Funding Advice Bureau SYFAB and TfR's guide to Evaluating Faith Based Projects What is evaluation? To evaluate is to assess the value of a piece of work, project, organisation or service. The main purpose of this is to help reflect upon what is trying to be achieved in order to establish how far it is succeeding. Evaluation can help establish quality assurance, provide a true picture of the work being undertaken and can highlight the good, the bad and the ugly! Evaluation also works as a tool by which you can learn from mistakes and can then progress, make changes and improve. This sheet will explore reasons why evaluating your project may be beneficial to it, your staff and volunteers and of course ultimately your beneficiaries; it will further offer suggestions about when and how to evaluate and who you might consider to be a valuable evaluator. Why Evaluate Your Project? Towards the end of a funding programme many funders require the evaluation of the projects that they have supported in order to ensure that the work has been conducted satisfactorily and to inform future strategies. Funders will want to know whether determined milestones have been reached, whether extra support is required and whether their money has been well spent. Evaluating provides accountability and is a particularly useful exercise in highlighting the philanthropic qualities of faith projects. Evaluations demonstrate stewardship and what ‘success’ means to and results in for faith organisations and their beneficiaries. However, if your project does not need to meet the requirements of funders it should still be considered good practice to carry out evaluations of your project to enhance your own learning and development about the difference you are trying to make. Evaluation enables refection upon purpose and goals – are things heading in the right and desired direction? The following questions may be asked and answered, for example: • How many people use our service? • What difference do we make, and who for? • How well are we meeting the needs of users and potential beneficiaries? • Does everything operate properly? Is it effective? Whose views count? • How can we make improvements? • Is this the best use of money, volunteers and worker time? Evaluation helps you to express your achievements to beneficiaries, partners, funders, the local community and others; its results act as an excellent source of publicity and a way to break down potential barriers towards ‘faith’ in secular society. ‘Sceptics’ can be shown the solid and positive impact that the group has as a resource of longevity in society. When Should You Evaluate? From the word go! Your project plan should include a section that specifically considers evaluation as the basis for continual improvement and refinement. An evaluation plan should outline SMART goals or outcomes (those that are specific, measurable, attainable, realistic and timely; see 1 below) - you can then implement ways to record evidence against these. Plan a periodic collection and assessment of evidence of longer term patterns of change (see 5) – avoid doing this ‘last minute’ as collecting the evidence, particularly if interaction with service users is needed, requires time, commitment and effort. Example of an evaluation plan that TfR can help you create. 1. Outcome Users are content with the church hall facilities 2. Aim a) Conduct evaluation of services b) Act on any matters arising from the service user questionnaire c) Conduct evaluation of services d) Act on any matters arising from the service user questionnaire 3. Action a) Create questionnaire for service users to find out their opinion of the church hall as a place to run their activity. Issue this and collate results. b) Review results; address any negative feedback. Produce report stating findings. Issue report to users, local community magazine; publish on church website. c) Create questionnaire for service users to find out their opinion of the church hall as a place to run their activity. Issue this and collate results. d) Review results; address any negative feedback and compare results to those collected in October 2010. Produce report stating findings. Issue report to users, local community magazine; publish on church website. 4. Who a) Volunteers b) Management committee c) Volunteers d) Management committee 5. To be completed by… a) September 2010 b) October 2010 c) September 2011 d) October 2011 How Should You Evaluate and What Framework to Use? There may be a limited amount of flexibility with what you can report upon in your evaluation if your outcomes and goals have been previously agreed with a funder. However, if this is not the case, think about how best you can measure whether you have or have not met your outcomes. For example, in the plan above it has been established that the outcome in can be best addressed by the action in 3. When you have collected and analysed your evidence, think how you could present the information most effectively. Example: Outcome: 25 families attending the Mosque open day in November 2010. Result: Through quantitative statistics: “In November 2010 30 families attended the Mosque open day; this represents a 20% increase on the intended target. Result: Through a qualitative case study, a simple structure for which could be…‘the situation’, ‘what we did’, ‘what happened’ and ‘what were the results’. The testimonies of beneficiaries provide an objective viewpoint and show the impact of your work first-hand. Sample from a qualitative case study. Ficton Jamia Mosque and Community Centre: Open Day November 2010 Who We Are: We are based in Ficton town centre, in an area highly populated by members of different cultural and religious backgrounds. Although the majority of users are from the Muslim faith our centre is open for use by the wider community. The Situation: We aim to be a space for meeting, learning and growing together as a community. In summer 2010 we decided that we would host an ‘open day’ for families to help address misconceptions about Islam and contribute to building a cohesive and friendly Ficton. What We Did and What Happened: We contacted the local press and other faith organisations in the area as well as local schools, advertising the open day for mid November. The Mosque would be open all day, with prayer observation, talks and refreshments throughout. What Were the Results: 30 families from the local area attended the Mosque open day, 20% more than we aimed for! “The open day really opened our eyes to how wonderful our town’s Mosque is; thank you for making it so interesting for our children!” The Jones’, Meadow Road. “I am delighted that so many families, of so many different backgrounds, took the opportunity to visit Jamia Mosque. I hope that this is the start of a long and prosperous relationship with them and the first of many more community events.” Imam Mahmood, Ficton Jamia Mosque. Remember, TfR can help you think about what information to present in a case study. Providing a balance of qualitative and quantitative data for your report makes for stimulating reading, particularly if being issued to a variety of people (funders, stakeholders, beneficiaries…) Using frameworks such as case studies and surveys are powerful, can be sent to the media easily and are less monotonous than 10 pages of prose! Measuring distance travelled by your beneficiaries may be enhanced by developing an application such as an Outcomes Star. This approach was developed for use in tenancy support by Triangle Consulting. Website: http://www.outcomesstar.org.uk Keeping data that has been gathered and evaluated periodically in one place provides a wealth of information to establish trends to state in annual reports, impact assessments, grant applications and essential updates that are provided to funders. Who Should Evaluate Your Project? The choice is yours! Staff and volunteers may be more than capable of evaluating the work with which they are involved and it may be beneficial for them to reflect upon the service that they provide. However, staff and volunteers must be prepared for the results, even if they are negative. When a nalysing work you may find it useful to look at the criteria that awarding bodies of quality assurance standards use to evaluate the compliance of applicant organisations. Quality assurance standards aim to ensure that the service being provided by an organisation is fit forpurpose and meets customer expectations. It can help improve local relations and help faith groups become community anchors. The Charities Evaluation Service offers training, consultancy, external evaluations and free advice about quality and evaluation systems, has a very accessible website and useful resources to introduce you to qualityassurance. Website: http://www.ces-vol.org.uk/index.cfm A number of quality assurance frameworks exist, the evaluation criteria for which may be useful to explore: VISIBLE: Community Matters is working with the Faith based Regeneration Network (FbRN), the Church UrbanFund (CUF) and the Faith and Civil Society Unit of Goldsmith’s to develop a quality assurance framework for faith based organisations. This will build on the Charity Commission-endorsed VISIBLE Accredited Standard; groups which meet this Standard’s criteria will have succeeded in showing that their organisation is strong and sustainable within its community and that it is, or has: A VOICE to represent issues of local concern An INDEPENDENT and politically neutral organisation A SERVICE provider for local people An INITIATOR of projects to meet locally identified needs A BUILDER of partnerships with other local organisations and groups A strong LOCAL network of people and organisations A way to ENGAGE local people to become active in their communities See http://www.visiblecommunities.org.uk for further information on VISIBLE’s evaluation criteria. PQASSO (Practical Quality Assurance System for Small Organisations): Created by and for the third sector it seeks to help take a systematic look at what you do, identify areas where you are doing well and not so well, and decide exactly where improvements are needed according to 12 quality areas. Website: http://www.ces-vol.org.uk/downloads/ pqassoguideforvcos-291-299.pdf Customer First: Open to any sector, but worth considering if your group is customer-focused, wants to respond to customer needs and create new business. See http://www. customerfirst.org/Standards.aspx for an overview of the 32 statements that applicants a re assessed against. International Organisation for Standardisation (ISO): ISO, is an international-standard-setting body. ISO has developed over 18,000 international standards on a variety of subjects; particularly popular with councils and their commissioning departments. The most popular standards are ISO 14001, environmental management systems and ISO 9001, managing for the sustained success of an organisation. http://www.iso.org/iso/iso_catalogue.htm features a directory of the standards, but requires payment to view the criteria for each. Investors in People: Outcome focused and has a non-prescriptive approach. This standard is popular with larger organisations such as the NHS and schools and may not be suitable for smaller, member-led groups. However, the framework summary may be a useful general resource when evaluating your work and services. Website: http://www.investorsinpeople.co.uk/ Documents/Branding2009/IIP_framework_summary09.pdf Social Enterprise Mark: A new quality standard to raise awareness of what social enterprises do in order to get more people to buy from, engage with, start up and work for social enterprises. The Mark represents enterprises working for social and environmental aims and at £99 plus VAT per year is much less expensive than other quality standards. Website: http://www.socialenterprisemark.org.uk Quality assurance standards are becoming increasingly popular with procurement and commissioning officers and often form part of tender bids, therefore if your group reaches a point where by it predominantly complies with a standard’s criteria, it may be worth considering official assessment, if finances allow. Another option regarding who should evaluate your project is to employ a consultant (working for the private or third sector). This may allow for a more professional and formal ‘feel’ than evaluations created ‘in-house’. However, do your funds allow for this? What would you learn from someone less familiar with your organisation conducting an evaluation? Would you gain a more objective view from an outside source? If you do decide to employ a consultant, TfR welcomes invitations to conduct evaluations for voluntary, community and faith groups throughout the country. South Yorkshire Funding Advice Bureau The Workstation 15 Paternoster Row Sheffield S1 2BX Tel: 0114 249 4343 Email: enquiries@syfab.org.uk Website: www.syfab.org.uk Together for Regeneration Sheffield Diocesan Church House 95-99 Effingham Street Rotherham S65 1BL Tel: 01709 309 119 Email: admin@tfr.org.uk Website: www.tfr.org.uk We can provide this information in other formats on request. Please get in touch to discuss your needs. Our information is produced for local community and voluntary groups. No permission is needed for limited reproduction if SYFAB is acknowledged. Large scale reproduction or inclusion in publications for sale must have written permission from SYFAB and TfR. SYFAB and TfR welcome your comments and criticism. You can talk to any member of staff, or contact us by post, phone, fax or email. Registered Charity No: 1061118 (SYFAB); 245861 (TfR) Reg. Company No: 3030641 (SYFAB); 196087 (TfR)