My Experience with Organizational Capacity Building for Evaluation - PowerPoint PPT Presentation

My Experience with Organizational Capacity Building for Evaluation Diane Trunk Los Angeles Center for Law & Justice (LACLJ)

Ag Agenda • Intro and Background • General Info re Evaluation Capacity Building • Action Steps for Evaluation and Insights from our Experience

Where We Started . . . Me LACLJ • Lawyer – civil litigation • Legal Aid w/ MSW background • Experience in evaluating one • Committed to Evaluation in program at another org Strategic Plan • Presenter at State Bar and Judicial • Shriver Civil Gideon research Council events re Evaluation subject • New managing attorney • “Case, client and time management” database system • What’s a logic model? meant for for-profit law firms • New executive director • What is an outcome?

Collect, Ask Analyze and Meaningful Interpret Questions Data Use Findings for Decision-making and Action

Ac Action Steps for Evaluation • Determine available organizational resources • Create an evaluation workgroup • Create a program logic model • Develop meaningful evaluation questions • Write an evaluation plan • Choose appropriate data collection instruments • Collect credible and reliable data • Analyze the data and draw conclusions • Communicate the learning

Ac Action Steps for Evaluation • Determine available organizational resources • Create an evaluation workgroup • Create a program logic model • Develop meaningful evaluation questions • Write an evaluation plan • Choose appropriate data collection instruments • Collect credible and reliable data • Analyze the data and draw conclusions • Communicate the learning

Who are your Organizational Members? • Doubters: see little value in evaluation • Proctors: feel that evaluation is best done by other people • Practitioners: have and use the level of evaluation skills expected and needed by the organization • Specialists: can teach about and coordinate program evaluations • Scholars: develop considerable expertise in evaluation and actively share their expertise outside their organization

Ac Action Steps for Evaluation • Determine available organizational resources • Create an evaluation workgroup • Create a program logic model • Develop meaningful evaluation questions • Write an evaluation plan • Choose appropriate data collection instruments • Collect credible and reliable data • Analyze the data and draw conclusions • Communicate the learning

Ac Action Steps for Evaluation • Determine available organizational resources • Create an evaluation workgroup • Create a program logic model • Develop meaningful evaluation questions • Write an evaluation plan • Choose appropriate data collection instruments • Collect credible and reliable data • Analyze the data and draw conclusions • Communicate the learning

Logic Model Summary Intrapersonal Cognitive and Legal & Orders Well- Emotional Interpersonal Supportive Referrals Being Social Changes Services Safety Plans Changes

EVIDENCE BASED LEGAL SERVICES MODEL FRAMEWORK

Ac Action Steps for Evaluation • Determine available organizational resources • Create an evaluation workgroup • Create a program logic model • Develop meaningful evaluation questions • Write an evaluation plan • Choose appropriate data collection instruments • Collect credible and reliable data • Analyze the data and draw conclusions • Communicate the learning

Ac Action Steps for Evaluation • Determine available organizational resources • Create an evaluation workgroup • Create a program logic model • Develop meaningful evaluation questions • Write an evaluation plan • Choose appropriate data collection instruments • Collect credible and reliable data • Analyze the data and draw conclusions • Communicate the learning

Ac Action Steps for Evaluation • Determine available organizational resources • Create an evaluation workgroup • Create a program logic model • Develop meaningful evaluation questions • Write an evaluation plan • Choose appropriate data collection instruments • Collect credible and reliable data • Analyze the data and draw conclusions • Communicate the learning

Ne Needs s slide t title

Ac Action Steps for Evaluation • Determine available organizational resources • Create an evaluation workgroup • Create a program logic model • Develop meaningful evaluation questions • Write an evaluation plan • Choose appropriate data collection instruments • Collect credible and reliable data • Analyze the data and draw conclusions • Communicate the learning

Look at Your Data Integrity • Specific Data Problems • Check boxes vs. drop downs • Defaults • Text fields vs. lookups • Outdated answer options Strategic Data Analytics 18

Ac Action Steps for Evaluation • Determine available organizational resources • Create an evaluation workgroup • Create a program logic model • Develop meaningful evaluation questions • Write an evaluation plan • Choose appropriate data collection instruments • Collect credible and reliable data • Analyze the data and draw conclusions • Communicate the learning

Closed Extended Services Matters And Exit Interviews Client has related open matter 19% Exit completed 53% Client did not reply to our attempts to contact them 26%

Ac Action Steps for Evaluation • Determine available organizational resources • Create an evaluation workgroup • Create a program logic model • Develop meaningful evaluation questions • Write an evaluation plan • Choose appropriate data collection instruments • Collect credible and reliable data • Analyze the data and draw conclusions • Communicate the learning

Clients Report Improved Satisfaction with Specific Areas +23% My emotional well-being My physical well-being +17% My sense of safety +23% My financial situation My housing situation +21% My ability to co-parent 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 Intake Exit

ASSM: Average Percent Change in Different Populations Job Readiness Employment Income Housing Food Health/Disability Transportation Health Care Social Relations Community Inv Mental Health All DV Survivors All Immigrants Undoc Life Skills Safety 0 5 10 15 20 25 30

Re Resources I’ve Found Helpful . . . • “You are Already an Evaluator” video by Changing River Consulting https://www.youtube.com/watch?v=lInjrhc45Dw • Jones, Sheri Chaney (2014) Impact & Excellence . • Evergreen, Stephanie (2018) Presenting Data Effectively 2 nd Ed. (and her blog) • The work of Dr. Cris M. Sullivan and https://www.dvevidenceproject.org • American Evaluation Association: www.eval.org • CDC: https://www.cdc.gov/obesity/downloads/cdc-evaluation-workbook-508.pdf • The work of Hallie Preskill – evaluation capacity building expert