Intervention Programs


Public Works has supported social services and support programs in the evaluation of their program processes and impact and has provided insight to clients regarding the provision of services targeted at specific populations such as foster care, juvenile delinquency and special education. We have conducted numerous evaluations of after-school and youth development programs.

 

After-School and Enrichment Program Evaluation

 

Evaluation of Environmental Literacy Initiatives:  TreePeople and ChangeScale. The California Regional Environmental Education Community (CREEC) Network is an educational project supported by the California Department of Education to support environmental literacy of California’s students by providing teachers with access to high quality environmental education resources. CREEC is housed and supported by TreePeople in the Los Angeles area and by ChangeScale in the San Francisco and Monterey Bay areas. Both TreePeople and ChangeScale have contracted with PW to develop, monitor, and analyze survey data to measure and monitor Environmental Education experiences and opportunities in partnered school districts. We conducted an evaluation in three school districts, to form the basis for a useful platform for districts, schools, and teachers to compare the Environmental Literacy efforts throughout the state.

 

The Huntington Library, Art Collections, and Botanical Gardens School Programs Evaluation.  The Huntington contains a world-renowned library, art-collection, and 120-acre botanical garden, which offers more than a dozen different educational programs for K-12 students. All programs are led by volunteer docents and address state and local education standards.  PW worked with The Huntington on a comprehensive collaborative evaluation of the School Programs component of the Education Division, with involvement of volunteer docents, participating teachers and students, and relevant employees.  Evaluation methods included observations, focus groups, student feedback, docent and teacher surveys, and listening meetings with all stakeholders.

 

Evaluation of the Reach for the Stars After-School Program, Whittier City School District (WCSD).  PW has served as the external evaluator for the WCSD’s after-school program funded through a combination of federal (21st Century Learning Center) and state (After School Education and Safety) and local (After School Enrichment Program) funding sources.  Evaluation methods include monitoring after-school attendance, conducting site visits to each of nine participating schools, rating each school’s after-school program based on a performance rubric, and surveying teachers of students involved in after-school programs.  In addition to reporting on implementation of the after-school programs, evaluation reports have included analysis of student outcomes (standardized test scores and regular school day attendance) of frequent after-school participants to students uninvolved in after-school programs controlling for demographic characteristics and prior academic achievement.

 

Evaluation of PasadenaLEARNs, Pasadena Unified School District (PUSD).  PW evaluated the effectiveness of an after-school program funded through a combination of County, State and Federal funds at 16 elementary and 3 middle schools in PUSD over three years. Evaluative research examined the program’s impact on academic achievement as well as student behavior, leadership and community involvement.  In addition, the evaluation included an in-depth examination of program implementation in a variety of areas including safety, program management, parent involvement, school linkages, and sustainability.  Evaluation methods included interviews and focus groups, site observations, surveys as well as descriptive and regression analysis of quantitative data linked to student performance on standardized tests and regular school day attendance.  Additional analyses focused on student performance on literacy and math assessments, and assessments of classroom performance completed by teachers. This evaluation was participatory in nature insofar as PW assisted PUSD and individual schools in the refinement of program implementation and the development and maintenance of electronic record keeping intended to create a data-driven model for after school programs.

 

Evaluation of PasadenaLEARNs, PUSD.  After the initial three year evaluation, PW continued to support the program by helping to meet the grant reporting requirements and providing training to the after-school program funded through County, State and Federal funds at 17 elementary and 3 middle schools in PUSD. Evaluation methods included descriptive analysis of quantitative data linked to student performance on standardized tests and regular school day attendance and the compilation of qualitative data collected in previous years’ evaluation into narrative for reporting purposes. In addition, PW designed instruments and provided training to all after-school sites for the purposes of developing a peer-monitoring model in which sites self-evaluated and provided feedback to peers regarding program implementation.

 

Evaluator, Champions Science Adventures. PW served as the evaluator for Champions Science Adventures, a national program focused on extra-curricular science education. PW developed eight pre-and post-assessments for four camps administered to a sample of the approximately 3,200 participating students. Evaluation activities included collection and analysis of data related to participants’ enrollment, and collection and analysis of the pre-and post-assessments for participants in grades K-3 and 4-6.

 

Evaluator, Mustangs on the Move, Flintridge Operating Foundation. PW served as the evaluator for Mustangs on the Move, a collaboration of nonprofit organizations and community members in Pasadena, CA. PW evaluated the effectiveness of the program that targeted high school students at John Muir High School. Evaluation activities included collection of attendance data; analysis of student outcomes data including a match to a demographically similar group of students; collection and analysis of partnership data; development and administration of a student survey; and development and administration of a parent survey.

 

Evaluation of the LACOE After-School Enrichment Program (ASEP), Los Angeles County Office of Education (LACOE).  PW evaluated the effectiveness of the multiple year, after-school program in place in 100 schools across Los Angeles County targeting CalWorks families. The evaluation examined the level of implementation at the site and overall program levels as well as the program’s impact on academic and behavioral achievement. Evaluation methods included interviews of key stakeholders at all sites as well as in-depth site visits at a sample of sites; student, parent and program staff surveys; and descriptive analysis of quantitative data including standardized test scores and attendance rates.  Additionally, site profiles were developed for all 100 sites and an anti-tobacco/alcohol/other drugs program, Friday Night Live Kids, were examined in depth. In order to better disseminate the evaluation results, a Website and data brochures were created.

 

Evaluation of the 21st Century High School After School Safety and Enrichment for Teens (ASSETs) Program Grant to the Keep Youth Doing Something (KYDS).  PW evaluated the baseline year of a five-year grant to implement an after school project at Polytechnic High School in the Los Angeles Unified School District, called the KYDS DREAMS project.  The DREAMS project aimed to increase language proficiency of participating English Learner students, enhance the participation of parents of participating students in family literacy programs, improve health and at-risk behavior among student participants, and strengthen and diversify community partnerships.  The evaluation called for PW to monitor and describe the implementation of after-school program; offer recommendations to improve the delivery of services; determine the impact of program activities on student achievement and related indicators of success; and, fulfill accountability and reporting requirements associated with the grant. Evaluation activities included a site visit to the school, KYDS staff interviews, a survey of student participants, and an analysis of student outcome data.

 

Evaluation of LA COPS After School Program, Los Angeles Unified School District (LAUSD). PW served as the evaluator for the federally funded LA COPS after-school program operated at five LAUSD high schools. Evaluation methods included school staff, student and parent surveys as well as the analysis of student achievement data. On-site qualitative research allowed for the examination of implementation progress in the key areas of academic support and tutoring services, enrichment programs, school-to-career opportunities, school safety and community collaboration.

 

Youth Development Evaluation

 

Counseling Works, Learning Works Charter School. Learning Works Charter School (LW), which operates under the umbrella of the PW organization, is committed to ensuring all youth in Pasadena, Altadena and Sierra Madre attain a high school diploma. This commitment extends to establishing strong support for youth through counseling and mental services to remain engaged and continue their education. PW established the Learning Works Mental Health Team to meet weekly to implement a federal counseling grant and case manage all enrolled youth. The grant focused on graduation, part-time jobs, and postsecondary placement in education or a job and measures of success for particular participants including probation youth and teen mothers. PW served as the evaluator for this grant and prepared all required federal reports.

 

Technical Assistance to the PUSD Positive Futures Grad Chasers and Dropout Advocates Grant. Using the model developed for students of the Learning Works Charter School, Learning Works assisted the district in the creation of a dropout recovery center and provided training to advocates and other staff such as attendance clerks, gang interventionists, social work interns, security, community based organizations, and others in how to “chase” at the regular middle and high school to intervene in the flow of school dropouts. The training included the development of modules including the importance of focusing on dropouts, the culture of schools, using data, what to do when you find dropouts, and engaging partners.

 

Evaluation of the PUSD Positive Futures Grant. In conjunction with the Positive Futures technical support, Grant Education Consultants contracted with PW to support the evaluation of this project. The evaluation focused on qualitative data that provided information and results necessary to determine if efforts were working to decrease the dropout rate and increase the graduation rate in the district. The evaluation included site visits to all PUSD middle and high schools; interview and focus groups for middle and high schools; ninth grade student survey to measure climate and culture of each school, and baseline data collection to meet federal requirements. The evaluation also included the collection of quantitative student outcome data.

 

Evaluation of the Valley Neighborhoods Collaborative, Los Angeles Education Partnership. PW served as the evaluator of the Valley Neighborhoods Collaborative, a community schools initiative led by the Los Angeles Education Partnership (LAEP) that centered on the three LAUSD high schools in the Northeast Valley of Los Angeles County.  Research focused on the extent to which the Valley Neighborhoods Collaborative and its neighborhood partnerships at San Fernando, Sylmar, and Arleta High Schools (beginning in Years 1, 2, and 3 of the grant, respectively) have: (1) increased parent involvement in their children’s education; (2) increased academic skills of high school students participating in academic enrichment and intervention programs; (3) increased youth eligibility and preparation for postsecondary opportunities; and (4) increased communication and service coordination among service providers, schools, and community members. Evaluation methods included staff, student, and stakeholder surveys, student focus groups, interviews with service providers, analysis of student and parent attendance data from partnering service providers, and analysis of a variety of school-wide and student academic achievement indicators. PW also participated in the initiative’s coordination team, monthly resource council meetings at the high schools, and quarterly collaborative meetings with all school stakeholders. PW submitted semi-annual Government Performance and Results Act (GPRA) reports to the U.S. Department of Education, as well as semi-annual summative evaluation reports to LAEP and other initiative stakeholders.

 

Evaluation of Safe Schools/Healthy Students Grant Initiative, Local District 8 of the Los Angeles Unified School District (LAUSD).  PW served as the local evaluator of the Safe Schools/Healthy Students grant initiative at Washington Preparatory Senior High School and its 19 feeder schools in south Los Angeles.  Research focused on the extent to which the initiative—a collaboration between 20 elementary and secondary schools, local law enforcement, county social service agencies, and community-based organizations—enhanced the safety of the twenty grantee schools and their surrounding communities and improved the health of students at those schools.  More specifically, the evaluation focused on the extent to which the initiative: (1) decreased violence and increased student perception of safety; (2) reduced use of alcohol, tobacco, and other drugs among youth; (3) improved the behavior, and social and emotional functioning of students and reduced bias-related harassment; (4) improved access to mental health services for students, families, and community members and improved the mental health of students; and (5) improved the physical, social, and emotional health of pregnant teens and their unborn children.  Evaluation methods included staff and student surveys, focus groups and interviews with school and project staff and community partners, as well as analysis of a variety of school-level data such as truancy rates, suspensions and expulsions, referrals to mental health services, probation youth outcomes, and student participation in alcohol and drug prevention and intervention programs; stakeholder meetings to share information; and submission of annual federal reports.

 

Evaluation of Project GRAD Los Angeles. Project GRAD, a non-profit designed to close the achievement gap between different groups of students, partnered with a community’s elementary schools and feeder middle schools to help expand the number of students at the high school level to taking advantage of resources and support offered to increase student access to higher education. Students who meet the Project GRAD requirements at the end of high school, receive scholarship money to help augment their college education. PW initially conducted a summative evaluation of the data collected by Project GRAD and a needs assessment of data collection procedures and processes with recommendations for improvement.

 

Cash for College Workshops Evaluation. PW served as the external evaluator of the Cash for College workshops, which included statewide and regional particpant surveys, information regarding incentive scholarships and FAFSA completion rates.

 

Los Angeles Youth Opportunities Movement (LAYOM) Community Technology Centers Grant Evaluation.  PW monitored the implementation of the Los Angeles Youth Opportunities Movement Community Technology Centers grant. Evaluation methods included surveys of Dolores Mission Alternative (DMA) seniors, youth and employers involved in internships, focus groups of case managers and students, and surveys of adult clients. PW analyzed data collected through the online LAYOM CTC database to assess levels of program participation, attendance rates, and data collection quality check-ins and support and student outcome data.

 

Lead Agency in providing Community-Based Teen Services, Department of Public Social Services, Long-Term Family and Self-Sufficiency Division. PW was the lead agency for a community-based teen services grant providing services to teens in the 6th to 12th grades in the PUSD attendance area. Activities included coordinating all subcontractors and partners in the development of services, which included two Learning Centers located in Pasadena to provide tutoring and academic intervention services to middle and high school students.  The Learning Centers also coordinated weekly “after-school specials” aimed at parents and youth on topics such as using the Internet, college and university admissions and career awareness. In addition, the services available through the Teen Services grant included individualized counseling on postsecondary and career options, access to virtual One-Stops, a technology center and a clearinghouse for job shadowing, volunteer and internship opportunities and referrals to social service agencies.

Evaluation System Development, Alumni House, Casey Family Programs and Transition Partners.  PW developed an implementation and evaluation process for the Alumni House in Pasadena, an organization providing housing, social services, job search and other information and support to graduates of foster care who are transitioning to adult independence.  PW conducted a needs assessment of foster care youth in independent living programs, graduates of foster care, and community-based organizations providing service. Based on the information gathered, PW developed a logic model that identified individual and program outcomes to be measured for progress in the evaluation plan. PW then developed a plan for revised intake procedures and a user-friendly data system intended to facilitate the efficient case management of youth served through the Alumni House.

 

Other Social Services Evaluation Projects

 

Consultant, The Frostig Center. PW served as a consultant to The Frostig Center, a company dedicated to improving education for students with learning disabilities. Consulting activities included three internal meetings, reviewing financial information and marketing materials, meetings with the Board and the development of a plan for key focus areas, especially Fund Development. PW drafted a plan for The Frostig Center based on these consultation services.

 

PUSD School Community Violence Prevention Program Evaluation. PW evaluated PUSD’s School Community Violence Prevention Program (SCVP) grant. The SCVP grant required that PUSD work in collaboration with community organizations, students, parents, school staff and law enforcement in implementing consistent, coordinated programs to increase school safety. PUSD, in partnership with the Pasadena Police Department, Western Justice Center Foundation and Operation Street Kidz, used the grant to develop such a collaborative to implement school safety and violence prevention programs at Pasadena High School and Washington Middle school, and to bolster existing programs at those two schools.

 

Los Angeles County Office of Education WIC. PW conducted an evaluation of the Early Advantage Initiative developed by the Los Angeles County Office of Education (LACOE) targeting large numbers of low-income parents and their children at Women, Infants, and Children (WIC) sites to improve the school readiness of young children. The evaluation included a review of data collection systems in both paper and electronic forms used including intake, surveys, and databases to assess the quality of information to be used for longitudinal evaluation purposes. Qualitative data was also collected from participants at one site. All data was summarized with recommendations to strengthen collection strategies. PW has also analyzed data collected by staff from parents about what they learned from the program to improve their child’s readiness for school.

 

Evaluation of Children’s Planning Council’s (CPC) Work to Date, Los Angeles County. PW measured the impact of the CPC based on their goals and objectives in a three-year Work Plan in order to examine whether it had been influential in bringing change over time. Through stakeholder surveys and interviews, PW measured the impact of the systems, partnerships, and linkages of children and families throughout the county and within sub-regions, assessed the impact of CPC from past to present (short-term) and project impact in the future (intermediate and long-term) to offer mid-term adjustments in the plan; and describe best practices.

 

Evaluation System Development, Haven Hills. Through a grant from the California Department of Health Services (DHS), Maternal and Child Health Branch/Domestic Violence, PW, and O’Malley International developed a plan for a two-year project centered on reducing relationship abuse among teens and young adults aged 14-22 in the West San Fernando Valley.  PW conducted a focused needs assessment that identified available programs and/or gaps in services. Techniques included one-on-one interviews, focus groups and surveys with teen clients of battered women’s shelters, staff from battered women’s shelters, staff from youth-serving agencies, and youth and adults from the local community. Subsequent data collected via interviews, focus groups, and surveys resulted in updated work and evaluation plans centered on developing new partnerships and effective strategies for reducing teenage relationship abuse.

 

Mothers’ Club Community Center Evaluation. Through a Proposition 10 grant (First Five Los Angeles) from the California Children and Families Commission, PW worked with Mothers’ Club Community Center to develop an evaluation plan for two separate projects designed to increase family literacy and enhance child care and early learning opportunities in the Pasadena area.  This evaluation focused on summarizing existing data instruments used by Mothers’ Club, guidance on the new Proposition 10 evaluation requirements, and determining data collection methods and instruments. In addition, PW recommended changes to and upgraded existing program database in order to produce multiple, useful reports.

 

Strategic Plan Development, Rosemary Children’s Services (RCS).  PW developed an organizational strategic plan for RCS in Pasadena, an organization that delivers services to abused, neglected and emotionally disturbed youth and their families.  PW conducted a needs assessment to identify strengths, weaknesses, opportunities, and threats to all program components of RCS.  Techniques included one-on-one interviews and focus groups with staff, volunteers, program directors, and executive board members.  The project resulted in a three-year strategic plan outlining the mission, guiding principles, program goals and objectives of the RCS program.

 

Evaluation System Development, Rainbow Services, Ltd. Through a grant from the California Department of Health Services, PW worked with Rainbow Services, Ltd. to develop an evaluation plan for a three-year project designed to help victims of domestic violence living in the greater San Pedro area. This evaluation plan focused on the organization’s particular strategies designed to help Spanish-speaking women become better able to identify abuse issues in their lives, cope with the emotional and physical problems suffered by both women and their children as a result of such abuse and to develop strategies for living independent lives free of abusive relationships.  Interviews and meetings with the director and other staff members focused the structure and location of the services to be provided under this grant and the evaluation tools to be used to determine progress in meeting grant goals. PW then conducted interviews and focus groups with staff and clients at year-end to assess agency progress under this grant and the new evaluation plan.

 

Strategic Plan Development, Pasadena Unified School District Family Centers. PW developed an organizational strategic plan for the Family Centers in the Pasadena Unified School District.  The Pasadena Education Foundation received a grant from the California Endowment to implement a three-year effort to build capacity at each of the five family centers.  These Family Centers serve the function of providing comprehensive and coordinated health and social services for the community within which the Centers are located. PW first conducted a brief needs assessment to inform the direction of the plan.  Through an inclusive process that included regular meetings with Family Center coordinators and PUSD stakeholders, goals and objectives as well as specific action steps were developed.  The project resulted in a three-year strategic plan outlining these program goals, objectives, and action steps.

 

Evaluation of the Early Childhood Program (ECP), Pasadena Unified School District (PUSD).  PW evaluated the ECP, a pre-Kindergarten program targeting the academic, social, and emotional development of children ages three and four. PW investigated the immediate effects of ECP using a combination of interviews, focus groups, questionnaires, and surveys. The evaluation assessed ECP in terms of its ability to prepare children for Kindergarten through an examination of program curriculum, intake and assessment procedures, instructional materials and resources, and interactions with parents. PW also analyzed quantitative measures of student achievement for ECP graduates compared to Head Start participants and PUSD students not involved in either early childhood program. A second phase of the evaluation examined the long-term effect of ECP participation, comparing ECP graduates to Head Start and other PUSD secondary school students transitioning to the third year, which focused on assessing program effectiveness and quality. PW used tools required by the California Department of Education, the Early Childhood Environment Rating Scale-Revised (ECERS-R), the Desired Results Developmental Profile (DRDP), and the DRDP Parent Survey.  PW also provided training to ECP teachers on administration of these tools.

 

Evaluation of the Academic Tutoring Partnership Program (ATPP) and Tutoring Assistance Grant (TAG), Pasadena Unified School District (PUSD).  PW conducted an evaluation of the ATPP to submit to the California Department of Education as partial fulfillment of TAG requirements. PW collected and analyzed the appropriate data to complete survey and student outcomes forms provided by CDE for each ATPP site.  Program personnel at each site were interviewed to develop narrative descriptions of each program.  In addition to submitting these data to the CDE, all collected data were combined to create individual school profiles that were provided to the schools and district.

 

Evaluation of the Hate Crime Prevention Program, National Conference on Community and Justice (NCCJ). PW evaluated two NCCJ programs aimed at improving youth awareness of intercultural understanding and reducing hate crimes in Los Angeles County.  Interviews and analysis of participant data were used to evaluate the effectiveness of the Juvenile Offenders Learn Tolerance (JOLT) program, a group therapy program for juvenile hate crime offenders and their parents. PW also employed surveys and conducted interviews as part of an evaluation of InterACTions, an interactive theatrical performance aimed at presenting alternatives to racism, hate and violence among secondary students and youth in County incarceration facilities.

 

Evaluation System Development, Armory Center for the Arts.  PW conducted a systematic needs assessment of the Armory Center for the Arts, a Pasadena-based non-profit arts education organization.  The needs assessment was designed to assess both programmatic and organizational evaluation needs.  In particular, PW focused on identifying standardized measures to assess the effectiveness and impact of the Armory’s educational programs serving children and youth.  As part of this process, PW assisted the Armory in the development of an electronic database for collecting data that can, in turn, be used to monitor and guide on-going organizational efforts and fundraising.

 

Non-Profit Strategic Planning, The Children’s Collective, Inc.  PW assisted The Children’s Collective, Inc., a non-profit organization involved in providing families and children with improved access to social and support services.  PW focused on the development of an indirect rate for cost accounting and development purposes.  Based on a complete review of accounting procedures, it was determined that the organization needed to restructure its internal control system.

 

Evaluation of Healthy Start, Van Nuys Elementary, Los Angeles Unified School District (LAUSD).  PW evaluated the second year of a State funded three-year Healthy Start program aimed at improving student and family access to basic health care and providing school-based access to mental health services at an elementary school in the San Fernando Valley.  Evaluation methods centered on the analysis of a case management sample of students and their families as well as the collection of service delivery data.  In addition, PW interviewed program staff, teachers, and school health personnel in as part of preparation of an annual State report.

 

Evaluation of Angel Gate Academy, Los Angeles Unified School District (LAUSD).  PW evaluated the Angel Gate Academy, a joint program between LAUSD, the National Guard and Cuesta Community College aimed at intervening with at-risk middle school students in a quasi-military, four-week experience. Evaluation methods included intensive focus groups with students and National Guard personnel, written student surveys, parent telephone interviews, transcript analysis and interview/observation techniques.  A final report documenting program success and areas for further improvement was developed.

 

 

© 2018. Public Works Group. All Rights Reserved.