Program evaluation has become an existential requirement for human services organizations — funders increasingly demand outcome evidence rather than accepting good intentions and activity counts as sufficient justification for continued support, and organizations that cannot demonstrate program effectiveness face increasing difficulty securing and renewing funding. HMSV8218 develops the data analytics and program evaluation competencies that doctoral-level human services leaders need both to lead evaluation efforts within their own organizations and to interpret and use evaluation evidence intelligently in decision-making.
Selecting evaluation methodologies and data types
Matching evaluation approach to evaluation purpose
- Types of program evaluation: HMSV8218 develops the foundational distinction between evaluation types that serve different organizational purposes. Needs assessment evaluation occurs before program design — determining whether a problem exists, its scope and characteristics, and what interventions might address it. Process (or formative) evaluation examines whether a program is being implemented as designed — are services being delivered with appropriate frequency, intensity, and fidelity to the intended model? Are the intended populations being reached? Process evaluation answers "is the program operating as planned?" rather than "is the program producing intended outcomes?" — a critical distinction, since outcome evaluation findings are uninterpretable without knowing whether the program was actually implemented as designed (a program that shows no outcome improvement might reflect either an ineffective program model or poor implementation of an effective model — process data is needed to distinguish these). Outcome (or summative) evaluation examines whether the program is producing its intended results — has client functioning improved, have target behaviors changed, have systemic conditions improved? Impact evaluation, the most rigorous form of outcome evaluation, attempts to isolate the program's causal contribution to outcomes from other factors that might also explain observed changes
- Determining necessary data types: The course develops systematic frameworks for determining what data an evaluation requires given its purpose and the available resources. Outcome data must align with the program's logic model and theory of change — measuring the specific short-term, intermediate, and long-term outcomes the program is designed to produce, using instruments with established reliability and validity whenever possible rather than ad hoc measures. Process data must capture the service delivery dimensions (dosage, fidelity, reach, participant engagement) that explain variation in outcomes. Contextual data (participant characteristics, environmental conditions, concurrent services) is needed to understand for whom and under what conditions the program is more or less effective — addressing the reality that few human services interventions work equally well for everyone. The course also examines data source options: primary data collection (surveys, interviews, observation, standardized assessments administered specifically for evaluation purposes) versus secondary/administrative data (data already collected through program operations, such as case records, service logs, and billing data) — with attention to the tradeoffs between data quality/specificity and cost/burden
Analytical strategies for program evaluation data
HMSV8218 develops the quantitative and qualitative analytical skills needed to extract valid, interpretable findings from evaluation data. Quantitative analytical strategies examined include descriptive statistics (means, medians, frequencies, and distributions that characterize program participants and outcomes); inferential statistics (t-tests, ANOVA, regression analysis, and chi-square tests that evaluate whether observed differences or relationships are statistically significant rather than attributable to chance); and the appropriate statistical tests for different evaluation designs (paired t-tests for pre-post comparisons within the same participants; independent samples tests for comparing treatment and comparison groups; regression analysis for examining the relationship between multiple predictors and outcomes while controlling for confounding variables). Qualitative analytical strategies examined include thematic analysis (systematically identifying patterns and themes across interview or focus group data); content analysis (systematically categorizing and quantifying the content of text data); and the coding processes (open coding, axial coding, selective coding) that qualitative analysis traditions use to move from raw data to organized findings. The course emphasizes matching analytical strategy to data type and evaluation question rather than defaulting to familiar techniques regardless of fit — and develops the practical skills of using statistical software (SPSS, Excel) and qualitative analysis software (NVivo, or manual coding methods) appropriate to a human services evaluation context.
Ethical standards in program evaluation
HMSV8218 examines the distinctive ethical considerations that arise in program evaluation — which, unlike academic research, occurs within ongoing organizational relationships with clients, staff, and funders, creating ethical complexities that purely academic research does not face. The American Evaluation Association's Guiding Principles for Evaluators (systematic inquiry, competence, integrity/honesty, respect for people, and responsibilities for general and public welfare) provide the professional ethical framework the course examines. Particular ethical tensions addressed include: the dual role conflict that arises when program staff are asked to evaluate their own programs (creating potential conflicts of interest that can bias both data collection and interpretation, even unintentionally); the power dynamics between evaluators and program participants (clients may feel pressure to provide favorable feedback to evaluators who they perceive as connected to the organization providing their services); confidentiality protection for evaluation participants whose service records and outcome data are being analyzed; and the ethical obligation to report unfavorable findings honestly even when they may jeopardize program funding or organizational reputation — an obligation that can create real tension with organizational self-interest.
Communicating evaluation results effectively
HMSV8218 develops the communication competencies needed to translate evaluation findings into formats that different audiences can understand and act upon. Written evaluation reporting examines the structure and content of formal evaluation reports (executive summary, methodology description, findings presentation, limitations discussion, and recommendations) appropriate for funders, boards, and other formal stakeholders. Visual data presentation examines the principles of effective data visualization (appropriate chart type selection for different data types and comparisons; avoiding common visualization errors that distort or obscure findings; designing visualizations for the specific audience and purpose) drawing on the data visualization literature (Tufte's principles of graphical excellence; Few's guidance on quantitative data display) to develop genuinely informative rather than merely decorative visual communication. The course also examines audience-adapted communication — recognizing that the same evaluation findings may need to be presented differently for a funder report (emphasizing accountability and impact), a board presentation (emphasizing strategic implications), a staff meeting (emphasizing practice implications), and a community presentation (emphasizing accessibility and relevance to community concerns).
HMSV8218 assignments include evaluation design plans, data analysis reports, and visual data presentations
Our human services data analytics specialists deliver expert support for HMSV8218.
Get Help With HMSV8218
Evaluation design plans, data analysis reports, visual data presentations.
Place Your OrderView All ServicesRelated courses
Frequently asked questions
HMSV8218 addresses this scenario directly because it represents one of the most professionally and ethically challenging situations doctoral-level human services leaders encounter — and how an organization responds reveals whether its commitment to evidence-based practice is genuine or merely rhetorical. The ethical obligation, established by the American Evaluation Association's principles, is clear: evaluators must report findings honestly regardless of whether they are favorable to the program or organization. But the leadership challenge of how to respond constructively to unfavorable findings is more complex. The course develops several principles for this situation. First, examine implementation before concluding the program model is flawed: poor outcomes may reflect implementation problems (inadequate staff training, insufficient dosage, poor fidelity to the program model, inadequate reach to the intended population) rather than a flawed program theory — process evaluation data, if available, can distinguish these possibilities. Second, consider measurement and design limitations honestly: small sample sizes, short follow-up periods, or weak comparison groups may produce null findings even for genuinely effective programs (a Type II error) — but this possibility should not become an excuse to dismiss inconvenient findings without genuine scrutiny. Third, communicate findings transparently to stakeholders, including the staff who have invested in the program, while framing the conversation around organizational learning and improvement rather than blame — research on organizational learning suggests that cultures of psychological safety (where staff can discuss failures without fear of punishment) produce better long-term organizational performance than cultures that punish honest reporting of disappointing results. Fourth, use unfavorable findings as the basis for a deliberate decision process: continue with modifications informed by the evaluation; pilot an alternative approach; or sunset the program and redirect resources — but make this decision through a transparent process rather than either reflexively defending the existing program or precipitously abandoning it without adequate analysis of what the findings actually demonstrate.