Training and performance improvement professionals face a persistent credibility challenge: how do you demonstrate that your programs produce real business value, not just satisfied participants? The Phillips ROI Methodology — developed by Jack and Patti Phillips over three decades and now used by hundreds of organizations globally — provides a rigorous, defensible framework for measuring training impact at every level from participant reaction to financial return on investment. ED7675 develops the full competency to plan, conduct, and report ROI impact studies.
The Phillips ROI Methodology
Five levels of evaluation in one integrated framework
- Level 1 — Reaction and Planned Action: ED7675 examines Level 1 evaluation (participant reaction) not as an end in itself but as a predictor of application and impact. The course distinguishes between reaction measures that predict transfer (learners' assessment of relevance, importance, and intent to apply) versus those that merely reflect enjoyment or satisfaction (instructor ratings, facility comfort), and develops Level 1 instruments that collect predictive reaction data efficiently
- Level 2 — Learning: The course covers Level 2 evaluation design — assessing whether participants acquired the knowledge, skills, confidence, and commitment needed to apply what they learned. ED7675 develops learning assessment designs that go beyond end-of-training knowledge tests to include skill demonstrations, simulations, role plays, and pre/post comparisons that measure genuine learning gains rather than test-taking performance
- Level 3 — Application and Implementation: ED7675 gives particular attention to Level 3 — the measurement of whether participants actually apply their learning on the job — because this is where most training ROI studies fall short. The course covers the timing and design of follow-up data collection (typically 60-90 days post-training), the methods for collecting application data (follow-up surveys, structured interviews, observation, supervisor assessments), and the barriers and enablers to application that Level 3 data reveals — information that is invaluable for improving both programs and the organizational environments that support or impede application
- Level 4 — Impact: The course covers Level 4 evaluation — measuring the business impact of training (productivity, quality, cost reduction, time savings, customer satisfaction, employee engagement, compliance). The key challenge at Level 4 is isolating the effect of the training from the many other factors that simultaneously affect business outcomes. ED7675 develops the isolation methods that make Level 4 credible: control groups, trend line analysis, expert estimation with confidence intervals, and statistical controls
- Level 5 — ROI: The course covers the conversion of Level 4 impact data to monetary value and the calculation of the ROI ratio: (Net Program Benefits / Program Costs) × 100. This includes the methods for converting training impact data to money (standard values from accounting records, historical costs, expert estimates), the categorization and calculation of program costs (including often-overlooked indirect costs like participant time and opportunity cost), and the interpretation of ROI results in organizational context
Evaluation planning
ED7675 develops the evaluation planning skills that make ROI studies feasible and defensible. The course covers the data collection plan (specifying the evaluation objectives, data collection instruments, timing, and responsibilities for each level of the framework), the ROI analysis plan (specifying the isolation methods, conversion methods, cost categories, intangible benefit categories, communication targets, and other study parameters before data collection begins — planning decisions made after data collection bias results), and the targeting decision (not every program warrants a Level 5 ROI study; ED7675 develops the criteria for selecting which programs to evaluate at which levels, including the ROI Forecasting methodology for high-cost programs that require pre-implementation ROI projections to justify funding). The course also addresses the organizational politics of evaluation planning — building evaluation into program design from the beginning rather than retrofitting it, managing stakeholder expectations about what evaluation can and cannot demonstrate, and developing the organizational infrastructure (evaluation policies, measurement standards, data collection systems) that makes sustained evaluation practice possible.
Data collection and analysis
ED7675 develops practical data collection and analysis skills applied to real ROI impact studies. The course covers instrument design for each level of the framework (reaction questionnaires, learning assessments, follow-up application surveys, business impact questionnaires, focus group protocols, and interview guides), sampling strategies (when to survey everyone versus representative samples, response rate management, handling non-respondents), data quality assessment (evaluating whether collected data is sufficiently reliable and valid to support ROI conclusions), and data analysis methods (descriptive statistics for reaction and application data, pre/post comparison for learning data, isolation method application for impact data, monetary conversion calculations, cost categorization and totaling, ROI ratio calculation, and intangible benefit identification and categorization). The course applies these skills to case study data and, where possible, to actual organizational projects.
Reporting and communication
ED7675 covers the communication of ROI study findings — a critical but often neglected component. The course covers the impact study report structure (executive summary, background, evaluation methodology, results by level, program costs, monetary benefits, ROI, intangible benefits, barriers and enablers, conclusions and recommendations), the adaptation of findings for different audiences (executives want ROI and business impact; program managers want application barriers and improvement recommendations; learning and development professionals want design and delivery implications), and the presentation of ROI results in organizational contexts where skepticism about training ROI is common. The course also addresses the ethical dimensions of ROI reporting: presenting results honestly (including unfavorable findings, limitations, and uncertainties) rather than selectively emphasizing positive results; acknowledging the limitations of isolation methods rather than claiming certainty about causal attribution; and making defensible but not exaggerated claims about the business value of learning initiatives.
ED7675 assignments include evaluation plans, data collection instruments, impact study analyses, ROI calculations, and executive briefings
Our training and performance specialists deliver expert support for ED7675.
Get Help With ED7675
Evaluation plans, data instruments, impact study analyses, ROI calculations, executive briefings, reporting.
Place Your OrderView All ServicesRelated courses
Frequently asked questions
ED7675 addresses the practical challenges of ROI measurement that cause many training professionals to give up on impact evaluation before they get to the ROI calculation. The most significant challenge is isolation — separating the effect of the training from all the other factors that simultaneously affect business outcomes. If sales increase after a sales training program, how much of the increase was caused by the training versus an economic upturn, a new product launch, a competitor's exit from the market, or seasonal fluctuation? Without a credible isolation method, the Level 4 data can't support a valid ROI claim. The Phillips methodology addresses this through several isolation techniques: control groups (comparing trained versus untrained groups when this is organizationally feasible), trend line analysis (projecting pre-training performance trends forward and comparing actual post-training performance to the projected baseline), and expert estimation with confidence adjustments (asking participants, managers, and other knowledgeable parties to estimate what percentage of the improvement was caused by the training, then adjusting the estimate down by an "error" percentage to account for estimation bias). None of these methods is perfect, but each is more defensible than making no isolation attempt. The second major challenge is monetary conversion — translating business impact data (productivity units, quality improvements, error reductions, time savings) into dollar values that can be compared to program costs. The Phillips methodology provides a hierarchy of conversion methods: use standard values from organizational accounting records when available; use historical cost data; use expert estimates with error adjustments; recognize that some impacts (employee engagement, leadership capability, culture) are too difficult to convert reliably and should be reported as intangible benefits rather than forced into an artificial monetary conversion that undermines credibility. The third challenge is cost completeness — most organizations dramatically undercount program costs by including only direct delivery costs (facilitator fees, materials, facilities) while ignoring participant time (often the largest cost), design and development amortization, technology, and administrative overhead. Accurate ROI requires comprehensive cost capture, which often means working with accounting to develop cost tracking systems that the organization doesn't currently have.