This week we read about logic models and ethics in evaluation. I am interested in your thoughts and how you understood the material . The purpose of this assignment is for you to practice developing a logic model and developing professional ethos so that you can better understand how program theory works and how ethics and logic models fit into the evaluation process. Please review all materials before you begin this assignment.
What is your understanding of a program logic model? Explain using your own words.
Discuss the limits of logic modeling as an approach to describing programs.
What roles do program logic models play in the process of evaluating programs? Please explain so we can learn from your response.
For program evaluators?
For other stakeholders in the evaluation process?
Select a policy or program and create a logic model – see examples provided in the PowerPoint. A logic model template is included in the module for this week. Upload a copy of your original logic model for your chosen program/policy.
Requirements: few sentences each question
W.K. Kellogg FoundationLogic Model Development Guide
IntroductionIf you donÕt know where youÕre going,how are you gonnaÕknow when you get there?ÐYogi BerraIn line with its core mission Ð To help people help themselves through the practical application of knowledgeand resources to improve their quality of life and that of future generations Ð the W.K.Kellogg Foundation hasmade program evaluation a priority.As our staff and grantees work on a spectrum of social improve-ment programs,the need for shaping and contributing to the body of knowledge regarding evaluationbecomes increasingly clear.Our first guide,the W.K.Kellogg Foundation Evaluation Handbook,was pub-lished in 1998,and has been made available to nearly 7,500 people.The Evaluation Handbookis a prac-tical,step-by-step manual for conducting evaluations.With the Handbook,we introduced the conceptof the program logic modeland the ways in which applying this concept has added value to ourown work.The program logic model is defined as a picture of how your organization does its work Ð the theory andassumptionsunderlying the program.A program logic model links outcomes (both short- and long-term)with program activities/processes and the theoretical assumptions/principles of the program.The W.K.Kellogg Foundation Logic Model Development Guide,a companion publication to the EvaluationHandbook,focuses on the development and use of the program logic model.We have found the logicmodel and its processesfacilitate thinking,planning,and communications about program objectives andactual accomplishments.Through this guide,we hope to provide an orientation to the underlying prin-ciples and language of the program logic model so it can be effectively used in program planning,implementation,and dissemination of results.The premise behind this guide Ð and our view of the role of evaluation in programming Ð is simple:Good evaluation reflects clear thinking and responsible program management.Over the years,ourexperience in using logic models in initiatives such as the Kellogg Youth Initiative Partnerships,Devolution,ENLACE (Engaging Latino Communities for Education),and the Native AmericanHigher Education Initiative,to name just a few,has provided ample evidence of the effectiveness ofthese methods.Learning and using tools like logic models can serve to increase the practitionerÕs voice in the domainsof planning,design,implementation,analysis,and knowledge generation.The process of developing themodel is an opportunity to chart the course.It is a conscious process that creates an explicit under-standing of the challenges ahead,the resources available,and the timetable in which to hit the target.Inaddition,it helps keep a balanced focus on the big picture as well as the component parts.In general,logic modeling can greatly enhance the participatory role and usefulness of evaluation as amanagement and learning tool.Developing and using logic models is an important step in buildingcommunity capacity and strengthening community voice.The ability to identify outcomes and antici-pate ways to measure them provides all program participantswith a clear map of the road ahead.Mapin hand,participants are more confident of their place in the scheme of things,and hence,more likelyto actively engage and less likely to stray from the course Ð and when they do,to do so consciouslyand intentionally.Because it is particularly amenable to visual depictions,program logic modeling canbe a strong tool in communicating with diverse audiences Ð those who have varying world views anddifferent levels of experience with program development and evaluation.Logic Model Development GuidePage III
IntroductionThe Logic Model DevelopmentGuidecontains four chapters and two comprehensive appendices.Chapter 1presents a basic introduction to the logic model as an action-oriented tool for programplanning and evaluation.It also offers an array of sample logic models.Chapter 2consists of exercises and examples focused on the development of a simple program logicmodel.Exercises include practical examples,checklists for reviewing content quality,and a templatefor developing a logic model.Chapter 3gives instructions on how to expand a basic logic model to explore and explain the theory-of-change that describes the rationale for your program.A template and checklist are provided.Chapter 4offers two exercises that afford the reader with an introduction to how the basic logicmodeling techniques introduced in the previous chapters can be applied to inform thinking aboutwhat should be included in an evaluation plan.Templates and checklists are also provided.The Resources Appendixprovides logic model development resources Ð references and Web sites worthvisiting.The Forms Appendixincludes blank templates to copy when developing your own logic models.AcknowledgementsThis work builds on the experience of many at the W.K.Kellogg Foundation who pioneered the appli-cation of logic modeling to their initiatives.For example,logic models were first used with the KelloggYouth Initiative Partnerships (KYIP).In this application,the models were instrumental in helping staffestablish program direction,implementation,an evaluation framework,and outcomes across three sites.In KYIP,logic modeling was used to facilitate and guide the development of the specific assumptionsand processes that ultimately led to the transition of the initiative from a WKKF-operated program to acommunity-owned program.WKKF program staff,including Tyrone Baines,Phyllis Meadows,GeraldSmith,Judy Watson Olson,Steve Peffers,Joyce Brown,and John Seita were instrumental in these efforts.Our work in developing the Logic Model Development Guidebegan at the request of Kellogg FoundationProgram Director Blas Santos who expressed a need for user-friendly tools and processes to support thework of grantees in Latin America and the Caribbean.The Logic Model Development Guide represents a collaborative effort.We particularly want to acknowl-edge the efforts of the Kellogg FoundationÕs former director of evaluation,Ricardo Millett,and histeam of evaluation managers,including Astrid Hendricks-Smith and Mark Lelle,who have since leftthe organization.Their tireless work among staff and grantees continues to promote the use of logicmodels to plan,design,and manage initiatives.Dale Hopkins and Karin Ladley were instrumental inbringing the material to print.We also wish to acknowledge the work of the Kellogg Foundation VicePresidents of Programs Rick Foster,Gail McClure,Dan Moore,and Gloria Smith,along with SeniorVice President of Programs Anne Petersen,who have underscored the importance of evaluation,embraced the logic model approach,and adopted it as a valued program support tool.Special thanks are extended to Cynthia Phillips,a primary writer and consultant throughout thedevelopment of this guide,and Work Volk Consultants,LLP,for formatting and editorial assistance.Thanks,also,to Beverly Parsons of In Sites;Andrew Hahn and the students at the Florence HellerGraduate School for Advanced Studies in Social Welfare,Brandeis University;Marc Osten,SummitConsulting Collaborative;Sally Bond,The Program Evaluation Group;Joel Meister and Eva Moya,University of Arizona;Amy Coates-Madsen and staff at Maryland Association of NonprofitOrganizations;and Gail Randall,Greater Worchester Community Foundation.ÐThe Program Staff of the W.K.Kellogg FoundationLogic Model Development GuidePage IV
Introduction to Logic ModelsChapter One defines logic models and explains their usefulness to program stakeholders. Youwill learn the relevance of this state-of-the-art tool to program planning, evaluation, andimprovement.Effective program evaluation does more than collect, analyze, and provide data. Itmakes it possible for you Ð program stakeholders Ð to gather and use information,to learn continually about and improve programs that you operate in or fund.The W.K. Kellogg Foundation believes evaluation Ð especially program logic modelapproaches Ð is a learning and management tool that can be used throughout a programÕslife Ð no matter what your stake in the program. Using evaluation and the logic modelresults in effective programming and offers greater learning opportunities, better docu-mentation of outcomes, and shared knowledge about what worksand why. The logicmodel is a beneficial evaluation tool that facilitates effective program planning, imple-mentation, and evaluation. The Whatand Whyof the Logic ModelThe WHAT: Logic Model DefinitionBasically, a logic model is a systematic and visual way to present and share your under-standing of the relationships among the resources you have to operate your program, theactivities you plan, and the changes or results you hope to achieve. Figure 1. The Basic Logic Model.The most basic logic model is a picture of how you believe your program will work. It useswords and/or pictures to describe the sequence of activities thought to bring about changeand how these activities are linked to the results the program is expected to achieve. Logic Model Development GuidePage 1A program logic model is apicture of how your pro-gram works Ð the theoryand assumptions underly-ing the program. …Thismodel provides a road mapof your program, high-lighting how it is expectedto work, what activitiesneed to come before others,and how desired outcomesare achieved (p. 35).W.K. KelloggFoundation EvaluationHandbook (1998)Chapter 1Resources/InputsActivitiesOutputsOutcomesImpact15432Your Planned WorkYour Intended Results
The Basic Logic Model components shown in Figure 1 above are defined below. Thesecomponents illustrate the connection between your planned workand your intended results.They are depicted numerically by steps 1 through 5. YOUR PLANNED WORK describes what resources you think you need to implementyour program and what you intend to do.1.Resourcesinclude the human, financial, organizational, and community resources a program has available to direct toward doing the work. Sometimes this component is referred to as Inputs.2.Program Activitiesare what the program does with the resources. Activities are the processes, tools, events, technology, and actions that are an intentional part of the programimplementation. These interventions are used to bring about the intended programchanges or results.YOUR INTENDED RESULTS include all of the programÕs desired results (outputs, out-comes, and impact). 3.Outputsare the direct products of program activities and may include types, levels andtargets of services to be delivered by the program.4.Outcomes are the specific changes in program participantsÕ behavior, knowledge, skills,status and level of functioning. Short-term outcomes should be attainable within 1 to 3years,while longer-term outcomes should be achievable within a 4 to 6 year timeframe.The logicalprogression from short-term to long-term outcomes should be reflected inimpact occurringwithin about 7to 10 years.5.Impact is the fundamental intended or unintended change occurring in organizations,communities or systems as a result of program activities within 7to 10 years. In the cur-rent model of WKKF grantmaking and evaluation, impact often occurs after the conclu-sion of project funding.The term logic modelis frequently used interchangeably with the term program theoryinthe evaluation field. Logic models can alternatively be referred to as theorybecause theydescribe how a program works and to what end (definitions for each employed by leadingevaluation experts are included in the Resources Appendix).The What: How to ÒReadÓ a Logic ModelWhen ÒreadÓ from left to right, logic models describe program basics over time fromplanning through results. Reading a logic model means following the chain of reasoningor ÒIf…then…Ó statements which connect the programÕs parts. The figure below showshow the basic logic model is read.Logic Model Development GuidePage 2Most of the value in a logicmodel is in the process ofcreating, validating, andmodifying the model ÉThe clarity of thinkingthat occurs from buildingthe model is critical to theoverall success of the pro-gram (p. 43).W.K. Kellogg FoundationHandbook (1998)Chapter 1
Figure 2. How to Read a Logic Model.The WHY: Logic Model Purpose and Practical ApplicationThe purpose of a logic model is to provide stakeholders with a road map describing thesequence of related events connecting the need for the planned program with the pro-gramÕs desired results. Mapping a proposed program helps you visualize and understandhow human and financial investments can contribute to achieving your intended programgoals and can lead to program improvements. A logic model brings program concepts and dreams to life. It lets stakeholders try an idea onfor size and apply theories to a model or picture of how the program would function. Thefollowing example shows how the logic model approach works. (If you are familiar with logicmodels, you may wish to skip ahead to the section entitled ÒWhy Use A Logic Model?Ó)Logic Model Development GuidePage 3Sample Factors influencing the trip:¥ Family membersÕ schooland work schedules¥ The holidays¥ Winter weather¥Frequent Flier availabilitySample Activities:¥ Creating/checking familyschedules ¥ Gathering holiday flightand FF information¥ Getting airport transportation ¥ Notifying Iowa relativesAn Example:We are proposing an inexpensive family trip from Charleston, South Carolina, to Des Moines, Iowa,to visit relatives during December school holidays. The seasonal trip we dream of taking fromCharleston to Des Moines is the Òprogram.Ó Basic assumptions about our trip ÒprogramÓ are:¥ We want to visit relatives between 12/10/00 and 1/5/01 while the children are out of school. ¥ We will fly from South Carolina to Iowa because it takes less time than driving and becausefrequent flier (FF) miles are available.¥ Using frequent flier miles will reduce travel costs.We have to determine the factors influencing our trip, including necessary resources, such as, thenumber of family members, scheduled vacation time, the number of frequent flier miles we have,round trip air reservations for each family member, and transportation to and from our home tothe airport. The activities necessary to make this happen are the creation of our own family holidayschedule, securing our Iowa relativeÕs schedule, garnering air line information and reservationsand planning for transportation to and from the airport. Resources/InputsActivitiesOutputsOutcomesImpactCertainresources areneeded tooperate yourprogramIf you have access to them, then you can use themto accomplishyour plannedactivitiesIf youaccomplishyour plannedactivities, thenyou will hopefully deliverthe amount ofproduct and/orservice thatyou intendedIf youaccomplishyour plannedactivities to the extent youintended, thenyour participants will benefit incertain waysIf thesebenefits toparticipants areachieved, then certain changes in organizations,communities,or systemsmight beexpected tooccur15432Your Planned WorkYour Intended Results
In this example, the results of our activities Ð or outputs Ð are mostly information, such asfamily schedules, flight schedules, and cost information based on the time frame of the trip.This information helps identify outcomes or immediate goals. For instance, if we makereservations as soon as possible, we are able to find flights with available frequent flier slotsand probably have more options for flights that fit within the time frame. Knowing this,our outcomes improve Ð reservations made well in advance result in flight schedules andairline costs that suit our timeline andtravel budget. Longer-term impact of our trip is notan issue here, but might be projected as continued good family relationships in 2010.Using a simple logic model as a trip-planning tool produced tangible benefits. It helpedus gather information to influence our decisions about resources and allowed us to meetour stated goals. Applying this process consistently throughout our trip planning posi-tions us for success by laying out the best course of action and giving us benchmarks formeasuring progress Ð when we touch down in Charlotte and change planes forCincinnati, we know weÕre on course for Des Moines.Typical logic models use table and flow chart formats like those presented here to cata-logue program factors, activities, and results and to illustrate a programÕs dimensions.Most use text and arrows or a graphic representation of program ideas. This is what ourtrip planning ÒprogramÓ could look like in logic model format.It was easy to organize travel plans in a flow chart, but we could also choose to organize anddisplay our thinking in other ways. A logic model does not have to be linear. It may appearas a simple image or concept map to describe more complex program concepts. Settling ona single image of a program is sometimes the most difficult step for program stakeholders.Logic Model Development GuidePage 4You canÕt do ÒgoodÓ evalua-tion if you have a poorlyplanned program.Beverly Anderson Parsons(1999) Resources/InputsActivitiesOutputsOutcomesImpactHolidayflightschedulesFamilyschedulesFrequentflyerholidayoptionsHolidayweatherCreatefamilyscheduleGetholidayflight infoGet ticketsArrangegroundtransportTicketsfor allfamilymembersFrequentflyer milesusedMoneysavedFamilymembersenjoyvacationContinuedgoodfamilyrelations15432Your Planned WorkTrip PlanningYour Intended ResultsTrip Results¥¥¥¥¥¥¥¥¥¥¥¥¥Chapter 1
Why Use a Logic Model?As you can see from the travel plan example, logic models are useful tools in many ways.Because they are pictorial in nature, they require systematic thinking and planning to better describe programs. The visual representation of the master plan in a logic model is flexible, points out areas of strength and/or weakness, and allows stakeholders to runthrough many possible scenarios to find the best. In a logic model, you can adjustapproaches and change courses as program plans are developed. Ongoing assessment,review, and corrections can produce better program design and a system to strategicallymonitor, manage, and report program outcomes throughout development and implementation. Effective evaluation and program success rely on the fundamentals of clear stakeholderassumptions and expectations about how and why a program will solve a particular prob-lem, generate new possibilities, and make the most of valuable assets. The logic modelapproach helps create shared understanding of and focus on program goals and method-ology, relating activities to projected outcomes.Logic Models Better Position Programs For SuccessMany evaluation experts agree that use of the logic model is an effective way to ensureprogram success. Using a logic model throughout your program helps organize and sys-tematize program planning, management, and evaluation functions. 1. In Program Design and Planning, a logic model serves as a planning tool to developprogram strategy and enhance your ability to clearly explain and illustrate program con-cepts and approach for key stakeholders, including funders. Logic models can help craft structure and organization for program design and build inself-evaluation based on shared understanding of what is to take place. During the plan-ning phase, developing a logic model requires stakeholders to examine best practiceresearch and practitioner experience in light of the strategies and activities selected toachieve results. 2. In ProgramImplementation, a logic model forms the core for a focused managementplan that helps you identify and collect the data needed to monitor and improve programming.Using the logic model during program implementation and management requires you tofocus energies on achieving and documenting results. Logic models help you to considerand prioritize the program aspects most critical for tracking and reporting and makeadjustments as necessary. 3. For Program Evaluation and Strategic Reporting, a logic model presents programinformation and progress toward goals in ways that inform, advocate for a particular pro-gram approach, and teach program stakeholders. Logic Model Development GuidePage 5If program planners donÕt have any hypothesesguiding them, their potential for learning fromthe initiative is low, andthe program is probably in trouble (p. 1). Everything You Wanted toKnow About Logic Modelsbut Were Afraid to Ask,Connie Schmitz andBeverly Anderson Parsons(1999)The bane of evaluation is apoorly designed program.Ricardo Millett, Director,WKKF Evaluation Unit
We all know the importance of reporting results to funders and to community stakehold-ers alike. Communication is a key component of a programÕs success and sustainability.Logic models can help strategic marketing efforts in three primary ways:¥ Describing programsin language clear and specific enough to be understood and evaluated.¥ Focusing attention and resourceson priority program operations and key results for thepurposes of learning and program improvement. ¥ Developing targeted communicationand marketing strategies.The Table below describes the relationship between a successful program and the benefitsderived from the use of logic models. How Logic Models Better Position Programs Toward Success.Logic Models Strengthen the Case for Program Investment Clear ideas about what you plan to do and why Ð as well as an organized approach tocapturing, documenting, and disseminating program results Ð enhance the case for investment in your program.Logic Model Development GuidePage 6Program ElementsPlanning and DesignProgramImplementation andManagementEvaluation,Communication, andMarketingCriteria for Program Success1Program goals and objectives,and important side effects arewell defined ahead of time.Program goals and objectives areboth plausible and possible.Relevant, credible, and useful per-formance data can be obtained.The intended users of the evalua-tion results have agreed on howthey will use the information.Benefits of Program Logic Models2Finds ÒgapsÓ in the theory or logic of aprogram and work to resolve them.Builds a shared understanding of whatthe program is all about and how theparts work together. Focuses attention of management on themost important connections betweenaction and results.Provides a way to involve and engagestakeholders in the design, processes,and use of evaluation.1Wholey, J. S., Hatry, H. P., & Newcomer, K. E. (Eds.). (1994). Handbook of Practical Program Evaluation.San Francisco: Jossey-Bass Publishers.2Barley, Z., Phillips, C., & Jenness, M. (1998). Decoding Program Logic Models. Workshop presented at theAnnual Meeting of the American Evaluation Association, Chicago, IL, November, 1998. There are many ways toconduct evaluations, andprofessional evaluators tendto agree that there is noÒone best wayÓ to do anyevaluation. Instead, goodevaluation requires careful-ly thinking through thequestions that need to beanswered, the type of pro-gram being evaluated, andthe ways in which theinformation generated willbe used. Good evaluation,in our view, should provideuseful information aboutprogram functioning thatcan contribute to programimprovement.W.K. Kellogg FoundationEvaluation Unit Chapter 1
Developing a Program Logic Model Requires a Simple Image and aStraightforward Approach A picture IS worth a thousand words. The point of developing a logic model is to comeup with a relatively simple image that reflects how and why your program will work.Doing this as a group brings the power of consensus and group examination of values andbeliefs about change processes and program results.Logic Models Reflect Group Process and Shared UnderstandingFrequently, a professional evaluator is charged with developing a logic model for programpractitioners. But a logic model developed by all stakeholders Ð program staff, partici-pants, and evaluators Ð produces a more useful tool and refines program concepts andplans in the process. We recommend that a logic model be developed collaboratively in aninclusive, collegial process that engages as many key stakeholders as possible. This guideprovides a step-by-step process to assist program planners.Like Programs, Logic Models Can Change Over TimeAs a program grows and develops, so does its logic model. A program logic model ismerely a snapshot of a program at one point in time; it is not the program with its actualflow of events and outcomes. A logic model is a work in progress, a working draft thatcan be refined as the program develops. Simple Logic Model BasicsCreating a logic model: What they look like and what needs to be included Logic models come in as many sizes and shapes as the programs they represent. A simplemodel focuses on project-level results and explains five basic program components. Theelements outlined below are typical of the model promoted by United Way of America tosupport an outcomes-based approach to program planning and evaluation. Developing and Reading a Basic Logic Model Read from left to right, logic models describe program basics over time, beginning withbest practice information or knowledge about Òwhat worksÓ from successful programpractitioners and other trusted authorities. Reading a logic model means following thechain of reasoning or ÒIf…then…Ó statements which connect the programÕs parts. The graybox in the left column defines the assumptions stated in ÒIf…then…Ó terms.Logic Model Development GuidePage 7LOGIC MODELIFÉTHENAssumptions:¥ Certain resources areneeded to operate yourprogram.¥ Ifyou have access tothem, thenyou can usethem to accomplishyour planned activities. ¥ Ifyou accomplish yourplanned activities, then,you will, it is hoped,deliver the amount ofproduct and/or servicethat you intended.¥ Ifyou accomplish yourplanned activities to theextent intended, thenyour participants willbenefit in specific ways. ¥ Ifthese benefits to par-ticipants are achieved,thencertain changes inorganizations, commu-nities, or systems mightoccur under specifiedconditions.
Building a Logic Model by Basic Program Components As you conceptualize your program, begin by describing your basic assumptions and thenadd the following program components in the order that they should occur.1.Factors are resources and/or barriers, which potentially enable or limit program effec-tiveness. Enabling protective factorsor resourcesmay include funding, existing organizations,potential collaborating partners, existing organizational or interpersonal networks, staffand volunteers, time, facilities, equipment, and supplies. Limiting risk factorsor barriersmight include such things as attitudes, lack of resources, policies, laws, regulations, andgeography.2.Activities are the processes, techniques, tools, events, technology, and actions of the planned program. These may include products Ð promotional materials and educationalcurricula; services Ð education and training, counseling, or health screening; and infrastructure Ð structure, relationships, and capacity used to bring about the desired results.3.Outputsare the direct resultsof program activities. They are usually described in terms of the size and/or scope of the services and products delivered or produced by the program. They indicate if a program was delivered to the intended audiences at the intended Òdose.Ó A program output, for example, might be the numberof classes taught, meetingsheld, or materials produced and distributed; program participation ratesand demography;or hours of each type of serviceprovided. 4.Outcomesare specific changes in attitudes, behaviors, knowledge, skills, status, or level of functioningexpected to result from program activities and which are most often expressed at an individual level. 5.Impactsare organizational, community, and/or system level changesexpected to result from program activities, which might include improved conditions, increased capacity, and/or changes in the policy arena. Thinking about a program in logic model terms prompts the clarity and specificityrequired for success, and often demanded by funders and your community. Using a sim-ple logic model produces (1) an inventory of what you have and what you need to oper-ate your program; (2) a strong case for how and why your program will produce yourdesired results; and (3) a method for program management and assessment.Other Logic Model Examples In practice, most logic models are more complex and fall into one of three categories: thetheory approach model (conceptual), outcome approach model, or activities approachmodel (applied) Ð or a blend of several types. It is not unusual for a program to use allthree types of logic models for different purposes. No one model fits all needs, so you willLogic Model Development GuidePage 8Chapter 1
need to decide exactly what you want to achieve with your logic model Ð and where youare in the life of your program Ð before deciding on which model to use. Types of Logic Models: Emphasis and StrengthsDescriptions of Three Approaches to Logic Models: Which Fits Your Program? 1.Theory Approach Modelsemphasize the theory of change that has influenced the design and plan for the program. These logic models provide rich explanation of the reasons for beginning to explore an idea for a given program. Sometimes they have additional parts that specify the problem or issue addressed by the program, describe the reasons for selecting certain types of solution strategies, connect proven strategies to potential activities, and other assumptions the planners hold that influence effective-ness. These models illustrate how and why you think your program will work. They are built from the Òbig pictureÓ kinds of thoughts and ideas that went into conceptual-izing your program. They are coming to be most often used to make the case in grant proposals. Models describing the beginnings of a program in detail are most useful during program planning and design.Logic Model Development GuidePage 9Types of Logic Models:Emphasis and StrengthsA program is a theory andan evaluation is its test. Inorder to organize the eval-uation to provide a respon-sible test, the evaluatorneeds to understand thetheoretical premises onwhich the program is based(p. 55). Carol Weiss (1998) Intended ResultsBeginningsShould contribute to the results you expect based on this theory of changeIf your assumptions about the factors that influence your issues hold true…Planned WorkThen, the activities you plan to do which build on these assumptions…Evaluation,Communication,MarketingImplementationPlanning &DesignGrant ProposalReports & Other MediaManagement Plantheory typeoutcomes typeactivities typeProgram LogicModelwhat we havedone so farhow we will dowhat we say we will dowhat wehopeto do
2.Outcomes Approach Modelsfocus on the early aspects of program planning and attempt to connect the resources and/or activities with the desired results in a workableprogram. These models often subdivide outcomes and impact over time to describe short-term (1to 3 years), long-term (4to 6 years), and impact (7to 10 years) that mayresult from a given set of activities. Although these models are developed with a theoryof change in mind, this aspect is not usually emphasized explicitly. Models that outlinethe approach and expectations behind a programÕs intended results are most useful in designing effective evaluation and reporting strategies.3.Activities Approach Models pay the most attention to the specifics of the implementationprocess. A logic model of this type links the various planned activities together in a manner that maps the process of program implementation. These models describe what a program intends to do and as such are most useful for the purposes of program monitoring and management. This type provides the detailed steps you think you will need to follow to implement your program. It shows what you will actually doin your community if your proposal is funded. Models that emphasize a programÕs planned work are most often used to inform management planning activities.Working Through Theory Approach Logic Models Emphasizes AssumptionsA theory approach logic model links theoretical ideas together to explain underlying pro-gram assumptions. The focus here is on the problem or issue and the reasons for propos-ing the solution suggested in your programÕs approach. Remember, the theory logic modelis broad and about Òbig ideas,Ó not about specific program Ònuts and bolts.Ó Noted evaluator and program theorist Carol Weiss (1998) explains that for program plan-ning, monitoring, and evaluation, it is important to know not only whatthe programexpects to achieve but also how. We mustunderstand the principles on which a programis based, a notion not included in evaluation until recently. Discussions about thewhethers, hows, and whysof program success require credible evidence and attention to thepaths by which outcomes and impacts are produced. The theory logic model is suitable for use by funders and grantees. A case example of itsuse is provided below. In this case, the model describes a WKKF cluster initiativeÕs (Comprehensive CommunityHealth Models of Michigan) programming strategy or its theory of change. Notice thatthis model places emphasis on ÒYour BeginningsÓ by including the assumptions identifiedby program planners as the principles behind the design of the initiative.Logic Model Development GuidePage 10The purpose of using pro-gram logic models inWKKF grantmaking is tohelp internal and externalstakeholders understandhow the FoundationÕsinvestment will contributeto achieving the intendedgoals. This understandingshould help these variousstakeholders makeinformed decisions aboutprogram priorities, fundingpriorities, assistance tograntees, evaluation of pro-gramming impact, andmarketing, communica-tion, and marketing strate-gies.W.K. KelloggFoundationEvaluation Handbook(1998)Chapter 1
Working with Outcome Approach Models Highlights Activities andProgram ImplementationOutcome approach logic models display the interrelationships between specific programactivities and their outcomes. On the next page is an example drawn from the CalhounCounty Health Improvement Program, funded under the Comprehensive CommunityHealth Models of Michigan initiative. This linear, columnar model emphasizes the causal linkagesthought to exist among pro-gram components. The arrows show which sets of activities program developers believedwould contribute to what outcomes. These statements serve as logical assertions about theperceived relationship among program operations and desired results and are the hallmarkof the logic model process. Notice that this model emphasizes ÒYour Intended ResultsÓ in the greatest relative detailand anticipates achievement outside the time allotted for the initiative.Logic Model Development GuidePage 11These models help build acommon understandingbetween managers andevaluators…. Such agree-ment is a prerequisite forevaluation work that islikely to be useful to man-agement. [These models]display the key events(inputs, activities, outcomes) that could bemonitored and the assumedcausal linkages that couldbe tested in evaluations of the program.Joseph S. Wholey, Harry P. Hatry, and K.E. Newcomer (1994) AssumptionsHealth is a community issue andcommunities will form partnershipsto resolve health care problems.Commnities can influence andshape public and market policy atthe local, state, and national levels.External agents, working inpartnership with communities, canserve as catalysts for change.Shifting revenues and incentives toprimary care and prevention will improve health status.Information on health status andsystems is required for informeddecision making.Your BeginningsYour Planned WorkYour Intended ResultsOutputsOutcomesImpactInputsActivitiesActiveParticipationin the Reform ProcessInclusiveCommunityDecision-MakingCommunity-wideCoverage andAccessComprehensive,IntegratedHealth CareDelivery SystemCommunity Health AssessmentCommunity-basedHealth InformationSystemsImprovedHealth StatusIncreasedHealth CareSystemEfficiencyMore EffectiveDistribution ofCommunityHealth CareResourcesAdministrative Processes forHealth Data, Policy, andAdvocacyExternalTechnicalAssistanceConsumersProvidersPayersStaffExample of a Theory Logic model (Adapted from WKKFÕs Comprehensive Community Health Models ofMichigan).
Using the Activities Approach Models to Track Outcomes The activities approach logic model also connects program resources and activities todesired results but does so in very great detail. Each outcome is usually dealt with sepa-rately by the activities and events that must take place to keep the program on track. The model emphasizing ÒYour Planned WorkÓ can be used as a work plan or manage-ment tool for program components and in conjunction with other models. Notice how it points out what program activities need to be monitored and what kind ofmeasurements might indicate progress toward results. Below is one model describing theconnections between project tasks and outcome achievement for the community coveragestrand from the outcome approach example provided earlier.Logic Model Development GuidePage 12Not only will a logic modelclarify each element of yourprogram, it will enable youto respond to the question:ÒTo what do I want to beheld accountable?Ó.The Evaluation Forum(1999) Activities that encourageconsumers, providers, and payersto seek support, and achievecommon goals.Activities that increase consumerawareness and access to healthpromotion, disease prevention, and primary care services.Activities that increase linkagesamong medical, health, andhuman service systems.Activities that lead to thedevelopment of a communityaccess and coverage plan.Activities that lead to thedevelopment of a community health information network.Activities that lead to the development of a community health assessment and reporting program.ActivitiesOutputsInputsConsumers,providers, and payers to participate in governanceprocesses.Sufficient staff withexpertise andleadership skills toimplement theprogram at thelocal level.Sufficient externaltechnicalassistance tosupport staff inprogramimplementationConsumers, providers, andpayers serving on the CCHIPGoverning Board seek, support, and achieve common goals.Increased community access andparticipation in health promotion,disease prevention, and primary care services.Linkages are forged amongmedical, health, and human service systems.Third-party administered contractfor community-wide coverageis in place.Fiber-optic information networkis in place (CHIN).Community health assessment andreporting program is in place.CCHIP Governing Board isdeemed inclusive andaccountable by the community stakeholders.Increased numbers of community members utilize the health promotion, disease prevention, and primary care service providedImproved access/coverage for theinsured, under-, and non-insuredin the community.Improved Health StatusCommunity members utilize theCHIN for information collection, storage, analysis, and exchange.Information provided by the Health Report Card is used tomake community health decisions.OutcomesImpactYour Planned WorkYour Intended ResultsChapter 1Example of an Outcome Approach model (example drawn from the Calhoun County Health ImprovementProgram, funded under the Comprehensive Community Health Models of Michigan initiative).
Adapted from the Calhoun County Health Improvement Program, one site of WKKFÕs ComprehensiveCommunity Health Models of Michigan initiativeThere Is No BestLogic ModelTry several on for size. Choose the model that fits your program best and provides theinformation you need in the format that is most helpful. Like anything else, it takes prac-tice to use logic models as effective program tools. We learn through trial and error tofind what works best for what program. DonÕt hesitate to experiment with program logicmodel design to determine what works best for your program. And donÕt be concerned ifyour model doesnÕt look like one of the case examples.The following show how the logic model forms gather information that can be usedthroughout your programÕs life Ð from defining the theory on which your program reststo evaluating program impact.Logic Model Development GuidePage 13Insurance market issues areidentified and documented.Insurance market issues areprioritized based on potentialfor successful reform.The Purchasing Alliance will identify insurance market issues and strategies to reform thoseidentified issues will bedeveloped and implemented.High priority issues areidentified and examined.Strategies to reform the high priority issues identified have been developed.Change agents with sufficientcapacity and resources tosuccessfully execute insurance market reform are identified.Equitable access to community-wide coverage.Change agents contracted toimplement insurance marketreform (minimum of 2).% decrease of peopleuninsured (201).% decrease of new Medicaideligible consumersachieving coverage beforein the hospital (203).% in Medicaid participatingproviders, using $1000 threshold level (204).DeliverableÐ6Your Planned WorkYour Intended ResultsMilestone ActivitiesOutputsOutcomesActivities to increase beneficiaryenrollment and provider participation in Medicaid and other third party sponsored insurance and reimbursment plans (2P1)
How to use a Logic Model Through the Life of Your Program:Logic Model Development GuidePage 14CLARIFYING PROGRAM THEORY:1.PROBLEM OR ISSUE STATEMENT:Describe the problem(s) your program is attempting to solve or the issue(s) your program will address.2.COMMUNITY NEEDS/ASSETS:Specify the needs and/or assets of your community that led your organization to design a program that addresses the problem. 3.DESIRED RESULTS (OUTPUTS, OUTCOMES AND IMPACTS):Identify desired results, or vision of the future, by describing what you expect to achieve near- and long-term.4.INFLUENTIAL FACTORS:List the factors you believe will influence change in your community.5.STRATEGIES:List general successful strategies or Òbest practicesÓ that have helped communities like yours achieve the kinds of results your program promises.6.ASSUMPTIONS:State the assumptions behind howand whythe change strategies will work in your community.1. Program Planning2. Program Implementation3. Program EvaluationDEMONSTRATING YOUR PROGRAMÕS PROGRESS:1.OUTPUTS:For each program activity, identify what outputs (service delivery/implementationtargets) you aim to produce.2.OUTCOMES:Identify the short-term and long-term outcomes you expect to achieve for each activity. 3.IMPACT:Describe the impact you anticipate in your community in 7 to 10 years with each activity as a result of your program.4.ACTIVITIES:Describe each of the activities you plan to conduct in your program.5.RESOURCES:Describe the resources or influential factors available to support your programactivities.PROGRAM EVALUATION QUESTIONS AND INDICATORS:1.FOCUS AREA:From your program theory logic model, list the components of the most important aspects of your program.2.AUDIENCE:Identify the key audiences for each focus area. Who has an interest in your program? 3.QUESTIONS:For each focus area and audience, list the questions they may have about your program.4.INFORMATION USE:For each audience and question you have identified, identify the ways you will use the evaluation information.5.INDICATORS: Describe what information could be collected that would indicate the status of your program and its participants for each question.6.TECHNICAL ASSISTANCE:Indicate the extent to which your organization has the evaluationand data management expertise to collect and analyze the data that relates to this indicator.StrategiesAssumptionsProblem or IssueInfluentialFactorsDesired Results(outputs, outcomes,and impact)Community Needs/Assets123456Focus AreaQuestionIndicators Technical AssistanceNeededÒ RESOURCESIn order to accomplishour set of activities wewill need the following:ACTIVITIESIn order to address ourproblem or asset we willaccomplish the followingactivities:OUTPUTS SHORTWe expect that onceaccomplished these activities will produce the following evidence or service delivery:SHORT & LONG-TERMOUTCOMESWe expect that if accom-plished these activitieswill lead to the followingchanges in 1-3 then 4-6years:IMPACTWe expect that if accom-plished these activitieswill lead to the followingchanges in 7-10 years: Evaluation Focus AreaAudienceQuestion UseChapter 1For more detail, see the ProgramPlanning Template on p. 57.For more detail, see the EvaluationPlanning Template on p. 59.For more detail, see the IndicatorsDevelopment Template on p. 61.For more detail, see the ProgramImplementation Template on p. 54.
Developing a Basic Logic Model For Your ProgramDrawing a picture of how your program will achieve resultsWhether you are a grantseeker developing a proposal for start-up funds or agrantee with a program already in operation, developing a logic model canstrengthen your program. Logic models help identify the factors that willaffect your program and enable you to anticipate the data and resources you will need toachieve success. As you engage in the process of creating your program logic model, yourorganization will systematically address these important program planning and evaluationissues: ¥ Cataloguing of the resources and actions you believe you will need to reach intendedresults.¥ Documentation of connections among your available resources, planned activities andthe results you expect to achieve.¥ Description of the results you are aiming for in terms of specific, measurable, action-ori-ented, realistic and timed outcomes.The exercises in this chapter gather the raw material you need to draw a basic logic modelthat illustrates how and why your program will work andwhat it will accomplish. Youcan benefit from creating a logic model at any point in the life of any program. The logicmodel development process helps people inside and outside your organization understandand improve the purpose and process of your work.Chapter 2 is organized into two sections Ð Program Implementation, and Program Results.The best recipe for program success is to complete both exercises. (Full-size masters of eachexercise and the checklists are provided in the Forms Appendix at the back of the guide foryou to photocopy and use with stakeholder groups as you design your program.)Exercise 1: Program Results. In a series of three steps, you describe the results you plan toachieve with your program.Exercise 2: Program Resources and Activities by taking you through three steps that con-nect the programÕs resources to the actual activities you plan to do. The Mytown ExampleThroughout Exercises 1 and 2 weÕll follow an example program to see how the logicmodel steps can be applied. In our example, the folks in Mytown, USA, are striving tomeet the needs of growing numbers of uninsured residents who are turning to MemorialHospitalÕs Emergency Room for care. Because that care is expensive and not the best wayto offer care, the community is working to create a free clinic. Throughout the chapters,MytownÕs program information will be dropped into logic model templates for ProgramPlanning, Implementation, and Evaluation.Logic Model Development GuidePage 15Chapter 2Over the past few years, Ihave markedly changed myapproach to logic modeling.I have become convincedthat it makes a consider-able difference if you do theoutcomes before planningthe activities. I definitely advocate doingthe outcomes first! I findthat people come up withmuch more effective activi-ties when they do. Use themotto, Òplan backward,implement forward.ÓBeverly Anderson Parsons,WKKF Cluster Evaluator
Novice logic modelers may want to have copies of the Basic Logic Model Template infront of them and follow along. Those readers with more experience and familiarity maywant to explore the text and then skip ahead to the completed Basic Logic Model for theMytown Example on page 34.Demonstrating Progress Toward Change The Importance of Documenting ProgressAccording to many funders, grant applications frequently lack solid descriptions of howprograms will demonstrate their effectiveness. Some grantees think activities are ends untothemselves. They report the numbers of participants they reach or the numbers of train-ing sessions held as though they were results. Conducting an activity is notthe same as achieving results from the accomplishment of thatactivity. For example, being seen by a doctor is different from reducing the number of unin-sured emergency room visits. Tracking data like meetings held or patients enrolled doesmonitor your programÕs implementation and performance, but those data are outputs(activ-ity data), not outcomes (which refer to the results you expect to achieve in future years). ÒDo the outcomes firstÓ is sage advice. Most logic models lack specific short- and long-term outcomes that predict what will be achieved several years down the road. Specifyingprogram milestones as you design the programbuilds in ways to gather the data requiredand allows you to periodically assess the programÕs progress toward the goals you identify.For that reason, Exercise 1 isnÕt filled out from left to right. This exercise asks you toÒdo the outcomes first.Ó We will focus our attention first on what we have called Òyourintended results.ÓAs you implement your program, outcome measures enhance program success by assess-ing your progress from the beginning and all along the way. That makes it possible tonotice problems early on. The elements (Outputs, Outcomes, and Impact) that compriseyour intended resultsgive you an outline of what is most important to monitor and gaugeto determine the effectiveness of your program. You can correct and revise based on yourinterpretation of the collected data. Exercise 1 Ð Describing ResultsDescribe the results you desire Ð Outputs, Outcomes and ImpactIf you were running the Mytown Free Clinic, how would you show that your desired out-come (a reduction in uninsured emergency care) didnÕt result from a mass exodus of unin-sured residents from Mytown, USA, or a sudden increase in number of employees offeredhealth insurance coverage by local businesses? Logic Model Development GuidePage 16Chapter 2
How will you demonstrate that your program contributed to the change you intend? Awell-crafted logic model can assert it is reasonable to claim that your program made a sub-stantive contribution to your intended change. When programs operate in real communi-ties where influences and forces are beyond your control, evaluation is generally moreabout documenting a programÕs contribution than about proving something. Community-based initiatives operate in complex environments where the scientific certainty of ÒproofÓis seldom attainable. This is where logic models can be especially helpful. INSTRUCTIONS:Exercise 1 will use the Basic Logic Model Development Template. Inparticular, you will use the information presented in the gray text boxes that follow aboutthe Mytown example program to determine what results are intended for this program.Example information about outcomes, impacts, and outputs are provided. You will fill inthe blank Basic Logic Model Development Template to illustrate first the outcomes andimpacts sought and then the outputs. You can then look at the completed template onpage 25 to see compare your interpretation with that produced by the Mytown folks. Exercise 1 uses the Basic Logic Model Development TemplateOutcomes and Impacts should be SMART:¥ Specific¥ Measurable¥ Action-oriented¥ Realistic¥ TimedLogic Model Development GuidePage 17ResourcesIn order to accom-plish our set ofactivities we willneed the following:ActivitiesIn order to addressour problem orasset we will con-duct the followingactivities:OutputsWe expect that oncecompleted or underway these activitieswill produce the fol-lowing evidenceofservice delivery:Short- & Long-Term OutcomesWe expect that if completed or ongo-ing these activitieswill lead to the fol-lowing changes in1Ð3 then 4Ð6 years:ImpactWe expect that ifcompleted theseactivities will lead to the followingchanges in 7Ð10 years:
Some logic models number the lists within a column to aid discussion. Some tabular logicmodels use rows to order and show the relationships among components. Some logicmodels, like the outcome and activity examples provided in Chapter One, use a box andarrow format to illustrate the Òcausal linkagesÓ demonstrating how your resources, activi-ties, outputs, outcomes, and impact connect to form chains. These depictions add to theclarity of your logic model/evaluation plan. However, for the most basic of logic models,the inventory approach we illustrate is sufficient to capture your thinking about how aprogram will work. The other techniques will improve its utility, but the most importanttask is to first get the component parts categorized and described. Once you have com-pleted the inventory table for this and Exercise 2 feel free to experiment with identifyingthe relationships among the items across columns.Short-term outcomes are results you expect to achieve one to three years after a programactivity is under way. Short-term outcomes are specific changes in things like attitudes, behaviors, knowledge,skills, status, or level of functioning expected to result from program activities. These usu-ally are expressed at an individual level among program participants. Long-term outcomes are results you expect to achieve in four to six years.Long-term outcomes are also specific changes in things like attitudes, behaviors, knowl-edge, skills, status, or level of functioning expected to result from program activities.These usually build on the progress expected by the short-term outcomes.Logic Model Development GuidePage 18EXAMPLES: Signed Memorandum of Agreement from the local technical college donatingclinic space, change in participantsÕ attitudes about the need for a medical home, increase innumbers of scheduled annual physicals, increased patient follow-up visits, change in staff’sawareness of patient scheduling challenges, increased appropriate referrals from ERÕs. Insert MytownÕs short-term outcomes in the Short- and Long-Term Outcomes Column of theBasic Logic Model Development Template.EXAMPLES: The clinic serves as a medical home for 500 uninsured patients. The clinic hassustained funding sources: patient co-payments ($10/visit) provide 20% of the ClinicÕs oper-ating costs, United Way provides 20%, Memorial Hospital donates 20%, the Medical Societycontributes 20% and an endowment established at the Community Foundation provides thefinal 20%. An annual golf tournament organized by the Kiwanis Club funds special clinic proj-ects. There has been a 25% reduction in uninsured emergency care since Mytown Free Clinicopened five years ago. In the ClinicÕs fifth year there is a 15% reduction in uninsured ER vis-its. Seventy-five volunteer administrators and 300 volunteer medical professionals regularlyserve at the clinic each year. Five companies donate all necessary medical supplies. Grantfunds purchase the computers and software needed to create electronic patient records. Forfive years patient satisfaction ratings have been 90%.Insert MytownÕs long-term outcomes in the Short- and Long-Term Outcomes column of theBasic Logic Model Development Template.Chapter 2
Impactrefers to the results expected seven to ten years after an activity is under way Ð thefuture social change your program is working to create.Impacts are the kinds of organizational, community, or system level changes expected toresult from program activities and which might include improved conditions, increasedcapacity, and/or changes in the policy arena.Outputs are data about activities. They are the direct results of program activities. They are usually described in terms ofsize and scope of the services or products delivered or produced by the program. Theyindicate whether or not a program was delivered to the intended audiences at the intendedÒdose.Ó A program output, for example, might include the number of classes taught, meet-ings held, materialsdistributed, program participation rates, or total service deliveryhours.Logic Model Development GuidePage 19EXAMPLES: Specific reduction in inappropriate emergency room use, increased donations ofclinic supplies to meet identified needs, a stable supply of medical volunteers, an endowmentsupporting 35% of the clinicÕs operating funds, 900 patients served/year. Insert MytownÕs impacts in the Impact Column of the Basic Logic Model DevelopmentTemplate.EXAMPLES:Number of patients referred to the Free Clinic from Memorial ER/year, the num-ber of patients screened/year, the number of qualified patients enrolled in the Free Clinic/year,the average number of patient visits/day, the total number of patient visits/year, the numberand specialties of medical volunteers, the number of volunteer administrators trained, thenumber and locations of clinic posters distributed, the number of potential patients calling forinformation/ month. Insert MytownÕs outputs in the Outputs Column of the Basic Logic Model DevelopmentTemplate.
Exercise 1 Checklist:Review what you have created using the checklist below to assess the quality of your draft.Logic Model Development GuidePage 20Progress Toward Results Quality Criteria1.A variety of audiences are taken into consideration whenspecifying credible outputs, outcomes, and impacts.2.Target participants and/or partners are described and quanti-fied as outputs (e.g. 100 teachers from 5 rural high schools).3.Events, products, or services listed are described as outputsin terms of a treatment or dose (e.g. 30 farmers will partici-pate in at least 3 sessions of program, or curriculum will bedistributed to at least 12 agencies).4.The intensity of the intervention or treatment is appropriatefor the type of participant targeted (e.g. higher-risk partici-pants warrant higher intensities).5.The duration of the intervention or treatment is appropriatefor the type of participant targeted (e.g. higher-risk partici-pants warrant longer duration). 6.Outcomes reflect reasonable, progressive steps that partici-pants can make toward longer-term results.7.Outcomes address awareness, attitudes, perceptions, knowl-edge, skills, and/ or behavior of participants.8.Outcomes are within the scope of the programÕs control orsphere of reasonable influence.9.It seems fair or reasonable to hold the program accountablefor the outcomes specified.10.The outcomes are specific, measurable, action-oriented, real-istic, and timed.11.The outcomes are written as change statements (e.g. thingsincrease, decrease, or stay the same).12.The outcomes are achievable within the funding and reportingperiods specified.13.The impact, as specified, is not beyond the scope of the pro-gram to achieve.CommentsRevisionsNot Yet Yes■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■Chapter 2
Exercise 2 Ð Describing ActionsLinking It All TogetherExercise 2 illustrates exactly how you plan to put your program theory to work. It leadsyou to identify the resources and activities your program will need to achieve your intend-ed results. This exercise documents your knowledge of the community resources you haveavailable and specific activities your program will implement. Program rationales in grant proposals are usually strong. Grantees tend to have a verygood sense of whatthey want to do. However, they frequently fail to make specific connections between their program and related best practice literature and practitionerwisdom that could and shouldsupport their approach and their work.To connect actions to program results, this exercise links your knowledge of what workswith specific descriptions of what your program will do. It requires you to anticipate whatwill be needed to support program activities. The elements that comprise your programimplementation act as a game plan for the program you propose. Most logic models list activity items and resources (like planning meetings, curriculumpurchase or design, training workshops, and service delivery). Depending on the nature of your effort, other types of products and processes may be included. Management-oriented logic models also include program and evaluation development, staff and volun-teer training, recruitment of partners and participants, and the publicity needed to supportyour work along the way.As mentioned earlier, if your program addresses multiple issues you may find it helpful togo through the exercises for each issue in turn and then aggregate them into a largermodel that highlights the relationships among issues. We recommend referring to a literature review on the problem your program is designedto address when you specify program activities. From this explicit knowledge of whatworks, you can more clearly connect the abstract strategies supporting the program to itsconcrete activities. When Exercise 2 is complete and you are satisfied that you have an accurate inventory ofthe Mytown programÕs component parts, transfer the information to the Basic LogicModel Development Template. Remember you have already filled in the three columnson the right with what you have learned about the intended results for the Mytown pro-gram example.What activities are planned?Based on what you know about effective ways to solveproblems or build assets, what specific activities have you planned?Logic Model Development GuidePage 21I would emphasize thatpeople may well changetheir minds about theactivities that are the mostuseful after having donethe results work.Beverly Anderson Parsons,WKKF Cluster Evaluator
What resources are needed?Once you have specified what you plan to do, determine theresources you will need to support the solutions your program proposes. For some typesof programs, it may also be helpful to describe the influential factors you are counting onto support your efforts in the community. Logic Model Development GuidePage 22EXAMPLES:Personnel Committee launches and completes search for full-time director.Director is hired and oriented to the board and the community. Board and staff visit theAnywhere Free Clinic to learn from its experience and to select documents to replicate (i.e.,policies and procedures, job descriptions, equipment needs, budgets, funding strategies, vol-unteer and patient records). Board and staff conduct program-planning retreat. Based uponAnywhereÕs funding plan, board secures Free ClinicÕs first-year funding. Marketing Committeecreates public relations campaign in collaboration with Volunteer Committee to secure volun-teers and patients. Facility Committee creates and completes MOA with technical college tosecure a clinic facility. Quality Assurance Committee creates evaluation plan in cooperationwith Memorial HospitalÕs Emergency Room staff and the local Chamber of Commerce. Summarize MytownÕs activities in the Activities column of the Basic Logic ModelDevelopment TemplateEXAMPLES:Medical Society/Memorial Hospital Task Force for the Uninsured will become aFree Clinic Board of Directors and secure a 501(c)(3) status from the IRS. The Board willrecruit 7Ð10 additional representatives from drug companies, the local technical school,MytownÕs United Way, the Chamber of Commerce, the Community Foundation, the VolunteerCenter, the Nurses Association, etc. During a 6-month planning period, board committees willbe launched; staff will be recruited/hired/oriented; a site visit will be conducted; and theClinicÕs first-yearÕs funding ($150,000/year) will be secured. Committees will create an MOAwith Memorial Hospital and the Medical Society to secure equipment required: 5 exam tables,7 desks, 5 blood pressure cuffs, 5 otoscopes, 5 stethoscopes, 5 PDRÕs, 1 set of scales, 10thermometers, three computers, one first aid emergency kit. Summarize MytownÕs resources in the Resources column of the Basic Logic ModelDevelopment Template.Chapter 2
Exercise 2 Checklist:Review what you have created using the checklist below to assess the quality of your draft.Logic Model Development GuidePage 23Theory into Action Quality Criteria1.Major activities needed to implementthe program are listed.2.Activities are clearly connected to thespecified program theory.3.Major resources needed to implementthe program are listed.4.Resources match the type of program.5.All activities have sufficient and appropriate resources. Comments/RevisionsNot Yet Yes■■■■■■■■■■■■■■■■
Here we include a flowchart that summarizes the steps to complete your basic logicmodel. Keep in mind that you could use this inventory style template to then furtherdescribe the relationships among the components using numbered items, rows, or boxesand arrows as we mentioned earlier.Flowchart for Exercises 1 & 2 Ð Describing Results, Resources, and ActivitiesLogic Model Development GuidePage 24Exercise 1 Describing ResultsExercise 2 Describing Resources and ActivitiesStep 1.1For each of the specific activities you have planned to do, whatshort-term and then long-term outcomes do you expect toachieve as indicators of the progress made by your programtoward its desired results?Step 1.2For each of the specific activities that you have planned to do,what outputs (service delivery or implementation targets) do youhope to reach through the operation of your program?Step 1.3For each of the specific activities you have planned to do, whatimpact do you expect to achieve in your community?Step 2.1Knowing what you know about what works to solve problems orbuild assets as specified in the theory of change for your pro-gram, what specific activities have you planned to do?Step 2.2What resources are available to your program to support thespecific activities you have planned to do (for some programs, itmay also be important to state those influential factors you arecounting on to support your work)?RESOURCESACTIVITIESOUTPUTSOUTCOMESIMPACTRESOURCESACTIVITIESOUTPUTSOUTCOMESIMPACTRESOURCESACTIVITIESOUTPUTSOUTCOMESIMPACTRESOURCESACTIVITIESOUTPUTSOUTCOMESIMPACTRESOURCESACTIVITIESOUTPUTSOUTCOMESIMPACTChapter 2OUTCOMESOUTPUTSIMPACTACTIVITIESRESOURCES34512
Logic Model Development GuidePage 25RESOURCESIn order to accomplishour set of activities wewill need the following:¥ IRS 501(c)(3) status¥ Diverse, dedicatedboard of directors representing potentialpartners¥ Endorsement fromMemorial Hospital,Mytown MedicalSociety, and UnitedWay¥ Donated clinic facility¥ Job descriptions forboard and staff¥ First yearÕs funding($150,000)¥ Clinic equipment¥ Board & staff orienta-tion process¥ Clinic budgetACTIVITIESIn order to address ourproblem or asset we willaccomplish the followingactivities:¥ Launch/completesearch for executivedirector¥ Board & staff conductAnywhere Free Clinicsite visit¥ Board & staff conductplanning retreat¥ Design and implementfunding strategy¥ Design and implementvolunteer recruitmentand training¥ Secure facility for clinic¥ Create an evaluationplan¥ Design and implementPR campaignOUTPUTSWe expect that onceaccomplished these activities will produce the following evidence or service delivery:¥ # of patients referredfrom ER to theclinic/year¥ # of qualified patientsenrolled in theclinic/year¥ # of patient visits/year¥ # of medical volunteersserving/year¥ # of patient flyers dis-tributed¥ # of calls/month seek-ing info about clinicSHORT- AND LONG-TERM OUTCOMESWe expect that if accom-plished these activitieswill lead to the followingchanges in 1Ð3 then 4Ð6years:¥ Memorandum ofAgreement for free clinic space¥ Change in patient atti-tude about need formedical home¥ Change in # of scheduled annual phys-icals/follow-ups¥ Increased # ofER/physician referrals¥ Decreased volume ofun-reimbursed emer-gencies treated inMemorial ERIMPACTWe expect that if accom-plished these activitieswill lead to the followingchanges in 7Ð10 years: ¥ Patient co-paymentssupply 20% of clinicoperating costs¥ 25% reduction in # ofuninsured ERvisits/year¥ 300 medical volunteersserving regularly eachyear¥ Clinic is a United WayAgency¥ Clinic endowmentestablished¥ 90% patient satisfactionfor 5 years.¥ 900 patientsserved/year Logic Model DevelopmentProgram Implementation Template Ð Exercise 1 & 2
PAD4320PROGRAMEVALUTION INPUBLICMANAGEMENT
WEEK3-AGENDA•Reflection on what we have coveredsofar.•This Week: Introduction and Understandingof Logic Models•How to CreateaLogicModel
BEFOREWE BEGIN –THIS SLIDE WILLAPPEAR ONALL POWERPOINTPRESENTATIONS•Pleasenotethatsomeofthematerial presentedhereinhasbeenadoptedoradapted fromvarioussources.Sincethisissolelyfor educationalpurposes,directcitationisnot mandatoryfortoday’sclassroom.However,if youplanonusingsomeoftheinformation presented herein, pleasemakesuretoproperly cite.
WHATISPROGRAMEVALUATION?•Evaluationinthiscoursereferstosocial programsandpolicyevaluations.•Whyisevaluationcarriedoutandforwhatreasons.Therearedistinct differencesinthevarioustypesofevaluationsandthecontextoftheevaluations.•Thetextidentified6P’s–programs,policies,products,personnel,performance, andproposals•Peopleevaluatedifferentthingsintheirpersonal,academic,orprofessionallives. Peoplecanevaluatethebookstheyread,thecarstheydrive,ortheemployees theysupervise.Inyouracademiclife,youalsoevaluateprofessorseachsemester. Thewhy,how,andforwhomshouldbeconsideredineachevaluation.Thereare differencesintheimportant detailsforeachtypeofevaluation.
WHATHAVEWELEARNEDSOFAR?Programevaluation consistsof asetof activitiesdeveloped tohelp correct,support, andextendthe waythat people,individually, collectively,or naturallymakesenseofpoliciesand programs implementedtomeethuman needs.•Wedevelopedanunderstandingofthefoundationforprogram evaluation.•Fourpurposes of evaluation(p. 13)–evaluatorsneedto clearlydistinguishthese fourpurposes.•Assessmentofmerit andworth•Programandorganizationalimprovement•Oversightandcompliance•Knowledgedevelopment•Four inquirymodes forevaluation practice(p.15)•The pathfromhuman needstosocialbetterment (p. 25)5
WHYISPROGRAMEVALUATIONIMPORTANT?Weexamined differenttypes ofprogramevaluation, andthedifferentreasonstoevaluateprograms.Wealso reviewed ethics and values in program evaluation, and we discussed appropriate methods for program evaluation.Evaluationsexists toimproveprograms andpolicies•SocialBetterment•MeetingHumanNeeds•LegitimacyandCredibility•ImproveSocialConditions•OversightandCompliance•Assessing the extent to which policies and programs follow the directives of statutes, regulations, and othermandates.•SeekingBetterWays•Improving,overseeing,selectingorunderstandingpoliciesandprograms•Supportingdecisions,deliberations,andactions
WHATAREPROGRAMLOGIC MODELS?Logicmodels areusefulforvisuallysummarizingthe structureofa program.Programlogic models arewidelyusedtodeveloptoshow the intended causallinkagesinaprogram.There aremanydifferent stylesoflogicmodels, buttheyidentifythemajor setofactivitiesin theprogram,their intended outputs,andoutcomes.•Logic model servesas a roadmap foraprogramtoproduceits intendedoutcomes.•Programlogic modelsalsohelp evaluatorsbetterunderstandthe rolesthat differenttypesofstakeholdersplaywithintheprogram.•The logic model isabeneficialevaluation toolthat facilitates improvementinprogramplanning, implementation&evaluation.7
WHATISA LOGICMODEL?•Alogicmodel:•provides a way of visually representing the essentials of a program, policy, or even a whole organization, as an open system•categorizeskeyactivities andoutlines the intended flow of these activities, linkingoutputstooutcomes•depicts networksof constructs (ideas/concepts) that can indicate intendedcause-and-effectlinkages8
DESCRIBETHEPROGRAMVIALOGICMODELSANDPROGRAMTHEORYA program logic model is a picture of how your program works –the theory and assumptions underlying the program.This model provides a road mapofyourprogram,highlightinghow itisexpectedtowork,whatactivities needtocomebeforeothers,andhow desired outcomesareachieved(Kellogg Foundation,p.35).Using evaluation and the logic model results in effective programming and offers greater learning opportunities, betterdocumentationofoutcomes,and shared knowledge about what works andwhy.9
CATEGORIESOF PROGRAMLOGICMODELS10•Programlogicmodelsareoftendevelopedwithfivecategories,illustratingmeanssuchas:1.InputsandResources2.Activities3.Outputs4.Outcomes5.Impacts.•In other words, inputs/resources and activities of a program are planned and implemented in order to produce its desirableoutputs, outcomes,andimpacts.
DEVELOPINGTHEEVALUATIONPLAN•Programcontext•Whatistheproblemorneedfortheprogram?•Whatarethegoalsand objectivesofyourprogram?•Who isinvolved?•Whatactivitieswilltakeplace?•Howwillyoumeasureprogressandimpact?
Createa‘Logic’ModelIllustrate the relationship among your program/project elements:Inputs: Resources necessaryforprogram implementationActivities: Interventions thatwillbeimplemented toachieveoutcomesOutputs: Direct products obtained as a result of programactivities
CREATEA‘LOGIC’MODELOutcomes:Theimpacts,changes,orresultsoftheprogramactivitiesandoutputs; linktoyourobjectivesandyourgoalsIntermediate-term:behaviorspracticespoliciesproceduresLong-term:environmentalsocialconditionseconomicconditionspoliticalconditionsShort-term:knowledgeskillsattitudesmotivationawareness
ESSENTIALSOFALOGICMODELInputsComponentsOutputsOutcomes(short,medium,longterm)Required resources, forexample, money people equipment technology facilitiesMajorclusters of activities, for example, intakeskilldevelopmentjobplacementTypically, units of work done or activities completed, for example,Number of clients served NumberofjobplacementsIntendedoutcomesrelatedtothe design of the program, policy, or organization, for example, Numberofpeopleemployed
1 5