Using Success Markers for
                    Programming In Extension Education

Success markers, and the process of creating them, offer a new way to look at planning, monitoring and evaluating programming efforts.  This process helps to carefully identify all those who may need to change to accomplish program goals and it identifies steps in which incremental successes can be continuously tracked.  Human behaviors that have been hard to measure are becoming more concrete when success markers are identified. 

  Initial reactions from Extension staff working with success markers are excitement and enthusiasm.  The process of writing a vision statement, identifying “whos”, writing outcome challenges for each “who” and then creating three levels of success markers seems to be an intuitive process for many.  It is easy to involve partners, clients, and others in the process and thus increase ownership in the anticipated outcomes of programs.  Those who have been frustrated at seeing progress when working with hard-to-reach, less motivated clients now have a method to help them track small, incremental steps along the path to success.  Several teams and groups within Extension have also used the process to write team goals, respond to strategic planning reports, and work with stakeholders.

         

S. Kay Rockwell Professor & Extension Program   Evaluation Specialist University of Nebraska-Lincoln Email: krockwell@unl.edu	LaDeane R. Jha Extension Educator Southeast Research & Extension Center University of Nebraska-Lincoln Email: ljha1@unl.edu
Susan N. Williams Extension Educator Southeast Research & Extension Center University of Nebraska-Lincoln Email: swilliams1@unl.edu	Carol E. Thayer Professor & Home Economics Extension Specialist South Central Research & Extension Center University of Nebraska-Lincoln Email: cthayer1@unl.edu

 Presented at the 2000 Annual Meeting of the American Evaluation Association, November 2, 2000, Honolulu, Hawaii.

 

          Using Success Markers for
                            Programming In Extension Education

A traditional approach to planning and evaluating Extension programming is to identify goals and objectives a program is expected to accomplish and then specify the indicators that will be used to measure outcomes.  Program success is then judged by how well the goals and objectives have been met by measuring change in the indicators.  In many programs, this process serves a system adequately because evaluation processes can be used which adequately measure the desired change within a given time frame. 

Frequently, however, these same evaluation methods may not adequately reflect program outcomes.  This is particularly true in many complex Extension programs in family life and youth areas.   These programs often have a transformational expectation and multiple partners may be required to accomplish desired outcomes.  In these areas, program goals and objectives might represent an ultimate expected outcome that few individuals reach rapidly, if at all.  These “far reaching goals” can include unrealistic expectations in a specified programming cycle because it may take years to actually accomplish the long-term, desired outcome.  In these situations, both the programmer and the clientele are set up for failure because the incremental gains may not be reflected for the given indicator.  In addition, there may be outside influences such as, governing policies or behaviors of other individuals, that need to change before an individual can successfully attain their goals.  And, with multiple partners, it is also difficult to ascertain the contributions that are a direct result of the Extension programming.

In one state, success markers, and the process of creating them, offer new way to look at planning, monitoring and evaluating programming efforts.  This process helps to carefully identify all those who may need to change to accomplish program goals and it identifies steps in which incremental successes can be continuously tracked.  Human behaviors that have been hard to measure are becoming more concrete when success markers are identified. 

The purpose of this paper is to describe success markers and how they are being tested in two complex programs in family life and youth programming. The family life program focuses on building strong families and family friendly communities.  The youth programming  focuses on mentoring,  juvenile diversion , and character building.  In both of these programming areas, success markers are also being used to build strong interdisciplinary teams and coalitions, identify the outcomes associated with the work of the teams and coalitions, and track their effectiveness. 

The Theory of Success Markers

Success markers, first introduced by Kibel (1999), are the actions/beliefs which indicate successful accomplishment of an outcome. They are a natural part of traditional strategic planning; they are the specific behavior changes that can result from program participation.  They are written in three ways for each program outcome: what one expects to see, would like to see, and would love to see (Esterling & Johnson, 1999).  The success marker process helps clarify program intentions and possibilities for all the program partners who will influence program outcomes; provide a structure in which program beneficiaries can participate in planning and self assessment; assist with process evaluation to correct or improve program performance; and provide a design for tracking and graphically portraying accomplishments.

Developing success markers for extension programming is a four-step process.  The steps are (a) creating the program goal/vision, (b) identifying whos , (c) writing expected outcomes, and (d) listing the success markers (figure 1). 

 

Whos

        Outcome 1                        Outcome 2                                      Etc. 

            SM_expect   SM_like   SM_love      SM_expect   SM_like   SM_love     SM_expect   SM_like   SM_love

Figure 1. Four-step process for identifying success markers.

Step 1 – Program goal/vision.  The goal is the ultimate vision for the program.  It reflects the optimum social, economic, or environmental condition for a given environment and is typically written in present tense.  All subsequent programming, (i.e., identifying specific objectives and outcomes, designing programmatic strategies, evaluating the activities) must be consistent with, and in line with, the goal/vision statement.  Examples of goal/vision statements on which one could build success markers are:

The goal/vision for the “Building Nebraska Families” program is to assist participants acquire skills to help themselves successfully maintain their families as they assume work responsibilities.

The goal/vision for the “Adventures in Mentoring Youth” program is to help youth improve social, educational, and pre-employment skills to ensure success in work, school and personal life by providing a mentoring relationship with an adult.

Step 2 –Whos .  In inter-disciplinary and inter-organizational programming, different agencies and organizations usually have distinct missions with independent goals that may influence program success.   While the different programming partners will take on specific roles, they must work together as they network and organize, plan and develop, and promote and implement a program (Rockwell & Bennett, 2000).  Kibel (2000) and Jha, et al. (2000) identify five categories for whos (who needs to change for the program to be successful).   These whos are not mutually exclusive and include: primary beneficiaries, partners, catalysts and overseers, change agents, and systems gatekeepers.  When writing success markers, one needs to develop separate outcomes and success markers for each “who” to know if (a) the programs are working or if adjustments need to be made, (b) the resources are being used efficiently to achieve the desired outcomes, and (c) the program is having, or is likely to have, a positive impact on the targeted social, economic, and environmental condition.

Primary beneficiaries are those who advance their skills or change their behavior due to the program services.  They participate in the program activities in some manner.  Usually, the primary beneficiaries are thought of as the end users of the program activities and may be an individual or a group of people within a community or region.

Partners include other agencies or organizations that offer interrelated services to the same primary beneficiaries.  Or, they may be defined as the different people on inter-organizational, inter-disciplinary teams who contribute to the given programming process.  In Extension programming, partners can also include the people who convert research findings into practical, applied programs for a target audience.  Specifically partners in Extension include, but are not limited to such people as educators, specialists, program assistants, and volunteers. Other partners may come from other educational units, governmental agencies, human service agencies, etc.

Change Agents are the person or persons who implement or develop best practices for a program.  They include, but are not limited to, those who design or teach research-based programs to be taught in response to identified needs.  

Catalysts & overseers are the individuals within organizations/agencies who are vital in providing program guidance and resources (i.e., time, money, and staff).  Many times they are also the people to whom program staff are accountable.  They include, but are not limited to, strategic planners, council board members, Deans, Department heads, and District Directors.

Systems gatekeepers are individuals and entities which are influential in the environment within which the program operates and that have the power to make it easier or more difficult for the program to be successful (Kibel, 2000 & Jha, et al., 2000).  They include, but are not limited to, advisory boards, funders, some administrators, and governmental agencies.

Step 3 – Writing outcome challenges.   Outcomes challenges should briefly describe intended impacts on the key program partners.  Outcome challenges typically have three distinct parts.  First the program partner is identified.  Second, a clause is added which describes the successful attainment of an identified desired change.  Third, the statement is completed with a behavioral intention that represents a significant attainment for the person targeted.

An example outcome challenge for a youth participating in an “Adventures in Mentoring” (AIM) program might be:

            Part I – Program partner: We intend to see youth in AIM who.......

                                                Part II – Clause describing change: We expect to see youth in AIM who take the necessary steps to improve their social skills, academic performance and pre-employment skills,.....
Part III – Behavioral intention: We expect to see youth in AIM who take the necessary steps to improve their social skills, academic performance and pre-employment skills to be successful in work, school and their personal life (Williams, 2000).

The number of outcome challenges for each program partner are dictated by the program vision – if the vision is quite narrow, there may be a very limited number of outcome challenges; if the program vision is quite broad, there may be several outcome challenges for a given program partner.  The process is continued until outcome challenges are written for each of the program partners previously identified.

Step 4 – Listing success markers.  Success markers are actions/behaviors which indicate successful accomplishment of the outcome challenge.  They are divided into categories according to those one would expect to see, like to see, and love to see.  While describing these categories over simplifies reality, it provides user friendly constructs for sequencing behaviors one expects from the programming.  Therefore, the behaviors in the three categories are not necessarily mutually exclusive, nor will every identified behavior be one that every participant will reach.

Expect to see success markers identify behaviors that must occur before there can be any successful program outcomes.  They are the very basic behavior markers.  Usually expect to see success markers will focus on participation activities because the program partners need to be engaged in the programming activity before they can begin to react to the subject matter and change their behavior patterns to be consistent with knowledge, attitude, skills, or aspirations promoted by the program.

Like to see success markers identify behaviors that come after, or start to emerge from, the ‘expect to see successes’.  They are the more immediate behaviors, or new practices, that program partners adopt as they start to apply new knowledge and skills, or alter their attitudes or aspirations in their work and life situations. They typically would relate to a change that has been sustained for at least six months.  Like to see success markers may be the highest level that many program participants ever attain.  While accomplishment of the like to see success markers indicates success, project leaders need to define the specific standards they will accept for judging if the program indeed accomplishes the desired outcomes.

Love to see success markers are longer-term or higher-order behavior changes that come after the ‘like to see successes’.   They are the new practices that program partners adopt as they use their new skills to effect their own life, or the environment in which they live, work, or play.  They are sustained over extended periods of time and become indicators of sustained transformational change.  Some program participants may never achieve love to see markers.

Case Study #1 – Building Nebraska Families is a state-wide initiative to assist participants acquire skills to help them successfully maintain their families as they assume work responsibilities.  

Vision – “Building Nebraska Families” program participants use newly acquired skills to successfully maintain their families as they assume work responsibilities.

Whos

Employment First participants/families
Building Nebraska Families educators
Building Nebraska Families program coordinator
Cooperative Extension administrators/faculty and staff
Health and Human Services Employment First case managers
Health and Human Services supervisors
Health and Human Services employment first administrators
Community members

Outcome Challenges