10/27/00

Marilyn Grantham and Jennifer Obst
University of Minnesota Extension Service

In keeping with the theme of this conference, we were interested in learning whether their experience with Minnesota Impacts! had an impact on the researchers and Extension staff who have entered impact statements in the database for their research projects or educational programs. The 140+ submissions thus far have been about equally divided between research projects and educational programs, so we drew a stratified random sample of 10 persons from each group and conducted a total of 20 telephone interviews.

1. We were interested in finding out

2. whether defining and reporting the impact of their research changed the thinking of researchers about
3. the importance of the outcomes and impacts of their work and
4. the importance of reporting research outcomes and impacts to the public and whether defining and reporting the impact of their educational programs changed
5. the thinking of Extension staff about
6. the role of program planning and evaluation in determining outcomes and impacts and
7. the importance of reporting program outcomes and impacts.

2. whether members of both groups had made use of the impact information prepared for Minnesota Impacts! for other reporting/accountability purposes?
3. And any suggestions that interviewees had for improving Minnesota Impacts! as a public accountability and reporting database.

Researchers: Eight out of ten said they prepared Minnesota Impacts! (MI) statements because they were told to do so by a dean or department head (only two cited their own initiative as the primary reason for preparing an impact statement).

Those who did take the initiative to report thought

• MI was a good thing.
• Their project was important.
• They wanted attention/publicity for their work.
• They should be responsive to the public.
• Describing impact wasn’t difficult to do.

They also reported changing their thinking about

• The importance of thinking about outcomes/impact from the beginning of their work
• The importance of communicating with the public.

They had used MI information for

• Reports to funding sources (federal, state, grantors)
• Background for faculty performance/research progress reports
• Presentations at public meetings

One had linked his MI report to his departmental website.

One had recruited an international grad student who saw his report on-line in MI.

Researchers’ concerns:

• Releasing research information via the web before the work was completed/formally published elsewhere.
• Fear that people opposed to research will use web-based information negatively.

Researchers’ suggestions for MI improvements:

• More internal/external promotion of website.
• Gather/report data on number of MI website "hits".
• Add more links to other U websites.
• Provide specific funding details (list sources/amounts).
• Provide an easy/timely way to update information.

Extension staff:

Why they prepared impact statements:

• Most of the 10 took the initiative to submit program impacts—they thought their program was deserving of public recognition.
• Most had heard about MI in work teams who talked about their best work/what should be submitted.

Changes in thinking about program planning, evaluation, accountability/reporting:

• Experience made them think more about the need to plan evaluations in advance—think about how to measure impact and collect data.
• A few said the experience spurred evaluation efforts.
• Experience raised awareness of importance of reporting all three kinds of impact—economic, environmental, quality of life.
• Database provides another vehicle for public reporting—accessible on demand.
• Documenting impacts helps in seeking additional, external funding.

Other uses found for MI information:

• Faculty reports
• Performance documentation
• Resume
• Reporting to local government officials
• Demo at professional meeting
• Exhibit
• Information shared with another faculty member for inclusion in a book
• Links to other websites

Suggested improvements:

• Expand database—more representative sample of programs
• Promote to internal/external audiences
• Explain importance of "value-added" reporting to Extension staff
• Tie reporting to state and district level program planning

Recommendations made by an MI "Think Tank" in May 2000:

• Continue MI as a major, external research/Extension accountability/reporting effort.
• Re-focus content on major research projects and priority Extension programs.
• Have AES administration and Extension Capacity Area and District Leadership Teams identify best work for entry into system.
• Show connections between research and Extension programs (links between entries).
• Continue to provide professional writing/reporting assistance and impact review.
• Review/update entries already in system.
• Continue to promote MI—especially to policymakers/funders.
• Promote linkages to other websites, e.g., county home pages and other UM sites.

Notable quotes from Extension staff:

"It made me think about not so much program planning but about the need for evaluation and gathering the [impact] information … and to think about it up front … not at the end of the program … to make sure you’re getting that information together."

"After doing the first one … it really made me think about what we’re doing and it’s really made me change what I’m doing … in evaluating not just this program but everything that’s important. Going through the process [of defining/reporting program impact] helped me learn how this is supposed to be done."