Summary Report of Actions and Accomplishments Facilitated by the
Tutor/Mentor Connection (T/MC), Chicago
to Increase the Number, Effectiveness and Survival of ALL Tutor/Mentor Programs

Based on Data Collected Through a Internet Pilot Project (OHATS)
September 2000 through August 2001

Produced by Steve Roussos and Daniel Bassill
September 2001

Please direct questions and comments regarding this report to Dan Bassill (tutormentor2@earthlink.net, 312-492-9614) or Steve Roussos (steve@akouo.org; 209-723-4398)

INTRODUCTION

Since 1993, Chicago’s Tutor/Mentor Connection (T/MC) has been creating a network of local, state and national partners dedicated to the development and expansion of tutoring, mentoring and school-to-work programs in Chicago and around the country.  Toward this end, T/MC uses research, training, marketing, and network development to create a supportive environment for organizations and support networks to develop and grow.  T/MC’s role as a support and infrastructure developer is unique and offers a model of a support system for community organizations serving youth.  T/MC was one of 50 groups nationwide to serve as a Teaching Example at the 1997 Presidents' Summit for America's Future, held in Philadelphia.  T/MC does not focus support on one specific program within one area, or a group of programs working on any one outcome.  Rather, it reaches out to partners who have expertise and ownership in particular specialties (e.g., media campaigns, volunteer training) and works to help them adopt a “distribution” vision that would lead ALL tutor/mentor programs to become more holistic in the services they offer youth, and more long-term in their ability to constantly upgrade services to meet the needs of youth as they change from children, to adolescents to teens to job-seekers.

T/MC's role as a support and infrastructure developer is not just to those who operate tutor/mentor programs, it is also to those who are, or can be, providers of resources (e.g., volunteers, dollars, publicity, training).  T/MC seeks to build leadership in every sector of business, commerce, civic and social life who will become responsible for the on-going distribution of volunteers, dollars, training, technology, and school-to-career opportunities into every tutor/mentor program on the T/MC database (currently over 13,000 active contacts).  This goal of providing resources that are essential to help programs improve is a revolutionary concept and one that can dramatically impact the distribution and effectiveness of an entire city of tutor/mentor programs.

T/MC’s approach and methods grew out of its founders’ 20 plus years of experience as volunteers in and eventually directors of youth programs.  These volunteers also had extensive business backgrounds in marketing and advertising, and have consistently modeled the T/MC after the most innovative business communications and information-sharing models, using concepts such as Total Quality Management as the vision behind a constantly learning and improving network of tutor/mentor communities.  The collective intelligence for building and supporting tutor/mentor programs exists within the current T/MC staff and its broader network of known and yet to be met peers.  It can be found in archives of documents and reports and seen in the successful start and growth of tutor/mentor programs and support systems for programs, such as the Lend A Hand grant program for best practices among tutor/mentor programs growing at the Chicago Bar Association/Chicago Bar Foundation since 1994.  Unfortunately, as in too many community change efforts, this intelligence and wisdom stays, and often dies, with individuals and organizations, and is not well distributed and used to improve the work.

This report is the result of a new project intended to prevent these problems, Project OHATS (Organizational History and Accomplishments Tracking System).  As of July 2000, collaborating with researchers who specialize on the study and improvement of community-wide change initiatives, T/MC began developing an Internet-based application to:

1. Enable staff and any stakeholder (such as program leaders, volunteers and clients, business and civic leaders, philanthropic organizations, social researchers, and policy makers) to document what they do to achieve the mission and objectives of the organization, and

2. Enable clearer and easier exchange of documented ideas, lessons and best practices within and outside the organization and its national network.

We describe the results from Tutor/Mentor Connection’s (T/MC) first year of tracking actions and accomplishments using OHATS.  In this first year of development, T/MC served as the primary beta-tester of OHATS.  Lessons learned during this time are also presented here and are being used to expand the use of OHATS to Cabrini Connections (T/MC’s parent organization), Kids’ Connection (a site-based program of Cabrini Connections), and other tutor/mentor staff and stakeholders.  A more complete explanation and illustration of OHATS can be found at http://www.tutormentorexchange.net/Intro_OHATS.htm.

METHODS

Measures

            The OHATS web data entry form prompts reporters to describe specific kinds of information for the following purposes:

1. To identify specific T/MC staff and stakeholders that become reporters

2. To identify information about the event or action so that someone, even if unfamiliar with T/MC, can understand what happened and why it was important.

3. To identify common elements across the reported events or actions so that they can be grouped for quantitative analyses and searched for specific information.

The data entry form in Appendix 1 shows the information collected.

            The common elements mentioned in number 3 allow one to search and analyze the OHATS database for events and actions by:

• Date, time period and intended duration

• New or different versus pre-existing events and actions

• People and organizations involved in bring about the events or actions

• T/MC strategy areas

• Internal (e.g., hiring staff, planning activities) and external organizational (e.g., networking, and providing services) events and actions

• Ways to share the event (e.g, by email, newsletter or the press)

OHATS measures are based on research and experience with nonprofit organizations seeking to improve long-term, community-wide outcomes in public health and community development.  A core component of OHATS is based on research of a documentation system created in the early 1990s at the University of Kansas for the U.S. Centers for Disease Control community initiatives to prevent cardiovascular disease.  That system enabled the study of how multiple organizations collaborate in a given community to create change in order to reduce the prevalence of illness and disease many years later.  That research showed that organizational staff and stakeholders could systematically document their contributions to community-wide, longer-term outcomes and also identified important measures and methods for tracking the progress of ongoing collaborative initiatives.

One important measure used in OHATS was adopted from this original research.  This is the measure of “community change.”  A community change is an observable change in the environment that indicates a new or modified program, policy or practice of an organization or system.  An event counts as a community change if it is facilitated by the organization under study (i.e., T/MC), is important to the mission of that organization, and influences or engages people or organizations outside of that organization.  Examples include the start of new tutor/mentor programs, adjusting the hours of tutor/mentor programs and businesses to make it easier for adult volunteers and youth to participate in programs, or adopting time-off policies that encourage employee volunteerism.  Preliminary studies and current research are testing the hypothesis that longer-term, community-level outcomes (e.g., crime rates, high school graduation rates) are likely related to specific attributes of community change.  These attributes include (a) the amount of change (e.g., new or modified community and systems changes within a specific period of time, goal, and/or geographic area), (b) the duration of a change (e.g., ongoing changes may be more influential than one-time events),  (c) the penetration or exposure to relevant populations (e.g., more people exposed to a change may increase the influence of that change; people may be exposed through multiple sectors, and/or settings (e.g., schools, businesses, and faith, and communities), or geographic areas, and (d) the intensity of strategy (e.g., a change that modifies access and removes barriers to services may be more powerful than a change to provide information about services).

T/MC’s use of OHATS is contributing to this research by documenting community changes in the tutor/mentor community that may improve longer-term, population outcomes related to youth development, economic indicators related to educational and employment outcomes attributable to tutoring/mentoring, and broader indicators of civic engagement and social capital.

Data Collection Procedures

            At the time of this report, OHATS data collection consists of one form (shown in Appendix 1).  Using any Internet browser, anyone, at any time can report event or action that are important to the mission of T/MC.

Known Limitations of Interpretations and Conclusions of the Results

Several limitations should be kept in mind for any interpretations and conclusions based on the results of this report.  Some limitations are the result of the learning curve in using a new tool such as OHATS.  The current results are based on reports of only five Cabrini Connections staff and stakeholders, with over 90% of them provided by Dan Bassill, T/MC’s CEO.  Although it is common in documentation systems of this kind, to have the majority of reporting done by the leaders or directors, this greatly under-represents the number of actions, accomplishments and lessons within T/MC and its network during period of this report.  We believe that as OHATS develops and stakeholders are introduced to and learn to use it, the number of reporters and reporting will increase.  Suggestions and steps being taken to increase the number of reporters are discussed later in the report.

Some limitations are the result of what is measured and the definitions and categories that make up certain measures.  First, a given event or action may be categorized differently by different reporters (see the types of categories in the form shown in Appendix 1).  And inconsistent categorization would bias the analyses of the events of this report.  To prevent this, OHATS provides clear and easy-to-understand categories and ways to group events.  If needed, the events shown in this report could also be re-categorized by an independent observer to assess the reliability of categorizing the events for the various analyses in the report.

Another limitation inherent in the measures is that they do not capture the amount of people influenced and resources used for any given event.  For example, the T/MC conferences are attended by more than 500 people and require dozens of meetings and tens of thousands of actual and in-kind dollars to create.  Yet they may only represent four reported events in this report (see Figure 3 on page 7).  Similarly, a media event that reaches 10,000 people is counted the same as an event that reaches 10.  OHATS, at this point, is designed to document events, not to quantify the numbers served in events.  Service participation and media reach numbers are collected separately by T/MC.  Future enhancements in OHATS will link participation and exposure data to documented events in order to more accurately analyze the impact of T/MC.

Finally, the accuracy and validity of data in OHATS depends on the memory, honesty and writing ability of its reporters.  Because T/MC uses OHATS to help it better understand and improve its efforts, purposeful deception in the reports is unlikely.  If needed, sufficient details are collected for each item reported in OHATS to allow an independent observer to check its occurrence and validity.

FINDINGS

Type of Action

Between September 2000 and August 2001, T/MC staff and stakeholders reported over 330 events, actions and accomplishments.  The complete list of actions can be found at http://www.tutormentorexchange.net/TMC_OHATS_page.htm.  Documented activities in OHATS are categorized in one or more of the following categories of action shown in Figure 1:

1.    Community changes: new or modified programs, policies, or practices in the community facilitated and/or created by T/MC to support and enhance the infrastructure needed for the success of tutor/mentor programs.

2.     Service provided: events produced by T/MC to provide information, instruction, or develop skills of people and organizations.  (The first time a service is provided it is coded as a community change; all subsequent events of that service are coded as a service provided.)

3.    Planning products: results, or products, of planning activities within T/MC.  For example, statements of objectives or the development of action plans, the formation of committees, and the hiring new staff.

4.    Resources generated: acquisition of funding through grants, donations, or gifts in kind.

5.    Critical Events: events considered important to the development and implementation of T/MC but which do not fit in the OHATS documentation categories.

6.    Media: coverage of T/MC or its projects in the newspaper, radio or television, or newsletter.

7.    Community actions: actions by T/MC with people and organizations outside of T/MC to bring about a new or modified program, policy, or practice.

8.    Endorsements: indications of external support for the T/MC mission and approach.  These events include examples of other organizations attempting to adopt one or more T/MC strategies for their own community.

Our first year’s experience using OHATS to document T/MC actions has shown that certain types of T/MC events are less accurately documented and are greatly underestimated in these results.  Media events are better documented in reports by PCI, T/MC’s public relations partner.  Resources generated and planning products are better documented in T/MC’s financial records and administrative reports.  Services provided by T/MC also are likely underestimated in these results (e.g., the Annual Recruitment Campaign and Bi-Annual Training Conferences).

T/MC shows a high amount of community action and community change (new or modified programs, practices and policies) within one year.  This is especially true given results of other organizations using the documentation system that OHATS is based upon (for example, some community initiatives do not achieve this amount of community change even within a 2-year period).  This is likely due in part to the fact this one-year of results is based on work that was preceded by and contributed to by actions and capacity developed since T/MC’s initiation in 1993.  Our goal is to be able to do a retrospective documentation that will show the actions that led to these results (and the ability to achieve greater amounts of change).

A significant amount of endorsements for T/MC, its events and methods are also seen in Figure 1.  Endorsements were reports of people outside of T/MC that contacted T/MC directly or spoke of T/MC in a listserv or news article in a way that acknowledged the value of T/MC’s methods and outcomes or that indicated replication of T/MC’s methods.  Many of these endorsements come from outside Chicago and throughout the U.S., and some from other countries.  These results regarding endorsement of T/MC should be followed in later study to better understand how other organizations are modeling and improving on T/MC’s approach.

Community Change

Of all the types of events and actions categorized in OHATS (see Figure 1, Types of Action), community change is of particular importance.  Community change is measured as new or modified programs, practices or policies related to the mission of T/MC.  These are changes in the environment that alter the conditions that shape individual and organizational behaviors.  Studies of comprehensive community initiatives indicate the rate of community change may be related to the rate of population-level health and development outcomes.  Therefore, community change is tracked as an intermediate marker of broader, more distal population outcomes

Nearly 70% of all events documented during this period were community changes.  This not only represent a how amount of change but as the graph in Figure 2 shows, the rate of change stimulated by T/MC over time is also high, and stable.  A flat line between any two points on this graph would mean that low or no amount of community change occurred during that period.  The results in Figure 2 mean that T/MC actions and accomplishments are not sporadic or occurring only during some parts of the year.  This result demonstrates the intentional strategy of the T/MC approach. 

Again, it is important to remember that these results show a limited number of actions and only a very brief slice in time of T/MC’s ability to stimulate community change in Chicago and beyond.  Figure 3 provides an illustration of how some key events influence hundreds of other organizations to create changes within their own programs and communities.  An analysis of T/MC-facilitated community change since its inception would allow us to see the variability and evolution of T/MC’s capacity to catalyze community change.  Future research and development of OHATS should enable the linking of participation and exposure results to each community change.  This would better show T/MC’s impact as a change agent by demonstrating how the process of changing the community to support tutor/mentoring results in meaningful differences in outcomes for programs, youth, adults and the broader community.

Type of Strategy

Five interconnected strategies guide actions and events employed by T/MC: public awareness, research, resource building, technology, training, and regular service and networking events.  Each of these strategies, and thus the events and actions that they represent, are interconnected and simultaneous.

For instance, T/MC research is a process of building awareness of what programs exists and building a connection between programs.  Every new contact documented in OHATS represents a new piece of knowledge in this research.  The T/MC contact database that began with 400 people in 1993 and now has over 13,000 contacts is one sign of this growth of knowledge.  Another example of the interconnection across strategies is how technology (as a strategy) is also a resource, but is so important to the T/MC mission that it is documented separately.  All events that T/MC leads are strategically designed to generate public awareness, resources and training.

These results show that the largest amount of action emphasized public awareness and training.    This reflects the T/MC priority of enlarging the number of adults, resource providers, etc. who get involved with this movement.  Public awareness and training examples include interactions on Internet discussion groups to raise awareness of key issues for tutor/mentoring and outreach and training services to programs and organizations.

Event Duration

Though a given event or action may be powerful enough to produce behavioral or organizational outcomes after a single exposure, experience suggests that this is not the norm.  The longevity of any given event can determine the opportunity for exposure to that event or change.  And, exposure to an event is a prerequisite for producing an outcome.

The vast majority (83%) of changes facilitated by T/MC were intended to be ongoing.  Some examples are the citywide recruitment fair every September and the training conference every May and November, each which have been occurring for several years.  The building of OHATS is an example of an ongoing event that will be enhanced and engage new partnerships over the years.  Another ongoing event is the Lend A Hand program began in collaboration with the Chicago Bar Association/Foundation in 1994.  From a single award of $2,000 given to one T/MC program in 1994 this program is now a growing marketing and fundraising activity led by the Chicago Bar Association.  Since 1995 more than $200,000 has been granted to over 40 different programs (some receiving repeat grants over multiple years).While each of these accomplishments facilitated by T/MC only shows up as a few listings in this year's report, each grew out of multiple, interconnected actions and partners.  A retrospective documentation and analysis would help to better understand the creation of such accomplishments and identify how others can replicate these changes in their community.  The ability of T/MC to stimulate, repeat, and sustain events and actions over time is one of its key strengths, and greatly increases the likelihood of contributing to Chicago-wide change in the availability, distribution and quality of comprehensiveness tutor/mentor programs.

Event Innovation

Equally important to the abilities to create actions that endure time and to sustain events over multiple years are the abilities to adapt to changing conditions and create new innovations.  Reporters were prompted to categorize an event into one of four groups: “new” (not existing in the prior history of T/MC work), “modified” (having occurred before, but modified in some important way), “existing,” or “other” (for uncertain or events with no clear category).  The vast majority of events were either new or modified.

            Examples of new events include the creation of OHATS and the start of new working relationships and collaborations to conduct the GIS mapping work, the volunteer recruitment campaign, and the training conferences.  Modified events included enhancements to the major T/MC events and actions that grow and reach more people each year, such as T/MC training conferences and Lend A Hand program.

SUMMARY

One-Year Documentation Demonstrates Significant Impact in Chicago

These results demonstrate the way T/MC helps improve the conditions in Chicago to help tutor/mentor programs and their support networks.  Given a short period, T/MC documented a large number of community changes (over 109 new or modified programs, practices and policies) that support and create the conditions for more and better tutor/mentoring.  Actions and accomplishments were well distributed across T/MC’s strategic action areas and were intended to be ongoing, not short-term.

T/MC is not the result of some grand and sustained funding initiative by the city and major foundations.  It is a vision that started with a few volunteers. They have had to build the T/MC from scratch, with no models to follow and with the help of volunteers and a patchwork of dollars.   The accomplishments of T/MC and the community changes documented here occur in the third largest city in America, where organization and mobilization of people and resources is often hindered by anonymity, mistrust and competition.  Getting more than 100 agencies to come together for a single goal is a huge accomplishment.  Getting them to come together three or four times a year for a sequence of goals is even more challenging.  These results begin to capture T/MC’s methods and strategies so that they can be enhanced and replicated.

T/MC Shows Massive Return On Investment

T/MC accomplished these results at a total cost of under $250,000 in 2000.  Many similar initiatives pay more than that just to organize one training conference of T/MC’s quality and reach (a single event documented in this report).  The results show how T/MC magnifies the impact of financial investments through collaboration and networking to produce and sustain community change.  This is not to say the T/MC is over funded.  It struggles to provide the impact it has documented here and to increase the pace of progress in each of its action areas as it finds dollars, partners and volunteers.  We invite you to consider taking on one of those roles.

Web Documentation System Shows Promise

These results also demonstrate the potential for the on-going use of a web documentation system.  Traditionally, documentation and exchange of history and lessons occur through personal conversations and journals, and organizational calendars, newsletters and conferences.  Increasingly, such information is posted on static websites and email discussion lists.  But these methods provide limited documentation and exchange of such intelligence.  OHATS simplifies the collection and organization of data using the Internet and leads to an increase in (1) the number of organization staff and stakeholders who document important actions and lessons, (2) the number and type of lessons and best practices reported, and (3) the rate of exchange of such intelligence.  Through systematic documentation and organization of its actions, accomplishments and critical events related to its progress, T/MC will:

·       Identify lessons and best practices that prevent problems and improve results,

·       Create a historical archive of its development and accomplishments that new members and those outside the organization can learn from,

·       Systematically track and demonstrate the influence of individual and cumulative contributions toward the achievement of objectives, and

·       Serve as a model for this type of documentation and share it with organizations in other sectors.

OHATS addresses a need present in every organization, but more acute in networks and collaborations such as T/MC.  It allows any organizational stakeholder to document and see what he/she does to achieve the organization’s mission, while allowing all recorders to see and demonstrate their cumulative actions and impact.  The application of OHATS is a learning process.  The challenge of any organization using a web or other documentation system is building a discipline of stakeholders to document action.  In the T/MC a few stakeholders are staff, most are volunteers that are part of other organizations.  This first year’s report demonstrates this learning curve.  Support is needed to conduct a retrospective documentation of T/MC actions and accomplishments, and to make enhancements in OHATS web features (such as enhanced security and analyses components).

Much Work and Many Opportunities for Support Remain

The positive nature of these results shows progress, as well as the immense work ahead.  The documented rate of accomplishment and community change must not only be sustained, but also increased to make a meaningful difference in youth and community outcomes.  Improvement and expansion of T/MC’s work by other communities requires a constant infusion of dollars, partners and volunteers.  We hope to engage you and you to engage your peers and network in this effort.  Opportunities to support this work, this report, and continued reports are found at www.tutormentorexchange.net

Appendix

Appendix 1. OHAT Data Entry Form