Learning about Organizational Learning - Creating a Learning History
George Roth
Art Kleiner
MIT Center for Organizational Learning
30 Memorial Drive
Cambridge, Ma 02139
tel. 617/253-8407 fax. 617/252-1998
April 7, 1995
Table of Contents
Managers today are drowning in information about organizational learning.
Nearly one hundred books and journal articles on the subject exist. Almost
sixty per cent of them have been published in the last three years.[1] Many organizations have experimented with
pilot projects, built around various theoretical or metaphorical ideas about
what "organizational learning" should be. A great number of these pilot
projects have produced noteworthy results -- both in "hard" performance
measures, and in the "soft" arena of increased commitment, innovation, and
morale. And yet the key questions about organizational learning remain as
elusive as ever: What types of learning efforts work effectively? Why do they
work, and why do they not? Everyone who works on building learning
organizations seems to run up against the challenge, sooner or later, of
"proving" the value of what they have done. Without some form of assessment,
it is difficult for organizations to learn from their own experience, to
transfer their learning, and to replicate the results so the benefits are not
lost when a pilot project comes to an end.
But "assessment" strikes fear in people's hearts. For many people, the word
itself draws forth a strong, gut-level memory of being evaluated, and measured
-- perhaps in the painful, learning-inhibited environment of a regimented
classroom. As writer Sue Miller Hurst has pointed out, most of us, beginning
in childhood, have an intrinsic ability to judge our own progress. This is a
natural aspect of our ability to learn. But schools and workplaces subjugate
that natural assessment to the judgment and ranking of teachers, supervisors,
and other "experts," whose appraisals can determine our tracking, promotion,
opportunity, wealth, status, and ultimately even self-esteem. Business
assessment systems such as Management by Objectives (MBO) tend to
institutionalize a destructive process where bosses propagate the same measures
and assessments of which they are also victims. Thus, assessment is not
emotionally neutral territory. Inherent fear and resentment diminishes the
quality of any assessment effort -- particularly those involved with assessing
organizational learning.
At the MIT Center for Organizational Learning, we've struggled for several
years to find a reasonable way to assess learning efforts. Our corporate
affiliates, as partners in learning projects, need to get some idea of the
return on their investments, and our own researchers need a better
understanding of their work. One year ago, a group of colleagues at the
Learning Center set out to develop a better form for making assessments. We
rejected the idea of traditional quantitative assessment and measurement
techniques, because learning cannot be quantified. Even analytic tools, such
as the tools of system dynamics, lead inevitably into unquantifiable realms
like the explication of mental models. We also rejected the maxim, as proposed
by Harvard professor David Garvin, that "if you can't measure it, you can't
manage it."[2] Many systems that can't be
measured must be managed.
We started, instead, by going back to the source: the people who initiate and
implement systems work, learning laboratories, or other pilot projects in a
large organization. We tried to capture and convey the experiences and
understandings of a group of people who have expanded their own capabilities.
The resulting document may become a new and much-needed form of institutional
memory. We call it a "learning history."
Every part of the learning history process is an intervention that should
increase learning: the interviews, the editing work, the circulation of drafts,
and the follow-up. The audience includes original participants in a learning
project, researchers and consultants who advised them, other people in the
organization, and ultimately anyone interested in organizational learning.
The learning history includes not just reports of action and results, but the
underlying assumptions and reactions of a variety of people (including people
who did not support the learning effort) in the organization involved in the
efforts, other people in that organization, and the action researchers
themselves. The final report might take many forms, but its story is always
larger than any one person's experience. We believe that the power of the
learning history depends on its ability to convey multiple perspectives on
controversial events. Every participant, reading the learning history, should
feel that their own point of view was treated fairly, and that they understand
how other people came to their perspective. Ideally, every reader will undergo
a little bit of a learning experience just from reading the learning history.
Currently, there are over a half dozen learning history projects in progress at
the Center, nearly all based in the Center's member companies. Only one or two
projects have been completed, and they have sparked extremely strong interest.
This article, the first to be published about learning history theories and
techniques, represents an initial effort to articulate an emerging body of
theory, and to report the results, so far, of the first learning history
work.
The learning history form draws upon theory and techniques from ethnography,
journalism, action research, oral history and theater. From ethnography come
the science and art of cultural investigation: systematic approaches for
participant observation, interviewing, and archival research, used in
understanding the day-to-day routines which make up people's lives. From
journalism come the skills of getting to the heart of a story quickly, and
presenting it in a way that draws people in; a good learning history is as
accessible and compelling as a newspaper story, but hopefully more accurate.
(We also draw heavily on the journalistic skill of fact-checking.)
Action research offers effective models and methods for exploring situations
where the researchers are actively involved in changing the system they are
helping. The typical action research intervention[3] follows a cycle in which managers observe
themselves acting and communicating, learn to recognize the assumptions
inherent in their actions, build an understanding of the norms and values which
drive those assumptions, and then plan new, more effective actions. Similarly,
with learning histories, we foresee a cycle in which teams will use the
insights of the learning history to guide new behavior -- which will then
become grist for the next rendition or edition of the history. The learning
history approach develops the capacities of learners to reflect and assess
their own efforts, and utilizes the data from that reflection and assessment as
the basis for documents that are more broadly disseminated.
Oral histories are often narratives which come from recorded in-depth
interviews. The tradition of oral historians provides a data collection method
for rich, natural descriptions of complex events, employing the voice of a
narrator who took part in the events.[4]
These techniques are vital for readers to understand the way that participants
attributed meaning to their experience. Finally, the learning history involves
deliberate presentations of emotionally charged narratives, which make the
skills of theatrical artifice essential.
Insiders versus outsiders: Traditional ethnographic researchers
define themselves as outsiders: strangers, developing an understanding of how
those inside the cultural system make sense of their world. There is an
underlying assumption that the researcher, and the reader, is separate from the
culture being studied.
Learning histories take an opposite approach. Members of the audience are also
participants; the participants tell the tale. The "subjects" under study are
also the readers of the story, keenly aware that their story is revealed here.
The story must balance insider's understanding and authenticity with an
outsider's perspective and context.
Perhaps for this reason, the most successful learning history projects to date
seem to involve teams of insiders -- managers assigned to produce and
facilitate the learning history -- working closely with outsiders -- trained
writers and researchers hired on a contracting basis. This collaboration seems
to be necessary to avoid the logistic perils involved in a deeply reflective
organizational activity. Managers inevitably find it difficult to reflect
"objectively" upon events, because they have ongoing relationships with the
participants and corporate culture. Outside researchers, particularly those
from an "action research" orientation, feel pressure to take on routine
"note-taking" work, which drains away their time for reflection and writing.
One key goal of our learning history work is developing the organizational
managers' own abilities to reflect, articulate, and understand complex issues.
Most learning projects teach people tools to conceptualize and understand
highly complex, dynamic interdependencies. A good learning history effort
might offer skills for understanding their own style of making assessments and
judgments, through the discipline of communicating those assessments to others.
These skills take time, experience and practice, perhaps several years, to
master.
Expert assessments versus organizational learning: The
inside/outside team structure also helps avoid another dilemma inherent in
traditional program evaluation approaches: the separation of experts'
assessments from the organization's own learning. In most assessments, experts
offer their judgment, and the company managers receive it, without gaining any
ability to reflect on and assess their own efforts. Thus, the stance of
a learning historian borrows from the concept of the "jointly told tale,"[5] a device used by a growing number of
ethnographers. In a jointly told tale, the story is "told" not by the external
anthropologist, nor by the "naive" native being studied, but by both
together.
The "subjective" stance, in itself, would not be sufficient, precisely because
people in a learning effort undergo a transformation. As they develop
capabilities together, gain insights, and shift their shared mental models,
they change their assumptions about work and interrelationships. This
collective shift reorients them so that they see history differently. They
find it difficult to communicate their learning to others who still hold the
old frame of reference. An outside expert can help bridge this gap by adding
comments, such as: "This situation is typical of many pilot projects," or
questions, such as: "How could the pilot team, given their enthusiasm, have
prevented the rest of the organization from seeing them as some sort of
cult?"
Results versus experience and skills: How do we help
organizations learn? Not by telling them what to do, but by
inspiring them with what others have been able to accomplish. People
are motivated to learn because they too want to change -- not for change's own
sake, but to accomplish better results. That's the positive side of
incorporating results into a learning history.
The negative side has to do with bottom-line pressure. Companies today don't
have a lot of slack resources or extra cash. Thus, in every learning effort,
managers feel pressured to justify the expense and time of the effort, by
proving it led to results. But a viable learning effort may not produce
tangible results for several years; its most important results may include new
ways of thinking and behaving that appear dysfunctional at first to the rest of
the organization. (More than one leader of a successful learning effort has
been reprimanded for being "out of control.") In today's company environment
of downsizing and reengineering, with its tremendous pressures to behave
compliantly, this pressure for results undermines the essence of what a
learning organization effort tries to achieve.
Yet incorporating results into the history is vital. How else can we think
competently about the value of a learning effort? We might trace examples where
a company took dramatically different actions because of its learning
organization efforts. But it is difficult to construct rigorous data to show
that an isolated example is typical. Alternatively, we might merely assess
skills and experience. A learning historian might be satisfied, for instance,
with saying, "The team now communicates much more effectively, and people can
understand complex systems." But that will be unpersuasive -- indeed, almost
meaningless -- to outsiders.
We believe it is possible to link learning organization efforts to results; but
not through analytic measurement. It can only be done by capturing and
constructing stories, gathering data from a wide enough group of people so that
judgments can be made about whether or not a story is typical. Assessment, in
this context, means listening to what people have to say, asking critical
questions, and engaging people in their own inquiries: "How do we know we
achieved something of value here? How much of that new innovation, honestly,
do we link to the learning effort?" We have found in our early work that
different people bring different perceptions of a "notable result" and its
causes; triangulating those perceptions together leads to a common
understanding with intrinsic validity.
For instance, in one corporation's learning history, a team developed a new
form of manufacturing prototype. On the surface, this achievement was a matter
of pure engineering; but it would not have been possible without the learning
effort. Some team members learned to communicate effectively with outside
contractors (who were key architects of the prototype), while others gained the
confidence to propose the prototype's budget, and others learned to engage each
other across functional boundaries to make the prototype work. A half-dozen
people, each affected in a different way by the learning effort, described
their work on the prototype. Until the learning history brought their stories
together, they were not aware of the root causes of each others' contributions,
and no one else in the company was aware of the process at all. The learning
history included a measurable "result" -- the new prototype saved millions of
dollars in rework costs -- but that result, in itself, was not enough for
duplicating the achievement. Only the stories, and the participants'
interpretations of those stories, brought the result to life.
Transformational learning versus retrospective reflection: Some
learning histories have been created after a project is over; participants are
interviewed retrospectively, and the results of the pilot project are
more-or-less known and accepted. Other histories are researched during the
time the story unfolds. The learning historian sits in on key meetings and
interviews people about events that may have taken place the day before.
"Mini-histories" may be produced from these interviews, so that the team
members can reflect on their own efforts as they go along, and improve the
learning effort while it is still underway. But such reflection carries a
burden of added discipline. It adds to the pressure on the learning historian:
pressure to "prove results" on the spot, to serve a political agenda, or to
justify having a learning history in the first place.
To make an ongoing learning history work, the organization must embrace a
transformational approach to learning. Instead of simply learning to "do what
we have always done a little bit better," transformational learning involves
re-examining everything we do -- including the way we think and see the world,
and our role in it. This often means letting go of our existing knowledge and
competencies recognizing that they may prevent us from learning new
things. This is a challenging and painful endeavor, and learning histories
bring us face to face with it. When the learning history is being compiled
over, say, a two-year period, side by side with the learning effort, then the
challenge and pain of examining existing frames is continuous.
Learning takes place through false starts and failures. A learning history
brings those out into the open. It is much more threatening to do this when
the calamity is fresh. To make the best of a "real-time" learning history, a
corporate culture should be cultivated in which admitting and publicizing
mistakes is seen as a sign of strength. Uncertainty would no longer be a sign
of indecisiveness, because reflecting on a learning effort inevitably leads
people to think about muddled, self-contradictory situations. Much work still
needs to be done on setting the organizational context for an ongoing learning
history, so that it doesn't set off flames that burn up the organization's good
will and resources.
We always end up with what we affectionately call "a mess of stuff,"
accumulated on a computer disk. Now we sort, trying to group the material into
themes, using some social science coding and statistical techniques, if
necessary, to help us judge how prevalent a given theme might be. We know that
we will have to reconsider our themes several times, as the work progresses.
This analysis produces a "sorted and tabulated mess of stuff," which will
become an ongoing resource for the learning history group as it proceeds. The
learning historians might work for several years with this material,
continually expanding and reconsidering it, using it as an ongoing resource,
and spinning off several documents, presentations, and reports from the same
material.
Notable results: How do we know that this is a team worth writing about?
Because they broke performance records, cut the times of delivery in half,
returned eight million dollars to the budget, or made people feel more
fulfilled. Whatever indicators will be regarded as significant by people in
your organization should be included; managers who are skeptical about the
project won't read the report unless the results are included. We find it's
helpful to use notable results as a jumping-off point, particularly if you are
willing to investigate the underlying assumptions -- the reasons why your
organization finds these particular results notable. Often, a tangible result
(the number of engineering changes introduced on a production line) signifies
an intangible gain (the willingness of engineers to address problems early,
because they feel less fear).
A curtain-raiser: What will the audience see when the drama opens? We
begin by thinking very carefully about how the Learning History opens. The
curtain-raiser must engage people, and give them a flavor for the full story,
without overwhelming them with plot details. The curtain-raiser may be a
vignette or a thematic point; often, it's a striking and self-contained facet
of the whole.
Nut `graf: A "nut `graf" is journalism jargon for the "kernel
paragraph," the thematic center of a news story. If you only had one or two
paragraphs to tell the entire learning history, what would you put in those
paragraphs? We try to force ourselves into the discipline of nailing down the
thematic point explicitly; even if it doesn't appear in the final draft, it
will serve to focus our attention all the way through the drafting.
Closing: What tune will the audience be singing when they leave the
theater? How do you want them to be thinking and feeling when they close the
report, and walk away from the presentation? You may not keep the closing in
its first draft form, but it is essential to consider the closing early in your
process, because it shapes the direction that the rest of your narrative will
take.
Plot: How do you get people from the curtain-raiser to the closing? By
deliberately devising a plot to lead them there. Will it be strictly
chronological? Will you break the narrative up into thematic components? Or
will you follow specific characters through the story? Every learning history
demands a different sort of plot, and we try to think carefully about the
effects before choosing one. So far, we have found that many plots revolve
around key themes, such as "Innovation in the Project" and "Engaging the Larger
System." Each theme then has its own curtain-raiser, "nut `graf," plot, and
closing.
Exposition: What happened where, when, and with whom. Here's where you
say there were 512 people on the team, meeting in two separate buildings, who
worked together from 1993 to 1995. The exposition must be told, but it often
has no thematic value. It should be placed somewhere near the beginning, but
after the nut graf.
The right-hand column (jointly-told tale): So far, the most effective
learning histories tell as much of the story as possible in the words of
participants. We like to separate these narratives by placing them in a
right-hand column on the page. Participants are interviewed; we condense their
words into a well-rendered form, as close as possible to the spirit of what
they mean to say, and then we check the draft of their own words with each
speaker before anyone else sees it.
The left-hand column (questions and comments): In the left
column, we have found it effective to insert questions, comments, and
explication that help the reader make sense of the narrative in the right-hand
column.
The
roots of a new form of story-telling
A learning history is a written document (or series of documents, perhaps
including multimedia productions) which is disseminated in some deliberate
fashion, to help an organization become better aware of a learning effort
within its boundaries. The history makes extensive use of participants' own
narratives, as well as outsiders' assessments of the story. It cuts back and
forth between different perspectives on events that have taken place, sometimes
using a two-column format to keep the research team's (and readers')
commentaries separate from the participant's descriptions and evaluations.
The
stance of a learning historian
The role of "learning historian" is new and it runs counter to many
organizational habits. Thus, even in the first year, paradoxical dilemmas have
emerged, all related to the stance that a learning historian takes.
Researchers, writers, managers and advocates who take on this role find
themselves beset by organizational and external pressures. That is why we see
a crucial need, right from the start, to define the role, so that this
relatively fragile innovation has an opportunity to grow and thrive.
Conclusion
A new philosophy and approach to assessment is embodied in learning history
work. At the Learning Center, we are very careful in using the word
"assessment." We now "write learning histories." We include a learning
historian as part of the team. The learning historian's job is to capture and
tell the story. That is the language we use. It is amazing how this approach
resolves a lot of psychological and emotional problems associated with
assessment. People don't want to be assessed. They want to share. They want
others to know what they've done -- not in a self-serving fashion, but so
others know what worked, and what didn't work. They want their story told.
Sidebar:
How to create a learning history
Like all recipes, this checklist of steps should be taken as a place to launch
from, not a strict formula to follow. Every learning history project will be
different. We have found the following steps and components useful for moving
forward:
Start by gathering information, through interviews, notes, meeting transcripts,
artifacts, and reports. We found that, for a project which involved about 250
people, we needed to interview at least 40 individuals, pulled from all levels
and varied perspectives, to get a full sense of the "real life" of the project.
We tried to interview key people several times, because they often understood
things more clearly the second or third time. We found it useful to come up
with an interview protocol, based on notable results: "Which results from this
project do you think are significant," we asked, "and what else can you tell us
about them?" All interviews in our work are audiotaped and transcribed. 
Accumulate
and Organize Data
Writing
the Learning History
At some point, whether the presentation is print or another medium, it must be
written. We have found that the following components are important. Generally,
we produce them in the order given here, although they will not necessarily
appear in that order in the final document.
Sidebar: Example of a learning history
This example describes part of a pilot program at a large manufacturing
company. Since the learning history has not yet been released for public
distribution, all specific references to people, places, and events have been
altered. Everyone mentioned is a real person; in this learning history, we
referred to all of them by titles only, except for Frank Jones, who was known
publicly as one of the leaders of the project, and a few other similarly
prominent individuals. Note the effort to keep each manager's voice distinct
in the right-hand column, and the need to make the left-hand column as
thought-provoking as possible.
| The learning lab created an opportunity for people on the team to begin to address deeper issues, with the team leaders involved. | Frank Jones (a top-level manager of the pilot project): Not long ago, two managers (call them "X" and "Y") began to attack me at a learning lab. I didn't understand them. So I continued to encourage them to say what they really felt. |
| . | Manager "Y" [telling his version of the same story]: My biggest pet peeve is that we were wasting our time in sometimes four or five meetings per week about making last-minute changes in the specs. This is not unique to our program; this was going on for years at the company. Frank would go after little details, rather than letting me manage them. |
| And managers, having promoted a climate of openness heard directly about the impact they had on engineers. | Frank Jones: "Frank," they finally said, "You're making our lives miserable. I can't get anything approved without coming to you and getting permission. Why do we need a system that is so cumbersome?" Lo and behold, I said: "Because I don't trust you." |
| Which required them to answer honestly...
...and that, in turn, allowed people to speak much more truthfully about underlying issues. |
Internal consultant: When Frank said that (and, actually, he shouted it),
there
was an uncomfortable silence in the room. What went through our minds was: We
always suspected Frank didn't trust us, and now he's telling us as much. Then
Frank
proceeded to say, "And let me tell you why I don't trust you. If I did nothing
to pressure you, you wouldn't meet your deadlines."
Frank Jones: I would have had a difficult time saying that to anybody in the past. It would have cut the cord of communication and any hope for trust. |
| How much self-esteem does it take, under these circumstances, to keep from feeling threatened or attacked? | But what happened next was amazing. They didn't get mad at me. They simply accepted that it was my position: I couldn't trust them to make changes correctly. And I accepted their position: that they were upset with the way I was acting. All of a sudden the truth came out. We finally got down to the nitty gritty -- a meaningful discussion about how to dispel the problem.. |
| But it did appear to have an effect in another sense...
...which others noticed some time later. |
Another manager: I'm one of the people that "X" and "Y" had fought with in the past. I've noticed already that they handle the issues differently than they did six months ago. "X" and I met and brainstormed together yesterday morning, and came up with a couple of ideas. That would have been unheard of in the past; he would have simply said, "I'm not helping you." I realized that I've got to be retrained too, because I still don't trust them. |