Developing a Theory of Change, Part 1: A Logical Process

This is the first in a two-part series about the Santa Cruz Museum of Art & History’s new theory of change. This week, Ian David Moss and I are each writing blog posts about our collaborative process to develop a theory of change at the Santa Cruz Museum of Art & History. Check out his blog post on the Fractured Atlas site. Next week, I’ll share more about what is in our theory of change, and why.

For three years, we hit the gas at my museum—hard. We were pointed in a new direction and knew we had to push ourselves to expand our community impact.

Three years in, the dust settled on the many changes. We had tripled our attendance. Doubled our staff. Experimented, launched, and retired many programs and exhibition formats.

We decided it was time to shift from exploring to deepening. A little over a year ago, we received a three-year grant from the James Irvine Foundation to strengthen our commitment to community engagement. One of the first things we decided we had to do was to grow some roots under the new strategies at the museum. We embarked on a process of “naming and claiming” the work we do.

Where to start? We decided to develop three things:

• Clear engagement goals that define how we do our work
• A theory of change to connect what we do to the impact we seek
• An engagement handbook to provide an overview of our goals, our theory of change, and the programs where they are manifest

I’ve written about the engagement goals before. I’ll write about the handbook soon. This blog post focuses on the theory of change and the process by which we developed it.

A theory of change is a model that connects: the activities we do, to the outcomes they effect, to the impact we seek to create in the world.

We wanted to build a theory of change for two reasons:

1. Externally, we need strong, data-driven arguments for support. We can’t just say,” fund this exhibition and the community will grow stronger.” We have to prove it. Donors want to understand the logic of how their dollars will translate into impact. A strong theory of change can make that case. 
2. Internally, everyone needs to know what “good” looks like and how their work helps contribute to the overall goals of the organization. A clear theory of change helps staff make strategic choices at every level.

We didn’t know how to develop a theory of change. But we knew we wanted to be rigorous about it. So we contracted with Ian David Moss, of Fractured Atlas and Createquity fame, to shepherd us through the process. Besides being brilliant and skilled in this area, Ian came in with an outsider’s perspective, which really helped us get out of the mindset of what we THINK we do and shift into what is actually observable and track-able.

Here’s what we did:

• Ian came to Santa Cruz, interviewed a bunch of staff, and drafted a very rough theory of change based on what he learned about our programming.
• We did a board/staff retreat where we built theories of change in two directions: UP from our activities to our intended impact, and DOWN from the intended impact to the activities that fuel it. The “DOWN” side was the most interesting, because it helped us understand the role we could play in our desired impact—and the community partners we would need to engage and support to see the impact realized to its fullest.
• We worked with Ian to revise the theory based on the retreat.
• Ian did a social psychology literature review to understand the research grounding the connections we made from activities to outcomes to impacts. We identified areas where the connections were weak and where we have to do more research to ensure that the logic is sound.
• We developed a final version of the theory in a wonky powerpoint flowchart model.
• We worked with a fabulous illustrator (Crista Alejandre) to transform the flowchart into an inspiring graphic.
• We started using the theory of change to focus our programs and partnerships, evaluate our work, and change the way we talk about what we do.

Here are some questions for Ian about his side of the experience working with us on this project. Next week, I’ll write about the actual content of the theory of change and how it is starting to impact our organization.

When you are working with an organization on a theory of change, how do you sort out the organizations' aspirations from the reality of their current activities?

Ian: This is always one of the most challenging (and interesting) parts of the engagement. One of the reasons why I find theories of change valuable as a tool for this kind of conversation is that they are really good at making the chain of logic – or lack thereof – between an organization’s activities and goals really clear. That sets up a process where I map out what I perceive the connections to be, and then I run it by the organization to make sure that I’m understanding their thinking correctly. If I spot a place in the logic chain that doesn’t make sense to me, all I need to do is ask some probing questions about it. It could well be that I’ve overlooked something important, in which case I’ll add in whatever’s missing, or it could be that I’ve uncovered something the organization hasn’t thought of, which could spark a much-needed reassessment of what the strategy is or even what the real goal is. (This is exactly what happened to us at Createquity: our theory of change precipitated a global rethinking of our entire content and engagement strategy because we discovered a gap between what we were doing and what our aspirations were.) Either way, the theory of change makes the assumptions embedded in a strategy transparent to everyone and provides a way to put those assumptions to the test.

In our work together, we ended up looking primarily to social psychology research to develop a strong logical basis for the MAH’s theory of change. Do you often find that these projects take you outside of the “arts” field in terms of defining the logic that connects activities to outcomes to impacts?

It depends. I would definitely say that you guys are unusual in how you see non-arts and non-humanities research and practices as not just relevant but central to your work at the museum. But I’d venture to say that it’s a rare arts organization that can’t learn something from how things are done in the wider world, whether that means understanding how and why potential audience members are motivated to make the choices they do, or understanding the policy context for the community-level changes you’re hoping to see, or whatever. I think a very common mistake people make is to draw the frame too narrowly, to say “well, we don’t have any data on this exact thing that we’re looking for, so there’s no point in trying to answer that question.” The reality is that we have many tools to understand and to estimate the way the world works around us, and there are a lot of parallels and inferences to be drawn either from examples in analogous fields or from initiatives that have a general focus that includes the arts but isn’t specific to them.

What do you think is the most challenging part of developing a theory of change?

Different projects present different challenges, but one thing I’ve found to be consistent is that the theory of change process can end up drawing out major differences in thinking styles. There’s a certain type of person who’s really comfortable breaking ideas down into orderly, modular components and analyzing the connections between them. Then there are other folks who are not at all accustomed to thinking that way – they’re much more at home in an open-ended, anything-goes brainstorming session that encourages divergent thinking and untethered creativity. For those people, the process of creating a theory of change or logic model can very easily feel confining if you don’t set it up carefully. What I’ve found is that things go better if I make sure that nobody has unrealistic expectations placed upon them. A lot of people find it easier to have a conversation and then react to a model presented to them than be tasked with having to figure everything out themselves. On the other hand, other folks want to be super involved and that’s great too.

Any words of wisdom about how to build buy-in and encourage use once a theory of change is developed?

A really good way to do this is to include it in training materials for both current and new staff. The more that the theory of change gets talked about, the more likely it is to be used. You can also use it as a reference point for other institutional capacity-building things your organization is doing. So the MAH used it as the basis for a measurement framework for the organization. At Fractured Atlas, it was a key input for a new brand book we developed to guide our internal and external communications. It can be an attachment to grant applications or included in annual reports to donors. And it’s important that the theory of change be periodically revisited to make sure that it doesn’t reflect stale thinking. That all being said, I would emphasize that going into the process with the intention for a theory of change to be useful is the number one predictor of whether it will actually be useful. Furthermore, the best way to build buy-in for a theory of change is by giving people a voice in creating it. That’s why as much as possible I try to involve front-line staff as well as leadership in the process, so that it will feel resonant at all levels of the organization.

Thanks to Ian for collaborating on this process with us. If you are reading this via email and would like to share a comment or question, you can join the conversation here.

Community Science Workshops and Shared Authorship of Space: Interview with Emilyn Green

Imagine the most community-based science center possible. Imagine it in a poor, immigrant farmworker community. It exists. It thrives. In California. In a Community Science Workshop.

A couple months ago, I visited a Community Science Workshop for the first time in Watsonville, CA. I was awestruck. A small room, packed with gadgets, packed with fossils, packed with tools, packed with PEOPLE everywhere making and exploring and building and learning. The people were of all ages--moms with babies strapped to their fronts, six year-olds using skillsaws, pre-teens building robots, teenagers doing homework. There was a spirit of conviviality and purpose and helpfulness and Spanglish in the air. The design and feel of the place was different than any science center I'd ever experienced. I knew I could learn a lot from it.

I sat down with Emilyn Green, Executive Director of the Community Science Workshop Network, to learn more about their history, design, and engagement strategy.

Can you give me the overview of Community Science Workshops? What are they and where did they come from? 

A Community Science Workshop is a place for kids to tinker, make, and explore their world through science. The first one was started in 1991 by a San Francisco educator, Dan Sudran, in his garage. The Exploratorium is great, but it wasn't super-accessible. Dan was living in the Mission neighborhood, which at the time was very kid-dense, mostly first-generation immigrants, and Dan noticed that when he was messing around in his garage with physics gadgets, he could not keep the kids in his neighborhood out of his garage. They were so fascinated and wanted to be there every day.
There are lots of great science museum resources, but not where these kids can walk after school. In most cases, they're not places where kids can go by themselves at all.

So the Community Science Workshop model is to put a drop-in, FREE community science center in a place that is walkable to kids' lives and schools. In a place where kids are already walking around after school.

The core program is a permanent, dedicated physical space, full of interactive hands-on physical exhibits, as well as a tinkering and making space, and recycled materials. Most Workshops also run a wide range of additional programs - supplemental school day programs, afterschool programs, mobile units that go to housing projects. There are a whole bunch of programs to disseminate the science but the central workshop space is the heart of it.

Where do Community Science Workshops fit in the informal science landscape?

It's kind of tricky. We don't fit the more common templates. The best explanation is "community science center." But in the more traditional lexicon, these might be defined as "informal spaces."

Now that there is a new emphasis on "ecosystems of STEM learning" - for us, that's really helpful. Our programs end up being the hub of the local science learning ecosystem--especially in communities where there isn't a science center for miles in any direction.

How are Workshop locations selected?

The first one was in the Mission. We received two rounds of NSF funding in the 1990s to expand. We've focused on farmworker communities--there are so many kids in these communities throughout California. At that time, we expanded to Watsonville, where strawberries come from. And Fresno in the Central Valley--a city of 500,000, hundreds of miles from the closest science center. And a couple other sites that didn't make it financially (more on that later).

I came on in 2010 to start the statewide nonprofit network. At that point, we opened three new locations: Sanger, Greenfield, and a new San Francisco workshop in the Excelsior neighborhood because the Mission has changed so dramatically. The Mission location is still a useful hub for San Francisco school programs. Excelsior is now the walkable neighborhood space.

What happened to the ones that folded?

We received NSF funding for three years and then it cut off. In places that succeeded, at that point, a local coalition was in place to fund the Workshop. Where it didn't work, no local support stepped up. That taught us a lot. Now, when people are interested in starting one, we emphasize permanence. It takes a local coalition of people who are really committed to this to have a program that lasts.

Who are those local coalitions?

We're really different from traditional science centers in our funding model. We don't have paying members. We're not going to do that. It's not a Community Science Workshop if it's not free to participants. In fact, we have almost no individual donors. We have a few shining, beloved exceptions, but that's not a significant part of our model at this time.

So we tend to have three legs of support: municipal, grants/donations, and fee for service (usually with the school district). The municipal support can be actual funding - from Parks and Rec, or Environmental Education/Public Works in Watsonville - or a free building, or free access to a van, or materials... or free access to the dump to get materials. Several Community Science Workshops also get Measure S grants - gang prevention grants - through their cities. Grants come from community foundations, small local family foundations, local businesses. And then the fee for service is mostly school districts that contract with the Workshop for science enrichment/science instruction.

What unique design elements make Community Science Workshops work?

Geography is key. We tend to overemphasize it, because it's the initial requirement for any kind of success in these communities. Kids in these neighborhoods are wandering around alone after school. So if kids can walk to us, they can participate.

Once in the space, there are a bunch of design features that continue to be about access. Our fundamental premise is that kids are really interested in stuff. Given a wide variety of stuff, they will find something that they are excited about and will take on projects. We assume the motivation is there. The interest is there. So if a kid is not engaging, it's likely some barrier to access.

One thing you'll notice when you walk in is a ton of user-generated content. Most is hand-made, by participants or staff or parents, and that is everything from our signage to our exhibits.

We tend to be fully bilingual where appropriate. Our staff is almost always Spanish/English bilingual. And we hire from the community. Over 30% of our staff statewide are either former Workshop students or parent volunteers. That's an important design principle for the space - who the kids see when they enter the space.

We have some considerations about height and accessibility. We make sure that kids can grab materials and tools without staff intervention. Part of this is practical: you can't facilitate making and tinkering for 30 kids if you have to hand them everything they need. And it shows kids that they can be the agents of their own learning.

And then there's the most important design element - it's MESSY. We've been playing around with different ways to describe this and not terrify people. It's not messy like “vermin-infested”. It looks like a space that is used by humans every day. It's "purposfully messy." It's organized enough so people coming in can learn the layout, but it's the opposite of sterile. Surfaces are dense, and covered, and richly layered, and there is nothing in the room that implies "don't touch me or you'll get in trouble." That bar is pretty low for kids - they really need to know they are welcome to explore.

I loved the feeling of the space. It made me think of the Spanish word "ambiente"--that convivial, welcoming feeling. It also made me think about some of what we learned in a recent ethnographic study in which some Latino moms talked about "American events being so organized, whereas Latin events have joyful chaos." I know that most of your design focus is on kids. Do you think there is also a cultural/ethnic aspect to the kind of access and design you use? 

I feel very careful talking about the ways that the particular populations who we choose to work with for social justice reasons are also the people who make our work possible because of cultural expectations. For example, kids and tools. It's much easier for us to work with kids with power tools in these Workshops than it might be with other families because a lot of these parents use tools in their own lives. They are comfortable with them.

I was amazed by how community-based and authentic it feels. Many science centers struggling to engage "underserved" people with informal science. You are succeeding. What do you think is the difference? 

I come back to geography. Easy access to the space. When I talk with people in science centers, some really dedicated people working on these questions, they acknowledge that geography is a big hurdle they have to get over.

But there also is a sense of community ownership. For example, the Exploratorium is an extraordinarily participatory museum, but it's not nearly as participatory as a Community Science Workshop. Any big museum has barriers and limitations to full community ownership. Anyone walking into a CSW could repair a broken exhibit--anytime.

And that's the way we respect the contributions of the families we are working with. They are authors in the space.

To make a sweeping generalization, it seems that the folks we work with--working class people, people who work with their hands for a living, people with larger extended families - are very comfortable with spaces where multiple people are authors. They are comfortable with shared authorship of space and events. Whereas formal organizations have a harder time facilitating a shared sense of space and events.

Really interesting. So there is authorship, but no bylines.

 No visible bylines. But they still exist informally. The bylines are in the community’s awareness of the space. "Aurora made that sign." "I helped paint that sign." "I was here when Sal was working on that." People know, but they know based on their real experience of it in the space.

When I visited, it was so clear that there are so many people who use the Workshop again and again, who build things over time, who get involved in different projects for different reasons. 

Yes. And it's hard for us to document. Half of our attendance per year is in enrollment and school-based programs, but half is in our drop-in spaces, where we don't track participants at all. We know anecdotally that a lot of these people are repeating and deepening their participation, but we don't have the data.

We're just starting to interview our alumni now and creating a catalogue of their stories. It's powerful. But that's not the focus of the program. The focus of the program is to make it work every day.


Big thanks to Emilyn Green and the Community Science Workshop Network for sharing insights in this post. Emilyn will be checking in on the comments here and can respond to your questions.  If you are reading this via email and would like to share a comment, you can join the conversation here.

Crossing the Professional-Amateur Line

Think back to the last time you crossed a line.
Did you feel brave? Deviant? Proud? Ashamed?

In art institutions, we typically treat the curator as the arbiter of quality. The curator draws the line between worthy and unworthy. He may explain the rationale behind where he drew the line--or not. Either way, visitors are expected to respect the line, respect the judgment, and appreciate the resulting display.

I'm not sure how well this is working for us in museum-land.

As our culture explodes in embracing creativity across the professional/amateur spectrum, museums have two choices:

1. we can sharpen the line.
2. we can embrace the embrace.

#1 is the weaker choice. It's the kid at the beach, frantically retracing the line in the sand with his foot as the sea swallows it. It's him yelling, "this is the way you are supposed to play!" and his voice getting lost in the waves. It's him standing on the beach, alone, as everyone continues the game around him.

#2 is the powerful choice. If done critically and with intention, it advances understanding of different types of quality and different levels of expertise. It covers the entire beach. If done poorly (uncritically), it turns the museum into yet another place to watch cat videos.

I actually believe that we have MORE ability to advance scholarship and curatorial expertise with #2 than #1. The line in the sand is not your expertise. It's a weak symbol of your expertise. And once everyone has trod over the line, it loses its power.

Expertise is worthy. Period. The classics don't lose their power when they share library shelf space with beach reads. Top boxers don't lose their status when they fight at the end of a long bill of lesser athletes. Why are we so afraid that great art will lose its power if we surround it with other work?

I've been grappling with this question as we enter the final month of a massively crowd-sourced exhibition at my museum called Everybody's Ocean. We took our inspiration for the exhibition's format from the vastness and complexity of the ocean. The ocean is a force. A home. An inspiration. A trash receptacle. We wanted to reflect this diversity of identities by inviting all kinds of artists and artworks into the gallery. The exhibition includes artwork about the ocean by over 250 artists of all backgrounds and abilities, grouped into ten poetic metaphors for the ocean.

The exhibition has been polarizing. Most visitors love the diversity, quality, and abundance of the art. But some visitors--especially some artists--hate it. As one upset artist said to me:

"We artists have so little. Being included in a museum exhibition used to be a sign of real achievement. With this exhibition, you've ruined the sanctity of that experience." 

When I hear this, I feel sad that the value of artistry is being reduced to a thin line of curatorial discretion between "in" and "out." I understand her concern. But I think it's a concern born out of weakness, not strength.

How can we be proud of the artistic ground that we are covering without worrying about where we draw the lines?
How can we go on the offensive instead of the defensive about the power of art and quality?
How can we cross lines joyously, thoughtfully, critically--and without fear?

If you are reading this via email and would like to share a comment, you can join the conversation here.

The Participatory Museum, Five Years Later

This week marks five years since the book The Participatory Museum was first released.

I wrote The Participatory Museum for two reasons:

1. to explore the "how" of participatory design in museums, cultural centers, libraries, and science centers
2. to create a version of this blog that was more "shareable" with organizational leaders and trustees 

By many measures, the book has been a success. Over 20,000 copies have been sold around the world. Over 150,000 people have accessed the free online version. I've given talks and workshops about it all over the world. Weekly, I hear from someone who is putting ideas from the book into action. 

Across the museum field, the questions about visitor participation have gone from "what?" and "why?" to "how?". I feel lucky to be a small part of that change. 

That said, there are a couple big things I got wrong in the book - or at least, that I've changed my perspective on since writing it. 

1. HUMANS ARE THE BEST AGENTS OF PARTICIPATION

When I wrote the book, I was coming from the perspective of an exhibit designer. I thought the pinnacle of participatory practice was an exhibit that could inspire collective visitor action without facilitation. A black box with people crowded around, talking and sharing and making and doing.

Now, I look back on the book and the biggest thing I see missing are the people inside that box. 

If participation was my mantra from 2007-2011, community has been my mantra since then. Over the past four years, I've been running a small regional art and history museum in Santa Cruz, CA. Our museum is highly participatory: plenty of opportunities for visitors to contribute, for artists to collaborate, for community members to co-create. But almost ALL of those opportunities are facilitated by people. Those people are driven not by the design precept of participation. They are driven by the human precept of inclusion and involvement. 

I no longer feel like the "best" forms of participation are unfacilitated. Like many engineers, I think I was overly presumptuous about what design could do on its own. Since 2010 I have seen, again and again and again, how valuable human facilitation is to the participatory process. Humans empower each other. Make space for each other. Invite each other in. Cheer for each other. Build community.

Five years ago, I was fixated on unfacilitated participation because I thought it was much easier to scale than facilitated participation. It is. But I've learned that humans can be agents of scale too. Every time we encourage a volunteer to launch her own collections research, or empower teens to launch their own program series, or invite new partners into our projects, we invite them to participate. That participation is powerful and scalable. We just have to think differently about how we craft job positions and expectations. We tell all our new colleagues: you will not be judged on what you do but how you empower others to do. We need employees who focus less on creating experiences directly for visitors and more on creating platforms for visitors to share with each other. Then, participation can be scalable--and human, too. 

2. PARTICIPATION IS POLITICAL

When I wrote The Participatory Museum, I wrote about participation as a design tool--a wrench that could be turned to reach certain goals in a cultural setting. As a designer, I wanted to present participation as "value-neutral," or, as I wrote, another technique "for the cultural professional's toolbox." I acknowledged that participatory projects best support goals like relevance, dynamism, personalization, socialization, and creativity--but I avoided much argument about why those goals might be more more important than others. 

This choice was strategic. I didn't think the field needed another argument for "why" participation. I wanted to dive into the "how" instead. But this choice also belied my background as an engineer and designer--someone trained to think of tools as apolitical.

Over the past few years, I've learned that participation can be highly political. When you seriously value the diverse experience and knowledge of community members, you challenge the traditional assignment of knowledge authority. I always knew this cognitively, but over the past few years, I've experienced it directly and viscerally. 

Our greatest champions and our loudest critics agree: our museum has become an inclusive community gathering place. Whether you think that's incredible or a disaster depends on your point of view. 

Terms like "inclusive," which have become commonplace in the field, are still highly contested when put into action. I know we're doing something right when community members are arguing about it. Over the past year, our board and staff have leaned into the political nature of what we do, developing a theory of change with community impact at its heart. I've gotten more comfortable and more confident with the idea of the museum as political body that advocates for empowerment and social bridging. When we really live our mission, that's where it takes us. 

But that's just our institution--not everyone's. I still feel strongly that there is no universal reason to encourage visitor participation. What is participation for? Is it intended to increase learning? Empowerment? Social bridging? I don't care what your reason is... as long as you have one. 

The reason, the mission, the underlying goal--that's what fuels me now. I don't identify as a designer/engineer the way I used to. I identify as a community activist. And if I ever write another book, it will probably be about that.

I'm curious to know: has The Participatory Museum played a role in your work? What do you think has changed around these issues in the past five years?

If you are reading this via email and would like to share a comment, you can join the conversation here.