The Participatory Museum Process Part 1: Overview and Statistics

This is the first of a four-part series on the behind-the-scenes experience of writing The Participatory Museum. This week, we'll look at the overview of the process of the creation of the book and some overall statistics for user participation. Next week, part 2 will focus on participants' experiences. Part 3 will focus on my experience, and part 4 will discuss the self-publishing process. Please let me know in the comments if there's anything in particular you want to know about - I'm happy to share whatever interests you.

Overview: Stages of Development and Participation Types

The Participatory Museum was written over a 15 month period that began in December of 2008. Participants were engaged in the following ways:

• Content review (open). I wrote the plans, outlines, and multiple drafts on a public wiki that was available for review, edits, and comments. While all stages were open for comment, I made an explicit ask right before releasing the second draft, and consequently, the second draft was most heavily edited. 65 people participated on the wiki, though the vast majority of the activity came from a core group of 15 (more on that below).
  
• Content review (solicited). In addition to the volunteers who signed up to help on the wiki, I directly solicited sixteen professionals in the field who I respect to provide feedback on particular chapters (or in some cases, on the entire draft).
  
• Content review (stealth). Many of the book sections started as blog posts on Museum 2.0. Sometimes, I'd put out a post on something I was struggling with for the book (see this early example). Readers' comments helped steer the book, even if these commenters never visited the book development wiki. At least 50 blog posts and 240 related comments fall into this category.
  
• Copy editing. I invited people to sign up to copy edit sections of the final draft. 17 people contributed to this effort. This process involved people downloading sections of the book, editing them in Microsoft Word, and reuploading them. Each section went through two copy editors for redundancy.
  
• Marketing copy. I invited people to help develop marketing copy for the book - to vote on the title of the book, help write the blurb for the back, and comment on the cover illustration. 210 people voted on the title, 6 contributed to the blurb, and about 30 commented on the cover.
• Image and content research. In a fairly traditional process, I asked professional colleagues to help me source images and examples that should be included in the book. In a couple of cases, I opened this up broadly to my Facebook or Twitter network, for example, when I was looking for a generic shot of visitors checking out a photography exhibition. I also cold-contacted people on Flickr with image reproduction requests, 100% of which were granted.
  

My general approach was to solicit as much participation as I felt I could usefully integrate. I didn't want to waste anyone's time with work I wouldn't be able to use, but I also wanted to stretch my own boundaries about how much help I really could accommodate. In the end, I felt great about the level of participation and it made a HUGE impact on the book. In particular, participants helped me:

• Cut the length of the book from 125,000 to 99,000 words. There were many redundancies in the original draft, and Bruce Wyman in particular was delightfully brutal at pointing them out.
  
• Restructure the book. You may notice that the graphs below show only six chapters, whereas the final book has eleven. This restructuring was based on their comments and edits.
• Streamline case studies, especially those in which I was personally invested. Conxa Roda was a queen of cutting.
• Improve the section on evaluation. Peter Linett, Mark Kille, and Andrea Bandelli were instrumental in making this happen.
  
• Track down examples from further afield, especially from smaller institutions, institutions outside the English-speaking world, and institutions that focus on living history.
• Shift the tone of the book. Sarah Barton and Elaine Gurian in particular helped me settle on a more generous, positive voice.
• Generally feel confident about making big changes. I made the vast majority of edits and restructuring on my own, but I was bolstered in doing so by the many comments and opinions of the contributors. They cheered me on conceptually as I worked late into the night. Without them, the differences among the drafts would have been much less significant (and the final result of lower quality).
  

When and how did people participate the most?

With the exception of spikes each time I made a blog announcement about the book development, the traffic to the wiki stayed pretty constant throughout 2009. The dead time in October was an error on my part--that data was lost.
Looking just at the returners (and excluding me), you can see how low the overall traffic was, and how concentrated in the various participatory time periods:
Despite the fact that the wiki enjoyed more visits during the first draft than the second, there was far more editing activity for the second draft. I took a different approach to the two drafts: the first draft was made available as I wrote it, whereas I released the second draft all at once (and gave people a fixed timeframe in which to make their comments). Here's a graph comparing wiki activity during the first draft (Feb-Oct, 2009) and the second draft (Nov 1 - Dec 18, 2009):

Clearly, the numbers of comments and edits were WAY up for the second draft. But this doesn't tell the whole story. When we look at the graph of the relative number of people involved, it looks like this:

Chapter 1, 5, and 6 received a lot more love in the second draft than the first, but the difference otherwise is not huge. The outline is an outlier because it was used as a planning tool and enjoyed lots of discussion among people who were interested in the book in its earliest phases. The number of people actively involved from draft one to two jumped from about 5 to 15--but those fifteen people collectively made hundreds more comments and edits to the draft. Of those fifteen, just six--Sarah Barton, Conxa Roda, Mark Kille, Mike Skelly, Louise Govier, and Claire Antrobus--contributed 95% of the edits. The power law is alive and well.

It's worth noting that I only knew one of these six people (Conxa) before this process began. In particular, Sarah Barton had incredible influence on the content development, and she is someone I would definitely go to in the future for content review.

While I was thrilled by the participation of this small group, it was obvious that a huge number of Museum 2.0 readers were not involved in the wiki process. For this reason, I stepped up the double-posting of case studies and book content on the blog and wiki so I could have the benefit of the comments of a wider group. Blog commenters, who not represented in the graphs above, represented the most diverse and numerous group of participants in content review. Hundreds of people offered a single comment on a post or tweet throughout the second draft process.

The solicited content reviewers also had incredible impact on the book, but in a way that was quite different from that of the wiki and blog commenters. The wiki and blog comments appeared in real time, whereas the solicited reviewers sent me their complete edited manuscripts in one pile in mid-December. This meant that throughout the writing stages, I could rely on blog and wiki commenters to steer me in the right direction. By the time I got back the manuscripts from solicited reviewers, I was pretty much "done" covering the wiki comments and could focus fairly exclusively on the solicited drafts. These solicited commenters in general provided incredibly detailed comments, though a few folks opted to offer an overall impression instead. Frankly, I'm glad that not everyone wrote line-by-line comments - I couldn't have handled it. Special thanks go to Georgina Goodlander, Ed Rodley, and especially Bruce Wyman for doing something very wonderful with their edits: making me laugh.

A few surprises

Every time I do a project that involves user participation, I'm always surprised to find some of my expectations are completely off the mark.

Here are four surprises I encountered in this project:

1. Unsolicited contributions were at least as valuable as those that were solicited. Part of me suspected that the people who I directly asked to review the drafts would be more honest, more critical, and just generally more helpful in the direction of the book. I expected that wiki volunteers would mostly be Museum 2.0 fans who might not feel comfortable being critical, especially in a public venue. This was not the case. I received FABULOUS critical comments on the wiki, including and especially from people I did not know. In one case, Chris Castle, one of the few people to comment on the first draft on the wiki (and someone I didn't know), became someone I solicited formally for the second draft because I had appreciated her early contributions to the wiki.
2. The numbers worked themselves out. I was nervous when I threw open the wiki and over a hundred people registered to edit. How would I deal with a hundred commenters and the 16 I'd solicited? But it turned out that only a small percentage of that hundred got deeply involved - a number (15) that was manageable for me. And that's not to say others who made a single comment didn't have impact--I got value out of every comment and edit, even if people only contributed once.
3. People preferred to comment on a finished draft rather than the work in progress. At the time, I thought people would be MORE excited to comment and help shape the book as I was first writing it than to comment on a complete draft. I was wrong. The second draft was offered to participants with a much more specific, time-limited ask, and it was much more successful than the open-ended "help me as I write it" approach to draft one. This makes sense - the second draft experience was much better-scaffolded - and it made me reconsider the extent to which participants want to be involved in the early development of other peoples' projects.
   
4. People loved to copy edit. I was nervous that no one would want to copy edit. I had lined up a good friend to save me if needed (and Dave Mayfield did make many key contributions). But I was totally wrong about this. People were THRILLED to copy edit. Copy editors were the most likely to enthusiastically blog or tweet about their experience. They were also more likely to be young or new to the museum field than other contributors. I think copy editing was a way that people felt they could make a meaningful contribution without having to be some kind of expert. They got a sneak peek of the most final draft. And apparently some people LOVE finding grammatical errors. Heck, I guess I do too.
   

Next week, tune in for a post focused on participants' experiences--how they were encouraged to participate, how they felt about the experience, and how the feedback and reward structures worked. If you were a participant, please consider filling out this short survey to add your voice to next week's post.

What else do you want to know about this book-writing process?