Friday, January 25, 2008

Listening in Multiple Directions: The Value of Tracking


Pop quiz. At any given time, do you know:
  • how many visitors are in your museum?
  • how many members are in your museum?
  • whether an individual visitor has visited before, and if so, when?
  • the purchase history of visitors buying items in your store?
Shockingly, for many museums, large and small, the answer to these questions is no. Ticket systems don't talk to membership databases. The store and the museum entrance are strangers. Each person who enters, whether they visit weekly or once in a lifetime, is treated the same.

Yes, some retail works that way. In Macy's or at the movie theater, I'm just a credit card. But not so in other facilities that encourage repeat visitation. Consider, for example, how my climbing gym uses data. When you arrive, you have to swipe your member card (comparable to museum membership) or pay for entry if you are not a member (again, comparable). The person behind the counter can instantly see how often you come, when you last visited, and can relay to you messages about your bill, classes you are taking, or warnings/compliments about behavior based on prior visits. It doesn't matter whether I've never met the person behind the desk or not--he or she can engage with me appropriately based on data. This kind of tracking enables faceless visitors to be treated as regulars, long-lost buddies, and or potential sales opportunities.

And on the web, this data use goes to the next level. Many months ago, there was a provocative post on the O’Reilly Radar called “If Google Were a Restaurant." In it, the author explored the Google's "pervasive culture of measurement," and considered how similar tracking and alterations might be made in the real world. Imagine a restaurant that tracks every single order, how much food is left on every plate, and uses that information to buy, market, and plate food differently. Imagine book stores that rearrange displays every day based on the sales of the previous day. Imagine museums that… well, we’ll get there.


The point is that Google monitors every single transaction on its site. It knows how many people in what geographic areas search for which terms when. It knows what people are most likely to click on for a given search. And it uses that data to prioritize content—to make the search functionality better.


Many interactive sites do this. Amazon tells you not only what product you might like, but when you click on a product, it tells you what people who clicked on it actually purchased. There are sites that change their inventory, their content, and their marketing to personalize your experience based on passive monitoring of your usage.


Is this Web 2.0? Absolutely. It’s not user-generated in the typical sense, but it fits the basic law of "architecture of participation with network effects": the services get better the more people use them. If Google wasn’t collecting information about what people click on for a given search term, they would have to hire thousands of people to prioritize the content on the web for meaningful results—that is, they’d have to curate the internet. Instead, Google lets users do this curating with their everyday searches for climate change and Paris Hilton. It is user-generated. It just doesn’t take any special effort on the part of the users.


So what about museums? Through the ubiquity and ease of web statistics tracking programs, museums are learning who does what where on their websites. Simply making your collection available for browsing in a database fashion—without any fancy tagging interactivity—can give you data about what parts of the collection are most interesting and or accessible to your web users. These kinds of stats have been used to help museums reorganize their sites; for example, to put PLAN A VISIT front and center when they determine that that’s the content most web visitors want.

On the web, it’s easy to track user actions, and, if you’re inclined, to act on them. But what about in the museum? As discussed in the O’Reilly post, it’s much harder to track and assimilate data in the real world than on the web. We can certainly do it at the ticket counter. Inside the museum, it becomes trickier; exhibit tracking requires human intervention or highly integrated technology. This is what evaluators do, though their monitoring requires resources and effort. Many educators hand out surveys at the end of programs. As more museums, particularly science and tech museums, move towards personalizing the museum visit using RFID and other technologies, these same technologies are being used to see how many people spend how much time interacting where. There are some privacy and data integration challenges, but real-time tracking of activity in the museum is possible.


And tracking isn’t the real challenge when it comes to “googlizing” the museum. It's not brain surgery to create a system, like that at my gym, that gives you historical information about each visitor that walks through the door.
In fact, there are many museums, including my own (The Tech), that collect data on visitor use of exhibits incidentally--but do nothing with the data. Now that RFID and barcode scanners are becoming more popular as tools to "personalize" the museum experience, some museums are generating a lot of data about who uses what exhibits for how long. And while we hand that back to visitors--in certificates congratulating them on completing several exhibits (Sony Wonderlab) or personal websites with images from their visit (The Tech)--we don't use it internally.

Why don't we get this data, and if we have it, why don't we use it? Because the challenge isn't tracking: the challenge is to listen to and act on the tracked data. That's the problem I've seen with educational program surveys--the results are tallied, the great quotes are emailed, but intelligent actions aren't taken. Let’s say you performed an evaluation in which you placed tape recorders in a museum wing for a month and recorded every single thing said. If the transcription revealed that visitors were confused, disappointed, or disaffected by exhibits, were skipping some in favor of others, how would you react? Would you repeat the experiment on other areas? Would you move the most popular exhibits to the front of the wing? Would you reconsider inclusion or implementation of the least popular ones? Or would you destroy the tape?


I think most of us would destroy the tape (or at least hide it in a drawer somewhere). On the web, we’re willing to reorganize content to improve the visitor experience. But in the museum, it’s more expensive, more painful, more personal. This is one of the secret sources of resistance to 2.0: it requires listening to people we're not used to listening to. Beth Kanter wrote about this recently, commenting:
The premise is that listening must become a priority in order to use the Web2.0 tools successfully. I think it is a pretty critical marketing practice despite what technology tools you are using.
And if we can get over ourselves and start listening, there are real financial and visitor rewards to be realized. Google and Amazon doesn’t track user actions to "appreciate" their users' interests. They do it to be more effective institutions. They do it to sell more stuff.

How can museums become more willing listeners? By designing for modularity. By acknowledging unsolved design problems (exhibits stuffed into hallways, nooks of random growth) and considering visitors' input to find solutions. By taking on more experiments so that we're generally more change-positive. By opening exhibits before they are finished so that evaluation isn't an onerous afterthought. By generating more data about visitor actions and analyzing it.

Good operations officers know how much specific exhibits cost in maintenance and disposables. Good marketing directors know that there's a direct relationship between number of visits and likeliness to buy or renew a membership. Why aren't we tracking the data that can make us more successful? Wouldn't you like to know what value different exhibits and programs have as you work out the equation of what's worthwhile and what direction to take?

7 comments, add yours!:

Anonymous said...

what kind of suggestions can others offer that would allow this kind of tracking in a museum that does not charge admission?

Nina Simon said...

Anon, great question (and good related point)!

Door sensors? Greeters who count? Occasional exhibits that measure traffic in some way (i.e. register each time they are used)?

Please share your ideas...

Beth's Blog said...

what's really interesting to me is the metrics for web1.0 are different from the metrics for web2.0. Some of the important social media measures aren't available with google analytics.

Great post!

Aging in Place Guide said...

Dear Nina,
Though you are one of my favorite people I do not always read your blog. But I read it twice lately and (not surprised) found the ideas incredibly relevant for my interests and work which are not about museums at all. You are a very smart and special person and i love that I know you. Thanks you sooo much.

Love, Louis

Anonymous said...

As an education researcher as well as someone who works in a university museum, I've found that the questions of who we track, why, how and should we, don't get as much consideration as I'm used to giving for my Human Subjects' permissions. I agree that there is a wealth of information out there, but at the same time, that information needs to be treated with respect. Some of the richest information that lower budget places can get, through paper surveys or interviews, also take visitors' time. I don't want to take up 10 min. of a grandfather/granddaughter's 1 hour visit to collect an interview if it (1) does not add to their experience or (2) (as encouraged in this post) actually used to make useful changes that help visitors.

Although the process university researchers have to go through can be cumbersome, to assure that data is being collected ethically (and helping the university avoid a lawsuit, let's be honest), I think it also raises great questions for us to consider the value of visitors' time and information. It may be easy to collect by the boatloads, but that doesn't mean we should treat it with less value.

Christopher1974 said...

I worked in a small restaurant years ago that tracked some of this kind of data. We had log books where we kept information about what the daily special was, what the weather was like, what events were happening external to the restaurant (it was in a college town so things like: alumni weekend, homecoming, freshman orientation) and then what the sales were at key points in the day as well as the total for the day and how much food was used and sold. If there were other notes, that was included too. This was a fantastic tool. And it was all done with paper and pencil. I'm sure now they have it all computerized. I've worked in several restaurants before and after -- none were as attuned to tracking data, and not all of them have continued to survive either.

Nina Simon said...

Audra,
I think that the issue you raise about legality and sensitivity is a good one. I've been talking recently with a museum director who wants to eliminate entry fee--but require every visitor to give personal information (demographics, email address). Clearly there's a huge $$ value to this information, and we need to be honest and open about soliciting it and using it.

On the other hand, services like Google have turned around the value of such data, giving it back to users in the form of better content. That's a win-win... as long as the user still understands what he or she is giving up. The privacy issue is a biggie, and yet I think most museums right now are SO far from sensitive areas in that regard, practically throwing data away, that we might as well start looking into using it and deal with the sticky rights issues as they arise.