Tuesday, November 18, 2008

Comment Cards 2.0: Three Tools to Check Out

In many museums, comment cards are currently the most "participatory" part of the visitor experience. It's the one place where visitors can offer direct, open-ended feedback on the institution's content and services. But there are three problems with museum comment cards:
  1. The comments are so scattered over a wide range of topics (including generic ones like, "Thank you!") that the signal-to-noise ratio is low. Unless you digitize them, they become unwieldy and impossible to search through and derive meaning from.
  2. In most institutions, the suggestions on comment cards don't get to the people in power. If they are read, it is primarily to address any chronic problems (i.e. complaints about the third floor bathroom), not ideas or opportunities.
  3. There are few, if any, ways to write back and continue the conversation with the visitor who commented. Relatedly, there is no way for other visitors to easily join threads of conversation--to do anything but offer their own discrete atomized comments.
In other words, comment cards aren't effectively organized for use. Enter the internet. Recently, a whole slew of new web-based applications have popped up to make it easier for companies to gather and prioritize feedback from users. While these began in the pretty geeky world of people suggesting new features for software (see featurelist as an example), there are now several services with slick user interfaces that allow you to offer feedback on everything from Whole Foods markets to Barack Obama's presidential agenda. Because these services present individual feedback in a social environment and allow users to vote for their favorite suggestions or questions, the institution can easily see the prioritized desires and concerns of visitors without having to read hundreds of cards.

These services could be a powerful, cheap alternative to comment cards--especially those that are focused towards making suggestions about the museum. If you are considering replacing comment books or cards with digital kiosks, why not put the kiosk online and use a system that will allow visitors to vote for others' suggestions, comment on new ideas, join conversations with staff about opportunities, see which suggestions have been adopted by the institution? These third-party applications provide a ready-made environment for comment cards to become more useful and usable to visitors and staff alike.

Here are the top three tools I've been exploring: IdeaScale, GetSatisfaction, and uservoice.

prioritizing suggestions for specific programs

A couple of weeks ago, I opened this Ideascale website to invite readers of this blog to suggest and vote on Museum 2.0 community activities of interest (please vote and comment--I will move to action stage at the end of the month). Ideascale is the most basic of these three tools, offering three actions a user can take: suggest an idea, vote for or against an idea, and comment on an idea. The ideas can be tagged and grouped into categories, and can be browsed in time order, by most popular, or by category.

My account is free, but you can pay $15 per month for a bunch of moderation tools and secure portals. There is also a way to award rewards based on the number of points accrued by a given user (you receive points for commenting, voting, and suggesting) - for example, IdeaScale's parent company, QuestionPro, will give you a $10 Amazon gift certificate when you accrue 100 points on their own virtual suggestion box.

IdeaScale is best for individual programs or events because it focuses on prioritizing via voting. The suggestions have to be reasonably focused so that people can make comparative judgments. It may be useful if you want to ask "What kind of teen programs should our museum offer?" because all of the answers will be related and can be judged as better or worse than each other. IdeaScale is less useful for questions like, "What should we change about our museum?"--it may be hard to compare suggestions like, "new bathrooms," to "longer hours" to "more tours"--and therefore, the content becomes less useful.

Two interesting examples to check out: ChoiceHotels, a booking software used by hotel managers, and AsktheSpeaker, in which Ideascale was used by Netroots Nation to select questions to ask Nancy Pelosi in an interview. Both of these are specific; the first, about the feature set for a software service, and the second, focused on a single event.

Positives of Ideascale: Easy to customize the look and feel to brand to your site. Simple, understandable functionality. Focuses users on prioritizing ideas. No ads in any version.

Negatives of Ideascale: users must register an account to comment, suggest, or vote.

Best use for museums:
When you want people to share their suggestions for a specific element of the institution (i.e. exhibition name, what kinds of programs do you want, what should we offer in our cafeteria) and want to gather both ideas and votes for each idea. While IdeaScale could be used as a standalone kiosk or a link from the website, but probably is better for specific, targeted projects than as an entire comment card solution.

ongoing conversation with users about visitor experiences

GetSatisfaction is the youthful giant of this field. While IdeaScale is about sharing suggestions for particular programs or services, GetSatisfaction is a "customer service and support" system. Rather than just suggesting ideas, users can "ask a question," "share an idea," "report a problem," or "give praise" to the company or institution. These types are color-coded so a user can quickly scan down and see the problem reports (red), which they may want to respond to quickly. Users can also submit their emotional feeling about the idea or problem via a set of emoticons that let you know generally whether people are happy or pissed off. GetSatisfaction makes it very clear which users are employees and which are customers, and lets users know at the top of the page how many employees are engaged in the forum (so you know whether you are in an entirely customer-based discussion environment or one that has a lot of active participation by the company).

GetSatisfaction is more about conversations with customers than prioritizing suggestions. There are great secondary tools to allow you to follow individual conversation threads and users have profiles that can be developed across the site (similar to Yelp!). While you can vote on a given question or item to say that "you also have this question," that feature is not as frequently used as the "reply" function. Where IdeaScale is about sorting suggestions by priority, GetSatisfaction is about connecting with users and their concerns and questions. It is primarily used by web companies, but there are some media providers like the BBC and venues like Whole Foods using it.

Positives of GetSatisfaction: Creates an ongoing forum for communication with users. Can be used for multiple kinds of requests--content questions as well as concerns about the cleanliness of the bathroom. Great user interface; see this in-depth article about its design.

Negatives of GetSatisfaction:
Free version has ads. Users must register an account to comment, suggest, or vote. Requires ongoing feedback and use by staff to adequately address user concerns.

Best use for museums: If you want to have ongoing conversations with visitors about their questions and concerns, GetSatisfaction is a good option. This is like a comment card system in which you are expected to respond to most of the questions and concerns. It could be a robust complete system, but there is a heavy staff time investment required.

voting fairly for new ideas

uservoice is still in beta, and it got on my radar through their clever creation of a Obama agenda suggestion implementation (it does not appear to be affiliated with or sponsored by the Obama team). It is very similar to IdeaScale, focusing on making suggestions, voting, and commenting, with one unique difference: users are given a set number of votes (10) to distribute among the ideas listed. While suggesting a new idea is up front on the site, the fact that you only have ten votes to spread around adds a game-like element that focuses you on checking out many ideas and distributing your votes wisely.

For this reason, uservoice may be an interesting tool to use if you want people to vote for options in a controlled way where different users' contributions are balanced (i.e. voting for a favorite exhibit). The concept is that some of the ideas in the uservoice list will be adopted by the institution, and then the votes for that idea will be "freed" back to the voters for use on other ideas. That concept relies on users returning to the site multiple times--something no museum can really count on.

Positives of uservoice: Does not require registering an account to suggest an idea, comment, or vote. There is only a free version currently with no ads.

Negatives of uservoice: The vote cap may be confusing and or limiting to users.

Best use for museums: If you want to invite people to vote on topics and require them to really value their votes, uservoice could be a strong tool. I could imagine it being used, for example, in a climate change exhibition to invite visitors and staff to recommend energy-saving options for the institution and for visitors to vote on which they think the institution should prioritize.

One key requirement to make any of these systems successful is that you must place it prominently in your physical museum or on your website such that people can easily access it when they have their question or comment. That is more likely to happen in the museum than online. This could even be the start of a great "online extension" activity for visitors. Rather than dropping their comment cards into the black hole of a suggestion box, they could start conversations and engagement with the institution--originating with both positive and negative impressions--that continue for a long time. One of the most interesting things I noticed as I scanned the Whole Foods GetSatisfaction site was how many topics started negative and ended up becoming polite, engaging conversations between customers and employees. And while the tone of the Whole Foods employees is very marketing-ish, it's also personal, and it seems to work. Annoyed people are converted. They are spending more virtual time with the brand, and with real people associated with it. They are having conversations. And I think that's encouraging some of them to go back to the store.

Why let Whole Foods have all the fun? How do you use comment cards in your museum, and how would you like to see them evolve?

7 comments, add yours!:

aaron said...

This might be a bit of a stretch, but museums could present their digital comment cards as part of their greening too.

Jennifer - Bay Area Discovery Museum said...

Great post, as always! Lots of food for thought...

Also check out crowdsound - it doesn't require a login but acts like ideascale...free, no ads. We set up an account but haven't implemented it yet on our web site.

So at our children's museum, we don't have any computer kiosks onsite.
And I think it would be unlikely that a visitor who has a suggestion or complaint woudl remember to login to a web site when they got home in order to register their issue. We might get comments on our web site if we put up crowdsound, but they will be of a different nature than comments made on site.

We actually read our comment cards out loud to our quarterback team meeting every week (the team is a member of operations, exhibitions, education, and marketing plus any other staff who have an event coming up or to debrief on). We do take action on items that need it ASAP, and we respond to visitors who leave their contact on the card. It gives a good sense of what the main issues of the week are.

However, we haven't found a good way of tracking the data - that's what I want. Ideally comment cards could be entered into a database so that we could observe rating trends over time - for example, is our cafe improving? Are restrooms decreasing in satisfaction?

Any ideas on that front would be much appreciated!

Seb Chan said...

My feeling is that the key is to have an integrated onsite/online comment form setup. I like the Brooklyn's steps in this direction.

We are in the process of moving this sort of stuff to Jira (http://www.atlassian.com/software/jira/) which started life as a bug tracking/IT helpdesk tool but is far more flexible than that. We are also trying Uservoice but Jira completes the circle for us.

Jira will give us the necessary tracking over time and also help manage the bigger issue - the need to ensure that these audience conversations are responded to and engaged in. Jira, with its roots in helpdesk systems can keep 'reminding' staff when they have unanswered visitor feedback for example . . . and can also alert the visitor when their query/suggestion has been 'escalated' to another staff member.

Getting the feedback is the easy part. Responding to it and acting on it is the real challenge - especially with limited resourcing.

I'll report on F&N after we've got a significant amount of data back to report on the trial.

Neil -- Boston Children's Museum said...

Using the comments and responding to them is something that we are grappling with, like Jennifer. Our bi-weekly "clean, safe, fun" meeting reviews and acts on the visitor comments. We have been toying with how to give visitors feedback on their feedback. These tools were great ideas, although creating a way for that to happen in the museum is challenging to think about. I'd be interested in experiences with the logistics issues in a high volume, lots of distractions, environment.

jlevinso said...

Does anyone know of museums or libraries that have started using any of these tools? I would love to check out site where they are implemented - that fully demonstrate their use in the museum world.

Jon said...

A company called VideoBooth Systems manufacture automated video feedback kiosks and booths designed for museums and galleries, I've seen these in use at the Tate and the National Gallery in the UK. Nice turnkey and secure solution.

Oberon515 said...

Great post but the main issue we see with feedback tools in museums, galleries etc is that they (web, email or paper) provide too little data, usually without analysis and way too late to take action. Its also unlikely to segment out different types of guests or profit centers.
I think feedback can only have value when it is delivered at the point of the visit and can be actioned immediately. This means issues can be fixed, guests salvaged / placated, ideas implemented, hazards removed, exhibit ideas implemented etc etc.
Anything afterwards simply misses the boat, as the brand experience has happened. This could mean the difference between a visitor NOT coming back or a guest NOT becoming a member.
New technology allows this 'live' feedback via the cell phone where guests can choose to give feedback / ratings / ideas on any aspect of the visit when they are there - using SMS, Voice or Browser. English and Spanish. Any cell phone, and web enabled device any land line.

http://opiniator.com/ is one for example.

FULL DISCLOSURE - the above is my company. We provide this for retail, events, and healthcare. These industries have similar challenges.

I disagree with Seb. There seems to be no point in asking for feedback unless you will act on it?