Thursday, July 02, 2009

Designing Recommendation Systems that Go Beyond "You'll Like This"

How would you design a recommendation system for a museum? Recommendation systems are tools that offer suggestions, most commonly in the "if you like that, you'll love this!" format. We've all become familiar with online retailers who address you by name and offer suggestions--some helpful, some annoying--based on your past activity and purchases.

When it comes to museums, recommendation systems are a natural solution for the problem of the customized tour. How can a museum offer each visitor suggestions for exhibits and experiences that will uniquely serve their interests? There are many lovely example of museums providing quirky tours based on particular interests. For example, The Tate Modern offers a set of pamphlets featuring different tours of the museum based on emotional mood. You can pick up the "I've just split up" tour and wallow in depression, or the "I'm an animal freak" tour and explore your wilder side. And the site I Like Museums lets you find whole institutions of interest based on your preference for trails like "making things," "nice cup of tea," or simply "pigs."

But what if you want to provide a truly emergent recommendation system, like the one used to recommend new songs to you on Pandora or new movies on Netflix? These systems use forms of collaborative filtering to analyze what you've liked and find things that might be similar based on both expert and user data. In this way, you could imagine a visitor moving through the museum, starting by expressing her love of optics, then discovering via an enjoyable exhibit that she also is into magnets, and so on.

There are two problems you have to address to create a great museum recommendation system.

Problem #1: Getting the Data

The first challenge is technical--the lack of explicit data. Recommendation systems use a combination of explicit and implicit information to provide you with suggestions. You make explicit designations by making purchases, expressing preference via ratings or reviews, or choosing some things over others. But you are also always generating implicit data passively via the things you click on, items you spend a long time looking at or listening to, and the choices your friends are making. In the physical space of a museum, visitors make very few explicit data contributions. You may buy a ticket to a special exhibition or show, or actively elect to take up an audio guide or exhibition brochure. But most of your preferences for one museum experience over another go unregistered and untracked. This means there's very little data on which museums can automatically offer recommendations for further experiences.

If we really want the explicit data, there are ways to encourage visitors to provide it. Consider the case of Netflix, the dominant US online movie rental company. Netflix makes movie recommendations based on your ratings of films you've watched. There is no reason in the life cycle of movie rental that a user should be expected to rate a movie. Pre-Netflix, there was never a history of people giving something "four stars" when they dropped it in the return slot. But Netflix realized that their ability to sell subscriptions was directly related to their ability to provide users with a steady stream of good movie recommendations, so they invested heavily in creating a rating system that is fun and easy to use.

Rating content from one to five stars may seem like a frivolous activity, but for Netflix, it's serious business. Netflix knows that good recommendations are key to their bottom line. If Netflix suggests too many movies that you don't like, you will either start ignoring the recommendation system or cancel your subscription altogether. The underlying message of the recommendation system is that there is always a movie you'll love on Netflix, so you should never stop subscribing.

This implicit promise is also the key to why people willingly rate hundreds of movies on Netflix. Netflix promises to give you better recommendations if you rate more movies. Your user profile is functionally an aggregate of the movies you have rated, and the more finely tuned the profile, the more useful the recommendations. The more you use it, the better it gets--and that symbiotic relationship serves customer and vendor alike. This promise is what is missing from so many museum rating systems. When museums allow visitors to rate objects or express preferences, the visitors' expressions are rarely, if ever, fed back into a system that improves the museum experience. The presumption on the part of museums is that rating things is a fun activity onto itself and that's why people use them on Netflix and other sites. But they aren't just fun ways to express yourself. They have direct personal impact. Whether you are panning a movie or gushing over a book, your explicit action is tracked and used to provide you with better subsequent experiences.

Problem #2: Designing the Value System

But what's "better" in the museum context? One of the biggest concerns about deploying recommendation systems in museums is that visitors will only be exposed to the narrow window of things they like and will not have "off path" experiences that are surprising, uncomfortable, and valuable.

Fortunately, not everyone is in the business of selling movie rental subscriptions (or woks, or books, or whatever). While online retail recommendation engines are unsurprisingly optimized to present you with things you will like, there are other ways to filter information based on preference.

For example, Librarything, a social network for sharing books, has a "books you'll hate" feature called the Unsuggester. Type in How Children Fail by John Holt, and you'll find its antithesis: Digital Fortress by Dan Brown. This is an undoubtably silly exercise.

When the BookSuggester was released in November of 2006, programmer Tim Spaulding wrote a blog post about the addition of the Unsuggester. After noting the patterns of opposition between philosophy and chick lit, programming manuals and literature, Tim writes:

"These disconnects sadden me. Of course readers have tastes, and nearly everyone has books they'd never read. But, as serious readers, books make our world. A shared book is a sort of shared space between two people. As far as I'm concerned, the more of these the better. So, in the spirit of unity and understanding, why not enter your favorite book, then read its opposite?"

The Unsuggester is based on different values than Netflix's Movies You'll Love and the BookSuggester. It's also based on different data. Whereas Netflix bases its recommendations on ratings, Librarything bases its recommendations on the books you have in your library (read why here). Instead of saying, "if you like this, you'll like that," Librarything says, "if you have this, you'll like that."

This may sound like a trivial difference, but it leads to a real value shift when it comes to the Unsuggester. The Unsuggester doesn't give you books you'll hate; it gives you books that you'd never otherwise encounter. The format is "if you have read this, you are unlikely to read this." The value system for the Unsuggester is based on the idea that we can learn something from things that are foreign to our experience. The books on the list are the ones that are least likely to be found in your Librarything collection or the collections of other users who also have your books. It's a window into a distant and somewhat unknowable world... not unlike the world of wild and disparate artifacts that curators would like to reveal to visitors.

And users have responded positively. When Tim suggested that few people were likely to actually read books on the Unsuggester list, an anonymous user responded,

"You underestimate Thingamabrarians. Some of us are just looking for new ways to branch out from our old ruts... and something flagged as 'opposite' to our normal reading might just be what we're all looking for. (Besides, a lot of the 'niche' books are throwing up classics in the unowned lists, and many people like to improve their lit-cred.)"

In other words, recommendation systems don't have to be optimized to give you something you like. They just have to be responsive to your personal inputs in some understandable and meaningful way.

The Unsuggester is based on the value of finding enjoyment in highly incongruous things. What other values might we want to base recommendation systems on, in museums or otherwise?

8 comments, add yours!:

Bruce Falk said...

Great post. I would think that relational database-building efforts like those described in Project Steve and on display by the Powerhouse Museum in Australia (among other folksonomy/tag cloud implementers) go a long way toward facilitating this kind of scheme.

Have you looked into formal semantic web ontology building? If so, what conclusions do you draw?

caleb tr said...

Nina I love this, as I too am interested in recommendation systems that do more than sell things. In my case, I'm interested in how a library can recommend a book to help someone be free. I've read your pieces where you talk about museums giving a free (and free) education.

Collecting the data could be fun - so many exhibits have audio tours, you could track which pieces people click to find out more about. This data gets better with traveling exhibits. You could also engineer an RFID grid on the floor of an installation, hand out tags for folks' shoes, and track where people walk and how long they stay in each place.

Then you can unsuggest the things not being looked at, highlight the ones that are, or identify patterns that match a visitor's behavior in order to suggest a different course (aren't you bored of looking at glow-in-the-dark fish? don't you want to see the turtles?)

I know you're all about getting people to interact at the museum, but as a visitor, I often want the place to myself. A recommendation system could simply send you to the currently-less-trafficked areas. As a museum member, I want to see old favorites but also what is new or recently hauled out of a crate. Even without tracking my behavior, museums can give me a tour of what I missed since my last visit.

Ultimately, you can always let visitors decide if they want to see 'more like this' or 'something different'. Fun stuff!

Lynda Kelly said...

Hi Nina. We've been thinking about this in terms of the technology that visitors walk into the museum with. Imagine if somehow you could get wirless access to their Facebook, Twitter, LinkedIn profiles (say by swiping in as they enter or something). These profiles contain often detailed information about friends, family, collegues, professional and personal interests, cites they've visited, things they've been doing recently, education and location data. The advantage is the information is what the person has decided to upload therefore it really is identity-based. Imagine then somehow based on this a series of recommendations pops up on their mobile device about exhibitions, programs, objects, tour routes etc on offer at the museum.

We're certainly not there yet but it's something we're investigating here with folks at Sydney University - we'll keep you posted (that is if we get the funding to do some pilot-testing).

Nina Simon said...

I'm really curious about this from the library angle. I use the library all the time, and it maintains a digital record of the books I check out (it must, to tell me when I'm late). And yet I can't access that list, export it to Librarything, or get any interesting recommendations because of it. I know libraries are very conscious about privacy issues, but is that the only reason I can't access my own checked-out books (behind a password and very long account number)?

Nina Simon said...

I'm very interested to hear more about this. This post was adapted from the section of my book that I'm working on now--a section on museums and personal profiles.

I've been thinking and writing about it from the starting point of encouraging museums to create custom, very simple profiles related to the experience--but it would be very interesting to glean that content from other sources where available. I'm curious what content from Facebook etc. you think will be really valuable as the basis for a personalized museum experience.

Marcus Kirsch said...

Yeah, the old 'I'm feeling lucky' experience.
There are a few issues that I came across having had to develop an alternative music search tool some 9 years ago. The interesting hurdle there was, that the german search engine company based their data on tagging each music title with over a hundred tags having specialists working day and night to achieve adding tags from "twoStep" to "autumn" to music in order to classify them. This was obviously years before user created content and voting was properly recognized as a pattern builder.
What noone over there had realized, was that language and it's abstract words are very inadequate to describe music, if you want to compare it. Take the case that fast music might not be fast for everyone and terms like Jazz don't necessarily have clear boundaries. So I recommended to completely hide all the tags when searching for music. Instead, I was using the songs themselves to search for new music. This lead to a search that was more based on someone going to a record store and singing to the clerk as sample of the music they might like to find. E.g. find me something that sounds like Beastie Boys, but with the instrumentation of Carl Orff's Carmina Burana. The Mendel-like search(mixing the underlying tags like DNA pieces) lead to an interesting and playful way to explore music by using your own perception as a guide/starting point. So on the surface, it did, what Pandora does in a very simplistic way.
So there are two things: One is that the presentation of the system is very important towards the users perception of how accurate the result is. Language is a big part of it. How can I push the complex aspect of generating an experience that a user can agree with and how can I best hide the complex underlying structure that might alienate the user.
Second, and that is very nicely put in the blog, how do I avoid long tailing my collection? It has been proven that elements like 'your top searches' and 'what others liked' are fairly distorting the choice on offer by peaking the first entries and shifting everything else way down the tail, making it often invisible during the process. This is, in the worst case scenario culturally limiting to the extend of hindering innovation.

Oh it's such a fragile thing that.

Gio said...

Great post, big problem. We've been working on this problem for a few years for the University of Arizona Science Center and made some progress in our research. The main problem, we found, is not just "getting the data", but getting the good ones. Since we did not have a Science Center yet (maybe we'll never have one, thanks to the recent Arizona State budget cuts) we experimented with the Web and created Zonebee as a complementary informal learning environment for the Web. With Zonebee, which is embodied in a browser Toolbar, we can collect data that, according to our research, might be "good ones": they define what we call the "organizing circumstance" of users learning around dimensions like the type of learning goal, the user expertise, or the access to strategic assistance. This information can be derived mostly implicitly from users' behavior, as long as we understand the meaning of their actions. For example, we need to know whether a user is "exploring" or "searching" or "focusing on" something. This applies to Web actions as well as a museum visit. Finally, we believe we can use a combination of clustering and collaborative filtering techniques to identify, based on the data, "communities of practice", which we can be used to make recommendations. Can all this research be transferred from the Web to a Museum? Absolutely! In order to allow this to happen, however, a museum experience must be designed and "instrumented" appropriately to collect "good" user behavior data. To make this possible we partnered with people and created an open-source "toolkit" that can be used to instrument museum exhibits economically and pervasively. I still hope we'll be able to get funded and go beyond our proof of concepts!

jason webber said...

A really interesting and well set out post and something i've thought about for years but never really found a neat solution to.

I've been tempted in the past to have a 'rate this object' or a 'was this interesting' button on each object but as mentioned this would only ensure that less things are viewed in the long run.

I run a website that has thousands of objects and it's a huge challenge to get the right people to the stuff they are interested in. I'd love to hear from someone in the museum world who has had a go at cracking this.