After dramatically expanding this enormous government Web site, we decided to tackle the top-level information architecture—particularly the first two levels of the Web site. (The expansion hadn’t touched this.) I had heard about the idea of card sorting, so arranged a card sort with internal staff and external users, focusing on the content directly available from the top two levels of the site. I basically jumped in with both feet and had a go at it. It worked out really well, and we got some great ideas on how to regroup the content at that level.
Then, I changed organizations to lead an intranet redesign. With my new-found experience in card sorting, I decided to do a card sort on the entire intranet. Boy, oh boy, did I learn a lot about card sorting and about IA during that project. Everything went wrong, which was a much better learning experience than when everything goes right. Oh, I still shudder at the memory of that project.
SB: I’m sure I’m not the only one who wants to know: What went so wrong in that card sort?
DS: This particular intranet was quite big. I first made the mistake of trying to cover the whole intranet in one card sort, with the result that, when users went to sort the cards, they really looked at them and said, “But none of this really goes together. There is too much here.” Of course, they tried, which meant that my results weren’t very good quality. Then, I paid too much attention to what people had called things in the card sort and used those for labels. I remember a point when others on my team gently pointed out that Computing wasn’t a very good navigation label, and I said, “But that’s what the users called it.”
Underlying all of that was a bigger problem—that I was trying to get a technique to do the thinking for me, instead of using it to help me think.
SB: Card sorting seems to be one of those techniques that is very simple in theory, but complex in practice.
DS: The underlying idea is really, really simple: Write content ideas on index cards and get your users to sort them in ways that make sense to them. The hard things are:
- What do you actually want and expect to learn from doing this sort of activity?
- How are you going to pull insights from data?
- What are you going to do with the data?
These are things that most people forget to think hard about. Lots of people say, Hey, we should do a card sort, without really figuring out what they want to learn from it. To do it well, you need to think about what you want to learn and make sure you construct the activity to actually learn that.
Then, you need to collect the right amount and type of data to meet your research goal. There’s little point involving 300 people in a remote card sort when you would get better insight from a face-to-face sort of 10 groups of 3. All you have is a big, fat analysis problem.
SB: What do you do with information after you’ve collected it?
DS: The main thing to do is to look for interesting patterns. To spot things that people did that you didn’t expect—or that confirm what you did expect. You can spot these patterns by just printing out all the results in a big spreadsheet and sticking them on the wall or by crunching numbers with a statistical program. But, however you do it, you are looking for things that tickle your brain cells and help you think through stuff that will help your project.
The thing you don’t do is take the raw data, stick it into a dendrogram—a slightly scary diagram that comes out of a statistical technique called hierarchical cluster analysis—and implement that as your hierarchy on your Web site. That won’t help you at all.
