SIMS 202 Fall '99 Assignment 5

SIMS 202 Information Organization and Retrieval

Assignment 5

Assigned 9/21. Due 9/30.

Readings: Sano, Hauser & Clausing, Rosenfeld & Morville

The goal of this assignment is to give you exposure to the process of participatory design of an information organization.

For this assignment, work with two other students (we'll set this up in class). This is going to require a bit of scheduling, so we've allocated some extra time, but you need to plan ahead.

First read the following excerpt from: SunWeb: User Interface Design for Sun Microsystem's Internal Web Jakob Nielsen and Darrell Sano, in the Electronic Proceedings of the Second World Wide Web Conference '94: Mosaic and the Web, 1994.

Card sorting is a common usability technique that is often used to discover users' mental model of an information space. A typical application is to get ideas for menu structures by asking users to sort cards with the command names: commands that get sorted together often should probably be in the same menu. A downside of the method is that users may not always have optimal models, and card sorting (or other similarity measures) is often used to assess the difference between the way novice and expert users understand a system.

Before our card sorting study, the SunWeb development group had brainstormed about possible information services to be provided over the system, resulting in a list of 51 types of information. We wrote the name of each of these services on a 3-by-4 inch notecard. In a few cases we also wrote a one-line explanation on the card.

Before each user entered the lab, the cards were scattered around the desk in random order. The users were asked to sit down at a table and sort the cards into piles according to similarity. Users were encouraged not to produce too small or too large piles but they were not requested to place any specific number of cards in each pile. After a user had sorted the cards into piles of similar cards, the user was asked to group the piles into larger groups that seemed to belong together and to invent a name for each group. These names were written on Post-It notes and placed on the table next to the groups. The users typically completed the entire process in about 30 minutes, though some took about 40 minutes.

The data from this study was lists of named groups of cards with subgroups corresponding to the original piles. Based on this information, it is possible to calculate similarity ratings for the various concepts by giving a pair of concepts one similarity point for each time both concepts occur in the same larger cluster and two points for each time they occur in the same pile. This similarity matrix can then be fed to one of the many standard statistics packages for a cluster analysis. It is also possible to use other statistical techniques such as multidimensional scaling.

Next, with your partners, choose a topic. This should be a fairly rich set of information that is to be used to organize information on a web site. Don't make it too complicated, however. Choose something that can be characterized by about 50 concepts as described above. You also need to specify what type of person is intended to use the information, and what the information is to be used for, in order to aid in the information organization part. One idea for topics might come from information centric web sites such as government sites. (Examples from Yahoo)

After choosing the topic, do the brainstorming step in which you generate a list of categories as described in the excerpt above and the Sano chapter (see pages 188-193 of the reader). You should have around 45 or 50 concepts. [This part should take about 2 hours.]

Then meet with another group. Each person in group A should individually organize the cards of group B, and vice versa (without looking at what the others are doing). You don't have to all be physically present at the same time -- just do the exchange in some way. You can give the other groups whatever instructions you feel will be helpful, although you may want to use the methodology described in the excerpt above ("not too few, and not too many" categories). [This should take 40 minutes to an hour per person.]

Finally, get back together with your original partners and make a final classification system based on the results of all three of the sorts. You don't have to use a clustering algorithm or other fancy statistical technique; instead try to eyeball where the best similarities are. You can also exercise your judgement and override or discard some of the results. Create an organization like that in Table 3.1 of the Sano chapter or a set of diagrams like those of figures 3.12-3.16 (if you have enough detail about what the site is to be about). You have some flexibility in what you end up making this information organization look like; just be sure you explain and justify it well. [This should take about 2 hours.]

Turn in the following (one per group): [Be sure to record information as you go so the writeup goes faster.]

Names of students in the group.
Topic and intended users and use of the topic.
List of original types of information.
Illustrations showing how these were grouped together by the members of the other group (these can be sketches or formal illustrations).
Final classification, its structure, and the justification for it.