Design Changes

Changes to the Design

The results of this informal test indicated significant changes were necessary to the TraveLite interface. As a consequence, we decided to significantly change both the interaction flow and business model of our site:

We now allow for people to experience TraveLite and create a guide as a 'guest'. This gives new users the opportunity to experiment with the site before making any commitment (without which, based on our testing, would cause the majority of users to leave the site).

We moved the 'create account' and 'purchase' stages to the end of the process, allowing users to create a guide with the option of saving it to return later, or downloading it directly (or abandoning it altogether). Furthermore, we also allow users to download their guide for a 48 hour free trial (this does require them to create an account).

We also decided to emphasize and focus on enabling this process for PDAs. Our users found this the most interesting and compelling element of the system: making a selection from a range of content in order to fit onto a PDA. Allowing downloads to other formats confused the tasks and made the entire system much more complex, resulting in a lack of focus in the design.

As an outgrowth of these changes, we were also able to simplify many of the pages and eliminate unnecessary functionality. Now the steps required are fewer: Select a Destination, Choose a method to Build a Guide, Edit the guide, and Download the guide (account required to download).

Updated Flow Diagram

Across the site, we looked at ways eliminate copy (text) and streamline the layout on all the pages, trying to achieve a cleaner interface. We also looked at ways to improve the navigation.

The Home page: We eliminated much of the text and highlighted the 'create a guide' aspect of the system. We also moved the sign in box up to the top header to separate it from the interaction in the main text box to further emphasize the ability to start making a guide, rather than creating an account.

	Previous Version:		New Version:

 

On the Create a TraveLite Guide page, we give visitors more information on the process, offering the opportunity to build a guide as a guest, while also allowing them to create an account if they wish. We also allow members another opportunity to sign in here if they missed the sign in box on the home page.

 

We then collapsed the two pages, "Destinations" and "Add/Remove Region", from the previous version to a single "Destinations" page, allowing the addition of finer-grained destinations to a guide.

	Previous Version:Destination		Previous: Add/Remove Regions		New Version: Destinations & Regions

Once destinations are added to a guide, users then move to building their guide. We simplified this page from the original, retaining the option to rename the guide but eliminating the necessity to choose a format and guide size at this stage. Now that we are focused entirely on creating guides for PDA formats, we decided to choose a default guide size for every user. Anyone who wishes to adjust this has the opportunity to do so by following the "change guide size" link on the edit page.

	Previous Version:		New Version:

We also changed the layout and options for the 'Build a Guide' method. On the layout we now present the options vertically instead of horizontally. From this section, each user will select one of two options, 'Build it for me' or 'Let me do it', that allow the user to determine how much they wish to personalize their guide. "Build it for me" takes them to a page where they can take a high level cut at the content across all the regions. "Let me do it" allows them to perform the detailed edit, section by section, region by region.

	Previous Version:		New Version:

From these pages, the user can then either preview their guide or download it to their PDA. When they download the guide, they are requested to create an account if they do not already have one. Then they can either download the guide and try it for 48 hours or purchase it outright.

On the Edit page, we also change the 'bookmark: edit later' link to read 'Save'.

Changes to the Experiment

As noted above, in future iterations of the experiment, we would eliminate the task of exploring and building a guide based on personal interest.

We would also consider using time to task completion for specific sub-tasks rather than broader guide-building tasks. We also would divide the tasks more discreetly and print them on individual sheets of paper handed to the user one at a time.

We plan to run another round of user testing, using these changes to the usability test design, to test these most recent changes to the interface indicated by the previous round of tests. The results of this second round of testing will inform the last design iteration of the interface.

 

Testing Results

Following is a summary of the results of the test organized by the page flow of the system. For more details, check the critical analysis log [HTML|Excel].

Home page: Several of our users tried to immediately sign in to the system without previously creating an account. When they arrived at the sign-in error page, they were still not sure what to do and again tried to sign in. One participant was unclear whether he needed an account to sign in to TraveLite. Most participants also found the Home page confusing and cluttered. Most did not want to read the text but rather wanted to start experimenting with the site immediately.

Add Destinations: Most users were confused by the Destinations page because, according to the task scenario, they wanted to build a guide for San Francisco but the only option they were offered was "United States: California & Nevada". They could not figure out how to drill down to the city. When they did select this destination, they were extremely concerned about only having an option to purchase (see under purchase below). Most wanted to see a deeper hierarchy so that they could know what regions were included in the guide. Many of the users wanted to double click the name of the destination to move it to the selected box.

Purchase: Most users were extremely disconcerted to not have an opportunity to play with the system before being confronted by a purchase page. This was further compounded by the need to purchase California when they only wanted San Francisco. In fact, all participants said that at that point they would normally discontinue their interaction with the site. Most wanted an opportunity to play before they pay. At this juncture they had not seen a compelling reason to pay. One user suggested presenting "Make a Guide" as the primary link rather than "Create an Account," and allowing the user to interact with the system and experience it as a guest before being required to sign up, which would permit the user preview the process and content before purchasing.

Add/Remove Regions: One user wanted to know if they were permanently removing a region by deselecting it or if they would have an opportunity to return to this page at a later date.

Guide Size page: Most users felt that this page was too complicated and required too much effort to comprehend it.

Step 2, Select Download Format: One user did not expect to have to specify a format at this stage. He also wanted to see the actual device names alongside the operating system options/document formats presented among the format selection options. He was also confused about whether the selection applied to the content or to this guide. Because he had subscribed to the content, he wanted reassurance that he could go back and change this if he wanted to create a new guide.

Step 3, Select customization method: Only one user actually read through the three options for this step, however he was not sure why he would care about the options. Most users just stopped at the first step and wanted to act on the radio buttons and then go. Even when they read through the options, they were not sure what the other options offered, 'Build it for me' was the only one that was direct and clear. One user suggested that we rename these 'In a rush', 'Picky', and 'Really picky'.

Edit page: Overall, the users really liked this page. They appreciated the functionality of allowing them to add and remove content and thought that it was nicely implemented.

Guide size feedback: Most found the guide size feedback very helpful in knowing how much content they had and/or could fit into their guide.

Tab checkbox (to include/exclude content section): Not everyone was clear about the functionality of this checkbox. One person understood that it would remove the section from his guide but was concerned that, as he had already edited the section, this would remove any work he had done if he decided to add it back in? His focus was on adjusting the content to fit into his device. He would have preferred a "select all|deselect all" option for the content rather than acting on the tab section itself. Two of the users also tried to remove a section while in the section, creating some confusion when the tab would not "deselect".

Two users explained that they would probably simply unclick all the checkboxes to remove all the content (and thereby the section) from their guide, although one of these testers did not notice the "include" checkboxes until they were pointed out to him in the discussion afterwards. One user suggested that we change the label next to the tab checkbox from "Include" to "Include Category".

Content categories: One user felt that the Activities, Entertainment and Attraction categories were very similar and partially redundant. Specifically on the contents of the tabs, one user recommended reordering content in The Essentials tab to include "Getting There and Away" information first.

Bookmark: No one understood 'Bookmark-edit for later'. In fact, most users were looking for a Save button. They either did not understand the notion of 'Bookmark' or felt that the metaphor did not quite fit the action. One user felt that this step was actually unnecessary and that he would simply assume that the guide was saved when he left the site. Furthermore, this collection of links was not very evident to most users.

Content: Several users wanted to be able to access fuller descriptions of individual content items by clicking directly on the name of the business and in fact tried to do this. One person felt that it was almost too customizable. He was not convinced that he would want to go through this amount of effort. Seeing the results did not really help, he would have preferred more interesting, subjective metadata such as 'romantic restaurants' or 'traveler recommended' rather than budget, cuisine and neighborhood for the purposes of selection for his guide.

Logout: Two users had a difficult time finding the logout button. One participant just closed the browser window.

Preview page: Most users liked that we actually provided the content on this page. All the users felt that the page needed to have better navigation and most wanted to know how the format on this page related to the format that would appear on the PDA. One user felt that having two scroll bars (one on the inset form, one for the browser window) was very confusing. Two users were not entirely sure that they would actually want to read through the content in this page. One indicated that seeing the listings on the edit page was enough of a preview. The other user recommended showing a graphical interface showing the relative percentage of content each section represented in the guide.

General

Design: Overall we need to improve the visual page-level design of the site. For example, on the destinations page, one user noted that the Selected Destination box was 'floating over there in space.' We also need to reduce the amount of text used (especially on the home page) and make the language more interesting and lively. One user also found the navigation particularly confusing but felt that an improved graphical design would make functional elements easier to locate.

Business Model: The most critical problem with our site is requesting users to pay too soon. Overall our participants did not find the idea of subscribing to a large chunk of information compelling, especially if they were forced to purchase California when they only wanted San Francisco. Most found the idea of taking a large collection of data and compacting it down for a PDA as the most compelling and interesting opportunity on the site. Furthermore, users wanted to get a taste for both the functionality of the site, the process required to create a guide and the quality and quantity of the content before paying for anything. They also wanted to know what the final portable version would be like - format, usability, etc. - although they would be comfortable simply downloading the trial version for this information.