The following is the full report of the two user studies completed by Lauryn Smith du Toit for the Basket Lace team on behalf of the International Organization of Lace (IOLI).
[3] Task Outline and Other Details
In preparation for this user study, the Basket Lace team pinpointed the important themes of the IOLI website. Each team member then formed five unique tasks pertaining to these themes to include in our respective studies. In addition, the two tasks italicized below were developed as a team and included in each user study that was completed. These two tasks regard the most prominent themes of the IOLI site, themes that we thus desire the most data for. Each team member completed two studies with seven tasks. One study was one exclusively on desktop, the other was exclusively on mobile. The wording presented here is the actual wording used during the user studies described below.
Find the date by which you much state your intent to submit materials for the Proficiency Program.
Find the name of the person you should contact if you want to donate lace or other materials.
Navigate to the website or Facebook page for your local charter chapter.
Find the purpose of the Teacher Development Program.
Find out how long you may keep borrowed library materials.
Find the pattern for bobbin lace spider earrings.
You want to borrow the video “Milanese Lace" by Louise Colgan from 2004 to show to a group of students, but you need to ensure that the length of the video fits your allotted time. Find the duration of this video.
The User Actions, User Non-Actions, Potential Reasons, Interesting Findings section of the report often refers to the IOLI home page. For the sake of space, a screenshot of IOLI’s full home page is shown below.
Please note that hyperlinks are generally included on the first reference of a particular page only.
[4-6, 8] User Actions, User Non-Actions, Potential Reasons, Interesting Findings
1. Find the date by which you much state your intent to submit materials for the Proficiency Program.
User actions:
User_1 moved the mouse around the home page, reading the list of the available links to get her bearings, until she located the Proficiency Program link in the right sidebar. Once on the Proficiency Program page, she skimmed the page and then selected the Proficiency Program General Information link. On the Proficiency Program General Information document page, she scrolled through about half of the document, which she called a “glossary.” She said that she was looking for date to jump out at her. She continued to scroll and skim the document. When she found a date (January 1st) in the document, she was confusion with the “vague” wording for the deadline and thought that maybe she was looking at the date for something else. The sentence in question was, “The deadline for notification is January 1st prior to the Convention where your assessment will be done.” She said, “I don’t understand that language.” She decided to go back to the Proficiency Program web page, where she paused and then clicked the Proficiency Program Registration Formlink. She stayed on the Proficiency Program Registration Form document page briefly, scrolling up and down twice in search of a date, to no avail. She again went back to the Proficiency Program web page, where she quickly selected the IOLI Proficiency Program for Lacemaking link. Again, she spent little time on the resulting page, scrolling up and down once to look for a date, finding nothing. She went back once more to the Proficiency Program page and then to the Proficiency Program General Information document page. She found the January 1st date again and said that she was not sure if that was the date for the submission of materials, or the date to submit intent. In giving January 1st as her answer, she re-read the sentence out loud, reiterated its confusing syntax, laughed, and said, “I feel kind of stupid.”
User non-actions:
She did not read the Proficiency Program General Information document in full.
Potential reasons:
The Proficiency Program General Information document is quite wordy, so it was likely skimmed for expediency, which ended up hindering the accomplishment of this task.
Interesting findings:
The Proficiency Program General Information document was likely deemed a glossary because of its first page, which consists of a list of definitions; this seems to be distracting, confusing the document’s true purpose.
2. Find the name of the person you should contact if you want to donate lace or other materials
User actions:
The user went back to the home page. From there, she navigated to the Contact Us page. On realizing that no names, “just the PO box,” were listed on the page, she returned to the home page. From there, she selected the Donation Policy link. She skimmed the page and quickly found that you must contact “the President.” She did more scrolling, back to the top, saying that the page does not specifically name who she must contact. She returned to the home page, where she selected the Officers link. Here, she quickly picked out “Loretta” as the President, but then backtracked when she saw that this name and title were listed under Board Members (see below).
She reviewed the page quickly and found no other mention of “President,” saying, “I guess I contact Loretta.”
User non-actions:
The user did not look around much before selecting Contact Us, despite the Donation Policy link having the same amount of prominence.
Potential reasons:
Contact Us is a reasonable first choice. I wonder, though, if the phrasing of this task, which used the word “contact,” unintentionally introduced a subconscious thought to the participant, leading her to the Contact Us page first.
Interesting findings:
The user stayed on the Contact Us page longer than expected given the small amount of content located there. Also, given the multistep nature of this task, the user was able to complete it relatively easily.
3. Navigate to the website or Facebook page for your local charter chapter
User actions:
The user immediately went to the home page to the navigate to the Charter Chapters page. On the Charter Chapters page, she scrolled down the page to review the list of regions and their relevant charter chapters. She then scrolled back up to the map at the top of the page and clicked the Northern USA region. She proceeded to skim the resulting list of charter chapters for ones that included “IL” in their name. She mentioned that she saw the Wisconsin charter chapter first, but she then clicked the link for the first Illinois charter chapter link that she noticed (the Lace Collectors and Exchange link). After visiting that site, she returned to the Northern USA list and stated that it was hard to determine which charter chapter was right for her, particularly because not all charter chapters had links (see below).
She navigated then to the Lighthouse Lacers page. Once there, she commented that she thought that there should be some consistency and regulation for the individual charter chapter websites. She again returned to the Northern USA list. She settled on the Lace Collectors and Exchange of Illinois as her local charter chapter and returned to its website.
User non-actions:
The user bypassed the interactive map at the top of the page. Potential reasons:
It is not immediately evident that the map is interactive, as you must hover over it to get the pointer to change. For finding a charter chapter for a specific locality, there is definite confusion, presumably due to geographic considerations, like the border of Wisconsin being closer to Chicago than southern Illinois. Interesting findings:
With this task, the user began to demonstrate familiarity with the organization of the home page. In addition, it was surprising that she spent a good amount of time on the Lace Collectors and Exchange landing page, mentioning how confusing it was and that she would have no hint as to how to get involved at this location. She said that she did not care for their site much due to difficulty in gathering and understanding what was being presented on the home page.
4. Find the purpose of the Teacher Development Program
User actions:
The user returned to the home page. She immediately selected the Teacher Info link. Once there, she quickly found and recited the purpose of the program.
User non-actions:
She did not spend any time browsing the home page. Potential reasons:
More time on the site is increasing her familiarity with the location of particular links on the home page.
Interesting findings:
The task was very easy for the user, even though the name of the link did not include “Teacher Development Program,” the topic of the task.
5. Find out how long you may keep borrowed library materials.
User actions:
The user returned to the home page. From there she selected the Library link. She slowly scrolled down the page. She offhandedly noted aloud how many library materials can be requested at a time, indicating that she was looking for numbers to stand out to her. Soon after, she found on the page that it says that you may keep items for three weeks.
User non-actions:
She did not spend any time browsing the home page.
Potential reasons:
Further increase in her familiarity with the location of particular links on the home page.
Interesting findings:
The user demonstrated a strong reaction to writing style and focused up on poor wording. She stumbled with the syntax of the sentence that reads, “Library materials may be borrowed for three (3) weeks, beginning on the day you receive your request.” She said that “receiving your request” must be a typo and should probably be “your requested item.”
6. Find the pattern for bobbin lace spider earrings.
User actions:
The user returned to the home page and navigated to the Member’s Galleriespage. From there, she quickly selected the Members Patterns link. Once there, she commented with a touch of frustration that the page only shows pictures with no descriptions (see below).
She clicked on one of the images and learned that doing so brings you to the pattern’s explanation. She said that since she did not know what the pattern looks like, she would have to click on all of the images until she found the one that she was looking for, which is what she preceded to do. She was not able to locate the pattern on this page, despite following the links associated with each image. She grew a bit annoyed, wondering aloud where else the pattern would be located if not on this page. She returned to the Member’s Galleries page and laughed when she saw another link called Earring Patterns above the Members Patterns link. She said that her mind went straight to what members contribute, rather than earrings, specifically. On loading the Earring Patterns page, she said, “Oh my goodness…” (see below)
She clicked an image that she said resembled a spider-like pattern. She claimed that this first selection was the pattern asked for. However, the pattern selected was for “Bobbin Lace Earrings.” She self-corrected when she read the name of the pattern aloud, realizing that “spider” was missing from its name. She returned to the main Earring Patterns page and preceded to click though all of the images until she found the one that was labeled “Bobbin Lace Spider Earrings.” She laughed again and said, “Thankfully there’s only nine [images to click through].”
User non-actions:
She did not initially read each of the links on the Member’s Galleries page before selecting a link. Potential reasons:
She is a high-level user and was moving quickly. She followed the first link where she saw “pattern” included in its name, filling in the gaps that were created by skimming the page quickly, in a way that was meant to meet her immediate need.
Interesting findings:
This task took the longest to complete. Also, the Earring Patterns page includes the directive, “Click on image to get the pattern,” unlike on the Member’s Patterns page, despite the same action having the same result, leaving users to intuit this functionality on the Member’s Patterns page.
7. You want to borrow the video “Milanese Lace" by Louise Colgan from 2004 to show to a group of students, but you need to ensure that the length of the video fits your allotted time. Find the duration of this video.
User actions:
The user returned to the home page and then navigated to the Member’s Galleries page. She did not find what she was looking for and returned to the home page. She then navigated back to the Library page, saying that she was looking for a catalog. Not finding one on the Library page, she again returned to the home page. Not finding a catalog link on the home page, she went back to the Library page. After a moment, she said, “Oh, duh,” and clicked on the Videos link under the Catalogue sidebar. (This all took place in a matter of seconds.) On the resulting document page, she scrolled down until she saw the Milanese Lace title, saying that it is 3 hours and 30 minutes long. User non-actions:
She did not use a keyboard search function.
Potential reasons:
The user may not have been aware of the ability to use a keyboard shortcut to locate the desired title. Or, it may not have crossed her mind, as it did not take her long to find the title in the list, which is organized alphabetically.
Interesting findings:
She did not notice that this video comes in two formats, citing only number 22 under the VHS section. There is also a DVD, labeled number 15 under the DVD section. See below.
[4-6, 8] User Actions, User Non-Actions, Potential Reasons, Interesting Findings
1. Find the date by which you much state your intent to submit materials for the Proficiency Program.
User actions:
User_2 began on the home page. The first thing he did was enlarge the font on his screen. He read through the list of available links in both sidebars. He was unsure which link to click, saying that none of them tell him anything about “submission.” After a bit more time, he reiterated that nowhere does it say something about submitting anything. He followed the Calendar link because “it is the only thing here that has to do with dates.” He scrolled through a few pages of the calendar before returning to the home page. (Here he commented, “Wow, not even a back button.”) He reread all of the link names, this time aloud, until he found the Proficiency Program link. On this page, he clicked the Proficiency Registration Program Form link. He tried to zoom in on the form but was unable to access this function. He skimmed the Proficiency Registration Program Form document page and found nothing. He returned to the Proficiency Program page and selected the Proficiency Program General Information. He was forced to download the document, to which he expressed annoyance. He was able to zoom in on this document. He scrolled and skimmed for a few moments, stating that he wished he could use a search function. He did not find what he was looking for and returned to the Proficiency Program page. At this point, he said, “I literally don’t know.” He returned once more to the Proficiency Program General Information document and read it more closely, until he found the date he was looking for — January 1st.
User non-actions:
He did not read the content word-for-word until he had no other choice.
Potential reasons:
Perhaps given the site’s simplicity, something with perceived importance, like an opportunity where users are active participants and where submissions are involved, is expected to be more prominent.
Interesting findings:
Both users struggled with this task. This task took User_2 the longest to complete. Also, he commented that there is no back button, where there is actually one present at the very bottom of the page.
2. Find the name of the person you should contact if you want to donate lace or other materials.
User actions:
The user returned to the home page, then clicked the Contact Us link. He soon said, “Never mind,” and went back to the home page. From there, he selected Donation Policy, where he enlarged the text and began reading, from the top. He found that the President is the one to contact. He said that he remembered seeing a link for Officers on the home page. He navigated back to the home page, then clicked on Officers. He quickly gave the name of Loretta Holzberger as the person to contact.
User non-actions:
This time he did not skim the page.
Potential reasons:
He likely learned from the previous task that skimming may not be appropriate on this site, even if he did not necessarily consider it his ideal method of garnering the required information.
Interesting findings:
For this task, both users’ first instinct was to navigate to the Contact Us page. This may be due to the wording of the task. Perhaps the results would be different if another word for “contact” was used instead.
3. Navigate to the website or Facebook page for your local charter chapter.
User actions:
The user started back at the home page. He then clicked Charter Chapters, and on that page began to scroll down to get his bearings, reviewing the list of regions and associated charter chapters. He stopped scrolling when he found the section that matched his region (Southwest USA). He spent more time reading within this section, saying that he had “no idea which one would be closest.” He then scrolled back to the top of the page, and on the map, clicked on the Southwest USA region. He was brought back to the region section that he was previously looking at. He looked at this list for more time, attempting to discern which charter chapter was most geographically appropriate. He selected the Freeway Lace Guild link as a “guess.” After landing there, he returned to the previous page and selected the Gold Country Lacemakers link. He then said that he does not know which would be his local charter chapter.
User non-actions:
He initially skipped over the interactive map, navigating to the Southwest USA section himself. Also, he did not spend as much time as User_1 on the charter chapter websites, such as to see which one would be geographically closer.
Potential reasons:
It is not immediately evident that the map is interactive, or in what way. Also, he was likely not interested in spending the time to go through each available charter chapter link to work out which once is closest to his locality.
Interesting findings:
This task had minimal success. Both users glanced over the map at first, apparently not realizing that is was interactive. Also, both were off put by the inability to narrow, or at least understand, the geographic focus of each charter chapter. Neither user was confident that they selected their local charter chapter.
4. Find the purpose of the Teacher Development Program.
User actions:
The user returned to the home page. He then navigated to the Proficiency Program page. He soon went back to the home page and then selected About Us. Then, he went to the home page again and clicked Teacher Info. On this page, he quickly found and recited the program’s purpose.
User non-actions:
He did not struggle here.
Potential reasons:
He seemed to be selecting high level subjects, mentally chunking the information and placing it into appropriate categories, despite the more broken-out nature of the links on the home page.
Interesting findings:
Given the word repeat in the task and the associated link (“teacher”), it took a surprising amount of time for the user to get to the appropriate page, opting instead for higher level topics.
5. Find out how long you may keep borrowed library materials.
User actions:
The user went to the home page and then navigated to the Library page. After enlarging the text, he began to slowly scroll and skim the page. He soon gave “3 weeks” as an answer to how long library materials may be kept.
User non-actions:
He did not skim as much as when attempting earlier tasks.
Potential reasons:
Not applicable.
Interesting findings:
This task was simple for the user to accomplish. However, he resorted more to reading than to skimming.
6. Find the pattern for bobbin lace spider earrings.
User actions:
The user returned to the home page and reviewed the available links. He “guessed” that he would go to “pictures” and selected Member’s Galleries. He zoomed in on the page, then selected the Earring Patterns link. Once the page loaded, he said, “Oh no, do I need to click each one of these?” (See the screenshots from User_1’s session, above.) He proceeded to click each image. (For the first image, it took him a bit of time to locate the name of what he was viewing.) He zoomed in on each individual image page, sometimes scrolling to the bottom of the page while giving it a brief skim. He repeated this pattern, selecting the images that to him most “looked like a spider web,” until he located the correct earring pattern. (He went through all of the images before finding the correct one.)
User non-actions:
Though the task took some time, the user did not really struggle.
Potential reasons:
Not applicable.
Interesting findings:
He associated patterns with picture, and thus with the galleries. I am not sure that most people would make this connection. Also, both users seemed to grasp pretty easily that to get to a pattern, you must actually click the image. However, both were exasperated that you must follow the image link to know what you are looking at.
7. You want to borrow the video “Milanese Lace" by Louise Colgan from 2004 to show to a group of students, but you need to ensure that the length of the video fits your allotted time. Find the duration of this video.
User actions: The user returned to the home page, then navigated to the Library page. He quickly expressed pleasant surprise when he saw there was a “videos” link (see below).
When presented with the resulting document page, he said, “OMG.” He enlarged the text and began reviewing the list of videos. When he located the video, he gave its correct duration.
User non-actions:
Though the task took some time, the user did not really struggle.
Potential reasons:
He was likely off-put by the text-heavy document and overwhelmed by the fact that he had to skim it to look for a certain video.
Interesting findings:
Like User_1, User_2 did not notice that this video comes in two formats. He cited only the DVD format, unlike User_1, who cited only the VHS format.
Review
[7] Surprising Findings
The study outcomes were relatively expected. The IOLI site is very simplistic, yet not intuitive. The site is probably more difficult for new users, or people who are new to lace, than for users who have used the site previously. In fact, the participants comfort with the site appeared to grow with each successive task.
Wrap-up interviews were unplanned, but both participants were inclined to speak afterward. This being the case, I made sure to note these post-session reflections. Much of this content will be useful in the redesign stage. User_1 ended up browsing unexplored areas of the site and said that she would not know what some things were, like specialty items, without going to its page. She also said that for much of the site, she navigated it “wondering what ‘this’ is.” She thought that the vendors page was self-explanatory, which was surprising, as I personally find it barren and unclear, without a leading hierarchy; perhaps this is a non-issue for someone with an immediate predefined need, or someone who was directed to this specific page as a resource. On her own accord, User_1 rated the overall site as an 8 on a scale of 0 to 10. User_2 called the site hard to use, saying that it would be that way even for a more frequent visitor. This was interesting feedback coming from someone in the marketing field, especially since his use seemed to ease as the session progressed. This user said that were he a regular user of the IOLO site, he would probably find ways around the difficulties.
It was interesting to identify the areas of the site that proved most frustrating for the participants. User_1 navigated the site in a fashion similar to User_2, but she did not express nearly as much disdain for it, which is likely due to the different format she experienced during the study. User_1 did not feel the need to enlarge the text and move it around on a pocket-sized screen like User_2 did, for instance. She also was not required to download any documents to her device, unlike User_2. Finally, the home page proved to be immensely important for the IOLI site. This may sound obvious, but here, it seems to act as a hub, housing many links for singular subjects, without being hidden in drop menus. Both users consistently returned to the home page as a starting point when presented with a new task.
[10] Redesign Speculations
Simplify the language of the document pages. People do not plan to spend much time searching for basic information. The documents are quite dense and text-heavy. Something as important as a deadline, for instance, could be made clearer and more prominent.
Ensure there is useful, explanatory information on each page. Some pages have little to no copy, others have too much. Also, the multiple document links create a lot of undue work, via clicking and scrolling, with the documents themselves wordy and clunky, as mentioned above. Have info on the pages themselves means less clicking though, plus mobile users will not be forced to download items to access the information.
Use information chunking, with appropriate headers, text differentiation, and the like. Also, eliminate language/content redundancies within pages/documents.
Include useful links. For donations, for instance, include the name and contact info, with a link to the proper email address, in the policy. Having the relevant information localized will save users time, as they will not need to search multiple pages for a single task. The names might change, but keeping the website updated in this sense will also lend for a sense of relevancy, or currency.
Label pattern photos in the pattern galleries. This simple addition will save users a lot of time. Maybe, too, consolidate the pattern galleries.
Have a template, or regulations, for charter chapters to use for their individual websites.
Make the design mobile responsive. Particularly important due to the small text size on the smaller screen.
Include a “search by zip code” function on the Charter Chapters page. At the very least, include addresses for each charter chapter (this is particularly relevant since not all of the charter chapters have websites, where more concrete location info might be available).
Add a general search function for the entire site.
[9] Relation to the Readings
Chapter 9 of Steve Krug’s Don't Make Me Think: A Common Sense Approach to Web and Mobile Usability was a great resource for planning the user study. I used his recommended outline (welcome, warm-up conversation, casual recap after the tasks) as a template for this assignment. I did not, however, do a home page tour given the nature of the IOLI landing page, which is of utmost simplicity. The information from the Krug text also allowed me to anticipate many of the ways in which the participants struggled, such as how the lack of visual clues impeded their sense of where to start some tasks. It also clarified problem areas that I did not anticipate, such as how the words the users looked for were not always there. Krug’s “Facts of Life” were also evidenced multiple times during each session. As can be seen from the notes above, both users exhibited satisficing, muddled through the site when a definite path was unclear, and leaned toward skimming rather than reading content.
Chapter 14 of Bruce Tognazzini’s Tog on Interface was likewise useful in preparing for the user study. I particularly latched on to the “think aloud” concept, so I used his suggestions for explaining the process at the outset of each session, then made sure to probe the participants if ever they became “too quiet.” I am glad I used this approach, as it seemed to help the participants think systematically about their actions and later proffer rationale and suggestions relating to the tasks. Some of their suggestions are even reflected in the Redesign Speculation section below. Finally, some of our more recent readings gave me a sense of which elements to be aware of and keep in mind for future attempts at redesign. For instance, Chapter 10 of Patrick J. Lynch and Sarah Hortons’s Web Style Guide discusses “linking meaningfully,” something lacking on, and apparently hindering the success of, the IOLI website. Further, the readings on accessibility, including discussion of headers and hierarchy, will surely prove to come into play in the redesign strategy of the IOLI website, based on the findings outlined above.