Work Inquiries
Ph: +1.585.406.2710

Evaluation Story — ConnectNY at Pratt Institute Library

ConnectNY, originally founded in 2001 by five academic libraries, grew to include 15 member institutions and its joint collection allows access to over 8 million resource materials. Of its members, Pratt Institute Library would like to improve students’ ability to access their local, and external resources using ConnectNY. In order to enhance the web experience of Pratt Institute Library, a team of three members was founded to conduct a user experience study.

Team members: Yi Chen, Leslie Lopez, Tinu Thomas

Table of Contents

Collections of Pratt Institute Libraries

Our goals were to: 

  • Determine if students can accomplish common research tasks and easily request books outside of the school’s catalog using ConnectNY.
  • Evaluate if students can navigate the Institute’s library site to find resources in their local collections.
  • Highlight and provide recommendations for potential usability concerns identified in Moderated Remote User Tests comparing the library’s current site against their test website.

My jobs in the team include managing research plans, deciding methodologies, and making recommendation mockups.



Our research plan for user testing.

After meeting with the Digital Learning Librarian of Pratt Library to discuss the goal of evaluating two ConnectNY related interfaces, the team designed a research plan in the sequence above.

After designing two sets of tasks for two interfaces, we designed both pre-test questionnaires and post-test questionnaires.


User Demographic

The six participants were sourced through each expert’s personal networks with students at the Pratt Institute. Below is the demographic breakdown of the participants:

  • Pratt Institute student
  • Median Age: 20
  • Mainly women
Sources where students get materials

The System Usability Scale (SUS) tables indicate that the overall score of Task B, which is for the Test Site interface is a better system. This gives the team an idea of directions to improve the current system.

System Usability Scale — Task A (Current Site)
System Usability Scale — Task B (Test Site)

After performing each task, users were asked to fill out a questionnaire. The post-task questionnaire evaluated the user’s emotional experience. We created an affect grid that visualizes the students’ emotional experience.

Affect Grid

Overall users had a positive experience with the library site. They appreciated the understandability, consistency and ease of use of the website with the need for very minimal learning.

  • They are motivated to use the library website because “there are a lot of resources.”
  • They tend to get frustrated when they’re unable to find items they’re looking for.
  • Even though they know what Connect NY is they “don’t think about ConnectNY at all”
  • They would rather “go to Google to download pdfs”
Screen recording of real-time user testing


Based on the analysis of our user test results, we concluded 4 main problems and 4 recommendations to improve the user experience for the current and test library desktop website.

Problem #1


  • Users got frustrated when the get a ‘No results were found’ response message creating a gulf of execution.

Why does this keep happening to me?


Provide users with a link to search for items from an external source.
Problem #2


  • Users did not expect the top navigation header options to be relevant to the search results.
  • The label ‘ConnectNY’ does not provide any actionable information.

I didn’t know the ConnectNY was on top, I thought that it would be part of the search results, so I wouldn’t go there (Top navigation).


Problem #3


  • Students are motivated to use the library to search for materials they need immediately.
  • Students would rather “search for the pdf on Google” than explore ConnectNY. They feel it would take longer to learn a new system.


Maintain a ‘Request material’ button with estimated delivery time mentioned below.
Problem #4


  • Students were overwhelmed by the number of filtering options available to them.
  • Even though there was a hierarchy in the advanced filter options, it wasn’t visible enough to guide users to the first step they should take.


Assign ‘Basic search’ as the default search bar and optimize advanced search to avoid cognitive overload.
Recommendations Summary

On the standard interface, users are unaware of a way to search for materials from an external source. The students are unfamiliar with ConnectNY and its role in searches.

On the test-site interface, users are overwhelmed with the search options provided and have a hard time locating a way to request the external materials they want.


By applying 4 recommendations to the website, Pratt Institute Library will be able to provide a better web experience to students. It will be more seamless for students to request materials from external sources like ConnectNY, even when they are not familiar with it.

In the end, we presented our user test report to the Digital Learning Librarian of Pratt Institute Library. We shared our data forms, video recordings, and recommendation mockup files to the library for reference, and hopefully, they will be able to craft a better user experience for all communities.

Author avatar
Yi Chen