Event Title

Turning Data into Insight: Analyzing Chat Transcripts to Revitalize Virtual Reference Services

Presenter Information

Hui-Fen ChangFollow

Session Format

Paper Presentation

Conference Tracks

Outreach, Services, and Programs

Short Description

As academic libraries transition to tiered reference services, public service points are relying more heavily on the work of student assistants. This presentation reports a study of analyzing reference chat transcripts data to assess chat reference service quality and performances of graduate student assistants, and how chat transcripts data were used to implement new procedures, revise chat reference training, and retool supervision strategies for online reference services.

Long Description

As academic libraries transition to tiered reference services, public service points are relying more heavily on the work of student assistants. In a library where chat services are monitored by a team of graduate student assistants, there is a concern about the quality of service being provided. Chat transcript data offer a valuable lens through which to access the content and assess the quality of online reference transactions. This presentation reports a study of analyzing reference chat transcripts data to evaluate chat reference service quality and performances of graduate student assistants in an academic library. Anonymized chat transcripts were analyzed because they provided an objective and non-intrusive source of data to observe and evaluate students' performances. A rubric was designed to help us break down chat performance into measurable skill sets such as assessing patron information need and quality of information resources offered. It also provided a framework to rate these skills on a scale from poor to excellent. Over the course of the project, we analyzed over 750 chat transcripts from the work of eight graduate assistants from two semesters. We classified transcripts by patron question type and analyzed both overall and individual chat performances. Ultimately, we used the findings of our data analysis as insight to revise chat reference training, implement new procedures, and retool our supervision strategies for online reference services.

Learning Objectives

At the conclusion of this presentation, participants will be able to:

Learn about the development and implementation of chat assessment rubrics for appraising chat reference services.

Gain a better understanding of the interdependence of library data and library services.

Use and apply learned materials in assessing chat services at their own institutions.

Share

COinS
 

Turning Data into Insight: Analyzing Chat Transcripts to Revitalize Virtual Reference Services

As academic libraries transition to tiered reference services, public service points are relying more heavily on the work of student assistants. This presentation reports a study of analyzing reference chat transcripts data to assess chat reference service quality and performances of graduate student assistants, and how chat transcripts data were used to implement new procedures, revise chat reference training, and retool supervision strategies for online reference services.