- Identify existing data and questions to ask of it
- Identify questions that do not have existing data to answer (Need to obtain data via an assessment methodology)
- Identify target audience:
xx By this point you should have defined what you are measuring. Whatever it is you are measuring, you need to identify and define the user population you will be measuring. The ‘what’ and ‘who’ here clearly are interlinked and will influence from both ends how you design your methodology.
Populations can be tricky. Our users are usually a very diverse group with many quirky variables any of which can skew data considerably. For students, these groupings can include some of the following:
- Where in the graduation pipe are they? Incoming freshmen, transfers or seniors?
- What is their ethnicity? Might that have an influence?
- Native English speaker or ESL? Domestic student or international?
- What’s their major, subject emphasis or discipline?
- Commuter or on campus housing?
- A particular group like honors students or student-athletes?
- Identified as an at-risk population?
- Primarily a library user or a remote user?
All of these variables are important to consider. For example, suppose you are trying to ascertain how important your library is as a study/socializing/group space for students. In that case, you may have a significant skew away from capturing science majors in any given sample since “their” lab won’t be the library. Science students, by the whole, tend to heavily skew toward being remote users of science databases rather than visiting the library to circulate print collections. How, then does this potentially skew your data?
Once you know these population differences and their potential variables, you should further identify your population(s) by methodological selection.
- What is a valid sample size for your identified user population?
- Define what you mean by ‘random sample.’ Is it whoever randomly walks in/shows up, or a centrally controlled sample randomly selected? Be clear when designing your methodology about what you mean by this.
- How self-selected is your sample? In other words, if you analyze how using your print collections impacts some rubric of student success, all students who have checked out a book are self-selected.
- Understand what a ‘control group’ is and how you might use one effectively in your assessment. Generally, a control group in an assessment study does not receive some library service or outreach. It is then used as a benchmark to measure how other tested users benefited from library services or outreach.
- Is your population a defined cohort or a set of cohorts? If so, what kind of cohort? It may include “all incoming freshmen” or some other subset.
- Whether randomly selected or self-selected, choosing cohorts has added benefits in that they become de facto control groups where you can test the impacts of different services on each. For example, one-course class section gets library information via online tutorials, and the other the same content through face-to-face instruction. Compare the difference using a defined rubric (e.g., class grade or successful assignment completion).
One of the most vexing populations of students to study is the “non-user.” We generally mean by that someone who neither visits the library nor checks out our analog collections. Of course, neither means they don’t ‘use’ the library; they generally don’t visit it physically. So how can we reach them?
- Class rosters (e.g., who shows up in the library, who doesn’t)
- Freshman (> Senior) class data sets (e.g., against the whole incoming Freshman class who among them used the library which didn’t)
- Compare random samples of the total population of non-users with random samples of in-house users
- Other
Finally, if you think you’ll get the best results by using swipe technologies, make sure you have campus backing and support to administer swipes anywhere and that you get permission to compare your swipe data to larger university data repositories where even richer student data are stored (e.g., Peoplesoft). xx