Getting Started with SoTL
What kinds of topics can I investigate when designing my own SoTL study?
Problems in Teaching
When thinking about a SoTL study, consider what "problem" you have been facing. While we usually think of problems in teaching as something to be ashamed of or hide from others, problems in SoTL are the foundation for intellectual inquiry into the complex, dynamic and often uncertain processes of teaching and learning. This inquiry becomes the basis for SoTL study and the reason why we share our work publicly for peer review. Randy Bass discusses this fundamental shift in the culture and discourse surrounding teaching in his foundational SoTL article, "The Scholarship of Teaching: What's the Problem?"
As you think about this problem, what is it that you want to know more about? Perhaps you’re unsure why something is happening in your classroom, or maybe you’re pretty sure you can explain why – but have you systematically collected data to support your explanation? Are you relying on largely anecdotal evidence, perhaps without realizing it? If you've ever found yourself saying, "I think that worked" about your teaching, then you might be in this latter category.
Here are two questions you might ask yourself to begin the brainstorming process:
- In your teaching, describe a strategy, activity or learning sequence you feel strongly “works” or “doesn’t work.” What reason(s) do you have for reaching this conclusion? What evidence would you need to prove to your department chair once and for all that this strategy does or does not work so that it can be widely adopted or abandoned?
Think about a moment or moments in your teaching where you were perplexed, confused, intrigued, or maybe even shocked about something related to your students’ thinking, learning and/or your teaching that made your curious to know more. Describe the moment and what you want to know more about and why.
These two questions speak to two simple categories into which SoTL scholar Pat Hutchings divides SoTL studies: 1) What works? and 2) What is happening? (Hutchings, 2000). Read more about these two types of research questions.
Resources to learn about SoTL
Bishop-Clark, C., Dietz-Uhler, B., & Nelson, C. E. (2012). Engaging in the Scholarship of Teaching and Learning: A Guide to the Process, and How to Develop a Project from Start to Finish (First edition). Sterling, Virginia: Stylus Publishing.
- Workbook style approach to doing SoTL. Available to check out in the CTL library. Contact email@example.com.
Hutchings, P. (2000). Opening lines: Approaches to the Scholarship of Teaching and Learning. Menlo Park, CA: The Carnegie Foundation for the Advancement of Teaching.
McKinney, K. (Ed.). (2013). The Scholarship of Teaching and Learning In and Across the Disciplines. Indiana University Press.
- Review of issues and critical questions in SoTL; cross-disciplinary focus
McKinney, K., & Cross, K. P. (2007). Enhancing Learning Through the Scholarship of Teaching and Learning: The Challenges and Joys of Juggling (1 edition). Bolton, Mass: Jossey-Bass.
- More of a how-to book, narrative (expository) approach with references
SoTL is about your role in teaching and learning both in your classroom, but also
your department, and university. Hutchings, Huber, and Ciccone (2011) suggested four
areas of institutional impact of SoTL:
1. Improving Teaching and Student Learning
2. Professional Growth and Faculty Development
4. Valuing and Evaluating Teaching
Thinking about #1, SoTL scholar Pat Hutchings (2000) divides SoTL studies into four types*, the most common of which are What works? and What is happening? studies.
'What works' studies are those that seek to collect evidence on the effectiveness of particular instructional strategies for particular instructional goals. Although Hutchings calls them, “What works,” it's often useful to ask “What works for what instructional goal?” to help clarify the intended goal of the strategy and better focus our research (Cohen, Raudenbush, & Ball, 2003). The instructional goal in these studies might be looking at student learning goals, but there might also be other outcomes of interest such as increase students’ self-regulatory skills, fostering a disciplinary way of thinking, or increasing students’ capabilities to ask good questions. Two examples of what works studies with impact at the institutional and department-level are:
Bernstein, D., & Greenhoot, A. F. (2014). Team-Designed Improvement of Writing and Critical Thinking in Large Undergraduate Courses. Teaching & Learning Inquiry, 2(1), 39–61. https://doi.org/10.2979/teachlearninqu.2.1.39
Gingerich, K. J., Bugg, J. M., Doe, S. R., Rowland, C. A., Richards, T. L., Tompkins, S. A., & McDaniel, M. A. (2014). Active Processing via Write-to-Learn Assignments: Learning and Retention Benefits in Introductory Psychology. Teaching of Psychology, 41(4), 303–308. https://doi.org/10.1177/0098628314549701
'What is happening' studies are driven by your desire to better understand something happening in your classroom or among your students. It seeks to make visible what was previously invisible or covered, like student thinking. It’s digging deeper into a problem of teaching. Teaching is rife with problems because of the nature of teaching (Lampert, 2003). Problems are nothing to be ashamed of; they are points of focus that are ripe for scholarly inquiry (Bass, 1999). Two examples of “What is happening” studies focusing on specific classrooms are:
Ciccone, A. A., Meyers, R. A., & Waldmann, S. (2008). What’s So Funny? Moving Students Toward Complex Thinking in a Course on Comedy and Laughter. Arts and Humanities in Higher Education, 7(3), 308–322. https://doi.org/10.1177/1474022208094414
Manarin, K. (2016). Interpreting Undergraduate Research Posters in the Literature Classroom. Teaching and Learning Inquiry, 41(1). https://doi.org/http://dx.doi.org/10.20343/teachlearninqu.4.1.8
Once you have brainstormed and found a problem worth investigating, you are reading for the next step: searching the literature.
Bass, R. (1999). The Scholarship of Teaching: What’s the Problem? (PDF) Inventio: Creative Thinking About Learning and Teaching, 1(1).
Cohen, D. K., Raudenbush, S. W., & Ball, D. L. (2003). Resources, Instruction, and Research. Educational Evaluation and Policy Analysis, 25(2). DOI: https://doi.org/10.3102/01623737025002119
Hutchings, P., Huber, M. T., & Ciccone, A. (2011). The Scholarship of Teaching and Learning Reconsidered: Institutional Integration and Impact. John Wiley & Sons.
Hutchings, P. (2000). Opening lines: Approaches to the Scholarship of Teaching and Learning. Menlo Park, CA: The Carnegie Foundation for the Advancement of Teaching.
Lampert, M. (2003). Teaching Problems and the Problems of Teaching. Yale University Press
SoTL projects are always situated in what we know about teaching and learning as well as the discipline. The University of Calgary has a comprehensive SoTL guide that is a gold mine of information (also UGA library's own resource guide for searching the educational literature). Highlighted here a few of the most important things from theses guide as well as a few tips that have helped other SoTL researchers find their way: literature databases, how to identify relevant search terms, and a brief orientation to higher education and SoTL journals.
This SoTL Annotated Literature database is curated by Nicola Simmons at Brock University in Canada.
It's good practice to try many different search terms related to your area of inquiry, framework for teaching and learning, or methodology in order to find relevant literature. Moreover, trying search terms at different levels of specificity may be helpful. For example, the term "active learning" covers a broad range of simple, short teaching strategies (e.g., writing to learn, or think-pair-shares) to more complex teaching pedagogies (e.g., case-based learning, project-based learning). In July 2017, searching ERIC for the term "active learning" yielded 7,685 results with an incredible range of foci: faculty perceptions of active learning, the impact of active learning on student teachers' competence, how to get active in large lectures, whether peer interaction is necessary for active learning.... so, narrowing your focus can help you find more relevant literature. A few ways to narrow your focus are:
- Type of pedagogy
- Target audience (e.g., undergraduates, first generation college students, etc.)
- Type of educational institution
- Qualifiers (e.g., benefits, challenges, obstacles, efficacy, effectiveness, significant difference)
- Methodology (e.g., qualitative, survey research, think-alouds)
While narrowing your focus is helpful, avoid getting to narrow. There are many studies done in K-12 schools or in different disciplines that might be useful to your study.
Of course, the best method to finding relevant literature is to pour over the reference lists from articles that you have found that are relevant to your area of inquiry, methodology, or framework for teaching and learning.
Higher Education and SoTL journals
There are an incredible number of higher education journals: over 700 at last count. Here are a few of our favorites, as well as some extensive databases provided by other institutions.
- For a complete directory of teaching journals, with capability to search by discipline and topic in higher education, visit Kennesaw State's directory. This can get you quite narrow rather quickly so try using the "general" or "any" search options, too.
- Southern Illinois University's selected list of higher education journals. Includes the major journals by discipline (one page).
- Early Career Higher Education Research Network's more selected list of higher education research journals. This is not an exhaustive list, but it does include acceptance rates.
Once you've conducted a literature review, the next step is to: design your study
Study design also answers the who, when, where and what of your SoTL study. Many of these answers depend on your research question, but here are a few questions to ask yourself:
- When does my study begin and end? It doesn't need to last all semester; a few classes may be all that is needed to meet your instructional goal, or perhaps just one afternoon of focus groups.
- Do I need to collect data from all students? Especially when you are interested in why students are struggling with a concept, idea,
or skill, it might be more useful to target particular students for data collection.
- This brief interview with a SoTL scholar at the high school level is quite useful for showing how to choose participants selectively and for maximum benefit.
- How often, and at what points, is optimal to collect data for my research question? You must balance your need for data with respect for your students' time and energy. This is why many SoTL researchers collect and analyze assignments the students are already producing in the course. Also consider that collecting data on learning immediately after students have been taught tells a different story than collecting it a week, or even 8 weeks, later.
- How can I remain ethical in my study design? Great question! You're ready for the next step, considering Ethics and IRB in SoTL;
Great questions! Being ethical in SoTL work means proactively making sure that your human subjects--your students--aren't feeling like they have to participate in your study because their grade or relationship with you might suffer if they do not participate; that students aren't missing out on what we believe to be best for their learning, and that they are fully aware of any risks or benefits of participation. It also means that you are confident that any risks associated with participation in the research are mitigated and any potential benefits from the study outweigh the risks. To help you navigate those questions, most institutions have their own IRB.
IRB = Institutional Review Board. This is the research oversight committee charged with ensuring that human subjects research is conducted ethically. Each research institution typically has one, and the one at UGA is lovely. At UGA, they are a part of the Human Subjects Office, which is part of the Office of Research. They hold IRB meetings monthly and welcome emails to firstname.lastname@example.org.
The decision about whether you need IRB approval is best handled by the IRB office. Therefore, you should submit your study for review and let them make the determination whether their approval is needed to conduct the study.
MacLean, Mark and Poole, Gary (2010). An Introduction to Ethical Considerations for Novices to Research in Teaching and Learning in Canada. The Canadian Journal for the Scholarship of Teaching and Learning. 1(2), 1-10. DOI: http://dx.doi.org/10.5206/cjsotl-rcacea.2010.2.7
Healey, R. L., Bass, T., Caulfield, J., Hoffman, A., McGinn, M. K., Miller-Young, J., & Haigh, M. (2013). Being Ethically Minded: Practising the Scholarship of Teaching and Learning in an Ethical Manner. Teaching & Learning Inquiry: The ISSOTL Journal, 1(2), 23–32. https://doi.org/10.2979/teachlearninqu.1.2.23
With a better understanding of the who, what, when and where, you can begin to think about the "how," that is what are the right methods of data collection for your SoTL study.
If the first question you're asking yourself is, "Should I use qualitative or quantitative methods?" you may be skipping some important questions.
You have likely considered final, or summative, assessments (assignments) in your course: exams, papers, presentations or other final products that students typically produce towards the end of your course. These are fine sources of data--keep in mind, however, that these assignments may represent the synthesis of different skills and forms of knowledge beyond the scope of your SoTL study. Think about formative assessments that you can use to collect data on how and what students are learning as your study progresses:
- Selected questions from an exam or standardized assessment measure
- Classroom Assessment Techniques, or CATs, a form of active learning
- Evidence of student thinking: Observation of students (recorded as field notes), audio recordings of student group work, think-alouds
- Instructor reflections: Your own written reflections on student learning and how that relates to your instruction
- Student self-report of learning: Interviews or focus groups with students (current or past), surveys. The UGA-CTL conducts mid-semester formative evaluations (MSFEs) each semester than can constitute data for a SoTL study. This means we can collect and synthesize data for you!
If you have a more exploratory SoTL study, qualitative measures such as case studies, interviews, and focus groups tend to be more useful for these studies.
- This brief yet informative guide (PDF) from the Vanderbilt University Center for Teaching talks a little about direct vs. indirect evidence of learning and breaks down qualitative and quantitative sources of data.
- In addition, this 2005 article (PDF) from Dan Bernstein and Randy Bass discusses the limitations of just using graded work, especially for uncovering the "intermediate processes" of student learning.
Once you've identified some methods you might want to try, remember to search the literature for other SoTL scholars who've used those methods. In addition, here are some resources when you're ready to dive deeper:
- Social Science Research Methods Knowledge Base
- Social Science Research: Principles, Methods, and Practices (free, open access textbook)
- Finding Pre-existing Questionnaires and Instruments (always recommended above creating your own)
- UGA offers all faculty and graduate students a free account with Qualtrics, an online survey development tool
- Guidance for Creating Likert Scales
- AAC&U's VALUE (Valid Assessment of Learning in Undergraduate Education) rubrics for assessing written work
Ideally, you identified a conference or a targeted journal way back when you were designing your study. I find having a concrete goal (and deadline!) and a collaborator to hold me accountable is the best way to see my SoTL study through from beginning to end.
While publication in SoTL and education scholarly journals is grand, there's no better place to begin presenting your SoTL work--ongoing or completed--and meeting other SoTL researchers who can become your support network than at conferences. Here are a few that I like, starting with some right here in Athens or within driving distance. In parentheses you'll find the date of the conference and, if known, proposal deadline:
Local and regional conferences
- Innovation in Teaching Conference, UGA College of Education (October; proposals due September 18, 2017)
- New in 2017: Opt-in to have your proposal recognized for the Emerging Landscapes in SoTL Award!
- USG Teaching and Learning Conference, University System of Georgia (April)
- Conference on Higher Education Pedagogy and Conference on Teaching Large Classes, Virginia Tech (February; proposals mid-October)
- American Educational Research Association, Annual Meeting (April, proposals in July; location varies)
- Lilly Conferences (location and timing varies; often in NC, MI and MD)
- The International Society for the Scholarship of Teaching and Learning Conference (ISSOTL; publisher of Teaching & Learning Inquiry)
The Center for Excellence in Teaching and Learning at Kennesaw State University has a comprehensive teaching conferences directory which you can filter by location, discipline, and topic. Thanks, Kennesaw!