Did your innovations produce the desired results? Is there evidence that you advanced your students’ higher order thinking skills?
For project goal #1 (finding ways to enhance the learning goal of analyzing sources critically) we found that the attention we gave to the analytical skills helped focus course goals and therefore focused student attention to those goals. We also appreciated greater opportunity for metacognitive reflection, partly demonstrated by the assignments, partly demonstrated in the post test, and also demonstrated in class discussion. By the end of the course students were readily discussing their presuppositions of biblical texts and able to identify and reflect on the kinds of analysis they were undertaking.
We developed narrated PowerPoints to allow more time in class for analysis of texts, and they achieved this goal. An unanticipated benefit of the narrated PowerPoints was that they also allowed the opportunity for reflective learners to internalize the information before coming to class, thus making them more prepared for analysis and discussion in class.
The rubrics provided greater transparency in assignment expectations, clarified what “analysis” means in this course context, and clarified expectations on the assignments. Unanticipated benefits of the use of rubrics were fewer grade disputes and no Honor Code violations. The rubrics also provided greater clarity in instructors’ grading, and more effective and consistent communication of feedback. At the outset of the project, the instructors found that the development of rubrics helped us in our thinking about the component parts and skills involved in analysis.
For project goal #2 (enhance the learning goal of responding to ethical challenges confronting the world) we found that the assignments demonstrated a greater awareness on the part of the students not only of their own presuppositions but also how their presuppositions impacted their analysis of texts. There was greater opportunity for metacognitive reflection related to presuppositions and their learning. The narrated PowerPoint (Ethical Interpretation of Scripture) developed a framework and language for analyzing interpretations of biblical texts; we found that the students clearly engaged the PowerPoint and were able to apply the questions presented in the PowerPoint to the video presentation that we studied related to the interpretation of scripture in a contemporary context.
For project goal #3 (assessment of achievement of these learning goals) we found in the data we have so far (Part 1) a measurable increase in the students’ understanding of academic study of the Bible. We are able to see that they have thought about what this kind of study is, and see improvement in their ability to think about it.
Were there unanticipated problems?
One unanticipated issue was the amount of time needed to create the narrated PowerPoints. Both participants found that there was a limit to the number of narrated PowerPoints that could be created for a given semester. We discovered this is definitely an area for “incremental change” as emphasized in the opening workshop. We also discovered that rubric development is a trial and error process – there are certain things you won’t find out until you use it. We discovered that it takes some trial and error to find the right number of assignments to teach and reinforce the desired analytical skills that is also sustainable for instructor grading time. Finally, a few comments on course evaluations suggested that one type of assignment felt like busy work to some students.
Are there changes you intend to make in another iteration of the course?
- We have already revised rubrics (included in the portfolio): Each instructor rethought the individual criteria under each type of analysis listed on the rubrics to give students a clearer assessment of their development. We also changed the language from “rhetorical” to “literary” analysis to clarify our expectations and to give students an opportunity to connect this work with thinking, knowledge, and skill in other disciplines. For the spring, Sean is now including a “Formulating a Hypothesis” rubric for assignments.
- We have already revised our syllabi (included in the portfolio): Each instructor revised the number and type of assignments included in the course to best address the learning goals and to clarify how each assignment relates to development of desired learning outcomes. We made explicit the connections between the learning outcomes and course requirements early in the syllabus.
- We have already revised our pre/post test: In Part I we replaced two questions that we thought may have been unclear to the students or we concluded were not directly related to our learning goals. In Part II we chose new texts that we thought were better conducive for demonstrating the analytical goals we emphasize in the course.
- “The Convergence” – some of Sean’s course revisions were about beefing up the study of historical/political/social context; some of Kristin’s course revisions were about beefing up attention to ideological construction of identity. Each of these revisions were rooted in the instructors’ individual course designs, but have also, as a result of project initiatives, help bring each course closer to intended learning outcomes.
Did new issues or questions emerge for you?
As a result of our work in this project, we are both interested in looking more broadly at Luther’s All College Learning Goals and how they are met across the curriculum. Our work with faculty has reinforced the need to connect “assessment” with student learning and faculty development and less with external accreditation requirements.
Faculty Work Report
We have had multiple opportunities to share our work with faculty colleagues and other constituents of the college. At the end of the Spring 2013 semester we gave a presentation to the Student Life and Learning Committee of the Board of Regents, communicating the goals of the project and how they grew out of assessment data. We gave a similar report at the final faculty meeting of the 2012-2013 academic year. During the fall semester, we were invited to share our work on the project with new faculty at one of their regular New Faculty Group meetings. In January 2014, we led a Faculty Development Workshop “Using data to improve teaching and learning in Religion 101” at which we shared the assessment data that gave rise to our project proposal, our work on the project, and the preliminary results of our pre/post test. We also moderated a discussion on topics such as development of rubrics, designing assignments with learning outcomes in mind, and development of a pre/post test. Jeff Wilkerson, Terry Sparkes, and Jon Christy, the three people who constituted the “virtual” third team member also attended the Faculty Development Workshop and participated in the discussion.