Roslyn Miller, Ph.D. and Amanda E. Major, Ed.D., CPLP, PMP are instructional designers at the University of Central Florida; their work and research focus on effective instruction and authentic assessment of student learning. As a team, their expertise covers both public and private educational institutions, secondary and higher education, and subject areas spanning both ELA and STEM.
In sum, Roslyn Miller and Amanda Major are incredible resources for guidance about assessment. To that end, we recently had the privilege of conversing with Major and Miller via Zoom to discuss assessment challenges, recommendations, and the future of assessment.
CHALLENGESFirst, we discussed the challenges of assessment in the sudden shift to remote learning.
According to Miller and Major, the quick shift from in-person to online learning environments exacerbated chronic issues and brought to the forefront the following challenges:
- Faculty fear of technology emerged as they transitioned to online instruction
- Any poor practices that faculty employed in person were transferred into remote instruction
- Faculty expressed uncertainty about academic integrity in remote learning
Amanda Major observed, “Faculty had to overcome their fear of technology. The quick shift to the remote platform was difficult for faculty and for students who were not accustomed to it. Many of our faculty members already had the tools they needed to make that shift and do it effectively, but many did not. Any poor practices that faculty employed in person were transferred as remote lectures to students if faculty did not have the requisite knowledge to transition to the LMS to deliver asynchronous instruction. Some faculty were just delivering lectures without the crux of what makes teaching and learning effective—the activities, the interaction. Nevertheless, instructional technology support teams, instructional designers, technologists, and multimedia specialists were there to guide and support them during this transition.”
2. Pedagogical practices:Strong pedagogical foundations were more critical than ever in the transition to remote learning and assessment. Miller stated, “One of the challenges stemmed from faculty who had not previously taught online trying to take what was done in the face-to-face classroom, including assessment, and just put[ting] it in the LMS. The results were then frustrating for both teachers and students. The online environment really needs to be taken into account,” adding that “the principles of effective teaching are the same, regardless of whether in a classroom or online; however, the environments are very different, and that’s where differences in techniques and strategies really need to be considered. Those considerations have to do with time—synchronous or asynchronous—and physical environment—whether you’re together in a room or all in different places. So, different times, different locations, and the way content is delivered and received and worked with all affect teaching strategies.”
“One of the biggest challenges I heard expressed from faculty members when we transitioned to remote learning was concerns about academic integrity in assessments,” said Miller. “Some of the concerns were that students would use books or the internet to complete an assessment that was intended to be closed-book, without resources. Faculty expressed concerns about students copying and sharing assessment items. Those were the biggest concerns that I heard, and those were some of the things on which we spent the most time working with faculty.”
“One immediate band-aid that faculty requested was technology to proctor students,” Miller added. “So, there was attention and money put into proctoring systems. However, even with providing those to faculty, we really tried to emphasize quality assessments where those proctoring tools are hopefully needed less.”
Major concluded, “Instructional designers aimed to collaborate with faculty to design authentic and multiple, varied assessments, including practice for low stakes exams worth just a few points or non-graded activities.”
RECOMMENDATIONS:So what are Miller and Major’s recommendations for assessment with integrity?
- Promote familiarity with technology:
Major shared, “I helped faculty who had not taught online or hybrid courses for quite some time; they needed a refresher. We had a conversation about incorporating application or analysis activities into their course using current events or challenging students to relate content to their lives to, hopefully, facilitate meaningful, learning experiences, given the times. One faculty who I worked with created a collaborative timeline activity for her students to map out what was going on with their lives during the COVID-19 crisis.”
2. Employ tools that support best pedagogical practices:Amanda Major recalled, “I had an influx of faculty who were teaching synchronously on Zoom and just a few who were dabbling in Panopto, too. I could see the need for collaborative, group activities/assessments because this could simulate activities faculty facilitate in the classroom. Sometimes, the activities don’t necessarily transfer one-to-one in the online format, requiring some ingenuity in the redesign of the activity in the online format. I began investigating ways for students to collaborate in real-time, through technology.” Major explored Sutori, Padlet, Google Docs, Jamboard, Materia activities. She also examined “activities like timelines, KWLs, and Rose, Bud, Thorn reflections, and more classic types of assessment documented by Angelo and Cross such as muddiest point, one-minute paper, concept maps, and so on.”
3. Uphold academic integrity with best pedagogical practices in assessment design:Vary your assessments. Miller stated, “If an assessment is made of exactly the same questions from semester to semester, yes, those assessment questions are out there and googleable, and so that creates a temptation for students to cheat when they have an opportunity; they’re alone in an online assessment, and they’ve got the time and technology to do it.”
Frequent, low-stakes assessments uphold academic integrity and increase the quality of assessments. Major said, “Instructional designers collaborated with faculty to design authentic and multiple, varied assessments, including practice for low stakes exams worth just a few points or non-graded activities.”
“I think one of the major challenges, related to the circumstances that create the concern with academic integrity, is that many higher education courses have two or three major multiple-choice tests—exams, and the total grade is calculated from those few exams,” said Roslyn Miller. “So, what that creates for academic integrity concerns, is that each assessment is high stakes enough that it makes a student weigh the risk of being dishonest: ‘Can I pass this test by studying enough on my own or can I pass it more easily by trying to do something else?’”
“One of the steps to address academic integrity concerns is to change the nature of the assessments so that they are not so high stakes by themselves and so that they don’t measure just facts that have been asked again and again and are just out there and can be looked up in a matter of seconds. So, instead of trying to find more technology tools to keep online students from looking up questions and trying to cheat otherwise, we need to change the nature of the assessment. Some ways that can be done, like what Amanda has already mentioned, is instead of having two or three major exams, spread assessment of student learning over more frequent assessments. When each one is lower stakes, the temptation to cheat is less; it’s not worth it to spend a lot of effort trying to cheat if you can demonstrate your learning more authentically.”
4. Enable formative feedback loops in assessment via tools and collaborative work:Major recommended that “Faculty members who are concerned about academic integrity can prompt students to turn in essay assignments through Turnitin Feedback Studio. Students can self-correct their work and see when they are using exact quotations via the Similarity Score. They can improve their writing and re-submit their essay assignment for a grade. Other activity ideas to encourage academic integrity are peer reviews and discussions because faculty can glance at these to determine that the student has performed substantively enough while students are holding each other accountable, too.”
“Hopefully,” said Major, “the student interactivity and collaborative activities offer learners a chance to bond with each other at a time in which isolation is very real. Group work is a great way to ensure that students are not cheating because a member of a group would likely speak up if a student was not present and active in the group. Faculty could build in an assessment for group members to rate each other’s contributions, then use that in each students’ grade calculations.”
THE FUTURE OF ASSESSMENT- What have we learned thus far about assessment?
According to Miller, “Certainly students have expressed a desire to get back in classrooms. They like to be in the presence of their professors and their classmates; many really feel like they learn best in that physical presence. So, what I hope, as we get back to the physical community, is that faculty and students (but especially faculty) will have learned their learning management system (LMS) so that they can use it to support their classroom teaching.
Some ways perhaps they’ll do that is to curate content in their LMS, maybe some short lecture videos or links to external resources so that students can process content that way, outside of the classroom, so that they can then use classroom time for more active learning and using that community environment for more collaborative work. That’s something I hope will be a change when we return to the classroom.”
Major responded, “Yes, I agree completely with you Roslyn. I’m hoping that we’re moving towards a flipped learning experience. Our Executive Director for the Center for Distributed Learning Kelvin Thompson often says, ‘The future is blended.’
“Additionally, we, as a cohort of faculty during the COVID-19 crisis, have learned that assessment can occur during live, synchronous video conferencing sessions, and that reflection, as well as planning activities, can be used as assessments. The lines between activities and assessments are blurring, and I think that’s a good thing.”
2. How will what we’ve learned in 2020 to date inform the future of assessment?Miller advised, “To begin any discussion of assessment, we should start with asking what the learning goals are. Those learning goals should be clear, not only to the instructor but should also be clearly, explicitly communicated to students in student-friendly language. This needs to be done at the course level overall and for each learning unit, so that students know what they’re going to be expected to learn, to know, to do by the end of this learning unit and by the end of the course.”
“Starting with communicating those learning objectives, then the content of the course needs to be aligned with those learning objectives. And whether we are in a physical classroom or online, we can use the LMS to make those connections explicit. We can use a graphic, like a matrix, to show what components of the content relate to which specific learning objectives. And then it will be helpful if we also make explicit for each learning activity or assignment what specific learning goal the activity is intended to help us meet. Then finally, the assessments should also be aligned with the objectives, content, and activities. The assessments should really look like the activities students do in class; they should look like the practice students have been doing.”
We thank Roslyn Miller and Amanda Major for joining us in this discussion about assessment with integrity. And we hope this helps you in your understanding of assessment and supports your authentic assessment of student learning.
