Assessing Outside the Box: the Open Format Assignment

By Caitlin Cornell and Megan Dean, Michigan State University

 

The COVID-19 pandemic has demanded unprecedented innovation and adaptation from instructors and students alike. Online learning has required many students to learn how to use new-to-them digital technologies like video editing, audio recording, and virtual presentation software. As we move back into more traditional classrooms, instructors may wish to incorporate students’ newly-honed digital abilities into their pedagogy. At the same time, we may have concerns about instructor investment in learning and teaching new technologies and about accessibility for students with disabilities, different learning styles, or varying access to resources.

In this article, we introduce a type of assignment that invites student use of digital technologies and critical thinking about the value of different modes of communication: open format assignments. An open format assignment works for online teaching and in-person courses and requires minimal instructor investment in teaching new technologies to students. Leaving the format of an assignment open to student choice has benefits for diverse student populations, including disabled students, and encourages student creativity and autonomy. The key to an open format assignment is a rubric which enables fair assessment across formats and emphasizes in-depth critical engagement with course concepts. 

In what follows, we discuss our use of open format assignments in two undergraduate learning contexts: an honors option mini-course on race, language, and disability, and in various philosophy courses. We offer narrative accounts detailing the context and rationale for our use of open format assignments, outlining assignment learning goals and rubric design, and noting key challenges and limitations. We suggest that these assignments can be easily adapted to a variety of learning contexts, and in the final section of the article we discuss the possibilities of incorporating open format in world language learning assignments. 

Case #1: Open Format Critical Reflections in a 5-week Race, Language, and Disability Course 

In the Spring of 2020, I (Caitlin) developed and taught a 5-week mini-course as an honors component (henceforth “course”) for a section of the Immigrants, Minorities, and American Pluralism course at Michigan State University. I asked the 12 students who signed up for the honors component to reflect critically on the language that American academia, society, and media use to shape current discourse about disability and race, via both traditional disabilities studies as well as Critical Disability Theory and DisCrit (Disability Critical Race Theory). 

I designed the course (pre-pandemic) to be maximally accessible and minimally anxiety-inducing, placing a strong emphasis on offering students choice in my course assessment. Their engagement with the ideas was paramount to me, regardless of format, and I wanted to see what students could create when given the freedom to choose. I also wondered if educators always need the level of specific control over student output that we often think we do.

On March 11, 2020, the morning after my first session of the course, the university announced the emergency transition of all face-to-face courses to remote delivery. The remaining four sessions of my course were held virtually, via Zoom. In the end, the purposeful design choices I made pre-pandemic became a saving grace during a tumultuous and uncertain time. I already understood that offering multiple modes for students to showcase their learning is one way of addressing accessibility (see, for example, Universal Design for Learning) and fostering more inclusive learning environments, but I only realized once the course had already begun the crucial flexibility that open format assessment would afford students during a global pandemic.

Assessment for the course comprised 4 critical reflections based on course readings and class discussion. I not only gave students the choice to complete any 4 reflections from 5 weeks of topics (i.e., they had one chance to not address one topic that was of less interest to them), but they were also able to choose how to design their reflections. Again, it was connection-making that was important to me, rather than format. Even if I had required traditional essays, I would not have prioritized formal writing skills in my evaluation of them. In order to make my assessment goals explicit for students, I included in the syllabus a paragraph explaining my motivations for infusing open format choice in the assignments as well as links to some general resources (e.g., TopHat Blog on Higher Ed Assessment Strategies) illustrating what “alternative” assessment (open format assignments are one type) can look like.

I developed a “universal” (my term) rubric to assess learning outcomes within the reflections. I was curious about its efficacy as a tool for evaluating student learning. The rubric was universal in two senses: (a) I used the same rubric for every reflection since each critical reflection served the same learning outcomes and (b) the rubric was bare of any binding information regarding mode, medium, design, or format of the reflections (I left the format of the reflections completely up to students). 

Critical Reflection Rubric:

Please address these criteria in your critical reflections that are based on the weekly course readings. Remember, the form your work takes is up to you. If you prefer to have more structure or guidance beyond this rubric, just ask! 

Criteria	5 points	4 points	3 points	2 points	1 point	0 points
Demonstrable understanding of concepts from the reading
Depth of reflection Response demonstrates an in-depth reflection on, and personalization of, the theories, concepts, and/or strategies presented in the course materials to date. Viewpoints and interpretations are insightful and well supported. Clear, detailed examples are provided, as applicable.
Clarity/explicitness of the medium (e.g., the symbolism of art pieces may need to be accompanied by some commentary by the artist in order to convey connections between the art and the reading to the instructor.)

Strengths of your critical reflection: 

 

Suggestions for growth in your critical reflection: 

 

Elements of this rubric were inspired by or adapted from the following resources: 

http://studentassessment.ucalgaryblogs.ca/files/2017/06/Reflection-Rubric.pdf

https://brocku.ca/pedagogical-innovation/wp-content/uploads/sites/53/Critical-Reflection-Rubric.pdf

http://carla.umn.edu/assessment/vac/improvement/p_6.html?sfns=mo

In your reflection, consider addressing the following elements:

1. What are the various authors’ perspective(s) relative to this week’s topic(s)?
2. In what ways do the authors’ perspectives contribute to general discourse on this topic in academia, society, or the media?
3. If you were to explain the key concepts from this week’s topics/readings/media to a family member at a barbecue, what do you think you would include in a 30-second summary?
4. What connections can you make from any of this week’s readings/media to your own experience? In other words, what did any particular reading/media/concept mean to you personally, and why do you think it had that effect on you?
5. What connections can you make from any of this week’s readings/media to your own discipline or academic interests?

 

Though I was inspired by its potential benefits, I wasn’t sure how open format assessment would manifest in reality. My primary concern was how much direction I should give students. On one hand, I didn’t want to color students’ formatting choices via the examples I gave them as a reference, but I also didn’t want them to struggle with too blank of a slate. I wanted to stimulate without controlling. I included in the syllabus only brief descriptions of what form critical reflections might take: “You choose the vehicle or manifestation of your work. For example, an essay, a video, artwork, a presentation, etc.” 

I was also concerned about explicitness in students’ work that weren’t text-based (like art or other visual representations, for example). In the case of artwork, for example, I warned students to be prepared to help me interpret their artistic metaphor with additional explanation if necessary. I chose to operate under the assumption that any lack of understanding was my own creative limitation, not related to the students’ output. I resolved that I would allow students to explain their intentions and interpretations casually, perhaps via virtual office hours; I did not require text-based accompaniment of visual representations so as not to create more work for students.

Another consideration was how to handle student work done in tactile modalities, a challenge resulting mainly from incorporating open format assignments during a pandemic in which students would be unable to deliver assignments to me physically. We discussed that in the case of any such work they could take a photo of it and send that to me, instead of presenting me with the object itself. One student made use of PowerPoint to create a digital collage (but whether that was in response to pandemic limitations or just her natural use of technology for creativity, I don’t know for sure). Just about any other mode or format could be achieved electronically.

The purposefully devised flexibility of my open format assignments led to extreme creativity and high engagement with course topics. While most students chose to write traditional essays, some embraced the freedom to choose a medium that spoke to them. Some students chose to work with a different medium for each reflection, making use of their freedom of choice over and over again. Among my favorite submissions were: a poignant art installation (a student video-recorded her sketch pad while she simultaneously sketched and narrated her visual connections to course topics), and an interview about course topics by a student of their family members after they had moved home from campus during the pandemic.

After the course, I made available to students a voluntary course evaluation survey. Here are highlights of their qualitative responses that they allowed me to share: 

At first I leaned into my comforts, but as the course went on and I learned more I thought the alternative assessment allowed me to try some new things that actually made me dive into the course content and grow my understanding.

I like freedom in my classes, it communicates to me my advisor is treating me like an adult who can adequately decide my output of work.

I felt a lot better once you gave more explanation on what you were looking for. It was helpful to know that we had total freedom as long as we followed the requirements on the rubric.

 I liked the idea of having total freedom, but I would’ve liked some examples to see at the beginning of the course. I just wasn’t totally sure at first how to produce a reflection with all of the requirements and I think it would’ve been cool to see how other people did it.

In the future, based on the latter two student comments above, I will showcase examples that former students have consented to share, even at the risk of influencing students’ creativity somewhat. 

Case #2: Open Format Assessment in Undergraduate Philosophy Courses

I (Megan) started using open format assignments in my undergraduate philosophy teaching at Georgetown University in 2016. Since then, I have used them as final assignments in various undergraduate courses, including Bioethics and Disability, Philosophy of Science, Philosophy of the Body, Ethics of Health Care, and Philosophical Issues of Feminism. These courses have varied in size and level, from small upper-year seminars at Hamilton College to 100-person lecture courses requiring no prerequisites at Michigan State University. 

The majority of my undergraduate teaching has been to non-philosophy majors. For example, my courses on health care ethics attract students going into the health professions and my recent Philosophical Issues of Feminism course attracted many students majoring in women’s and gender studies. So rather than focusing on formal philosophical argumentation, I aim to help students build up a “toolbox” of philosophical concepts, theories, and approaches that they can use to analyze and evaluate real-life experiences. The open format assignment is an opportunity to demonstrate in-depth understanding of a particular philosophical “tool,” to reflect on how it may be useful and to whom, and to practice articulating philosophical insights in accessible and engaging ways. While I developed the assignment with non-majors in mind, these skills are also valuable for philosophy majors interested in public philosophy or interdisciplinary work. And because students can choose to write a traditional philosophical essay, majors who wish to demonstrate and develop their essay writing skills can do so within the confines of the assignment.

The most recent iteration of the assignment was used in Spring 2021’s Ethics of Health Care, a 100-person lecture course with a teaching assistant, and Philosophical Issues of Feminism, a 35-person seminar-style course, at Michigan State University. Both courses were taught online due to the pandemic. The assignment was worth a total of 20% of the final grade: 5% for a proposal, and 15% for the final product.

The central task of the assignment is for students to demonstrate in-depth understanding and critical engagement with concepts from one or more course texts. The specific topic of the assignment is left open; students may choose any concept, theory, or text from the course syllabus. They are encouraged to choose a topic that will be useful or interesting to others, and asked to identify this potential audience and consider how to effectively package the ideas to reach them. They are not required to actually attempt to reach this audience, but considering how to do so is a useful heuristic for thinking about the pros and cons of different formats.

Students can choose any format they wish to achieve this task. Example formats such as slide presentation with voiceover, podcast, video, op-ed, and website are provided in the assignment instructions, but students may propose alternatives. I have received a wide variety of creative formats, including short animated videos, infographics, interactive websites, TikTok video series, comic strips, photo essays, visual art, and short pieces of fiction. I encourage students to think about which formats will enable them to get their ideas across in a clear and engaging way and how to make effective use of their chosen format. For example, students who choose slide presentations should think about how to use images and graphics to help them get their ideas across rather than just relying on text or voice-over. Students who choose traditional essays are encouraged to take advantage of the depth and detail that the format allows for. 

I do not provide explicit instruction on how to use different formats or necessary technologies. It is up to students to learn the technology they want to use if they are not already familiar. However, I do provide extensive resources to students to support this. I liaise with university librarians to provide resource guides with links to online tutorials and how-tos for common formats. When possible, I have a librarian visit class to discuss online and on-campus resources with students–such as borrowable recording equipment or editing software–and, when feasible, to offer one-on-one support. 

Students are required to submit their projects electronically through the course platform. Where appropriate, links to externally-hosted content can be submitted instead of files; for instance, videos can be uploaded to MSU’s Kaltura Mediaspace or Youtube and podcast files to Soundcloud. In the past I have also accepted physical versions of posters or other project formats, but the online submission is more streamlined and enables easier evaluation. 

To encourage students to get the projects started in advance and to head off common problems, I require a proposal about a month prior to the project due date. Students must name and briefly describe their chosen topic, relevant course materials, and the intended audience for their work. They must identify their chosen format and explain why it would be an effective means for reaching that audience. The proposal also asks if the student will need help with their chosen format, which is an opportunity to provide further resources if needed and/or to encourage students to select a format that will be feasible given their access to technology and the time they have to dedicate to the project. 

I give personalized feedback on the proposals, and students are required to respond to this feedback via email or in a meeting. Common issues that can be addressed at this stage include overly broad or ambitious topics, inappropriate length or size of project, or basic misunderstandings of the chosen topic. The proposal process is time-consuming, but it reduces final grading time by improving the quality of the projects overall and helps students feel confident in their chosen format and topic. In an online pandemic teaching context, it was also a valuable opportunity to connect with students one-on-one and learn more about their interests. I use a few strategies to minimize the workload of the proposal process, such as “project workshops”: class time when my teaching assistant and I are available to discuss proposals and give feedback on drafts and outlines. Next time I will incorporate recommended length or size for common formats and detailed submission instructions on the assignment sheet to further reduce the labor involved in feedback. 

The final project is graded according to a rubric. The rubric is a key aspect of this assignment and ensures consistency in grading across various formats. I make the rubric available early in the semester and encourage students to refer to it while they are working on their projects. Two-thirds of the grade is awarded for demonstrating a thorough and insightful understanding of chosen course concepts. One-sixth of the grade is awarded for use of format along with grammar, spelling, and citations. Another one-sixth is awarded for appropriate choice of topic, meant to reward students for following through on approved topics and reinforce that they must stick to course concepts and texts rather than doing outside research. This distribution of points frames format as a more or less transparent tool for communicating course content. Importantly, it limits the number of points that can be lost for clumsy uses of format that do not undermine communication of key concepts, such as awkward transitions in a podcast episode or poor video editing, for instance. 

Final Project Rubric:

Criteria	Excellent	Very Good	Satisfactory	Not up to standards
Topic	2.5 points Appropriate choice of topic, project shows excellent understanding of the topic and its relevance to the course	2 points Appropriate choice of topic, project shows good understanding of the topic and its relevance to the course, some errors, omissions, or inconsistencies	1.5 points Appropriate choice of topic, project shows some understanding of the topic and its relevance to the course, some significant errors, omissions, or inconsistencies	1 point Topic is inappropriate for the assignment, project shows poor understanding of the topic and its relevance to the course, major errors or inconsistencies
Engagement with concepts from course	10 points Project shows excellent understanding of key concepts and familiarity with relevant texts, makes thorough and insightful use of concepts	8 points Project shows good understanding of key concepts and familiarity with relevant texts, some small errors, omissions, or inconsistencies, makes good use of concepts	7 points Project shows some understanding of key concepts and familiarity with relevant texts, some errors, omissions, or inconsistencies, makes some attempt to use concepts	5 points Project shows poor understanding of key concepts and/or no familiarity with relevant texts, major errors or inconsistencies, makes poor use of concepts
Citations, spelling, grammar, format, style	2.5 points Project is nearly flawless, follows directions precisely, makes excellent use of the chosen format to communicate ideas, citations are complete and in an appropriate format	2 points Project contains a few mistakes, no mistakes that impede meaning, mostly follows directions, makes good use of chosen format to communicate ideas, citations are mostly complete and in proper format	1.5 points Project contains some mistakes that impede meaning, mostly follows directions, makes some use of chosen format to communicate ideas, citations are missing key info or formatting	1 point Project contains many mistakes that impede meaning, does not follow instructions, makes poor use of chosen format to communicate ideas, citations are missing or are not formatted properly

Suggestions for Incorporating Open Format Assignments in World Language Courses

While the examples we showcased here are from courses taught in the English language, open format assignments can be easily adapted for and provide several benefits to world language learning contexts. Not unlike other assessment decisions you make, your open format assignments will depend on your learning outcomes and course context. 

In higher-level world language courses or seminars focused on literature or other content, or for language courses with a significant number of learning outcomes focused on culture rather than discrete language skills, the opportunities for open format engagement seem limitless. Very often, writing skills like genre or style do not need to be restricted to traditional text-based modalities. A comparative analysis of literature, for example, can be achieved multiple ways, as long as the comparison itself is paramount in your assessment. Even such skills as writing for coherence or cohesiveness or demonstrable vocabulary use need not be fettered to traditional modalities.  

In lower-level language courses, it may be more challenging to engage in wholesale open format assessments because genre and mode are often prescribed within the learning objectives (e.g., students need to be able to write an email, and their practice should be done as authentically as possible within the appropriate mode – an email platform). However, the spirit of open assessment, that is offering students choice when appropriate, can easily be achieved on a smaller scale. You might offer a more controlled assignment than ours by limiting format options to those that make the most sense for your learning goals. Giving students even a small menu of choices gives students the opportunity to demonstrate what they have learned in ways that appeal to them and play to their strengths.

We recognize that a rubric used for the evaluation of specific productive skills (i.e., speaking or writing) would necessarily be focused on the skill(s) being assessed. For example, some learning outcomes related to certain speaking skills may not be effectively assessed through written work (e.g., pronunciation, certain types of fluency, etc.). However, it might sometimes be possible to build in choice related to the format that speaking output can take: perhaps it doesn’t matter to the essence of the learning outcome whether or not students engage in spontaneous speech, conduct an interview, or prepare a more traditional presentation. Giving students choice in how they demonstrate their speaking skills and allowing them to play to their strengths can mitigate the effects of anxiety which so often plagues many language learners (e.g., Horwitz, 2010). Spontaneous speech can also be anxiety-inducing for autistic language learners (Garrity et al., 2018). In the case of writing, audience matters. But, you may find in revisiting your learning outcomes that students could choose from multiple formats within a single genre or rhetorical style on a given assignment. In process writing, for example, you might encourage students to choose mind-mapping applications to demonstrate their outlining stage as an alternative to traditional paper and pencil or word processing formats.

Conclusion 

As noted throughout our narratives, open format assignments do present certain challenges. For example, some students may feel lost without examples or produce projects of very different lengths or sizes. We’ve suggested some ways to mitigate these issues, and have found that overall the benefits of these assignments greatly outweigh these limitations. Open format assignments are a highly flexible and accessible option appropriate for many learning contexts. We found that it made grading less tedious for us and required little familiarity on our part with specific software or technology. Most importantly, they provide significant benefits for students. For students, open format assignments: 

1. Provide an opportunity to make use of existing technological knowledge/skills, or to learn a new technology they are interested in;
2. Allow students to avoid certain platforms or technologies entirely if uninteresting or inappropriate for them;
3. Encourage student autonomy and ownership over their work;
4. Encourage creative engagement with course material;
5. Accommodate different learning styles or preferences;
6. Create flexibility for students with disabilities;
7. Accommodate differential access to resources.

Acknowledgements

Caitlin would like to thank the students who joined the 5-week course and were so generous in sharing their work and evaluations of the assessment design, as well as Dr. Anna Pegler-Gordon who mentored the development of the course. Megan would like to thank her former teaching mentor Professor Kate Withy, who encouraged the development of the open-format assignment, and A.Y. Odedeyi, teaching assistant for Spring 2021 Ethics of Health Care.

References

CAST (2018). Universal Design for Learning Guidelines version 2.2. Retrieved from http://udlguidelines.cast.org

Garrity, M., McGlowan, T., Chen, S., Wall, J., Alonso, M. R., Mia, L., Vivien, N., & Caldwell-Harris, C. (2018). Adults with autism discuss their experiences of foreign language learning: an exploration of the “different strategies” hypothesis. Retrieved from https://hdl.handle.net/2144/37459

Haynes, J. (2017). 3 New Assessment Strategies for Testing in Higher Ed. Top Hat, October 16, 2017. Retrieved from https://tophat.com/blog/assessment-strategies-testing-higher-ed/.

Horwitz, E. K. (2010). Foreign and second language anxiety. Language teaching, 43(2), 154.