Written by Mona Maxwell, Senior Instructional Designer

Edited by Nancy Fischer and Erin Thomas

Perhaps this winter has provided you with an opportunity to make soup “from scratch” or to sneak cut vegetables into an existing can of Campbell’s soup. Regardless of your level of love or distaste for soup, join me in a soup-making analogy to help frame your thinking around two tips to increase validity and learner success with summative assessments.

Tip #1: Serving of the Soup should be Preceded by Sampling of the Soup

Summative assessment preceded by formative assessment often results in better soup – I mean, more learner success on summative assessments. 

What are Summative and Formative Assessments?

Formative assessment – the act of monitoring student learning and providing ongoing feedback – is like sampling the soup to get feedback on what it needs more or less of.  We can make incremental improvements based on formative assessment of the soup’s taste, color, and texture. To do this, it helps to know what criteria defines a soup as “fantastic” in the particular circle of people consuming the soup.

Summative assessment – the act of evaluating what has been learned, usually for a reporting purpose in the higher education context – is like serving the soup.  This is the “grand finale” when we may receive feedback again – not for the intent of improving this particular pot of soup, but for improving the next pot of soup.

Effective Feedback on Formative Assessments

Feedback as to whether you need more basil or less cooking time can come from your own judgement, from other trusted household members, or your instructor if you are in culinary arts school!

Regardless of the source of the feedback, effective feedback is likely based on pre-determined criteria that may be written or passed down orally or gleaned from past experiences. The opportunity to “check in”, provided by formative feedback, enables one to improve in a low-risk environment and ideally, enjoy the satisfaction of those incremental improvements on summative assessments. You can determine what steps (if any) are needed to achieve mastery. 

Similarly, learners can be given a chance to self-assess themselves against a checklist or a rubric related to what makes their research paper or laboratory report “good”. Many learning management systems (LMS) offer a rubric tool to specify criteria for “good”, “better”, and “best” as well as self-assessment tools that provide immediate feedback to learners without any numeric evaluation. Some LMS platforms may provide similar options such as a questionnaire tool.

A quiz tool can also provide formative feedback using a numeric evaluation. These numeric results do not need to influence the final mark in the course to have a positive impact on learning.  One study involving 471 health sciences students attending classes both online and face-to-face reported that

“…a significant positive relationship was found between final grade achieved and percentage of self-tests attempted. This relationship was significant regardless of study status (on-site or distance), course studied or total activity logged”.

(Thomas et al., 2017)

Instructor and Learner Benefits

Instructors can also benefit from the result of a formative assessment. For example, instructors might proactively identify areas in need of further reinforcement if numeric results are low on a particular set of questions. The timely source of data provided by formative assessments can help to provide effective further practice if needed by particular individuals. One can even use the results of formative assessment to make changes in the course design for future offerings of the course. If learners often seem to systematically produce soup that is too salty, they might benefit from a new recipe!

Given an opportunity to provide feedback to each other, learners can also benefit from rating a peers’ work against criteria on a rubric. Check whether your learning organization’s LMS has tools for enabled for peer assessment.

If your next step is to decide which tool in a learning management system helps instructors and learners “taste the soup” (formative assessment), feel confident that the tools in your LMS may be able to accommodate your particular context.  Most learning management systems allow the instructor choices on the following:

• Whether feedback is…
  • Instantaneously released to the learner upon completion of each question of an assessment, or
  • Released after the completion of an entire assessment in the form of a report
• Whether instructors can view…
  • Scores on all answers for all students, or
  • Anonymized scores of all students
• Whether it is desirable to…
  • Have all students view the answers of all others (as in a survey, for example), or
  • Have student answers completely private.

Contact an instructional designer or another staff member at your institution to explore ways that the various tools can be used to accomplish your goals.

Tip #2: Keep the Flies Out of the Soup

Dare I say that no one wants to be distracted by a fly in their soup!  Similarly, preparing students for summative assessments can alleviate any distracting elements that prevent a valid assessment of the soup itself.

Communicate overtly about the intent of assessments, both formative and summative.  Would you consider using this soup analogy to describe to learners the intent of your formative and summative assessment? I have had some success with it. 

Swat the Flies Away

I have a few suggestions that you can follow to ensure that external factors (flies) do not interfere with your assessments:

First, provide specific information about the format of the summative assessment in the course syllabus or course information module online. For example: Will it be timed? What materials, if any, are allowable for use during the summative assessment?

Also, provide links or advice on where students can find specific support for skills that may be assumed, such as researching skills and writing skills. Your campus may have many untapped supports for them! For project work in particular, ensuring that access to appropriate presentation software or providing presentation ideas is helpful, especially in an online environment. You may not need to write these supports yourself! Reach out to your learning organization and you may be surprised at what already exists. Simply encouraging students to share among themselves can yield a soup bowl full of great ideas! I have learned much about presentation software and the pros and cons of each one through my students and my children!

Eliminate a Few More Flies

Specific to online summative assessments, one of the distracting “flies” can be the learning management system itself. When possible, students should be relieved of any concerns about the format of the tool they will be using for summative assessments. For example, providing specific information about the quiz details and the question types that students will experience on a summative assessment can be very helpful. One way to efficiently provide this support is by creating a sample quiz in your course. By doing so, the level of comfort with the LMS is less likely to be an impediment that causes assessment results to be less valid.

For example, as shown below, the quiz instructions and description (as they appear in the LMS) should be demonstrated well in advance of a summative assessment. Typical information that is seen in a learning management system quiz can be found below:

Quiz Details

Current Time: 10:11 am

Time Allowed: 2 hours

          Attempts: Allowed – 1 Completed 0

Introduction: The introduction is seen on the page before a student begins the quiz.  Provide reminders such as supplies allowed, whether navigation to previous questions will be possible or not, and/or how/when/whether the results of the assessment will be available to the learner.

Instructions: Instructions are also seen on the page before a learner begins the assessment. Provide helpful information such as what happens when the allowed time has expired and whether there is a pre-warning about remaining time left for the quiz.

Click “start” to begin the quiz. The timer will not begin until after you have pressed start and the start-up process is finished. 

A sample quiz also allows the variety of types of questions be explored in advance of the summative assessment. For example, will there be true/false questions, multiple choice questions, or will there be a text box to provide a short answer of a limited number of words? Sample quizzes work best when you include every question type that you plan to use.

Again, this may not require too much additional effort on the part of instructors.  Your organization may have access to sample quizzes for you to import into your LMS in a matter of minutes. 

Using these strategies to limit the number of “flies” and the probability that one will land in your soup can yield more valid results of your summative assessments.

“Soup for Thought”

I hope that preceding the serving of the soup with one or more soup samplings as well as keeping those distracting “flies” out of your soup can at least provide some “soup” for thought. The goal is to increase validity of summative assessments and increase learner success.

References

Thomas, J. A., Wadsworth, D., Jin, Y., Clarke, J., Page, R. & Thunders, M. (2017). Engagement with online self-tests as a predictor of student success, Higher Education Research & Development, 36:5, 1061-1071, DOI: 10.1080/07294360.2016.1263827