Practice guidelines for evaluation, examination and continuous assessment of student knowledge during online education

Background from the survey

Background from teachers

The questions related to the examinations and continuous assessment of the students were frequently skipped by the teachers. Even some of those who answered the questions expressed the approach that “this doesn´t apply to them” or had no specific needs. Many of the teachers thus seem to have only limited experience with the online assessment.

The needs regarding online examination and continuous assessment of students' knowledge in the survey frequently pointed out the preference of the personal contact during the exam (about 1/3 of the teachers who answered this topic) and the concern about the limited objectivity of the online form of examination.

The most focused needs regarding online examination and continuous assessment of students' knowledge in the online space were following:

• uniform rules of examination at the faculty
• training for teachers
• to prevent opening webpages and study materials
• eliminate unauthorized tools by students
• the ability to see students' activities during the exam
• better equipment, better internet connection
• mandatory use of the camera
• unified assessment platform
• list of available platforms for creating test materials or quizzes (MS Forms, LMS Moodle, Kahoot etc.)
  • A software for creating quizzes, tests, interactivity have been used at least once a month (or more frequently, even “on daily basis”) by 54 % of responding teachers (and this particular question was answered only by slightly more than a half of the teachers). Seven percent of them used such a software utility only once a semester. On the other hand, it has never been used by 33% of the teachers and the use of similar software tools was evaluated as “not suitable for their teaching” by 6% of them.
• larger base of questions
• central database of questions
• display only one question without returning to the previous question
• possibility to set custom test conditions

Most of the recommendations provided by the teachers who answered the questions related to the examinations and continuous assessment of the students was focused on the prevention of cheating and ensuring the objectivity, since the teachers are mostly afraid of cheating and some of them point out the low objectivity of online exams.

The most focused recommendations from the teachers included:

• verify student identity
• see not only the student but also his screen
• elimination of the use of unauthorized means
• immediate answers to questions, exam in the form of a dialogue on a given topic
• do not allow getting back to previous questions
• share problems with each other and solve them together
• uniform rules of examination at the faculty
• small tests or quizzes during and after the lectures are also mentioned as one of the tools for the motivation of students during the online education

Problems with tests and shortcomings in creating tests are also pointed out among the experiences with online education.

Quite an interesting point mentioned in the topic of students possibly disadvantaged in the online education era: “cheating on online testing puts decent and fair students at a disadvantage“.

Background from students

There were no questions focused on the online testing and examinations in the student version of the survey. In some opened questions, however, some students commented on the online testing or exams. The relevant comments were the following:

• in the sections focused on bad experiences with online education and negatives of this form of education, the students mentioned:
  • no technical support during online exams
  • time to wait for the test
  • more cheating
  • unfair exams
  • stress during tests
  • unaccepted test results in case of late submission
• in the section focused on good experiences with online education, the students mentioned:
  • tests in MS Forms
  • organisation of exams
• in the section summarizing the student´s needs regarding the online education, they pointed out:
  • minimizing the amount of tests or even no tests during lectures but on the other hand also continuous revision of knowledge
  • possibility of going to previous questions during online tests
  • more time for the tests
  • easier exams
  • exam system based on oral communication
• in the section focused on the preferred forms of online education (“others”):
  • online quizzes each week

Summary of the background information

To sum it up, the teachers point out the importance of unified examination rules in the faculty and teacher´s training in the online examination and testing. Sharing the problems with other teachers and solving them together may also help on this point. They put the major stress on the prevention of cheating: the verification of student´s identity, the elimination of the use of unauthorized tools by students (opening webpages or study materials during the oral or written exam) by monitoring the student by mandatory use of camera and sharing the students screen. In oral exams, they recommend the form of immediate answers to questions, i.e. exam in the form of a dialogue on a given topic. In written tests, they recommend the existence of unified testing platform and the use of larger and possibly unified test database. They further emphasize the possibility to set custom test conditions, e.g. the possibility to display only one question without returning to the previous question. It´s important to set the conditions in the way that doesn´t disadvantages the decent and fair students not trying to cheat.

The students seem to be opened to the online testing and exams and some of them consider such a type of knowledge of their control as an advantage of the online education. Some of them might, however, be stressed by the online tests. Similarly to the situation in the contact education, some students prefer the oral exams in the while the others find the written tests to be more suitable. Fully in this line, some of them recommend the reduction or cessation of the online testing, while the others prefer regular revision of the knowledge by the online tests or quizzes for keeping attention and for the motivation of the students. They appreciate easier exams and more time for the tests in the online setting. From the student´s perspective, it´s also important to set the exam or test rules to be fair and clearly pre-defined. In contrast to the teacher´s recommendations, they appreciate the possibility to go back to the previous questions during online tests.

For both sides (students and teachers), the use of online testing platforms seem to bring positive experience.

Both sides emphasize also the technical requirements for efficient online testing and examinations: the sufficient internet connection and equipment, availability of technical support during online testing. The later submission of the tests due to the technical problems may represent a complication of student´s site and should be solved technically.

Neither the teachers not the students addressed the topic of recording the online exams or tests in their survey.

Practice guidelines for evaluation, examination and continuous assessment of student knowledge during online education

General recommendations on the online assessment

• The testing and examination of students has to be performed with the respect to the requirements of the law, relevant government measures and internal regulations of the university/ faculty. Furthermore, it´s necessary to respect the specifics of faculties and individual programmes which must have the possibility to define their needs in terms of requirements for the level of knowledge and form of testing or examinations.
• The conditions of continuous assessment and written or oral exams should always be set in the way that provides equal conditions for all students across course and doesn´t put non-cheating students at a disadvantage.
• Limited access to some parts of the education process (e.g. practice) in the era of full transition to online education should be considered by teachers during the setting of assessment/ exam requirements in particular course. These requirements should thus be modified accordingly in relevant cases.
• It should always be verified whether the conditions of written and oral examination of students with special needs comply with such needs, and the adequate technical and organizational adaptations need to be performed, if necessary.
• Before the beginning of the testing, the technical means used during the exam/ test should be checked.
• The teachers and students are encouraged to use the assistance of technicians and user support teams.
• The teachers are encouraged to share their experiences with the online assessment with the others for further improvement of the online testing/examination methods.

Online assessment recordings

• In general, it´s recommended to make an official recording of oral and written examinations. Furthermore, there might be specific rules for recording the examinations /tests in particular universities/ faculties based on the decisions of their rectors/deans. The university/ faculty rules should contain among others:
  • the manner or instrument for making the recordings
  • the group of persons entitled to handle them
  • the storage place for keeping the recordings
  • the storage period. In Czech conditions, it is recommended that the recordings should be kept for the minimum period of 30 days of the date of the oral examination assessment, during which the student may lodge a request for the assessment review. This period may, however, differ in different countries and should be adapted according to the local rules. For example, audio recordings of state exams at UPJS (Slovakia) organised online during pandemic have to be archived three months since the end of the crisis situation.
• The purpose of such recordings is an unbiased processing of:
  • a request for the review of the assessment or
  • a dispute connected with the use of unauthorized recordings
• The consent of the student or the examiner to making the recording is not required; however, the student must be informed that the recording is to be made.
• The students must be informed that their own recording the exam is not acceptable.

Preparation for examinations

• The adequate form (e.g. written testing or an oral exam or their combination) and the environment must be selected first.
  • The choice of the adequate form (e.g. written testing or an oral exam or their combination) depends mainly on the type of the course:
    • Oral examinations may be more suitable for courses focused on knowledge acquisition; an interview with a specific topic and targeted questions may significantly reduce the chances of cheating. Some teachers examine two students at a time and let them interact.
    • Among written exam forms, the „open-book exams“ represent the most relevant and highly recommended option for online assessment. This type of exam doesn´t test only the knowledge acquired, but also the student´s ability to find and apply information. For the purpose of this exam, students are allowed to use their written notes, or even other texts or books according to the rules set by the teacher.
    • The simple written testing of knowledge is less suitable for the final exams performed in the online mode (but might be appropriate for continuous assessment).
  • The choice of the appropriate environment:
    • For oral examining, the use of online communication platforms allowing secure online meetings with a high-quality video and audio connection and instant screen sharing is recommended. The most frequently used platforms for this purpose are currently MS Teams and Zoom, but there are many others available and the teachers should follow the recommendation of their university/faculty concerning the preferred platform to be used (since the preferred platform is usually available for the employees and the technical support is provided).
    • For written tests (including those performed during continuous assessment of the students), the teachers may use:
      • Specific platforms for online testing provided by particular universities/faculties (an available technical support and relevant protection from cheating represent major advantage of such a type of platforms) (KvIS in Masaryk University may represent a good example).
      • Public platforms for creating test materials (MS Forms, LMS Moodle, Kahoot, Quizizz) mainly if no specific platform is available in particular university/ faculty.
    • Students must be informed properly and provably of changes in the requirements for the completion of a course (methods and manner of evaluation) and manner and procedure of online testing at least few weeks before the start of the exam according to the specific rules in particular university/ faculty (e.g. at least 3 weeks before the first date of the examination).
    • For other oral examinations held online with the use of video conferencing technologies, a longer time may be necessary (the teachers should count with at least 5 minutes before for the potential solution of minor technical issues and sufficient time for the exam itself).
    • The teachers should determine in advance the method of resolving potential technical difficulties that will lead to the interruption of the exam (loss of connection etc.) and should inform the students of the method.

Recommendations on continuous assessment

• Continuous assessment, i.e. the evaluation of a student´s progress throughout a course of study, as distinct from by examination, represents an important part of online education. Among others, the small tests or quizzes during and after the lectures may serve and an important tool for keeping the attention and motivation of students during the online education.
• If the open book type exam is planned, the students should become familiar with this type of testing already during the course and should be aware of the type of the questions they may face in the exam.
• For written tests or quizzes used for continuous assessment, the teachers may use:
  • Specific platforms for online testing provided by particular universities/faculties (an available technical support and relevant protection from cheating represent major advantage of such a type of platforms) (KvIS in Masaryk university may represent a good example).
  • Public platforms for creating test materials (MS Forms, LMS Moodle, Kahoot, Quizizz) mainly if no specific platform is available in particular university/ faculty.
• Larger database of questions is highly recommended since it is technically impossible to prevent the disclosure of the test questions to students who will pass the assessment in later days (and the test in later days should thus change at least partly).
• The recommended setting is to display only one question without returning to the previous one.
• The time for taking the test and the total number of resits should be limited.
• Another solution is to determine a very large number of questions (that will disable answering them all) and make the evaluation according to the results. Such an approach limits the undesired cooperation among students due to the lack of time, and natural competitiveness.

Recommendations on oral exams

• Examined students must prove their identity (e.g. by showing their ISIC or identity card to the camera)
• The students must be notified that their recording the exam is not acceptable and told what the procedure will be in case the connection is lost (see above).
• The most relevant form of online exam is the dialogue on a given topic, i.e. the form of immediate answers to questions which may prevent cheating.
• Inspecting the student’s environment or fairness of conditions of the oral examination are not supported in terms of technology, ethics or law. Teachers are recommended to modify the questions and process of the exam so that it is not essential for the evaluation whether the student might find facts on the internet or in his/her own notes.
• Video conferencing software enables sharing the student’s desktop and his/her procedure of searching the relevant information; the teacher may then evaluate for instance such aspects as the correct choice of electronic sources, their relevance and trustworthiness, and discuss the choices with the student.
• Some universities may use the proctoring systems for the direct monitoring of the online examination (software for monitoring the student’s eye movements, his/her typing on the keyboard, noises in the microphone, blocking the student’s computer and mobile phone). For many universities, however, this approach is not a path that they might want to take.

Recommendation on written exams

• Examined students must prove their identity (e.g. by showing their ISIC or identity card to the camera). The identification of students should, however, be made individually (e.g. in “Breakout Rooms”) to prevent compromising personal data.
• It´s recommended to prefer the “Open Book” form of final exam if possible.
  • For the purpose of this exam, students are allowed to use their written notes, or even other texts or books according to the rules set by the teacher.
  • Open book exams tend to ask students to apply, analyse, synthesize, compare/contrast or evaluate information. They test whether the student understands whole the course and/or how course concepts work together. It´s thus the exam form which allows to test the way the student thinks about the topic, how he/she can connect it into broader units within the subject, and where he/she sees overlaps with other subjects.
  • The student is able to find the relevant information in the open source only if he/she previously studied the book in detail.
  • The students should be trained for this type of testing during the course.
  • There are, however, fields of study where this type of exam is more natural to apply (fields of social sciences).
• The simple written testing might be less suitable for online form of written exams:
  • The teachers should be aware that it is technically impossible to prevent the disclosure of the test questions to students who will pass the exam in later days. It´s thus recommended to change the set of questions among the examination terms, i.e, to have larger database of questions.
  • It´s recommended to limit the time for taking the test and the total number of resits.
  • Another solution is to determine a very large number of questions (that will disable answering them all) and make the evaluation according to the results, in line with the European Credit Transfer and Accumulation System (ECTS). Such an approach limits the undesired cooperation among students due to the lack of time, and natural competitiveness.
• Inspecting the student’s environment or fairness of conditions of the written examination are not supported due to technological, ethical and legal reasons. Preventing the examined student’s use of the internet is not feasible.
• It is advisable to maintain online connection with students also during a written test. Their questions and problems can be discussed in the chat of the meeting (so as not to disturb the other students); the teacher can also have a private chat with the student by means of “Breakout Rooms”.