Introduction

This document lists the recommended steps to run an electronic OSCE exam throughout the Preparing the Exam, Running the Exam and Completing the Exam phases. Because the process of evaluating the exam results is highly specific to each institution, this document contains no recommendations on that subject.

Vocabulary

  • Candidate: the student taking the exam
  • Examiner: a qualified doctor assessing the candidates
  • Rotation: a series of stations a candidate will rotate through. No student will participate in more than one rotation. There are usually multiple rotations in an exam day. Normally a rotation will be followed by a rotation-break.
  • Station: the place in which an examiner will usually assess the candidate’s performance according to a checklist.
  • Break–Station: a station in which the candidates aren’t assessed but rotate through. There’s no examiner in a break-station.
  • Break-Candidate: an imaginary candidate rotating through the stations. in this case the examiners won’t have anybody to assess at the time the break-candidate is in their station.
  • Course: if parallel rotations take place (e.g. two rotations at the same time) they are named “rotation 1: blue”, rotation 1: yellow” etc… Usually each course is on a different floor.

Planning the Exam

The following section list the tasks you ought to complete before running an EOSCE examination. The tasks are structured by date you ought to have completed them: “3 months before the exam”, “1 month before the exam”, “2 weeks before the exam”, “1 week before the exam” and “The day before the exam”.

A) 3 months before the exam

  • Familiarize yourself with the IT-infrastructure (internet access, private WiFi) available at your examination premises.
  • Gather as much information as possible about the upcoming exam (e.g. the number of days, stations, rotations, course, timetable etc.)
  • Ensure you are sufficiently equipped with iPads and organize additional devices if necessary (remember to factor in spare devices as well).
  • Schedule a test to ensure the iPads will be able to connect to the Internet at the examination premises. Involve University-IT services if necessary.
  • Test for working Internet connectivity using a few iPad devices by trying out both HTTP and HTTPS connections. If necessary repeat until devices can access the Internet.

B) 1 month before the exam

  • Ensure that the information about your examination (e.g. number of days, stations, rotations, course, timetable etc.) is still correct or reconsider hardware requirements (see above).
  • Create a Amazon Web Services (if necessary). This will require a valid credit card.
  • Create a Amazon S3 Bucket for your examination or reuse an existing bucket. The encrypted examination backups will be stored to this bucket.
  • Organize transport to exam premises for all iPad devices on the day of the exam (if necessary).
  • Contact examination administration to reserve a spare room on the examination floor for the EOSCE team. Remember to organize a key for the room as well.
  • Prepare documents to inform examiners and students about the usage of EOSCE at their exam. For the examiners this will include workshop-dates on which they can learn how to use the app. Students might have questions as well.
  • Schedule & prepare a training workshop for the examiners using the sample checklists.
  • Configure a few iPads with Amazon S3 credentials (using OSCE-Editor and the QR-Code) and make sure to confirm the cloud-storage access works. You may use the OSCE-Eval example-files and OSCE-Track for this.

C) 2 weeks before the exam

  • Set aside all EOSCE equipment (iPads & AC-Adapter, MacBook & Bretford PowerSync Cart etc.) for the exam.
  • Get the detailed exam-information (names of all the examiners, checklists, a list of all students, timetable & rotation plan etc.)
  • Create the electronic checklists by using the OSCE-Editor
  • Copy all electronic checklists to an iPad and make sure that all items can be selected and all titles can be read.
  • Create EOSCE learning material to train the examiners (if needed). Also inform all other stakeholders about your plans to use EOSCE.

D) 1 week before the exam

  • Configure all iPads you will be using in this exam (+ the spare devices)
  • Update checklists with latest changes using OSCE-Editor and check if the changes look good on the devices (if necessary)
  • Prepare and set aside several iPads to train the examiners (use the example-files in OSCE-Eval for this purpose)
  • Create a list of all iPads used in the examination (including their ID, inventory number) and assign each iPad to an examiner.
  • Prepare stickers for each iPad and examination day containing the examiner name, the assigned station, course, room etc…
  • Prepare paper receipts for the examiners to sign when they receive and return their iPads. These receipts can be printed directly from the app and should include the iPads public key. Beware: reinstalling the app on the device will generate a new public key.

E) The day before the exam

  • Hold app-training workshop for new examiners (if necessary)
  • Finish the configuration for the iPads
  • Reconfirm that the iPads can connect to the exam server (both WiFi and cloud storage connectivity). You may not need to test all devices. But make sure to confirm the settings haven’t changed.
  • Label each iPad using the sticker prepared earlier (containing the examiners name, station, etc…)
  • Prepare iPads for each examiner by downloading their checklists. You may need to open each examination template file and select the correct examiner to create the examiners’ working copy.
  • Charge all iPads overnight (including any spare devices)

Running the Exam

A) On Each day BEFORE the exam starts

  • Introduce the examiners to the OSCE-Eval app and remind previous examiners of the process (if needed)
  • Hand out the iPads + AC-adapters to the corresponding examiners. Alternatively, distribute them by leaving them in the correct station.
  • Make sure all examiners are ready to use the iPads and be ready to answer any last-minute questions examiners might still have.

B) While the exam is running: During Rotations

  • Use OSCE-Track to monitor the state of the exam. It will tell you when the last back-ups was uploaded to the exam server. As long as all iPads are frequently backing up to the exam server you have nothing to worry about.
  • Ensure that all examiners have a back-up on the exam server. If the back-up of one or more examiners are missing: take notes and visit that station during the next break.
  • Ensure that you can be easily reached by the administrative personal in case of emergencies (especially when the candidates rotate)
  • Have your spare devices ready and easily accessible
  • Make sure you know how to react in the different error scenarios that may occur during the examination

C) While the exam is running: During Rotation–Breaks

  • Use OSCE-Track to ensure examiners have finalized their rotations.
  • Walk through all stations and check if the iPads are plugged in (iPads should have enough battery life to get you through the day, but better be safe than sorry).
  • If you have devices that haven’t synced to the server, make sure to make a back-up of that device using iTunes.
  • Use OSCE-Track to download all rotation data to your device (for back-up reasons). Alternatively, use an S3-Client (e.g. Cyberduck) to download all files to a hard drive.

D) After the exam has finished for the day

  • Use OSCE-Track to ensure all rotations have been finalized and ask examiners to finalize their rotations if necessary.
  • Collect all iPads and ask examiners to sign the sign-in/sign-out receipt (if needed)
  • Use OSCE-Track to download all examination data. Connect and sync the device used for this to a computer and back up your data. Check if all examination excel sheets look complete.
  • Connect all iPads to iTunes and copy all files on the device to the computer. Delete them after the transfer has completed successfully. Your iPads should now be ready for you to start downloading the examination files for the next exam day (if necessary).
  • Copy the examination results excel-sheets to a separate folder (to hand over to the evaluation-team).
  • See steps described in “Until the day before the test” to prepare the iPads for the next exam day (if applicable)

Finishing the exam

  • Copy all examination result excel-sheets into one folder and hand them over to the evaluation team.
  • Burn all back-ups of each examination day onto DVDs and keep safe. Alternatively, use memory sticks (but the former have the benefit of being write-once).
  • Verify your back-ups are readable and contain all the data you require (this is important). And now verify again!
  • Archive the paper receipts signed by the examiners (you may want to scan them to do away with the paper).
  • ONLY AFTER having verified your backups: delete the examination data from the examination server (e.g. by using Cyberduck), delete all data from the devices (e.g. by using iTunes) and delete all examination data from the laptop used to sync the devices.
  • Remove the examination-sticker from this day from every iPad
  • Restore the iPads to the factory settings (you only need to do this if the devices are used for purposes other than EOSCE). See “After the Exam is before the exam” for more information on how to do this.
  • Turn off iPads and lock them away securely

After the Exam is before the exam

After making sure that all on the iPad device generated examination-data (excel-files) is backed up on an additional disk, they can be deleted on the device. Since we recommend to reset every device before the next exam anyway, the deletion of this files can be taken by the reset process itself:

Resetting iPads to the factory settings:

  • Directly on the iPad: “Settings” > “General” > “Reset” > “Erase all Content and Settings”
  • Using iTunes: “Restore iPad or define as new iPad”
[/fusion_builder_column][/fusion_builder_row][/fusion_builder_container]