SLIDE 1 Question Answer Can tables be attached as a picture? Yes, images of tables may be incorporated into your exam questions as either in-line images or attachments. Can students submit their hand writing as a photo right after the test? We recommend using the in-software notes feature, as students may type notes securely and directly into the Examplify application during an exam. This mitigates the need for outside paper. Do I have the option of giving both open book or closed book examinations? UWA has the option to deliver any exam as a secure "closed book" or non-secure "open book" exam to deliver the exam in the intended style. Is there an option to randomise question order for the entire exam and response order for MCQs? Yes, question and answer choice randomisation is available. Can UWA add customised categories to match UWA learning outcomes? Can each school have customised categories to match unique learning
Yes, UWA has the ability to create as many customised categories for each department as well as global UWA- centric categories. ExamSoft will partner with UWA to assist with set-up of the UWA hierarchal category structure that best suits the vision for learning assessment. What are the minimum system requirements for ExamMonitor? What if I have a student that does not have a devices that meets these minimum system requirements? ExamMonitor requires a camera and a microphone for use. ExamSoft will provide UWA with minimum system requirements which can be shared with students. How does using ExamSoft and its assessment tools help UWA ensure assessment integrity and prevent a student from using secondary resources, such as phone, tablet, books, etc., or determine if instances such as constantly looking away, during a secure "closed-book" exam? ExamSoft provides the tools empower UWA deter and detect instances of academic dishonestly to help UWA enforce institutional policies regarding academic integrity. ExamMonitor will take a video and audio recording of the
- student. That file is then reviewed by an AI engine and live human invigilator. A report will be sent to UWA faculty to
review and determine if any additional action is required. Can Examplify work on iPad/tablets as many students don't have a PC with or without a camera? Examplify works with Mac, PCs, iPads, and Surface Pro devices. For ExamID and ExamMonitor use, students are required to use a compatible Mac or PC with a microphone and webcam.
SLIDE 2 Can ExamMonitor be used independently of ExamSoft/Examplify? Engineering students need to show working in layout form and use complex / mathematical symbols and exam questions may use diagrams and complex symbols. We have trialed handwritten test/exam solutions which are uploaded using MS Office Lens, but the invigilation
- f the student has to be done manually (Zoom
video of student doing the exam by human). Can ExamMonitor assist in this process? ExamMonitor must be used with Examplify to deliver secure assessments. Show your work functionality, including the ability to display work via a spreadsheet, is on the ExamSoft development team's roadmap with a target implementation by the end of 2020. How does ExamSoft account for changes in student appearance when validating identity? ExamID establishes a baseline image by taking a picture prior to the first high stakes assessment. ExamID is able to discern between nuanced changes to appearance. Does ExamMonitor detect usage of outside devices such as a calculator? ExamMonitor will detect the use of outside devices such as a calculator. What about writing equations and draw diagrams
- n paper, and take photos/scan to submit?
UWA can enable students to use outside resources coupled with ExamMonitor as deemed appropriate. Students can input notes directly into the Examplify software. Any drawn diagrams or written equations may be completed
- n separate paper and UWA decides on how to incorporate outside resources to supplement an exam utilising such
resources. How is this video data managed and stored? What assurances can we give students that this data will be managed appropriately? Please refer to the Student FAQ for more information on data and remote invigilation services. How do students that prefer to write out their notes to work through problems on paper? UWA determines assessment policies as to whether students are permitted to use paper outside of the system and provide permission to do so as deemed appropriate. Student may also directly enter notes in Examplify. What student data is collected? How do you protect student privacy? Student privacy is of paramount importance. ExamSoft collects limited personally identifiable data including first name, last name, email address, userID, and assessment data. Please refer to the Privacy Policy for additional information.
SLIDE 3
Can UWA mark assessments online or offline? UWA faculty may mark exams online or print exams and enter scores into the system. ExamSoft recommends online marking, providing the reporting and analytics to inform learning outcomes and provide tailored student support. Will support/training be provided to UWA faculty to teach creation and management of exams? Yes, ExamSoft has partnered with UWA to create a formal training plan. ExamSoft values the partnership with UWA and will provide training and ongoing support. How accurate is the AI system in detecting abnormal activities? The AI system is very nuanced in detecting abnormal behaviors and activities. To ensure consistent and fair evaluation of behaviours, all AI reports are reviewed by a professional proctor, at which point, a comprehensive report is provided to UWA. How will reports be useful in the event that ExamSoft is only used for high-stakes exams? ExamSoft provides institutional administration the ability to report on course, program, and college-wide metrics. Reports may be customised to view metrics on multiple levels spanning across a course or within a specific program. Reports may be utilised to view how students and the institution's programs are performing across time with robust longitudinal reporting capabilities. Students receive in-depth performance reports upon exam completion with instantaneous marking of objective items. Reports can be customised to include individual performance such as question category performance. Is the software used by AACSB accredited business schools? Yes. In Non-Secure "open book" mode can you allow students to access a website or other specified materials? Yes, exams which are created and posted in the Non-secure format will enable the student to identify resources such as a website, software, or document(s) provided by an instructor. Can ExamSoft system incorporate outside assessments to integrate results with exams delivered through ExamSoft? ExamSoft can only analyse assessments created, delivered, and marked through the ExamSoft platform.
SLIDE 4
How does ExamSoft ensure security of the platform? Our development, engineering, and QA teams routinely conduct vulnerability and security checks on Examplify. A thorough QA process is in place to verify security mechanisms and ensure product stability and reliability. Vulnerability scans and tests are carried out to ensure that the applications are functioning correctly, the security procedures in place remain effective, and that all traffic that is transmitted is encrypted. For secure exams, can students manipulate any of their device setting to circumvent ExamSoft security metrics? Examplify prevents the student from tampering with their device settings and students will not be able to access any assessment details or information outside of the exam's specified timeframe. Does ExamSoft follow industry-standard privacy policies? ExamSoft's Privacy Policy may be viewed here: https://examsoft.com/privacy-policy How does ExamSoft assist with UWA administering exams in an administrative work from home and student take at home environment? ExamSoft's Implementation and training team provide comprehensive resources to ensure adequate training and comprehension of using the ExamSoft platform. Training sessions cover the administrative web portal: question authoring/editing/banking, assessment management, integration & account management, and reporting use. Students participate in a mock exam session during implementation to complete registration and also familiarise themselves with the platform prior to completing a high-stakes summative assessment. Can the software accommodate different time allocations for those students with UniAccess arrangements giving them extra time in exams? Yes, UWA will be able to assign students extra time as a percent (e.g. 150%, 200%, etc.). Extra time accommodations are set at the student profile level and all exams will adhere to this extra time accommodation. How can results be exported? Results may be exported to a .CSV to use the data for analysis or to upload into other software. Is the data collected provided to any third parties? ExamSoft partners with remote invigilation services to analyze AI flagged exam videos.
SLIDE 5
How large are the ExamMonitor recordings? Where are they stored? ExamMonitor recordings are approximately 150 MB for a 60-minute assessment. Videos files are stored locally on the device only until confirmation of successful upload after which those files are deleted from local storage. What types of attachments may be assigned to individual questions or the exam as a whole? Individual questions and the exam paper may have an associated attachment. Compatible attachment types include: image (JPG, JPEG, GIF, PNG), audio (MP3), and video (MP4, MOV, AVI), PDF, and RTF. Are students able to flag items to review later in their exam session? Students will see the flags on the left and will also be able to filter items by the flagged questions. What happens with the students' exam if there is a power outage during the exam or their computer crashes? Students should reconnect their power supply and restart the device. ExamSoft saves exam progress locally to the device every 60 seconds. What if students do not have a device that meets minimum system requirements for using ExamID and ExamMonitor? [UWA response]: Students can apply to access the hardship fund. UWA IT have acquired 500 laptops for students struggling with IT infrastructure. This is something Student Life is working hard on with the guild et al [UWA response]: The Reid, Law and J Robin Warren Libraries remain open with available PCs. These devices do not have cameras or microphones, however they may also be an option for some students. If we email a password to the students, what happens if students share that password with students outside the unit? Only students enrolled in the course can download the exam and the exam cannot be shared as it is indelibly linked to each student's computer upon download. Can you use ExamID and ExamMonitor without locking down the student’s computer? ExamMonitor can only be used on secure assessments and cannot be used on a non-secure exam. Are students able to type or paste special characters in their answers (eg in Essay type), and will the system preserve these accurately? Some special characters are available directly within Examplify for essay responses. Special characters cannot be copy/pasted from outside the system in a secure exam.
SLIDE 6 Does the question bank contain any outside or 3rd party content or is the bank exclusively UWA imported and created questions? The bank will only contain your questions. ExamSoft is the warehouse for them. What is ExamSoft planning to introduce to enhance the platform and support additional specialised department assessment needs? ExamSoft is currently in the process of adding new features such as a rubrics tool and a show your work module. These features are expected to be released later in 2020/2021. What is the earliest we can have students do a mock exam and can we do more than one? ExamSoft training team will work with UWA to determine adequate timing for mock exams prior to actual exams. Multiple mock exams can be provided to students. We could invest a lot of time here in creating Qs and exam papers that can be tracked over time (which is great) - what is the University's plan here? Is this a 2020 solution only or a program we are looking to use into the future as well? ExamSoft has over 1790 clients and a 98% retention rate. It is our expeirene that the data acquired over time delivers invaluable information to positively impact learning outcomes, student retention and accreditation
- preparation. We will defer to UWA regarding plans for partnering with ExamSoft for both immediate needs and
future needs. Can students overseas use this system? e.g. where there may be restrictions on internet access?
- Yes. ExamSoft is used successfully across the globe and there are no known geographic restrictions.
How will the system work with multi-step calculation based assessments, including maths and physics exams? Questions can be grouped together, even if question randomisation is used. Each question within the grouping could be used to show the progression of the problem over time. Can we transfer our existing question banks to ExamSoft?
- Yes. The system allows for question importation via MS Word docs. Questions can also be exported from
Blackboard and imported into the ExamSoft system. Will a bank of student focused FAQs be created? Where will the FAQs be hosted? [UWA Response]: There is a working group being convened to create the resources and tools for students (including FAQs). Is there an easy way to include a case study in the exam, which students can use as a reference when answering questions? Yes, we recommend adding a resource such as this as an "assessment level attachment." This resource is available throughout the entirety of the exam.
SLIDE 7 Is there a streamlined way of teaching students how to use the software? The mock exam will be utilised to educate students on how to use the testing application. Can students elect to answer only a subset of questions on an exam? It is not currently possible to structure an exam to allow students to pick and choose which questions to answer, however questions can be weighted differently pre or post exam via adjusted scoring. Is it possible for students to print a hard copy, work
- n it with hand writing, and then upload the
completed paper? Students cannot print a Word copy, however faculty can print a Word copy if desired. Students would then be able to complete their exam via bubble sheet. Results can then be imported via CSV. What access will students have to the software prior to their exams? Will UWA be required to establish mock exams on the system as well as the actual exam? ExamSoft will coordinate with UWA to develop a mock exam policy. Students are highly encouraged to complete a mock exam and establishing a baseline image will be required before ExamID can be used for a high stakes assessment. Does ExamSoft offer an integration with Turnitin? ExamSoft does not currently offer a native integration with Turnitin. Essay responses can be exported via Excel. I've heard ExamSoft offers an OSCE tool, how can I learn more? ExamSoft offers OSCE solutions but this is not available as part of the current UWA product suite. ExamSoft would be happy to discuss those products centrally when desired. Can instructors use International Phonetic Alphabet characters in questions? Is it possible for students to use them in answers? Some IPA symbols are available when using the essay question type within Examplify. Further conversations may be required to scope this further. If students need clarification during the exam, how do they reach out to UCs without internet? UWA to suggest an INTERNAL PROCESS in collaboration with ExamSoft for this issue. Are students completing the same exam simultaneously or at different intervals? ExamSoft provides the ability to test synchronously or asynchronously. ExamSoft recommends that all exam takers test at the same time for high stakes assessments. Exam options can be enabled to provide flexibility in how assessments are delivered. When does UWA plan to begin using ExamSoft? Semester 1 Examinations
SLIDE 8
Can an exam be drafted and access given to another staff member to review and edit the same exam? Yes, user permissions can be structured to allow for multiple users to co-create an assessment. Will ExamSoft be used in parallel with existing LMS tools? ExamSoft will be used for assessment services at UWA. ExamSoft offers an integration with Blackboard that syncs roster and selected course information on a nightly basis. Final exam grades can be pushed from ExamSoft back to the Blackboard gradebook. Can students access the exam paper through the software but answer using an excel spreadsheet, which they can upload at the end of the exam? If exams are created in a non-secure mode students can complete questions in Excel outside the system and paste them into an essay prompt. Invigilation services are not available in a non-secure mode. What happens if there is a technical glitch during a student's exam? Can the student resume from where the system left off will they need to start again from scratch? Students are encouraged to utilise ExamSoft's 24 hour support team, which is available via phone, email, or chat. The vast majority of technical concerns can be solved by restarting the device. The device saves the exam file locally to the device every 60 seconds so students will not need to restart the exam from scratch once their technical issue is resolved. Can essay responses be anonymised for marking? Faculty members can anonymise essay responses when marking. When and where will the recording of this meeting be available? Available now on EDUFlex website (hosted by the EEU) Can the scientific calculator key strokes be recorded by examsoft and submitted for review to the examiner? ExamSoft will not track specific key strokes from the calculator. Exam logs will show that a student accessed the calculator. What is the best way to handle the scenario in which the student got the workings correct but the final answer incorrect? If an essay question is used, the student can include their response and workings directly into Examplify. From there, marks can be assigned as appropriate, and marker comments can be shared with students, where desired.
SLIDE 9 How do students provide questions regarding the exam during a secure exam session? ExamSoft offers a "Notes & Feedback" option that allows for students to request feedback directly within the testing application, Examplify. The student feedback is available for review within the faculty portal. A lot of equation based exams- has been confirmed that it won’t work with Examplify- can these be put
For invigilated exams via ExamSoft, UWA can enable students to use outside resources coupled with ExamMonitor as deemed appropriate. Students can input notes directly into the Examplify software. Any drawn diagrams or written equations may be completed on separate paper and UWA decides on how to incorporate outside resources to supplement an exam utilising such resources. Can the completed exam papers be downloaded
- ut of exemplify? Would like this to be an option
for accreditation recording. TBD Invigilation on Examplify – staff are concerned about students cheating - there is a function to allow invigilation – how do the students feel about their privacy in relation to this? Please refer to the Student FAQ for more information on security concerns, student data and remote invigilation services: https://www.web.uwa.edu.au/novel-coronavirus/online-exam-faq#3-03 Will Examplify be more stable than LMS? TBD Is marking going to be done online – how easy is this? UWA faculty may mark exams online or print exams and enter scores into the system. ExamSoft recommends online marking, providing the reporting and analytics to inform learning outcomes and provide tailored student support. Who does the uploading of the exam papers? Unit Coordinators OR Education Transformation Team What happens if the students internet/Wi-Fi access drops out – what is the security like when offline? Can the exams be modified during this period? Exams are taken offline. A lot of the exams in EMS require students to show their workings to come to a result – Examsoft doesn’t allow multiple lines of workings ExamMonitor must be used with Examplify to deliver secure assessments. Show your work functionality, including the ability to display work via a spreadsheet, is on the ExamSoft development team's roadmap with a target implementation by the end of 2020.
SLIDE 10 Exams in computer science require students to use
- ther software- this does not appear to be
compatible with Examsoft TBD Change in exam upload format – previously PDFA – knew that the unit co-ordinator has signed off the exam – concerned that moving to word document would allow for errors TBD How do you proof that the exam paper is correct
To be determined during Training Will PhD students have access to the platform for
Access permissions to ExamSoft to be determined by Unit Coordinators/UWA's policy on permissions
