On-line Computer Exam

The University's local network, the used e-mail system and all other necessary information technology were taught at the beginning of each Academic year. The latter one included a middle advanced level course on browsing, text editing, and a minimal excel sheet manipulation. The conclusion of the course consisted of finding a plain text file stored in the local network, formatting it according to a detailed long list, and sending it as an attachement to a given e-mail address.

The IT awareness grew significally during the past several years. Thus our department decided to replace the obligatory course by an on-line exam. Of course, several training sessions had been offered as well. The main points focused on while determining the method and content of the exam were

• the exam should be taken anytime, but only in the computer labs;
• the exam can be taken an arbitrary number of times;
• only skills and knowledge necessary for the students' further studies are required for a successfull exam.

Based on these ideas, the Computer Proficiency Exam could be reached through a local web address. On the starting page the student identified herself giving her login name and password. After a successfull authorization, the system generated a new folder in the student's home directory, and created five identically named, but different in length and in type (extension) files in that folder. The first task was to match the type and the length of the files, create a new folder, copy, delete, and move certain files into the new folder. This task not only mesured the file manipulating skills of the examiner, but also ensured that she has access to her home directory. This way we wanted to be sure (not totally successfully) that not a single person should complete all exams in "en gros".

The second task turned out to be quite challenging. A meaningful English text of about 700 character was to be typed in. The text appeared on the top of the screen, and the typed text appeared on the bottom. Without mistypes we accepted as low speed as 80 charcters per minute; the minimal required speed increased with each mistype.

The third task was to fill out three multiple choice questionaire, each consisting of 20 questions. Each question had three to five possibility, and only one was accepted as correct. The first qustionaire focused on general issues, the second one on the University's e-mail system, and the third one on text editing with a few questions about excel sheets.

The concluding fourth task was a repetition of the task of the previous years. A plain text document was to be formatted according to a detailed formatting requirements, and the result to be e-mailed. The result had been checked by one of our colleagues.

Considering all details and consequences, the on-line exam was a success. We had all together 534 exams taken by 431 students, 394 of them had been successfull. The average typing speed was 156 characters/minute, while the average number of misprints was below 1 percent. The percentage of the correct answers for all three questionaries was quite uniformly aroung 80%. The formatting task turned out to be more difficult than the others. There were 675 submissions for the 394 accepted ones. On typical error was setting the margin erroneously. We required it to be 1 cm. The number 1 was (almost) always correct, however they the default inch was not changed to centimeters.

The main goal of the exam was to ensure that students possess all computer skills necessary for their university studies. If a student can hack the exam pages to forge a successfull exam, then he definitely has all those skills. Thus this was not out main concern, rather that such a villain may teach the others to cheat. For example, a quick internet seach followed by a copy&paste is faster and less error prone than typing.

Fortunately, the only browser available in CEU computer labs is IE. Futhermore no external programs can be installed or run on the machines. Exam pages use dynamic cookies and have dynamis addresses as well. Pages are shown in a separate window without headers; no local menu is working (right click is disabled), and printing a page (if someone arranges for this) results in an empty page. Even screen shots are disabled. All requests and answers are logged, which is necessary anyway should any complain arise. Theachers can look through the logs using a password protected page, and can revise the system's decision. In one case only we needed this when a student with a broken arm cannot type fast enough.

By our experience the countermeasures worked fine, and the exam material did not leak out. What we cannot achieve, and we have to make serious consideriation concerning it in the future, is that quite a few student logged in and then let someone else do the exam. This was quite evident in view of the numoruous similar mistakes in document formatting.

The software was developed locally. It requires a version 2 Apache web server with lobapreq module, and consist of several small perl programs. It can be configured easily, the multiple choice question and typing tasks can be configured separately. The software is available under the GNU General Public License.