MUNI
C4E
Preventing Cheating in Hands-on
Lab Assignments
Jan Vykopal, Valdemar Švábenský, Pavel Seda, Pavel Čeleda
Masaryk University, Brno, Czech Repu blic
March 2022 | ACM SIGCSE Technical Symposiu m
Format of Hands-on Cybersecurity Classes
Student Student
I 1Lab environment Lab environment
Virtual machine 1 Virtual machine 2 Virtual machine 1 Virtual
Motivation
Assignment
Student Communication Student
A
T
A
•
Lab environment Lab environment
Virtual machine 1 V l r t u a l
machine 2 Virtual machine 1 V l r t u a l
machine 2
3/18
Paper Contribution
Methods and toolset for
automatic problem
generation for tasks in a
lab environment.
Case study in an
authentic teaching
context.
Personalized lab environment Personalized lab environment
Virtual machine 1 Virtual machine 2 Virtual machine 1 Virtual machine 2
4/18
Toolset
/ @ interact with — I
Environment generator f Student
I generate
Personalized lab environment
Virtual machine 1
(D associate personalized
answers with the student
Virtual machine 2
>
CTFd plugin
Infrastructure for
hands-on lab tasks
Novel APG toolset
and the control flow
>
(§) submit
-personalized- - ^
answers
CTFd portal
(submission server)
A
Instructor
Configuration Generation
web:
type: port
challenge_id: 1
min: 8000
max: 65000
prohibited: [8080,8888]
secret:
type: t e x t
challenge_id: 2
6/18
Submission Server
run
, (§) interact with —[
Environment generator f Student
I generate
•
Personalized lab environment
Virtual machine 1
(D associate personalized
answers with the student
Virtual machine 2
>
CTFd plugin
Infrastructure for
hands-on lab tasks
Novel APG toolset
and the control flow
>
(§) submit
-personalized- -
answers
CTFd portal
(submission server)
A
(D get logs
Instructor
Case Study
• Individual homework assignment in an introductory computer security course.
• Taught at Masaryk University in the Czech Republic in Spring 2021.
• The course was enrolled by 207 undergraduate students.
• Topics covered: network attacks on authentication of Telnet and SSH servers,
securing an SSH server, and analyzing SSH network traffic.
8/18
Case Study - Personalized Environment
Each student had a personalized environment:
• a host running the Telnet server at a random network port,
• one user account with a random username,
• another user account with a random password, and
• a file containing a random sentence.
9/18
Tasks
• 8 tasks in total.
• 1 chain of 6 consecutive tasks.
• At the beginning, students can choose from 3 tasks (Al, T l , and T2).
>l
A2 ) A3 ) A4 ) SI ) S2 J
10/18
Cheating Detection
• Someone else's answers - the most reliable; incorrect submissions of correct
answers of other students.
• Task chains - students' solve time for consecutive tasks Less than minimal possible
solve time.
• Submission proximity - time proximity or location proximity of two or more
submissions.
11/18
Results
• Someone else's answers - 3 cases.
• The most conclusive case:
Student A submitted the correct answer 41247 for Al.
Student B submitted the incorrect answer 41247 twice, several days later, and before
the first interaction with the lab environment.
• Task chains (consecutive tasks) - 2 cases.
• One of two cases:
Three students completed the A3 task in 58 seconds.
The minimal possible solve time was 45 seconds. The assignment text: 102 words.
• Submission proximity - 2 cases.
• One confirmed case using location proximity:
Students K and L submitted their answers to T2 within 68 seconds.
Student K confessed he had cooperated with L. They share the same dormitory room.
12/18
Post-Homework Survey
• Optional survey after the assignment - 45 students answered.
• Forty students (89%) would prefer the provided format of completing assignments.
• Only one student would prefer the traditional homework assignment.
• Students' answers to other questions are reported in our paper.
13/18
Limitations
• A single exercise in one course - however, the number of participants is
considerably Larger than in the vast majority of published works.
• The detection methods analyze only students' actions at the submission server.
• Estimating the Location proximity using the same IP address of the submission is a
double-edged sword.
• Advanced students may reverse-engineer the environment generator and obtain the
answers without interaction with the personalized Lab environment.
• The answers of 45 out of 195 students may not represent opinions of all students,
particularly the critical voices.
14/18
Conclusions
• Prevention and detection of cheating in hands-on assignments involving the Lab
environment is possible in large and remote classes.
• Automated provisioning of the Lab environment with personalized values generated
Locally at students' computers is a feasibLe approach.
• Our case study reveaLed seven suspicious cases using three detection methods.
• Students enjoyed the assignment and its format and did not perceive cheating
prevention disruptively.
15/18
Publicly Available Contributions
Full paper and slides:
@) https://www.muni.cz/en/research/publications/1816366
Open-source toolset:
sf> https : / / g i t l a b . f i . muni. cz/cybersec/apg
16/18
Stay in Touch
Jan Vykopal
B vykopal@ics.muni.cz
Cybersecurity Laboratory
¥ https://twitter.com/cybersecmuni
17/18
Acknowledgments
• ERDF project "CyberSecurity, CyberCrime and Critical Information Infrastructures
Center of Excellence" (No. CZ.02.1.01/0.0/0.0/16_019/0000822).
• Special thanks to Daniel Kosc for developing the toolset.
18/18
MUNI
C4E
EUROPEAN UNION
European Structural and Investment Funds
Operational Programme Research,
Development and Education MINISTRY OF E D U C A T I O N ,
Y O U T H A N D S P O R T S
C 4 E . C Z