MASARYK
U N I V E R S I T Y
FACULTY OF INFORMATICS
Analytics in OnlyDevs E-learning
Platform using xAPI
Bachelor's Thesis
JAN NOUZA
Brno, Spring 2025
MASARYK
U N I V E R S I T Y
FACULTY OF INFORMATICS
Analytics in OnlyDevs E-learning
Platform using xAPI
Bachelor's Thesis
JAN NOUZA
Advisor: Ing. Lukáš Grolig
Brno, Spring 2025
Declaration
Hereby I declare that this paper is my original authorial work, which
I have worked out on my own. A l l sources, references, and literature
used or excerpted during elaboration of this work are properly
cited and listed in complete reference to the due source. During the
preparation of this thesis, I used the following AI tools:
• Grammarly1
for grammar checking and sentence rephrasing
• ChatGPT2
to improve my writing style and assist with structuring
the thesis
I declare that I used these tools in accordance with the principles of
academic integrity. I checked the content and took full responsibility
for it.
Jan Nouza
Advisor: Ing. Lukáš Grolig
1. https://www.grammarly.com/
2. https://chatgpt.com/
iii
Acknowledgements
I want to thank my thesis advisor, Ing. Lukas Grolig, for his guidance
during the creation of this thesis. M y thanks also go to my family, girlfriend,
and friends for their support, encouragement, understanding,
and all the time we spent together throughout my studies.
iv
Abstract
Modern e-learning platforms collect and analyze user engagement
data in order to improve the overall quality of their content. This thesis
focuses on improving learning interaction tracking on the OnlyDevs
platform. The improvement is achieved by designing an enhanced
tracking model and implementing a backend application capable of
collecting, storing, and processing learning records based on the Experience
API (xAPI) standard.
After analyzing the platform's existing tracking mechanisms, several
shortcomings were identified, including structural deviations
from the xAPI standard and insufficient coverage of key learning interactions.
A new tracking model was proposed to address these issues,
introducing new categories of tracked interactions, including video
and question-level learning events.
A backend application serving the role of a Learning Record Store
(LRS) was implemented using ElysiaJS, ClickHouse, and RabbitMQ.
The resulting application supports structured learning record processing
and storage, establishing a solid foundation for advanced analytics
on the platform.
Keywords
E-learning, xAPI, LRS, Learning analytics, Interaction tracking, ElysiaJS,
ClickHouse
v
Contents
Introduction 1
1 Learning data tracking 3
1.1 Experience API 3
1.1.1 xAPI Statements 4
1.2 Other technologies 8
1.2.1 SCORM 9
1.2.2 Cmi5 9
1.3 Summary 10
2 OnlyDevs learning tracking analysis 11
2.1 Existing tracking structure 12
2.1.1 Actors 12
2.1.2 Objects 12
2.1.3 Verbs 12
2.1.4 Results 13
2.2 Identified shortcomings and limitations 13
2.2.1 Deviations from the xAPI standard 13
2.2.2 Wrong usage of the achieved verb 14
2.2.3 Missing context and extensions 14
2.2.4 Lack of fine-grained tracking 14
3 Tracking model improvements 15
3.1 Achieved verb replacement 15
3.2 Object improvements 15
3.2.1 Activity type 16
3.2.2 Extensions 16
3.3 Score changes 17
3.4 Context addition 17
4 New categories of tracked interactions 18
4.1 Video interactions 18
4.2 Question interactions 19
4.3 Course interactions 21
4.4 Session lifecycle 22
vi
5 Learning analytics and metrics 23
5.1 Question difficulty 23
5.2 Video engagement 24
5.3 Future advanced metrics 25
6 Learning Record Store implementation 26
6.1 Functional and non-functional requirements 26
6.2 Used technologies 27
6.2.1 ElysiaJS 28
6.2.2 ClickHouse 28
6.2.3 RabbitMQ 29
6.3 Implementation details 29
6.3.1 Statement ingestion and validation 29
6.3.2 Retrieval endpoints 30
6.3.3 CQRS pattern 31
6.3.4 Database model and migrations 33
6.3.5 Project structure 34
6.3.6 Testing 35
6.3.7 Implementation challenges 35
7 Future improvements 38
7.1 Advanced analytics endpoints 38
7.2 Dashboard integration 38
7.3 Custom xAPI metadata endpoints 39
7.4 Full platform integration 39
8 Conclusion 41
A Thesis archive content 42
Bibliography 43
vii
Introduction
As digital education evolves, collecting and analyzing learner engagement
data has become essential for improving the quality of educational
content and the effectiveness of learning platforms. By understanding
how students interact with course materials, platforms can
identify areas for improvement and tailor learning experiences to individual
needs. One standardized method for capturing such data is
the Experience API (xAPI), a flexible specification designed to track
learning experiences.
This thesis aims to use xAPI to improve the analytics of the OnlyDevs
e-learning platform by enhancing the tracking model, designing
new tracking opportunities, and developing a backend web application
for analytics data storage and processing. The OnlyDevs platform
focuses on providing microlearning lessons on various topics related
to programming and development, and was previously used as a
learning resource for the PB138 Basics of web development and markup
languages course.
The thesis starts with a brief discussion on the significance of learning
data tracking. Next, a detailed description of the xAPI standard is
provided, focused mainly on the importance and structure of its core
component - xAPI Statements.
The following chapter analyzes the current implementation of interaction
tracking on the OnlyDevs platform. It describes the structure
of currently collected learning events, identifies the limitations and
shortcomings, and outlines proposed improvements to the tracking
model.
Based on the tracking analysis, the thesis then explores enhancements
to the current tracking structure in the third chapter. These
enhancements intend to improve the structure of xAPI Statements
generated by the current tracking processes without introducing entirely
new categories of tracked interactions.
The following chapter focuses on identifying new types of interactions
that could be tracked to provide more complete information
about student behavior on the platform. The interactions are introduced
with their analytic potential and structural specifics of the xAPI
Statements representing them.
1
INTRODUCTION
Building upon the proposed improved tracking model, the thesis
then explores the types of learning metrics and insights that could be
derived from the collected data. The focus is on identifying indicators
that reflect student engagement and potential difficulties with the
learning content.
The next chapter describes the technical implementation of the
backend system responsible for receiving, validating, and storing the
xAPI Statements known as a Learning Record Store (LRS). This includes
an overview of the technologies used, the technical aspects of
the application, and the challenges encountered during development.
Lastly, the thesis discusses possible future improvements, such
as extending the analytics system with more advanced metrics or
steps needed for full OnlyDevs platform integration. These suggestions
are intended to guide further development beyond the current
implementation.
2
1 Learning data tracking
Modern educational platforms often include mechanisms for tracking
the progress of their users. This tracking enables the platforms to analyze
their content from both a technical perspective (user experience)
and an educational perspective, revealing possible weaknesses in the
platform's effectiveness.
The results of those analyses create a way for the platform maintainers
to continuously improve the platform by identifying gaps
in content quality and providing students with educational content
suited to their specific needs. One of the possible ways to enable this
kind of tracking is by using the Experience API specification.
1.1 Experience API
The Experience API (or xAPI for short), formerly known as Tin Can
API, is an IEEE-approved standard developed by Rustici Software
and Advanced Distributed Learning (ADL) for collecting data about
learner interactions and experiences [1].
The main objective of the standard is to define a consistent but flexible
and expandable way to express learning interactions and events.
Furthermore, the tracking itself is independent of the platform and is
not restricted, for example, to a web browser. It allows tracking activities
from various sources, including external applications, learning
platforms, or offline activities [1].
The xAPI defines its own ecosystem consisting of multiple parts.
The most relevant parts for this thesis are described below based on
information from the official xAPI website [1].
Learning Management System (LMS) A software application designed
to provide and manage learning materials and their delivery
to the intended audience. In the scope of this thesis, the
Onlydevs platform serves the role of an LMS.
xAPI Statement The core unit of information describing how exactly
the learner interacted with the LMS. A detailed structure of xAPI
Statements (also referred to as Statements) is further discussed
in Section 1.1.1.
3
l . LEARNING DATA TRACKING
Activity provider A system or a tool generating data (xAPI Statements)
about learning actions and experiences. The LMS itself is
typically an activity provider, but other external software, such
as an attendance tracker or other external e-learning platforms,
can also serve this role.
Learning Record Store (LRS) A component that is the recipient of
the xAPI Statements (learning records). Its main responsibilities
are processing and storing the records. It can also provide a way
to aggregate the stored data into meaningful analytical metrics.
LRS can either be an integrated part of the L M S or an external
standalone service.
1.1.1 xAPI Statements
As previously stated, xAPI Statements are the core unit of information
inside the xAPI ecosystem. They are an essential component of the
xAPI specification and represent learning events and actions.
Statements are represented in the JSON format, a widely adopted
data interchange format popular thanks to its human-readable nature.
The basic structure of a Statement is composed of three main object
properties - actor, verb, and obj ect. These three objects are supposed
to provide an answer to a simple question, "Who did what?" in the
context of a given LMS. A n example of such an answer might be "John
answered Question No. 1" [2].
Other object properties containing additional information about
the action represented by the Statement are result and context. The
result represents the outcome of the interaction (e.g., a test score),
while the context provides additional information about the learning
environment (e.g., language of the interaction).
As mentioned before, xAPI is supposed to define a consistent,
flexible, and expandable way of describing learning actions. This trait
is achieved by the extensions property. The structure of this property
is very flexible, as it can hold multiple key-value pairs to capture
any additional information about the interaction not defined by the
standard fields. It can be added as a sub-property into the main object
properties - object, result, and context [1].
4
l . LEARNING DATA TRACKING
Examples and more detailed descriptions of the properties based
on the information on the official xAPI website [1], the IEEE standard
specification [3], and the xAPI Data Specification [4] are provided
below to enhance understanding of the Statement structure.
Actor
The actor property is used to identify the user performing the interaction.
There are several ways to uniquely identify the user using subproperties,
one of which is required to be used: mbox, mbox_shalsum,
OpenID1
URI, or a custom account object, which defines a user's account
on a specific platform.
A n optional property holding the actor's name can also be included,
although it is not a unique identifier for obvious reasons. Examples of
a whole actor property using the mbox and account object identifiers
are shown in Listing 1.1 and Listing 1.2.
Listing 1.1: Example actor property #1
{
"name " : " John Doe" ,
"mbox" : " m a i l t o : J o h n . d o e © e x a m p l e . c o m "
}
Listing 1.2: Example actor property #2
{
"name": "John Doe",
"account " : {
"homePage": " h t t p s : / / x . c o m " ,
"name": "johndoe123"
}
}
1. https://openid.net/
5
l . LEARNING DATA TRACKING
Verb
The verb property describes the action that happened between the actor
and the object, and is composed of two fields: i d and display. The
id is a required Internationalized Resource Identifier (IRI). Although
using any full IRI is possible to identify the verb, it is recommended
to try and choose a verb already predefined in the xAPI Registry [5],
created by the maintainers of the xAPI standard to provide a centralized
way to store and explore all possible verbs and other types of
attributes connected to xAPI Statements.
The display property is of the language map type, which consists
of multiple key-value pairs. The keys in this map are language
tags, defined by the RFC 56462
standard (for example, en-US), while
the values are strings that provide a human-readable description or
explanation of the verb.
Listing 1.3: Example verb property
{
" i d " : " h t t p : / / i d . t i n c a n a p i . c o m / v e r b / v i e w e d " ,
" d i s p l a y " : {
" e n - U S " : " v i e w e d "
}
}
Object
The last of the three main properties is the obj ect, which defines what
the actor has interacted with. The only required sub-property is an id,
which is once again an IRI, but this time it references an object or an
activity within the given LMS.
Information about the object can be expanded by the def i n i t i o n
property. The def i n i t i on is a JSON object holding other sub-properties
itself, allowing it to store as much supplementary information about
the activity as possible, including the type of the object, language
maps to describe the activity, and extensions holding any additional
information. Listing 1.4 illustrates the object's structure with its defi-
nition.
2. https://www.rfc-editor.org/info/rfc5646
6
l . LEARNING DATA TRACKING
Listing 1.4: Example object property
{
" i d " : " h t t p : / / e x a m p l e . e d u / c o u r s e s / x A P I " ,
" d e f i n i t i o n " : {
" t y p e " : " h t t p : / / a d l n e t . g o v / e x p a p i / a c t i v i t i e s
/ c o u r s e " ,
"name " : {
"en-US": " E x p e r i e n c e API 101"
} ,
" d e s c r i p t i o n " : {
"en-US": " P r a c t i c a l usage of xAPI"
}
}
}
Result
The result property stores measurable outcomes of the interaction. It
defines fields, such as score, success, and completion, which indicate
the results of a given interaction. Another measurable attribute is
duration, which shows how long the activity that led to the result
took and is represented in the ISO 86013
format. Result can be further
enriched with extensions as well.
Listing 1.5: Example result property
{
score " : {
" r a w " : 43,
" m i n " : 0,
"max": 50
} ,
" c o m p l e t i o n " : t r u e ,
"success " : t r u e ,
" d u r a t i o n " : "PT1M20S
}
3. https://www.digi.com/resources/documentation/digidocs/90001488-13/
reference/r_iso_8601_duration_format.htm
7
l . LEARNING DATA TRACKING
Context
As indicated by the name of the property, it is intended to provide additional
context not included in the other properties of the Statement.
This information can be used to clarify the relationships between individual
activities, reference other users such as instructors, and specify
the platform or language of the interaction. Such details enhance the
understanding of the learning environment and can be beneficial when
creating reports or aggregating data. Note that the context data can
also be further expanded with extensions.
Listing 1.6: Example context property
{
" i n s t r u c t o r ": {
"name": " Jane Doe" ,
"mbox": " m a i l t o : j a n e . d o e @ example.com"
},
" c o n t e x t A c t i v i t i e s " : {
" p a r e n t " : [
{
" i d " : " h t t p : / / e x a m p l e . e d u / c o u r s e s / x A P I "
}
]
},
" p l a t f orm": "Example Edu",
"language " : "en-US"
}
1.2 Other technologies
In addition to xAPI, other technologies used to track learning activities
exist. Some predate the development of xAPI and contributed to its
design, while others aim to complement or integrate with it. This
section presents two important technologies: SCORM, the most widely
used standard for e-learning courses [6], and cmi5, a specification that
combines the benefits of both SCORM and xAPI.
8
l . LEARNING DATA TRACKING
1.2.1 SCORM
Sharable Content Object Reference Model, or SCORM, is a collection
of technical standards designed for e-learning platforms made
by the co-authors of xAPI, A D L . Unlike xAPI, which is a single standard,
SCORM comprises multiple standards and provides guidance
to developers on how to effectively use them in conjunction.
While SCORM offers a method for tracking a user's learning achievements
to some degree, its primary purpose is not focused on this aspect.
Instead, SCORM's main goal is to package and deliver educational
content - such as a course or lecture - so that it can run independently
on any platform that supports SCORM. This delivery is done by using
a so-called manifest file - an X M L file which provides all the necessary
information about the structure of the content and how to launch it
[7]The
main disadvantage of SCORM, in contrast to xAPI, is that
it is strongly tied to a particular L M S and, therefore, cannot gather
information from any other source. Moreover, SCORM defines only a
very limited set of information it can track, which is composed of only
four attributes - completion of the activity, duration of the activity,
whether the activity was successful, and the score of the activity [1].
To address the weaknesses of SCORM, Rustici Software was selected
by A D L as the leading group for researching and developing a
new standard - a "new generation of SCORM." This effort resulted
in the creation of the Tin Can API, which was later evolved into xAPI
[8]. Thus, SCORM can be considered the predecessor of xAPI, though
only regarding learning data tracking.
1.2.2 Cmi5
Based on the information regarding S C O R M and its relationship to
xAPI provided in Section 1.2.1, it has been established that S C O R M
primarily focuses on packaging and delivering learning content. At
the same time, xAPI provides a comprehensive solution for tracking
learning activities. A unified technology combining the strengths of
both could address their limitations and offer a more robust e-learning
standard, which is precisely the goal of the cmi5 specification.
9
l . LEARNING DATA TRACKING
The cmi5 specification can be described as an xAPI profile or an
instantiation of xAPI. It packages and launches educational content in
a manner similar to SCORM, utilizing an X M L document that outlines
the content and the launch process while also employing xAPI as the
communication and data layer [9, p. 26-30].
The specification describes the learner's progression by outlining
specific verbs representing certain events and, more generally,
by defining xAPI Statement patterns that illustrate the communication
between activity providers and the LRS. However, custom-made
Statements or their parts can still be specified [9, p. 31-33].
The main disadvantage of the specification related to learning
data tracking is, similarly to SCORM, that it is tied exclusively to cmi5compliant
platforms and applications due to the very specific structure.
Unlike xAPI, this means that data cannot be tracked from any source,
but only from an activity provider implementing cmi5 [9, p. 27].
1.3 Summary
This chapter introduced the concept of learning data tracking and its
role in educational platforms. It presented xAPI as a flexible, platformindependent
standard for capturing a wide range of learning activities
through structured Statements consisting primarily of actors, verbs,
and objects while also supporting extensions. In comparison, SCORM
and cmi5 were discussed, but their platform dependency and limited
tracking capabilities highlighted xAPI's advantages. Consequently,
xAPI is chosen as the foundation for designing the learning analytics
system on the OnlyDevs platform, leveraging its extensibility to
capture various interactions.
10
2 OnlyDevs learning tracking analysis
The OnlyDevs platform offers microlearning lessons on programming
and development topics, with basic user interaction tracking using
xAPI Statements to monitor learning progress. To support the analysis
of the current tracking based on the platform's code, Figure 2.1 shows a
simplified class diagram of the core platform entities to illustrate their
relationships. The overview below describes different types of lectures
from a content perspective to detail the course structure further.
Text lecture A lecture composed of written text defined in the markdown
format, which allows for the inclusion of code blocks or
images to demonstrate programming concepts better.
Video lecture A pre-recorded video presentation.
Checkpoint Consists of one or more questions testing the student's
knowledge of a certain topic. It may consist of single-choice and
multiple-choice questions, with student results scored in points.
Code lecture Enables the student to practically test their knowledge
by providing a coding assignment and an in-browser coding
sandbox, where the student can write and execute their code.
This lecture is not graded, but the student can show and hide a
sample solution and also reset their coding progress.
Visual Parsdi;m Slandardjan Ncuz&IMasaryk UnrjcrsrlyVI TextLectu re
Course 1 Section 1 Lecture
0.* 0.*
Video Lecture
Checkpoint
CodcLccture
Figure 2.1: Simplified class diagram of the course structure
11
2. ONLYDEVS LEARNING TRACKING ANALYSIS
2.1 Existing tracking structure
The existing tracking on the OnlyDevs platform currently covers only
basic user interactions with the learning content, primarily focusing on
the lectures. This section covers the structure of the xAPI Statements
produced by the platform, including descriptions of the contents of
individual properties based on the type of interaction when necessary.
Note that in this section, the objects are discussed before the verbs to
outline the semantics of tracked interactions better.
2.1.1 Actors
The actor property has the same structure in all the tracked interactions
and contains the mbox identifier of the user. Identification of
the actor through the mbox only is sufficient, since the users register
themselves with their emails, and linking their OnlyDevs account to
other platforms is currently not available.
2.1.2 Objects
In the current version of the platform, the contents of the object
property reference only the lectures. Aside from the required i d , the
objects contain the name and description language maps, and a type
string, which specifies the type of the lecture. The type property holds
only a simple string value, for example text or checkpoint.
2.1.3 Verbs
Together with objects, verbs define the semantic meaning of the interaction.
As already established above, the current tracking mechanisms
of the OnlyDevs platform regard only the lectures as objects. Therefore,
the verbs are tightly connected to the lectures as well.
A l l used verb objects have an i d , and a display language map as
specified in the xAPI standard. The i d of each verb is a URI referencing
a verb from the Experience API Registry as defined in Section 1.1.1.
Below is a list of used verbs along with a brief description of their
usage. For convenience, the human-readable representations of the
verbs are used instead of their URIs.
12
2. ONLYDEVS LEARNING TRACKING ANALYSIS
• Opened: Student opened the lecture
• Completed: Student marked the lecture as completed (does not
apply to checkpoint lectures)
• Achieved: Student submitted the checkpoint lecture quiz
• Showed: Student revealed the sample solution in the code lecture
• Hid: Student hid the sample solution in the code lecture
• Reset: Student reset their progress in the code lecture
2.1.4 Results
The result property is only present in Statements that represent the
submission of a checkpoint lecture, and therefore it is accompanied
by the verb achieved. It holds information about the student's score
in the checkpoint quiz, which is depicted by three self-explanatory
attributes - success, achievedScore, and maxScore.
2.2 Identified shortcomings and limitations
Following the previous analysis of current xAPI Statements, this section
highlights the key shortcomings and limitations of the existing
tracking system, addressing both the structural and semantic flaws of
the Statements.
2.2.1 Deviations from the xAPI standard
Although the current tracking follows the high-level structure of xAPI,
some properties deviate from the official specification. While the issues
described below are not critical, addressing them would improve data
consistency and support more robust analytics due to compliance with
the standard.
First, the type property inside object . d e f i n i t i o n is incorrectly
represented as a plain string indicating one of the four lecture types.
According to the xAPI specification [3, p. 40], this property must be a
valid IRL
13
2. ONLYDEVS LEARNING TRACKING ANALYSIS
Second, the achievedScore and maxScore properties in the result
object are not compliant either. The xAPI standard defines a dedicated
score sub-object that should include fields such as raw and max to
represent the user's result [3, p. 44].
2.2.2 Wrong usage of the achieved verb
During the analysis of the current xAPI Statement processing, it was
discovered that the verb achieved is used incorrectly. The current LRS
accepts string representations of verbs and maps them to their corresponding
i d and display properties before storing the Statement.
However, for the achieved verb, this mapping assigns an incorrect IRI
pointing to the verb archived. Moreover, achieved is not a valid verb
in the xAPI Registry [5]. Therefore, an entirely new verb or multiple
verbs should take its place instead of just replacing the IRI to make
the interactions currently using this verb semantically correct.
2.2.3 Missing context and extensions
Even though the core properties and attributes are present in the
Statements, they lack both context and any extensions, which limits
their expressiveness. The context property can provide additional
information about the learning environment, such as details about
the parent object or the platform. Additionally, extensions allow for
the inclusion of custom data, enriching the information about the
interaction and enabling more detailed analytics. Extensions can be
included not only in the context but also in the object and result,
depending on the type of data being recorded.
2.2.4 Lack of fine-grained tracking
The current tracking focuses exclusively on lectures as whole units,
without capturing detailed interactions within them. For example,
checkpoint lectures only record whether the student passed the entire
quiz, but provide no insight into individual questions. Tracking of
more granular interactions, such as responses to specific questions,
would enable deeper analysis of learner behavior and more targeted
improvements to course materials.
14
3 Tracking model improvements
To address the limitations identified in the current tracking model on
the OnlyDevs platform, this chapter proposes a set of improvements
and extensions that enhance the expressiveness of existing tracked
learning interactions. These changes are compliant with the xAPI
specification and primarily focus on improving the current types of
tracked events, without introducing entirely new categories of tracked
interactions, which is the aim of Chapter 4.
3.1 Achieved verb replacement
As described in Section 2.2.2, the current tracking model uses the verb
achieved, which is not part of the official xAPI Registry and is incorrectly
mapped in the LRS. The following standardized verbs from the xAPI
Registry are proposed to replace this verb and correct the semantics
of quiz results:
• Passed: Student completes the quiz with enough points to pass
the threshold for successful completion
• Failed: Student completes the quiz but does not achieve the
required score to pass the quiz
• Submitted: Student has submitted an ungraded quiz
This distinction improves clarity when analyzing outcomes and allows
for differentiation between successful and unsuccessful attempts and
between graded and ungraded (training) quizzes, which is one of
the planned platform features. It also aligns with how many existing
learning systems represent assessment interactions using xAPI [10].
3.2 Object improvements
Enhancing the object property is vital for ensuring consistent interpretation
of the learning interactions. This section introduces two main
improvements to the object structure on the platform - compliance
with the xAPI specification for type values, and the use of extensions
to enrich the metadata associated with each object.
15
3. TRACKING MODEL IMPROVEMENTS
3.2.1 Activity type
As mentioned in Section 2.2.1, the platform's use of the type property
inside obj ect. d e f i n i t i o n does not conform to the xAPI specification.
To ensure consistency with the standard, this subsection introduces a
custom set of IRIs to represent the existing lecture types.
Each lecture type is assigned a corresponding IRI under the namespace
https : //onlydevs. cz/xapi/activities/. The specific lecture
types follow the namespace prefix, and their names (e.g. Text lecture)
are converted into kebab-case (e.g. text-lecture).
These custom types avoid the ambiguity found in generic types
such as article, assessment, or performance from the xAPI Registry [5],
which do not fully capture the structure or purpose of the platform's
learning activities. Moreover, defining activity types within a dedicated
namespace allows the platform to introduce new types consistently
in the future.
3.2.2 Extensions
To support more detailed analysis and a better understanding of the
learning content, additional metadata can be attached to the object
using the extensions. Extensions are a collection of key-value pairs,
where each key must be a valid IRI, as required by the xAPI specification.
A n extension can be defined by a profile dedicated to a particular
category of interactions, or it might hold specific data valuable to only
one application [3].
Some extensions can be adopted from existing definitions. In particular,
the tags extension from the xAPI Registry [5] is used to associate
lectures and other learning objects with relevant keywords. This standardized
extension holds an array of strings (e.g., ["HTML", "CSS",
"JavaScript"]), supporting content categorization and enabling future
features like content recommendation and personalization.
In addition to standard extensions, this thesis proposes a set of custom
extensions under the https : //onlydevs . cz/xapi/extensions/
namespace to capture platform-specific metadata not covered by the
xAPI Registry. Defining custom extensions allows for a more precise
definition of Statements while maintaining full compatibility with the
xAPI standard.
16
3. TRACKING MODEL IMPROVEMENTS
For text lectures, the following extensions capture structural attributes
of the markdown content, helping estimate reading time and
content complexity:
• characters: total number of characters, including whitespace
• words: total word count, excluding code blocks
• images: number of embedded images
3.3 Score changes
As discussed in Section 2.2.1, the current implementation of checkpoint
tracking does not follow the xAPI specification, which defines a
dedicated score sub-object inside the result. To align with the standard
and enable well-structured score tracking, the score object containing
all four required attributes is proposed to replace the current
achievedScore and maxScore fields.
The raw, min, and max attributes of the new sub-object represent
the number of points achieved, the minimum possible score, and the
maximum possible score, respectively. The scaled value is a normalized
number between -1 and 1 used to represent the percentage of
gained points [3, p. 44].
3.4 Context addition
To add some basic information about the learning environment to the
existing tracking model, the addition of context and the following
standard context-level properties is proposed:
platform A string indicating the platform from which the Statement
originated, which allows data to be distinguished in case multiple
systems are integrated in the future. This value should
always be set to OnlyDevs in the current setup.
language A language tag (e.g., en-US) specifying the primary language
of the learning interaction that generated the Statement,
which can be used to differentiate multiple language versions of
the same object if content localization is implemented.
17
4 New categories of tracked interactions
While the previous chapter focused on extending and refining the
structure of existing Statements on the OnlyDevs platform, this chapter
introduces new types of learning interactions that the current tracking
model does not capture. These interactions aim to provide a more
complete picture of student engagement and behavior.
4.1 Video interactions
To facilitate a deeper analysis of how students interact with video
content, particularly in video lectures, the adoption of a communitydeveloped
specification, the xAPI Video Profile, is proposed. The
profile introduces a set of dedicated verbs (e.g., played, paused, seeked)
to define interactions with a video, a specific activity type, and multiple
extensions to both result and context that allow platforms to
capture detailed information about video interactions. The complete
specification of the profile, including a full list of used verbs, field
definitions, and Statement templates, is available in the official documentation
[11].
Aside from the verbs, extensions, and their usage defined in the
profiles data model, the profile requires some additional attributes
from the xAPI standard, not yet defined by the new tracking model, to
be contained inside the Statement. That includes two attributes in the
result - completion and duration. The former indicates whether the
activity connected with the Statement was completed, and the latter
represents the time spent to complete the activity.
When it comes to the context property of the Statement, it has to be
extended with a regi strat i on attribute. The registration is an optional
UUID value used to associate multiple Statements within a single
learning experience or attempt. Watching a video in a video lecture
can be an example of such an experience, which can include a sequence
of multiple interactions like playing, pausing, resuming, etc. Suppose
they are made within a single user session. In that case, all of these
interactions should generate a Statement with the same registration,
because they are all parts of the same learning experience.
18
4. N E W CATEGORIES OF TRACKED INTERACTIONS
Although the attributes described above are specified by the profile,
their usefulness extends beyond video interaction tracking. Incorporating
the completion, duration, and registration properties into
other types of tracked learning experiences - such as completing a
coding assignment or submitting a quiz - would improve the semantic
consistency of statements and enable more unified analytic data
processing. For instance, using the same registration identifier to
group statements within a single learning session can provide a clearer
picture of user behavior and enable session-based analytics. Similarly,
standardized tracking of time spent (duration) and task completion
status (completion) across other interaction types can enhance engagement
analysis and support cross-activity comparisons.
4.2 Question interactions
This section proposes tracking individual quiz questions to improve
the quality of assessment analytics. In the current implementation,
checkpoint lectures yield only one aggregated result, failing to provide
details on how students answered individual questions or which questions
posed the greatest difficulty among a group of students. This
design introduces detailed question-level tracking, where student
responses are captured using the answered verb and other attributes.
Object
The object represents the question and follows the structure of an
interaction activity, as defined by the specification [4]. Specifically,
it uses the cmi-interact ion activity type designed for assessment
interactions, and requires the following fields:
• interactionType: specifies the interaction type (e.g., "choice")
• description: contains the question text in a language map
• choices: an array of all answer options, where each option in-
cludes:
- id: a unique identifier for the option within the list
- description: a language map with the option text
19
4. N E W CATEGORIES OF TRACKED INTERACTIONS
• correctResponsesPattern: a list of one or more valid combinations
of correct answer IDs, with the following format:
- single correct answer: ["a"]
- multiple correct answers: [" a [, ] c" ]
- multiple correct answer combinations: [" a [, ] b", " c" ]
These fields defined for the choice interaction type consistently represent
question structure and logic, enabling accurate tracking and
analysis. Interaction types other than choice, such as performance or
l i k e r t , may require a different array attribute to store the interaction
components (e.g., steps or scale), or a different pattern format for
correct responses. The specification defines specifics of these fields for
other interaction types as well, but they are not described in this thesis
in detail, since the current types of interaction or assessment activities
on the platform do not follow the semantics of the other interaction
types.
Result
The result captures the outcome of the learner's answer to an individual
question. The following fields are relevant:
• response: a string representing the IDs of answers selected by
the learner, where multiple answers are delimited using [, ]
(e.g., "a" or "b[,]c")
• success: a boolean indicating whether the answer was correct
• score: a structured score object as defined in Section 3.3
Context
The context of the Statement should include contextActivities to
describe how the current interaction relates to other learning activities.
The most relevant relationship for question-level tracking is parent,
which links a question to a lecture, but it can also link a lecture to a
course. Other activity relationship types are:
20
4. N E W CATEGORIES OF TRACKED INTERACTIONS
• grouping: used to define an indirect relationship to the Statements
obj ect, for example, questions belonging to a course
• category: defines an activity to categorize the Statement, is used
within the xAPI Video Profile described in Section 4.1
• other: captures any additional or custom relationships not covered
by the previous types
4.3 Course interactions
After defining individual interactions within lectures, capturing interactions
one level higher enables a better understanding of which
courses are most popular or insufficiently engaging for students to
complete. The following custom IRI defines the course activity type for
the interactions: https : //onlydevs . cz/xapi/activities/course.
The first crucial interaction at the course level is enrollment, captured
using the joined verb. A n alternative verb purchased could be
considered, in case a mechanism for buying courses in exchange for
credits or money is implemented in the future.
Course completion is also an essential event to track. It is captured
using the completed verb and includes the result. completion field to
indicate whether the student successfully completed all the lectures.
Unlike lectures, courses are typically completed over a longer period
of time, and this interaction may therefore not require a duration or
registration field.
Another valuable interaction might be rating, which provides feedback
on the perceived quality of the course. This should be tracked
using the rated verb. The rating is represented by the score in the
result. score. Optionally, result. response can include free-form
textual feedback provided by the student. Although this section focuses
on course interactions, the same rating mechanism could be
applied to lectures if feedback is desirable at multiple levels.
All interactions related to courses may include the tags extension
to classify the course by topic or technology. This extension, defined
in the xAPI Registry, is described in more detail in Section 3.2.2.
21
4. N E W CATEGORIES OF TRACKED INTERACTIONS
4.4 Session lifecycle
To gain a better understanding of how learners interact with lecture
content over time, it is important to monitor the entire lifecycle of a
learning session, especially given the microlearning nature of the lectures.
This includes tracking everything from the moment the lecture
content is opened to when the session concludes. The xAPI Registry
[5] provides a set of verbs that can describe different events within a
session, that are not directly connected to the learning content. The
following verbs are proposed for capturing the session continuity:
• Initialized: Student started or opened a learning activity but
has not yet started interacting with it
• Unfocused: Student has lost focus on the activity (e.g., switched
browser tabs)
• Focused: Student regained focus on the activity
• Exited: Student left the activity (e.g., refreshed the page)
• Terminated: Implies that the student completed all intended
interactions, and marks the formal conclusion of a session
Generation of Statements with verbs such as unfocused,focused, or exited
can be triggered by browser events such as visibilitychange [12]
and bef oreunload [13], allowing the frontend to react appropriately
to changes in the learner's focus or presence. Statements with all of
the session lifecycle verbs and other interactions in the lecture should
include a shared context. registration value to logically group all
related interactions, as described in Section 4.1.
To mark the learning activity session starts consistently across all
types of interactions, the verb initialized should replace the verb opened
in the current platform tracking model described in Section 2.1.3. The
verb opened would be better suited to situations like opening a file or a
document, rather than starting to interact with a lecture.
22
5 Learning analytics and metrics
With a more robust and structured tracking model in place, the platform
can support a wide range of analytics that were previously not
achievable. This chapter outlines examples of such metrics, each targeting
a specific aspect of user behavior or content effectiveness.
5.1 Question difficulty
One of the most valuable insights from detailed question tracking
described in Section 4.2 is the ability to measure question difficulty
using two key attributes: the success rate and the answer distribution.
The success rate, represented simply as the percentage of users who
answered correctly, may indicate that a question is too complicated,
unclear, or based on poorly explained content.
The answer distribution shows how often each option was selected.
It helps identify common misconceptions and detect distractors that
are either too obvious or unintentionally misleading. Figure 5.1 shows
how such a distribution could be visualized for a single-choice question.
In this example illustration, the correct answer is CREATE, yet
a significant number of users selected PUT, which is incorrect. This
might indicate student confusion that requires content clarification.
Which of the following options Is not a valid HTTP method?
• CREATE
PUT
POST
Figure 5.1: Answer distribution for a single-choice question
23
5. LEARNING ANALYTICS A N D METRICS
5.2 Video engagement
Tracking how students interact with video lectures can reveal practical
insights into learner attention, interest, and content effectiveness. The
xAPI Video Profile described in Section 4.1 standardizes how video
playback interactions are recorded, allowing the platform to capture
relevant metadata.
A valuable metric enabled by this tracking is the view count per
time interval, revealing how frequently students viewed each segment
of the video. This is made possible by using the played-segments
result extension, which records time ranges of the video that each student
viewed. A n example of this metric being visualized is illustrated
in Figure 5.2, where a line chart shows the number of views for each
10-second segment of a sample video.
Although simplified and with large segments, this example demonstrates
how certain spikes in view count may indicate moments in
the video that students found difficult to understand and therefore
watched multiple times. On the other hand, as seen in the chart, a gradual
decline in view count towards the end of the video may suggest
decreasing student attention or perceived redundancy of the closing
segments. Such insights can help the learning content creators identify
confusing parts that may require clarification, and less engaging
sections that could benefit from structural or other improvements.
Video Segment
Figure 5.2: Video view count per 10s segments
24
5. LEARNING ANALYTICS A N D METRICS
5.3 Future advanced metrics
Once the proposed tracking system is deployed and begins collecting
real user data regarding learning interactions, more sophisticated analyses
can be created and evaluated. These analyses can reveal patterns
in student behavior and can support data-driven improvements to
both content and user experience. The following examples depict how
the platform's analytics may use the collected data:
Optimal video length estimation By aggregating watched durations
and other video metadata across different videos, the platform
can detect how learners respond to videos of varying lengths. If engagement
tends to drop off in longer videos, this can help determine
guidelines for optimal video duration and structure, and encourage
content creators to split complex topics into smaller segments.
Session interruption patterns Using session lifecycle events (initialized,
suspended, resumed, exited), the platform can identify common
patterns in how and when users interrupt their learning sessions. For
example, frequent suspensions early in a session may suggest that
some content fails to capture attention quickly or that the topic is not
interesting to the targeted audience. Such data can guide adjustments
in lecture structure or delivery.
Interest-based course recommendations Thanks to the tracking of
course enrollments and completions, lecture interactions, and tags
associated with lectures and courses, the platform can analyze the
historical behavior and interests of the learners. This opens the door
to personalized content recommendations - for example, suggesting
follow-up courses in topics frequently viewed or rated highly by the
learner, proposing learning materials, or generating training quizzes
based on the learner's weaknesses.
25
6 Learning Record Store implementation
This chapter describes the implementation of a Learning Record Store
(LRS) capable of processing Statements based on the proposed tracking
model for the OnlyDevs platform described in previous chapters.
It outlines the system's requirements and technologies, provides an
overview of the core functionality, and addresses selected challenges
encountered during development.
6.1 Functional and non-functional requirements
This section defines the essential requirements for the backend web
application that serves as the Learning Record Store for the OnlyDevs
platform. These include functional requirements specifying the application's
expected behavior and non-functional requirements related
to implementation constraints and system properties.
xAPI The LRS must support the reception of learning events (xAPI
Statements) that conform to the tracking model structure defined
in this thesis.
REST API The LRS should expose a REST API that allows submitting
either a single Statement or a list of Statements. A read-only
endpoint for retrieving a Statement by its ID should also be available,
primarily for testing and debugging purposes. At least one
example endpoint returning aggregated analytical data should
also be included to demonstrate how the LRS could support
future analytical use cases.
Message queue integration Besides the REST API, the LRS must implement
a message consumer capable of receiving Statements
asynchronously via a message queue, allowing decoupled and
scalable data ingestion.
Statement validation The LRS must validate all incoming Statements
before processing them. The LRS must reject any request that
fails to conform to the expected Statement schema. Additionally,
values with predefined formats, such as language tags or
26
6. LEARNING RECORD STORE IMPLEMENTATION
UUIDs, must be checked. Validation errors received through the
REST API must result in a descriptive error message with an
appropriate HTTP status code. For Statements received through
the message queue, validation errors should be logged for later
inspection.
Statement storage The LRS must store valid Statements in an analytical
database optimized for high-performance querying over
large datasets to enable efficient analytic queries over stored
learning records.
Database migrations Database schema changes must be handled by
a structured migration mechanism to enable safe and consistent
evolution of the LRS data model.
CQRS pattern The system must implement the Command Query Responsibility
Segregation (CQRS) pattern to separate read and
write operations into distinct logical components. This pattern
improves code maintainability, promotes clear responsibility
boundaries, and enables future performance improvements.
Technology constraints The implementation must use the ElysiaJS
web framework, a lightweight and modern TypeScript framework
chosen for its planned integration into the core backend
of the OnlyDevs platform. Using ElysiaJS ensures architectural
consistency across platform components and enables easier future
integration. The ClickHouse analytical database should be
used to store learning records, as it is already utilized by the current
tracking system and meets the criteria for efficient querying.
Reusing this technology simplifies infrastructure management
and enables straightforward data migration between legacy and
new systems.
6.2 Used technologies
The implementation of the LRS relies on a modern technology stack.
This section introduces each core technology used in the project and
explains the motivation for its selection.
27
6. LEARNING RECORD STORE IMPLEMENTATION
6.2.1 ElysiaJS
ElysiaJS is a minimal community-driven web framework for building
backend applications in TypeScript. It offers an intuitive API design,
built-in TypeScript support, and a modular plugin system that encourages
clean separation of concerns and code reusability. Plugins can
be composed hierarchically, decorating the highest-level application
instance with new routes or other application logic [14].
The framework runs on the Bun JavaScript runtime, which contributes
to its high performance and fast startup times. Bun is a modern
all-in-one toolkit that combines a JavaScript engine, package manager,
and native TypeScript support [15].
Although Elysia is a relatively recent technology, having its first
stable release in March 2024, it has already shown promising results
in web backend framework benchmarks. In the TechEmpower Web
Framework Benchmarks [16], ElysiaJS placed 14th out of more than
five hundred tested frameworks in plaintext responses per second.
Thanks to being one of the best-performing JavaScript frameworks,
ElysiaJS was well-suited for implementing the LRS web server in
this thesis, which needs to handle a large number of incoming xAPI
Statements.
6.2.2 ClickHouse
ClickHouse is an open-source column-oriented database management
system designed for Online Analytical Processing (OLAP). It is optimized
for high-speed aggregation, filtering, and scanning of large
data volumes, making it particularly suitable for applications performing
real-time queries on large datasets. Unlike traditional row-based
databases, ClickHouse stores data by columns, allowing for better
compression and faster query execution when working with a subset
of columns [17].
In the context of this thesis, ClickHouse is used as the primary
storage for validated xAPI Statements. Its performance on analytical
workloads is a key factor, as the LRS must support querying large
numbers of learning events, often filtered or grouped by fields such
as actor, verb ID, object ID, session ID, or timestamp.
28
6. LEARNING RECORD STORE IMPLEMENTATION
6.2.3 RabbitMQ
RabbitMQ is a widely used open-source message broker that implements
the Advanced Message Queuing Protocol (AMQP). It enables
asynchronous communication between components by allowing them
to send and receive messages through queues. This decouples producers
and consumers, improves scalability, and enables load balancing
in distributed systems [18].
In the LRS, RabbitMQ is used to support asynchronous ingestion
of xAPI Statements. As no specific message broker technology was
specified by the application requirements, RabbitMQ was chosen from
available alternatives (e.g., Apache Kafka) due to the thesis author's
prior experience with the technology. The message-based ingestion of
Statements follows the same validation and persistence logic as the
REST API, ensuring consistency across both entry points.
6.3 Implementation details
This section outlines the core processing flow of xAPI Statements in
the implemented LRS. It describes how incoming data is handled,
how it is stored, and how it can be queried. The section also covers the
testing strategy and highlights selected implementation challenges.
6.3.1 Statement ingestion and validation
The primary entry point for learning records is a REST API endpoint
POST /statements that accepts Statements in JSON format. Clients
can submit either a single Statement or a list of Statements in the
request body, and either a single UUID or a list of UUIDs is contained
in the response. Upon receiving a request, the application performs
validation in schema-based and application-level stages.
The ElysiaJS web framework handles the first stage automatically,
since it integrates tightly with Typebox - a runtime schema declaration
system for TypeScript. Typebox allows the developers to define precise
JSON schemas using a syntax similar to TypeScript types and use
them for runtime validation [14]. In the LRS, schemas validate the
top-level structure of Statements, including nested objects and field
types. The validation also covers constraints such as value ranges (e.g.,
29
6. LEARNING RECORD STORE IMPLEMENTATION
score .scaled must be between -1.0 and 1.0) or string formats like
u r i and uuid. If this validation fails, the framework returns an HTTP
response with status code 422 Unprocessable Entity and a JSON
body describing the schema violation.
After schema validation, a second layer of application-specific
validation is applied. This step verifies the semantic correctness of
selected fields. Specifically, language tags are validated using the
language-tags1
package to ensure they conform to RFC 5646 syntax,
and the result. duration field is parsed using the iso8601 -duration2
package to verify its format. If any of these checks fail, the system responds
with an HTTP 400 Bad Request error and a relevant error
description.
In addition to the REST API, the LRS application also includes
a RabbitMQ message consumer that listens on a specific queue for
JSON Statements submitted asynchronously. O n application startup,
the consumer attempts to subscribe to a queue. If the subscription
fails, more attempts to subscribe are made before the consumer logs
its unavailability without shutting down other parts of the application.
Both the number of retries and the delay between each attempt are
configurable through environment variables. After a successful subscription,
the application consumes messages and applies the same
validation principles to the received Statements as the REST API, and
logs any potential errors.
6.3.2 Retrieval endpoints
The LRS exposes two data retrieval endpoints. The first is a simple
Statement retrieval endpoint that allows clients to fetch a single Statement
by its unique identifier: GET /statements/{id}. This endpoint is
primarily used for debugging and testing purposes, but it also demonstrates
the read capabilities of the system.
The second endpoint GET /analytics/question/{id} provides a
basic example of how analytical endpoints can be created on top of
the recorded data. It requires the ID of a question (object .id), and
returns aggregated statistics about the given question - the number
of total and successful answers, and the distribution of responses.
1. https://www.npmjs.com/package/language-tags
2. https://www.npmjs.com/package/iso8601-duration
30
6. LEARNING RECORD STORE IMPLEMENTATION
6.3.3 CQRS pattern
The LRS implementation uses the CQRS (Command Query Responsibility
Segregation) pattern, which segregates writing and reading
operations into separate logical paths. This separation improves code
maintainability and scalability [19].
In CQRS, write operations are handled through commands. Commands
modify the system's state (e.g., insert new data), and their
execution encapsulates business logic and invariants. On the other
hand, queries handle the read operations. Queries do not modify the
system's state, serving the read-only role, and can retrieve data from
pre-cached read models, such as a materialized view in the database
[20].
In the LRS, CQRS is implemented using the elysia-cqrs3
plugin,
which provides a mediator-based dispatching mechanism and interfaces
for commands, events, queries, and their handlers. To ensure
proper functionality within the application, a custom wrapper registering
all command and query handlers was created. This integration
is discussed further in Section 6.3.7.
When it comes to Statement write operations in the LRS, Statements
are wrapped in the InsertStatementCommand after validation, and
the command is then dispatched to a command handler. The handler
calls a persistence layer that stores the Statement in ClickHouse. The
command handler serves as a coordination layer and does not contain
additional logic.
As for data queries, the GET endpoint for Statements is implemented
by dispatching a GetStatementByldQuery, which is handled
by a query handler that fetches the corresponding record from the
database. Similarly, the analytic endpoint described in Section 6.3.2
dispatches a query that returns aggregated statistics for a question.
This mediator-based approach, where each command and query
is processed by its dedicated handler, contributes to better separation
of concerns and improves the overall structure of the codebase. It also
enables future performance improvements - for example, the CQRS
pattern allows separating data sources for reads and writes, where
write operations update one data source and read operations query
from a different, potentially more read-optimized one [19].
3. https://github.com/jassix/elysia-cqrs/
31
6. LEARNING RECORD STORE IMPLEMENTATION
The diagram displayed in Figure 6.1 provides a simplified overview
of the data flow in the LRS implementation, highlighting the separation
of write and read operations as defined by the CQRS pattern. For
clarity, the diagram omits response handling (i.e., returning UUIDs
and Statement objects) and the example analytical endpoint, which
follows a similar processing pattern to the Statement retrieval.
Message
Message queue
GET
/statements/{id}
POST
/statements
Schema
validation
W— Stalemsnl-
Message
consumer
GetStatementByldGuery
Query
Mediator
GetSlalementByld Query
_ L _
Query
Handler
App-level
validation
Ins&rtStatemenlCommand
, i
Command
Mediator
hsertG:a:eren1Cor rani
Command
Handler
Figure 6.1: Processing flow of xAPI Statements
32
6. LEARNING RECORD STORE IMPLEMENTATION
6.3.4 Database model and migrations
Statements are stored in a single denormalized table in the ClickHouse
database to prioritize read performance and efficient analytical querying
over large learning record datasets. While flat in structure, the
table accommodates all necessary xAPI properties.
Selected nested fields, such as object. d e f i n i t i o n , choices, use
the Array( JSON) type, while flexible fields like the extensions properties
are stored as stringified JSON in String columns. Filtering by
specific extension values is possible using built-in JSON functions4
without requiring schema changes when new extensions are defined.
To support advanced filtering and aggregation, some xAPI fields
are also extracted into dedicated columns. For instance, the standard
tags extension is stored as a string array, and the contextActivities
structure is split into four separate columns reflecting xAPI semantics.
Figure 6.2 displays the schema, but omits columns for interaction
components other than choices for clarity.
M id
1 verfajd
verb_display
objectjd
Q] objeci_defiriition_riame
LH object_definition_descripticn
A-z object_definition_type
3bjcci_d(it
ini:irtn_in:pfaciion_^r.c
[ _ abjfict_d«'ini:iftn_coffftr.t.response.^pflttfti
!_._ cbject_de'initioii_cho ces
a 7 object .do^ni'.iorucx'.ersions
r- 3bjcct_df'lniiion_crtf»rsiors_lan^
res u It_score_sc a Ied
result_score_raw
result_score_min
123 resiilt_score_max
result_sjecess
result_completion
resLll_djratiori
result_response
A-z result_e)densions
context_rGgiEtratiori
context_instructor
c• ntext_activ ities_ps rent
co ntext_activities_category
context_activities_grouping
context_activitiss_otrisr
context_platform
context_language
co ntext_eirtensio ns
Qtimestamp
stored
Figure 6.2: Schema of the statements table
4. https://clickhouse.com/docs/en/sql-reference/functions/
j son-functions
33
6. LEARNING RECORD STORE IMPLEMENTATION
To manage changes to the schema in a maintainable way, the project
uses the clickhouse-migrations5
tool. This community-driven migration
tool allows defining SQL-based migration scripts that can be
applied in a controlled and versioned manner during deployment
or application startup. This approach ensures that schema evolution
is traceable, automatable, and safe to apply across different environ-
ments.
6.3.5 Project structure
The implementation aims to follow the principles of Clean Architecture,
though its application is limited due to the nature of Elysia
plugins (i.e., no interfaces, hierarchically injected plugins). The src
folder structure described below aims to clearly separate concerns
across the application and ensure code maintainability.
• application: Contains use case logic, specifically CQRS commands,
queries, their handlers, a CQRS wrapper defining the
mediators, and a message handler.
• controllers: Consists of route handlers that expose the REST API.
These handlers translate requests into commands or queries and
dispatch them via the CQRS mediators.
• data: Responsible for data persistence. It contains database models
and a repository that encapsulates access to ClickHouse,
acting as a bridge between the application layer and the infras-
tructure.
• domain: Encapsulates the core domain logic, including data
transfer objects (DTOs), validation logic, and JSON schemas that
define the expected structure of incoming Statements. Domain
validation ensures that only semantically valid Statements are
accepted by the system.
• infrastructure: Contains integration objects such as ClickHouse
client and RabbitMQ subscriber.
5. https://github.com/VVVi/clickhouse-migrations
34
6. LEARNING RECORD STORE IMPLEMENTATION
6.3.6 Testing
The application is tested using bun: test, the built-in testing framework
provided by the Bun toolkit. The test suite consists of integration
tests that verify the behavior of the system through its REST API. The
tests focus on verifying correct behavior in response to both valid
and invalid input. The tests are located in the test folder, and in order
to execute them, a local ClickHouse instance must be running,
as the application writes data to an actual database even during test
execution.
6.3.7 Implementation challenges
During the development of the LRS implementation, a few challenges
emerged that required special attention. This section describes selected
challenges and their solutions.
CQRS setup
The first issue was related to the integration of the elysia-cqrs plugin.
Although the plugin provides a straightforward mechanism for
handler registration by calling a function, it does not seem to extend
the application context properly with mediator objects that handle the
dispatching.
This issue is probably due to the usage of derive () instead of
decorate (), which are Elysia plugin lifecycle methods, as seen in the
source code inside the plugin's repository.6
This led to TypeScript
errors when accessing the context properties in other Elysia plugins,
since the CQRS setup needed to be used not only in the REST API controllers
that use Elysia methods such as post () to handle the requests,
but also in the RabbitMQ message handler.
To resolve this, a custom wrapper plugin was created that manages
the registration of all the handlers, explicitly decorates the application
context with mediator instances, and fully encapsulates the CQRS
setup. The wrapper uses classes and interfaces (e.g., CommandMediator)
from the original elysia-cqrs plugin, and its code snippet can be seen
in Listing 6.1.
6. https://github.com/jassix/elysia-cqrs
35
6. LEARNING RECORD STORE IMPLEMENTATION
New commands or queries and their handlers can be easily registered
in the wrapper. Other Elysia plugins can use it by injecting it
using .use(cqrsWrapper) and calling the decorated mediators from
the context.
Listing 6.1: Custom CQRS wrapper Elysia plugin
e x p o r t const cqrsWrapper = new E l y s i a ( )
. d e c o r a t e ( ' c m d M e d i a t o r ' , new CommandMediator())
. d e c o r a t e ( ' e v e n t M e d i a t o r ' , new E v e n t M e d i a t o r ( ) )
. d e c o r a t e ( ' q u e r y M e d i a t o r ' , new Q u e r y M e d i a t o r ( ) )
. o n S t a r t (
({ d e c o r a t o r : {
cmdMediat o r ,
e v e n t M e d i a t o r ,
q u e r y M e d i a t o r ,
// ... o t h e r d e c o r a t e d p r o p e r t i e s
}
}) => {
const commands = [
[XYZCommand, new XYZCommandHandler(. . . )]
]
f o r ( c o n s t [command, h a n d l e r ] of commands) {
c m d M e d i a t o r . r e g i s t e r ( c o m m a n d , h a n d l e r ) ;
}
// ... event and query r e g i s t r a t i o n s
}
) ;
Timestamp formatting
Another challenge was connected to consistent timestamp handling.
The xAPI standard requires timestamps to be in the ISO 8601 format,
and specifically recommends the usage of the RFC 3339 format, which
is a profile of ISO 8601 [21]. Although both of these standards enable
multiple possible timestamp formats, the following format, which
conforms to both standards, has been selected to enable a consistent
36
6. LEARNING RECORD STORE IMPLEMENTATION
schema for timestamp validation: 2025-05-15T12:34:56.789Z. This format
represents timestamps with millisecond precision and a time zone
indicator (note that the Z can be replaced with a time zone indicator
like +01:00), and is validated at the schema level by using a Typebox
regular expression.
Since ClickHouse's datetime types7
do not preserve time zone
information, timestamp columns were defined using DateTime64(3,
' UTC'), ensuring that ClickHouse stores timestamps with millisecond
accuracy, and interprets them as U T C timestamps. Incoming timestamps
are parsed using JavaScript's Date object, which converts any
provided time zone offset into U T C (i.e., with a Z time zone). The
result is serialized via .toISOStringO, and the trailing Z is removed
to match ClickHouse's expected input format. On retrieval, the stored
UTC timestamps are returned in the same specified format with the Z
UTC indicator added.
7. https://clickhouse.com/docs/sql-reference/data-types/datetime
37
7 Future improvements
Although the proposed tracking model and the LRS implementation
fulfill the core objectives of this thesis, several areas remain open
for further development and enhancement. Potential future improvements
described in this chapter focus primarily on expanding the
system's analytical capabilities and preparing it for full-scale production
deployment.
7.1 Advanced analytics endpoints
While the application includes one example of an endpoint serving
analytical data, more complex features should be implemented to fully
realize the analytical potential of the system. Future work could consist
of endpoints that provide advanced aggregated analytics, including
the examples described in Section 5.2 and Section 5.3. This extension
of the platform's analytical capabilities could require more sophisticated
and optimized ClickHouse queries and potentially the use of
materialized views or other performance enhancements to maintain
low latency.
7.2 Dashboard integration
To make the collected learning data accessible to instructors and administrators,
it would be beneficial to integrate the LRS with a dashboard
interface. Such a dashboard could visualize key metrics, highlight
problematic areas, and support automated evaluation of content effectiveness.
A n example of suitable technology for this purpose is
Grafana.1
Depending on future architectural decisions, the integration
could be implemented via a separate API layer or by connecting
directly to the existing REST endpoints.
1. https://grafana.com/
38
y. FUTURE IMPROVEMENTS
7.3 Custom xAPI metadata endpoints
Several custom IRIs were introduced to represent domain-specific
activity types and extensions as part of the proposed tracking model.
According to the xAPI specification, each IRI used in a Statement
should ideally dereference to a human and machine-readable document
that provides semantic context [3, p. 47].
To comply with this best practice, the system could expose dedicated
endpoints that serve metadata for these custom IRIs. For example,
accessing a U R L such as https://onlydevs.cz/xapi/activities/textlecture
could return a JSON describing the meaning and structure of
the text-lecture activity type.
7.4 Full platform integration
Currently, the LRS is a functional but standalone backend service. To
integrate it fully into the OnlyDevs platform production environment,
the following improvements are advisable:
Tracking model adoption To fulfill the potential of the implemented
LRS, the tracking on the platform should be extended with the
new tracking model described in this thesis and integrated with
the LRS itself.
Security A l l API endpoints are currently unprotected. To ensure secure
access, authentication and role-based authorization should
be implemented in accordance with the platform's architectural
requirements.
Error monitoring and observability Integration with logging, tracing,
or alerting systems is necessary for production use to ensure
reliability and traceability.
Testing enhancement To increase the test coverage, improve the developer
experience, and ensure testing consistency, a Dockerized
testing setup could be introduced using proper configuration
and tools such as Testcontainers.2
This would eliminate the need
2. https://testcontainers.com/
39
7- FUTURE IMPROVEMENTS
for a manually configured ClickHouse instance during local testing
and in the CI/CD pipelines, and could enable automated
tests for message consumers.
Message dead lettering Messages can get dead-lettered (i.e., sent to
a dead letter exchange) if they cannot get delivered to their
designated queue for reasons like exceeding the length limit of
the queue. Implementing a robust dead lettering mechanism,
which would allow republishing of messages that end up in
the dead letter exchange, would ensure better message queue
reliability [22].
40
8 Conclusion
This thesis explored the Experience API (xAPI) standard for tracking
learning interactions in e-learning systems and the structure of xAPI
Statements. A comparison with other standards, such as SCORM and
cmi5, highlighted xAPI's greater flexibility and extensibility.
Subsequently, the existing tracking model used by the OnlyDevs
platform was analyzed, and several shortcomings were identified,
including deviations from the standard and limited granularity of
recorded events. Based on these findings, a new tracking model was
proposed. It introduced structural improvements to xAPI Statements
and added support for new interaction types to enhance the analytic
value of collected data. The design also proposed a set of possible
learning analytics and metrics that the enriched data could enable.
The second part of the thesis focused on developing a Learning
Record Store (LRS) for the OnlyDevs platform. It outlined both the
application requirements and the technical implementation, which
was based on ElysiaJS, ClickHouse, and RabbitMQ. The application accepts
xAPI Statements via a REST API and message queue, validating
their structure and the semantic correctness of individual fields.
Although the current implementation fulfills the main objectives
of this thesis, some areas remain open for further development. These
include the addition of more advanced analytic data retrieval or integration
with analytical dashboards.
The main contribution of this work lies in the creation of a functional
xAPI-based tracking system designed to be integrated into the
broader context of the OnlyDevs platform. The proposed tracking
model and the implemented backend service lay the groundwork for
more advanced learning analytics on the platform.
41
A Thesis archive content
The thesis archive includes the attachment.zip file, which (after unzipping)
contains two folders with the following contents:
• Irs: Source code of the implemented LRS described in this thesis
• xapi-demo: Source code of a simple frontend demo that demonstrates
an example of generating xAPI Statements regarding
question tracking, including session lifecycle events (initialized,
unfocused, exited, etc.)
For information on how to run the applications, please refer to the
README. md files located in each of the folders.
42
Bibliography
1. xAPI.com [online]. Rustici Software LLC, 2025 [visited on 2025-
03-16]. Available from: https : //xapi . com.
2. VIDAL, Juan C ; RABELO, Thomas; L A M A , Manuel. Semantic
Description of the Experience API Specification. In: 2015 IEEE
15th International Conference on Advanced Learning Technologies.
2015, pp. 268-269. Available from DOI: 10.1109/ICALT. 2015.128.
3. IEEE Standard for Learning Technology-JavaScript Object Notation
(JSON) Data Model Format and Representational State
Transfer (RESTful) Web Service for Learner Experience Data
Tracking and Access. IEEE Std 9274.1.1-2023. 2023, pp. 1-103.
Available from DOI: 10.1109/IEEESTD. 2023.10273185.
4. xAPI Data Specification [online]. Advanced Distributed Learning
(ADL) [visited on 2025-03-16]. Available from: https : //github.
com/adlnet/xAPI-Spec/blob/master/xAPI-Data.md.
5. xAPI Registry [online]. Rustici Software LLC [visited on 2025-03-
20]. Available from: https: //registry. tincanapi. com/#.
6. CLOKE, Harry. What is SCORM? Your Guide to the Most Popular
ehearning Standard [online]. Growth Engineering Authoring Tool,
L&D, Learning Technology, 2021 [visited on 2025-04-02]. Available
from: https : //www. growthengineering. co . uk/what-is-
scorm/.
7. SCORM.com [online]. Rustici Software LLC, 2025 [visited on
2025-03-22]. Available from: https : //scorm. com.
8. BAKHOUYI, Abdellah; DEHBI, Rachid; TALEA, Mohamed; H A JOUI,
Omar. Evolution of standardization and interoperability
on E-learning systems: A n overview. In: 2017 16th International
Conference on Information Technology Based Higher Education and
Training (ITHET). 2017, pp. 1-8. Available from DOI: 10 .1109/
ITHET.2017.8067789.
43
BIBLIOGRAPHY
9. MILLER, Brian; RUTHERFORD, Tammy; PACK, Alicia;
VILCHES, George; INGRAM, Jim. cmi5 Best Practices Guide: From
Conception to Conformance [https: //adlnet .gov/publicat ions/
2021 / 09 / cmi5 - Best - Practices - Guide/]. 2021. Accessed:
2025-03-23.
10. MILLER, Brian. Deep Dive: Verbs [online]. Rustici Software LLC,
2013-06-20 [visited on 2025-03-25]. Available from: https : //
xapi.com/blog/deep-dive-verb/.
11. AGRAWAL, Pankaj et al. xAPI Video Profile [online], [visited on
2025-05-01]. Available from: https: //liveaspankaj .gitbooks.
io/xapi-video-prof ile/content/.
12. Document: visibilitychange event [online]. Mozilla Developer Network
( M D N ) [visited on 2025-05-04]. Available from: https :
//developer . mozilla . org/en-US/docs/Web/API/Document/
visibilitychange_event.
13. Window: beforeunload event [online]. Mozilla Developer Network
( M D N ) [visited on 2025-05-04]. Available from: https :
/ / developer . mozilla . org /en -US /docs /Web /API /Window/
beforeunload_event.
14. ElysiaJS [online]. ElysiaJS [visited on 2025-05-08]. Available from:
https://elysiaj s.com/.
15. Bun [online]. Bun [visited on 2025-05-08]. Available from: https:
//bun.sh/.
16. TechEmpower Framework Benchmarks - Round 23 results [online].
TechEmpower, 2025 [visited on 2025-05-08]. Available from:
https : //www . techempower . com/benchmarks/#hw=ph&test =
plaintext&section=data-r23.
17. ClickHouse [online]. ClickHouse, Inc., 2025 [visited on 2025-05-
08]. Available from: https: //clickhouse. com/.
18. RabbitMQ [online]. Broadcom, 2025 [visited on 2025-05-08].
Available from: https: //www. rabbitmq. com/.
19. CQRS Pattern [online]. Microsoft, 2025 [visited on 2025-05-09].
Available from: https : //learn.microsoft. com/en-us/azure/
architecture/patterns/cqrs.
44
BIBLIOGRAPHY
20. K H O N O N O V , Vlad. Learning Domain-Driven Design. O'Reilly
Media, 2021. ISBN 978-1-098-10013-1.
21. N E W M A N , Chris; KLYNE, Graham. Date and Time on the Internet:
Timestamps [RFC 3339]. RFC Editor, 2002. Request for Comments,
no. 3339. Available from DOI: 10.17487/RFC3339.
22. Dead Letter Exchanges [online]. Broadcom, 2025 [visited on 2025-
05-21]. Available from: https : //www. rabbitmq. com/docs/dlx.
45