For several years, virtual reality (VR) has been revolutionizing many sectors, and training is no exception. By offering immersive and interactive experiences, VR allows learners to immerse themselves in simulated environments, thus promoting a better understanding of concepts and increased retention. However, the rise of virtual reality in the field of training raises new questions. How can we track and assess learners’ progress in virtual reality? How can we collect and analyze the data generated by their interactions? This article aims to identify the current concepts, tools, and best practices to meet the specific needs of tracking training in virtual reality.
LMS, SCORM: Definitions and Limitations
Before diving into the analysis of the state of the art on tools and methods for tracking learning in virtual reality, it is worth recalling a few concepts.
An LMS (Learning Management System) is a platform that allows the creation, management, and tracking of online training programs: e-learning modules, quizzes, video capsules, virtual classes… LMSs notably allow:
- to create users (learner, administrator, tutor, etc.),
- to upload training content (learning units) into courses or programs
- to assign them to learners.
Most LMSs allow the recording and analysis of basic feedback sent by training content using the SCORM (Sharable Content Object Reference Model) communication standard. A reference model for shareable learning objects, SCORM encompasses a set of technical standards that enable training content to communicate with the LMS in a standardized manner.
The main existing learning management tools (LMS) are primarily designed for traditional online training: these LMSs do not allow hosting virtual reality content and, therefore, do not collect primary training data (score achieved, time spent, number of attempts, etc.).
SCORM vs. xAPI: What are the differences?
Although it has been a reference standard for many years, SCORM has certain limitations compared to new learning technologies such as virtual reality:
- Firstly, SCORM is strongly tied to the LMS environment. Its use in other contexts, such as immersive training or VR simulators, is therefore not possible.
- Moreover, when used for any other type of learning, SCORM primarily focuses on final results (score reporting, pass/fail), without providing a detailed (granular) view of the learner’s journey or their interactions with the content.
In response to the limitations of SCORM and the need for greater precision (or granularity) in data collection, particularly in immersive learning environments, xAPI (Experience API) emerged.
Unlike SCORM, xAPI allows capturing all interactions of a learner with content, whether it is a click, an answer to a question, or an action in a virtual environment.
Then comes the LRS...
The LRS (Learning Record Store) plays a central role in the xAPI ecosystem.
Unlike the LMS, which primarily manages online courses, the LRS is designed to store all data related to learning experiences, in the form of xAPI statements.
A true “logbook” of learners, it collects and analyzes in-depth their interactions with the training content. Whether it’s movements, gestures, facial expressions, or even eye tracking, the LRS records a multitude of data. It can track both formal experiences (such as e-learning modules) and informal ones (such as browsing websites, attending events, reading books, etc.). Few activities escape this tracking capability, allowing for a personalized and detailed analysis of each learner’s journey.
This information is called “learning traces”, “xAPI traces”, or “xAPI statements”.
The challenges of data reporting for immersive VR training
The integration of virtual reality into training programs opens new perspectives for learning but also raises challenges in terms of tracking and assessment. Collecting and analyzing the data generated by learners’ interactions in VR is a major challenge.
By enabling a better understanding of learners’ interactions, this data offers new perspectives for personalizing training paths, improving content effectiveness, and developing new teaching methods tailored to the immersive environment.
- Engagement measurement: by measuring the time learners spend on each task or module, it is possible to identify the most engaging content.
- Identification of difficulties: By analyzing learners’ actions, it is possible to identify the areas where they encounter difficulties. Analyzing the mistakes made highlights misunderstood concepts and allows for adjustments in explanations.
- Personalization of learning: Based on the data collected, it is possible to personalize each learner’s training path by offering activities tailored to their profile and needs.
- Evaluation of content effectiveness: Data analysis allows for the assessment of the effectiveness of different contents and improvements to be made accordingly.
Current limitations of LMS in the face of VR challenges.
LMS were originally designed to meet the needs of traditional online training (i.e., through a computer interface). Although they are powerful tools, almost all LMS do not allow hosting or creating content that can be played through a virtual reality headset.
A partial solution exists and involves using the LMS platform’s web service. When available and configured, it allows an external application to communicate with the platform to exchange information (user authentication and SCORM data transmission).
The implementation of this web service is rarely included in the installation and configuration services of the platform and generally requires additional services. Moreover, integrating the communication component with the web service into VR content requires web development skills in addition to VR application development expertise, which represents an extra cost for each new VR content.
LMS, LRS: Market trends.
Faced with the limitations of traditional LMS, the market is rapidly evolving to meet the specific needs of virtual reality training. Several trends are emerging:
Emergence of VR-specialized LMS platforms.
New LMS platforms are emerging, specifically designed to manage VR training. These solutions can natively integrate game engines (Unity, Unreal Engine) and offer advanced features for the creation, deployment, and tracking of immersive experiences.
Unfortunately, these commercial platforms are mostly proprietary solutions that inevitably add to the cost of traditional LMS platforms. This results in a doubling of the subscription cost for training platform services. Moreover, these solutions may raise legal issues regarding personal data protection (SaaS mode with cloud storage).
On the side of open-source LMS...
In open-source LMS platforms like Moodle, the lack of native support for xAPI learning traces represents a major limitation, especially for immersive training (VR, AR, and XR). This means that, without the addition of specific features or plugins, these platforms cannot record and analyze in detail the complex interactions of users in immersive environments, such as movements, gestures, or specific actions performed in a virtual space. This gap significantly reduces the ability to track progress in rich and interactive learning experiences, which are crucial for immersive training.
However, with the rapid evolution of technologies and standards like xAPI, opportunities are emerging to overcome these limitations. Recent developments allow for smoother integration of immersive content into LMS platforms like Moodle, particularly through plugins or gateways to external LRS (Learning Record Stores). These LRS, interconnected with Moodle, can collect and store data generated by immersive environments, enabling detailed and comprehensive tracking of learners’ interactions in virtual, augmented, or mixed reality (XR).
Thanks to these advancements, it is now possible to better integrate immersive training within learning paths managed by open-source LMS, thus paving the way for more engaging learning experiences and a more precise assessment of skills developed in immersive contexts.
From LMS to a specialized tool ecosystem: Xlearning (Experience Learning Platform).
The combination of open-source and specialized tools, connected by standard xAPI protocols, allows for the creation of a customized learning environment that is more flexible and tailored to specific needs. The choice of the most suitable solution will often be based on the combined and specific use of specialized tools to create a true “learning experience ecosystem.” This involves the specialization of tools, such as:
- The LMS, which will be used solely to manage learner authentication, host modules, and assign learning paths.
- The “launcher”: installed on the immersive device, it will link the connected user to the list of VR modules they can access by communicating with the LMS. This component will automatically download and install the VR module on the headset (if it’s not already installed).
- The LRS will be solely dedicated to storing xAPI learning traces, including those generated outside the LMS.
The learning trace data from the LRS will be leveraged by a data analysis solution (Data Learning Analytics) to generate customized reports with key performance indicators (KPIs) relevant for managing your training programs.
The LMS market for VR is undergoing rapid transformation. As a result, organizations must adopt a pragmatic and tailored approach to selecting the right combination of solutions. Supported by strong technical expertise, you will be able to create immersive and effective learning experiences.
Conclusion
To fully leverage the potential of VR training, it is recommended to adopt a modular and flexible approach:
- Prioritize open and modular solutions: Open-source solutions offer greater customization and better integration with other tools.
- Define a data collection and analysis strategy: It is essential to clearly specify the data to be collected and the key indicators to monitor in order to select the appropriate analysis tools.
- Focus on interoperability: Choose tools compatible with xAPI standards to facilitate data exchange between the different components of the ecosystem.
- Collaborate with experts: Engage specialists in pedagogy, technology, and data to support you in designing and implementing a solution tailored to the specific needs of your organization.
The market for VR learning tracking tools is constantly evolving. We can therefore expect the emergence of new, increasingly advanced and specialized solutions. Artificial intelligence is also expected to play an increasingly significant role, particularly in data analysis and the personalization of learning paths. Open standards like xAPI will continue to evolve, promoting interoperability between various tools.
