Navigating The Corporate eLearning Marketplace: Analytics (Part 3)

All About Analytics And Reporting
everything possible/Shutterstock.com
Summary: What is AICC, SCORM, xAPI, and CMI5? What is a Learning Record Store (LRS)? And what is a Learning Analytics Platform (LAP)?

Analytics And Reporting

Now that a description has been given for the types of users, the data collected on them, and the various pieces of eLearning technology used to do so, an explanation of reports and analytics may be provided. A report refers to a summary of a particular data set; and analytics refers to an interpretation of a report that helps modify business or educational practices.

In order to produce reports and analytics, trainee activity must be recorded, stored, and retrieved. In order to do so, a standard communication protocol must be used. The purpose of these standards goes beyond making reporting and analytics possible; their main purpose is to provide a common language that allows content and systems in an eLearning platform to be interoperable. In this section, we will first cover communication standards and then we will discuss reporting and analytics more directly.

Communication Standards

Three common communication standards for eLearning include: AICC, SCORM, and xAPI. Each standard has different reporting and analytic capabilities. While reporting and analytic functionality can sometimes be added to these standards, it may require significant modification.

AICC

AICC was originally developed before the public release of the internet, and is still the second most widely used standard; however, its popularity is projected to decline, as it is no longer being updated. It is also the standard, of the three mentioned, with the most limited tracking capabilities. This standard allows administrators to track whether a trainee has completed a course, passed or failed an examination, and track the time a trainee spends in a course.

One of the drawbacks of AICC is that it is structured in a way that can make it time-consuming to send and retrieve data to and from an LMS, as it requires multiple actions.

When evaluating AICC compliant solutions it is important to be aware that there are few requirements in order to be AICC compliant. The ease of compliance usually results in the need for additional coding to obtain certain features.

SCORM

The difficulties and expenses associated with deploying an AICC standard led to the development of SCORM. SCORM was originally developed prior to the proliferation of smartphones, and is now the most widely used standard for LMSs. In addition to having the same reporting capabilities of AICC, SCORM is able to set the pace of trainees' learning (i.e., “sequence" the content presented). SCORM also solves the multi-step data transfer problem associated with AICC by utilizing a more efficient packaging process.

There are, however, some instances where AICC is preferred over SCORM. For example, AICC is able to avoid the “cross domain scripting problem that plagues SCORM’s ECMAScript-based communication [2]” because it is HTTP-based. In other words, AICC allows for content located on different servers to be transferred more easily than SCORM.

xAPI And CMI5

To address some of the shortcomings of AICC and SCORM the developers of both standards individually developed new standards. The creators of AICC, the Aviation Industry Computer-Based-Training Committee (also AICC), developed CMI5. And the creators of SCORM, the Advanced Distributed Learning (ADL) Initiative (part of the U.S. Department of Defense), developed xAPI. These new standards were similar but not entirely the same. When the AICC and ADL realized they had similar goals and created similar standards they decided to put their work together. From their collaboration, they concluded xAPI could be used for more applications. They decided CMI5 would be modified to support xAPI. CMI5 can now be thought of as an additional “set of rules” or “profile” within xAPI.

xAPI is the standard with the greatest analytic capabilities. In addition to having the same reporting and analytics abilities as SCORM, xAPI has the ability to track: virtual environments, games, multimedia, group activities, offline activities, mobile activities, forums, and informal learning (i.e., actions conducted outside of the LMS, which may include: emails, social media, web browsing, or actions performed in other systems). The addition of tracking informal learning in this standard is not insignificant, one theory suggests that 90% of learning occurs in an informal way.

The interoperability of xAPI is due largely to it being based off of an “Activity Stream specification” developed by “Google, Facebook, Microsoft and other industry giants to interchange social experiences in a standard format [1]." This standard format records data in both a human and machine-readable way (i.e., “subject-verb-object strings [1]."

With xAPI retroactive reporting can also be performed. Retroactive reporting or the collection of “semi-unstructured data” is a feature not available in SCORM. Retroactive reporting allows for data to be collected prior to knowing what reporting and/or analytic purpose it will have.

The data collected is also more “granular” than SCORM (e.g., instead of only collecting whether a trainee has passed or failed a test, xAPI can record the trainee’s answer to every question in a test and the number of times the test was attempted).

The data collected by the xAPI specification is stored in a database called a Learning Record Store (LRS). The LRS is partitioned into two parts: local and cloud-based. The local LRS stores data when a trainee is offline or has weak connectivity. When the trainee is back online the data from the local LRS is uploaded to the cloud LRS. The local LRS is what allows xAPI to effectively track offline and mobile activity. The cloud LRS can also be located outside of the LMS, in SCORM this database is part of the LMS.

It is important to recognize that xAPI is an augmentation of SCORM, it is not a replacement. “It only (potentially) replaces the data communications protocols and models of SCORM; the other aspects of the SCORM such as content packaging and delivery are not covered by the xAPI [1]." This aspect of xAPI is what makes CMI5 important.

The rules defined by CMI5 allow xAPI to be used in such a way that it defines how content should be packaged and delivered in an LMS. CMI5 brings many of the desired features found in SCORM (that xAPI does not define) into the xAPI environment, giving you one of the greatest levels of functionality and interoperability in an eLearning platform.

Learning Analytics Platform

The databases these standards generate are not enough to produce reports and analytics; in order to do this, they sometimes require additional software to retrieve, organize, and present the data found in a database. This additional software may include a dashboard (User Interface for viewing reports and analytics) and an analytic engine.

In xAPI, the analytic engine is sometimes referred to as a Learning Record Consumer (LRC); and the combination of LRS, LRC, and dashboard may be referred to as a Learning Analytics Platform (LAP).

Be aware that some LAPs are marketed as LRSs. When integrating an LRS it is important to know whether it is only an LRS or whether it is also a LAP. If it is only an LRS the integration of additional plugins or third-party software is required to produce analytics and reports. It is also important to know that some LMS solutions build their own LRS and LAP and others integrate third-party software into their platform.

The quality and functionality of LAPs vary greatly. In addition to LAPs having a different set of default reports and dashboard quality, some may or may not feature: specific, dynamic, ad-hoc, or hybrid reports.

Specific reports are reports that are customized for a particular user or group of users. Dynamic reports are reports that are generated based upon the results of previous reports. Ad-hoc reports are reports that can be generated by an admin user setting parameters. And hybrid reports are reports generated from multiple databases. LAPs with hybrid reporting capabilities are usually reserved for enterprise-level companies due to their cost.

Please leave a comment below or reach out to the author for source citations. This article is Part 3 of a four-part series, read Part 1, Part 2, and Part 4. 

References:

[1] Choosing Series

[2] Evolution of SCORM

Sources

Read Also

Learning Analytics LMS SCORM
Write or read Comments
Close
Change your privacy settings to see the content.
In order write or read comments you need to have functional cookies enabled.
You can adjust your cookie preferences here.