Data-informed decision-making can take your organization’s learning & development strategy to the next level. As a learning leader, you do not need to become a data scientist to design and implement a data-informed learning strategy, but you will need an appreciation for the central concepts so that you can implement the right strategy for your organization—and guide learners to their optimal learning path.
This article will introduce seven concepts that learning leaders need to understand to begin crafting their data-informed learning strategy.
What L&D leaders need to know about data skills
When deciding which data skills you need in your skill set, focus less on specific tools, such as statistics software or programming languages. Instead, focus on how you will obtain the data you need and what you do with that data. That entails understanding the need for—and having the skills or resources to do—the following:
- Figure out what data you need to answer your business- and performance-related questions
- Identify or create a “supply chain” for your data
- “Clean” the raw data you gather to make it usable
- Create data profiles or governance protocols
- Examine or “interview” the data to spot trends, identify outliers, and begin to reveal the stories the data tell
- Understand the “levels” of data analysis and operate at levels beyond evaluation
- Communicate the data in visualizations, dashboards, and other communications to your team, to executives and managers, and to others who will use it in data-driven decisions
The next sections provide an overview of these foundational skills.
1. Define your goals and questions
Before the L&D team can gather learning data, they need to know what their goals are and identify the data that can help them achieve those goals. As a learning leader, you will likely have goals at the organizational level; you will also need to consider individual learners’ goals as well as the goals of the L&D team.
Instructional designers, performance consultants, and L&D professionals have a unique lens on how to reach those goals and what data might be useful. This is an area to collaborate with your colleagues on the business side, too, as their goals are generally measured in business metrics, not training metrics.
Learning what questions to ask and what data might answer them is the first skill needed to develop a data-informed learning strategy.
2. Define or create your data supply chain
A supply chain is a series of processes: Raw material enters the system; one or more processes transform that material; and the end product is something that is useful (to you or to others). In a data supply chain, the data are the raw material; analytics performed on the data transform it; and the end result is information and insights that your company can use to improve performance.
Your data supply chain includes your data sources, how and where you store data, and where you are using it. The supply chain includes stakeholders and challenges as well. One example: If your organization has relied heavily on classroom-based training or informal learning, you may have very little data beyond course completions and dates. One of your first challenges in building up your supply chain may be to figure out how to capture the data you need from these learning events.
3. How and why you need to ‘clean’ your data
Raw data is usually filled with “junk”: duplications, inconsistencies, similar data gathered by systems that label each data point differently. The most basic example is the vast range of sources of learning data. Each source, whether a SCORM eLearning module, a quiz, a record from the HRIS, a learner’s login credentials, a mobile microlearning platform that sends data to the LMS, or something else, might identify the learner differently and record data in fields with different names.
“Cleaning” your data means:
- Standardizing this information and eliminating duplicate data
- Fixing any structural errors, such as inconsistent labeling or classifications
- Identifying errors and missing data
- Validating whether your “cleaned” data set makes sense and enables you to prove or disprove your working theories and gain insights
4. Data profiles & governance
Data governance encompasses the processes and stakeholders that determine how you handle and use data in a consistent way throughout your organization.
In a limited data environment, such as SCORM-based eLearning courses or a classroom-intensive delivery approach, there isn’t that much data or very many systems to worry about, and data governance may not have even crossed your mind.
However, as your learning strategy matures and you engage more systems as data sources and encounter richer data, you will need to pay attention to governance. Following a consistent data governance protocol can also improve data security and make it easier for your organization to comply with standards and use your data effectively and efficiently.
5. Initial data exploration & analysis
Understanding enough about data and statistics to identify trends, determine whether a correlation is meaningful, and know whether an outlier is likely due to a data entry error or represents legitimate data enables you to begin exploring your data set.
This is where partnering with the analytics team in your organization may be key. Look around your organization for the people analyzing business intelligence (BI) data or the marketing team that handles a lot of similar data about your customers. These colleagues use and explore data in their job roles and can likely help you figure out how to start exploring your learning data.
6. Analyze the data at multiple levels
Data use and analysis within L&D often starts with evaluation — looking at what is happening — as this is a familiar space for us. Evaluation can occur at the individual learner level, looking at progress and results in training modules. It can also be at a bigger-picture level: Is the organization meeting compliance training requirements or business targets? What is the experience within the learning like? Either way, this type of descriptive analysis is the most basic level of data use and analysis.
Stepping up a notch, L&D might move to diagnostic analysis, asking why things happened the way they did and you got the results you did. For example, if one team’s performance suddenly improves, you can dive deep into the data to determine what sets this team apart from the others: What are they learning? Did team leaders or members take a course, attend a conference, complete a mentorship? What outside sources are they relying on?
The next level is asking “what will happen if we do …?” This predictive analysis happens all the time in marketing and consumer apps: Companies, whether Amazon, Netflix, or your local burger joint, look at patterns and then proactively offer those combinations of products. People who liked this show also watched that show. People who bought product in your cart also bought other product.
Finally, pulling all of this data together, L&D can move into prescriptive analytics. If you want to drive performance in a specific way, what do you need to put in place for that to happen? Knowing whether learners engaged with and succeeded with specific training, and whether those learners had better performance, can help you develop training and performance support that could drive the results you seek.
For a deeper dive into this, check out Ben Betts’ Learning Analytics Maturity Model—you can benchmark your organization against others.
7. Communicate your data
The last concept and basic data skill brings L&D professionals back into our “wheelhouse”: communicating data requires choosing the best format to tell the story the data reveals. This means choosing a visualization format and using a tool—Excel, an analytics platform, a learning record store, or other—to put the data into a format that non-data-experts, such as business leaders, individual learners, or their managers, can understand.
What’s next?
L&D leaders with a mature data analysis strategy can then use data in other ways as well, such as personalizing training recommendations and delivery or optimizing workflow with the right performance support and tools offered to individual employees based on their performance and results.
Join the Learning Leaders Alliance & build your skills
Are you seeking the strategies and skills required to navigate the needs of today’s ever-changing workplace? Are you an experienced or aspiring leader looking for a community to connect with to explore today’s biggest learning leadership challenges?
The Learning Leaders Alliance is a vendor-neutral global community for learning leaders who want to stay ahead of the curve, and for aspiring leaders wanting to build their skillsets. The Learning Guild’s Alliance Membership package includes access to exclusive digital events and content curated for today’s modern learning leader, as well as opportunities to attend in-person learning leadership events held around the globe. See the details here.