Let’s be honest, we could all use a little help from our friends, and when it comes to Degreed you don’t have to be one of The Beatles to get that help. At the core of the platform is a simple question: How do we help Degreed users discover, develop, and measure the skills they need to accomplish their goals? The data science team constantly strives to improve the product’s understanding of how learning happens and how the learning ecosystem is evolving. On a practical level, this means enabling connections, including connecting users to:
- Jobs that they can aspire to
- The skills they need to excel at these jobs, and
- Content that will help them develop these skills.
One of the most direct ways that we can help encourage users to continuously build their skills is through learning recommendations in their feed, a daily blend of personalized content recommendations for each user. Think of it like your super-smart friend recommending a good book or a podcast to you. Here’s how it works.
How do we populate the learning feed?
If we look at what one particular person learns in a day, it becomes clear that learning spans many topics, can come from many sources, and often involves the serendipitous discovery of interesting things in the course of daily life. Accomplishing all of this in a single stream is challenging, but it is this type of organic learning that we strive to emulate in our platform. To capture this diversity, we combine several distinct types of curation: personal, social, and skills-driven learning, the last of which is machine-curated.
Personally-curated content
Personally-curated learning recommendations consist of the materials that a user has already discovered and saved for later (e.g. recently saved content and enrolled pathways). By working these items into the learning feed, we help remind users of the great content that they’ve bookmarked so it won’t be forgotten. Personally-curated content comes in two forms: from the queues and Pathways.
How are queues utilized?
Think of the queue as your “to learn next” list. It’s a great way to keep track of your learning for the future. If we look at engaged Degreed learners who are using the queue over the last 6 months, we see the following trends:
- Courses are most frequently added to users’ queues, followed by articles, videos, and books.
- On average, users have 3-4 pieces of content in their queues from the last 6 months
- On average, users who interact with their queues have completed 2-3 items from their queue within the last 6 months, most of which are articles.
How are Pathways utilized?
Pathways are one of the most highly used and engaging formats for learning recommendations — 63% of the engaged users we examined are following at least one pathway, the average being 3.8 pathways. And users continually add new pathways, with one-third of these users adding at least one pathway in the last 6 months.
Socially-curated content
Social curation happens in two ways. The most direct form of social curation is the material that others recommend to you. Indirectly, we also leverage content consumed by the people in your social network on the Degreed platform — your peers, people you follow, people in the same groups as you, etc. For each person in your network, we calculate a connectedness score; the more connections you share, the more likely that you’ll see content that they’ve completed. Additionally, content is ranked by relevance to your interests, by its recency, and by its overall popularity.
Skills-driven learning
Skill expansion
Degreed also mixes in machine-curated learning recommendations based on the user’s selected skills and interests. Finding the right learning by topic can be difficult because terminology can differ between sources and jargon in a field may change rapidly. To address this problem, Degreed learns how people talk about the universe of learning. One way we do this is to use the tags that people apply to content by feeding them through a neural network to derive hidden relationships between them. For example, if one article is tagged with both machine learning and artificial intelligence then the network learns that there is an association between the two concepts. By doing this with millions of tag pairs, the algorithm becomes able to suggest closely related terms given any user interest. When we search for content to deliver in a feed, we search not only for the exact topic but also on the related terms in the skill graph.
Inferred interests
One of the improvements we’ve made recently is based on the fact that users consume content far more frequently than they change their skills. To adapt to the user’s changing interests, we derive which broad topics the user is learning about based on the content that they are completing. We then add items to the feed based on which items in that topic area are the most popular. These topic areas are intentionally broader than skills to help bring diversity to our suggested learning items.
How do we measure how well we are doing?
It is important to remember that our goal is to help learners meaningfully and pointedly build skills through learning recommendations rather than just offer content for the sake of consumption. As a result, we want to use metrics that optimize for bringing value to users rather than myopically focusing on things like click-through rate. We track and adjust the performance of the feed by monitoring engagement from users, comparing click-through rates for different sources, listening to user feedback, and conducting user testing.
What’s next?
We will continue to experiment with our approach and convert the best results into enhancements throughout 2019 and beyond. Here are two key questions that we are looking into to move our platform forward:
What is quality content?
How do we optimize for better user engagement without falling into the click-bait trap? And how can we better measure which types of learning contribute effectively to skill development?
How do we match the user to the right learning opportunity at the right time?
On-the-go learning: this is learning in the flow of life — an audiobook you listen to on the way to work, an article you read while waiting in line at the grocery store, or a documentary you watch while riding a spin bike at the gym. How do we identify these times in a user’s day and suggest content in the right size at the right time?
Just-in-time learning: this is learning in the flow of work. How can we help users to discover and track all the things that they learn while they are doing their jobs and connect them to the skills that they are developing?
Deep learning: not the deep learning of AI, rather this is the dedicated learning time that people find — to take a course, to read a book, to learn about something in more depth than they do when in the flow of work. How can we help users remember to carve out this time, recognize when they are engaged in this type of learning, and make the best possible suggestions for what the user wants to learn?
As we say, in true Degreed fashion, we’re never finished learning, optimizing, and improving. Our goal is to work for you and your people, and while we think we’re doing a pretty good job, we’re getting even better. Through recommendations, you have the opportunity to share the wealth — making everyone around you smarter, better, and more skilled. That way we can all discover, develop, and accomplish goals.