fbpx
Home Blog Page 512

Commonality in Learning Systems – Functionality

By
onlinefreecourse
-
0
Commonality in Learning Systems – Functionality

I never thought about it before. I see it all the time—and I suspect you do, too.

You take it for granted. Why would a restaurant, or at a “Farmers Market, ” or even a franchise misstate, okay, lie, about it?

Farm to Table.

I admit, upon reading the first article, I read the others.

Yes, it was from Tampa, and yes, it is from 2016; however, there is no doubt, at least in my mind, that it exists in your location or any place you visit that mentions “farm to table.”  

Or, in the case of the “farmers market” from my farm (not mine, as I do not have one), it isn’t always the case.

The question then is, what will you or I do about it?

Will we ask to speak to the head of the restaurant, or in many cases, it is the manager who is there, to find out? I doubt the server will know.

Will we think about it? Will we forget until after we get home and then say, “Hmm, I should have asked..”

I bring all of this up because we often have questions when looking at buying any learning technology (Learning Systems, AI tools, e-learning tools, VR/AR, content creators, etc.) and even 3rd party content, whether you are hiring someone to build it for you, or you are purchasing it.

The Common with Learning Systems (LMS, Learning Platform, Talent Development, Combos)

It is very common for people looking to buy any learning technology (I now place learning systems under this overall umbrella term) or content to ask the typical questions that many people ask.

Many go so vague or ambiguous that every system has it, but the prospect isn’t aware, so they push out the common and then expect people to retort as though it is uncommon.

The List of Common Questions – If you are asking these, I can tell you that every system offers it

Can accept videos – i.e., upload them to be used in their system

  • Modern UI/UX that is easy to use – Every vendor will claim they have this, even though I see many that do not, unless you believe that Microsoft should have never discontinued Windows 2000.
  • Has a catalog
  • Assign users to _____
  • Can you take quizzes or assessments, or can you do so
  • Self-registration, forget password, custom domain (this gets a bit tricky, I’ll explain shortly), ability to schedule webinars/seminars – for webinars, you can connect to X web conferencing, or the vendor says that you can do webinars or talk to folks via a webcam.
  • Some level of skills management – skill capabilities; ditto for the calendar – widespread here, scheduling sessions – common,
  • Customer Training – Can skin and add your logo – and can skin (i.e., change the colors) and add a logo to any additional tenant that the system has – think child or sub-portal
  • Can skin and add logo – any system that exists out there
  • Provides outstanding support – They all claim to do so. Take it for what it’s worth.
  • Offers integrations – most common are HCMs
  • Mobile Responsive – This doesn’t mean they have a mobile app.
  • It can be seen on a mobile device – think access via the browser.
  • Micro-Learning – They can all do it – micro- only means short. Oh, and vendors were able to do this in 2000; it’s not a new concept.
  • Certificates – Most have templates, the ability to edit the certificate, a certificate can be e-mailed to the learner, a certificate can be provided upon completion of content/course/path, and so forth
  • The system offers assigned learning – with a lot of things you can do, from assigning learners to a specific course or courses they have to complete to locking down a course, so they have to go step by step before moving on to a learning path
  • Support for the hierarchy at the enterprise, business unit, or other desired levels
  • Repeatable creation of events based on a standard course template (e.g., number of days, min/max seats, title, description)
  • Has LXP capabilities
  • Create workflows
  • Compliance – Audit trail is quite common

Administration Common items – I very rarely see a vendor saying they do not have this – but there is one that isn’t common (you won’t see it below, but will shortly)

  • Administrators can set multiple levels of approval (e.g., no approval needed, supervisor approval, instructor approval, etc.)
  • Administrators can create a learning path, journey, or curriculum path.
  • Administrators can add content/courses to the system and add or remove it from a curriculum/learning path
  • Assign and enforce access based on user profile
  • Assign content/path based on skills, ditto for a job role
  • Administrators can limit access to catalog items based on skill profile or skill proficiency
  • Administrators can create skills – needed, requested, and assigned to learners, learners, or in general in the system
  • Administrators can compare learner to learner based on skill(s) tied to a role
  • Capture multiple user attributes (e.g., Organization, Department, Manager, Job role, etc.)
  • Create user groups based on defined attributes (e.g., organization, department, country, etc.).
  • Control profile settings by field (e.g., allow learners to change e-mail addresses but not change user name)
  • The administrator can limit access to catalog items based on user profiles (e.g., job role, location, customer, partner, etc.).
  • Allow administrators to create and assign system roles
  • Administrators can create job roles and job titles, edit and delete
  • Administrators can view reports can schedule to go to X, Y, and Z
  • Administrator home page
  • Allow administrators to track attendance status (e.g., no show, canceled, attended) of all types of training (e.g., instructor-led, online, virtual)

If the system offers or has a manager’s section

  • Manager see their team in a manager dashboard or view
  • Managers can assign training to teams and/or individuals
  • Managers can update on the job training, mark training complete
  • Managers can approve learner requests or deny it
  • Manager can add/reject/update/approve learners/users
  • Managers can compare learner to learner (within their team) on a variety of items, including skills
  • If skills are involved – Managers can review skill ratings and offer what they see as the appropriate rating
  • If skills are involved – The manager can assign skills or recommend skills the learner needs or, ideally, needs
  • Managers can receive reports (if the administrator approves them)

If the system offers an Instructor area

  • Instructors have access to course enrollment information
  • Instructors can push surveys/assessments during active class or post-class
  • Instructors can update rosters
  • Allow instructors ability to print rosters, send e-mails to participants, and update course attendance post-class

Reporting

  • Offers reports – and provides data, including what content X took, what they completed, and other learner data
  • Reports provide overall information by group, department, etc., and individual
  • Ad-hoc

Learner Environment

Think the learner side – Now, whether it is excellent or not, is another story – think of a chapter in the life of learning

  • Search functionality allows learners to find specific courses, materials, and delivery formats based on keywords, titles, delivery formats, and other tagged identifiers (Some systems limit what can be generated, so think of this more as a search capability)
  • Courses have standard attributes regardless of delivery mechanism, including title and description.
  • If the system offers “opportunities” – it is viewable on the learner side with additional information and specifics – the learner can apply. Opportunity will list requirements – this won’t stop someone from applying anyway.
  • Learners can enroll in or request enrollment for a course, content, group, etc.
  • Learners can view the entire catalog and select their courses
  • Learners can be assigned courses/content, learning paths
  • Learners can view documents and download them (Common is PDF)
  • The learner can see their status, i.e., a progress bar, percentage of completion, or whatever format the vendor has presented it in – some do use colors and expect the learner to know what they mean.
  • Learners can be assigned materials, videos, etc.

Skills

I will note that if you are buying a system specifically focused on customer training, as in 95% or higher of their clients are using the system for customer/client/partner/distributor training, etc. – they will not match the skill capabilities of a system that is only dedicated to skills OR a system that is combo – i.e., customer and employee, or employee only – think a talent development system.

Equally how good it is, or robust for these items is debatable – a lot stink at it, or uh, not good – but hey, someone loves them – and that is all that matters – I bet they love candy corn too.

The List of Skills

  • Create catalogs of content only for job roles tied to skills
  • Curriculum can be set by skills, interests, job roles matched to skill or skills
  • Catalog Search by content related to a specific set of skills/interests (Example: Leadership)
  • Recommends courses/content based on job role, skills
  • Skill gap analysis
  • Skills tied to opportunities (openings or, for example, specific limited opportunities such as a project manager for an upcoming project) in the company/organization – This applies only if the system offers opportunities within their platform
  • Analytics directly tied to career development and acquisition of skills for specific job roles
  • Identify by each learner the skills assigned to them AND the skill(s) they selected
  • Skill Ratings – I can tell you the majority lack one key capability, okay two (specifically what each rating means, and what are the base requirements for that skill – i.e., skill three means you have achieved a level of blah, can do blah blah, blah; and have done blah blah) – I mean what does a two mean without such information? Think this way – I am training you at an ice cream store. What does one mean? At two, you should know how to scoop ice cream and talk to a human across the other side of the counter.

I won’t dive into security, but there are common questions folks ask, and the vendor can do or does; nor will I cover mentoring or cohorts, which will be noted on my newest template. Additionally, mentoring, including coaching, is new to many systems. Some claim that they do, but they do not.

I left out AI – because it is new, and so assumptions that all systems have X or even 10% have X are erroneous.  

The most common one I hear is the ability to create content or courses using AI.

The reality here is that if a vendor has jumped into AI, they will likely focus on the low-hanging fruit areas of content and course builder in the system, creating quizzes and assessments.

For whatever reason, nobody seems to ask about token fees – or whether the AI authoring tool or capability, if offered by the system, is included or not, i.e., whether an additional fee applies.

Nor what LLM they are using. An important question to ask. Most use Open AI.

I omitted the common questions about the system’s cost, use case, and so on.

Lastly, I avoided commonalities in a couple of other areas simply because it is easy for a vendor to say, ‘We have it,’ and this is often achieved by them implementing a workaround to achieve the desired results.

Bottom Line

Speaking of workarounds to give the impression that the system has this capability, do you assume that the salesperson is familiar with the entire system?

What about they know what is in the works and can provide such information to you?  

Even if you are talking to the head of sales?

Do you believe everything that comes out of the salesperson’s mouth? Because they are the authority here, and thus, if they say it is true, it must be.

Do you ask for specific information only to receive an SLA that is presented as providing such details?

If the salesperson and you are not meshing, do you deal with it, grumbling under your breath?

All of these are the farm-to-table experience.

Just without the vegetables and meat.

Oh, and

the

Farm.

E-Learning 24/7

Are You a Fan of Anime?

By
onlinefreecourse
-
0

Once a niche, countercultural art form, anime is now mainstream. What are your experiences with anime, and what do you think of its explosive popularity?

Word of the Day: montage

By
onlinefreecourse
-
0

This word has appeared in 190 articles on NYTimes.com in the past year. Can you use it in a sentence?

Teach Taxicab Geometry With Steven Strogatz and The New York Times

By
onlinefreecourse
-
0

Welcome students to a world of math where pi equals 4 and circles aren’t round.

Are Youth Sports Getting Too Expensive?

By
onlinefreecourse
-
0

Organized sports for young people have become a multibillion-dollar industry in America. How is this affecting high school athletes’ experiences and access to play?

Braces

By
onlinefreecourse
-
0

Metal braces are suddenly cool. What do you think of the trend?

Word of the Day: comatose

By
onlinefreecourse
-
0

This word has appeared in 31 articles on NYTimes.com in the past year. Can you use it in a sentence?

Weekly Student News Quiz: Charlie Kirk, Test Scores, Mars Rock

By
onlinefreecourse
-
0
Weekly Student News Quiz: Charlie Kirk, Test Scores, Mars Rock

Above is an image related to one of the news stories we followed this past week. Do you know what it shows? At the bottom of this quiz, you’ll find the answer.

Have you been paying attention to current events recently? See how many of these 10 questions you can get right.

The Future of Learning: Identifying Disruptors

By
onlinefreecourse
-
0
The Future of Learning: Identifying Disruptors

The disruptors are not just LMS providers; they encompass a range of other learning systems.

Three knowledge management platforms, not your typical LMS or two, and even an LXP.

How did I go about finding these disruptors?

What do I see as a disruptor versus one that isn’t?

Why does Dairy Queen refuse to offer almond or coconut milk Blizzard shakes?

I mean, when you can’t enjoy a heath bar mashed up with Almond milk “dairy”, then you are losing out on a significant audience worth of sales, err, customers.

There is a lot to digest here, and rather than go about it in a traditional sense of a post, I went the Q&A route.

I surmise there will be plenty of people who think, “What about blah blah?” To which I respond, “Who is blah blah?”

A system named blah blah (literally blah blah) would get a view to see what is blah blah.

Q: What is a disruptor in the industry?

A: I went about looking at it from three different perspectives and ways.

First

A well-known vendor with a considerable reputation, numerous clients, substantial capital, and a strong marketing budget is not a disruptor.

They can be in one way – let’s say AI or skills, for example, but across the industry, they wouldn’t slide into my disruptor category.

A vendor who has a shiny UI/UX isn’t a disruptor in the sense of the word.

Yes, you want a slick UI/UX (especially the latter, which vendors often ignore), but having those items alone doesn’t take you into the force of a disruptor.

A disruptor is a vendor that has something – let’s call it panache – that, if leveraged successfully, would take the industry by storm.

I’ve seen disruptors in the past.

They either end up in two categories – you know where this is going.

a. Fail – They disrupted early on – led a segment – was known for it, and then for whatever reason, failed as the market adapted.

I shouldn’t say “whatever” reason, as I tend to see the “Rabbit Hole” effect taking place.

This is where the vendor’s roadmap or feature set is influenced by a major client or a group of clients (not everyone), and then the vendor adjusts rather than deviates.

By the time they realize it, they are no longer disrupting – instead, they try to play catch-up.

They never get there; yes, they land clients, but disruption isn’t a short fix or a quick “tada”.

Tech is full of disruptors that failed to leverage – and failed to push forward.

b. Outpace. Some people will say they lack the capital, funding, staff, or other resources to continue disrupting.

I say poppeycock.

This industry has always had early adopters.

That isn’t new.

The industry has always had disruptors – again, not new.

But at some point, a vendor or two will catch up if the disruptor sits on its laurels and believes by reputation, or number of clients, or who its clients are, or another factor, it ignores the competition.

The dot-com industry was full of disruptors.

A lot of them dropped back – not because of ignoring a bubble, but instead because they never paid attention to the smaller vendors in the back of the room.

They saw only whom they wanted to see.

They – the vendors – ignored the warning signs of – watch what this vendor is doing.

Then whammo – the disruptor vendor becomes a has-been, and in the dot com world ended up with stock worth about one dollar and twenty-six cents (for 40,000 shares).

I should have framed that check.

Actually, the additional shares they found and the sending of a check for twenty-four cents should have been framed.

Anyway, from being the leader of the pack, to being swallowed up by not paying attention.

There are plenty of vendors in our industry that have a gold nugget or two staring them in the face, but instead of powering that as a UVP, let alone a USP, they look at the usual and push that narrative.

That’s not a disruptor; that’s just not understanding the market and what you can tap into to set yourself apart.

A USP means Unique Sales Proposition – it is a must for any vendor in any industry, and in our case, this industry.

You can’t survive, well, you can, without having a USP.

Nevertheless, again, too many push the same USP as everyone else has, and then can’t figure out why competitor J has more clients than they do.

The reason?

Lack of UVP – which is Unique Value Proposition.

You, as the prospective buyer, are looking for that UVP, without necessarily knowing it.

When I look at disruptors, they have that “it” – the UVP.

As a buyer, the biggest issue – the reason folks pick the wrong system is they fail to see the UVP, because either the vendor lacks it (most often the case) or the UVP isn’t being tapped as it should be.

Overall, about the LXP market.

At one time, they truly had the USP and UVP; however, over a short period, they lost the UVP due to a functionality they introduced to the space – “assigned learning.”

Later on, the word LXP – as a whole went sideways from what it was designed to be and do, and what it became.

One of the vendors in my disruptor list is an LXP – what it was designed to be, but at the same time, changing the format enough, the approach that, with some fine-tuning, is going to disrupt.

Q: Are all disruptors perfect systems?

A: No. I do not believe there is a perfect system out there, hence no one scores 100%. All systems can be better.

Q: Do disruptors have the potential to break out even further, truly?

A: Yes. The industry is full of those folks. Docebo started as open source. Nobody knew who they were – at least IMO.   

Then they went commercial, did a few of this and that, and hello – early disruptor.

Others on the “never saw them coming – early disruptors included”

  • ExpertusOne – original incarnation. First vendor to have geolocation in their mobile app – even today, you find systems that lack that, heck even trade show apps – I mean, how can I find the Cornerstone booth in section 309, when I have no idea what section I’m in, because your signage is awful – attend a LTUK show – and you will know first hand.
  • Litmos – Original version – not the version you see today. Strong UI/UX out of the gate and proof that you could be streamlined yet very effective in understanding your audience.
  • Cobent – (RIP) – 100% all compliance platform. Disruptor – they couldn’t adapt enough for compliance
  • Training Orchestra – Training management system – it is all about even management and scheduling here – you want the best? Here you go.
  • Growth Engineering – Truly a gamification platform that understands what is achievable by disruption. Sadly, they couldn’t sustain. Still playing catch-up.
  • Blue Volt – First gamified platform – and first to really target blue-collar workers.
  • Degreed – The behemoth of LXP. I’d add EdCast here too – both disruptors early.
  • Xyleme – an LCMS that was worth its weight. Disruptor – just failed to understand the market (although one vendor, not X here, thinks LCMS is back – no, it’s not)

I could go on, but you get my drift.

Even today, there are dominant players in their respective segments who disrupted the market as a whole – but again, for this list, it’s a bit different.

Secondly

How are they disrupting – and does anyone realize it?

One of the vendors has been around for a while but has pivoted to a more effective target segment, and thus has the real potential to disrupt as a result.

Sometimes, a change in leadership and direction can do that.

A new set of eyeballs – and those that lead that segment better be paying attention.

Thirdly,

Technology change.

Knowledge Management Platforms are a perfect example.

They have been around for more than two decades, but the KM of then is not the same as now, and along the way, the KM segment flattened out and just sputtered along.

It required AI for all things to revitalize the KM side, and I can state unequivocally that you will see more KM features in learning systems to come, not just the standalones I will be referring to.

If ever there was “disruptor alert,” – look, look – this is one of them – the KM side.

Disruptors

Let’s get to the list.

The list Abbreviation

  • Type of system (LMS – Learning Management System, LXP – Learning Experience Platform, KM – Knowledge Management Platform/System)
  • PR – Product Review – Coming by Nov 25. The specific month is shown.

Each vendor presented – click on their name and it takes you to their website.   

I added some text for each vendor, but this will not be an extensive analysis – just the highlights and what I see as the disruptor.

Knowledge Management Platforms

For a system to be defined as a KM, it needs the following (bare basics)

  • Answer Engine – it drives the entire platform. The AE is using Gen AI – the basics are that the learner asks a question, and a response is presented. The learner can click text or a word, and it goes right to that article, content, or video that exists within the platform. Includes cites.

That’s the basics.  

As with any AI function, they can create fake or false information, and thus, you should always verify that the information is correct rather than assuming.

One KM displays the content where the response is located, and nothing else. The usual approach is to show the specific line, etc., and then display the entire document or video as well.

An AE (answer engine) can also answer a question or a series of questions.  

Using a KM is only as good as knowing what to enter into a prompt.  

Being specific is far better than being ambiguous.

Some KM can already tap into multimodal, which means you can add a PDF right in the AE window, and the platform chat assistant, if you will, scans it and summarizes or does what you ask.

You will get to see this firsthand with

Capacity – Which sells an entire system on other customer support stuff, but they do sell the Answer Engine as a separate piece, which is what I am referencing here – i.e., the AE.

Of all the KMs I have seen, capability-wise, Capacity’s Answer Engine does the best job. CAE – my acronym, not theirs, is made up of the former lucy.ai solution.

Could you add CAE to your learning system? Yes.

Could it be used in conjunction? Yes

Can it be solely a standalone, and does it replace your entire learning structure?

Not really, unless you are only using your learning for this type of knowledge on the go.

There are numerous use cases where I can see a CAE excelling, especially with deskless workers and others who cannot access a desktop or prefer not to.

Hence, the on-the-go approach.

However, its power across the board is primarily for the L&D side of the house, particularly in terms of employee onboarding.  

Why tap them in with your top person for some OJD when you can have them tap into a KM to solve similar or likely questions without taking your top talent away from their job?

Of all the KM platforms – the new versions here – Capacity to me – is the most mature, and that isn’t saying much – it just means it has more out of the gate than say what I have seen in the past or currently with other KMs.

The best way to think of a KM is to think up and beyond – and this is why I believe that KMs are going to make a huge impact, plus wise in the industry.

Product Review – Capacity Answer Engine coming in October!

Two other KMs

Joggle Learning – A true newbie out of the gate. Different in appearance compared to Capacity, but still enough intrigue for me to want to know and see more.

Part of the challenge with KMs is their inability to effectively describe what they are for the general population, which is why you see the marketing aspect all over the map.

Knowledge Management on the go is what it is, or say Knowledge Management and leave it at that.

Ardoise is a take on what the capabilities of a KM entail, as it incorporates the development of skills through practice scenarios. However, being new to the space, the vendor that is, the scenarios need some help, as practice is one thing, but reality is another.

No.

I was impressed by how the “Avatar” voice matches their lips – a true lip-sync even at a lower speed, when I tested it.

The Ardoise platform also comes with a “content creator component,” and yes, as a result, an output of some content.

The AI drives this unquestionably.  

And they – Ardoise shows their methodology in the learning angle, which I found intriguing. Hypertextual learning – prescribed. (I added the last part to make it sound more impressive – i.e., prescribed)

LXP

Like the dinosaur, the LXP market is a dying breed – in terms of what it used to be, what it is now, which is really ubiquitous to an LMS, let alone a learning platform.

To stand out then, you need a way to leverage two parts: the LMS aspect with enough legit LXP juice and some AI (I note this, although AI is becoming common every day – quality-wise, it’s a different story).

Bealink, which notes itself as an LXP, is the combo of the two with an AI piece. The content is there. The UI/UX is there, and nobody outside of Western Europe realizes that Bealink has some real potential here.

Before just going “Degreed” for your LXP needs, if you want that, Bealink is enough of a disruptor to get you thinking, “hmm, I never thought about this, this way.”

What is nice about Bealink is that it has everything you need without going for a piecemeal approach, which the original LXPs attempted to go for – and, well, sometimes that isn’t the right strategy to take.

When an LMS is not an LXP

Let me tell you a funny story.

It’s about this vendor with a great learning story to tell, one that pushes gamified learning, which, to me, is far different from gamification.

They have a clever name, a unique journey, a fun front end – which is out of the box with the system, tweaked for each client (included BTW), adds some flair, and screams let’s get it.

Despite having LXP in their name, they definitely are not that – you can ask them why it appears, and it has to do with a domain thing, I won’t bore you.

This is a combo system for L&D and customer training/learning. You could even add this to the association space, and people would like it.

The moment I saw LemonadeLXP.

I thought to myself, ‘total USP plus UVP, plus I want my learners to use it – if I were running training, L&D, HR, or whatever department.’

They, for whatever reason, initially focused on Financial Services, but they are pivoting. This is designed for all verticals/market segments and types of learners.

It shows that you do not have to use a system that looks as though the Louvre oversaw its museum’s sterile design.

It’s also a key reason it’s a disruptor – oh, that, and its functionality, of course, plus the love surrounding gamified learning.

Product Review coming in October.

Checks all the boxes for those wondering.

HALiGHT – A system that also pulls enough weight that all you can have it with cake, too, pushes it off.

Name me another vendor that can have “snippets” which are snippets of video or images, ala Instagram, in appearance for the learners – created and uploaded on the home page?

Please show me a system where the learners are truly the drivers of their own learning engagement and empowerment, tapping into retail frontline/deskless workers as the core audience.

Product Review coming in Oct/Nov.

LMS for the Blue-Collar Workforce

I hear about so many systems that will name a client who is in the blue-collar industry, and therefore, they target blue-collar workers, even though a large percentage of their learners are in the office workforce.

They are not in the construction segment, nor manufacturing on the floor, only, and the list can go on.

There have been those and continue to be those who play here and there, but when you push the narrative of construction as a segment, for example, and more importantly, the idea that learning for a blue-collar workforce can be designed in such a way that it still delivers what learning should be, with a user interface/experience to boot. Knowledge Anywhere is here to greet you.

After years in the desert, when they were solid, KA has found its proper footing and is a disruptor in the way it understands the blue-collar workforce. The challenges, whereas companies sadly see their non-office workforce as limited in terms of dedication – i.e., they can’t take an hour away from the floor to train – that so many have missed.

Every L&D person I have met who has a blue-collar workforce wants them to have the same availabilities for learning as their office workforce, yet the L&D folks run into the C-suite person who doesn’t see everyone as the same in terms of what is available and time.

Knowledge Anywhere, though, gives those C-suite folks who think the blue-collar is secondary (without saying it out loud), that this is a system for your workforce out on the job site or on the shop floor with enough key areas to disrupt.

Product Review coming in Oct/Nov.

Bottom Line

The Disruptors of 2025

LemondadeLXP, Ardoise, Capacity, Joggle, Bealink, Knowledge Anywhere, and HALIGHT.

E-Learning 24/7

This blog is dedicated to the memory of a close friend, who started as my boss, then became my mentor, and ultimately became a disruptor in the association management segment: Joe Lynch.

Behind the Build: Skills Tracking

By
onlinefreecourse
-
0
Behind the Build: Skills Tracking

It’s one thing to make it to the end of a course — it’s another to know what skills you’ve actually gained from it. Whether you’re switching careers, leveling up in your current role, or just trying to stay sharp, it can be hard to pinpoint what you know and how well you know it. 

We recently rolled out a new skills tracking feature for Codecademy learners, born from user research into our “upskiller” audience — people actively building new skills or deepening existing ones to advance their careers or stay current in fast-changing fields. 

“We did a deep dive into our upskiller segment and discovered re-emerging themes about what learners really wanted,” explains Mark Hannallah, Group Product Manager at Codecademy. The research revealed three core pain points: learners wanted to understand what skills they’d gained, assess their proficiency levels and knowledge gaps, and find meaningful ways to apply those skills. 

The solution seemed straightforward, but building a system that could automatically extract skills from our diverse content library and deliver precise, skill-based recommendations (beyond just course and path recommendations) was the real challenge our engineering team had to solve. We needed AI to parse and structure skills from an enormous, unstructured content library — something that simply wouldn’t have been possible with traditional approaches. 

The project: Create a system that tells learners what skills they have, lets them self-assess, and provides project-based practice opportunities. 

This cross-functional effort involved Product, Design, Engineering, and Curriculum teams using generative AI to parse course content, backend services to integrate skill metadata across systems, and front-end experiences to surface skill tracking in learner journeys. 

Read on to hear how the team brought this feature to life — from early brainstorming to launch and beyond. 

Investigation and roadmapping  

Mark Hannallah, Group Product Manager: “The first step in any project is having a clear enough understanding of the problems you’re aiming to solve, and then breaking those into discrete steps to get started. We were trying to diagram out what was a pretty complex problem — the relationship between what you learn, the skills you walk away with, and how those things ladder up. We had lots of FigJam sessions and brainstorms.” 

Jerimie Lee, Staff Product Designer: “In the earlier stages, there was a lot of jumping on a call and really taking into consideration the goals that we’re looking at and then each other’s expertise. Everybody on the team has primary expertise, but we also kind of flexed across each other’s domains a little bit.”  

Neil Daftary, Engineering Manager: “One of the challenges was philosophical at the beginning where it was like, what is a skill? The language that we were using for it was challenging to define — but also I like that kind of stuff. For me it was it was really fun to be in that space and in the definition stage. I really enjoyed all the FigJams and brainstorming.”  

Mark: “I’ll give credit to the process we follow — it’s called the ‘shape up’ process. It’s somewhere between quarterly planning and sprints. We had a really robust approach where engineering, design, and product disciplines came together to talk through different use cases, sketch out flows, and work backwards from there to build out the system requirements.” 

Good to know: We define skills as broader capabilities that are built from specific techniques or concepts. 

Implementation  

Mark: “Prior to this project, we didn’t have structured metadata about skills associated with our content. So with generative AI, we were able to parse all our course content — past and present — and derive the skill metadata. We then structured that across our catalog.” 

Neil: “One of the cool things I’d highlight is using generative AI, but with a human in the loop. There was a ton overview that Mark and [Senior Instructional Designer Alex DiStasi]  did to make sure our skill metadata and outcomes was correct. 

I worked with one of our engineers to write out scripts that basically query our content and generate these nice CSV or Excel files.” 

Mark: “There’s also a platform layer: as learners complete content, we needed to track their progress toward skills. That meant iterating on and expanding robust platform services to pass metadata between the front end and back end — essentially tracking how learners engage with skill content. 

Then there’s the front end — the user experiences and touchpoints. We had to figure out where in the learner journey the feature should appear and test those ideas with users.”  

Neil: “Language-wise, we used JavaScript and on the platform side was Ruby.” 

Troubleshooting  

Mark: “When designing a system, it’s got to be a complete, functioning system — or else it just doesn’t work. It took a lot of thinking through, discussing, and trying different things to come up with an end-to-end system that works. If something fails, then the system isn’t designed the right way. 

That kind of thing can be tough — you think you’ve made progress, and then it’s a couple steps forward, a couple steps back. But for me, we were really grounded in strong user research, and I think we understood what was important to the audience we were designing for. 

At the end of the day, we were able to return to the problems and figure out how to solve them. That’s what gave me energy throughout the process. Product teams are in a ton of meetings, but this class of meeting — where you’re creatively solving problems that benefit the user and the business — tends to be more fun and less draining.” 

Neil: “One of my main lessons was just the importance of working cross-team — and the ‘contracts’ we formed to deliver these features. It was a learning process, especially since our team hadn’t worked closely with platform historically. Figuring out a good way to collaborate, staying up to date on what’s possible with the current data model and what’s not — that was key. 

Feature-wise, it came down to things like APIs — the shape of the data, how we were going to request it, and scalability. We pushed the system to its limits in ways we didn’t expect at the beginning, so performance became a big focus.”  

Ship  

Jerimie: “Even before this project, one of the key things that added fuel to it was the release of the Skill XP feature. We got really positive feedback from learners and saw statistically significant increases in engagement. That told us, There’s something here. There’s a lot more we could do to help learners see their progress. 

The way it happened to be built — partly because we had to scope down and pace ourselves — was that we launched it for Java first. That gave us a mechanism to get input partway through the project as we continued building. So, it wasn’t like we had to wait 8 months to get feedback. We were able to get something out earlier, and I thought that was really nice.” 

Mark: “We spent two cycles getting the MVP product to market for a subset of our catalog, and then two more cycles to scale the product across the full catalog. After our MVP launch, we got mostly positive feedback, and people started asking for more: ‘This is really cool, this is what I’ve wanted. Can you do this, this, and this next?’ It felt very validating and fueled us to continue scaling across the full catalog.” 

Retrospective  

Mark: “I think the pace of development these days is just getting faster. And the threshold for getting buy-in for big, platform-shifting projects is higher because people expect development to happen faster — especially with the use of AI. One learning with any project is staying mindful of scope and figuring out how to get to market as fast as possible. 

The other learning is that when you build a system that powers a platform, it’s really important for the organization to understand its value and how it can be leveraged. In any company or product team, you want to be aware of how your features can empower other teams — and making that well known can be a big driver of success.” 

Jerimie: “Honestly, the entire thing was a great case study to reference. A lot of the initial ideation definitely came full circle. We slowly engaged more of the organization to get buy-in, eventually pitching upwards to get the official stamp of approval. That process, even though we’ve gone through it in our careers before, felt extremely gratifying in this case given how complex the project was.”  

Snaps 

Mark: “Definitely the LX [learner experience] team, the engineering team, the platform team came in clutch, and then the leadership team. There was a lot of key support at the right times.” 

Neil: “Data science was involved a lot at the beginning in understanding how our content is related to each other, so I’d give them snaps.” 

1...511512513...1,523Page 512 of 1,523

Recent Posts