Adapt Learning Community: tracking options

Skip to main content

So, including SCORM 1.2 support 'out of the box' seems the obvious choice

Because we use the excellent pipwerks SCORM API wrapper, implementing SCORM 2004 is generally as straightforward as just switching over to the correct imsmanifest.xml* and commenting out one line of code**

In terms of what is tracked in SCORM (and how it's implemented in the UI) - Kev Adsett is probably best placed to tell us what the current functionality is. Though I do know it does not support cmi.interactions tracking at this stage.

I'm guessing we'd all like to have support for Experience API (AKA Tin Can) on the roadmap - but what about AICC? In my experience it's little used in the UK but is strangely popular with our US cousins across the pond... I personally have very little experience with it, so would love for somebody with more experience to volunteer on this one!

* and supporting files - of which SCORM 2004 seems to require an inordinate amount

** which forces the pipwerks wrapper to default to SCORM 1.2 for those odd LMSes that make both versions of the API available

Hi Matt, good discussion topic.

Adapt's SCORM module, which is called Spoor, is able to track and report to LMSs in the following way.

There are two optional criteria settings for LMS tracking:

Completing all the modules
Passing the quiz

These can be set independently from each other, meaning there are four possible combinations for tracking criteria:

No completion criteria, the course will never report a completion status to the LMS (this is unlikely to ever be required!)
Just completing the modules
Just passing the quiz
Both completing the modules and passing the quiz

The way in which the LMS reports is also configurable. There are two settings here:

onTrackingCriteriaMet - what to report when the tracking criteria (above) have been met
onQuizFailed - what to report if the user fails the quiz

The expected values here are the SCORM compliant lesson statuses: 'completed', 'passed', 'incomplete', 'failed'.

All of this information is encapsulated in a simple json file: scoreData.json, which is held in the spoor folder with all the other files Spoor requires.

As it stands at the moment, if passing the quiz is a criterion, it is up to the assessment plugin to actually trigger the method in Spoor. This is because there is no default 'quiz' functionality in Adapt, just question components. The Assessment plugin changes that, and is a requirement if you wish to track by quiz.

Show parent | Reply

As I understand it, we haven't implemented 'bookmarking' yet, is that right?

Show parent | Reply

Yeah that's right. We so far don't use SCORM's LessonStatus object to track the last known location, but it's certainly feasible; Spoor could pass that data to the router to navigate to, if a configurable bookmarking option is turned on.

Show parent | Reply

Reading back over this - I've made a typo -

We so far don't use SCORM's LessonLocation object to track...

Show parent | Reply

I agree that tin can should to be on the roadmap but also in terms of what it can do in the coming CMI5. I think there is quite a hype about Tin Can and LRS's at the moment so in a way it would be great to piggyback on that to get people enthusiastic about using adapt.

Show parent | Reply

Hi,

Just want to give a little input from Sweden here as a developer of my company's AuthoringTool (Flashbased that we do not have the means to rewrite to HTML5). Hoping and believing in Adapt as a future tool for us.

In my experience we have never had to deliver AICC content. SCORM 1.2 is still the most commonly used version among our customers. SCORM 2004 is also very important.

Implementing TinCan from the start would of course be great as it could help spread the buzz about Adapt.

I hope there are plans to implement cmi.interactions tracking before release as we have customers who depend upon the data gathered from that.

As for completion criteria, I find that different LMSes handle the statuses very differently in SCORM 1.2. Will there be a possibility to configure when the SCORM package is delivered how it will report to the LMS? The possibility to decide if for example completing all modules should be reported as "Completed" or "Passed" would be great.

Show parent | Reply

Hi Martin.

Adapt's SCORM reporting will be configurable as you say. It will allow for the SCORM-compliant statuses:

'completed', 'passed', 'incomplete', 'failed'

Cheers,

Kev

Show parent | Reply

I hope it will be fully configurable so that even if you have a Quiz and the user fails you can have a SCO that does not report "failed" for that.

I have come upon LMSes that do not allow the user to try again after they have failed once.

Show parent | Reply

Hi Martin,

Yes it is fully configurable in that way. There is an "onQuizFailure" value, which can be used to specify what happens when a user fails an assessment. It can be left blank, or set to 'incomplete', and will not send any extra data to the LMS upon quiz failure.

If it is set to anything else ("failed"), it will send that status to the LMS.

Show parent | Reply

Hi,

I think we might have to split this conversation into SCORM 1.2 and 2004 as the standards are slightly different.

As 1.2 is more prevalent we could discuss that first

The value of cmi.core.lesson_status can only be one of the following:

passed
failed
completed
incomplete
browsed
not attempted

So if I have a quiz included in the SCO and I want to include that to indicate passed/failed. I would have to consistently report passed/failed, with a score (we can discuss where a score is stored later, including mastery score and so on).

If I flipped between passed/failed complete/incomplete my reporting would get confusing really fast as I would have a mix of statuses.

If I have no quiz I'd expect the package to report complete/incomplete, also if I have a quiz but I mark that quiz as not contributing to completion, it should also report complete/incomplete.

Would you be able to provide an example on how you'd like this to work?

Thanks,

Mark.

Show parent | Reply

Hi Martin,

thanks for the input. As you say, we see SCORM 1.2 as very important as it's still largely the default tracking mechanism used in the industry. I would also like to add Tin Can support very early on. As you say, we see less demand for AICC but may well support it in the medium to long term.

With regards to cmi.interactions, we will need to think a little differently, as the content is less likely to follow the page turning methodology. It's therefore less easy to tell when a user has started to engage with a particular interaction, meaning that data fields such as 'time' and 'latency' aren't so easy to pin down. Having said that, I'm sure there are ways and means to implement this and we would like to make it quite configurable over time.

In principle, we are building the architecture so that there are publishing plug-ins. This means that we / anyone can add several methods of processing the data into the relevant output. The first two types should be tracking related and cover SCORM 1.2 and Tin Can.

Sven

Show parent | Reply

Hi Matt,

Thanks for giving us a basic overview of the current SCORM implementation; this looks like it covers most basic course scenarios.

SCORM 1.2. is still very much the most dominant method for tracking and I feel the adapt project should be moving forward with this for the first release. But it looks like this has already been decided :)

Thinking about SCORM, even if we are not considering it in the core, I think we should think about ways in which we can extend the basic functions if possible, allowing “hooks” if you wish (not sure if this is the correct phrase). Essentially if a developer (bespoke) wants to push data to the suspend_data variable, he/she should be able to. Likewise retrieve this data. Just a thought as there is often the requirement to push data which needs to be saved for returning visitors.

AICC we see used very rarely, but as Sven said is something that can be looked at in the long run.

SCORM 2004, again I’ve personally only come across a handful of client’s who have adopted this. But I would certainly say this could be something that we should look at supporting next. If from what Matt has mentioned is possible then this might be a small story to add to the project at a later date just to limit its barrier to the market for some users.

Have people seen many organisations adopting TIN-CAN LMS functionality? Although there is lots of hype about this, prioritising it due to the buzz phrase might gain it good ground in the market place, but will it be used by many, could we maybe put the efforts elsewhere on the outset to support 2004, and AICC maybe, just a thought?

Alan

Show parent | Reply

Great comment about Suspend_data. Yes it would be good if the framework contained functions for storing and retrieving data from Suspend_data.

This is something that I know I would use quite often as customers often want specific data stored and retrieved upon the next entry into the course.

Another one that I find important is lesson_location so that the user does not have to navigate back to where he / she left of the previous time.

Just want to stress again that I find support for interactions important too and that as with Suspend_data it would be great if the framework contained functions for this so that developers can "push" data to interactions.

Apart from lesson_status I find I use these the most.

Oh and one more thing. Retrieving information about the student, like student_name and student_id would be great to have functions for as well. Making it possible to for instance creating a game with a highscore list as a plugin to Adapt. How to save that highscorelist is another discussion altoghether.

Show parent | Reply

Martin - I completely agree that 'bookmarking' via lesson_location is an absolute must-have. But I think there are design questions around how this should work that need to be answered.

For example, what actually gets bookmarked? The page, the article, the block or the component? Or should this just be configurable? If so, what should the default* be?

Also, how are you returned back to that location on resuming the course? In many courses you'll get some sort of popup/prompt asking you if you want to resume/return to last page visited - but I've always gone with the thinking that because that's generally what you'd want to do, why make you read a message and click a button to achieve that, why not just send you right back there? You could always display a little Gmail style notification message explaining what just happened and giving the user the option to return back to the menu.

* I'm a great believer that the default setting/value for something should be the option you'd go for the majority of the time. This may sound obvious but it's amazing how often this isn't the case!

Show parent | Reply

Tracking to cmi.interactions - in my experience this is more a nice-to-have rather than an essential, mainly because so few LMSes actually support this (or have decent reporting options even when they do).

So personally I think this would be something to be added in phase 2.

Which raises the question - what do we do in situations like this? Should we put to the vote?

Show parent | Reply

[apologies, posted this with the wrong account earlier]

Hi Matt,

thanks for raising the question of decision making and voting. Just to elaborate on that point:

Generally, voting should be a last resort when it comes to decision making. We should at all times try to reach consensus via discussion. However, I do appreciate this may not always be possible.

It's worth recognising how the stage the project is at affects this:

Currently, we are in the process of taking the existing code library and refactoring it so that we can launch an initial release of the codebase. This stage is slightly different to what the future will be like in so much that there is no open codebase just yet which people can work on to pursue their own goals and requirements. We have set a target of getting the Adapt codebase re-architected within three months. Daryl is hoping to upload a first portion of proof of concept code today. This means that what will be delivered during this stage is very much down to the guys who are working on the code now.

Once the codebase is open, everyone can develop and add features. This is when the meritocracy kicks in, i.e. the solid implementation of good ideas will then start to shape the project. The main decision points will then be over what gets incorporated into the core codebase and how well this aligns with the overall vision and roadmap.

The core team will always be guided by the requirements that come form the community so it's important to hear these items and get a sense of how much they resonate in the community and we will at times put up polls to get a sense of relative priorities.

Hope this helps!

Show parent | Reply

Maybe we could start a list of "champion plugins" for interested parties to pick off and accomplish. Be it interaction tracking, Tin Can or AICC implementation or IE6 support (had to!). Just a thought.

Show parent | Reply

Alan, Martin - that's a very good point about allowing other developers to be able to store data in suspend_data. Obviously Adapt core will need to store its internal tracking data there - but there's no reason why a developer shouldn't be able to add in other information - probably using some sort of basic get/set API that forces you to assign your data to a variable. We would just need to add in some soft of check to ensure that the storage limit of the suspend_data field (4KB in SCORM 1.2, 62KB in SCORM 2004) is not exceeded (with the internal tracking data always being given priority).

Show parent | Reply

Tin Can/xAPI - yes there is a lot of buzz about this at the moment but I'm not actually aware of that many LMSes that have support for it. None of the major players (Cornerstone, SABA, Moodle etc) anyway.

Consequently, although I think it's an important thing to include for Adapt to be seen to support modern/emerging standards, I think it's not a 'phase 1' feature - and actually almost deserves its own special phase of development to ensure it's done right, taking full advantage of what it has to offer.

Some thought will also have to put into how this works in the authoring tool as I think it will be necessary to allow authors to specify what verbs and statements are associated with particular pieces of content/interactions. If you have a copy, the latest version of Lectora does this and might well be worth a look.

Show parent | Reply

Hi Matt,

It might be worth considering tincan adoption outside the LMS market too. We have a large ECM we'll be implementing now within the company that will giving us tincan triples too. I think you are absolutely right it is very early to be thinking Tin Can but if it was possible to provide some form of Scorm - Tin Can wrapper it might be a quick win, although I have no idea how feasible that is.

Show parent | Reply