Adapt Learning Community: Two issues relating to assessment

Hi,

we are using Adapt to build courses with final quiz consisting of 5-20 questions on a final page of the course.

We are struggling with two issues:

1. We would like to show the user question marking after they answer all the questions and see the final score (assessment result component). When I enable "Show Marking:" checkbox under "Question Behaviour" in Assessment settings the marking is shown immediately after each question is marked. "Suppress Marking" does not do the trick either, probably because we set attempts allowed to -1 to enable users to take the quiz as many times as they want. Is there any other way to make it work?

2. We set Spoor to record responses but only answers to open-ended questions get correctly recorded in LMS (we use Litmos). For text input component we see the text typed in by users in LMS but for multiple choice questions (which is 99% of our questions) there is no interaction data in Litmos (same for drag and drop component). What may be the reason for that?

I would appreciate any suggestions. Thanks.

Hi Dan

Please could you state whether you are using the authoring tool to build your course or you are working directly with the framework. Whichever you are using please could you also supply version numbers.

Show parent | Reply

Hi Matt,

thanks for your quick response.

We are using Authoring tool and it seems it's 0.4.1 version.

Show parent | Reply

Hi Dan

Regarding 1) no I don't think there's any way to make 'suppress marking' work if you're also allowing the learner infinite attempts at the assessment.

Regarding 2) if you're using AAT 0.4.1 that's quite an old version so I suspect also has quite an old version of the framework. In which case you are probably running into this issue as far as the MCQ components go. As for the drag and drop component, that's going to be a third party component so you'll need to speak to the developer of whichever one you're using. It's likely that they simply haven't added support for interaction tracking.

Show parent | Reply

Thanks a lot for your response, Matt. Really appreciate that.

1. I though so after testing different options.

2. Makes sens. We have not been updating Authoring Tool for a long time now as we did quite a lot of customization but now may be a good time for that. We use much newer version for a different project so I will test the reporting there.

Thanks.

Show parent | Reply

I have done some quick test on version 0.6.0 of AAT. Still not getting interaction records for multiple choice questions. Here is sandbox registration state log from cloud.scorm.com:

[test] question reporting

Satisfied: unknown
Completed: true
Progress Status: true
Attempts: 1
Suspended: false
Activity Objective #1
- Id:
- Measure Status: true
- Normalized Measure: 0.5
- Progress Measure: false
- Satisfied Status: false
[test] question reporting
- Satisfied: unknown
- Completed: true
- Progress Status: true
- Attempts: 1
- Suspended: false
- Activity Objective #1
  - Id:
  - Measure Status: true
  - Normalized Measure: 0.5
  - Progress Measure: false
  - Satisfied Status: false
- Runtime Data
  - cmi.completion_status: completed
  - cmi.credit: NoCredit
  - cmi.entry: Other
  - cmi.exit: Unknown
  - cmi.learner_preference
    - cmi.learner_preference.audio_level: 50
    - cmi.learner_preference.language:
    - cmi.learner_preference.delivery_speed: 1
    - cmi.learner_preference.audio_captioning: 0
  - cmi.location:
  - cmi.mode: Review
  - cmi.progress_measure:
  - cmi.score_scaled: 0.5
  - cmi.score_raw: 50
  - cmi.score_min: 0
  - cmi.score_max: 100
  - cmi.total_time: 0000:00:16.18
  - Total Time Tracked by SCORM Engine: 0000:00:19.70
  - cmi.success_status: Unknown
  - cmi.suspend_data: {"lang":"en","assessment":{"quiz":[1,1,2,1,0,{"5cac8529a75f7414167be692":true,"5cac8a19a75f7414167be697":false}]},"completion":"-111","questions":"","_isCourseComplete":true,"_isAssessmentPassed":false}
  - cmi.interactions.0
    - cmi.interactions.0.id: 5cac8a19a75f7414167be697
    - cmi.interactions.0.type: FillIn
    - cmi.interactions.0.timestamp: 2019-04-09T14:22:28
    - cmi.interactions.0.weighting:
    - cmi.interactions.0.learner_response: sample text
    - cmi.interactions.0.result: incorrect
    - cmi.interactions.0.latency:
    - cmi.interactions.0.description:

I added two questions to the assessment: one multiple choice and one text input. I can see "sample text" answer for text input so I guess global settings are good and spoor is working but there are no records for multiple choice question in the very same assessment. What am I doing wrong?

Here is the scorm engine debug log: https://cloud.scorm.com/sc/guest/ViewDebugLog?logId=62a4b807-f1c6-42b2-b7f8-b0b4f2e8c0f2&courseTitle=%5Btest%5D+question+reporting

Show parent | Reply

Hi Dan

It's not about what version of the authoring tool you're using - it's about what version of the framework and the multiple choice question component you're using.

This actually came up already in these forums a couple of weeks ago, see this post.

Show parent | Reply

Thanks Matt.

I'm not sure about framework version we use now but can see that we use 2.2.1 version of MCQ. Which seems to be the latest one. Should we go back to some previous versions? Or rather check the framework version and update to the newer one?

Show parent | Reply

If you're wanting to show markings after the entire quiz is complete (not after each question is complete) it sounds like you'll need submitAll, if your questions can all be contained in an article and you're happy with a single submit. This works fine in mine with show markings / infinite attempts enabled.

Show parent | Reply

Thanks a lot you both for your help.

I forgot about submitAll as I have never used it before but it does exactly what I wanted.

I have downgraded MCQ to 2.1.2 and it seems to report interactions correctly now!

Show parent | Reply