Picture of Daniel Ghost
Question scoring
by Daniel Ghost - Friday, 14 November 2014, 1:19 PM
 

The current scoring doesn't work as I would expect and having discussed this with a few team members I would question whether this should be the default scoring behaviour. For example with the current develop branch of adapt-contrib-mcq, you are given a score based on how well you answered the question. This means that you could answer every question incorrectly in an assessment but still end up with a positive score, which depending upon configurations could actually result in a passing score.

As default, would it be best to only give a user a postive score for correctly answered questions? If you wanted to allow postive scores for partially correct answers then this could be a configrable attribute on the model.

There looks to be an issue with the way partially correct scores are currently calculated too. It bases the percentage of the _questionWeight on how mnay items there are rather than how many correct answers there are. This means that a partially correct question with more options will generally provide a lower score than one with fewer options. For example:
10 options - 2 correct options - _questionWeight:10 - partially correct answer score = 1

3 options - 2 correct options - _questionWeight:10 - partially correct answer score = 3.33

Surely in both of these cases the score should actually be 5?

I will raise an issue on github for this also, but wanted to see what the general opinion was regarding this default functionality. I haven't yet checked other question types to see how they work. I assume they also reward partially coprrect answers?

Paul Welch
Re: Question scoring
by Paul Welch - Friday, 14 November 2014, 2:02 PM
 

Hi Dan,

 

Yes, I think you’re correct. In the vast majority of cases questions weights are 1 with zero for partials and incorrect. Having the ability to have weightings is great, but it should be configurable and it shouldn’t be determined in my opinion by the number of options (after all, we don’t give people a higher score for answering a question with 2 correct options over a question with only 1 correct option).

 

Thanks,
Paul

 

Picture of Matt Leathes
Re: Question scoring
by Matt Leathes - Friday, 14 November 2014, 4:51 PM
 

we've been discussing this amongst the other developers and agree that getting a question partly right should by default result in a score of 0.

it would clearly be useful in some situations to be able to change this, but it shouldn't be the default.

does anybody disagree with this? If so, now is the time to say...

I would also add that a better explanation of how 'question weighting' actually works would be helpful (maybe I'm just being a bit thick though!)