Hi Amy
Just been giving this a try for myself... I can get it working fine using the settings you describe and it will only give me a single attempt at the assessment but only during the same session.
If I quit the course then relaunch from the LMS I can then attempt the assessment again.
To fix this, add a property '_shouldStoreResponses' (set to true) to config.json, in the same place as _requireAssessmentPassed, _shouldSubmitScore etc.
This will tell the tracking extension to remember the user's answers across sessions, thereby allowing the assessment to remember what the user did in the assessment in the previous session.
Hope this helps...