But how do I set Spoor to record "incomplete" when the course starts?

You don't have to do anything, it does this by default.

Right now, if I try to run the course it sets the "lesson status" to "failed", not "incomplete".

Then this is the source of your problem, something in your course is triggering a 'failed' status. I would suggest running the course with the SCORM debugger enabled so that you can see what it is doing and when.