It's the same issue when you add a new component or extension, for instance when uploading a new version of an extension. Before you can see the effect in the preview, you must re-pick a theme/menu, as you describe Lars.

It would be very helpful if the AT would rebuild the entire course, on clicking the preview button, if a new plugin is added.