Adapt Learning Community: test-based development

So - I know we want to keep the barrier to entry for new developers set pretty low BUT these days there's an expectation that any decent framework should include a certain amount of tests.

The question I therefore put to the community: do you think Adapt Core should require tests; to what sort of level and using what testing framework(s)?

I know Daryl has already been looking into this a bit and has identified a unit testing framework he most likes the look of; also our PHP development team have been looking at Selenium for testing the interface of their sites.

I would argue that *not* having automated tests actually increases the barrier to entry for an open source project.

As a potential contributor how do I know if the project is working? If I want to make a change can I do so without busting up the whole project and breaking it for everyone else? If someone has doubts about these things they will probably just not contribute.

If a project does have automated test included with the source then anyone can checkout the source and run the test suite and see that everything is working ok; after making changes to the source I can run the test suite and double-check that nothing is broken so I can confidently submit my change.

Secondly there is a difference between unit tests (Mocha/QUnit/Jasmine) and acceptance/UI tests(Selinium/Cucumber). Both perform slightly different jobs. Unit Tests describe and verify behaviour of an object or function, acceptance/UI tests describe and verify behaviour of a feature or user story.

Unit test are cheap to write and to run and give instant feedback (near real-time), acceptance test are more expensive to create and maintain and are time intensive and give you slower feedback.

In my opinion unit tests are the easiest cheapest and most reliable way to ensure quality, and acceptance tests provide a great deal more confidence on top of that.

If you are thinking about using Selinium you should defintely check out BrowserStack and Saucelabs.

Show parent | Reply

I think that passing tests and a high coverage level should be a pre-requisite for an approved merge into the main development branch in git.

Show parent | Reply

I agree with this

Show parent | Reply

Thanks for raising this, Matt. Going with unit tests has been agreed in previous discussions but not yet documented - so good to capture this here. We now need to see this flow through into documentation, specification and, of course, the code.

Daryl and Dennis, I presume you are building this into the task list / product backlog. Has there been a agreement on how we will do this and which approach we will take?

Show parent | Reply

Hey,

For unit testing we've been looking at Mocha - documentation is good and the ability to extend is great. Chris gave a very good walk through of Mocha and from getting up and started within a couple of minutes I thought this along with Jasmine was fast and easy.

Mocha can also be used on a node server. Would be good to know what Dennis will be using before we make a final decision? Keeping the same processes (where we can) for framework and editor developers as similar as possible will enable more developers to get involved.

Thanks,

Daryl

Show parent | Reply

Hey,

Rob explored a few options for testing a few months back we and eventually settled on Mocha with Should.js. We've both found that it makes writing tests a pleasure. Mocha meets the criteria for low barrier to entry for new developers, and it seems like everyone here likes it, so it gets my vote.

On a related note - we use Gerrit for code review and Jenkins for continuous integration, so any time we submit a change for review, Jenkins runs our tests and reports back whether the change breaks anything - a change cannot be merged into the git repository if it breaks the build. I have yet to read the Git flow document Fabien has uploaded, but I hope there is room for similar automated build options in the process?

Show parent | Reply

Hi Dennis,

I like the idea of running Gerrit and Jenkins and it tallies with the setup I have seen elsewhere and aspire to. What do we have to do to get this in place for Adapt Learning?

Show parent | Reply

Hi Sven,

From other correspondence it seems that you may already have made some progress on this, but here are a couple of links:

https://gerrit.googlecode.com/svn/documentation/2.0/install.html

https://wiki.jenkins-ci.org/display/JENKINS/Installing+Jenkins

Both applications are simple enough to install on a *nix host. Gerrit is effectively an interface to git, so would operate as a gatekeeper to your GitLab git repository. It allows developers to submit changes, but those changes must be reviewed before they are merged into Gerrit's git repository.

We require that at least two other developers must approve a submitted change, and we also require verification from Jenkins that the new code does not break the build (usually a run of unit-tests and some lint checking). Jenkins supports a Gerrit plugin that triggers a build everytime a developer submits a new patch set for review. We nominate a small number of users as 'Integrators' and only they are allowed to approve a change for merging into the authoritative codebase.

This can work alongside your GitLab installation - once a change has been submitted Gerrit can 'replicate' (i.e. push) the affected branch to a remote repository - in this case to GitLab.

I not sure this process is compatible with git flow, which I believe is based on pull requests, but I admit that I am not intimately familiar with git flow.

Show parent | Reply