Just wanted to mention, if you aren’t using the github “status” API, you should be. It’s a super simple way to make it clear that unit tests have passed (or, more importantly, failed) on a branch before a pull request is merged.
Our original idea for this was to have a robot comment the pull requests as to whether the tests had passed or failed. Turns out this is pretty tricky to get right, because you have to keep some state locally about which commits have passed or failed the tests, and monitor this state when iterating over pull requests. Github status manages all of this for you, where you just mark a commit as passed or failed, and it does the right thing for the pull request as it is updated.
And the code is super simple (we’re using the ‘github3’ wrapper):
gh = github3.GitHub(login=GITHUB_USERNAME, password=GITHUB_PASSWORD) repo = gh.repository(GITHUB_ORG, GITHUB_REPO) repo.create_status( commit, status_string, # see below # this is the link that the 'details' button goes to build_url, status_description)
The API supports four statuses: ‘success’, ‘failure’, ‘error’, and ‘pending’. I’m not sure the intent of the difference between ‘error’ and ‘failure’ is—we’re just using the latter—but I would like to start using ‘pending’ as well. I think the way to do this is have a separate jenkins job that’s tagging commits that haven’t yet been covered by the other job (since running the unit tests, for us, takes several minutes). If you’ve had success with this, let us know!