We’re extremely proud at Jana at having achieved continuous delivery; for well over a year now, assuming all tests pass, code that is merged goes all the way out to production through an entirely automated process using Jenkins. The release pipeline is quite complicated, but it boils down to what you might expect:
- Run unit tests
- Deploy to a staging environment
- Run integration tests against the staging environment
- Deploy to the production environment
Step (3) in particular is interesting, because there is a whole host of integration tests that need to be run, and since they’re typically independent they can be run in parallel—otherwise it takes forever. It’s also convenient for us to implement many of them as separate Jenkins jobs that can be run independently:
The Join Plugin’s Fatal Flaw
Our first stab at implementing this used the Jenkins Join Plugin, which purports to support exactly this: once all of the integration test jobs are finished, and passed, it will kick off the next job in the sequence.
However, it turns out the join plugin has a fatal (for us) flaw: because the join trigger is handled by the parent job, and the parent job is not considered “running” while the child jobs are running, there’s a brief window of time where no job is running at all (between when the child jobs finish and the next job in the sequence starts). This allows for other jobs to sneak in between, including potentially starting the whole pipeline over again. Sad face.
Multijob Plugin to the Rescue!
Once we figured out what was going on with the join plugin, we switched to using the Multijob Plugin, which has a more human-readable interface, introduces the concept of phases that each run their jobs in parallel, and keeps jobs in the “running” state the whole time.
Thought this was worth posting in case others ran into the same issue—I wasn’t able to find much on this subject on the internet, just this one issue that sounds like the same thing . If you have your own solution for parallel jobs in Jenkins, we’d love to hear it—and also, we’re hiring :).