Thursday, June 4, 2015

Managing dependent jobs in Jenkins


The problem

It is well known that Jenkins can handle job dependencies based on maven dependencies.
But, how can we manage dependencies between Jenkins jobs that are based on .Net code?
Of course, we can manage job dependencies manually, marking in each job what are its dependent jobs, this takes a lot of time to maintain and is also error prone.

We looked into some options (there weren't many) and couldn't find anything that quite fitted our needs.
This is because here in Picscout, each of our Jenkins jobs is mapped to a single solution.
It can run MSBuild , or unit tests on each relevant project in the solution.
If, for example, we take NDepend powerful API and try to adjust it to our needs, we can use it to know what assemblies are referenced by our solution's projects.
But what are the solutions that hold those assemblies? and by what order should we build them? this is for left for us to implement.
Surely there must be a better alternative.


The solution

We decided to write something of our own.
We wanted a tool that will integrate well with Jenkins.
While Jenkins can run any type of script or executable, if you run your code within Jenkins itself (using groovy system script) you will be able to use its internal API, to do things like start new jobs, wait for them to complete and analyze their build result quite effortlessly.
This however means you have to write your code on a JVM based language-
That is why we opted to write our tool in Java.

The dependency tool we wrote can map solution and project files in given folders and build a dependency chain for each project and solution.
Given a project, it can tell you what projects (by order) will be affected as a result of a change in this project and same is true for solutions.
Now, all that is left to do is to integrate it with Jenkins and Git.

Git/Jenkins Integration

The process we built is made of a few steps:
1. We use git-hooks to identify changed folders, we than find what solutions are in those folders.
2. We send the list of changed solutions to a special job we wrote in Jenkins.
3. The job runs a small Groovy system script , this script finds the dependent jobs using the dependency tool, runs them and and decides if the git commit can be accepted to our master branch (if all the jobs passed).

finally, the dependency tool can also be integrated with Jenkins dependency grpah view plugin to draw the dependencies :

(we are still working on this one)

Source/Deployment

The dependency tool can be found here along with some example usage projects and groovy scripts.
The tool is also deployed in Maven repo.
Please notice it is licensed under the MIT license.

4 comments:

  1. Opening of such center would be very useful for the fresh graduated students but students should focus on Online part time jobs till they get their desire jobs.

    ReplyDelete
  2. I have read your blog its very attractive and impressive. I like it your blog.

    Java Training in Chennai Core Java Training in Chennai Core Java Training in Chennai

    Java Online Training Java Online Training Core Java 8 Training in Chennai Core java 8 online training JavaEE Training in Chennai Java EE Training in Chennai

    ReplyDelete
  3. Getting hold of the correct London Modeling occupations Agency is essential as once you have chosen the Modeling employments from their profiles, the rest would be dealt with by them, making it simple and advantageous for you. owl.lk

    ReplyDelete