Using Jenkinsfiles to Change Jenkins Jobs

In my previous post, we looked at the pain of changing 100 different Jenkins jobs manually when most of them need the exact same change.  In this post, we will look at the Jenkinsfile configuration as a way of preventing the maintenance nightmare of changing Jenkins jobs through the web GUI.

What Is a Jenkinsfile?

A Jenkinsfile is, like the Job DSL scripts I talked about last time, a configuration script that tells Jenkins how to create a job.  This files live directly with your source code.  Here’s an example of a Jenkinsfile, taken directly from Jenkin’s website:

Jenkins has the concept of a pipeline, which basically is a way of breaking up your jobs into separate steps.  Not necessarily build steps, which are probably all shell or batch scripts, but different stages of the job itself.

As an example, this Jenkinsfile has 3 steps:

  1. Build
  2. Test
  3. Deploy

Most developers are immediately going to know what that means.  Those steps also get tracked independently on the Jenkins job’s page.  Here’s an example of a job that has a checkout, testing, reporting, and emailing stage.

Jenkins Pipeline

You see each stage’s result (pass if it is green and fail if it is red) as well as the amount of time it took to complete.

Since pipelines are more of a core feature of later Jenkins versions, it has more “eye-candy” like the above and more support from the Jenkins core code.

Using Jenkinsfiles in Your Projects

While it is nice to have the Jenkins configuration live with your source code, it doesn’t exactly change the problem if you have one hundred different jobs that need to be changed.  If they all lived in separate projects, you would have to check out every single project then likely do a search and replace in all of those files.

Let’s look at a better solution.

Pipelines have the idea of a shared library.  This means you can share pipeline scripts with several different projects.  There is a src, vars, and resources libraries.  For our projects, I used a file in the vars directory.  I went off the example they give in the shared library link I gave above (you would put this in a file in the vars library and check that into source control as part of another project – I named ours jenkins-utils):

You can then call this shared Jenkinsfile from other Jenkinsfiles:

So for example, say you have one hundred projects that all need the even or odd build step as defined in the shared library above.  The @Library declaration tells what file contains that function call.

I used this concept to setup our Python testing and deployment in a common Jenkins file.  Every Jenkinsfile that belongs to a specific Python project is about a 2 or 3-liner, similar to the example above calling evenOrOdd.  The only parameter I pass in is the email notification for who to notify on a broken build.

Using Jenkinsfiles with this shared library approach allows us to change several jobs at once by modifying that shared library Jenkinsfile project.

In order to use these Jenkinsfiles, when a job is created, you must use the “pipeline” job type (or a similar type job that depends on pipelines).  From within that job, you will tell it the repo from which to pull code as well as the location of the Jenkinsfile in that project/

Pros and Cons

Pros:

  • If you use a shared library, you can modify many jobs at once.
  • Does not overwrite existing jobs.  You must change the Jenkinsfile and commit it before the job will reflect the change.  So no accidental loss of changes (you can’t actually configure the job without modifying the Jenkinsfile).
  • Any developer should be able to modify how a job is built without ever touching the Jenkins web GUI.  Usually once a job has been configured through a Jenkinsfile, making small changes to it is trivial.

Cons:

  • Learning curve for learning how to write a Jenkinsfile.  It is harder than using the web GUI.
  • Still requires you to initially create the job in Jenkins.  It should be simple, as only the repo and pointing to the Jenkinsfile is required, but you still must create each on initially.  Contrast that to the Job DSL approach I talked about before, which doesn’t require you to manually create those jobs through the web GUI.
  • It can be tricky modifying the Jenkinsfile and testing it before actually committing code.  It’s possible, albeit not intuitive.

Conclusion

Of the three approaches we have discussed (regex, Job DSL , and Jenkinsfiles), using Jenkinsfiles is my favorite method of changing multiple jobs.  The one thing I don’t like about it as much as Job DSL is the need to create the jobs initially.  However, the additional support given by the Jenkins framework more than makes up for this in my opinion.  Being able to see each stage individually, as well as some of the additional plugins built around it, makes it a fantastic feature for maintaining many Jenkins jobs.

I hope this short series on maintaining multiple Jenkins jobs has been helpful.  Think about these different approaches when you build up your build server, as a little investment in infrastructure at the beginning can really pay dividends down the road.

Please follow and like us:
error

Leave a Reply

avatar
  Subscribe  
Notify of