Using Regex to Change Jenkins Jobs

In my previous post, we looked at the pain of changing 100 different Jenkins jobs manually when most of them need the exact same change.  In this post, we are going to see one way of preventing that by using a regular expression to change the job configuration files on the server.

Search and Replace

How Jenkins Stores Job Configurations

As mentioned in the previous post, using regular expressions requires access to the server.  You will need to know the location of the Jenkins root directory.  On Linux, it is normally /var/lib/jenkins.

Once in that directory, you should see a “jobs” folder.

Once inside that folder, you should see either job names or other folders (if you use them in Jenkins).  Either way, it should look very similar to your top-level view within Jenkins.

Navigate to a specific job’s folder, and you should find a file called “config.xml”.  This is the configuration for your job.  If you open it, you should see XML tags and values that are related to your job.  If it has build steps, you should see them in here.  For example…

This code is for a shell build step that echoes “Hello”.

You should be able to somewhat understand what is going on in your job by reading this file.  It is wordy, but the configuration is in there.

Changing a Build Step

Imagine there is a build step that does “make”, and you want it to instead do “make clean all”.  Here’s how we can go about that using a Linux server.

  1. Go to the Jenkins jobs directory (/var/lib/jenkins/jobs is the default at the time of this writing).
  2. Execute the following command:

    1. The “find” gets all the config.xml files for all jobs.
    2. sed does the replacement.  It searches for the command tag that has make, then replaces it with make clean all.
    3. IMPORTANT!  If you have multiple make commands in jobs and only want to replace some of them, you may need a smarter regex.  Please understand what this regex does before attempting it blindly.  To be completely safe, make a backup of the “jobs” directory first.
  3. Restart Jenkins:

     

Pros and Cons

Pros:

  • This can bulk change just about anything in all your jobs.
  • Requires no managing of all your jobs from an existing script (such as DSL scripts or Jenkinsfiles)

Cons:

  • Coming up with an appropriate regex could be challenging.  Be careful you test it before trying it, or make a copy of your jobs folder first so you don’t mess up all your jobs.
  • You must have access to the Jenkins server, which may be problematic if it is managed by another admin.
  • If you mess this up without backing it up, you’ve just messed up a lot of stuff!  There’s no clean way of undoing all those changes.  As I saw in a Quora answer:

    In Unix there is no undo, only do.

  • You must restart Jenkins to see the changes in these jobs.

Conclusion

Using regex to replace a lot of configuration in multiple jobs is better than having to manually click a hundred web pages, but I would use it as a last resort.  When lots of jobs have been created with copy-pasting in the browser without another way to manage it, this may be your best bet.  In the next few posts, we are going to look at some alternatives.

P.S. Based on a comment from my previous blog post, a special mention goes out to the Configuration Slicing Jenkins plugin.  It looks like it does something similar to this in a much cleaner way, although I don’t think it would be as configurable as the regex can be.

Leave a Reply

Your email address will not be published. Required fields are marked *