Automating NPM Package Versioning

Welcome back to another newsletter!

Today, I want to discuss a topic that came up recently with a client of mine. Automating our NPM package publishing.

We were handling manually previously. We knew at some point we would migrate it to be automated and the time finally came.

I ended up learning more about Github Actions than I did about anything in NPM itself.

Let’s go over what I found out and how it’s all done.

The Problem

First off, why did they even need to automate this? The packages are published as a shared package between our frontend and backend code (TypeScript FTW). Previously, these packages were published essentially whenever it was deemed necessary. However, this is tedious and error-prone. It was also quite difficult to dictate whether the packages needed to be published or not!

When to run the solution

In this case, there are a few branches that are worked off of. There’s a develop branch that is used for development. This branch gets merged to staging for QA before finally being merged to master for production. There’s a deployment each step of the way so that integration tests can be ran.

Which is not super abnormal. Many organisations follow this model and it works fine. It’s not the ideal DevOps experience, but you need a full test suite with guaranteed success for that to exist and there’s just not enough resources to do all of this at this time. With time, this will come 😀.

Therefore, it was decided that the most obvious thing for us to do would be to automate this process once a pull request has been merged to develop. The repository already has a build & unit test check for these packages when pull requests are created & updated. Which means all that was needed was to publish the packages after bumping and building.

However, it’s not necessary to run this workflow on every pull request that gets merged. There is a little git check before (which is built into Github Actions) to see if there have been any changes in the package directory. If there are no changes, the workflow can be skipped automatically, like magic 🪄 

Here’s the snippet that makes that happen:

It’s also important to check if there are changes to this workflow as well. This makes it easier to develop against and also pick up any new changes to the workflow.

Tip: When developing a workflow on github, you can use act to run it locally. It’s great for fast feedback! It’s not perfect, but works well enough for me to recommend it.

How does it work?

Let’s have a look at the steps that are needed:

1. Checkout the repo within the action

2. Install package dependencies

3. Bump the version

4. Do git-related activities (tag, commit, push) after making the version change in the package.

5. Build with new version

6. Publish the new version

The Implementation Process

Checking out the repo was simple enough. There’s an action step for this. If you’re not using it in GH actions and your repo is in GH… shame.

The first challenge that came up along the way was that a NPM token needed to be injected into the environment. There are private packages that are utilized by this code and a token is required for publishing.

This isn’t so hard in Github actions. All you need to do is add the secret to your Actions environment for the repository. I’m not going to dive into this since it’s covered in the docs here.

Once it’s in the environment, I’ll simply run a little script that generates a .npmrc file that saves the URL & token for the repository. If you are not a TypeScript/JavaScript developer, this is similar to have an maven config for Java projects. FYI, this is different for each artifact store. In this case, NPM was the target, which is straight forward to configure.

Perfect. This part is configured properly. Now to bumping the version…

It’s certainly possible that I could write the script myself to do this. NPM has a version command for bumping versions. It also updates files, but why do that? I’m a lazy developer who enjoys standing on the shoulders of giants. I also like writing as little code as possible because it’s one less thing to maintain.

I used this action step to handle task. It’s pretty straight forward to use and well-documented. It also handles the git activities such as tagging, committing, and pushing!

This process bumps the patch version for all changes by default. The action step automatically will look for certain keywords such as “breaking”, “feature”, etc. to determine if it should bump major or minor versions instead. These keywords are also configurable!

All is great except for one thing…

Quite an annoying problem comes up…

This repository has status checks and branch security on it for the target branch (develop in this case). By default, Github produces a token for the user that created the pull request for the workflow to use. Which means that if the pull request creator does not have access to override these status checks and branch security, they cannot make a commit on the branch. This makes sense.

I try what any sane person does, passing a PAT token into the commit step… no dice. Weird. I can use this token to push from my local machine… What’s happening?

The token produces by the Github mentioned a paragraph early is static and read only. You cannot overwrite it in other steps (even if you name your environment variable the same name). Also, no matter where you put it, that token is what the actions runner will use to run any git command in the workflow. Kind of annoying, but we have a PAT token that we know works.

Well, I lied. You can overwrite it. It is in a sneaky spot though. In the github checkout action step, there is an option to override that token. After overriding the token like this:

Works like a charm.

The remaining steps are to build and publish. This is nothing special as the process uses a simple build script to compile TypeScript to JavaScript and then publishing is as trivial as running npm publish when you have a proper .npmrc configured.

The Result

Woohoo! We’re done. Another bit of manual work turned into a continuous process that requires zero human intervention. Everything has been working great for us. I’d love to hear what you think of this solution. Do you see any issues in this that could come along? There’s a few things that I won’t mention today.

Regardless, automation makes life easier. Not to mention the potential business value achieved. Imagine a developer needing to take context switch for 10-15 minutes every week to bump & publish new package versions? It adds up…

Thanks for reading this week’s newsletter!

If you find this content valuable, please share it with a fellow developer to keep them up to date with the latest in the DevOps and automation space.