Reproducible builds with npm

Background

Previously, we’ve been using a combination of git submodule, npm install, and npm install --force to build the blog. This has led to non-reproducible builds and a lot of confusion. Here’s how we can fix it.

One: switch from git submodule to npm for theme package

Previously, we used git submodule to add the theme to the blog.
This caused some issues, as the theme was also a npm package with its own dependencies and build process. hexo generate would build them concurrently.

We added the theme git repo to package.json.

Two: switch from npm install to npm ci in base target

Current blog is packaged with npm [1]: https://docs.npmjs.com/about-packages-and-modules. The base needs to provide the necessary dependencies for the blog to build.

Two files are important here:

1. package.json: specifies the package name, version, and dependencies.
2. package-lock.json

We do not want to use npm install here because:

1. it does not delete node_modules tree.
2. it modifies package.json if a new package is added.
3. it generates package-lock.json to describe the exact tree that was generated.

Leading to non-reproducible builds.

For instance, consider calling the following problematic Dockerfile

FROM node:22.5.1-alpine3.19 AS base

COPY package.json package-lock.json /workspace/
RUN npm install

FROM base AS build

COPY . /workspace  # package*.json overwrite
RUN npm run build

A correct base Dockerfile target is:

FROM node:22.5.1-alpine3.19 AS base

RUN apk add --no-cache git
WORKDIR /workspace
COPY package.json /workspace/
RUN npm ci

npm ci installs the exact dependencies listed in package-lock.json and does not modify package.json or package-lock.json.

Just build

The build target should not contain any npm install or npm ci commands. It should only contain the build command as everything should be installed in the base target.

FROM base AS build

COPY . /workspace
RUN npm run build

Three: handling updates to package.json and package-lock.json

This required a little bit of creativity. Here’s what we want to accomplish:

1. A dev env a.k.a “change, build, test” workflow in “real-time”.
2. Avoid maintain two separate Dockerfiles.
3. Allow changes to package.json and package-lock.json to git-commit.

The dev service

From the base target, mount the source tree (instead of COPY). Ergo, the dev can/must run npm install / npm ci to update dependencies or node modules.

Then npm run build to build the site with any new changes.

The service runs a persistent shell process.

[!NOTE]

If you wanted to use docker to re-build then you should probably use the prod target of the main Dockerfile instead.

The serve service

Since we cannot now serve (i.e. run nginx) from the same container we are goingto use a separate service for that.

The serice as such binds to the source tree where public/ is generated by dev service and runs nginx with default args.

Notes:

1. npm ci disregards node_modules. Hence, using devcontainer will not affect production builds unless package*.json is/are updated.