Reducing Docker build time by -96%
I took over a slow CI pipeline, reworked the shell scripts, restructured the Dockerfiles for layer caching, and added multi-stage builds. The result cut delivery time and carried over to several other projects.
Problem
A CI pipeline with 8-minute Docker build times cost the team on every push, PR, and deployment. The slowness came from a structural problem in how the Dockerfiles and shell scripts were written, not from one isolated bottleneck.
Approach
I audited the Dockerfiles for cache-busting patterns: dependency installation steps sat after source files that changed often, so no layer could be reused. I rewrote the shell scripts that drove the build, reordered the Dockerfile layers to hit the cache more often, and added multi-stage builds to keep build-time dependencies out of the final runtime image.
Architecture
Multi-stage Dockerfiles with an explicit layer order: system dependencies → package installation → source copy → build step → minimal runtime stage. I refactored the shell scripts to stop invalidating the cache without reason, documented the pattern, and applied it across several projects in the same codebase family.
Outcome
Build time dropped from 8 minutes to 20 seconds. I rolled the new Dockerfile structure out across several projects, so the gain reached past the original pipeline. The change paid back in developer time within the first week.
The problem
The pipeline I inherited ran an 8-minute Docker build on every push. At that cadence the cost adds up: developers wait, feedback loops slow down, and teams deploy less often. The cause was a set of structural choices that defeated Docker's layer cache at every step, not one expensive operation.
Layer cache analysis
The pattern behind most of the slowness: source files copied before dependency installation. Source changes on almost every build, so each one invalidated the layers above the COPY instruction and forced a full reinstall of every dependency, even when the requirements had not moved. The fix is a strict ordering rule: copy only what each step needs, in order of change frequency. System packages first, then dependency manifests, then the source.
Multi-stage builds
Multi-stage builds separate the build environment from the runtime image. Build tools, compilers, and test dependencies have no place in the image you deploy. A builder stage and a runtime stage shrink the final image and remove a class of dependency-related cache churn. Each stage keeps its own cache, so a change to build tooling leaves the runtime layer intact.
Shell script refactoring
The CI shell scripts wrapped Docker commands so that they passed different arguments on each run, including timestamps, dynamic tags, and computed variables. BuildKit could not match the build context to a cached layer. I refactored the scripts to use stable, deterministic build arguments, which restored cache reuse at the CI level, not only inside individual Dockerfiles.
Rollout
I wrote the pattern up as a reusable Dockerfile template and applied it across multiple projects in the same codebase family. The 20-second build became the baseline for later work, so each new project started with the faster push and deployment cycle.