Hello, world!<br> My name is Justin.

Hello, world!
My name is Justin.

These are the things that are on my mind.

Docker and Makefiles

1 minute read

A whale of a time!

I’m learning PyTorch!

I’m writing a Dockerfile using a PyTorch base image and installing some Python packages that’ll be useful when developing my models.

I use Makefiles a lot to make my Docker-based workflows easier. I stumbled across a nice Makefile pattern in the PyTorch repo and wanted to share it with y’all.

The original Makefile

With my simple Dockerfile in the same directory as my Makefile, I started out writing my Makefile like this:

.PHONY: build
build:
	docker build --progress=plain -t pytorch .

.PHONY: check-gpu
check-gpu:
	docker run --rm --gpus all pytorch nvidia-smi

.PHONY: bash
bash:
    docker run --rm --gpus all pytorch bash

The PyTorch Makefile pattern

In my journey into the PyTorch repo, I found this Makefile, which is used in the build_publish_nightly_docker.sh script.

It extracts the docker build and docker push commands into the DOCKER_BUILD and DOCKER_PUSH Makefile variables:

DOCKER_BUILD  = DOCKER_BUILDKIT=1 \
		docker build \
			--progress=$(BUILD_PROGRESS) \
			$(EXTRA_DOCKER_BUILD_FLAGS) \
			--target $(BUILD_TYPE) \
			-t $(DOCKER_FULL_NAME):$(DOCKER_TAG) \
			$(BUILD_ARGS) .
DOCKER_PUSH = docker push $(DOCKER_FULL_NAME):$(DOCKER_TAG)

It also extracts Docker build args into their own variable:

BUILD_ARGS  = --build-arg BASE_IMAGE=$(BASE_IMAGE) \
		--build-arg PYTHON_VERSION=$(PYTHON_VERSION) \
		--build-arg CUDA_VERSION=$(CUDA_VERSION) \
		--build-arg CUDA_CHANNEL=$(CUDA_CHANNEL) \
		--build-arg PYTORCH_VERSION=$(PYTORCH_VERSION) \
		--build-arg INSTALL_CHANNEL=$(INSTALL_CHANNEL)

To build and push the Docker images, other Makefile targets make use of the above variables. In subsequent Makefile targets, some build args are replaced before executing the command contained within the DOCKER_BUILD and DOCKER_PUSH variables:

runtime-image: BASE_IMAGE := $(BASE_RUNTIME)
runtime-image: DOCKER_TAG := $(PYTORCH_VERSION)-runtime
runtime-image:
	$(DOCKER_BUILD)
	docker tag $(DOCKER_FULL_NAME):$(DOCKER_TAG) $(DOCKER_FULL_NAME):latest

runtime-push: BASE_IMAGE := $(BASE_RUNTIME)
runtime-push: DOCKER_TAG := $(PYTORCH_VERSION)-runtime
runtime-push:
	$(DOCKER_PUSH)

My new Makefile

My Makefile is far simpler than the PyTorch one. However, thanks to their Makefile pattern, my simple Makefile is a little bit cleaner and a little bit more maintainable:

IMAGE_TAG = pytorch
INTERACTIVE = 
DOCKER_RUN = docker run \
		--rm \
		--gpus all \
		$(INTERACTIVE) \
		$(IMAGE_TAG)

.PHONY: build
build:
	docker build --progress=plain -t $(IMAGE_TAG) .

.PHONY: check-gpu
check-gpu:
	$(DOCKER_RUN) nvidia-smi

.PHONY: bash
bash: INTERACTIVE := -it
bash:
	$(DOCKER_RUN) bash

Thank you, PyTorch maintainers!

Justin

You may also enjoy