Using Go for Multi Arch Kubernetes - Part 1

Part 1 - Building Multi-arch Containers

This is Part 1 of a series that’s going to take you through the concept of a heterogenous Kubernetes cluster supporting multiple different CPU architectures. It will be primarily focused on using Go since that’s my currently favorite language and might be one of the easiest languages to use that supports multi-arch. In this series we’re going to cover building Go containers for multi-arch, how to secure these containers against intrusion, benefits and tradeoffs of different container configurations, creating a build system that can generate multiple architectures, and finally deploying a hardware agnostic service to Kubernetes.

Since I started using Kubernetes I’ve thought that it’s the perfect platform for a hardware agnostic(more or less) software deployment system. It already supports the most common hardware architectures such as x86,x64,armhf, and aarch64. The problem lies in being able to build and deploy software that can run on all these architectures. Thankfully though, Go is built to not require dependences at runtime due to its statically compiled nature. Unlike other languages you don’t have to rely on system libraries at runtime so you have the flexibility of building your application then copying it into a container without worrying about installing extra software into the container. The one exception being cgo, but we’ll cover that in a different post. In fact if you want more secure containers you do not need a runtime at all for Go applications.

With the intro out of the way let’s dive right in.

First, let’s create a simple http service. It’s not really going to do much other than return a JSON response from an API call.

package main

import (
	"encoding/json"
	"log"
	"net/http"
)

type APIReq struct {
	Hello string `json:"hello"`
}

func main() {
	log.Println("Starting simple API service")
	http.HandleFunc("/", apiRequest)
	log.Fatal(http.ListenAndServe(":10000", nil))
}

func apiRequest(w http.ResponseWriter, r *http.Request) {
	log.Println("Received, request")
	var apiReq APIReq
	apiReq.Hello = "world"
	json.NewEncoder(w).Encode(apiReq)
}

So far so good, this is just about the simplest go API we can write. Now, we should make sure it runs first:

> go run main.go
2020/06/13 14:00:52 Starting simple API service

In another terminal run this command to make sure it’s working:

> curl http://localhost:10000
{"hello":"world"}

Now, Go lets you build for different architectures and operating systems with the GOOS and GOARCH environment variables. Since most Kubernetes deployments run on Linux we’re going to target Linux for the GOOS variable. First, let’s target the x64 architecture and build an artifact:

> GOARCH=amd64 go build -o amd64
> file amd64
amd64: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2,

As you can see this generated a 64 bit x86-64 compiled binary. Let’s try building a different CPU architecture:

> GOARCH=arm64 go build -o arm64
> file arm64
arm64: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), statically linked, 

This arm64 artifact won’t run on a typical x64 based laptop, but it demonstrates how easy it is to build Go for different CPUs. Now we’re going to take this a step further with a Dockerfile that will generate containers with both CPU architectures. Docker lets you define multiple builds in a single Dockerfile which makes this process much easier that it might seem. In the same directory as your Go source we’ll create a Dockerfile with the following contents:

FROM golang:latest as build-arm64
RUN mkdir /app
WORKDIR /app
COPY ./ .
RUN GOOS=linux GOARCH=arm64 go build -a -installsuffix cgo -ldflags="-w -s" -o api

FROM golang:latest as build-amd64
RUN mkdir /app
WORKDIR /app
COPY ./ .
RUN GOOS=linux GOARCH=amd64 go build -a -installsuffix cgo -ldflags="-w -s" -o api

FROM scratch as arm64
COPY --from=build-arm64 /app/api /go/bin/api
ENTRYPOINT ["/go/bin/api"]

FROM scratch as amd64
COPY --from=build-amd64 /app/api /go/bin/api
ENTRYPOINT ["/go/bin/api"]

Let’s digest each of the different blocks you’re looking at, starting with the first type:

FROM golang:latest as build-arm64
RUN mkdir /app
WORKDIR /app
COPY ./ .
RUN GOOS=linux GOARCH=arm64 go build -a -installsuffix cgo -ldflags="-w -s" -o api

This block is defining a build step and we tag it with the architecture that we want to build. It’s an intermediate step before we generate the artifact we’re going to use in our cluster. The important piece is the RUN block that tells Go to build the artifact for Linux and arm64. Continuing on to the next block:

FROM scratch as arm64
COPY --from=build-arm64 /app/api /go/bin/api
ENTRYPOINT ["/go/bin/api"]

This block is where the actual artifact we’re going to use is generated. For this example we’re using scratch containers, these allow us to have containers that do not have a runtime. This enhances the security of the container because if an attacker were to somehow break out of the container application it prevents them from being able to take any actions. The only binary in this container is the api that we built, there is no shell or other means of interacting with the system. The only thing this block does is copy the binary from the previous step into this new container.

At this point you should be able to see how you can use Go to generate containers that span differnet operating systems and hardware. It’s a start down the road to having a truly heterogenous Kubernetes cluster. In Part 2 we’re going to discuss CGO and how to build containers that compile in C libraries in the same fashion as we have with plain Go binaries.