In this tutorial, we will walk through the process of setting up a local environment to deploy a containerized application to a Kubernetes cluster running on KIND (Kubernetes IN Docker) using a GitLab CI/CD pipeline.

Prerequisite

  • Basic knowldge of Kubernetes & GitLab CI/CD.
  • Docker: Both GitLab and KIND cluster deployments use Docker. Since I work on macOS, I will be using Colima instead of Docker Desktop for this tutorial. However, a similar setup is possible with Docker Desktop.
  • GitLab: Run GitLab instance locally, Please follow this tutorial to setup GitLab instance on your machine before proceeding.
  • Kind: Tool for running local Kubernetes cluster.
  • Kubectl: To interact with Kubernetes cluster.
  • Source Code: Python app along with kubernetes manifests.

Overview

GitLab CI/CD for KIND Kubernetes cluster Architecture

Assuming that GitLab is already configured as described in the prerequisites section, we will use Kind (Kubernetes IN Docker) to deploy a Kubernetes cluster. Kind runs the cluster within a single Docker container, which resides on a dedicated Docker network called kind.

To ensure communication between GitLab and the Kubernetes cluster—allowing Pods to pull images from the GitLab Container Registry—the Kind container will be added to both the GitLab network and the kind network. This dual-network setup enables the Kind container to access resources from both environments.

Operational Flow

We will follow the setup outlined in the diagram above.

  1. Code Push: When a user pushes code to the GitLab repository, it triggers the pipeline using the GitLab Runner.
  2. Job Container: The GitLab Runner creates job container to run the stages defined in the pipeline.
  3. Registry DNS Resolution: The job container uses the host machine’s(macOS) /etc/hosts file to resolve the IP address of the gitlab container, which will allow it to connect to gitlab container registry.
  4. Image Push: The job container builds the application image and pushes it to GitLab container registry.
  5. K8s DNS Resolution: The job container uses the host machine’s(macOS) /etc/hosts file to resolve the IP address of the kind container, allowing it to connect to kubernete api server.
  6. Application Deployment: The job container authenticates with the Kubernetes cluster and deploys the application using the resolved API server address.
  7. Image Pull: The Kind cluster’s container runtime uses host machine’s(macOS) /etc/hosts file to resolve the registry’s domain to pull container images.
  8. App Run: After resolving the registry’s IP and pulling the container image, the application starts running within the Kubernetes cluster.

Create Kubernetes cluster

Create a config file required for creating kind cluster on your local machine and save it as kind-config.yaml

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
networking:
  apiServerAddress: "127.0.0.1" # This is the address used to expose the API server
  apiServerPort: 6443
nodes:
  - role: control-plane
    kubeadmConfigPatches:
      - |
        kind: ClusterConfiguration
        apiServer:
          certSANs:
            - "k8s.example.com"
            - "localhost"
            - "127.0.0.1"

Run the command below to deploy the kubernetes cluster on your local machine:

1

kind create cluster --name hello-world --config kind-config.yaml

This command will create a Kubernetes cluster with a single node (a Docker container) and a dedicated Docker network named kind. It also sets up the current context to point to this cluster.

By default, Kind creates the Kubernetes cluster in its own dedicated Docker network called kind. However, attempting to access the cluster from a different network may result in the following error:

1

failed to verify certificate: x509: certificate is valid for...

This error occurs due to a mismatch in IP addresses when trying to verify the Kubernetes API server’s certificate. To resolve this and allow access to the API server from other networks(gitlab-network), we have used a domain name k8s.example.com instead of relying on an IP address.

To achieve this, we have included k8s.example.com in the apiServer.certSANs block of the Kind cluster configuration. This ensures that the Kubernetes API server’s certificate recognizes and accepts requests made to the k8s.example.com domain from other networks.

In order to enable the connectivity between Kind cluster and GitLab, add the Kind container to GitLab network.

1

docker network connect gitlab-network <kind container id>

DNS Resolution

We already have entries for gitlab.example.com and registry.gitlab.example.com mapped to 127.0.0.1 in the /etc/hosts file of local/host machine, as described in the previous tutorial. However, to access the GitLab registry from within the Docker executor (i.e., job container during pipeline execution) and Kubernetes pods, we need to map registry.gitlab.example.com to the IP address assigned to the GitLab container within the gitlab-network. Additionally, we need to add an entry for k8s.example.com (the Kubernetes API server) with its IP address allocated within the gitlab-network to ensure the Docker executor and job containers can access the Kubernetes API and deploy applications.

Steps to Update /etc/hosts:

  1. Retrieve IP addresses:

    Run the following commands from your local machine to get the IP addresses of the GitLab and Kind containers in the gitlab-network.

    1
2

    docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' <gitlab-container-id/container-name>
docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' <kind-container-id/container-name>

  2. Edit the /etc/hosts file:

    After retrieving the IP addresses, update your /etc/hosts file with the correct IP addresses for registry.gitlab.example.com and k8s.example.com.

    1
2

    <gitlab-container-ip> registry.gitlab.example.com
<k8s-container-ip> k8s.example.com

❗Important Note: The GitLab registry will now be accessible from within the gitlab-network (i.e., Docker containers and Kubernetes pods), but if you need to push images from your local/host machine, this configuration will not work, as registry.gitlab.example.com will point to the IP allocated within the gitlab-network. To push images from your local machine, you must temporarily change the entry for registry.gitlab.example.com in your /etc/hosts file back to 127.0.0.1.

Trust certificate

As we are using self-signed certificate for GitLab container registry registry.gitlab.example.com, We need to add the certificate to Kubernetes nodes(docker container) so that the container runtime(like containerd or docker) trusts it.

  1. Get into the Kind container’s shell:

    Connect to the kind container using the command below

    1

    docker exec -it <kind-container-id>  /bin/bash

  2. Obtain the certificate:

    Extract the certificate by running following command

    1

    openssl s_client -showcerts -connect registry.gitlab.example.com:5005 </dev/null 2>/dev/null | openssl x509 -outform PEM > gitlab-registry.crt

  3. Copy Certificate:

    Copy the certificate to /usr/local/share/ca-certificates/.

    1

    docker cp gitlab-registry.crt /usr/local/share/ca-certificates/

  4. Update the Certificate Store:

    Run the following command to update the trusted certificates

    1

    update-ca-certificates

  5. Restart the Container Runtime:

    Restart Docker or containerd on each node

    1

    systemctl restart containerd

If you are running kind with multi node setup , you need to run these steps for each node.

Create GitLab Repository

We will use a simple Hello World Python app along with a test script for demonstration purposes. Follow the steps below to create the GitLab repository.

  1. Download the source code of sample Python application as mentioned in the prerequisites section. Once the download is complete, unzip the file.

    After unzipping the file, you’ll see the following structure:

     1
 2
 3
 4
 5
 6
 7
 8
 9
10

    hello-world/
│
├── README.md
├── app.py               
├── dockerfile           
├── k8s-manifests
│   ├── deployment.yaml
│   └── service.yaml
├── requirements.txt     
└── test_app.py

  2. Login into your GitLab account, Create a Git repository for the Python app with name hello-world and clone this repo on your machine.

  3. Copy the content of downloaded Python application source code to the newly cloned repository and push it to gitlab.

    1
2
3

    git add .
git commit -m "first commit"
git push origin main

Configure GitLab CI/CD

Configure kubeconfig

Kind generates a kubeconfig file when the cluster is created. You can use this kubeconfig to authenticate in the GitLab CI/CD pipeline.

  1. Get the kubeconfig for the Kind cluster: Run the following command on your local machine:

    1

    kind get kubeconfig --name hello-world > kubeconfig.txt

  2. Store the kubeconfig as a GitLab CI variable:

    • Open the kubeconfig.txt file in a text editor and locate the server endpoint in the content of kubeconfig (it will look like server: https://127.0.0.1:6443). Replace the 127.0.0.1 IP address with k8s.example.com, so that gitlab-runner can connect to kubernetes cluster and deploy the app on it.
    • Go to your GitLab project → Settings → CI/CD → Variables.
    • Add a new variable KUBECONFIG_CONTENT and paste the contents of the kubeconfig.txt file.

.gitlab-ci.yml file

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45

stages:
  - test
  - build
  - deploy

variables:
  IMAGE_TAG: $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA
  KUBE_NAMESPACE: default
  KUBE_DEPLOYMENT_NAME: hello-world

test:
  stage: test
  image: python:3.9
  script:
    - pip install -r requirements.txt
    - pytest ./test_app.py

build:
  stage: build
  before_script:
    - apk update && apk add --no-cache docker openssl bind-tools #ca-certificates
    - mkdir -p /etc/docker/certs.d/registry.gitlab.example.com:5005
    - openssl s_client -showcerts -connect registry.gitlab.example.com:5005 </dev/null 2>/dev/null | openssl x509 -outform PEM > ca.crt
    - cp ca.crt /etc/docker/certs.d/registry.gitlab.example.com:5005/ca.crt
  script:
    - echo $CI_REGISTRY_PASSWORD | docker login -u $CI_REGISTRY_USER --password-stdin $CI_REGISTRY
    - docker build -t $IMAGE_TAG .
    - docker push $IMAGE_TAG

deploy:
  stage: deploy
  before_script:
    - apk update && apk add --no-cache curl bash
    - curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
    - chmod +x kubectl
    - mv kubectl /usr/local/bin/kubectl
    - kubectl version --client
    - mkdir -p /root/.kube
    - echo "$KUBECONFIG_CONTENT" > /root/.kube/config
    - export KUBECONFIG=/root/.kube/config
  script:
    - kubectl apply -f deployment.yaml 
    - kubectl set image deployment/$KUBE_DEPLOYMENT_NAME $KUBE_DEPLOYMENT_NAME=$IMAGE_TAG
  only:
    - main

The pipeline consists of three stages: test, build, and deploy.

  • In the test stage, pytest is executed to run the application’s test suite, ensuring code quality and functionality before moving forward.

  • In the build stage, the before_script section includes commands to configure the certificate for registry.gitlab.example.com. These commands allow the job container to trust the GitLab registry, ensuring that the container can push the application image securely. Without this step, the job container would fail to communicate with the registry, as SSL verification errors would prevent it from pulling or pushing images. By installing the registry’s SSL certificate, the container bypasses this issue, ensuring secure interactions with the GitLab registry.

  • In the deploy stage, we retrieve the Kubernetes configuration (kubeconfig) stored as a GitLab CI variable. This allows us to interact with the Kubernetes cluster and deploy the application. Using this configuration, the deployment process updates the Kubernetes environment by applying the new image built in the previous stage.

💡 To speed up pipeline execution and avoid downloading packages during each run, you can create a custom Docker image that includes all the necessary packages, as well as the certificates required to trust the registry. Once you build this image, you can register it with the GitLab Runner for use in your CI/CD jobs.

Test the setup

  1. Add the .gitlab-ci.yml file created in the previous section to the root of the python app repo and push the changes.

    1
2
3

    git add .
git commit -m "add .gitlab-ci.yml file"
git push origin main

    Navigate to GitLab UI and check if a pipeline is triggered or not. Once the pipeline is executed successfully, The python app will be deployed in the kubernetes cluster.

  2. Run the below command in the terminal to check if the python app is running in the cluster or not.

    1

    kubectl get po

  3. To test the app on browser port-forward

    1

    kubectl port-forward svc/hello-world 8080:80

And that’s it for this guide on setting up GitLab CI/CD with a Kind Kubernetes cluster! If you have any questions or run into issues, feel free to leave a comment below.

That’s all Folks! 🚀