In this part of my series on the TKG Extensions I will walk you through my installation of the Harbor Container Registry. As I was exploring the use case without NSX-T for a customer of mine, I was not able to use the embedded Harbor Registry that is only available with NSX-T and vSphere Pods. As with most of the open source components in the TKG Extensions it would be easily possible to deploy them also on top of a physical or virtual machine like Sadegh Khademi did it. I am limited to nearly 2 TB of vSAN storage and a container registry can consume a lot of disk space. That’s why I took an old workstation, put an ubuntu 20.10 server on it and installed an NFS server with at 4 TB data disk.
Preparation Steps (Pod Security Policies)
The harbor manifests were not prepared with service accounts unlike other components I covered in previous parts of my write up. You can simply create a service account “harbor” by issueing
kubectl create sa harbor
but unlike the ClusterRoleBindings these service accounts live in a namespace. So you are forced to create the namespace first. That’s the reason why I just added the necessary yaml definition to the first file of the manifests:
These definitions are adding the service account and the ClusterRoleBinding to the PSP with privileged access. ytt uses the files in /overlay to preprocess the yaml files in the main directory. As you can see I took the namespace “harbor-system” for the service account. This gets customized to “tanzu-system-registry” by the overlay “change-namespace.yaml”. That’s why I have to enable this preprocessing:
Attention: I didn’t do that for the ClusterRoleBinding! I didn’t want to invest in ytt syntax, please post the solution for this mystery if you find it 😉
Now we just have to use the service account:
I had to edit the following files:
- /registry/harbor/02-clair.yaml
- /registry/harbor/04-database.yaml
- /registry/harbor/06-jobservice.yaml
- /registry/harbor/09-redis.yaml
- /registry/harbor/10-registry.yaml
- /registry/harbor/11-trivy.yaml
Deploy the Services
Follow the README.md to generate password and copy them from the config.yaml to values.yaml. Afterwards you can deploy everything with
ytt --ignore-unknown-comments -f common/ -f registry/harbor/ -v hostname=harbor.ne.local | kubectl apply -f-
After a minute or two you should be able to access your newly installed registry, in my case with https://harbor.ne.local :
Now let’s test the access. If you have a self-signed certificate – and if you followed my lead, you’ll unfortunately have one -, you have to prepare your client docker installation for it. Get the registry server certificate, here are two methods:
kubectl get secret harbor-tls -n tanzu-system-registry -o jsonpath='{.data.tls\.crt}' | base64 -d
sudo openssl s_client -connect harbor.ne.local:443
Get the PEM and put it in a file harbor.crt. On ubuntu I put it on /usr/local/share/ca-certificates and add it to the system by issueing “sudo update-ca-certificates”.
Test a push:
daniele@ubuntu-dt:~/dev/tkg-extensions$ docker pull busybox Using default tag: latest latest: Pulling from library/busybox e5d9363303dd: Pull complete Digest: sha256:c5439d7db88ab5423999530349d327b04279ad3161d7596d2126dfb5b02bfd1f Status: Downloaded newer image for busybox:latest docker.io/library/busybox:latest daniele@ubuntu-dt:~/dev/tkg-extensions$ docker login harbor.ne.local Authenticating with existing credentials... Login Succeeded daniele@ubuntu-dt:~/dev/tkg-extensions$ docker tag busybox:latest harbor.ne.local/library/busybox:latest daniele@ubuntu-dt:~/dev/tkg-extensions$ docker push harbor.ne.local/library/busybox:latest The push refers to repository [harbor.ne.local/library/busybox] 0064d0478d00: Pushed latest: digest: sha256:0415f56ccc05526f2af5a7ae8654baec97d4a614f24736e8eef41a4591f08019 size: 527
This push was successful, if you check the project /library you should see the pushed image:
I mentioned in the first part that I will come with complete solution if you want to use your own CA throughout all components in the next parts.
Leave a Reply