Harbor Container Registry

Harbor is an open-source container registry that provides security, identity, and management for container images.

Table of Contents

Overview

Harbor serves as the central container registry for the homelab environment, providing:

  • Image Storage: Secure storage for container images
  • Security Scanning: Vulnerability scanning with Clair
  • Access Control: Role-based access control (RBAC)
  • Content Trust: Image signing and verification with Notary
  • Replication: Multi-registry synchronization

Integration Points

  • Tanzu Kubernetes Grid: Primary registry for Kubernetes workloads
  • Build Service: Image repository for Cloud Native Buildpacks
  • cert-manager: Automated TLS certificate management
  • Contour: Ingress controller for HTTPS exposure

Prerequisites

Hardware Requirements

  • Minimum CPU: 2 cores
  • Minimum RAM: 4GB
  • Storage: 20GB for Harbor components + image storage

Software Requirements

  • Kubernetes cluster with ingress controller
  • Helm 3.x
  • cert-manager (for TLS certificates)
  • DNS resolution for Harbor FQDN

Network Requirements

  • VLANs: Accessible from management and workload networks
  • Ports: 443 (HTTPS), 80 (HTTP redirect)
  • DNS: harbor.tkg.markalston.net, notary.tkg.markalston.net

Required Credentials

  • HARBOR_ADMIN_PASSWORD: Admin password for Harbor

Installation

Step 1: Add Harbor Helm Repository 

helm repo add harbor https://helm.goharbor.io
helm repo update

Step 2: Download and Customize Chart 

helm fetch harbor/harbor --untar

Step 3: Create Namespace and Service Account 

namespace=harbor
release=harbor

# Create namespace
kubectl create namespace "${namespace}"

# Create service account with cluster admin permissions
kubectl create serviceaccount kubeapps-operator --namespace="${namespace}"
kubectl create clusterrolebinding kubeapps-operator-harbor \
    --clusterrole=cluster-admin \
    --serviceaccount="${namespace}":kubeapps-operator

Step 4: Install Harbor with Helm 

helm upgrade -i "${release}" \
    --namespace "${namespace}" \
    --values k8s/harbor/values.yaml \
    --set harborAdminPassword="${HARBOR_ADMIN_PASSWORD}" \
    --set expose.ingress.hosts.core="harbor.tkg.markalston.net" \
    --set expose.ingress.hosts.notary="notary.tkg.markalston.net" \
    --set externalURL=https://harbor.tkg.markalston.net \
    bitnami/harbor

Configuration

Basic Configuration

The Harbor deployment uses a custom values file (k8s/harbor/values.yaml) that configures:

  • External database (PostgreSQL)
  • Redis cache
  • Image scanning with Clair
  • Notary for content trust
  • Ingress configuration

Retrieving Admin Credentials 

# Get Harbor admin password from Kubernetes secret
export HARBOR_ADMIN_PASSWORD=$(kubectl get secret --namespace "${namespace}" \
    harbor-core-envvars -o jsonpath="{.data.HARBOR_ADMIN_PASSWORD}" | base64 --decode)

# Display credentials
echo "Username: admin"
echo "Password: ${HARBOR_ADMIN_PASSWORD}"

Environment Variables

Variable Description Default Required
HARBOR_ADMIN_PASSWORD Harbor admin user password None Yes

Certificate Management

Verify TLS Certificate 

# Check certificate details
openssl s_client -showcerts -servername harbor.tkg.markalston.net \
    -connect harbor.tkg.markalston.net:443 </dev/null

Update Certificate for Harbor

When Let’s Encrypt certificates are renewed, update Harbor’s certificate:

# Extract current certificates from cert-manager
kubectl get secrets prod-wildcard-certs -n cert-manager \
    -ojsonpath={.data.'tls\.key'} | base64 -d > prod.key
kubectl get secrets prod-wildcard-certs -n cert-manager \
    -ojsonpath={.data.'tls\.crt'} | base64 -d > prod.crt
  1. Extract the first certificate from prod.crt and update the server_cert_key cert_pem
  2. Use prod.key to update the server_cert_key private_key_pem
  3. Verify certificate matches:
# Compare certificate details
openssl x509 -in ./prod.crt -text
openssl s_client -showcerts -connect harbor.markalston.net:443 </dev/null 2>/dev/null

Integration with Tanzu

Configure Tanzu Kubernetes Clusters to Trust Harbor

For Tanzu Kubernetes clusters to trust Harbor’s certificates, configure the TkgServiceConfiguration:

# Edit the TKG service configuration
kubectl edit TkgServiceConfiguration tkg-service-configuration \
    --kubeconfig=/Users/malston/.kube/supervisor

Add the Harbor CA certificate to the trust bundle to automatically configure new Tanzu Kubernetes clusters.

kapp-controller Certificate Trust

For kapp-controller to trust Harbor with self-signed certificates, create a certificate configuration:

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Secret
metadata:
  name: kapp-controller-config
  namespace: tkg-system
stringData:
  caCerts: |
    -----BEGIN CERTIFICATE-----
    [Full certificate chain here]
    -----END CERTIFICATE-----
  httpProxy: ""
  httpsProxy: ""
  noProxy: ""
  dangerousSkipTLSVerify: ""
  kappDeployRawOptions: '["--diff-changes=true"]'
EOF

Verification

Health Checks 

# Check Harbor pods status
kubectl get pods -n harbor

# Check Harbor services
kubectl get services -n harbor

# Check ingress configuration
kubectl get ingress -n harbor

Functional Tests 

# Test Docker registry functionality
docker pull nginx
docker tag nginx:latest harbor.markalston.net/tanzu/nginx:v1
docker push harbor.markalston.net/tanzu/nginx:v1

# Test Kubernetes image pull
kubectl create deployment --image harbor.markalston.net/tanzu/nginx:v1 tanzu-nginx

# Verify deployment
kubectl describe pod -l app=tanzu-nginx

# Cleanup test
kubectl delete deployment tanzu-nginx

Registry Trust Verification 

# Test imgpkg with Harbor registry
kubectl run -it --rm --restart=Never test-imgpkg \
    --image=harbor.markalston.net/tanzu-application-platform/tap-packages@sha256:... \
    -- imgpkg pull -i harbor.markalston.net/tanzu/test:latest -o /tmp/test

Troubleshooting

Common Issues

Issue: Certificate Trust Issues with kapp-controller

Symptoms:

  • Tanzu Application Platform packages fail to install
  • imgpkg operations fail with certificate errors
  • kapp-controller logs show TLS certificate verification failures

Solution: Configure kapp-controller to trust Harbor’s certificate chain:

# Create certificate configuration for kapp-controller
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Secret
metadata:
  name: kapp-controller-config
  namespace: tkg-system
stringData:
  caCerts: |
    [Full certificate chain from Harbor]
EOF

Issue: Docker Push/Pull Authentication Failures

Symptoms:

  • docker push fails with authentication errors
  • kubectl image pulls fail

Solution:

# Login to Harbor registry
docker login harbor.markalston.net
Username: admin
Password: [Harbor admin password]

# Create docker registry secret for Kubernetes
kubectl create secret docker-registry harbor-registry \
    --docker-server=harbor.markalston.net \
    --docker-username=admin \
    --docker-password="${HARBOR_ADMIN_PASSWORD}" \
    --docker-email=admin@markalston.net

Issue: Certificate Mismatch After Renewal

Symptoms:

  • Browser certificate warnings
  • TLS handshake failures

Solution:

# Verify certificate alignment
openssl x509 -in ./prod.crt -text
openssl s_client -showcerts -connect harbor.markalston.net:443 </dev/null 2>/dev/null

# Update Harbor certificate configuration if needed

Log Locations

  • Harbor logs: kubectl logs -n harbor [pod-name]
  • Ingress logs: kubectl logs -n contour [pod-name]
  • cert-manager logs: kubectl logs -n cert-manager [pod-name]

Debug Mode 

# Enable debug logging for Harbor core
kubectl edit configmap harbor-core -n harbor
# Set LOG_LEVEL: debug

# Check Harbor database connectivity
kubectl exec -it harbor-database-0 -n harbor -- psql -U postgres -d registry

References

Internal Documentation

External Documentation

Last Updated: 2025-01-11 Maintained by: Mark Alston

This project is for educational and home lab purposes.