Private DNS Workaround

Current situation

Private DNS is planned for 2027.

Until then, it is possible to use a dedicated CoreDNS instance in SKE. CoreDNS is exposed through an internal STACKIT LoadBalancer. The private LoadBalancer IP is then used as DNS nameserver in the required STACKIT networks.

Target setup

VMs / SKE nodes
    |
    v
Network DNS nameserver = <COREDNS_LB_IP>
    |
    v
Internal STACKIT LoadBalancer
    |
    v
CoreDNS in SKE
    |
    +-- static private records for stackit.mt.internal
    +-- forward all other DNS queries to upstream resolvers

Important:

• DNS needs UDP and TCP port 53.
• The STACKIT Service annotation for an internal LoadBalancer is lb.stackit.cloud/internal-lb: "true".
• The internal LoadBalancer IP is not persistent if the Service is deleted and recreated.
• Use CoreDNS as the only DNS resolver for networks that must resolve stackit.mt.internal. Public resolvers belong inside CoreDNS as upstreams, not directly on the network.

CoreDNS deployment

File: coredns-mt.yaml

apiVersion: v1
kind: Namespace
metadata:
  name: mt-dns
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: mt-coredns
  namespace: mt-dns
data:
  Corefile: |
    stackit.mt.internal:53 {
        errors
        log
        cache 30
        hosts {
            ttl 30
            192.214.181.240 proxy.stackit.mt.internal
            10.0.1.205 vpnserver.stackit.mt.internal
            fallthrough
        }
        forward . 192.214.161.53 213.17.17.17 188.34.111.111
    }

    .:53 {
        errors
        health
        ready
        log
        cache 30
        forward . 192.214.161.53 213.17.17.17 188.34.111.111
        reload
    }
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: mt-coredns
  namespace: mt-dns
spec:
  replicas: 2
  selector:
    matchLabels:
      app: mt-coredns
  template:
    metadata:
      labels:
        app: mt-coredns
    spec:
      containers:
        - name: coredns
          image: coredns/coredns:1.14.4
          args:
            - -conf
            - /etc/coredns/Corefile
          ports:
            - name: dns-udp
              containerPort: 53
              protocol: UDP
            - name: dns-tcp
              containerPort: 53
              protocol: TCP
            - name: health
              containerPort: 8080
              protocol: TCP
            - name: ready
              containerPort: 8181
              protocol: TCP
          livenessProbe:
            httpGet:
              path: /health
              port: 8080
          readinessProbe:
            httpGet:
              path: /ready
              port: 8181
          securityContext:
            runAsNonRoot: true
            runAsUser: 65532
            runAsGroup: 65532
            readOnlyRootFilesystem: true
            allowPrivilegeEscalation: false
            capabilities:
              drop:
                - ALL
              add:
                - NET_BIND_SERVICE
          volumeMounts:
            - name: config
              mountPath: /etc/coredns
              readOnly: true
      volumes:
        - name: config
          configMap:
            name: mt-coredns
---
apiVersion: v1
kind: Service
metadata:
  name: mt-coredns
  namespace: mt-dns
  annotations:
    lb.stackit.cloud/internal-lb: "true"
    lb.stackit.cloud/service-plan-id: "p10"
spec:
  type: LoadBalancer
  selector:
    app: mt-coredns
  ports:
    - name: dns-udp
      port: 53
      targetPort: 53
      protocol: UDP
    - name: dns-tcp
      port: 53
      targetPort: 53
      protocol: TCP

Current upstream resolvers are the STACKIT EU01 DNS Resolver IPs:

192.214.161.53
213.17.17.17
188.34.111.111

For EU02, replace them with the STACKIT EU02 DNS Resolver IPs:

45.137.172.101
45.137.172.102
45.137.172.103

Apply

export KUBECONFIG=./privatedns.yml

kubectl config current-context
kubectl get nodes -o wide

kubectl apply -f coredns-mt.yaml
kubectl -n mt-dns rollout status deployment/mt-coredns --timeout=180s
kubectl -n mt-dns wait \
  --for=jsonpath='{.status.loadBalancer.ingress[0].ip}' \
  service/mt-coredns \
  --timeout=300s

export COREDNS_LB_IP="$(kubectl -n mt-dns get svc mt-coredns -o jsonpath='{.status.loadBalancer.ingress[0].ip}')"
echo "$COREDNS_LB_IP"

kubectl -n mt-dns get pods -o wide
kubectl -n mt-dns get svc mt-coredns -o wide

Wait until the Service has an IP in the EXTERNAL-IP column. For an internal LoadBalancer this is still a private IP.

Use this value as:

<COREDNS_LB_IP>

Current test deployment:

Context: privatedns
Internal LoadBalancer IP: 10.0.1.39
Status: CoreDNS pods running, DNS test successful

Add the nameserver to STACKIT networks

The <COREDNS_LB_IP> must be reachable from the network it is added to, so it needs to be either in the same project or the same SNA (STACKIT Network Area).

Existing network:

1. Open STACKIT Portal.
2. Go to Networking -> Network.
3. Select the relevant network.
4. Click Edit.
5. Add <COREDNS_LB_IP> as DNS Name Server.
6. Repeat this for every network that must resolve stackit.mt.internal.

New network:

1. Open STACKIT Portal.
2. Go to Networking -> Network.
3. Click Create network.
4. Add <COREDNS_LB_IP> under DNS Name Server.

If Zscaler or another central DNS layer is used, add a conditional forwarder:

Zone: stackit.mt.internal
Forward to: <COREDNS_LB_IP>
Port: 53 UDP/TCP

Update DNS records

DNS records are defined in the hosts block of the CoreDNS ConfigMap.

For services that currently only have a public IP, you can point the record to the public IP.

Example:

hosts {
    ttl 30
    192.214.181.240 proxy.stackit.mt.internal
    10.0.1.205 vpnserver.stackit.mt.internal
    10.240.1.101 mongodb.stackit.mt.internal
    fallthrough
}

Apply the change:

export KUBECONFIG=./privatedns.yml

kubectl apply -f coredns-mt.yaml
kubectl -n mt-dns rollout restart deployment/mt-coredns
kubectl -n mt-dns rollout status deployment/mt-coredns --timeout=180s

The LoadBalancer Service is not recreated by this. The internal DNS IP stays the same as long as the Service is not deleted.

Validate the changed record:

export COREDNS_LB_IP="$(kubectl -n mt-dns get svc mt-coredns -o jsonpath='{.status.loadBalancer.ingress[0].ip}')"

kubectl -n mt-dns run dns-test \
  --rm -i \
  --restart=Never \
  --image=busybox:1.36 \
  --command -- sh -c "nslookup mongodb.stackit.mt.internal $COREDNS_LB_IP"

Validate

From a VM in the network:

dig @<COREDNS_LB_IP> proxy.stackit.mt.internal +short
dig @<COREDNS_LB_IP> vpnserver.stackit.mt.internal +short
dig @<COREDNS_LB_IP> stackit.cloud +short

Expected result:

proxy.stackit.mt.internal     -> 192.214.181.240
vpnserver.stackit.mt.internal -> 10.0.1.205
stackit.cloud                  -> public DNS response via upstream resolver

From inside the cluster:

kubectl -n mt-dns run dns-test \
  --rm -i \
  --restart=Never \
  --image=busybox:1.36 \
  --command -- sh -c "nslookup proxy.stackit.mt.internal $COREDNS_LB_IP && nslookup vpnserver.stackit.mt.internal $COREDNS_LB_IP && nslookup stackit.cloud $COREDNS_LB_IP"

Expected result:

proxy.stackit.mt.internal     -> 192.214.181.240
vpnserver.stackit.mt.internal -> 10.0.1.205
stackit.cloud                  -> public DNS response via upstream resolver

Operational notes

• Update the hosts block when private service IPs change.
• Keep the Service object stable. Recreating it can change the internal LoadBalancer IP.
• Allow UDP/TCP 53 from the required network CIDRs.
• Use two CoreDNS replicas minimum.
• This is a workaround until native STACKIT Private DNS is available.

References

• STACKIT SKE Load Balancing: https://docs.stackit.cloud/products/runtime/kubernetes-engine/basics/load-balancing/
• STACKIT DNS Resolver network settings: https://docs.stackit.cloud/products/network/core-networking/dns-resolver/how-tos/create-and-manage-dns-resolver/
• STACKIT DNS Resolver IPs: https://docs.stackit.cloud/products/network/core-networking/dns-resolver/basics/concepts/
• CoreDNS version: https://coredns.io/