Automate Export certificates and keys from Kubernetes and import in Palo Alto Networks

Sat, 25 Mar 2023 13:03:12 +0100

In my precious post I explained how to export certificates and keys from Kubernetes and how to set a password on the key file for import in Palo Alto Networks (by hand). Since I’ve several services running on Kubernetes, all with there own certificate and key. I wanted to automate this process. This is good practice, especially since the certificates are relatively short lived, which means I need to renew the certificates on the firewall at least every 90 days. The first step I did was creating a bash script that would take care of this.

I know I’ve hardcoded the passphrase on the certificate key, for this temporarily solution this is fine. I can delete all crt and key files after the import.

Next step is of course to see if I can automate this when a certificate is renewed. The watch command could help with this.

#!/bin/bash
#
# Description: Get all certificates from all namespaces and import them into the Palo Alto Networks firewall
# Uses: awk, kubectl, openssl

FW_HOST="1.2.3.4"
API_KEY="VERYSECRETAPIKEY"

while read ns cert
do 
    echo "retrieving: $ns $cert"
    # Get the certificate and key from the secret
    kubectl get secrets -n $ns $cert -o json | jq -r '.data."tls.crt"' | base64 -d > $cert.crt
    kubectl get secrets -n $ns $cert -o json | jq -r '.data."tls.key"' | base64 -d > $cert.key
    # Set the password for the key - This is required for the import on the Palo Alto Networks firewall
    openssl rsa -aes256 -in $cert.key -out $cert.key -passout "pass:P@ssw0rd!" 
    # Import the certificate and key into the Palo Alto Networks firewall
    curl -k -X POST -F "file=@$cert.crt" "https://$FW_HOST/api/?key=$API_KEY&type=import&category=certificate&certificate-name=$cert&format=pem"
    curl -k -X POST -F "file=@$cert.key" "https://$FW_HOST/api/?key=$API_KEY&type=import&category=private-key&certificate-name=$cert&format=pem&passphrase=P@ssw0rd!"
done < <(kubectl get certificates -A -o custom-columns=NAMESPACE:.metadata.namespace,SECRET:.spec.secretName --no-headers --sort-by=.metadata.namespace | awk '{print $1 " " $2}')

Export certificates and keys from Kubernetes

Mon, 20 Feb 2023 19:03:12 +0100

One of the things I really believe in is a layered security approach, if one of the layers fail you will have other layers still providing protection. One of the first layers is often the network. However even if you have a “next-gen” / layer-7 firewall, it is only fully utilized when the traffic is “readable” and therefore not encrypted. Think of protection against SQLi, XSS, etc.

Nowadays it is very easy (and free) to protect your services with TLS, think of services like Let’s Encrypt. With Kubernetes it is even easier, as you can use the cert-manager to automatically request and renew certificates.

These certficates and keys are stored in Kubernetes as secrets. When you want your firewall to decrypt the traffic, you need to export the certificates and keys from Kubernetes and import them to your firewall, in my case a Palo Alto Networks firewall. This is called SSL Inbound decryption, since we own the certificate and private key, the firewall can simply read along. Just to note, this is different with SSL Outbound (Forward Proxy) decryption, wich is a bit more complex or at least it involves additional components.

To list all the certificates on your Kubernetes cluster run the following command:

> k get certificates -A
NAMESPACE       NAME                              READY   SECRET                               AGE
website         blog.breekeenbeen.nl              True    blog.breekeenbeen.nl                 110d

The output is slightly modified to not expose all applications and URLs.

Now that we now the namespace and the name of the secret we can export the certificate and key to a file, this however has a few caveats. The secret, basically consistes of two entries, the certificate and the key. Secondly the certificate and key are base64 encoded. So we need to decode the base64 encoded certificate and key and write them to a file.

The easiest way is to print the output to json format and use jq to export the specific field, this output can then be base64 decoded and written to a file.

k get secrets -n website blog.breekeenbeen.nl -o json | jq -r '.data."tls.crt"' | base64 -d > blog.breekeenbeen.nl.crt
k get secrets -n website blog.breekeenbeen.nl -o json | jq -r '.data."tls.key"' | base64 -d > blog.breekeenbeen.nl.key

Now that we have the files, we can normally import them into our firewall. In my case I use a Palo Alto Networks firewall, which won’t accept the key file without a passphrase set (when using the WUI import). So I need to add a passphrase to the key file.

openssl rsa -aes256 -in blog.breekeenbeen.nl.key -out blog.breekeenbeen.nl.key

Now we can import the certificate and key into the firewall and create a decryption rule.

See the column on the right, this is indicates that the traffic is decrypted.

This work great, however since (Let’s Encrypt) certificates will renew every 90 days (not the private key), it would make sense to automate this process.

K8S | Breek Een Been

Automate Export certificates and keys from Kubernetes and import in Palo Alto Networks

Export certificates and keys from Kubernetes