Automatic Certificate creation in Private Public Key Infrastructure (PKI)
Over the past one week ago, I try to spike and setup private PKI in my lab environment mostly for ingress and gateway testing. There is a reason why I don’t go with public one like Let’s Encrypt because you need to buy a domain and host in ACME supported DNS01 challenge like AWS route53 and the pricing is too high.
Currently, I can resolve this problem by integrating multiple platform like:
- step-ca, online certificate authority for secure, automated certificate management.
- CoreDNS, as private DNS server.
- acme-dns, A simplified DNS server with a RESTful HTTP API to provide a simple way to automate ACME DNS01 challenges.
With this three platform (all running in Docker), I can have a private certificate generated by private root CA automatically via ACME DNS01 challenge with private domain all is done via certbot command like:
For Kubernetes, I use cert-manager, a native Kubernetes certificate management controller that support ACME protocol.
below picture is the flow generate private certificate with private domain via certbot. In this example I will use:
- Certificate requested for server-1.zufar.io
- acme-dns resolve *.auth.zufar.io
- User register with acme-dns, acme-dns will expose an limited random domain (example:
- acme-dns will return a JSON response contain a domain that exposed, username and password for API call to populate the domain with TXT record later.
- User create a CNAME record in CoreDNS based on limited random domain that acme-dns give to the user.
_acme-challenge.server.zufar.io IN CNAME 173a5dcb-4777-498d-9abd-87d84bb4fe54.auth.zufar.io.
- Certbot request an certificate to the certificate authority step-ca (example:
server.zufar.io). step-ca will request the certbot to prove that user is authorized with HTTP01 or DNS01 challenge.
- Certbot will call acme-dns and populate the TXT record in
keythat step-ca give.
- Certbot will create a private key, create a CSR and send to certificate authority step-ca to sign the CSR.
- step-ca will verify the authority by contacting DNS server to resolve
_acme-challenge.server-1.zufar.io(CoreDNS in this case).
- CoreDNS will resolve to
173a5dcb-4777-498d-9abd-87d84bb4fe54.auth.zufar.io. CoreDNS will forward this request to acme-dns via
- acme-dns resolve the DNS request and give the key in the TXT record
- CoreDNS will send the result to step-ca.
- step-ca verify the authority of the domain, sign the CSR and send the cert to the certbot.
I use CoreDNS as the main DNS server in my lab. The configuration is pretty easy for me. You only need to provide the Corefile and Zone file. Also some modification in the docker entrypoint.
It is pretty simple right?
Configuring step-ca probably is the most difficult one in this spike. In the official documentation running step-ca in docker, the first thing is we need to bootstrap the directory used by step-ca and run
step ca init to create the whole infrastructure certificate like root CA and intermediate CA.
There is a problem in my side because I already have self signed root CA created and mounted in all my VM.
step ca init can support
--key flag for exisiting root CA. Follow this step to bootstrap initial directory:
- Create a directory in your machine where you want to run the step-ca via docker (this will be mounted to the container)
- for simplicity, copy your root certificate and root private key (this is not recomended, please check this document on how the
step ca initactually works)
- Run initial docker container to bootstrap the directory
- Run the init command
- Note your password and root fingerprint, also create a password file in the
- Enable ACME support
- Exit the initial container for bootstraping
After that, we can run with docker-compose:
Try to curl the step-ca
We use acme-dns to store the
key in TXT record automatically with API call.
CoreDNS will translate the ACME magic domain into domain in the acme-dns (check the flow).
acme-dns API will listen in
https://auth.zufar.io. So you need to create a certificate from the root CA and mount that to the acme-dns container.
When its finish, you can try to register a domain through the API (you must limit this if acme-dns access publicly)
It will return a JSON response
certbot with acme-dns-certbot-hook or cert-manager support acme-dns for automatically register the TXT via API in the domain that acme-dns created (in this cases
173a5dcb-4777-498d-9abd-87d84bb4fe54.auth.zufar.io). We need to change the structure of the JSON file and use the file in the certbot hook or cert-manager: