One year ago I built a complete and dedicated lab in order to permanently test and demonstrate Veeam Cloud Connect. The lab had been designed to operate as a production environment, and was also used for the Veeam Cloud Connect book I wrote. After a year, my SSL certificate was about to expire, so I had to either renew it, or find a different solution.
UPDATE: There’s a new and improved version of the script. You can read this blog post to learn how the entire process work, and then go here to find the improved script: Improved Powershell script for Let’s Encrypt certificate renewals.
Some basics about Let’s Encrypt
My previous certificate was a classic SSL certificate sold by one of the many Certificate Authorities available on the internet. I have nothing to complain about regarding my experience with them, the entire request process was quick and easy, and the certificate is universally recognized. My personal issue was just the price: at 249 USD a year, this was totally out of my budget. My company paid for the first year in order to let me write the book, but for the following years I had to find a different solution on my own. I had two months to find a solution, while my certificate was expiring:
While reading around, I was “geekily” intrigued by the offer from Let’s Encrypt.
Let’s Encrypt (can I call it LE for the rest of the article for brevity? ok…) is a new certificate authority backed by some of the internet’s biggest players, including: the Electronic Frontier Foundation, Mozilla, Google and many others. LE eliminates the complex process of manual certificate creation, validation, signing, installation and even renewal by instead leveraging an automated DevOps style approach with open source command line tooling built upon an open standard called ACME (Automated Certificate Management Environment). Another difference is the price. They offer full certificates FOR FREE. And they are not one of those trail certificates that last for 15 days to let you complete your tests, and that you can only request once. LE certificates last for 90 days, but you can renew them as many times as you want (well, they have a limit of 5 certs for the same host per week, but still is a lot!). So, if you find a way to make the renewal process as automated as possible, it’s like having perpetual free certificates. At first, I was also a bit unhappy with the 90 days policy, but after reading their explanation, it really makes sense. In short (the text is copied from their website):
– They limit damage from key compromise and mis-issuance. Stolen keys and mis-issued certificates are valid for a shorter period of time
– They encourage automation, which is absolutely essential for ease-of-use. If we’re going to move the entire Web to HTTPS, we can’t continue to expect system administrators to manually handle renewals. Once issuance and renewal are automated, shorter lifetimes won’t be any less convenience than longer ones.
So, I was going to figure out how to automate the registration and renewal process of my Veeam Cloud Connect certificates.
Step 1: interact with Let’s Encrypt via PowerShell
My first activity was to learn more about the release and renewal process of LE, so to test every step in a manual way, before automating it. The ACME standard has been only recently supported by Windows, but there are around enough tools and documentation to succeed in this task. Also, thanks to Rick Stahl and his article Using Let’s Encrypt with IIS on Windows, even if I was not going to use IIS, some information there has been really useful.
The tool I’ve chosen is ACMESharp: it’s based on a .NET library that provides the core interface to the ACME APIs so you can also automate your own applications, and it has several PowerShell commandlets. Also Veeam Cloud Connect uses PowerShell, so we are in a good situation. ACMESharp requires PowerShell 4.0 or greater, and .NET Framework 4.5. It has been tested on Windows 2008R2, 2012 and 2012R2 (at the time of this blog post); so, my Windows 2012 R2 Veeam server satisfied all the requirements.
Ok, time to start. The first step is to check the PoweSshell version on the Veeam server, because ACMESharp can be installed using Powershell Gallery, which is only available natively starting from Powershell 5. I’ve run this command in PowerShell to verify its version:
$PSVersionTable.PSVersion Major Minor Build Revision ----- ----- ----- -------- 5 0 10586 117
My version is 5, so I can proceed. Otherwise, I would have needed to upgrade to version 5 first. The ACMESharp module can be quickly installed using:
install-Module -Name ACMESharp
Next, I import the installed module:
Import-Module ACMESharp
You may want to add this import operation at the beginning of every script involving Let’s Encrypt, just like you are used to do with other modules. Then, I need to prepare ACMESharp. I need first to initialize a Vault, that is a location in the filesystem where ACMESharp will store Certificates and related artefacts. Note that if you run as Administrator, your Vault will be created in a system-wide path, otherwise it will be created in a private, user-specific location. I chose to run all the PowerShell commands as Administrator.
Initialize-ACMEVault
Then, I needed to register myself with the Let’s Encrypt CA, providing a method of contact like an email:
New-ACMERegistration -Contacts mailto:ldelloca@gmail.com -AcceptTos
Then, I submit a DNS domain name that I want to secure with a PKI certificate. In my example, it’s my Veeam Cloud Connect service dns name:
New-ACMEIdentifier -Dns cc.virtualtothecore.com -Alias vcc
The answer from the API is like this:
IdentifierPart : ACMESharp.Messages.IdentifierPart
IdentifierType : dns
Identifier : cc.virtualtothecore.com
Uri : https://acme-v01.api.letsencrypt.org/acme/authz/aseriesofrandomletters
Status : pending
Expires : 1/29/2017 9:45:33 PM
Challenges : {, , }
Combinations : {2, 0, 1}
Now, as this is the first time I ask a certificate to LE for this domain, I need to prove my domain ownership. I can do this in different ways, but as my DNS zone is hosted on AWS Route53 and it’s super quick to edit it, I opted for the DNS challenge. You can also place a text file in your website, if this is what you are going to encrypt. To see which specific information Let’s Encrypt wants, I run this command:
Complete-ACMEChallenge vcc -ChallengeType dns-01 -Handler manual == Manual Challenge Handler - DNS == * Handle Time: [1/22/2017 10:59:59 PM] * Challenge Token: [somerandomletters] To complete this Challenge please create a new Resource Record (RR) with the following characteristics: * RR Type: [TXT] * RR Name: [_acme-challenge.cc.virtualtothecore.com] * RR Value: [6iS2hd2l1MuxvOAm-KvIrBB-v2YBPO6r92iU8E9MUFw] ------------------------------------
Ok, so I need to create a new TXT record in my DNS zone, using the value I received. This is pretty common among Certification Authorities, nothing strange. I went into my DNS Zone, created the TXT record, and so I was able to complete the challenge:
Submit-ACMEChallenge vcc -ChallengeType dns-01
The status may still be “pending”, as the LE server immediately accepts the submission, but performs the validation asynchronously. So you will not get an immediate response to the status of your submission, but you can check the status of the submission by running multiple times this command:
(Update-ACMEIdentifier vcc -ChallengeType dns-01).Challenges | Where-Object {$_.Type -eq "dns-01"}
At some point (it took me only a few seconds, but your experience may be different) the status will change from pending to valid. If the Challenge fails for any reason you will see a status of “invalid”. At this point, you cannot re-attempt the same exact Challenge without first Submitting a new DNS Identifier. Once the Challenge has been successfully validated, I can check the overall status of the Domain Identifier, which should be valid as well:
Update-ACMEIdentifier vcc
Step 2: create the first Let’s Encrypt certificate
Now that I proved my domain ownership, I can request my first certificate. This is where I created a modified version of Rick Stahl’s script:
#Load ACMESharp module
import-module ACMESharp
$alias = "vcc"
$certname = "vcc-$(get-date -format yyyy-MM-dd-HH-mm)"
$pfxfile = "C:\ProgramData\ACMESharp\sysVault\$certname.pfx"
# Change to the Vault folder
cd C:\ProgramData\ACMESharp\sysVault
# Generate a certificate
New-ACMECertificate ${alias} -Generate -Alias $certname
#Submit the certificate
Submit-ACMECertificate $certname
# Hit until values are filled in
update-AcmeCertificate $certname
pause
# Export Certificate to PFX file
Get-ACMECertificate $certname -ExportPkcs12 $pfxfile
The final result is the certificate, already released by LE and ready to be installed:
If you ever had any previous experience managing public SSL certificates, the idea of receiving a new SSL certificate after a few seconds, in a completely automated way that does not involve any email exchange, it’s pretty amazing!
Then, by adding this additional line to the script, I was also able to import the certificate into the Local Machine store:
Import-PfxCertificate -CertStoreLocation cert:\localMachine\my -Exportable -FilePath $pfxfile Directory: Microsoft.PowerShell.Security\Certificate::LocalMachine\my Thumbprint Subject ---------- ------- 597D2247AEF1A60E6A2E6C945044881A8676A433 CN=cc.virtualtothecore.com
And I can see the new certificate in my Certificate Store:
Step 3: Import the certificate into Veeam Cloud Connect using PowerShell
Replacing an old certificate with a new one in Veeam Cloud Connect is simple, and can be done with a few steps from the console, but I wanted to completely automate this process, so I went to have a look at the PowerShell options. There are two commands in Veeam PowerShell related to Cloud Connect certificates, Get-VBRCloudGatewayCertificate and Add-VBRCloudGatewayCertificate.
Before importing the certificate, I had to change a configuration option in the registry of the Windows machine where Veeam Backup & Replication server is installed:
HKLM\SOFTWARE\Veeam\Veeam Backup and Replication\CloudIgnoreInaccessibleKey (DWORD) = 1
With this key, I instructed Veeam to avoid checking the private key presence too strictly; this check fails on recent certificates, such as the ones created by Let’s Encrypt. This new behaviour is going to become the default in VBR 9.5 Update 2, for now you just need to enable the registry key.
Once the Veeam services have been restarted to enable the new registry key (this has to be done only once), I was ready to complete the script with the two PowerShell commands:
asnp VeeamPSSnapin
Connect-VBRServer -Server localhost
$certificate = Get-VBRCloudGatewayCertificate -FromStore | Where {$_.SerialNumber -eq $serialnumber}
Add-VBRCloudGatewayCertificate -Certificate $certificate
Disconnect-VBRServer
Return
This last part of the script loads the Veeam PowerShell snap-in, connects to the local Veeam Backup Server, identifies the new LE certificate by using the serial number, and installs the certificate into Veeam Cloud Connect. The operation completes successfully, and if I then go into a tenant installation to refresh my connection to Cloud Connect, I see the new certificate:
The complete script is below. If you schedule it to run for example every month, or whatever date that is below the 90 days expiration date of the cert, your Veeam Cloud Connect will always have an upto-date certificate. Also, the first part of the script can be used for any other windows service where you can automate the certificate management via PowerShell.
### VARIABLES ###
# alias for the ACME request
$alias = "vcc"
# Let's Encrypt certificates expire after 90 days, so you will have many of them in the local
# certificate store after some time. It's easier to identify them if we give them a unique name.
# We use the date here to do so.
$certname = "vcc-$(get-date -format yyyyMMdd-HHmm)"
# Give a name to the PFX file on disk, based on the certificate name
# This is a default folder created by ACMESharp, so it's easy to use it
$pfxfile = "C:\ProgramData\ACMESharp\sysVault\$certname.pfx"
# Store the certificates into the Local Store of the Local Machine account
$certPath = "\localMachine\my"
# Configure the FQDN that the certificate needs to be binded to
$domain = "cc.virtualtothecore.com"
# Give a friendly name to the certificate so that it can be identified in the certificate store
$friendlyname = "letsencrypt-$(get-date -format yyyyMMdd-HHmm)"
### SCRIPT RUN ###
# Load ACMESharp module
import-module ACMESharp
# Change to the Vault folder
cd C:\ProgramData\ACMESharp\sysVault
# Generate a new certificate
New-ACMECertificate ${alias} -Generate -Alias $certname
# Submit the certificate request
Submit-ACMECertificate $certname
# Wait until the certificate is available (has a serial number) before moving on
# as API work in async mode so the cert may not be immediately released.
$serialnumber = $null
$serialnumber = $(update-AcmeCertificate $certname).SerialNumber
# Export the new Certificate to a PFX file
Get-ACMECertificate $certname -ExportPkcs12 $pfxfile
# Import Certificate into Certificate Store
Import-PfxCertificate -CertStoreLocation cert:\localMachine\my -Exportable -FilePath $pfxfile
# Use the new Certificate into Veeam Cloud Connect
asnp VeeamPSSnapin
Connect-VBRServer -Server localhost
$certificate = Get-VBRCloudGatewayCertificate -FromStore | Where {$_.SerialNumber -eq $serialnumber}
Add-VBRCloudGatewayCertificate -Certificate $certificate
Disconnect-VBRServer
Return