Field Notice: FN74166 - Expired Device Certificate Causes Cisco SD-WAN Umbrella DNS Connections to Fail - Software Upgrade Recommended

The SSL digital certificate that is used by Cisco Catalyst SD-WAN Routers to register with Cisco Umbrella DNS expires on September 30, 2024. Cisco SD-WAN Routers with the expired certificate will fail to register with the Cisco Umbrella DNS service. The result of this failure is that all subsequent client DNS requests will be dropped.

The Cisco Umbrella DNS security solution uses digital certificates during the SSL handshake process to establish secure HTTPS connections. For DNS security, the SSL certificate facilitates device registration. The current SSL certificate on affected Cisco SD-WAN Routers expires on September 30, 2024.

This problem affects the following Cisco products if they are running an affected Cisco IOS XE Software release and if they are in SD-WAN Controller mode:

• Aggregation Services Routers (ASRs)
• Catalyst 8000 Routers
• Catalyst 8000v Edge
• Catalyst IR Rugged Series Routers
• CSR 1000v Cloud Services Routers
• Integrated Services Routers (ISRs)
• Integrated Services Virtual Routers (ISRVs)

This problem also affects these devices if they are running in Autonomous mode and are configured to use the Cisco Umbrella API with API keys for registration.

Note: Devices that are running Cisco IOS XE Software Release 17.14.1 or later are not affected, regardless of operating mode. These newer software releases already contain the updated digital certificate.

Affected devices will fail to establish secure connections with the Cisco Umbrella DNS service and then DNS registration will fail.

After registration fails, DNS capability is not available on Cisco routers that are running Cisco IOS XE SD-WAN Software. All DNS requests from clients fail. Without an available DNS service, client devices will experience a variety of network reachability failures, such as websites and cloud services becoming unavailable, and so on.

Note: Devices that are configured for Cisco Umbrella DNS Security that are already in operation will not be impacted until reboot. The expired certificate is used only during device registration with the Cisco Umbrella DNS service, not for individual DNS requests. Device registration occurs when the Cisco Umbrella DNS service is initially configured or when the configured device is rebooted.

Solution

Affected devices must have the affected certificate replaced with a new, unexpired certificate. The new certificate is valid until the year 2035. Customers who do not currently use Cisco Umbrella DNS, but who expect to deploy it in the future, can replace the affected certificate by upgrading the Cisco SD-WAN Router software to a release that contains the new certificate. The new certificate is installed automatically during the upgrade. Devices that are running Cisco IOS XE Software Release 17.14.1 or later already contain the updated digital certificate.

For affected devices, the following X1 certificate must be downloaded and installed. The installation method depends on the software release that is installed on the affected device.

This certificate can also be downloaded from https://letsencrypt.org/certs/isrgrootx1.pem

Complete the following installation instructions that correspond to the software release currently installed on the affected device.

Cisco Devices that are Running Cisco IOS XE Software Release 17.5.x or Earlier in Controller Mode

1. Download the new unexpired certificate from the following website and place it on a device that has access to the affected router(s) in the SD-WAN overlay:

   https://letsencrypt.org/certs/isrgrootx1.pem

2. Enter the Linux scp command or similar mechanism to perform a secure file copy from the download device onto each affected router. For example:

   scp ./isrgrootx1.pem admin@<EdgeIP>:bootflash:trustidrootx3_ca.ca

   Substitute <EdgeIP> with the IP address of the affected router.

3. Once the file copy completes, reload the router to complete the installation process.

Alternatively, the new ISRG Root X1 rooted certificate can be downloaded to Cisco vManage and copied to each affected router. It is not possible to copy the new certificate directly into the router bootflash with this method. Instead, the new certificate must be copied into a temporary directory first and then copied into the final bootflash location while logged into the router.

1. Log in to Cisco vManage and access vshell.

   vManage# vshell
   vManage:~$ pwd
   /home/admin

2. Download the new unexpired certificate from letsencrypt.org.

   wget https://letsencrypt.org/certs/isrgrootx1.pem --no-check-certificate

3. Enter the Linux scp command to perform a secure file copy from Cisco vManage into a temporary location on each affected router. For example:

   scp -P 830 isrgrootx1.pem admin@<EdgeIP>:/bootflash/sdwan/trustidrootx3_ca.ca

   Substitute <EdgeIP> with the IP address of the affected router.

4. Log in to the affected router.

5. Enter the copy CLI command to copy the new certificate from the temporary location into bootflash.

   router# copy bootflash:/sdwan/trustidrootx3_ca.ca bootflash:
        Destination filename [trustidrootx3_ca.ca]?

6. Enter the delete CLI command to remove the certificate file from the temporary location.

   router# delete bootflash:/sdwan/trustidrootx3_ca.ca

7. Reload the router to complete the certificate installation process.

Cisco Devices that are Running Cisco IOS XE Software Release 17.6.x to 17.8.x in Controller Mode

1. Download the new unexpired certificate from the following website and place it on a device that has access to the affected router(s) in the SD-WAN overlay:

   https://letsencrypt.org/certs/isrgrootx1.pem

2. Enter the Linux scp command or similar mechanism to perform a secure file copy from the download device onto each affected router. For example:

   scp ./isrgrootx1.pem admin@<EdgeIP>:bootflash:trustidrootx3_ca_092024.ca

   Substitute <EdgeIP> with the IP address of the affected router.

3. Once the file copy completes, reload the router to complete the installation process.

Alternatively, the new ISRG Root X1 rooted certificate can be downloaded to Cisco vManage and copied to each affected router. It is not possible to copy the new certificate directly into the router bootflash with this method. Instead, the new certificate must be copied into a temporary directory first and then copied into the final bootflash location while logged into the router.

1. Log in to Cisco vManage and access vshell.

   vManage# vshell
   vManage:~$ pwd
   /home/admin

2. Download the new unexpired certificate from letsencrypt.org.

   wget https://letsencrypt.org/certs/isrgrootx1.pem --no-check-certificate

3. Enter the Linux scp command to perform a secure file copy from Cisco vManage into a temporary location on each affected router. For example:

   scp -P 830 isrgrootx1.pem admin@<EdgeIP>:/bootflash/sdwan/trustidrootx3_ca_092024.ca

   Substitute <EdgeIP> with the IP address of the affected router.

4. Log in to the affected router.

5. Enter the copy CLI command to copy the new certificate from the temporary location into bootflash.

   router# copy bootflash:/sdwan/trustidrootx3_ca_092024.ca bootflash:
        Destination filename [trustidrootx3_ca_092024.ca]?

6. Enter the delete CLI command to remove the certificate file from the temporary location.

   router# delete bootflash:/sdwan/trustidrootx3_ca_092024.ca

7. Reload the router to complete the certificate installation process.

Cisco Devices that are Running Cisco IOS XE Software Release 17.9.x to 17.12.x in Controller Mode

1. Log in to Cisco vManage.
2. Choose Administration > Settings > Umbrella DNS Certificate.
3. In the Umbrella Root Certificate window, delete the old certificate text and paste in the new certificate. Alternatively, upload the certificate from a file by choosing the Select a File option.
4. Click Save.

Cisco Devices that are Running Cisco IOS XE Software Release 17.13.x in Controller Mode

1. Log in to Cisco vManage.
2. Choose Administration > Settings.
3. Choose Cloud Credentials.
4. Choose Umbrella DNS Certificate.
5. Paste the new certificate in the edit window. Alternatively, upload the certificate from a file by choosing the Select a File option.
6. Click Save.

Cisco Devices that are Running Any Affected Release of Cisco IOS XE Software in Autonomous Mode

1. Download the new unexpired certificate from the following web site and place it on a device that has access to the affected routers:

   https://letsencrypt.org/certs/isrgrootx1.pem

2. Copy the certificate to bootflash.

3. Verify the Certificate File: Use the dir CLI command to ensure the new certificate file is indeed present on the bootflash and the filename is correct. 

   Router# dir bootflash:isrgrootx1.pem

   This command will list the file if it exists, along with its size and other details.

4. Configure a Trustpoint: If one is not already present, create a trustpoint that will be used to store the certificate.

   Router# configure terminal
   Router(config)# crypto pki trustpoint MY_TRUSTPOINT_NAME
   Router(config-trustpoint)# enrollment terminal
   Router(config-trustpoint)# exitRouter(config)# exit
   Router#

   Replace MY_TRUSTPOINT_NAME with the name you want to assign to your trustpoint.

5. Import the Certificate: Use the crypto pki import CLI command to import the certificate from the bootflash into the trustpoint.

   Router# crypto pki import MY_TRUSTPOINT_NAME certificate bootflash:isrgrootx1.pem.

   This command tells the router to import the certificate file isrgrootx1.pem from the bootflash into the trustpoint named MY_TRUSTPOINT_NAME.