RCM Core Dump Collection and Generation

RCM allows network operators to generate a full memory dump of the applications or pods for analysis and troubleshooting purposes. In addition to logs and statistics, the full core dump eases debugging and provides better serviceability. This feature applies to both SMI and VM-based RCM deployments.

You can generate core dump for the Controller, BFDMgr, CheckpointMgr, and ConfigMgr pods.

You can use this debug command kubectl -n rcm exec -it pod_name -- kill -SIGUSR1 1 to generate a core dump for the required pod. The core dump must be generated upon request from your Cisco account representative only.

Default Setting: Not Applicable

UCS C220 M7 Server Qualification

In this release, RCM is functionally qualified on the Cisco UCS C220 M7 server.

The Cisco UCS C220 M7 Rack Server is a versatile general-purpose infrastructure and application server. This high-density, 1RU, 2-socket rack server delivers industry-leading performance and efficiency for a wide range of workloads, including virtualization and bare-metal applications.

ClusterIP Service for Internal Communication

RCM supports a clusterIP service for the Keepalived and BFDMgr host networking pods to listen on for internal requests.

Previous Behavior: The Keepalived and BFDMgr pods listened on virtual IP address (VIP) for internal REST communication.

New Behavior: The pods will now listen on the cluster IP address. You can configure custom ports in the RCM Ops Center to run multiple RCM instances in host and non-host networking modes.

This release supports new CLI commands in Config mode to configure custom ports for the Keepalived and BFDMgr pods to listen on.

• k8 smf profile rcm-keepalived-ep custom-port-keepalived port_number —Configure the Keepalived endpoint with the specified port number.

• k8 smf profile rcm-bfd-ep port_name port_number —Configure the ports specific to the BFDMgr endpoint with the specified port number.

Example:

[local] rcm# config
[local] rcm(config)# k8 smf profile rcm-keepalived-ep custom-port-keepalived 8082
[local] rcm(config)# k8 smf profile rcm-bfd-ep infraadmin-port 8890
[local] rcm(config)# k8 smf profile rcm-bfd-ep infraprometheus-port 10091
[local] rcm(config)# k8 smf profile rcm-bfd-ep ipc-ep-port 9014
[local] rcm(config)# k8 smf profile rcm-bfd-ep metrics-port 8083
[local] rcm(config)# k8 smf profile rcm-bfd-ep pprof-ep-port 8860
[local] rcm(config)# k8 smf profile rcm-bfd-ep rest-port 9192
[local] rcm(config)# k8 smf profile rcm-bfd-ep resthealth-port 8182

Input Valid Range Values for VRRP Delay

Previous Behavior: In the RCM Ops Center configuration, the VRRP delay parameter accepted the old default value of "0" seconds.

New Behavior: The VRRP delay parameter accepts any value in seconds, that ranges from 1 to 86400. The value "0" is invalid with this release.

This parameter is configured to delay the start of VRRP instance.

Example:

[local] rcm(config)# k8 smf profile rcm-keepalived-ep vrrp-config group s1
[local] rcm(config-group-s1)# tuning-params vrrp-delay 1
[local] rcm(config-group-s1)# end

Customer Impact: Prior to the RCM Ops Center upgrade, you must ensure that the vrrp-delay parameter has a value in the valid range. If "0" was used in the old configuration, use the no tuning-params vrrp-delay command to remove the configuration, and then perform Ops Center upgrade.

Reset Stale Connections on Standby RCM

Previous Behavior:

This behavior was observed during RCM switchover:

• When RCM starts and transitions to the MASTER state (due to system mode running, an RCM VM reboot, or a switch from MASTER > FAULT > BACKUP > MASTER), the CheckpointMgr pods are restarted even if RCM reports itself as MASTER.

• In the BACKUP state, Keepalived restarts the ConfigMgr pod and all CheckpointMgr pods to clean up the state. Although RCM may transition from BACKUP to MASTER, the CheckpointMgr pods may not restart completely by the time the transition occurs.

New Behavior:

The ConfigMgr pod will not restart in BACKUP state. Instead, it will clean up its internal state when RCM enters the non-MASTER state.

This release introduces the following behavior during RCM switchover:

• RCM supports a new Ops-center CLI command to reset stale TCP connections from the backup CheckpointMgr to the standby UPF:

  k8 smf profile rcm-config-ep reset-stale-connection { true | false }

  Default value: false

  When the configuration is enabled (set to true ), the CheckpointMgr pods will terminate TCP connections with all UPFs and perform other clean-up tasks when RCM enters the non-MASTER state. The CheckpointMgr pods will not be restarted in the BACKUP state.

• When RCM transitions to the non-MASTER state, the RCM controller notifies the ConfigMgr and CheckpointMgr pods about this state change.