Added SmartSwitch support in chassisd and enabling chassisd #467

rameshraghupathy · 2024-04-15T03:38:50Z

Added SmartSwitch support in chassisd and enabling chassisd for fixed SmartSwitches

Description

chassisd is enabled only for modular chassis. Smartswitch is a fixed chassis. However it has been treated like a modular chassis to manage the DPU cards just like the line-cards of a modular chassis. chassisd will be enabled only on the smartswitch NPU and hence the scope of these changes are limited only to NPU and not applicable to DPU.

Motivation and Context

chassisd is enabled only for modular chassis. Smartswitch is a fixed chassis. However it has been treated like a modular chassis to manage the DPU cards just like the line-cards of a modular chassis. Hence, chassid is needed for SmartSwitch and enabled here. Also, some of the table updates and clean up are not required for SmartSwitch platform and hence using the is_smartswitch API to selectively enable it. the text "fixes #xxxx", "closes #xxxx" or "resolves #xxxx" here

How Has This Been Tested?

Enabled and built and image and checked if the chassisd is running all the time without crashing
Verify if the config change handler is working as expected by issuing config CLIs to startup and shutdown DPUs

Additional Information (Optional)

sonic-chassisd/scripts/chassisd

SmartSwitches

platform identifier.

temp fix to get the build passing.

sonic-chassisd/scripts/chassisd

oleksandrivantsiv · 2024-06-11T01:57:45Z

What are the states supported by the DPUs in the Smart Switch?

rameshraghupathy · 2024-06-11T02:22:09Z

What are the states supported by the DPUs in the Smart Switch?

”dpu_midplane_link_state”
”dpu_control_plane_state"
"dpu_data_plane_state"

sonic-chassisd/scripts/chassisd

accidental removal of a few lines from the original chassisd file

sonic-chassisd/tests/test_chassisd.py

oleksandrivantsiv · 2024-07-16T21:08:27Z

sonic-chassisd/scripts/chassisd

@@ -184,7 +188,7 @@ class ModuleUpdater(logger.Logger):
        self.chassis = chassis


I think that instead of modifying the existing ModuleUpdater class, we should implement SmartSwitchModuleUpdater(ModuleUpdater). SmartSwitchModuleUpdater should be derived from ModuleUpdater and overwrite the methods that should behave differently for the Smart Switch. This approach allows us to keep the original implementation untouched and guarantees full backward compatibility with the chassisd.

Will consider this.

Implemented this change.

@oleksandrivantsiv

sonic-chassisd/tests/test_chassisd.py

oleksandrivantsiv

as commented

gpunathilell · 2024-11-14T22:30:02Z

sonic-chassisd/scripts/chassisd

+                self.update_dpu_state(key, "UP")
+
+            elif midplane_access is False and current_midplane_state == 'False':
+                if self.is_module_reboot_system_up_expired(module_key):


Do we need this function call?

@gpunathilell No we don't, cleaned it.

gpunathilell · 2024-11-16T00:38:13Z

sonic-chassisd/scripts/chassisd

+            midplane_state = 'False'
+            if isinstance(fvs, list) and fvs[0] is True:
+                fvs = dict(fvs[-1])
+                midplane_state = fvs[CHASSIS_MIDPLANE_INFO_ACCESS_FIELD]


We are checking for the midplane_state here in the DB (and if it is not present in the DB we are assuming that the module is not reachable since midplane_state=False by default) but in case of Nvidia platform the midplane_state is being updated to the DB by Chassisd itself, so chassisd here is checking for midplane state in DB before it is supposed to update the midplane state itself. Either we need to re order code so that we update midplane_state to db before we check the DB for values, or instead of checking the DB we can directly invoke the midplane reachability API

Because this may cause issues, assuming the dpu is powered on initially, by default if there is no data in DB we assume the midplane_state is false, if no config is present the DPU is supposed to power off, but admin_state == 'down' and midplane_state is false, so the operation is skipped, causing DPU to stay on

@gpunathilell Using dpu operational_state and the API to get this as planned.

… to call the platform API

gpunathilell · 2024-11-22T16:55:15Z

sonic-chassisd/scripts/chassisd

+
+        formatted_time = dt_obj.strftime("%a %b %d %I:%M:%S %p UTC %Y")
+
+        reboot_cause_dict = {


The get_reboot_cause is supposed to return a tuple - a reboot cause and an accompanying description (from HLD) when we perform json.dump with a tuple, it is saved as a list can we split here as "cause" and "comment" separately? otherwise the # Publish the reboot cause information to CHASSIS_STATE_DB step fails in chassisd because we are performing hset to chassis_state_db with a list (Example return value)

@gpunathilell Done

gpunathilell · 2024-11-22T19:49:13Z

sonic-chassisd/scripts/chassisd

+
+            # Prepare the updated data
+            updates = {
+                "dpu_midplane_link_state": state,


Formatting suggestion, Can we have the midplane state similar to admin state and have in lower case? the show interfaces status also displays in lower case, Can be changed later if it takes time, also the time format can be changed to "%Y-%m-%d %H:%M:%S" (here or in sonic-utilities system_health command - since we are displaying this data to user)

sonic-platform-daemons/sonic-chassisd/scripts/chassisd

Line 1259 in f371df7

if admin_state == 'up' and operational_state != ModuleBase.MODULE_STATUS_ONLINE:

@gpunathilell Made dpu_midplane_link_state lowercase. The time is already in the format "%Y-%m-%d %H:%M:%S" right?

gpunathilell · 2024-11-23T01:12:34Z

The REBOOT_CAUSE_DIR is not accessible from the pmon docker where chassisd is running, (theres a diffierent /host/reboot created in the docker )REBOOT_CAUSE_DIR = "/host/reboot-cause/module/" The first-boot file which is created by sonic-host-services is not seen by chassisd, we need to use a different location which is available in the docker, and is persistent across reboots
Resolved after considering the buildimage PR

rameshraghupathy · 2024-11-23T01:28:54Z

The REBOOT_CAUSE_DIR is not accessible from the pmon docker where chassisd is running, (theres a diffierent /host/reboot created in the docker )REBOOT_CAUSE_DIR = "/host/reboot-cause/module/" The first-boot file which is created by sonic-host-services is not seen by chassisd, we need to use a different location which is available in the docker, and is persistent across reboots

@gpunathilell Have you used the PR? This is needed for pmon docker to mount the volume.

gpunathilell · 2024-11-23T01:34:30Z

The REBOOT_CAUSE_DIR is not accessible from the pmon docker where chassisd is running, (theres a diffierent /host/reboot created in the docker )REBOOT_CAUSE_DIR = "/host/reboot-cause/module/" The first-boot file which is created by sonic-host-services is not seen by chassisd, we need to use a different location which is available in the docker, and is persistent across reboots

@gpunathilell Have you used the PR? This is needed for pmon docker to mount the volume.

No, I see that the location is mounted in pmon in that PR, will include, please ignore comment

gpunathilell · 2024-11-25T23:58:08Z

sonic-chassisd/scripts/chassisd

+            updates = {
+                "dpu_midplane_link_state": state,
+                "dpu_midplane_link_reason": "",
+                "dpu_midplane_link_time": datetime.now().strftime("%Y%m%d %H:%M:%S"),


@rameshraghupathy I mean this line

Suggested change

"dpu_midplane_link_time": datetime.now().strftime("%Y%m%d %H:%M:%S"),

"dpu_midplane_link_time": datetime.now().strftime("%Y-%m-%d %H:%M:%S"),

Will be displayed as such when we do show system-health dpu:
DPU2 Online dpu_midplane_link_state up 2024-11-25 23:38:21

vvolam · 2024-11-26T09:50:12Z

sonic-chassisd/scripts/chassisd

 PLATFORM_ENV_CONF_FILE = "/usr/share/sonic/platform/platform_env.conf"
+PLATFORM_JSON_FILE = "/usr/share/sonic/platform/platform.json"


I see platform.json file exists in /usr/share/sonic/devices//platform.json, am I missing something?

@vvolam The file exists, we are accessing it here

oleksandrivantsiv suggested changes Apr 25, 2024

View reviewed changes

sonic-chassisd/scripts/chassisd Outdated Show resolved Hide resolved

oleksandrivantsiv reviewed Apr 26, 2024

View reviewed changes

sonic-chassisd/scripts/chassisd Outdated Show resolved Hide resolved

Added SmartSwitch support in chassisd and enabling chassisd for fixed

0785bb1

SmartSwitches

rameshraghupathy force-pushed the master branch from d5ed9fc to 0785bb1 Compare April 26, 2024 22:03

rameshraghupathy added 2 commits April 28, 2024 13:28

Made temp fix to avoid chassisd tests passing. Need to handle it using

c4725c4

platform identifier.

The test_chassisd needs to updated for smartswitch. The change is a

d1a6b2a

temp fix to get the build passing.

rameshraghupathy mentioned this pull request Apr 29, 2024

PMON High Level Design for SmartSwitch (Replaces the PR #1538) sonic-net/SONiC#1584

Merged

rameshraghupathy added 4 commits May 31, 2024 12:09

Addressing review comments

af4ff55

Merge remote-tracking branch 'upstream/master'

356fa8b

Fixed the merge conflict breakage in "asic_table"

97b6518

checking test_chassisd.py is ok

60cb8ed

rameshraghupathy force-pushed the master branch from 4f9d602 to 60cb8ed Compare June 5, 2024 20:00

rameshraghupathy added 3 commits June 5, 2024 13:13

Trying to resolve test failure

cd6db9b

Using is_smartswitch in MockChassis

8e958ba

Disabling the test until the dependency is committed

fb7b4ac

oleksandrivantsiv reviewed Jun 11, 2024

View reviewed changes

sonic-chassisd/scripts/chassisd Outdated Show resolved Hide resolved

oleksandrivantsiv reviewed Jun 11, 2024

View reviewed changes

sonic-chassisd/scripts/chassisd Outdated Show resolved Hide resolved

oleksandrivantsiv reviewed Jun 11, 2024

View reviewed changes

sonic-chassisd/scripts/chassisd Outdated Show resolved Hide resolved

Addressing review comments

312129f

vvolam reviewed Jun 19, 2024

View reviewed changes

sonic-chassisd/scripts/chassisd Outdated Show resolved Hide resolved

sonic-chassisd/scripts/chassisd Outdated Show resolved Hide resolved

Addressed review comments: 1. removed SWITCH module 2. Restored

ce3bc30

accidental removal of a few lines from the original chassisd file

oleksandrivantsiv reviewed Jul 16, 2024

View reviewed changes

sonic-chassisd/tests/test_chassisd.py Outdated Show resolved Hide resolved

oleksandrivantsiv reviewed Jul 16, 2024

View reviewed changes

sonic-chassisd/tests/test_chassisd.py Outdated Show resolved Hide resolved

oleksandrivantsiv reviewed Jul 16, 2024

View reviewed changes

sonic-chassisd/tests/test_chassisd.py Outdated Show resolved Hide resolved

oleksandrivantsiv suggested changes Jul 16, 2024

View reviewed changes

rameshraghupathy added 2 commits August 5, 2024 15:58

As per one of the review comments created derived class for SmartSwitch

ca1fe7c

Uncommented the previously commented blocks in test_chassisd

b2ac82c

rameshraghupathy added 6 commits November 14, 2024 07:38

Did some cleanup

e65d8eb

Working on coverage

961ed0a

Working on coverage

4d4c815

Working on coverage

7dc1ab7

Working on coverage

778c342

working on coverage

c9f503d

gpunathilell reviewed Nov 14, 2024

View reviewed changes

gpunathilell reviewed Nov 16, 2024

View reviewed changes

rameshraghupathy added 2 commits November 16, 2024 10:18

Addressed review comments

2469f7f

Using config_db CHASSIS_MODULE table admin_status change notification…

364f522

… to call the platform API

rameshraghupathy force-pushed the master branch from 30fe529 to 364f522 Compare November 18, 2024 08:57

rameshraghupathy added 2 commits November 18, 2024 01:29

coverage

77fd578

Working on coverage

3d35649

rameshraghupathy force-pushed the master branch from 25cfe16 to 3d35649 Compare November 18, 2024 14:54

rameshraghupathy added 8 commits November 18, 2024 17:48

Working on coverage

67604f8

Working on coverage

fe3302a

Working on coverage

74904e6

Working on coverage

7c3ed24

Working on coverage

d982499

working on coverage

3ca60cf

Working on coverage

efc70b0

working on coverage

f371df7

gpunathilell reviewed Nov 22, 2024

View reviewed changes

Addressed a few review comments

a23cf28

gpunathilell reviewed Nov 25, 2024

View reviewed changes

vvolam reviewed Nov 26, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added SmartSwitch support in chassisd and enabling chassisd #467

Added SmartSwitch support in chassisd and enabling chassisd #467

rameshraghupathy commented Apr 15, 2024 •

edited

Loading

oleksandrivantsiv commented Jun 11, 2024

rameshraghupathy commented Jun 11, 2024

oleksandrivantsiv Jul 16, 2024

rameshraghupathy Jul 19, 2024 •

edited

Loading

rameshraghupathy Aug 5, 2024

rameshraghupathy Sep 3, 2024

oleksandrivantsiv left a comment

gpunathilell Nov 14, 2024

rameshraghupathy Nov 19, 2024

gpunathilell Nov 16, 2024

gpunathilell Nov 16, 2024

rameshraghupathy Nov 19, 2024

gpunathilell Nov 22, 2024 •

edited

Loading

rameshraghupathy Nov 23, 2024

rameshraghupathy Nov 23, 2024

gpunathilell Nov 25, 2024

gpunathilell Nov 22, 2024

rameshraghupathy Nov 23, 2024 •

edited

Loading

gpunathilell commented Nov 23, 2024 •

edited

Loading

rameshraghupathy commented Nov 23, 2024 •

edited

Loading

gpunathilell commented Nov 23, 2024

gpunathilell Nov 25, 2024

vvolam Nov 26, 2024

rameshraghupathy Nov 26, 2024

		@@ -184,7 +188,7 @@ class ModuleUpdater(logger.Logger):
		self.chassis = chassis


		formatted_time = dt_obj.strftime("%a %b %d %I:%M:%S %p UTC %Y")

		reboot_cause_dict = {

	"dpu_midplane_link_time": datetime.now().strftime("%Y%m%d %H:%M:%S"),
	"dpu_midplane_link_time": datetime.now().strftime("%Y-%m-%d %H:%M:%S"),

		PLATFORM_ENV_CONF_FILE = "/usr/share/sonic/platform/platform_env.conf"
		PLATFORM_JSON_FILE = "/usr/share/sonic/platform/platform.json"

Added SmartSwitch support in chassisd and enabling chassisd #467

Are you sure you want to change the base?

Added SmartSwitch support in chassisd and enabling chassisd #467

Conversation

rameshraghupathy commented Apr 15, 2024 • edited Loading

Description

Motivation and Context

How Has This Been Tested?

Additional Information (Optional)

oleksandrivantsiv commented Jun 11, 2024

rameshraghupathy commented Jun 11, 2024

Choose a reason for hiding this comment

rameshraghupathy Jul 19, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

oleksandrivantsiv left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gpunathilell Nov 22, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rameshraghupathy Nov 23, 2024 • edited Loading

Choose a reason for hiding this comment

gpunathilell commented Nov 23, 2024 • edited Loading

rameshraghupathy commented Nov 23, 2024 • edited Loading

gpunathilell commented Nov 23, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rameshraghupathy commented Apr 15, 2024 •

edited

Loading

rameshraghupathy Jul 19, 2024 •

edited

Loading

gpunathilell Nov 22, 2024 •

edited

Loading

rameshraghupathy Nov 23, 2024 •

edited

Loading

gpunathilell commented Nov 23, 2024 •

edited

Loading

rameshraghupathy commented Nov 23, 2024 •

edited

Loading