I am currently testing to use the zimablade, along a nvidia rtx 3090, to host hermes and ollama. Although when using models in hermes they fail, at the end or in the middle of inference, and in the web GUI in zimaos NaN appear inside the box. I though it could be an issue related to power management, from going to low to high power states, due to GSP firmware bug. The zimablade is connected to the gpu via oculink and the egpu dock from the zima store. I have added pcie_aspm=off pcie_port_pm=off nvidia.NVreg_DynamicPowerManagement=0x00 to /mnt/boot/cmdline.txt but I havent had any different result. I have ran the command (dmesg -T | grep -i -E “nvidia|NVRM|Xid|GSP” | tail -n 200) to check what happens after NaN appear:
[Fri Sep 18 19:34:38 2026] NVRM: _issueRpcAndWait: rpcSendMessage failed with status 0x0000000f for fn 10 sequence 8780!
[Fri Sep 18 19:34:38 2026] NVRM: rpcRmApiFree_GSP: GspRmFree failed: hClient=0xc1d00085; hObject=0x5c000056; paramsStatus=0x00000000; status=0x0000000f
[Fri Sep 18 19:34:38 2026] NVRM: _issueRpcAndWait: rpcSendMessage failed with status 0x0000000f for fn 10 sequence 8781!
[Fri Sep 18 19:34:38 2026] NVRM: rpcRmApiFree_GSP: GspRmFree failed: hClient=0xc1d00085; hObject=0x5c000048; paramsStatus=0x00000000; status=0x0000000f
[Fri Sep 18 19:34:38 2026] NVRM: _issueRpcAndWait: rpcSendMessage failed with status 0x0000000f for fn 10 sequence 8782!
[Fri Sep 18 19:34:38 2026] NVRM: rpcRmApiFree_GSP: GspRmFree failed: hClient=0xc1d00085; hObject=0x5c00003a; paramsStatus=0x00000000; status=0x0000000f
[Fri Sep 18 19:34:38 2026] NVRM: _issueRpcAndWait: rpcSendMessage failed with status 0x0000000f for fn 10 sequence 8783!
[Fri Sep 18 19:34:38 2026] NVRM: rpcRmApiFree_GSP: GspRmFree failed: hClient=0xc1d00085; hObject=0x5c000003; paramsStatus=0x00000000; status=0x0000000f
[Fri Sep 18 19:34:38 2026] NVRM: _issueRpcAndWait: rpcSendMessage failed with status 0x0000000f for fn 10 sequence 8784!
[Fri Sep 18 19:34:38 2026] NVRM: rpcRmApiFree_GSP: GspRmFree failed: hClient=0xc1d00085; hObject=0x5c000072; paramsStatus=0x00000000; status=0x0000000f
[Fri Sep 18 19:34:38 2026] NVRM: nvAssertFailedNoLog: Assertion failed: (status == NV_OK) || (status == NV_ERR_GPU_IN_FULLCHIP_RESET) @ rs_client.c:844
[Fri Sep 18 19:34:38 2026] NVRM: nvAssertFailedNoLog: Assertion failed: (status == NV_OK) || (status == NV_ERR_GPU_IN_FULLCHIP_RESET) @ rs_server.c:259
[Fri Sep 18 19:34:38 2026] NVRM: nvAssertFailedNoLog: Assertion failed: (status == NV_OK) || (status == NV_ERR_GPU_IN_FULLCHIP_RESET) @ rs_server.c:1375
[Fri Sep 18 19:34:38 2026] NVRM: _issueRpcAndWait: rpcSendMessage failed with status 0x0000000f for fn 76 sequence 8785!
[Fri Sep 18 19:34:38 2026] NVRM: _issueRpcAndWait: rpcSendMessage failed with status 0x0000000f for fn 76 sequence 8786!
[Fri Sep 18 19:34:38 2026] NVRM: _deviceTeardown: Disable of Cuda limit activation failedNVRM: _issueRpcAndWait: rpcSendMessage failed with status 0x0000000f for fn 10 sequence 8787!
[Fri Sep 18 19:34:38 2026] NVRM: rpcRmApiFree_GSP: GspRmFree failed: hClient=0xc1d00085; hObject=0x5c000002; paramsStatus=0x00000000; status=0x0000000f
To be honest I am not sure at all what the problem is but I would appreciate any feedback or experience from someone that has had a similar issue
Kind regards