CPU Usage and Sleep(.1)

[mholzma]

I noticed my CPU usage is relatively high. This may not be an issue but I also saw a sleep(.1) commented out. Should I uncomment that to cause the raspberry pi to take a momentary break?

Thanks.

[David00]

Hi Mike, I have a couple of questions to get some more details:

• What utilization numbers are you seeing on your Pi, and how are you determining utilization?
• What model Pi are you using?

Uncommenting the sleep likely won't have any impact because it's such a short duration to sleep. I am seeing the power monitor process have 100% utilization on my Pi 4, but overall, the Pi CPU is loaded just above 33%. The power monitor process is running as fast as possible, so it is expected to have 100% utilization.

[beadon]

It's a good qustion here, the solution of adding a sleep will only cause more problems based on the design today.

In code like this running in a 'hot loop' to acquire data as fast as possible is comment for high performance systems, or for systems that need high precision to get sampled data properly. The question to ask is should the code be polling as fast as it is ? Or, is there a method we can use, or existing hardware (DMA) that will allow the CPU to only be used as new data arrives ?

The heavy lifting ultimately should not be getting done by a CPU in the best case, but what else is on the board that can be used to manage this polling and shift to interrupts ?

[David00]

I'm definitely open to discussions on design. I do not quite know what percentage of the CPU load is consumed by the sample collection vs the calculations part, but if the OS is putting the main power monitor process to sleep and doing context switches in between every sample taken, that's going to be a ton of overhead.

The heavy lifting ultimately should not be getting done by a CPU in the best case, but what else is on the board that can be used to manage this polling and shift to interrupts ?

On my board - there's nothing else that can manage the polling process. On the Pis, there is onboard DMA, but I've never used it. I believe this would mean introducing C into the project, which also means a more complex build/distribution process. This looks like an interesting resource, and they mention pigpio, which I've used on a different project in a C Extension and have had positive experiences with.

[David00]

but if the OS is putting the main power monitor process to sleep and doing context switches in between every sample taken, that's going to be a ton of overhead.

Actually, I think that might be exactly what's happening. Per my last deep-dive into the spidev module, I learned that the spidev.xfer2() function is using the Py_BEGIN_ALLOW_THREADS macro to (I believe) setup for a context switch, up to 4 times per SPI transaction.

spidev's xfer2

The non "2" version of xfer does not use the macro, so I'd assume the CPU context switches less often.

I do not know if the CPU utilization stats include the overhead of performing context switches, but it might be worth a shot to see if the utilization decreases by using the xfer function instead of the xfer2 function.

[beadon]

Ok, then we should start by profiling the code at the right depth.

My go-to bag of tricks include:

Code

• strace
• flame charts
• valgrind

From an OS level

• sar
• collectd
• vmstat
• iostat
• pcm
• many more ...

We want to profile in the code space, so selecting the best tool to profile first is the right step. It seems like python offers cProfile, which could be a great first look at where the cycles are going ( https://docs.python.org/3/library/profile.html )

Based on results there, we may need to instrument deeper into modules, or take more profiling hints from the spidev python folks.

[David00]

Thanks @beadon. That is great info.

I am currently working on a large update to resolve #75 and #76 (and more), so I want to wait until I release the update (which will be tagged v0.3.0) to get into profiling.