#44 Line interpolation for fan speed. #60
Conversation
|
id be glad to test this out for you if you have it in a docker image on dockerhub |
|
@barnhill I needed to create dockerhub account, but now you can test it: docker-compose.yaml example: version: '3'
services:
Dell_iDRAC_fan_controller:
image: 7adrian/dell_idrac_fan_controller_with_line_interpolation:latest
container_name: Dell_iDRAC_fan_controller
restart: unless-stopped
environment:
- IDRAC_HOST=<your.IP>
- IDRAC_USERNAME=<login>
- IDRAC_PASSWORD=<password>
- ENABLE_LINE_INTERPOLATION=true
- FAN_SPEED=10
- HIGH_FAN_SPEED=45
- CPU_TEMPERATURE_FOR_START_LINE_INTERPOLATION=42
- CPU_TEMPERATURE_THRESHOLD=60
- CHECK_INTERVAL=3
- DISABLE_THIRD_PARTY_PCIE_CARD_DELL_DEFAULT_COOLING_RESPONSE=falsePS. In this image default values is changed to above. |
|
running this now ... will see how this works. This is a much needed change to allow the fan curve to scale on utilization |
|
With a CPU_TEMPERATURE_FOR_START_LINE_INTERPOLATION set at 40 it appears that its not ramping fans up till the cooler CPU is detected to pass this threshold and in this case its CPU1. Also for a 1 degree intrusion into the interpolation range it looks like its a bit aggressive by adding 12% to the fan speed with: FAN_SPEED=10
|
|
You right, I setup increase fan speed only when CPU1 is above CPU_TEMPERATURE_FOR_START_LINE_INTERPOLATION, but if that happens program start to find higher temperature of CPUs and calculate fan speed of that that the reason why it adding 12%. I will fix it in a moment :) |
Dell_iDRAC_fan_controller.sh
Outdated
| # Check if TEMP_WINDOW is grater than 0 | ||
| if [ $TEMP_WINDOW -gt $FAN_VALUE_TO_ADD ]; | ||
| then | ||
| FAN_VALUE_TO_ADD="$((FAN_WINDOW * TEMPERATURE_ABOVE_LOWER_THRESHOLD / TEMP_WINDOW))" |
There was a problem hiding this comment.
Bash natively doesn't support floating numbers, so if first you calculate TEMPERATURE_ABOVE_LOWER_THRESHOLD / TEMP_WINDOW you will have always 0...
Dell_iDRAC_fan_controller.sh
Outdated
| if $ENABLE_LINE_INTERPOLATION | ||
| then | ||
| CURRENT_FAN_SPEED=$FAN_SPEED | ||
| if [ $CPU1_TEMPERATURE -gt $CPU_TEMPERATURE_FOR_START_LINE_INTERPOLATION ]; |
There was a problem hiding this comment.
I think this needs an OR
$CPU2_TEMPERATURE -gt $CPU_TEMPERATURE_FOR_START_LINE_INTERPOLATION
or else it will never trigger interpolation when CPU2 temp is over the threshold for interpolation which is what I was seeing.
|
I pushed fixed version, you can check now. |
|
@barnhill The reason probably you had 12% when 1 degree change are: |
Dell_iDRAC_fan_controller.sh
Outdated
| if $ENABLE_LINE_INTERPOLATION | ||
| then | ||
| CURRENT_FAN_SPEED=$DECIMAL_FAN_SPEED | ||
| if [ $CPU1_TEMPERATURE -gt $CPU_TEMPERATURE_FOR_START_LINE_INTERPOLATION ] || [$IS_CPU2_TEMPERATURE_SENSOR_PRESENT] && [$CPU2_TEMPERATURE -gt $CPU_TEMPERATURE_FOR_START_LINE_INTERPOLATION]; |
There was a problem hiding this comment.
I think order of operation is a problem here now because now CPU2 is over the threshold and CPU1 isnt and its not ramping fans up.
Threshold set to start interpolation set at 40
Inlet CPU 1 CPU 2 Exhaust Active fan speed profile
26°C 40°C 47°C 38°C Interpolated fan control profile (10%)
There was a problem hiding this comment.
Actually getting this error in the script:
/Dell_iDRAC_fan_controller.sh: line 243: [true]: command not found
There was a problem hiding this comment.
[$IS_CPU2_TEMPERATURE_SENSOR_PRESENT = true] ?
There was a problem hiding this comment.
I see this changed and will pull again in a few minutes to see if it fixes that issue.
|
I changed if order and pushed new docker image. You can check now. |
|
Looks like its working as expected so far 27°C 39°C 46°C 38°C Interpolated fan control profile (20%) |
|
Im not sure which sensor this script was original using for CPU temp but it is slightly hotter than what TrueNAS is reporting as the hottest core temp. So its apparently using different sensors. That is something outside this PR though. |
|
Great! :)
With PID you can tweak values to achieve different behavior when temperature going to high: Tell me what you think about that. It's overkill, or can be very useful? Edit. And I thinking about add night time mode to set different max fan speed during day and night. |
|
I think proportional would be the most useful ... integral might let CPUs run hot too long. I think it might be over engineering a solution a bit. A complicated config would be a barrier to entry and we want people using it. |
I think program use the same CPU temperature as you can find in IDRAC 8 -> Power / Thermal -> Temperatures -> CPUX Temp. |
|
I set my thresholds slightly higher to prevent fan ramp when the CPU temps are tolerable in truenas. |
|
Also I think first things first is we should get this merged with approval from the repo owner. Im going to keep an eye on it for a few days and see what happens. |
With day/night I think about during day you can set something like starting increase fan speed when temp above 35 degree and going really fast when temperature is 10 degree before warning/critical point. But during night starting temperature going to 45 degree and fast fan speed when temperature is 5 degree before warning/critical point. To simplify usage for entry users I thinking about create profiles (program should automatically read cpu warning/critical temperature) and based on that user will be able to only set environment variable (ultra_quiet, quiet). And program will calculate fan speed to stay at healthy level. |
|
right now at 15% fans mine is super quiet in the closet. I dont know that I would even use night mode since I couldnt hear it anyway outside the closet its so quiet. I would like to figure out how to read local ipmi data instead of using the network if I can. |
If you using Truenas Scale and want to use local ipmi you need to:
|
|
worked like a champ. Had to enable priv mode first then save ... then come back in and put the device in |
|
@tigerblue77 Just wondering if you are interested in this PR. It adds some capabilities to configure the fan curves more and I have verified @7Adrian 's work and it works as expected. |
|
I'm going to give this a test right now. My server is jumping from my set fan speed to the obnoxiously loud dell default. Really could use a ramp response like this PR provides. @7Adrian using a PID algorithm is an interesting idea. It might be overkill but it would definitely be fun...I probably wouldn't want to try to implement PID in bash though, that doesn't sound fun. |
|
Ran the below values on a Dell R720XD. I don't have much time on this server and I think it might've taken a bad fall in shipping, so the heatsinks might need to be reapplied. These values were not great. It seemed like the cpu temp was jumping all over the place and the Dell default kicked in a couple times. Output: It seemed like something odd was going on with the sensor or the reading. You can see the cpu temps jump 9 degrees celsius in less than 10 seconds. I can't necessarily attribute that behavior to these changes since line interpolation seemed to perform correctly. Fan speeds went up as CPU temp went up. When the threshold was hit, the Dell default kicked in. Then I tried these values which worked amazingly well: It seems like the algorithm is sensitive to the Haven't had a chance to look at the code yet, been looking at code all day already. May have spoken too soon, as I was writing this the CPU temp spiked again. I think there's either something wrong with my CPUs or with the temp readings. Let me revert back to the original docker image and see if the temperature readings are stable. |
|
Yeah, my CPU temps are just jumpy for some reason. I reverted to the original docker image/settings and same behavior. Also verified the same temps in the idrac browser interface. Looks like I'll be redoing my CPU heatsinks. Edit: |
|
@kyle-blair what's your CPU TDP? In your case I think using new high grade thermal paste can help, because your exhaust temperature doesn't change much under different fan speed, so it looks like air doesn't get temperature from CPU. ENABLE_LINE_INTERPOLATION=true
HIGH_FAN_SPEED=60
CPU_TEMPERATURE_FOR_START_LINE_INTERPOLATION=35
FAN_SPEED=15
CPU_TEMPERATURE_THRESHOLD=60
CHECK_INTERVAL=1Short interval time should safe your CPU from being overheat, and you definitely need higher HIGH_FAN_SPEED if you got Dell fan control from time to time. PID controller I mentioned above I'm planning to start new project from scratch in mostly C++. |
tigerblue77
force-pushed
the
master
branch
2 times, most recently
from
0568f14 to
ae625ab
Compare
|
@7Adrian I'm not a lawyer, but from my point of view it only concerns the application (here the Docker container) and its use. In concrete terms, I don't want the work of the people who participate in this repository to be taken over, copied, repackaged or reused in any way whatsoever for profit. For my part, As far as I'm concerned, I won't rule out adding a donation link to the README.md file in the future, but I want this tool to remain free 😊 I stop my work here for today, here are a few notes :
Any help is welcome ! Try to split in different branches/PR when possible :) |
|
hi @tigerblue77 I'm going to get an R740 as a lease to create another episode of my Superserver series, and I just wanted to ask you if you want me to test something. I know iDrac 9 up to the .30 version is okay, but I thought I should ask just in case you need testing or feedback! Thanks for your great work; I use it every day <3 |
|
Hello @jcastro, |
For sure! I'm just a bit lost on what's the docker image I need to test from @7Adrian's fork? thanks! |
|
I'm going to test the changes, and I'll publish image to the Docker repository with different tag once I've finished my tests. Edit: |
|
@7Adrian oops, sorry for that... |
tigerblue77
force-pushed
the
master
branch
4 times, most recently
from
fa62b41 to
931c80c
Compare
Hi, I create changes for letting users to use line interpolation of fan speed. We covered this upgrade in #44.
How that interpolation work:
ENABLE_LINE_INTERPOLATION= trueFAN_SPEED= 10HIGH_FAN_SPEED= 50CPU_TEMPERATURE_FOR_START_LINE_INTERPOLATION= 30CPU_TEMPERATURE_THRESHOLD= 70By default application work the same as before. You can change it if you want enable line interpolation by default.
Main changes:
PS. Changes tested on dell server with one CPU.