Pythone.exe closing down on start up.

[Tinbum1]

Ever since the windows GPU was brought out I've been having problems with the instances of pythone.exe closing down on start up of Deepstack. After a few tries starting Deepstack it normally managed to start without this and once started it's run ok most of the time. Now it's got to the stage where I can't get it to start without the 2 instances of python closing down. It used to be mainly with multiple instances but its doing it with just one now.

Process Process-1: Traceback (most recent call last): File "D:\obj\windows-release\37amd64_Release\msi_python\zip_amd64\process.py", line 297, in _bootstrap File "D:\obj\windows-release\37amd64_Release\msi_python\zip_amd64\process.py", line 99, in run File "C:\DeepStack\intelligencelayer\shared\detection.py", line 71, in objectdetection queue = db.lrange(IMAGE_QUEUE, 0, 0) File "C://DeepStack\windows_packages\redis\client.py", line 1975, in lrange return self.execute_command('LRANGE', name, start, end) File "C://DeepStack\windows_packages\redis\client.py", line 901, in execute_command return self.parse_response(conn, command_name, **options) File "C://DeepStack\windows_packages\redis\client.py", line 915, in parse_response response = connection.read_response() File "C://DeepStack\windows_packages\redis\connection.py", line 739, in read_response response = self._parser.read_response() File "C://DeepStack\windows_packages\redis\connection.py", line 340, in read_response raise error redis.exceptions.BusyLoadingError: Redis is loading the dataset in memory

see also;
#63

[jeremypsl]

Yes, exact same problem for me !

[jeremypsl]

@Tinbum1 are you on Windows 10/11 ? Does it work sometimes or never ?
Did you tried AITool ?

[Tinbum1]

Windows 10, it was working OK once I got through the startup which I had to do a few times but now it's always. I use AiTool and always have. Much better than the BI version in my opinion.

[VorlonCD]

I'm just now trying the GPU version again now that I have a 1080ti with 11 GB RAM. (Win11) Seems to be OK so far but its only been running a few hours. The 4 GB ram on my old card was the reason I always assumed deepstack GPU kept crashing. I notice that with only TWO instances of DEEPSTACK running (ports 86, 87) my video ram is almost completely used up 10 out of 11 MB, so thats pretty tight. The 2 instances of deepstack and python.exe's's's take up about 6 GB of GPU/video ram by themselves. So if your card only has 6, it is certainly not enough to be used by deepstack GPU version.

Keep an eye on how much of your video ram is being used to see if it could be a factor - if it seems tight that may be why its crashing - Task manager > performance tab > GPU > RAM

Based on the error, I might try running the FLUSHALL command?
https://bobcares.com/blog/error-loading-redis-is-loading-the-dataset-in-memory/

Are you running the LATEST recommended versions of CUDA and CuDNN referenced here? (Custom install > Uncheck everything except CUDA, let it keep any NEWER drivers, but make sure that exact version of CUDA is installed)
https://docs.deepstack.cc/windows/

If BlueIris is running on the same machine, try turning off all its HARDWARE ACCELERATION options for encoding video, etc so it uses less video ram. Side note on that, I found BI hardware acceleration would fail unless I hacked my driver to remove restriction on maximum number of simultaneous NVENC video encoding sessions imposed by Nvidia to consumer-grade GPUs.
https://github.com/keylase/nvidia-patch/tree/master/win

[jeremypsl]

@VorlonCD
I run the FLUSHALL command but now :

• i don't have the redis-server.exe running
• python instances are running
• deepstack not working

[jeremypsl]

Oups i'm sorry ITS WORKING, it's just redis process use so much less ram than before i can't see it !!!
Before 2Go of RAM for redis server and now 3Mo...

[Tinbum1]

@VorlonCD Thanks- i'd seen that link but wasn't sure how to do it. Bit of investigation and

c:\DeepStack\redis>redis-cli FLUSHDB
OK

Worked for me also, thanks

[VorlonCD]

LOL nice, it was just a random guess. I wonder if it was due to this file being corrupt? You can just DELETE it and it gets re-created after a while:
"C:\DeepStack\redis\dump.rdb"

[jeremypsl]

Is there a way to prevent this behavior of redis server ? To accumulate that shit ?

[VorlonCD]

Not sure. First time ive heard of the issue but I'm no redis expert. I know that DUMP file gets periodically written to to preserve the contents of what its doing in memory. Then upon start, IF the file exists it tries to read it into memory to help speed things up initially.

I might speculate that it got corrupt when your computer blue screen crashed JUST as the DUMP file was being written.

So once and a while, with deepstack closed, delete the file I guess. Or run FLUSHDB with deepstack running.

[spammenotinoz]

Suggest consider moving to docker, any issues just re-start the container. I have a scheduled restart of the container, seems to keep things very healthy, ie: fixes the nothing found and not responding issues. You need to obviously stop BI starting Deepstack though.
Once the docker desktop is installed and working, this is the command I run once from Powershell to download the image and run the container. Suggest a quick overview on docker and containers, they work very differently to VMs.

docker run -d --name='deepstack' --gpus all -e VISION-DETECTION=True -e TZ="Australia/Sydney" -v localstorage:/datastore -p 80:5000 --restart unless-stopped deepquestai/deepstack:gpu

Suggest looking at Docker help so you know the parameters, but the above basically has deepstack running on port 80.
As you have the GPU version already working you already have the required nvidia junk installed.

[Tinbum1]

@spammenotinoz FYI You can set AiTool to automatically restart Deepstack if their is no response from the server.