Hi there,
i'm running zoneminder 1.36.32 + event server on a Dell R730XD on Proxmox in a LXC with a Tesla P40.
CPU is loaded with 120GB RAM and 25 Cores CPU.
I got 13 cams. All of them are integrated in zm with the main (nodect mode, 4k) and the substream (modect mode, 640x480...).
Unfortunately the database is growing from time to time. So i increased mysql to 700.
I think the main problem is that my 24GB Tesla P40 is running full. See the Host nvidia-smi.
I also got sereval log errors. Such this:
Error running model: Timeout waiting for pyzm_uid33_gpu_lock portalock for 100 seconds
--------------------------------------------------------------------------------------
Timeout waiting for pyzm_uid33_gpu_lock portalock for 100 seconds
--------------------------------------------------------------------------------------
Error loading same model variation for face:Error while calling cudaMallocHost(&data, new_size*sizeof(float))
in file /tmp/pip-install-bg4asj5v/dlib_ffc5739eb1204a7e9def39b29673b390/dlib/cuda/gpu_data.cpp:211. code: 2, reason: out of memory
Not all hooks are failing. So it is working. But not all...
Can you please help me with this problem? Would it be helpful when i create a additional LXC with mlapi?
Thank you
Host:
Code: Select all
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 525.60.12 Driver Version: 525.60.13 CUDA Version: 12.0 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 Tesla P40 On | 00000000:82:00.0 Off | Off |
| N/A 87C P0 67W / 250W | 21224MiB / 24576MiB | 3% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| 0 N/A N/A 1464885 C /usr/bin/python3 192MiB |
| 0 N/A N/A 1465149 C /usr/bin/python3 192MiB |
| 0 N/A N/A 1465544 C /usr/bin/python3 192MiB |
| 0 N/A N/A 1465775 C /usr/bin/python3 192MiB |
| 0 N/A N/A 1466093 C /usr/bin/python3 192MiB |
| 0 N/A N/A 1466095 C /usr/bin/python3 192MiB |
| 0 N/A N/A 1466355 C /usr/bin/python3 192MiB |
| 0 N/A N/A 1466356 C /usr/bin/python3 192MiB |
| 0 N/A N/A 1466972 C /usr/bin/python3 192MiB |
....
Code: Select all
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 525.60.13 Driver Version: 525.60.13 CUDA Version: 12.0 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 Tesla P40 On | 00000000:82:00.0 Off | Off |
| N/A 88C P0 74W / 250W | 21362MiB / 24576MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
+-----------------------------------------------------------------------------+