NVR replacement ZM high CPU

Discussions related to the 1.36.x series of ZoneMinder
Post Reply
airtite
Posts: 9
Joined: Wed Sep 01, 2021 12:07 pm

NVR replacement ZM high CPU

Post by airtite »

Hi

I have 2 x 32 camera dahua NVRs that I am trying to replace with a single ZM server, the server is running debian bullseye with ZM 1.36.5-buster

The server specs are 2 x Intel(R) Xeon(R) CPU E5620 @ 2.40GHz with hyperthreading enabled thus 8 vcpus in total, it has 64gigs of RAM and is an HP dl380 with sas drives in raid 5.

The cameras are all dahua IP cameras running the mainstream at 720p x 10fps (keyframe interval at 20) and the substream at 352x288 x 7fps (keyframe interval at 14)

Currently I have 39 cameras connected using both the mainstream and the substream so 78 monitors in total, I have the substream for each monitor in modect and the mainstream in nodect linked to the substream for recording events.

save jpegs is disabled and its running camera passthrough with crf=23 on all cameras.

The system seems to be running ie its recording and allows viewing of cameras but is sitting at a load of 10.91 ie 100% BUT /dev/shm is at 8% so the system seems to be using more CPU than RAM currently its showing 6Gigs used of the 64Gigs.

I still need to add another 21 cameras, whats confusing is that the NVRs which dont seem to have that high resources seemed to handle the cameras/recording way better than what I thought is a fairly high spec server.

Any pointers on what I could do to lower CPU?
parky
Posts: 45
Joined: Sun Aug 28, 2011 8:41 am

Re: NVR replacement ZM high CPU

Post by parky »

I don't know of anything ZM wise to suggest, but am sure others will though it sounds about right based on my setup

What I am thinking is, as I assume you have a G7 DL380 is to ditch the E5620 and get a matched pair of X5650's like I have in my server which will give you 24 cores to play with. My ZM box has a single X5650 and runs 720p @ a load of 2 and 1080p @ load 3 with 8 cameras on Mocord
User avatar
iconnor
Posts: 2879
Joined: Fri Oct 29, 2010 1:43 am
Location: Toronto
Contact:

Re: NVR replacement ZM high CPU

Post by iconnor »

Load is not necessarily indicative of 100% cpu. I like to use htop to visualise cpu usage.

The old nvrs were likely not decoding, doing motion detection, etc. Also, there is work to be done in ZM in reduce cpu use.

/dev/shm is no longer indicative of anything. We are no longer using to store all the raw images. They are in still in ram, just not in /dev/shm. Again, use htop.
airtite
Posts: 9
Joined: Wed Sep 01, 2021 12:07 pm

Re: NVR replacement ZM high CPU

Post by airtite »

iconnor wrote: Wed Sep 08, 2021 12:31 pm Load is not necessarily indicative of 100% cpu. I like to use htop to visualise cpu usage.

The old nvrs were likely not decoding, doing motion detection, etc. Also, there is work to be done in ZM in reduce cpu use.

/dev/shm is no longer indicative of anything. We are no longer using to store all the raw images. They are in still in ram, just not in /dev/shm. Again, use htop.
htop is showing CPU usage fluctuating between 80-100% across all 8cpu's, Mem is at +/-*g of 62G and swap is at 0K/5.5G

Regarding the OLD NVRs would it be less CPU intensive to just record with no motion detection or how would I go about emulating what the OLD NVRs did in ZM?
User avatar
burger
Posts: 390
Joined: Mon May 11, 2020 4:32 pm

Re: NVR replacement ZM high CPU

Post by burger »

Try lowering Analysis FPS to 2. Can be done via shell.

Code: Select all

update Monitors set AnalysisFPSLimit="2.00" where AnalysisFPSLimit is NULL;
Ref: https://wiki.zoneminder.com/MySQL

I like mocord more than modect, personally. Modect can miss events.
fastest way to test streams:
ffmpeg -i rtsp://<user>:<pass>@<ipaddress>:554/path ./output.mp4 (if terminal only)
ffplay rtsp://<user>:<pass>@<ipaddress>:554/path (gui)
find paths on ispydb or in zm hcl

If you are new to security software, read:
https://wiki.zoneminder.com/Dummies_Guide
User avatar
kitkat
Posts: 193
Joined: Sun Jan 27, 2019 5:17 pm

Re: NVR replacement ZM high CPU

Post by kitkat »

airtite wrote: Wed Sep 08, 2021 12:39 pm how would I go about emulating what the OLD NVRs did in ZM?
If you just want to record the streams without any motion detection or other analysis, and if the camers send AVC/h.264 video, then the best move is likely to be to set the Source Type on the General tab to FFMpeg, and to set the Video Writer on the Storage tab to Camera Passthrough.
airtite
Posts: 9
Joined: Wed Sep 01, 2021 12:07 pm

Re: NVR replacement ZM high CPU

Post by airtite »

kitkat wrote: Wed Sep 08, 2021 1:31 pm
airtite wrote: Wed Sep 08, 2021 12:39 pm how would I go about emulating what the OLD NVRs did in ZM?
If you just want to record the streams without any motion detection or other analysis, and if the camers send AVC/h.264 video, then the best move is likely to be to set the Source Type on the General tab to FFMpeg, and to set the Video Writer on the Storage tab to Camera Passthrough.

would this be less CPU intensive than motion detection?
User avatar
kitkat
Posts: 193
Joined: Sun Jan 27, 2019 5:17 pm

Re: NVR replacement ZM high CPU

Post by kitkat »

airtite wrote: Wed Sep 08, 2021 2:27 pm
kitkat wrote: Wed Sep 08, 2021 1:31 pm
airtite wrote: Wed Sep 08, 2021 12:39 pm how would I go about emulating what the OLD NVRs did in ZM?
If you just want to record the streams without any motion detection or other analysis, and if the camers send AVC/h.264 video, then the best move is likely to be to set the Source Type on the General tab to FFMpeg, and to set the Video Writer on the Storage tab to Camera Passthrough.

would this be less CPU intensive than motion detection?
Yes.

It's the decoding and analysis that really eats the CPU cycles - Simply passing the stream straight through to the disk will use fewer resources.


e2a: Oh, I seem to have missed the bit about setting Function on the General tab to Record rather than MoDect/Cord, but hopefully you guessed that part!
User avatar
burger
Posts: 390
Joined: Mon May 11, 2020 4:32 pm

Re: NVR replacement ZM high CPU

Post by burger »

It might be wise to use a combination of modect/mocord and record. For some cameras, you don't really care if you get statistics for each event. Say you are watching a store shelf, and if something is removed from the store shelf, it will be clear in the before / after of the video stream. In that case, you just go back in history until the item is back, and record will suffice. It works because the item you are watching is in the frame of the camera.

But, if you have say, a hallway and a door, and you want to know when people enter this room, if you have it on record, you won't know (via e.g. show timeline) when someone opens or closes the door. In this case, modect or better mocord will give you stats of any motion on the stream, which you can use to quickly find out when someone entered the room. In this case, if you had record, you'd have to go back through all the footage... Depending on the camera and time range it can be difficult.

Also, I'm not sure if record works with Linked Monitors. I know modect/mocord/nodect will though. If you have record on the main stream, you won't need it on the sub stream, obviously.
fastest way to test streams:
ffmpeg -i rtsp://<user>:<pass>@<ipaddress>:554/path ./output.mp4 (if terminal only)
ffplay rtsp://<user>:<pass>@<ipaddress>:554/path (gui)
find paths on ispydb or in zm hcl

If you are new to security software, read:
https://wiki.zoneminder.com/Dummies_Guide
airtite
Posts: 9
Joined: Wed Sep 01, 2021 12:07 pm

Re: NVR replacement ZM high CPU

Post by airtite »

so I deleted all the sub stream cameras and just have the 39 x 720p streams, I have switched to record and the load now is at Load: 10.86, I guess I would just have to play around with the setups to try and get a balance of what we need out of the system vs what it can actually do. That or revert back to the old dual NVR setup.
dougmccrary
Posts: 1172
Joined: Sat Aug 31, 2019 7:35 am
Location: San Diego

Re: NVR replacement ZM high CPU

Post by dougmccrary »

FWIW, more cores make a big diff, if that's an option.
Also, if you can drop from 10 to say 6FPS will probably also be a big help.
And crf doesn't matter with passthrough.
parky
Posts: 45
Joined: Sun Aug 28, 2011 8:41 am

Re: NVR replacement ZM high CPU

Post by parky »

yes I found that dropping to 6fps made a difference and still gives a decent video

A swap from the E5620 to a X5650 will give you the extra cores without going dual. £5 for a single chip or £16.50 for a pair from a quick search of the usual auction site
Mamonick
Posts: 5
Joined: Mon Sep 27, 2021 12:07 pm

Re: NVR replacement ZM high CPU

Post by Mamonick »

Try to downgrade to ZM 1.34. I am using 1.34.26 on i9-10850K and 64GB ram with 40 nonDahua cameras with Record function (1080p/720p 10fps) and 77 Dahua (1080p 10fps) with Nodect function. CPU usage is about 50%. After upgrade to 1.36 CPU usage goes to 100% and ZM doesnt record. Decoding in this version is very CPU intensive. Without decoding works fine if you dont need liveview.
If your cameras have Motion Detection you should use it. Download Dahua SDK (there is AlarmListen demo) and you can use ZM external telnet trigger to start and stop recording.
parky
Posts: 45
Joined: Sun Aug 28, 2011 8:41 am

Re: NVR replacement ZM high CPU

Post by parky »

Running 1.36.7 now and all my cameras are set to Mocord and not record jpegs and do passthrough. Live view still works just fine and the load is half of what 1.34 was
Post Reply