ZoneMinder Forums

Hi,
new user on the forum, but I've been using Zoneminder for about two years now.
I've recently set up a new system running 1.36.33. I'm just using it for 24/7 recording and the mobile app, no motion detection required.
I have 8 camera's, 2 Dahua 3MP, 4 Dahua 4MP, and 2 Hi3516 based (Chinese) 2MP.
After experimenting with various settings I found out that the only setup keeping CPU low is set all camera's to H264 (the Dahua 4MP also support H265) and use camera passthrough. Decoding is ON, except for the 2 Chinese camera's. When I switch decoding on for those two, CPU increases enormously, and I constantly see the zmc processes restart on ALL camera's. Might be due to CPU usage; I've seen loads of 27 on a 6 core system so basically unusable.

So at the moment I don't have live for these two camera's. Any idea what could cause this?
Capture method is ffmpeg (with the rtsp stream url, port 554, UDP) for all of them, not using Onvif at all. I tried both CBR and VBR encoding on the camera's, didn't make a difference. I can't set the iframe interval higher than 12 on these camera's whereas on the Dahua's I can't set it LOWER than 25 but that's the only real difference I see. It almost looks as if the Chinese camera's send "bad" rtsp traffic, but does that even exist?

Some additional information: these Chinese camera's + the 2 Dahua 3MP camera's have been there for a while. In the past I used Milestone software on old hardware, and when previewing it always took a lot longer for the Chinese camera's to be visible than for the Dahua ones. No difference in recording though.

For what it's worth: system is running on Debian and consists of a i5 9600k with 16GB of ram.

They likely have a huge keyframe interval. We have buffer everything since the last keyframe, so this can consume a lot of ram.

We need to change how ZM records in order to deal with this problem, or throw a lot of ram at it.

Well, shouldn't it be BETTER for these cams then? For the 4MP Dahua's I can set the I Frame interval between 20 and 150 (it is now set to 40). For the 3MP I can set it between 25 and 150 (now set to 50 which is default).
For the Chinese ones, I can't set it HIGHER than 12. So the interval is a lot lower than for the Dahua's which run fine.

Lower is better. I have a couple at 2. Meaning there is a complete frame, followed by one with only changes, followed by another complete frame. The higher the number (interval) the more partial, changes only, frames there are. Those partial frames are useless without the first full, complete keyframe.
HTH.

Well, assuming you defined your bitrate, a key frame interval of 2 means that you'll have to compress your key frames A LOT more to squeeze that information in the requested bitrate. So whilst it may be better from a memory consumption point of view, it certainly will decrease image quality.

I don't think it's really memory related anyway. I did a few more experiments; as soon as I switch on decoding for one of the Chinese camera's, almost immediately ALL of the zmc processes (also the ones of the Dahua camera's) start crashing every ~20 seconds. Memory utilization stays low. It's only after a few minutes that I see load increasing significantly but thay may be because each time all of the zmc processes have to restart.

Maybe I should rephrase my question to: Chinese camera's crash all zmc processes. Because that's what seems to happen.

Yeah, but in your case you can't cant go that low. So have you set whatever the minimums are?

I've just set it to 1; exactly the same behaviour.
As soon as I switch on decoding for this camera all of the zmc processes begin to crash, and after a bit you see load increasing (I assume this is because the processes clog up the CPU and then crash).

top - 13:14:18 up 2 days, 16:29, 1 user, load average: 4.03, 2.13, 1.70
I had 15G of free memory when this started happening so it really doesn't look like it's a memory problem.

It's very interesting that enabling decoding for this one camera also impacts all of the other capture processes. As if we're hitting a bug in a common part of the software.

Probably worth analysing some of the video if someone is equipped to do that from a capture.

I can take a capture if that would help anyone.
Some extra information: I have a second Zoneminder setup running on a VM in another location. There are also two Chinese camera's plus a few others (Hikvision and Dahua) on that installation; not the same ones but the interface of all of these Hisilicon camera's looks exactly the same so they are certainly similar. They don't cause any problem on this installation though.

I could simply replace the two camera's, they were inexpensive so it's not a big loss. But I'd like to know what's causing the problem nevertheless.

debug logs would likely shed some light.

There isn't much to see in the individual zmc logs.
I suppose zmdc.log is the one to look at.

Interestingly, it looks like usually one of the OTHER camera's crash when I enable decoding for one of the "faulty" ones.
The problematic camera's are 1 and 2.
As soon as I start decoding camera 1 or camera 2, I see in the logs:
03/06/23 12:50:51.111547 zmdc[118219].INF [ZMServer:716] ['zmc -m 5' crashed, signal 8]
03/06/23 12:51:12.446343 zmdc[118367].INF [ZMServer:716] ['zmc -m 5' crashed, signal 8]
03/06/23 12:51:33.682761 zmdc[118516].INF [ZMServer:716] ['zmc -m 7' crashed, signal 8]
03/06/23 12:51:54.964906 zmdc[118672].INF [ZMServer:716] ['zmc -m 7' crashed, signal 8]
03/06/23 12:52:16.651106 zmdc[118847].INF [ZMServer:716] ['zmc -m 5' crashed, signal 8]
03/06/23 12:52:37.976564 zmdc[119002].INF [ZMServer:716] ['zmc -m 5' crashed, signal 8]
03/06/23 12:52:59.223078 zmdc[119160].INF [ZMServer:716] ['zmc -m 8' crashed, signal 8]
03/06/23 12:53:20.665181 zmdc[119312].INF [ZMServer:716] ['zmc -m 7' crashed, signal 8]
03/06/23 12:53:41.977085 zmdc[119465].INF [ZMServer:716] ['zmc -m 8' crashed, signal 8]
03/06/23 12:54:03.252749 zmdc[119614].INF [ZMServer:716] ['zmc -m 5' crashed, signal 8]
03/06/23 12:54:24.608055 zmdc[119765].INF [ZMServer:716] ['zmc -m 6' crashed, signal 8]

Quite often it's one of the 4MP camera's that crash, but not always. Never the problematic one though.

I occasionally see this login the zmc logs
[non increasing dts, fixing. our dts -2400 stream 0 last_dts 0. reorder_queue_size=0]
but I see that for ALL of the camera's, also the ones that don't cause problems. Might not even be relevant since I'm using camera passthrough.