ZM administraive problems: Overruns and Crashing Server

[FuriousGeorge]

First of all, I'd like to thank the writers of this program. Despite the headaches its causing me, I'm sure it does a lot of good for other people in the community.

The first thing I noticed was that Modect mode was unreliable. No matter how much fiddling I did, the detection was either far too sensitive (one never ending event) or nowhere near sensitive enough (no events, ever)

My workaround was to use mocord mode, and delete any event with no alarm frames. This works well enough, but unfortunately, despite the speed of my computer, the load always seems too high.

I get buffer overruns and corrupted db tables. I wish I could get you my settings. but about one second ago my load jumped to 30, and the remote machine has become totally unresponsive.

When zoneminder crashes, it may take out the entire server. It will hang when I try to ssh in, and I need to call the location to have them restart the server.

When I add a monitor, I lower the blend percentage to 5, and double all the buffers in the buffer section. Capture speed is 3fps. Nonetheless, zoneminder will only run from between a few minutes and a few days.

When zoneminder breaks, it sometimes breaks catastrophically. The db seems to get at least a little corrupted every few days. Does anyone think this is partly a result of lowering my event length to 1 minute?

Unfortunately, there is no way to view a feed at 15fps and record at three, even when there is an alarm. I actually believe it may be possible to do this in Modect mode, but modect mode has never been very reliable for me.

If anyone has any suggestions I would really appreciate it. I'm trying to avoid installing a different OS / software solution.

[timcraig]

Here's my advice on buffer overruns:

When you get buffer overruns, it because you CPU got interrupted or your CPU can't keep with the motion detection. Zoneminder puts a captures frames and buts them in the buffer for the zone motion detection to analyze. If the motion detection demon falls behind, the capture demon will start overwriting capture frames that haven't been analyzed yet and the sequence of the frames are corrupted.

To address overruns you have 3 options:
1) lower the monitor FPS.
2) reduce the monitor zone complexity (e.g use alarm pixels instead of blobs).
3) increase your monitor's image buffer size (found under the 'Buffers' tab in the monitor settings screen).

The best way to address overruns is option #3.
Just keep raising the Image buffer size until you don't see buffer runs anymore. The Image Buffer Size uses shared memory, so you'll have to watch out for hitting the shared memory limit (look for error messages about unable to access shared memory with command "cat /var/log/syslog | grep ERR"). You will need to raise your shared memory settings and possible add more memory to your PC.

If you tight on memory and you have monitors that you only use for monitoring or always have set to function 'None', lower the image buffer settings to the minime (10) as ZM will still allocate shared memory for 'None' and 'Monitor' monitors (so that you can change their functions to a motio without having to restart ZM).

[Lee Sharp]

Lots of complaints but short on information.

1) It sounds like this computer is doing other things. ZoneMinder is very IO intensive, and shared duty is NOT recomended.

2) High load, see above... Also, what hardware is it on? How many cameras, or what type ate what resolution and frame rate?

3) Modect is hard... But if the server is overloaded, it is near impossable to tune properly... See number 1 and 2 above.

4) I only see database crashes when the system hangs during a write. This can happen often if power is bad, or the system itself is unstable. You can stick a fix in cron, but first lets see why the system is crashing. I have cameras up for months with no issues.

It can work. I have several that work well myself, and so do others. And we are even willing to help you fix your system. But try just a little bit of humility...

[FuriousGeorge]

To address overruns you have 3 options:
1) lower the monitor FPS.
2) reduce the monitor zone complexity (e.g use alarm pixels instead of blobs).
3) increase your monitor's image buffer size (found under the 'Buffers' tab in the monitor settings screen).

The best way to address overruns is option #3.
Just keep raising the Image buffer size until you don't see buffer runs anymore. The Image Buffer Size uses shared memory, so you'll have to watch out for hitting the shared memory limit (look for error messages about unable to access shared memory with command "cat /var/log/syslog | grep ERR"). You will need to raise your shared memory settings and possible add more memory to your PC.

I did set aside about 700 megs for ShMem so I should be good there.
I've doubled my buffers from their default values already, but I'll keep increasing them nontheless.

I'll let you know if it works.

[FuriousGeorge]

Lots of complaints but short on information.

Sorry about that. Even though I couldn't ssh in, there was some info I could have given, but left out.

1) It sounds like this computer is doing other things. ZoneMinder is very IO intensive, and shared duty is NOT recomended.

I am using it with KDE to display all 13 feeds in montage mode in FF. Could that be a part of the problem

2) High load, see above... Also, what hardware is it on? How many cameras, or what type ate what resolution and frame rate?

I notice my load is real low until Mozilla Starts.

I have 13 cameras, which 'came with the building' of unknown make and model.

Some are B&W, some are RGB24, others are B&W with IR.

I run them at 320 x 240

As for framerate, they are all at 3fps.

3) Modect is hard... But if the server is overloaded, it is near impossable to tune properly... See number 1 and 2 above.

4) I only see database crashes when the system hangs during a write. This can happen often if power is bad, or the system itself is unstable. You can stick a fix in cron, but first lets see why the system is crashing. I have cameras up for months with no issues.

It can work. I have several that work well myself, and so do others. And we are even willing to help you fix your system. But try just a little bit of humility...

Sorry, I don't know what came over me. My frustration leaked into my post way too much.

I recently set up all 13 cameras identically, and more conservatively on the motion detection.

While I still get overruns eventually, the system fails catastrophically less often. I changed a bunch of settings in one shot so it's hard to know what caused the slight improvement.

All 13 cameras are now set up like this

Function: Mocord
Max FPS: 3
Alarm Max FPS: 3
Ref Image Blend %: 5


Device Format: NTSC
Capture Width: 320
Capture Hieght: 240


Image buffer size: 80
Warmup Frames: 50 (should I be lowering that instead of raising it?)
Pre Event Image Count: 20
Post Event Image Count: 20
Stream Replay Image Buffer: 1000
Alarm Frame Count: 0


Frame Skip: 0
Section Length: 60


Zones: One zone, encompassing the entire image, using the 'Fast / Medium" preset.

I'm not worried about the accuracy of motion detection, atm.

Tonight, I logged in after hours to check on it, and even after restarting ZM, it was having issues:

Code: Select all

# tail -fn 100 /var/log/messages
Oct  8 04:53:00 isabel zma_m17[20428]: INF [Downstairs-BarI: 2900 - Starting new event 1055787]
Oct  8 04:53:00 isabel zma_m8[20394]: INF [Upstairs-Dining: 2902 - Ending event 1055778]
Oct  8 04:53:00 isabel zma_m8[20394]: INF [Upstairs-Dining: 2902 - Starting new event 1055788]
Oct  8 04:53:00 isabel zma_m16[20416]: WAR [Waiting for capture daemon]
Oct  8 04:53:00 isabel zma_m10[20382]: INF [Outside-Front: 2903 - Ending event 1055779]
Oct  8 04:53:00 isabel zma_m10[20382]: INF [Outside-Front: 2903 - Starting new event 1055789]
Oct  8 04:53:00 isabel zma_m9[20372]: WAR [Waiting for capture daemon]
Oct  8 04:53:00 isabel zma_m42[20476]: WAR [Waiting for capture daemon]
(last line repeats many times)

A quick reboot resolved it.

[Lee Sharp]

1) It sounds like this computer is doing other things. ZoneMinder is very IO intensive, and shared duty is NOT recomended.

I am using it with KDE to display all 13 feeds in montage mode in FF. Could that be a part of the problem

Yep. Firefox is a fat bloated pig, and the 13 streams are not nice to your memory either. Do a 'free -m' while you are in next time. Ouch... Look at compiling the xlib_shm program to just watch the streams as ZM gets them. Much less overhead.

Also, how much memory is in the system?

[FuriousGeorge]

I have 2GB of memory.

We've looked into xlib_shm, but the people wont use it, opting for the slightly glitzier web interface instead.

Here's a stability update:

It went a few days without crashing so i changed all the zones to 'high sensitivity, best'.

The good news is its been a few hours with no overruns, but unfortunately even a bar full of people at 1am may not trigger any alarms. My blend %ge was as low as five, but I dont want to go messing around with all those settings and run into stability issues again.

As such, I think I'm going to be forgoing motion detection for the near future.

All the other settings are same as my last post, but i'm switching to record mode.

[SlovakJoe]

Lee's right. You need to turn this into a headless server. The viewing should be done on another machine that sits on the same LAN. The ZM documentation itself points out that the same machine used for running ZM should never be used for viewing.

Then again, I sympathize with your problem. Maybe the client thinks of this as his one and only computer. It can be tough to tell a small business owner he needs to spend $$$ on another computer because the security system can't show him the video.