Zoneminder "Error" Messages

[crickel]

Hey everyone,

We're tuning our zoneminder installation, and run into a snag. We want to be notified automatically whenever there's a problem with recording or playing back video to make sure we're not missing anything.

Today I did a dump of the last 24 hours worth of logs. I have found the following error messages on our devices:

[Can't get shared memory id '7a6d0005', 5: No such file or directory]
[Waiting for capture daemon]
[Can't insert event: MySQL server has gone away]
['zma -m 3' exited abnormally, exit status 214]
[Can't find process with command of 'zma -m 10']
[Buffer overrun at index 15, image 15, slow down capture, speed up analysis or increase ring buffer size]
[Signal: Lost]
[Signal: Reacquired]
[Failed to set video format: Invalid argument]
[Backtrace: /lib/tls/i686/cmov/libc.so.6(gsignal+0x55) [0xb7037085]]
[Signal address is (nil), from 0xb7f2c410]
[Terminating, last frame sent time 1284045431.091325 secs more than maximum of 10.000000]
[Can't shmget, probably not enough shared memory space free: Invalid argument]
[Can't open events/7/10/09/09/10/39/37/006-capture.jpg: Interrupted system call]

These errors range from WARning, ERRor, and FATal levels. I've spent most of the morning trying to resolve the most frequent errors (the first two in the list). I have had absolutely zero success at even tracing down a root cause - all of the solutions appear to be pot-luck (Have you tried A? How about B? C? Ok, that fixed it).

Our system configuration is as clean as we can make it. We're already running INNODB, we've got shmmax and shmall set up properly for our system, we've got a clean database that doesn't need repairs, we're not capturing video at obscene rates.

But here's the bigger problem: According to manual checks on the actual system, everything is just peachy. Video is being recorded. I can play video back. All the cameras work just fine and are displaying streaming video fine. Motion detection is working properly. All the proper processes are running... I haven't found any events that have been recorded improperly.

If these errors are actually indicative of serious errors on our install, why is everything running properly? Alternately, if we can't trust the logs to provide valid error warnings, what can we trust to test our functionality?

Craig

[bb99]

Bump up your shared memory values by 25 to 50% and see what you get.

[crickel]

Bump up your shared memory values by 25 to 50% and see what you get.

I'm not sure we can. We're running at 4% memory free as it is. Of course, it's possible that I'm just completely misinterpreting what 'free' reports. Shared memory is still more or less kernel voodoo to me...

Here's what we have right now:

Code: Select all

$ cat /proc/sys/kernel/shmmax
134217728
$ cat /proc/sys/kernel/shmall
134217728

$ free -b
             total       used       free     shared    buffers     cached
Mem:    3708805120 3538436096  170369024          0  155054080 2856140800
-/+ buffers/cache:  527241216 3181563904
Swap:   3306520576      98304 3306422272

$ uname --all
Linux xxxx 2.6.24-16-server #1 SMP Thu Apr 10 13:58:00 UTC 2008 i686 GNU/Linux

These are Ubuntu 8.04 servers.

I believe what that's saying is that we've basically used everything we reasonably can without hitting swap like a hammer. If I'm incorrect, let me know - I spent a fair amount of time trying to figure out exactly how much memory we could use for these, but I could well be wrong. I'm still not sure if shmmax and shmall are supposed to be specified in bytes or kb or 'blocks'.

Craig

[bb99]

Have a look at this one: http://www.zoneminder.com/forums/viewtopic.php?t=16237

I personally start at your value and gradually increase until everything is working. After you clear up the "can't shmget" most of the rest (if not all) will disappear.

The shared memory value you have is good for a resolution slightly greater than 320*240.

[crickel]

Hmm. We're only recording at 320x240, though.

And if we increase the value for shared memory on the cameras, won't it be assigning blocks of such size for each camera that we start having to swap memory to disk, with all the badness that entails?

[crickel]

We've upgraded Zoneminder to a more recent version that uses mapped memory by default. This has actually resolved a lot of the error messages we were getting around memory and seems very helpful.

Still getting 'interrupted system call', 'buffer overrun', 'backtrace' and 'signal lost/reacquired'. I'm betting that the signal two are due to camera/cable issues, and the backtraces will disappear if we resolve the other issues, but the first two (especially the 'interrupted system call') have me stymied.

I do note there's something new on the buffers tab. Here's our settings:

Image Buffer Size: 200
Warmup Frames: 25
Pre Event Image Count: 5
Post Event Image Count: 20
Stream Replay Image Buffer: 150
Alarm Frame Count: 1

I tried to look up the 'stream replay image buffer' for more info on what it does, but the documentation simply says, 'This option is ...'

Does anyone know what that option actually does, or see any problem with the above settings?

Craig