2 or more camera streams stop daily

Forum for questions and support relating to the 1.27.x releases only.
Locked
TangoAlpha
Posts: 3
Joined: Mon Jun 16, 2014 8:02 pm

2 or more camera streams stop daily

Post by TangoAlpha »

New to zoneminder, but I now have (8) 720p@10 FPS cameras connected to a zoneminder 1.27 VM. Everything is working, motion detection, email alerts, etc., but I have to restart every 15 hours or so, or at least one of the camera streams gets "hung". Sometimes, the capture appears red, and sometimes it just won't populate a picture.

Below's a snippet from /var/log/syslog that happened around the time the camera feed went offline:

Jun 15 16:04:22 zoneminder zmwatch[3455]: INF [Restarting capture daemon for Front_door, time since last capture 13 seconds (1402873462-1402873449)]
Jun 15 16:04:23 zoneminder zmdc[3315]: INF ['zmc -m 2' stopping at 14/06/15 16:04:23]
Jun 15 16:04:23 zoneminder zmc_m2[3351]: INF [Got signal 15 (Terminated), exiting]
Jun 15 16:04:29 zoneminder zmdc[3315]: INF ['zmc -m 2' crashed, signal 8]
Jun 15 16:04:29 zoneminder zmdc[3315]: INF [Starting pending process, zmc -m 2]
Jun 15 16:04:29 zoneminder zmdc[3315]: INF ['zmc -m 2' starting at 14/06/15 16:04:29, pid = 5273]
Jun 15 16:04:29 zoneminder zmdc[5273]: INF ['zmc -m 2' started at 14/06/15 16:04:29]
Jun 15 16:04:29 zoneminder zmc_m2[5273]: INF [Starting Capture]
Jun 15 16:04:29 zoneminder zmc_m2[5273]: INF [Priming capture from rtsp://192.168.1.203:554/cam/realmonitor?channel=1&subtype=0&authbasic=YWRtaW46YWRtaW4=?tcp]

If I set the motion detection to disabled on the camera, then set it back to modect, the camera comes back, but a lot of functions seem sluggish until I reboot the zoneminder VM.

Here's a snippet from reenabling modect on a camera:

Jun 16 13:10:40 zoneminder zmc_m3[4130]: INF [Starting Capture]
Jun 16 13:10:40 zoneminder zmc_m3[4130]: INF [Priming capture from rtsp://192.168.1.204:554/cam/realmonitor?channel=1&subtype=0&authbasic=YWRtaW46YWRtaW4=?tcp]
Jun 16 13:10:40 zoneminder zma_m3[4135]: WAR [Waiting for capture daemon]
Jun 16 13:10:43 zma_m3[4135]: last message repeated 2 times
Jun 16 13:10:43 zoneminder web_php[3991]: ERR [socket_sendto( /tmp/zm/zms-247351s.sock ) failed: No such file or directory]
Jun 16 13:10:43 zoneminder zma_m3[4135]: WAR [Waiting for capture daemon]
Jun 16 13:10:46 zma_m3[4135]: last message repeated 2 times
Jun 16 13:10:46 zoneminder zma_m3[4135]: INF [In mode 3/1, warming up]
Jun 16 13:10:46 zoneminder zma_m3[4135]: WAR [Approaching buffer overrun, consider slowing capture, simplifying analysis or increasing ring buffer size]
Jun 16 13:10:47 zoneminder web_php[4113]: ERR [socket_sendto( /tmp/zm/zms-247351s.sock ) failed: No such file or directory]
Jun 16 13:10:50 zoneminder web_php[3982]: ERR [socket_sendto( /tmp/zm/zms-247351s.sock ) failed: No such file or directory]
Jun 16 13:10:53 zoneminder web_php[4113]: ERR [socket_sendto( /tmp/zm/zms-247351s.sock ) failed: No such file or directory]


I googled around and found this:
http://www.zoneminder.com/wiki/index.ph ... esolutions.

But I'm not sure whether to go the shared or mapped memory route, how best to implement on 1.27, or even if that is really the issue.

Any ideas before I start executing commands I don't really understand?
TangoAlpha
Posts: 3
Joined: Mon Jun 16, 2014 8:02 pm

Re: 2 or more camera streams stop daily

Post by TangoAlpha »

Increased the image buffer size to 75 on all the cameras, and the ring buffer error went away. Will see if the hang still happens.

top - 13:43:05 up 3:57, 1 user, load average: 1.51, 1.51, 1.57
Tasks: 154 total, 3 running, 151 sleeping, 0 stopped, 0 zombie
Cpu(s): 17.1%us, 1.4%sy, 0.0%ni, 81.3%id, 0.0%wa, 0.0%hi, 0.1%si, 0.0%st
Mem: 4129508k total, 2576856k used, 1552652k free, 88648k buffers
Swap: 4190204k total, 0k used, 4190204k free, 1866460k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
5425 www-data 20 0 331m 214m 201m S 15 5.3 0:28.30 zma
5483 www-data 20 0 331m 214m 201m S 14 5.3 0:25.65 zma
5457 www-data 20 0 331m 213m 201m S 13 5.3 0:23.31 zma
5536 www-data 20 0 331m 213m 201m S 13 5.3 0:23.62 zma
5407 www-data 20 0 331m 213m 201m R 13 5.3 0:23.45 zma
5470 www-data 20 0 331m 213m 201m R 13 5.3 0:23.06 zma
5442 www-data 20 0 331m 213m 201m S 12 5.3 0:23.59 zma
5498 www-data 20 0 331m 213m 201m S 12 5.3 0:23.04 zma
5485 www-data 20 0 443m 253m 201m S 5 6.3 0:09.66 zmc
5409 www-data 20 0 444m 254m 201m S 5 6.3 0:08.94 zmc
5430 www-data 20 0 337m 214m 201m S 5 5.3 0:10.48 zmc
5444 www-data 20 0 337m 214m 201m S 5 5.3 0:09.97 zmc
5398 www-data 20 0 337m 214m 201m S 4 5.3 0:08.76 zmc
5421 www-data 20 0 446m 255m 201m S 4 6.3 0:08.51 zmc
5458 www-data 20 0 444m 254m 201m S 4 6.3 0:09.06 zmc
5471 www-data 20 0 445m 254m 201m S 4 6.3 0:08.55 zmc


Jun 16 13:41:35 zoneminder zmc_m1[5398]: INF [Driveway: 1000 - Capturing at 9.90 fps]
Jun 16 13:41:36 zoneminder zmc_m2[5409]: INF [Front_door: 1000 - Capturing at 9.90 fps]
Jun 16 13:41:36 zoneminder zmc_m3[5421]: INF [Front_lawn: 1000 - Capturing at 9.90 fps]
Jun 16 13:41:36 zoneminder zma_m1[5407]: INF [Driveway: 1000 - Processing at 9.80 fps]
Jun 16 13:41:36 zoneminder zmc_m4[5430]: INF [Side_yard_AC: 1000 - Capturing at 9.90 fps]
Jun 16 13:41:36 zoneminder zmc_m5[5444]: INF [Pool: 1000 - Capturing at 9.90 fps]
Jun 16 13:41:36 zoneminder zma_m4[5442]: INF [Side_yard_AC: 1000 - Processing at 9.90 fps]
Jun 16 13:41:36 zoneminder zmc_m6[5458]: INF [Back_lower_porch: 1000 - Capturing at 9.90 fps]
Jun 16 13:41:36 zoneminder zmc_m7[5471]: INF [Back_upper_deck: 1000 - Capturing at 9.90 fps]
Jun 16 13:41:36 zoneminder zma_m5[5457]: INF [Pool: 1000 - Processing at 9.90 fps]
Jun 16 13:41:37 zoneminder zmc_m8[5485]: INF [Side_garage: 1000 - Capturing at 9.90 fps]
Jun 16 13:41:38 zoneminder zma_m2[5536]: INF [Front_door: 1000 - Processing at 10.20 fps]
Jun 16 13:41:39 zoneminder zma_m3[5425]: INF [Front_lawn: 1000 - Processing at 9.62 fps]
Jun 16 13:41:39 zoneminder zma_m6[5470]: INF [Back_lower_porch: 1000 - Processing at 9.62 fps]
Jun 16 13:41:39 zoneminder zma_m7[5483]: INF [Back_upper_deck: 1000 - Processing at 9.62 fps]
Jun 16 13:41:39 zoneminder zma_m8[5498]: INF [Side_garage: 1000 - Processing at 9.71 fps]
TangoAlpha
Posts: 3
Joined: Mon Jun 16, 2014 8:02 pm

Re: 2 or more camera streams stop daily

Post by TangoAlpha »

No love, now another camera went red, and I'm seeing:

Jun 16 15:31:46 zoneminder zmc_m8[5485]: ERR [Unable to read packet from stream 0: error -541478725]
Jun 16 15:31:46 zmc_m8[5485]: last message repeated 5 times
Jun 16 15:31:46 zoneminder zmc_m5[5444]: INF [Pool: 67000 - Capturing at 10.00 fps]
Jun 16 15:31:46 zoneminder zmc_m8[5485]: ERR [Unable to read packet from stream 0: error -541478725]
Jun 16 15:31:46 zmc_m8[5485]: last message repeated 12 times
Jun 16 15:31:46 zoneminder zmc_m8[5485]: INF [Side_garage: 67000 - Capturing at 10.20 fps]
Jun 16 15:31:46 zoneminder zmc_m8[5485]: ERR [Unable to read packet from stream 0: error -541478725]
Jun 16 15:31:47 zmc_m8[5485]: last message repeated 17 times
Jun 16 15:31:47 zoneminder zma_m8[5498]: INF [Side_garage: 66828 - Gone into alarm state]
Jun 16 15:31:47 zoneminder zmc_m8[5485]: ERR [Unable to read packet from stream 0: error -541478725]
Jun 16 15:31:47 zoneminder zmc_m8[5485]: ERR [Unable to read packet from stream 0: error -541478725]
Jun 16 15:31:47 zoneminder zma_m8[5498]: INF [Side_garage: 66828 - Opening new event 16117, alarm start]
Jun 16 15:31:47 zoneminder zmc_m8[5485]: ERR [Unable to read packet from stream 0: error -541478725]
Jun 16 15:31:47 zmc_m8[5485]: last message repeated 63 times
Jun 16 15:31:47 zoneminder zmc_m8[5485]: WAR [Buffer overrun at index 33, image 67083, slow down capture, speed up analysis or increase ring buffer size]
Jun 16 15:31:47 zoneminder zmc_m8[5485]: ERR [Unable to read packet from stream 0: error -541478725]
Jun 16 15:31:48 zmc_m8[5485]: last message repeated 8 times
Jun 16 15:31:48 zoneminder zmc_m8[5485]: WAR [Buffer overrun at index 42, image 67092, slow down capture, speed up analysis or increase ring buffer size]
Jun 16 15:31:48 zoneminder zmc_m8[5485]: ERR [Unable to read packet from stream 0: error -541478725]
Jun 16 15:31:48 zmc_m8[5485]: last message repeated 14 times
Jun 16 15:31:48 zoneminder zmc_m8[5485]: WAR [Buffer overrun at index 57, image 67107, slow down capture, speed up analysis or increase ring buffer size]
Jun 16 15:31:48 zoneminder zmc_m8[5485]: ERR [Unable to read packet from stream 0: error -541478725]
Jun 16 15:31:48 zmc_m8[5485]: last message repeated 26 times
Jun 16 15:31:48 zoneminder zma_m8[5498]: INF [Side_garage: 66837 - Gone into alert state]
Jun 16 15:31:48 zoneminder zmc_m8[5485]: ERR [Unable to read packet from stream 0: error -541478725]
Jun 16 15:31:48 zmc_m8[5485]: last message repeated 22 times
Jun 16 15:31:48 zoneminder zmc_m2[5409]: INF [Front_door: 67000 - Capturing at 10.00 fps]
Jun 16 15:31:49 zoneminder zmc_m8[5485]: ERR [Unable to read packet from stream 0: error -541478725]
Jun 16 15:31:49 zmc_m8[5485]: last message repeated 18 times
Jun 16 15:31:49 zoneminder rsyslogd-2177: imuxsock begins to drop messages from pid 5485 due to rate-limiting
Jun 16 15:31:49 zoneminder zma_m8[5498]: INF [Side_garage: 66862 - Left alarm state (16117) - 59(9) images]
Jun 16 15:31:49 zoneminder zma_m8[5498]: INF [Side_garage: 66862 - Closing event 16117, alarm end]
Jun 16 15:31:51 zoneminder zma_m8[5498]: INF [Side_garage: 67000 - Processing at 11.63 fps]
Jun 16 15:31:52 zoneminder rsyslogd-2177: imuxsock lost 409 messages from pid 5485 due to rate-limiting
Jun 16 15:31:52 zoneminder zmc_m8[5485]: ERR [Unable to read packet from stream 0: error -541478725]
Jun 16 15:31:54 zmc_m8[5485]: last message repeated 199 times
Jun 16 15:31:54 zoneminder rsyslogd-2177: imuxsock begins to drop messages from pid 5485 due to rate-limiting
Jun 16 15:31:58 zoneminder rsyslogd-2177: imuxsock lost 459 messages from pid 5485 due to rate-limiting
Jun 16 15:31:58 zoneminder zmc_m8[5485]: ERR [Unable to read packet from stream 0: error -541478725]
Jun 16 15:31:59 zmc_m8[5485]: last message repeated 199 times
Jun 16 15:31:59 zoneminder rsyslogd-2177: imuxsock begins to drop messages from pid 5485 due to rate-limiting
Jun 16 15:32:00 zoneminder zma_m1[5407]: INF [Driveway: 67000 - Processing at 10.00 fps]
Jun 16 15:32:01 zoneminder zma_m5[5457]: INF [Pool: 67000 - Processing at 10.00 fps]
Jun 16 15:32:03 zoneminder zmfilter[5496]: INF [Archiving event 16117]
Jun 16 15:32:03 zoneminder zmfilter[5496]: INF [Creating notification email]
Jun 16 15:32:03 zoneminder zmfilter[5496]: INF [Sending notification email 'ZoneMinder: Alarm - Side_garage-16117 (63 - 34 9)']
Jun 16 15:32:03 zoneminder postfix/smtpd[6105]: connect from localhost[127.0.0.1]
Jun 16 15:32:03 zoneminder postfix/smtpd[6105]: 4F77D3E01D4: client=localhost[127.0.0.1]
Jun 16 15:32:03 zoneminder postfix/cleanup[6108]: 4F77D3E01D4: message-id=<20140616223203.4F77D3E01D4@zoneminder>

Grepping for zma_m8, I see:

Jun 16 15:30:25 zoneminder zma_m8[5498]: INF [Side_garage: 66000 - Processing at 10.00 fps]
Jun 16 15:31:47 zoneminder zma_m8[5498]: INF [Side_garage: 66828 - Gone into alarm state]
Jun 16 15:31:47 zoneminder zma_m8[5498]: INF [Side_garage: 66828 - Opening new event 16117, alarm start]
Jun 16 15:31:48 zoneminder zma_m8[5498]: INF [Side_garage: 66837 - Gone into alert state]
Jun 16 15:31:49 zoneminder zma_m8[5498]: INF [Side_garage: 66862 - Left alarm state (16117) - 59(9) images]
Jun 16 15:31:49 zoneminder zma_m8[5498]: INF [Side_garage: 66862 - Closing event 16117, alarm end]
Jun 16 15:31:51 zoneminder zma_m8[5498]: INF [Side_garage: 67000 - Processing at 11.63 fps]
Jun 16 15:32:06 zoneminder zma_m8[5498]: INF [Side_garage: 68000 - Processing at 66.67 fps]
Jun 16 15:32:21 zoneminder zma_m8[5498]: INF [Side_garage: 69000 - Processing at 66.67 fps]
Jun 16 15:32:36 zoneminder zma_m8[5498]: INF [Side_garage: 70000 - Processing at 66.67 fps]
Jun 16 15:32:52 zoneminder zma_m8[5498]: INF [Side_garage: 71000 - Processing at 62.50 fps]
Jun 16 15:33:11 zoneminder zma_m8[5498]: INF [Side_garage: 72000 - Processing at 52.63 fps]
Jun 16 15:33:26 zoneminder zma_m8[5498]: INF [Side_garage: 73000 - Processing at 66.67 fps]
Jun 16 15:33:45 zoneminder zma_m8[5498]: INF [Side_garage: 74000 - Processing at 52.63 fps]
Jun 16 15:34:09 zoneminder zma_m8[5498]: INF [Side_garage: 75000 - Processing at 41.67 fps]
Jun 16 15:34:26 zoneminder zma_m8[5498]: INF [Side_garage: 76000 - Processing at 58.82 fps]
sleuth
Posts: 5
Joined: Mon Jun 23, 2014 5:24 pm

Re: 2 or more camera streams stop daily

Post by sleuth »

What are the camera settings, are you using ffmpeg or libvlc? I'm new to ZM too...
linuxsense
Posts: 374
Joined: Wed Nov 07, 2007 1:59 am
Location: Huntington Beach, California
Contact:

Re: 2 or more camera streams stop daily

Post by linuxsense »

You want to use memory map unless you have a specific reason not to use it.

It looks like you are not using a capture method that is completely compatible with your cameras. Dont use the 'tcp' option as it has been depreciated in recent versions of libvlc and ffmpeg...I think. Are you able to stream the cams via RTSP using VLC or other software? It looks like zm simply cant talk to the cam. Also, restart zm when you make changes to the camera settings or you will end up chasing your tail.
rceccleston
Posts: 9
Joined: Wed Jan 22, 2014 10:00 am

Re: 2 or more camera streams stop daily

Post by rceccleston »

After experiencing similar problems, I found the following works well on a 4x 1080P cameras capturing at 25 frames per second.

Source Type - Libvlc
Maximum FPS - 5 (monitoring zones at 5 FPS to reduce CPU load)
Alarm FPS - 15 (recording at 15 FPS to reduce storage demand)

Source Path - rtsp://xxx.xxx.xxx.xxx
Target Colorspace - 24 bit
Capture Width - (720P = 1280) or (1080P = 1920)
Capture Height - (720P = 720) or (1080P = 1080)

Image Buffer Size (frames) - 50
Warmup Frames - 25
Pre Event Image Count (frames) - 25
Post Event Image Count (frames) - 25

I would recommend setting up a single camera in 'monitor' function before adding more cameras.

Once an image can be viewed on all camera in 'monitor' function, progressively switch each camera to 'modetect' with limited zone coverage.
linuxsense
Posts: 374
Joined: Wed Nov 07, 2007 1:59 am
Location: Huntington Beach, California
Contact:

Re: 2 or more camera streams stop daily

Post by linuxsense »

Small tip: Many cameras seem to advertise audio as an option even if they don't actually provide audio and even if they do you cant do anything with it on zm so adding an option to ffmpeg of "allowed_media_types=video" seems to aid in stability. I am not sure if a similar option is available for libvlc as I am getting better results and lower overhead with ffmpeg on my new build.
User avatar
knight-of-ni
Posts: 2404
Joined: Thu Oct 18, 2007 1:55 pm
Location: Shiloh, IL

Re: 2 or more camera streams stop daily

Post by knight-of-ni »

rceccleston wrote: Maximum FPS - 5 (monitoring zones at 5 FPS to reduce CPU load)
Alarm FPS - 15 (recording at 15 FPS to reduce storage demand)
The Maximum FPS and Alarm Maximum FPS fields are intended for local cameas only. For IP cameras, do not place values into these fields. Doing so introduces undesrieable side effects that may not be immediately noticed. Significant lag is introduced into the live view and entire motion events are missed.
rceccleston wrote: Target Colorspace - 24 bit
Maybe you are already aware, if one has libjpeg-turbo installed on the system you can switch the color space to 32bit for a significant performance boost.
Visit my blog for ZoneMinder related projects using the Raspberry Pi, Orange Pi, Odroid, and the ESP8266
All of these can be found at https://zoneminder.blogspot.com/
rceccleston
Posts: 9
Joined: Wed Jan 22, 2014 10:00 am

Re: 2 or more camera streams stop daily

Post by rceccleston »

knnniggett wrote:
rceccleston wrote: Maximum FPS - 5 (monitoring zones at 5 FPS to reduce CPU load)
Alarm FPS - 15 (recording at 15 FPS to reduce storage demand)
The Maximum FPS and Alarm Maximum FPS fields are intended for local cameas only. For IP cameras, do not place values into these fields. Doing so introduces undesrieable side effects that may not be immediately noticed. Significant lag is introduced into the live view and entire motion events are missed.
rceccleston wrote: Target Colorspace - 24 bit
Maybe you are already aware, if one has libjpeg-turbo installed on the system you can switch the color space to 32bit for a significant performance boost.
How do you confirm if libjpeg-turbo is installed and active within Ubuntu 12.04?

Thank you.
linuxsense
Posts: 374
Joined: Wed Nov 07, 2007 1:59 am
Location: Huntington Beach, California
Contact:

Re: 2 or more camera streams stop daily

Post by linuxsense »

Easiest way to see if ZM found the libs and was built against them is to use ldd on one of the binary files. Find the dir where 'zmc' is located and do "ldd zmc | grep turbo" and see if libjpeg-turbo shows up.
linuxsense
Posts: 374
Joined: Wed Nov 07, 2007 1:59 am
Location: Huntington Beach, California
Contact:

Re: 2 or more camera streams stop daily

Post by linuxsense »

One thing to keep in mind is many IP cams can only support two streams at a time so if you are feeding a stream to ZoneMinder and accessing it directly you can get the oldest stream to be ended to supply the newer ones. If you use a mobile app to watch the cams streams set it up to pull the stream from ZoneMinder instead of going direct to the can unless you know you won't max out the cams capability to feed multiple sources at the same time.
dazed
Posts: 66
Joined: Thu Feb 13, 2014 5:32 pm

Re: 2 or more camera streams stop daily

Post by dazed »

linuxsense wrote:Easiest way to see if ZM found the libs and was built against them is to use ldd on one of the binary files. Find the dir where 'zmc' is located and do "ldd zmc | grep turbo" and see if libjpeg-turbo shows up.
I finally got my libvlc to work but when I tried "ldd zmc | grep turbo" in my /usr/bin folder i got no message.

Can I just edit it in? I got my libvlc to finally start working when I followed this http://lachlanmiskin.com/blog/2012/06/2 ... rker-0xd9/

using
sudo apt-get install dpkg-dev
sudo apt-get build-dep libjpeg-turbo
apt-get source libjpeg-turbo
cd libjpeg-turbo-1.1.90+svn733
cp -p jdmarker.c jdmarker.c.orig
vi jdmarker.c
dpkg-buildpackage -rfakeroot -uc -b
sudo dpkg -i ~zmadmin/libjpeg-turbo8_1.1.90+svn733-0ubuntu4.1_i386.deb
sudo service zoneminder restart

with some errors at after the dpkg-buildpackage -rfakeroot -uc -b and the one after that.

But it did get my VLC to start working...just no turbo, and no errors so far....
linuxsense
Posts: 374
Joined: Wed Nov 07, 2007 1:59 am
Location: Huntington Beach, California
Contact:

Re: 2 or more camera streams stop daily

Post by linuxsense »

No, don't do that. Sorry if I overlooked it, half asleep, but what distro are you running and what version of ZoneMinder are you using? Built from source or installed with a package?

You are running this in a VM? I have a few VM images that are ripe to build the current ZoneMinder code with all the goodies. When I get a few minutes I can clean up my 14.04 x86_64 image and share it. It uses turbo, recent libvlc, a good ffmpeg with x264, etc. I use the same confg on my physical servers and it works extremely well.
dazed
Posts: 66
Joined: Thu Feb 13, 2014 5:32 pm

Re: 2 or more camera streams stop daily

Post by dazed »

Im using Ubuntu 12.04 and zm 1.27 on a dedicated server. No VM.

Well as far as the errors go they returned and the picture on the libvlc started making wierd pixel shifts (dont know how to describe) so
I'm back to using ?tcp....thought maybe this libturbo would be my answer to get the libvlc working instead of ffmpeg.
linuxsense
Posts: 374
Joined: Wed Nov 07, 2007 1:59 am
Location: Huntington Beach, California
Contact:

Re: 2 or more camera streams stop daily

Post by linuxsense »

Ahh. I wasn't sure. You said "..connected to a zoneminder 1.27 VM" so I thought it might be running in a virtual machine.
Locked