RTSP stream, image smudging and ZM Warnings

[asker]

Hello, I am trying to debug the following situation:

My ZM is monitoring a Foscam 9831W camera (1280x960)

Very often, when looking at live view, or even timeline recordings, I see the recorded image is "smudged" significantly

Around that time I get this problem, no errors are generated, no buffer overruns, except this message:

2015-03-17 15:07:41.599100 zms 29296 WAR Shared data not initialised by capture daemon, some query functions may not be available or produce invalid results for monitor Garage zm_monitor.cpp 426

I don't see any CPU spikes

This happens for some of my other cameras too.

My FPS is set to 1 in ZM
I am using FFMPEG and RTSP

I tried switching to VLC - I don't get this smudging, but it turns green once in a while, so net result is the same (in addition to much higher load - consistent 95% CPU). I also tried reducing capture to 720x480 -- the problem doesn't go away. "df -h" shows I am only using 18% of tmpfs

Any ideas on what is going on and how I can solve it? This is causing unnecessary events. I know this is not a camera problem because I always get crystal clear images on both my mobile phone (using TinyCam monitor) and on my browsers (directly connected to the cameras)

[asker]

Okay, so here is my analysis:

a) During transcoding, it seems FFMPEG is smudging an RTSP stream rather often. My guess I can play with options somewhere in ffmpeg?
b) If I use libvlc, there is no smudging, but it hogs all my CPU (97-98% consistent) and every once in a while I get green frames for long durations
c) Now Foscam allows me to set a sub stream as MJPEG, I did that, but it only returns 640x480. Not good
d) Foscam also allows us to directly get images from the camera at full res using http://ip address:port/cgi-bin/CGIProxy.fcgi?cmd=snapPicture2&usr=admin&pwd=xxx --> if I set ZM source as "remote" and use this URL instead, I'm consistently getting proper images (no transcoding) and after changing all my cameras to snap images at 2FPS, my CPU load hovers between 6-33% (higher when it detects motion)

So any reason why I should not be using option d? It seems to avoid transcoding (which seems buggy), there is ZERO artifacts when panning (yeah!), and it seems to consume less resources! (my camera does not seem to be clogged either - I can operate it from my mobile and desktop browser while ZM is grabbing images).

I can imagine why RTSP may be a better choice for high FPS live viewing, but ZM anyway recommends between 1-3 FPS as sufficient for recording, which is its main purpose (and I agree). So it seems to me, if all my cameras are in the same LAN, no switch or internet hopping, raw JPEG capture seems to be a win as far as ZM goes

[bbunge]

Use the ?tcp option and adjust the key frame rate at the camera to send a full image more often. Also lower the frame rate a bit if you can.

bb

[asker]

Thanks Bbunge,

On ?tcp, I think that would depend on the camera supporting that parameter and from what I've read, I don't believe Foscams support it. I am using the following workaround instead: Instead of using my internal IPs for monitoring, I am routing it through my ddns (public) IP and in my router port forwarding, I am only allowing TCP.

Also, last night, I got fed up of the smearing and reformatted my box, installed ubuntu server instead of desktop and installed ZM from github instead of iconnor PPAs. So far I am not seeing smearing. Its running since morning, so I'll keep a tab. Interestingly I have a theory that reducing frame rate may be causing pains to the transcoder - if the FPS was too fast, I would expect to see buffer overruns, which I never did. Anyway, I'll keep a tab

Edit: Missed your point on key frame rate --- AHA. Logical. I'll keep that in mind if I see that problem again.

(PS: The other idea of switching it to image snapshot mode bombed - I started getting many timeouts even at 1FPS...., so back to FF)

[asker]

Okay, so the smudging started again and I've been trolling other threads here and in the foscam forum. Some notes:

a) Adding ?tcp to the URL makes ZM go berserk - I get logs saying "Unable to open input rtsp://xxx:xxx@xxxxxxxx:20001/videoMain?tcp due to: Operation now in progress" for all my streams, and in a few minutes ZM DB stops function (too may socket opens). Plus, I only got a blue screen (Foscam 9831W)

b) For each camera, I've reduced in-camera frame rate to 10 and a key frame rate to 10 (from 30:30)
c) Inside ZM I've reduced FPS to 5

let's see what happens - everything works great for a while and then I get smudging after a while - so lets see

Any other tips?

[PacoLM]

Try setting up the maximum FPS in the ipcam web interface and leave the field as empty in your ZM box.

[asker]

Try setting up the maximum FPS in the ipcam web interface and leave the field as empty in your ZM box.

Thanks, I've set in-camera FPS to 5, and Key frame interval to 15. Kept FPS overrides blank at ZM. I'll let you know how it goes.
Out of curiosity, what is your thought process behind this change? Are you thinking by reducing it in-camera, the RTSP stream will be lesser from the camera resulting in lesser smear (packet drops, I think) as opposed to ZM force reducing it later?

[PacoLM]

Hi Asker,

I read the proposal many times in the forum. What I understand from my readings: If the ipcam has a fixed fps set (15, for instance), and ZM is told to capture only 10fps, there's a lack of sync in the images, so some of them are garbage.

I had the same problem in 2 cams of my setup and I solved using this.

Hope it helps!

PacoLM

[asker]

Hi PacoLM,
thank you! So far the results are extremely positive. I'll continue to monitor my monitors
I kept all my cameras in modect all through yesterday and I only found two frames with smearing - and that is in my basement which is farthest away from my router. I've reduced in camera FPS to 3 for that camera and key frame to 12 to see if it helps completely eliminate.

Overall very happy, PacoLM. You've saved me a lot of grief. I would have never guessed syncing issues between ZM FPS and camera FPS.

I really do think the ZM Wiki needs updating with many of these tips. Do they allow regular users to update? I'd be happy to update my findings and these tips.

Hi Asker,

I read the proposal many times in the forum. What I understand from my readings: If the ipcam has a fixed fps set (15, for instance), and ZM is told to capture only 10fps, there's a lack of sync in the images, so some of them are garbage.

I had the same problem in 2 cams of my setup and I solved using this.

Hope it helps!

PacoLM

[PacoLM]

Glad to know that my comments helped you. For the wiki, I believe you only need to be registered, so, please, go and update it!

Congratulations,
PacoLM

[asker]

Okay great! I've set up my own page here http://www.zoneminder.com/wiki/index.ph ... .04#Topics
Will keep updating as I find time

Glad to know that my comments helped you. For the wiki, I believe you only need to be registered, so, please, go and update it!

Congratulations,
PacoLM

[themagicm]

I have a Foscam 9805P and when viewing the image live in ZM, its very very slow. CPU usage is very low also. When you view your stream, is it fluid? Meaning, if a person walked by, is it fluid there or slow also?

[asker]

I have a Foscam 9805P and when viewing the image live in ZM, its very very slow. CPU usage is very low also. When you view your stream, is it fluid? Meaning, if a person walked by, is it fluid there or slow also?

Yes, its fluid for me (depending on FPS, though). When I had FPS of around 15-20, it was very fluid. I use FPS of 3 to manage CPU loads and even @ 3/fps its not slow. It's pretty real time but with choppy movements which is expected due to the low FPS.

How have you setup the cameras in ZM?

[themagicm]

General:
source: ffmpeg
function: monitor
enabled check
max fps 5
alarm fps blank


source: rtsp://<username>:<password>@<IP>:<PORT>/videoMain
remote method: rtp/unicast
options blank
target colorspace 32 bit color
capture width 1280
capture height 720

I get no image above. ZM is just too flakey. I might just have to stick with Blue Iris because it just works.

Errors:
Mar 31 16:41:31 a zms[22799]: ERR [Terminating, last frame sent time 1427838090.633596 secs more than maximum of 10.000000]
Mar 31 16:41:31 a zms[22799]: INF [Terminating Logger]
Mar 31 16:41:33 a web_php[20831]: ERR [Timed out waiting for msg /var/lib/zoneminder/sock/zms-347296s.sock]
Mar 31 16:41:33 a web_php[20830]: ERR [getStreamCmdResponse stream error: Timed out waiting for msg /var/lib/zoneminder/sock/zms-347296s.sock - checkStreamForErrors()]

[knight-of-ni]

Couple of follow up notes:

- For ip cameras, always set the frame rate in the camera, *not* in zoneminder. Trying to reduce the frame rate by putting values into the monitor max fps & alarm max fields is guaranteed not to work correctly.

- Yes, the WiKi is editable by anyone. Please help out by keeping it up to date.