Ffmpeg and smearing issues

Forum for questions and support relating to the 1.30.x releases only.
pgrunwald
Posts: 71
Joined: Mon Mar 06, 2017 9:26 pm

Re: Ffmpeg and smearing issues

Post by pgrunwald »

p.s. I have been using ffmpeg with RTP/RSTP the whole time.

p.p.s. Of course, everything appears to be working fine today, I just checked the logs for the whole day.
daffodils
Posts: 1
Joined: Tue Mar 21, 2017 1:33 am

Re: Ffmpeg and smearing issues

Post by daffodils »

johnnytaco wrote:Hi -

First off - I **LOVE** ZoneMinder. I've really enjoyed installing, configuring, tweaking, and playing with it over the last month. It's been a great home project for me. It gets me back in Linux which is always fun... I plan to contribute somehow, if possible. I'm not much of a dev but I have been spreading the word to friends who are looking at security cam systems in their homes.

I have Reolink cameras and (at least) 2 of them give me smearing issues after a few days of running. Once smearing starts for a camera, it doesn't stop until I restart ZoneMinder or the camera itself. Then it takes at least a day or more for it to happen again. It seems that changing from FFMPEG (RTP/RTSP) to Remote is a fix (2 days in, at least), but I've read all over the forums not to use Remote as it will likely go away. Also, I get random "unexpected channel errors" in the logs for my Remote systems.

Notes:
ZoneMinder 1.30.0 (distro, not compiled, used the read-me/install doc and followed to the T)
/dev/shm size fine (using <50%)
CPU load under 2, often under 1 in a 4 core system
other disk space is fine
clean CentOS/minimal build, yum updated everything (including ffmpeg) (nothing else running to speak of)
TCP (not UDP) for FFMPEG
cameras are all hardwired, cat6, 1Gb POE switch
cameras do NOT allow keyframe changes so I can't adjust those
VLC and direct URL camera viewing are fine during the smearing issue w/ ZM
once smearing starts, it doesn't stop on its own and thus records motion events
all cams are MODECT with minimal zone sizes

I really don't want to get rid of my Reolink cameras - I have 5 of them and really like them (other than this issue). Does anyone have any ideas on what else I can look at, tweak, or do to fix? I suspect it's an FFMPEG bug or something between that and ZM, but I don't know how to prove it. Also the fact that it just happens on 2 of my cameras is really weird (and suspect). But since Remote seems to avoid (or hide) the issue, I'm further stumped.

Thank you.
jt
Last edited by daffodils on Mon Mar 27, 2017 3:53 am, edited 1 time in total.
johnnytaco
Posts: 26
Joined: Sun Feb 26, 2017 3:16 pm

Re: Ffmpeg and smearing issues

Post by johnnytaco »

Interesting... Mine are facing south and west and get a ton of sun (especially compared to my other 3 cameras). The one that smears the most, the 410, is facing west but has a ton of sun. I'm trying to recall when it starts smearing. I'm thinking mid-day when it's brightest. The 410 does it almost every few days, regardless. The 422's do it less often but they do it (well, 2 of them at least).

Does your 410 do this only on ffmpeg/rtsp/tcp? Have you ever seen it do it with Remote?

Once yours starts smearing, does it continue? Mine doesn't stop until I restart the camera (which seems to always work). Smearing artifacts, you name it.

So weird.
pgrunwald
Posts: 71
Joined: Mon Mar 06, 2017 9:26 pm

Re: Ffmpeg and smearing issues

Post by pgrunwald »

I have not used remote and it did not smear today as far as I can tell. Today was not as sunny as yesterday. I have only had the camera outside since Saturday. I had it running inside for a week and didn't see it.

I also sent an image to Reolink.
Last edited by pgrunwald on Tue Mar 21, 2017 5:07 am, edited 1 time in total.
johnnytaco
Posts: 26
Joined: Sun Feb 26, 2017 3:16 pm

Re: Ffmpeg and smearing issues

Post by johnnytaco »

SteveGilvarry wrote:You didn't mention changing the "remote method" option in the ffmpeg camera to RTP/RTSP, did you do that?
Yes - it has always been RTP/RTSP. :( I'll have a perfect image for hours, usually days, then the smearing starts and doesn't stop (only in ffmpeg/rtp/rtsp so far). Is there a way to see the exact ffmpeg call that is being made from ZM to see what options it's using? Happy to try others...

jt
johnnytaco
Posts: 26
Joined: Sun Feb 26, 2017 3:16 pm

Re: Ffmpeg and smearing issues

Post by johnnytaco »

I turned off "3D-NR" on the 3 cameras that have had the smearing issue to see if that's related. I've also read it could be something with the h264 profile (I have the cameras all at high, but there are main and baseline options too). Also there's a contrast option "anti-smearing" that is interesting... I just don't want to sacrifice image quality.

Funny, I've been reading about the Reolinks on other sites (Blue Iris mostly) and some people call them junk. I have a few Avigilon and Amcrest cameras and I think the Reolinks are just as good if not better for the money. Of course time will tell with firmware updates, support, and their longevity being outside, but we'll see. I could spend $200-$300+ on a camera ... but why? :)

See advanced options on the Reolink for grins:
Screen Shot 2017-03-21 at 12.09.08 AM.png
Screen Shot 2017-03-21 at 12.09.08 AM.png (69.99 KiB) Viewed 17965 times
jt
pgrunwald
Posts: 71
Joined: Mon Mar 06, 2017 9:26 pm

Re: Ffmpeg and smearing issues

Post by pgrunwald »

I should have mentioned it. I also played with "exposure". "Low-noise" allows gain changes, "anti-smearing" allows shutter changes, and manual is both. I think they are both upstream of the encoding like the rest of the image settings so I don't expect they would have any effect.
johnnytaco
Posts: 26
Joined: Sun Feb 26, 2017 3:16 pm

Re: Ffmpeg and smearing issues

Post by johnnytaco »

Yeah, makes sense. Did you tinker with the h264 profile? Though I still think that's not it, since again, Remote works for me. You should put a Remote monitor in for your offending camera so you have an ffmpeg and a remote one to watch. It'd be nice to know it's not just me...
pgrunwald
Posts: 71
Joined: Mon Mar 06, 2017 9:26 pm

Re: Ffmpeg and smearing issues

Post by pgrunwald »

what is the low stream snap command?

This gives me the high res: /cgi-bin/api.cgi?cmd=Snap&channel=0&rs=spam&user=admin&password=neverUmind

found her:
https://reolink.com/wp-content/uploads/ ... -v1.61.pdf
johnnytaco
Posts: 26
Joined: Sun Feb 26, 2017 3:16 pm

Re: Ffmpeg and smearing issues

Post by johnnytaco »

I had no idea you could even do snaps with this cam. LOL!
pgrunwald
Posts: 71
Joined: Mon Mar 06, 2017 9:26 pm

Re: Ffmpeg and smearing issues

Post by pgrunwald »

I had a "smear storm" today on both my low feeds.

I was able to clear it by switching each monitor from function "modect" to "none" and back again.

Restarting Zoneminder would not clear it. I could not boot my cameras remotely from work.

I noticed using "htop" that the "zmc - m n" process for each feed was locked to a core and almost completely pegged out at 100%. The process did not sleep the way they normally do.

The camera frame rates were very erratic. The feed itself is set to 7fps at the camera. ZM was showing 1.x -19fps, again, very erratic. when the feed is clear they are a very consistent 7.0 - 7.2fps.

The logs are basically clear during these storms, just showing continuous alarms.
pgrunwald
Posts: 71
Joined: Mon Mar 06, 2017 9:26 pm

Re: Ffmpeg and smearing issues

Post by pgrunwald »

https://github.com/ZoneMinder/ZoneMinder/issues/811

Are we not chasing this old problem?

I have been able to completely reinvent the wheel! Is figuring all of this out independently part of the Zoneminder club secret handshake? :roll: :wink:
johnnytaco
Posts: 26
Joined: Sun Feb 26, 2017 3:16 pm

Re: Ffmpeg and smearing issues

Post by johnnytaco »

Just read through that whole thread... that's all about UDP though, isn't it? I've been using TCP (I think) via RTP/RTSP all along. If I do RTP/Unicast, it gets worse for me (more artifacts, at least).

I had smearing on my 410 today. It started at 8:30AM. It was getting bright, but not nearly as bright as it gets hours later. I still have my Remote monitors alongside Ffmpeg.

You never responded to the h264 baseline/main/high profile on your Reolinks. What are you using? Did you try different settings? Also have you tried having a Remote and Ffmpeg identical monitor to see what happens?
pgrunwald
Posts: 71
Joined: Mon Mar 06, 2017 9:26 pm

Re: Ffmpeg and smearing issues

Post by pgrunwald »

On Monday I did play with the profiles but it didn't do anything. See earlier post.

I had smearing today when I restarted ZM. It would not clear until I powered off my POE switch... I also boosted wmem, rmem, and optmem_max all to 512k. No effect.

What are your source tab settings for "remote"? (I sent you a PM).
johnnytaco
Posts: 26
Joined: Sun Feb 26, 2017 3:16 pm

Re: Ffmpeg and smearing issues

Post by johnnytaco »

Remote settings are all the defaults, basically break up the URL you used in ffmpeg into the various fields. Just replied to your PM.

That sucks that you had to resort to restarting the switch. That's really weird.
Locked