1.28.107 VLC segmentation fault.

Forum for questions and support relating to the 1.28.x releases only.
Locked
segordon
Posts: 37
Joined: Tue Aug 11, 2015 10:04 am

1.28.107 VLC segmentation fault.

Post by segordon »

New update went swimmingly, but now libvlc appears to be crashing.

ffmpeg works great, but I need libvlc in order to deal with smudging issues on my cheap china-cams.

Code: Select all

2015-10-16 02:33:44.278220	zmdc	1645	ERR	'zmc -m 1' exited abnormally, exit status 11	zmdc.pl	
2015-10-16 02:33:44.272463	zmc_m1	2025	INF	addr2line -e /usr/bin/zmc 0x7ffab192ca78 0x7ffab0b89d10 0x7ffab191b600 0x7ffab191b68e 0x7ffa89a080cf 0x7ffa89a085cb 0x7ffaa79a1365 0x7ffaa79a192e 0x7ffaa795639f 0x7ffaa79661b8 0x7ffaa7967843 0x7ffaa796b9f6 0x7ffab0b806aa 0x7ffaad68beed	zm_signal.cpp	113
2015-10-16 02:33:44.272326	zmc_m1	2025	INF	Backtrace complete, please execute the following command for more information	zm_signal.cpp	112
2015-10-16 02:33:44.272189	zmc_m1	2025	ERR	Backtrace 13: /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7ffaad68beed]	zm_signal.cpp	105
2015-10-16 02:33:44.272049	zmc_m1	2025	ERR	Backtrace 12: /lib/x86_64-linux-gnu/libpthread.so.0(+0x76aa) [0x7ffab0b806aa]	zm_signal.cpp	105
2015-10-16 02:33:44.271915	zmc_m1	2025	ERR	Backtrace 11: /usr/lib/libvlccore.so.8(+0x539f6) [0x7ffaa796b9f6]	zm_signal.cpp	105
2015-10-16 02:33:44.271778	zmc_m1	2025	ERR	Backtrace 10: /usr/lib/libvlccore.so.8(+0x4f843) [0x7ffaa7967843]	zm_signal.cpp	105
2015-10-16 02:33:44.271640	zmc_m1	2025	ERR	Backtrace 9: /usr/lib/libvlccore.so.8(+0x4e1b8) [0x7ffaa79661b8]	zm_signal.cpp	105
2015-10-16 02:33:44.271503	zmc_m1	2025	ERR	Backtrace 8: /usr/lib/libvlccore.so.8(+0x3e39f) [0x7ffaa795639f]	zm_signal.cpp	105
2015-10-16 02:33:44.271362	zmc_m1	2025	ERR	Backtrace 7: /usr/lib/libvlccore.so.8(vlc_module_load+0x47e) [0x7ffaa79a192e]	zm_signal.cpp	105
2015-10-16 02:33:44.271222	zmc_m1	2025	ERR	Backtrace 6: /usr/lib/libvlccore.so.8(+0x89365) [0x7ffaa79a1365]	zm_signal.cpp	105
2015-10-16 02:33:44.271081	zmc_m1	2025	ERR	Backtrace 5: /usr/lib/vlc/plugins/access/liblive555_plugin.so(+0x75cb) [0x7ffa89a085cb]	zm_signal.cpp	105
2015-10-16 02:33:44.270938	zmc_m1	2025	ERR	Backtrace 4: /usr/lib/vlc/plugins/access/liblive555_plugin.so(+0x70cf) [0x7ffa89a080cf]	zm_signal.cpp	105
2015-10-16 02:33:44.270662	zmc_m1	2025	ERR	Backtrace 3: /usr/bin/zmc(_ZN13AuthenticatorD1Ev+0x2e) [0x7ffab191b68e]	zm_signal.cpp	105
2015-10-16 02:33:44.270519	zmc_m1	2025	ERR	Backtrace 2: /usr/bin/zmc(_ZN13Authenticator5resetEv+0x10) [0x7ffab191b600]	zm_signal.cpp	105
2015-10-16 02:33:44.270235	zmc_m1	2025	ERR	Backtrace 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x10d10) [0x7ffab0b89d10]	zm_signal.cpp	105
2015-10-16 02:33:44.270042	zmc_m1	2025	ERR	Backtrace 0: /usr/bin/zmc(_Z14zm_die_handleriP9siginfo_tPv+0x78) [0x7ffab192ca78]	zm_signal.cpp	105
2015-10-16 02:33:44.269595	zmc_m1	2025	ERR	Signal address is 0xffffffffffffffe8, from 0x7ffab191b600	zm_signal.cpp	84
2015-10-16 02:33:44.267161	zmc_m1	2025	ERR	Got signal 11 (Segmentation fault), crashing	zm_signal.cpp	54
2015-10-16 02:33:44.106861	zmc_m1	2018	INF	Priming capture from rtsp://admin:admin@192.168.1.89:554/11/	zm_libvlc_camera.cpp	145
2015-10-16 02:33:44.104616	zmc_m1	2018	INF	Starting Capture version 1.28.107	zmc.cpp	250
2015-10-16 02:33:44.013530	zmdc	2018	INF	'zmc -m 1' started at 15/10/16 02:33:44	zmdc.pl	
2015-10-16 02:33:44.013530	zmdc	1645	INF	'zmc -m 1' starting at 15/10/16 02:33:44, pid = 2018	zmdc.pl	
2015-10-16 02:33:44.009890	zmdc	1645	INF	Starting pending process, zmc -m 1
addr2line gives little useful information

Code: Select all

addr2line -e /usr/bin/zmc 0x7ffab192ca78 0x7ffab0b89d10 0x7ffab191b600 0x7ffab191b68e 0x7ffa89a080cf 0x7ffa89a085cb 0x7ffaa79a1365 0x7ffaa79a192e 0x7ffaa795639f 0x7ffaa79661b8 0x7ffaa7967843 0x7ffaa796b9f6 0x7ffab0b806aa 0x7ffaad68beed
??:0
??:0
??:0
??:0
??:0
??:0
??:0
??:0
??:0
??:0
??:0
??:0
??:0
??:0
libvlccore8 is installed, and was working before the update to 1.28.107

Code: Select all

apt-cache policy libvlccore8
libvlccore8:
  Installed: 2.2.0-1
  Candidate: 2.2.0-1
  Version table:
 *** 2.2.0-1 0
        500 http://us.archive.ubuntu.com/ubuntu/ vivid/universe amd64 Packages
        100 /var/lib/dpkg/status
only thing I found that seemed related was this https://github.com/ZoneMinder/ZoneMinder/issues/343, but there were no solutions found as far as I could tell..

Any ideas appreciated.

(p.s. on a happy note, ffmpeg is now usable for me with the right settings. Would still like to resolve this vlc thing.)
bbunge
Posts: 2930
Joined: Mon Mar 26, 2012 11:40 am
Location: Pennsylvania

Re: 1.28.107 VLC segmentation fault.

Post by bbunge »

Did you install the vlc "extras?"

For ffmpeg there are things you can do to overcome smudging. Force TCP, shorten key frame interval in the camera, lower frame rate, lower resolution. for a few.

bb
User avatar
knight-of-ni
Posts: 2404
Joined: Thu Oct 18, 2007 1:55 pm
Location: Shiloh, IL

Re: 1.28.107 VLC segmentation fault.

Post by knight-of-ni »

That's probably the same issue as this one, which has existed since libvlc was first implemented:
https://github.com/ZoneMinder/ZoneMinder/issues/343

The fault is happening in the libvlc library.

No one knows for certain why the problem occurs. I am fairly certain is does not have anything to do with any specific version of ZoneMinder because, for each version, it works for some and doesn't for others.

It might have something to do with a particular combination of resolution, frame rate, i frame interval, or some other parameter.

In any case, we are looking for a volunteer to spend some time on this.
Visit my blog for ZoneMinder related projects using the Raspberry Pi, Orange Pi, Odroid, and the ESP8266
All of these can be found at https://zoneminder.blogspot.com/
segordon
Posts: 37
Joined: Tue Aug 11, 2015 10:04 am

Re: 1.28.107 VLC segmentation fault.

Post by segordon »

knnniggett wrote: The fault is happening in the libvlc library.

No one knows for certain why the problem occurs. I am fairly certain is does not have anything to do with any specific version of ZoneMinder because, for each version, it works for some and doesn't for others.

It might have something to do with a particular combination of resolution, frame rate, i frame interval, or some other parameter.
Yeah, I noticed that github issue too; was crossing my fingers that it was unrelated when I saw the date and bounty.

The thing is, I never changed any of my camera settings during the update; they were carried over from 1.28.2 (I think that was the previous iconnor/zoneminder version before the recent update?) directly into use with the iconnor/zoneminder-master ppa, using the procedure here: http://www.zoneminder.com/wiki/index.ph ... o_1.28.107,

and to remedy the issue all I did was flip from libvlc to ffmpeg on each monitor.

No on-camera settings changed. There are more options per monitor available in 1.28.107, so maybe one of the new options changed something?
bbunge wrote:Did you install the vlc "extras?"

For ffmpeg there are things you can do to overcome smudging. Force TCP, shorten key frame interval in the camera, lower frame rate, lower resolution. for a few.

bb
Yeah, the vlc extras are installed. This was a previously working ZM installation using libvlc for 6 monitors before the update to 1.28.107.

As for ffmpeg, i'm still trying to find a constellation of settings that works well for my particular cams while using it. I have gone through a lot of the stuff available for smudging/banding reduction, and the only settings that seem to work are ones that cripple the cameras to either a very low resolution, or a very low frame-rate. libvlc allowed me to utilize the cameras to their intended specifications.

I'd love to help someone with more experience track this issue down. If anyone decides to work on it and needs logs or data from someone affected, let me know!
User avatar
knight-of-ni
Posts: 2404
Joined: Thu Oct 18, 2007 1:55 pm
Location: Shiloh, IL

Re: 1.28.107 VLC segmentation fault.

Post by knight-of-ni »

and to remedy the issue all I did was flip from libvlc to ffmpeg on each monitor.
Right. That's because the fault is happening in the vlc libraries, not in ZoneMinder. Changing to ffmpeg uses the ffmpeg libraries, which does not experience this fault. It is possible that ZoneMinder is indirectly causing the fault, but no one has figure that out yet.
No on-camera settings changed. There are more options per monitor available in 1.28.107, so maybe one of the new options changed something?
That's what has been so frustrating about this. We did not make any libvlc changes following the release of 1.28.1.
The new options you see are not libvlc related.

Trying to duplicate the problem from one machine to another has been troublesome. One users' Linux Distro + version of zoneminder + libvlc libraries might work while another user's does not, even when the versions match exactly. This is why I think it might have something to do with incoming stream from the camera.
Visit my blog for ZoneMinder related projects using the Raspberry Pi, Orange Pi, Odroid, and the ESP8266
All of these can be found at https://zoneminder.blogspot.com/
McFuzz
Posts: 181
Joined: Tue Aug 28, 2012 7:03 am

Re: 1.28.107 VLC segmentation fault.

Post by McFuzz »

Hi all,

I just hit the same exact issue on my box - ubuntu 14.04 with 1.28.107 and an FI9821 v2.2 -- same exact error messages. I am stuck - libav has epic smearing, mjpeg has 4 fps capture and libvlc does not work :(

Any ideas/progress?

Thanks in advance.
bbunge
Posts: 2930
Joined: Mon Mar 26, 2012 11:40 am
Location: Pennsylvania

Re: 1.28.107 VLC segmentation fault.

Post by bbunge »

I have a FI9821W V2 on Ubuntu 14.04-3 ZM 1.28.107 running FFMPEG with no issues. Control even works! There are several posts in the forum where I've given settings and in teh WIKI for Foscam.

bb
goatman64
Posts: 31
Joined: Wed Oct 21, 2015 7:41 pm

Re: 1.28.107 VLC segmentation fault.

Post by goatman64 »

Same issue on Ubuntu 15.04, libvlc 2.2.0-1

Oct 20 06:46:14 mss zmc_m4[12166]: INF [Starting Capture version 1.28.107]
Oct 20 06:46:14 mss zmc_m4[12166]: INF [Priming capture from rtsp://cam4/live1.sdp]
Oct 20 06:46:14 mss zmc_m5[12165]: ERR [Got signal 11 (Segmentation fault), crashing]
Oct 20 06:46:14 mss zmc_m5[12165]: ERR [Signal address is 0xffffffffffffffe8, from 0x7f0d406585d0]
Oct 20 06:46:14 mss zmc_m5[12165]: ERR [Backtrace 0: /usr/bin/zmc(_Z14zm_die_handleriP9siginfo_tPv+0x78) [0x7f0d40669a48]]
Oct 20 06:46:14 mss zmc_m4[12166]: ERR [Got signal 11 (Segmentation fault), crashing]
Oct 20 06:46:14 mss zmc_m5[12165]: ERR [Backtrace 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x10d10) [0x7f0d3f8c6d10]]
Oct 20 06:46:14 mss zmc_m4[12166]: ERR [Signal address is 0xffffffffffffffe8, from 0x7f3dd42315d0]
Oct 20 06:46:14 mss zmc_m5[12165]: ERR [Backtrace 2: /usr/bin/zmc(_ZN13Authenticator5resetEv+0x10) [0x7f0d406585d0]]
Oct 20 06:46:14 mss zmc_m4[12166]: ERR [Backtrace 0: /usr/bin/zmc(_Z14zm_die_handleriP9siginfo_tPv+0x78) [0x7f3dd4242a48]]
Oct 20 06:46:14 mss zmc_m5[12165]: ERR [Backtrace 3: /usr/bin/zmc(_ZN13AuthenticatorD1Ev+0x2e) [0x7f0d4065865e]]
Oct 20 06:46:14 mss zmc_m4[12166]: ERR [Backtrace 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x10d10) [0x7f3dd349fd10]]
Oct 20 06:46:14 mss zmc_m5[12165]: ERR [Backtrace 4: /usr/lib/vlc/plugins/access/liblive555_plugin.so(+0x70cf) [0x7f0d1e01b0cf]]
Oct 20 06:46:14 mss zmc_m4[12166]: ERR [Backtrace 2: /usr/bin/zmc(_ZN13Authenticator5resetEv+0x10) [0x7f3dd42315d0]]
Oct 20 06:46:14 mss zmc_m5[12165]: ERR [Backtrace 5: /usr/lib/vlc/plugins/access/liblive555_plugin.so(+0x75cb) [0x7f0d1e01b5cb]]
Oct 20 06:46:14 mss zmc_m4[12166]: ERR [Backtrace 3: /usr/bin/zmc(_ZN13AuthenticatorD1Ev+0x2e) [0x7f3dd423165e]]
Oct 20 06:46:14 mss zmc_m5[12165]: ERR [Backtrace 6: /usr/lib/libvlccore.so.8(+0x89365) [0x7f0d366de365]]
Oct 20 06:46:14 mss zmc_m4[12166]: ERR [Backtrace 4: /usr/lib/vlc/plugins/access/liblive555_plugin.so(+0x70cf) [0x7f3db1c0f0cf]]
Oct 20 06:46:14 mss zmc_m5[12165]: ERR [Backtrace 7: /usr/lib/libvlccore.so.8(vlc_module_load+0x47e) [0x7f0d366de92e]]
Oct 20 06:46:14 mss zmc_m4[12166]: ERR [Backtrace 5: /usr/lib/vlc/plugins/access/liblive555_plugin.so(+0x75cb) [0x7f3db1c0f5cb]]
Oct 20 06:46:14 mss zmc_m5[12165]: ERR [Backtrace 8: /usr/lib/libvlccore.so.8(+0x3e39f) [0x7f0d3669339f]]
Oct 20 06:46:14 mss zmc_m4[12166]: ERR [Backtrace 6: /usr/lib/libvlccore.so.8(+0x89365) [0x7f3dca2b7365]]
Oct 20 06:46:14 mss zmc_m5[12165]: ERR [Backtrace 9: /usr/lib/libvlccore.so.8(+0x4e1b8) [0x7f0d366a31b8]]
Oct 20 06:46:14 mss zmc_m4[12166]: ERR [Backtrace 7: /usr/lib/libvlccore.so.8(vlc_module_load+0x47e) [0x7f3dca2b792e]]
Oct 20 06:46:14 mss zmc_m5[12165]: ERR [Backtrace 10: /usr/lib/libvlccore.so.8(+0x4f843) [0x7f0d366a4843]]
Oct 20 06:46:14 mss zmc_m4[12166]: ERR [Backtrace 8: /usr/lib/libvlccore.so.8(+0x3e39f) [0x7f3dca26c39f]]
Oct 20 06:46:14 mss zmc_m5[12165]: ERR [Backtrace 11: /usr/lib/libvlccore.so.8(+0x539f6) [0x7f0d366a89f6]]
Oct 20 06:46:14 mss zmc_m4[12166]: ERR [Backtrace 9: /usr/lib/libvlccore.so.8(+0x4e1b8) [0x7f3dca27c1b8]]
Oct 20 06:46:14 mss zmc_m5[12165]: ERR [Backtrace 12: /lib/x86_64-linux-gnu/libpthread.so.0(+0x76aa) [0x7f0d3f8bd6aa]]
Oct 20 06:46:14 mss zmc_m4[12166]: ERR [Backtrace 10: /usr/lib/libvlccore.so.8(+0x4f843) [0x7f3dca27d843]]
Oct 20 06:46:14 mss zmc_m5[12165]: ERR [Backtrace 13: /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f0d3c3c8eed]]
Oct 20 06:46:15 mss zmc_m4[12166]: ERR [Backtrace 11: /usr/lib/libvlccore.so.8(+0x539f6) [0x7f3dca2819f6]]
Oct 20 06:46:15 mss zmc_m5[12165]: INF [Backtrace complete, please execute the following command for more information]
Oct 20 06:46:15 mss zmc_m4[12166]: ERR [Backtrace 12: /lib/x86_64-linux-gnu/libpthread.so.0(+0x76aa) [0x7f3dd34966aa]]
Oct 20 06:46:15 mss zmc_m5[12165]: INF [addr2line -e /usr/bin/zmc 0x7f0d40669a48 0x7f0d3f8c6d10 0x7f0d406585d0 0x7f0d4065865e 0x7f0d1e01b0cf 0x7f0d
1e01b5cb 0x7f0d366de365 0x7f0d366de92e 0x7f0d3669339f 0x7f0d366a31b8 0x7f0d366a4843 0x7f0d366a89f6 0x7f0d3f8bd6aa 0x7f0d3c3c8eed]
Oct 20 06:46:15 mss zmdc[11622]: ERR ['zmc -m 5' exited abnormally, exit status 11]
Oct 20 06:46:15 mss zmc_m4[12166]: ERR [Backtrace 13: /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f3dcffa1eed]]
Oct 20 06:46:15 mss zmc_m4[12166]: INF [Backtrace complete, please execute the following command for more information]
Oct 20 06:46:15 mss zmc_m4[12166]: INF [addr2line -e /usr/bin/zmc 0x7f3dd4242a48 0x7f3dd349fd10 0x7f3dd42315d0 0x7f3dd423165e 0x7f3db1c0f0cf 0x7f3d
b1c0f5cb 0x7f3dca2b7365 0x7f3dca2b792e 0x7f3dca26c39f 0x7f3dca27c1b8 0x7f3dca27d843 0x7f3dca2819f6 0x7f3dd34966aa 0x7f3dcffa1eed]
Oct 20 06:46:15 mss zmdc[11622]: ERR ['zmc -m 4' exited abnormally, exit status 11]
McFuzz
Posts: 181
Joined: Tue Aug 28, 2012 7:03 am

Re: 1.28.107 VLC segmentation fault.

Post by McFuzz »

bbunge wrote:I have a FI9821W V2 on Ubuntu 14.04-3 ZM 1.28.107 running FFMPEG with no issues. Control even works! There are several posts in the forum where I've given settings and in teh WIKI for Foscam.

bb
unfortunately - even with the wiki settings i am getting smearing :(

http://i.imgur.com/Wg4c6i3.png


But - that is using libav. I want to try and completely move to ffmpeg and get rid of libav. Are there any good instructions out there to do that without breaking anything? :D
AnotherBrian
Posts: 112
Joined: Tue Jul 27, 2010 6:36 am

Re: 1.28.107 VLC segmentation fault.

Post by AnotherBrian »

i get this same problem under ubuntu 15.10 but was previously working under 15.04
c128
Posts: 88
Joined: Sat Sep 14, 2013 3:36 pm

Re: 1.28.107 VLC segmentation fault.

Post by c128 »

I have the same issue on 15.04 though:

http://www.zoneminder.com/forums/viewto ... 473#p89473

...the only change I made, from a previously "working fine with libvlc" installation, was upgrading my ZM master release to the later ZM version.

So, don't think it's an issue specific to an Ubuntu version, as such.
PacoLM
Posts: 971
Joined: Wed Dec 02, 2009 9:55 pm
Location: Spain

Re: 1.28.107 VLC segmentation fault.

Post by PacoLM »

Same thing -> http://www.zoneminder.com/forums/viewto ... 32&t=23563

It does not seems a weird combination of parameters, as knnniggett suggested
It might have something to do with a particular combination of resolution, frame rate, i frame interval, or some other parameter.
It was working fine with V1.28.1, started to fail in V1.28.107, so must be (slight) change in the way this last version deals with VlcLib connections.

Developers: let us know what kind of testing we need to do in order to help troubleshooting this, please.

Thanks in advance,

PacoLM

After more than 15 years, no longer using ZM as surveillance system.
Now in the dark side, using a commercial system...
SteveGilvarry
Posts: 494
Joined: Sun Jun 29, 2014 1:12 pm
Location: Melbourne, AU

Re: 1.28.107 VLC segmentation fault.

Post by SteveGilvarry »

Getting a debug version of libvlc looks like fun. Got some better output from Valgrind, but still nothing in liblive555 where it is occurring from the looks of this. I am scratching my head on why Authenticator is turning up in here. I will have to bring up a 1.28.1 version and confirm it is working and then swap to master with no other updates so see if maybe our inputs to libvlc have somehow changed, but it has to be from monitor or camera as livvlc_camera has not changed for ages.

Code: Select all

==26625== Invalid read of size 8
==26625==    at 0x9DA66B7: std::string::clear() (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19)
==26625==    by 0x4707C5: Authenticator::reset() (zm_rtsp_auth.cpp:50)
==26625==    by 0x470677: Authenticator::~Authenticator() (zm_rtsp_auth.cpp:46)
==26625==    by 0x20C45A6B: ??? (in /usr/lib/vlc/plugins/access/liblive555_plugin.so)
==26625==    by 0x20C45F59: ??? (in /usr/lib/vlc/plugins/access/liblive555_plugin.so)
==26625==    by 0xD1B4177: module_load (in /usr/lib/libvlccore.so.7.0.0)
==26625==    by 0xD1B472D: vlc_module_load (in /usr/lib/libvlccore.so.7.0.0)
==26625==    by 0xD174247: demux_New (in /usr/lib/libvlccore.so.7.0.0)
==26625==    by 0xD180B95: InputSourceInit (in /usr/lib/libvlccore.so.7.0.0)
==26625==    by 0xD181B6A: Init (in /usr/lib/libvlccore.so.7.0.0)
==26625==    by 0xD1850E5: Run (in /usr/lib/libvlccore.so.7.0.0)
==26625==    by 0x7605181: start_thread (pthread_create.c:312)
==26625==  Address 0xffffffffffffffe8 is not stack'd, malloc'd or (recently) free'd
Production Zoneminder 1.37.x (Living dangerously)
Random Selection of Cameras (Dahua and Hikvision)
SteveGilvarry
Posts: 494
Joined: Sun Jun 29, 2014 1:12 pm
Location: Melbourne, AU

Re: 1.28.107 VLC segmentation fault.

Post by SteveGilvarry »

Took a while but we have found the cause of this segfault. Libvlc and zm both have an Authenticator class so we had a conflict between them patch not in yet but have demonstrated a fix just doing a better job over the weekend.
Production Zoneminder 1.37.x (Living dangerously)
Random Selection of Cameras (Dahua and Hikvision)
Locked