ZM 1.25.0 Performance Patch - Completed

If you've made a patch to quick fix a bug or to add a new feature not yet in the main tree then post it here so others can try it out.
tibetfreedom
Posts: 163
Joined: Fri Oct 22, 2010 11:21 am

Re: ZM 1.25.0 Performance Patch - Completed

Post by tibetfreedom »

mastertheknife wrote:
tibetfreedom wrote:You are attempting to use a format that requires libjpeg-turbo, but ZoneMinder was not compiled against libjpeg-turbo. Make sure that libjpeg-turbo's header files are used, and not standard libjpeg's header files.

Please I can't find out how to enable this… I have installed libjpeg-turbo, but have not either compiled gainst it or enabled it in the lib conf where ever that is……

can you do a simple how to on this… ?

I have compiled with the following…

Many Many thanks, your distro is still loads better than the standard non patched…

./configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --datadir=/usr/share --sysconfdir=/etc --localstatedir=/var/lib --with-libarch=lib --with-mysql=/usr --with-ffmpeg=/usr --with-webdir=/var/www/zm --with-cgidir=/usr/lib/cgi-bin --with-webuser=www-data --with-webgroup=www-data --enable-debug=yes --enable-crashtrace=yes ZM_SSL_LIB=openssl CPPFLAGS="-D__STDC_CONSTANT_MACROS"
I will recompile again but no I downloaded the source and compiled.



Did you install libjpeg-turbo from the debian\ubuntu repository? if yes, then thats the issue. Their libjpeg-turbo package only provides the libraries, but not the header files.

mastertheknife
mastertheknife
Posts: 678
Joined: Wed Dec 16, 2009 4:32 pm
Location: Israel

Re: ZM 1.25.0 Performance Patch - Completed

Post by mastertheknife »

Hi all,

I have recently began working on adding common deinterlacing techniques to ZM and relocating the V4L2_CAPTURE_FIELDS option to be monitor specific.
You can see this here: http://i.imgur.com/sBXhR.jpg
The best deinterlacing technique so far is the four field motion adaptive one. This one compares the current frame with the next frame for motion and only applies linear deinterlacing to motion areas. So far it uses a lot of CPU and is not perfect, but i'm working on improving this.

Here is a picture of a bus captured by my PAL analog camera at 768x576 connected to a bt878 card:
Image

After deinterlacing (Four field motion adaptive):
Image

This should allow us all to use the full resolution of our analog cameras (640x480 for NTSC, 768x576 for PAL).

mastertheknife
Last edited by mastertheknife on Fri Feb 10, 2012 1:33 pm, edited 1 time in total.
Kfir Itzhak.
mastertheknife
Posts: 678
Joined: Wed Dec 16, 2009 4:32 pm
Location: Israel

Re: ZM 1.25.0 Performance Patch - Completed

Post by mastertheknife »

tibetfreedom,

What does this command show?

Code: Select all

cat /usr/include/jpeglib.h | grep EXTENSIONS
On my machine:

Code: Select all

kfir@GentooB0X ~ $ cat /usr/include/jpeglib.h | grep EXTENSIONS
#define JCS_EXTENSIONS 1
mastertheknife
Kfir Itzhak.
tibetfreedom
Posts: 163
Joined: Fri Oct 22, 2010 11:21 am

Re: ZM 1.25.0 Performance Patch - Completed

Post by tibetfreedom »

mastertheknife wrote:tibetfreedom,

What does this command show?

Code: Select all

cat /usr/include/jpeglib.h | grep EXTENSIONS
On my machine:

Code: Select all

kfir@GentooB0X ~ $ cat /usr/include/jpeglib.h | grep EXTENSIONS
#define JCS_EXTENSIONS 1
mastertheknife
root@CCTV:/home/cctvadmin# cat /usr/include/jpeglib.h | grep EXTENSIONS
root@CCTV:/home/cctvadmin# ln -s /opt/libjpeg-turbo/lib/libjpeg.a /usr/lib/libjpeg.a
ln: creating symbolic link `/usr/lib/libjpeg.a': File exists
root@CCTV:/home/cctvadmin# ln -s /opt/libjpeg-turbo/include/jconfig.h /usr/include/jconfig.h
ln: creating symbolic link `/usr/include/jconfig.h': File exists
root@CCTV:/home/cctvadmin# ln -s /opt/libjpeg-turbo/include/jerror.h /usr/include/jerror.h
ln: creating symbolic link `/usr/include/jerror.h': File exists
root@CCTV:/home/cctvadmin# ln -s /opt/libjpeg-turbo/include/jmorecfg.h /usr/include/jmorecfg.h
ln: creating symbolic link `/usr/include/jmorecfg.h': File exists
root@CCTV:/home/cctvadmin# ln -s /opt/libjpeg-turbo/include/jpeglib.h /usr/include/jpeglib.h
ln: creating symbolic link `/usr/include/jpeglib.h': File exists
root@CCTV:/home/cctvadmin# cat /usr/include/jpeglib.h | grep EXTENSIONS
root@CCTV:/home/cctvadmin#
pok
Posts: 9
Joined: Tue Feb 14, 2012 1:29 pm

Re: ZM 1.25.0 Performance Patch - Completed

Post by pok »

mastertheknife, thank your fo your`s Patch.

But i have some problem while use it...
When remote camera work as ffmpeg stream - all ok, but picture some times destroyed.
When remote camera work as rtsp source i have error "swscale does not support the codec format".

Seems trouble in "mFormatContext = rtspThread->getFormatContext();"
I have not more experience in cpp.

As i can see rtsp frame contents send to ffmpeg, so error "swscale does not support the codec format" is not correct.

Please, help me understand what happen in zm_remote_camera_rtsp.cpp.
mastertheknife
Posts: 678
Joined: Wed Dec 16, 2009 4:32 pm
Location: Israel

Re: ZM 1.25.0 Performance Patch - Completed

Post by mastertheknife »

tibetfreedom wrote:
mastertheknife wrote:tibetfreedom,

What does this command show?

Code: Select all

cat /usr/include/jpeglib.h | grep EXTENSIONS
On my machine:

Code: Select all

kfir@GentooB0X ~ $ cat /usr/include/jpeglib.h | grep EXTENSIONS
#define JCS_EXTENSIONS 1
mastertheknife
root@CCTV:/home/cctvadmin# cat /usr/include/jpeglib.h | grep EXTENSIONS
root@CCTV:/home/cctvadmin# ln -s /opt/libjpeg-turbo/lib/libjpeg.a /usr/lib/libjpeg.a
ln: creating symbolic link `/usr/lib/libjpeg.a': File exists
root@CCTV:/home/cctvadmin# ln -s /opt/libjpeg-turbo/include/jconfig.h /usr/include/jconfig.h
ln: creating symbolic link `/usr/include/jconfig.h': File exists
root@CCTV:/home/cctvadmin# ln -s /opt/libjpeg-turbo/include/jerror.h /usr/include/jerror.h
ln: creating symbolic link `/usr/include/jerror.h': File exists
root@CCTV:/home/cctvadmin# ln -s /opt/libjpeg-turbo/include/jmorecfg.h /usr/include/jmorecfg.h
ln: creating symbolic link `/usr/include/jmorecfg.h': File exists
root@CCTV:/home/cctvadmin# ln -s /opt/libjpeg-turbo/include/jpeglib.h /usr/include/jpeglib.h
ln: creating symbolic link `/usr/include/jpeglib.h': File exists
root@CCTV:/home/cctvadmin# cat /usr/include/jpeglib.h | grep EXTENSIONS
root@CCTV:/home/cctvadmin#
The problem is that you are using stock libjpeg's header files, instead of libjpeg-turbo's header files which include extensions.

What does this show?

Code: Select all

cat /opt/libjpeg-turbo/include/jpeglib.h | grep EXTENSIONS
If it shows the correct output, then do this:

Code: Select all

rm /usr/lib/libjpeg.a
rm /usr/include/jconfig.h
rm /usr/include/jerror.h
rm /usr/include/jmorecfg.h
rm /usr/include/jpeglib.h

Code: Select all

ln -s /opt/libjpeg-turbo/lib/libjpeg.a /usr/lib/libjpeg.a
ln -s /opt/libjpeg-turbo/include/jconfig.h /usr/include/jconfig.h
ln -s /opt/libjpeg-turbo/include/jerror.h /usr/include/jerror.h
ln -s /opt/libjpeg-turbo/include/jmorecfg.h /usr/include/jmorecfg.h
ln -s /opt/libjpeg-turbo/include/jpeglib.h /usr/include/jpeglib.h
Then recompile ZM (make) and reinstall ZM (make install) and hopefully it will work this time.
mastertheknife
Kfir Itzhak.
beerygaz
Posts: 28
Joined: Wed Jul 08, 2009 9:43 am
Location: Johannesburg, South Africa

Re: ZM 1.25.0 Performance Patch - Completed

Post by beerygaz »

Firstly I'd like to applaud your work and innovation. I'm really looking forward to the interlacing code, especially.

I compiled your source but can only get my cameras (local via bt878) to work in greyscale. If I select BGR24 (and 24-bit color) or BGR32 I get the following errors:

Code: Select all

Mar  3 08:28:37 dvr zms[8835]: ERR [Bogus input colorspace]
Mar  3 08:28:37 dvr abrt[8838]: saved core dump of pid 8835 (/data/www/cgi-bin/nph-zms) to /var/spool/abrt/ccpp-2012-03-03-08:28:37-8835.new/coredump (94789632bytes)
Mar  3 08:28:37 dvr abrtd: Directory 'ccpp-2012-03-03-08:28:37-8835' creation detected
Mar  3 08:28:38 dvr abrtd: Executable '/data/www/cgi-bin/nph-zms' doesn't belong to any package
Mar  3 08:28:38 dvr abrtd: Corrupted or bad dump /var/spool/abrt/ccpp-2012-03-03-08:28:37-8835 (res:2), deleting
Mar  3 08:28:39 dvr web_php[18346]: ERR [socket_sendto( /tmp/zm/zms-903499s.sock ) failed: Connection refused]
Mar  3 08:28:39 dvr auditd[1501]: Audit daemon rotating log files
Mar  3 08:28:42 dvr web_php[18345]: ERR [socket_sendto( /tmp/zm/zms-903499s.sock ) failed: Connection refused]
In addition, it seems zmtrigger.pl is not working. My zmtrigger.log shows it receiving events, but nothing happens in ZM (I send an on+30 event and the monitor remains idle).

Can you shed any light on what's going wrong please? I'd love to get my colour images back.

Gaz
mastertheknife
Posts: 678
Joined: Wed Dec 16, 2009 4:32 pm
Location: Israel

Re: ZM 1.25.0 Performance Patch - Completed

Post by mastertheknife »

beerygaz,

Thank you!

The reason you are seeing "Bogus input colorspace" is because standard libjpeg is being loaded, instead of libjpeg-turbo which is required for BGR24 or 32bit.
About zmtrigger.pl, thanks for reporting, i'll look into this.

mastertheknife
Kfir Itzhak.
mastertheknife
Posts: 678
Joined: Wed Dec 16, 2009 4:32 pm
Location: Israel

Re: ZM 1.25.0 Performance Patch - Completed

Post by mastertheknife »

pok wrote:mastertheknife, thank your fo your`s Patch.

But i have some problem while use it...
When remote camera work as ffmpeg stream - all ok, but picture some times destroyed.
When remote camera work as rtsp source i have error "swscale does not support the codec format".

Seems trouble in "mFormatContext = rtspThread->getFormatContext();"
I have not more experience in cpp.

As i can see rtsp frame contents send to ffmpeg, so error "swscale does not support the codec format" is not correct.

Please, help me understand what happen in zm_remote_camera_rtsp.cpp.
swscale is being used by ZoneMinder to convert from certain formats to formats that ZM can understand and work with (greyscale and RGB).

mastertheknife
Kfir Itzhak.
beerygaz
Posts: 28
Joined: Wed Jul 08, 2009 9:43 am
Location: Johannesburg, South Africa

Re: ZM 1.25.0 Performance Patch - Completed

Post by beerygaz »

mastertheknife wrote:beerygaz,

Thank you!

The reason you are seeing "Bogus input colorspace" is because standard libjpeg is being loaded, instead of libjpeg-turbo which is required for BGR24 or 32bit.
About zmtrigger.pl, thanks for reporting, i'll look into this.

mastertheknife
Cn you tell me how to comile libjpeg-turbo then please. I've followed as much as I can from this thread.

Code: Select all

[root@dvr zm]# cat /opt/libjpeg-turbo/include/jpeglib.h | grep EXTENSIONS
#define JCS_EXTENSIONS 1
#define JCS_ALPHA_EXTENSIONS 1
I followed the previous post of removing the existing libjepg files and creating the symbolic links. Then I re-ran configure and make / make install - but I'm still getting the same [Bogus input colorspace] error.

Am I missing a step?

Gaz
mastertheknife
Posts: 678
Joined: Wed Dec 16, 2009 4:32 pm
Location: Israel

Re: ZM 1.25.0 Performance Patch - Completed

Post by mastertheknife »

These commands simply take care of libjpeg-turbo's header files and the static library.
The correct libjpeg.so still needs to be loaded. There are few days to do this, the easiest one is to replace the standard libjpeg.so with the one from libjpeg-turbo's by copying /opt/libjpeg-turbo/lib into /usr/lib but this affects all applications on the system.

mastertheknife
Kfir Itzhak.
rtindle
Posts: 2
Joined: Fri Feb 24, 2012 11:13 pm

Re: ZM 1.25.0 Performance Patch - Completed

Post by rtindle »

I'm getting the following error during make:

Code: Select all

g++ -DHAVE_CONFIG_H -I. -I..  -I/usr/include -I/usr/include -Wall -Wno-sign-compare -finline-functions -I/usr/include   -D__STDC_CONSTANT_MACROS -DZM_STRIP_SSE -MT zm_ffmpeg_camera.o -MD -MP -MF .deps/zm_ffmpeg_camera.Tpo -c -o zm_ffmpeg_camera.o zm_ffmpeg_camera.cpp
zm_ffmpeg_camera.cpp: In member function âvirtual int FfmpegCamera::Capture(Image&)â:
zm_ffmpeg_camera.cpp:219: error: âSWS_CPU_CAPS_SSE2â was not declared in this scope
make[2]: *** [zm_ffmpeg_camera.o] Error 1
make[2]: Leaving directory `/home/rtindle/mastertheknife-ZoneMinder-kfir-8d30497/src'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/home/rtindle/mastertheknife-ZoneMinder-kfir-8d30497'
make: *** [all] Error 2
This is on a CentOS 6.2 ESXi VM; ffmpeg-0.6.5-1.el6.rf.i686/ffmpeg-devel-0.6.5-1.el6.rf.i686 along with libjpeg-turbo-1.2.0-20120209.i386 are installed.

Code: Select all

flags           : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss constant_tsc pebs bts tsc_reliable nonstop_tsc unfair_spinlock pni hypervisoru
mastertheknife
Posts: 678
Joined: Wed Dec 16, 2009 4:32 pm
Location: Israel

Re: ZM 1.25.0 Performance Patch - Completed

Post by mastertheknife »

pok,
I have updated the patch and the error now includes the unsupported format in the error. So please re-download the source and this should make troubleshooting easier.

rtindle,
In the update i have also added a workaround against this error. Please re-download the source and try again.

mastertheknife
Kfir Itzhak.
rtindle
Posts: 2
Joined: Fri Feb 24, 2012 11:13 pm

Re: ZM 1.25.0 Performance Patch - Completed

Post by rtindle »

mastertheknife wrote:rtindle,
In the update i have also added a workaround against this error. Please re-download the source and try again.
This works great; compiled and now running; thank you for all the help you've done here; this patch is very useful and already I can see an improvement in my setups performance.

One thing I had to do after the fact was fix the jpeg corruption issues my cameras (Foscam) are sending out. The fix is here:

http://www.zoneminder.com/forums/viewto ... 440#p73440

The corruption isn't a problem; but the spamming of the logs makes it hard to see real issues.
beerygaz
Posts: 28
Joined: Wed Jul 08, 2009 9:43 am
Location: Johannesburg, South Africa

Re: ZM 1.25.0 Performance Patch - Completed

Post by beerygaz »

mastertheknife wrote:These commands simply take care of libjpeg-turbo's header files and the static library.
The correct libjpeg.so still needs to be loaded. There are few days to do this, the easiest one is to replace the standard libjpeg.so with the one from libjpeg-turbo's by copying /opt/libjpeg-turbo/lib into /usr/lib but this affects all applications on the system.

mastertheknife
I just wanted to say thanks for the assistance. I have your patch up and running and ZM is working better than it ever has. RTSP streaming is working, fps rates are up, CPU is down. Great stuff.

Any ideas on why zmtrigger.pl is still not working as expected though?

Gaz
Post Reply