ZM 1.25.0 Performance Patch - Completed

[Carrier82]

Now i've come up with another problem when using my Acti over RTSP (ffmpeg):

2012-08-26 23:29:19.869490 zmdc 28001 ERR 'zmc -m 6' exited abnormally, exit status 255 zmdc.pl
2012-08-26 23:29:19.852460 zmc_m6 28147 FAT You must compile ffmpeg with the --enable-swscale option to use ffmpeg cameras

I did recompile ffmpeg with --enable-swscale, but this has no effect. I wouldn't want to revert back to the non-optimized zoneminder... I configured ZM compilation with:
./configure --prefix=/opt/zm --with-libarch=lib64 --with-mysql=/usr --with-ffmpeg=/usr --with-webuser=www-data --with-webgroup=www-data --disable-debug --disable-crashtrace --enable-mmap=yes --with-webdir=/var/www --with-cgidir=/usr/lib/cgi-bin --with-ffmpeg --with-extralibs="`pkg-config --libs libavcodec libavdevice libavfilter libavformat libavutil libpostproc libswscale`" ZM_SSL_LIB=openssl

[Carrier82]

Tried hard to find a work-around for this but wasn't succesful. Apparently the function av_open_input_file is not supported by recent ffmpeg anymore and wasn't able to compile zoneminder performance patched version with the new ffmpeg I have installed. Reverting back to basic repository enabled ffmpeg functionality, as I haven't been able to use my Acti camera without ffmpeg. I have Ubuntu 12.04 32-bit.

[Carrier82]

Using basic zoneminder with motion detection with my high-res cameras jumped the CPU use on my poor atom to 100% so I had to retry. I removed the ffmpeg I had compiled and installed ffmpeg, libpostproc-dev, libavfilter-dev and libswscale-dev from repositories and compiled ZM performance version again with the aforementioned configure input. This time compilation was successful and ffmpeg works!

[McFuzz]

Wooo.... this looks quite promising... is there a guide that would be applicable for Debian Squeeze? I think some of the paths are different (I followed this guide - it'd need to be translated into English: http://www.zoneminder.com/wiki/index.ph ... mbre_2011) )

Thanks!

[2Psycho]

Dear All,

I discovered that at least one patch is not included into the build from mastertheknife.
This patch can be included into the modified code from mastertheknife.
This fixes the OPT_FRAME_SERVER in case it cannot capture a frame at one and has to read multiple times.
Regards,
Erik

[themagicm]

One thing I'm noticing is that when I go to a Timeline view and there is an event, if I mouse-over it, the preview is RED but an event with no alarm previews fine.

[ridwan]

haii mastertheknife.. how to upgrade zoneminder 1.250 to your new zm? and if i already update your zm our algorithm, may i to roll back to native zoneminder.. how i do it? thanks

sorry my english not good

[themagicm]

mastertheknife:

any clue as to why it goes red???? How do we roll back (thats what I want to do)?

Reinstall the good version?

[mobil1]

hello
my memory is

Code: Select all

             total       used       free     shared    buffers     cached
Mem:      32824236   32524936     299300          0      61984   25451128

my cpu is

Code: Select all

clflush size    : 64
cache_alignment : 64
address sizes   : 46 bits physical, 48 bits virtual
power management:

processor       : 15
vendor_id       : GenuineIntel
cpu family      : 6
model           : 45
model name      : Intel(R) Xeon(R) CPU E5-2650 0 @ 2.00GHz
stepping        : 7
cpu MHz         : 1995.044
cache size      : 20480 KB
physical id     : 0
siblings        : 16
core id         : 7
cpu cores       : 8
apicid          : 15
initial apicid  : 15
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 x2apic popcnt aes xsave avx lahf_lm ida arat epb xsaveopt pln pts dts tpr_shadow vnmi flexpriority ept vpid
bogomips        : 3990.08
clflush size    : 64
cache_alignment : 64
address sizes   : 46 bits physical, 48 bits virtual
power management:

my question is
will it help me your patch for 63 cameras that I stand now? and the memory is filled almost completely, although it lived up to 15 cameras at 2GB , so there should be about 8 GB , not 32 GB full. I use zm 1.24.4, and what advice would you give the value or shmmax shmmal.

[petekol]

thnx for the patch first -)

on my setup Athlon 4450e, Livestudio Webcam, 1080p i still get ~70% CPU for zmc usage in nodect mode independent if i use MJPEG or YUYV format. What can eat so much cpu power? In case of nodect/MJPEG zoneminder/zmc even does not need make any convertations at all.

[postfux]

Hello mastertheknif

I also use your performancepatch Version "mastertheknife-ZoneMinder-kfir-0ef2974.tar.gz", great work but anyway i had some little problems with it.

First: i got this error in the LOGs
'zmtrigger.pl ' exited abnormally, exit status 9
witch i could solve with disabling mmap with the

Code: Select all

./configure --enable-mmap=no

command found here:
http://www.zoneminder.com/forums/viewto ... 020#p76020
The error is gone, but now ZM uses arround 15% more CPU, this is for the moment OK
Mybe you have another idea how to fix this without disablig mmap. All other hints in the forum
didn't help.

Second: i got this erro in the LOGs
'zmaudit.pl -c' exited abnormally, exit status 9
when running from the commandline i got:

Code: Select all

directory (/tmp/) aptitude-root.9670:cpaCgf is still tainted at /usr/share/perl/5.14/File/Find.pm line 702.

this happend whenever aptitude was running or when exited with CTRL+C ( i know i sould not)

I found a hint and so changed the following line in the zmaudit.pl
From:

Code: Select all

File::Find::find( { wanted=>\&deleteSwapImage, untaint=>1 }, $swap_image_root );

To:

Code: Select all

File::Find::find( { wanted=>\&deleteSwapImage, untaint=>1 ,untaint_skip=>1 }, $swap_image_root );

With this modification the errors are gone and until now i had no problems anymore, but i have no idea if i broke something else or it has negative sideeffects.

Could you check this?

Greetings from Austria

PS:By the way i am using Debian wheezy and ffmpeg Version N-35287-g7076967 (with newer once i couldn't compile zm) if this is from interrest.

[knight-of-ni]

Thank you for all your hard work mastertheknife.

For anyone interested, I have created an RPM & SRPM of the latest kfir-proper branch here:
http://www.zoneminder.com/forums/viewto ... 428#p80428

[mastertheknife]

Thank you for all your hard work mastertheknife.

For anyone interested, I have created an RPM & SRPM of the latest kfir-proper branch here:
http://www.zoneminder.com/forums/viewto ... 428#p80428

Hi,
I'm a huge fan of RHEL\Centos (Got RHCE ), those are very good stable distros and having proper ZM RPMs and SRPMs for them is awesome. It makes ZM installation a piece of cake even for the under-average Joe.
Looking forward to your 1.26 RPMs!

[knight-of-ni]

Hi,
I'm a huge fan of RHEL\Centos (Got RHCE ), those are very good stable distros and having proper ZM RPMs and SRPMs for them is awesome. It makes ZM installation a piece of cake even for the under-average Joe.
Looking forward to your 1.26 RPMs!

Yeah, I too prefer the stability of RHEL/CentOS in a server environment. I admit though I use Ubuntu as my desktop OS.

Just a few days after creating an RPM from your performance patch, I discovered Keyljohnson's request to help test the upcoming 1.26 release. I have since plugged myself into the github branch to help out wherever help is needed.

I've got an RPM & SRPM of the latest 1.26 beta available for testing now:
http://www.zoneminder.com/forums/viewto ... 501#p80501

I could really use another set of eyes on these rpms! For one, I've done nothing to make it SELinux friendly.

[mastertheknife]

Yeah, I too prefer the stability of RHEL/CentOS in a server environment. I admit though I use Ubuntu as my desktop OS.

Just a few days after creating an RPM from your performance patch, I discovered Keyljohnson's request to help test the upcoming 1.26 release. I have since plugged myself into the github branch to help out wherever help is needed.

I've got an RPM & SRPM of the latest 1.26 beta available for testing now:
http://www.zoneminder.com/forums/viewto ... 501#p80501

I could really use another set of eyes on these rpms! For one, I've done nothing to make it SELinux friendly.

I also used to use Ubuntu as my desktop OS, but i am much more familiar with RHEL\Centos (especially with yum\rpm over apt\deb) so i am now a Fedora 19 user
You should give that a try, its great!

RHEL\CentOS\Fedora by default ship with selinux enabled, and it pretty much just works. Now and then need to change the default contexts for certain paths with semanage but other than that, its very good and there should be no reason to disable it just for ZM. We will just create a policy module for ZM. A policy module basically defines what our service is allowed to access (assuming it has the correct permissions to access in the first place) and what default access we would like to deny. Pretty much all policy modules in RHEL\CentOS for services at this moment only use the "type" field for controlling access, by having the services run on a special type that can only access what its allowed to access.

The first thing we need is to define types. I thought of least access model where we have type for everything, e.g. zm_exec_t for zm processes and scripts, zm_conf_t for config files such as zm.conf, zm_log_t for files in /var/log/zm, etc, but i believe that wont be possible because the CGI scripts won't be able to access those types because CGI scripts run as httpd_sys_script_exec_t. If we use another type for CGI scripts, apache won't be able to access those CGI files. And if we permit in the policy to allow access from httpd_sys_script_exec_t, every cgi script will be able to access ZM specific stuff, so no point.
So, we can use a minimal approach for now, just to be able to keep selinux in enforcing mode.
Types:
- All executables, including cgi scripts, should their default types. Same goes for logs and everything else.
- Events folder and other zm dynamic content that should be shared between multiple services (such as images/ folder which contains previews for zone editing) - should use public_content_rw_t because i can't think of any easy way of providing access to both ZM and apache.
Access:
- By default, all the default types already have access to other default types.
Ports:
- As far as i remember, ZM doesn't listen on any port, so no need for ports in the policy
Booleans:
- Cant think of any that we might want right now.

Not too difficult it seems