Raspberry pi3 and h264_mmal

Forum for questions and support relating to the 1.29.x releases only.
cmisip
Posts: 39
Joined: Sun Apr 30, 2017 10:09 pm

Re: Raspberry pi3 and h264_mmal

Post by cmisip »

Managed to get master up and running and integrate the fixes for motion vector analysis.

The motion vector code required a few database commands to setup the Ffmpeghw Monitor type and the Mvdect Function. These need to be run by mysql root.

Code: Select all

ALTER TABLE Monitors modify column Type enum('Local','Remote','File','Ffmpeg','Ffmpeghw','Libvlc','cURL') NOT NULL default 'Local';
ALTER TABLE MonitorPresets modify column Type enum('Local','Remote','File','Ffmpeg','Ffmpeghw','Libvlc','cURL') NOT NULL default 'Local';
ALTER TABLE Controls modify column Type enum('Local','Remote','Ffmpeg','Ffmpeghw','Libvlc','cURL') NOT NULL default 'Local';
alter table Monitors modify column Function enum('None','Monitor','Modect','Mvdect','Record','Mocord','Nodect') NOT NULL default 'Monitor';
I can't figure out why the Function 'Mvdect' is appearing in the web UI as 'FnMvdect'.

I have yet to test Ffmpeghw Monitor. It is really only for RPI right now. I don't know of any other h264_mmal codec implementations in other devices that currently work with ffmpeg.

The motion vectors are only available for software decoding of h264 video using Ffmpeg Monitor. This works if the ffmpeg version supports the AVFrameSideData export flag (version 3.x and up).

So if the installed ffmpeg supports it, to use the motion vectors, set Monitor Source Type to Ffmpeg and Function to FnMvdect with your h264 camera.

The analysis is equivalent only to Alarm Pixels in Modect. I don't have any filtering code to ascertain clusters of motion vectors. I don't know if that is even necessary as there are a lot fewer motion vectors from Mvdect than alarm pixels from Modect.

The user can select the sensitivity of the MvDect function by setting Min/Max Alarmed Area in the Zone configuration UI. It is recommended to use 'Percent' for Units.

I figured that I could use that config setting instead of tackling how to introduce new config options to the database.

The analyser tries to save cpu cycles by analysing only if there are enough vectors to satisfy the minimum user specified score.

No guarantees.

Running on an x86, I get these results from zmsample.sh

Mvdect at idle.

Code: Select all

PIDSTAT zmc : Reporting average of 12 readings at 5 seconds intervals

ZMC Process ID : 773 ==> ( 6.60 + 7.00 + 6.60 + 6.80 + 6.40 + 6.80 + 7.00 + 7.00 + 6.20 + 6.80 + 6.80 + 6.40 ) / 12
AVERAGE : 6.70

PIDSTAT zma : Reporting average of 12 readings at 5 seconds intervals

ZMA Process ID : 781 ==> ( 1.40 + 1.40 + 1.40 + 1.40 + 1.20 + 1.40 + 1.40 + 1.40 + 1.00 + 1.60 + 1.20 + 1.40 ) / 12
AVERAGE : 1.35
Mvdect in action:

Code: Select all

PIDSTAT zmc : Reporting average of 12 readings at 5 seconds intervals

ZMC Process ID : 773 ==> ( 9.20 + 9.60 + 7.40 + 0.00 + 6.20 + 9.20 + 7.40 + 7.60 + 8.80 + 6.40 + 8.40 + 8.00 ) / 12 
AVERAGE : 8.18

PIDSTAT zma : Reporting average of 12 readings at 5 seconds intervals

ZMA Process ID : 24781 ==> ( 4.80 + 3.60 + 3.80 + 3.80 + 3.40 + 4.80 + 3.40 + 3.20 + 2.80 + 2.00 + 3.20 + 3.20 ) / 12 
AVERAGE : 3.50
Modect

Code: Select all

PIDSTAT zmc : Reporting average of 12 readings at 5 seconds intervals

ZMC Process ID : 773 ==> ( 6.80 + 6.60 + 6.60 + 7.00 + 7.20 + 6.80 + 7.19 + 6.60 + 6.80 + 7.20 + 6.60 + 7.20 ) / 12
AVERAGE : 6.88

PIDSTAT zma : Reporting average of 12 readings at 5 seconds intervals

ZMA Process ID : 16633 ==> ( 5.60 + 5.40 + 5.20 + 5.40 + 5.59 + 5.20 + 5.40 + 5.20 + 5.40 + 5.20 + 5.00 + 5.40 ) / 12
AVERAGE : 5.33

The code is at https://github.com/cmisip/ZoneMinder

Thanks,
Chris
cmisip
Posts: 39
Joined: Sun Apr 30, 2017 10:09 pm

Re: Raspberry pi3 and h264_mmal

Post by cmisip »

Getting this error on the neon stuff when trying to build a package on the raspberry pi 3.

Code: Select all

make[3]: Entering directory '/mnt/DISK1/build/ZoneMinder/obj-arm-linux-gnueabihf'
[  1%] Building CXX object src/CMakeFiles/zm.dir/zm_image.cpp.o
/mnt/DISK1/build/ZoneMinder/src/zm_image.cpp:3418:128: warning: target attribute is not supported on this machine [-Wattributes]
 void neon32_armv7_fastblend(const uint8_t* col1, const uint8_t* col2, uint8_t* result, unsigned long count, double blendpercent) {
                                                                                                                                ^
/mnt/DISK1/build/ZoneMinder/src/zm_image.cpp:3809:110: warning: target attribute is not supported on this machine [-Wattributes]
 void neon32_armv7_delta8_gray8(const uint8_t* col1, const uint8_t* col2, uint8_t* result, unsigned long count) {
                                                                                                              ^
/mnt/DISK1/build/ZoneMinder/src/zm_image.cpp:3885:131: warning: target attribute is not supported on this machine [-Wattributes]
 void neon32_armv7_delta8_rgb32(const uint8_t* col1, const uint8_t* col2, uint8_t* result, unsigned long count, uint32_t multiplier) {
                                                                                                                                   ^
{standard input}: Assembler messages:
{standard input}:364: Error: selected processor does not support ARM mode `vdup.8 q12,r12'
{standard input}:370: Error: selected processor does not support ARM mode `vrshl.u8 q8,q0,q12'
{standard input}:371: Error: selected processor does not support ARM mode `vrshl.u8 q9,q1,q12'
{standard input}:372: Error: selected processor does not support ARM mode `vrshl.u8 q10,q2,q12'
{standard input}:373: Error: selected processor does not support ARM mode `vrshl.u8 q11,q3,q12'
{standard input}:374: Error: selected processor does not support ARM mode `vrshl.u8 q4,q4,q12'
{standard input}:375: Error: selected processor does not support ARM mode `vrshl.u8 q5,q5,q12'
{standard input}:376: Error: selected processor does not support ARM mode `vrshl.u8 q6,q6,q12'
{standard input}:377: Error: selected processor does not support ARM mode `vrshl.u8 q7,q7,q12'
{standard input}:378: Error: selected FPU does not support instruction -- `vsub.i8 q4,q4,q8'
{standard input}:379: Error: selected FPU does not support instruction -- `vsub.i8 q5,q5,q9'
{standard input}:380: Error: selected FPU does not support instruction -- `vsub.i8 q6,q6,q10'
{standard input}:381: Error: selected FPU does not support instruction -- `vsub.i8 q7,q7,q11'
{standard input}:382: Error: selected FPU does not support instruction -- `vadd.i8 q4,q4,q0'
{standard input}:383: Error: selected FPU does not support instruction -- `vadd.i8 q5,q5,q1'
{standard input}:384: Error: selected FPU does not support instruction -- `vadd.i8 q6,q6,q2'
{standard input}:385: Error: selected FPU does not support instruction -- `vadd.i8 q7,q7,q3'
{standard input}:1878: Error: selected processor does not support ARM mode `vabd.u8 q0,q0,q4'
{standard input}:1879: Error: selected processor does not support ARM mode `vabd.u8 q1,q1,q5'
{standard input}:1880: Error: selected processor does not support ARM mode `vabd.u8 q2,q2,q6'
{standard input}:1881: Error: selected processor does not support ARM mode `vabd.u8 q3,q3,q7'
{standard input}:33683: Error: selected processor does not support ARM mode `vdup.32 q8,r12'
{standard input}:33689: Error: selected processor does not support ARM mode `vabd.u8 q0,q0,q4'
{standard input}:33690: Error: selected processor does not support ARM mode `vabd.u8 q1,q1,q5'
{standard input}:33691: Error: selected processor does not support ARM mode `vabd.u8 q2,q2,q6'
{standard input}:33692: Error: selected processor does not support ARM mode `vabd.u8 q3,q3,q7'
{standard input}:33693: Error: selected processor does not support ARM mode `vrshr.u8 q0,q0,#3'
{standard input}:33694: Error: selected processor does not support ARM mode `vrshr.u8 q1,q1,#3'
{standard input}:33695: Error: selected processor does not support ARM mode `vrshr.u8 q2,q2,#3'
{standard input}:33696: Error: selected processor does not support ARM mode `vrshr.u8 q3,q3,#3'
{standard input}:33697: Error: selected FPU does not support instruction -- `vmul.i8 q0,q0,q8'
{standard input}:33698: Error: selected FPU does not support instruction -- `vmul.i8 q1,q1,q8'
{standard input}:33699: Error: selected FPU does not support instruction -- `vmul.i8 q2,q2,q8'
{standard input}:33700: Error: selected FPU does not support instruction -- `vmul.i8 q3,q3,q8'
{standard input}:33701: Error: selected processor does not support ARM mode `vpadd.i8 d0,d0,d1'
{standard input}:33702: Error: selected processor does not support ARM mode `vpadd.i8 d2,d2,d3'
{standard input}:33703: Error: selected processor does not support ARM mode `vpadd.i8 d4,d4,d5'
{standard input}:33704: Error: selected processor does not support ARM mode `vpadd.i8 d6,d6,d7'
{standard input}:33705: Error: selected processor does not support ARM mode `vpadd.i8 d0,d0,d0'
{standard input}:33706: Error: selected processor does not support ARM mode `vpadd.i8 d1,d2,d2'
{standard input}:33707: Error: selected processor does not support ARM mode `vpadd.i8 d2,d4,d4'
{standard input}:33708: Error: selected processor does not support ARM mode `vpadd.i8 d3,d6,d6'
{standard input}:33709: Error: selected processor does not support ARM mode `vst4.32 {d0[0],d1[0],d2[0],d3[0]},[r2]!'
src/CMakeFiles/zm.dir/build.make:398: recipe for target 'src/CMakeFiles/zm.dir/zm_image.cpp.o' failed
make[3]: *** [src/CMakeFiles/zm.dir/zm_image.cpp.o] Error 1
make[3]: Leaving directory '/mnt/DISK1/build/ZoneMinder/obj-arm-linux-gnueabihf'
CMakeFiles/Makefile2:176: recipe for target 'src/CMakeFiles/zm.dir/all' failed
make[2]: *** [src/CMakeFiles/zm.dir/all] Error 2
make[2]: Leaving directory '/mnt/DISK1/build/ZoneMinder/obj-arm-linux-gnueabihf'
Makefile:127: recipe for target 'all' failed
make[1]: *** [all] Error 2
make[1]: Leaving directory '/mnt/DISK1/build/ZoneMinder/obj-arm-linux-gnueabihf'
dh_auto_build: make -j1 returned exit code 2
debian/rules:45: recipe for target 'build' failed
make: *** [build] Error 2
dpkg-buildpackage: error: debian/rules build gave error exit status 2
/proc/cpuinfo shows 4 processors with features:

Code: Select all

processor	: 0
model name	: ARMv7 Processor rev 4 (v7l)
BogoMIPS	: 38.40
Features	: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32 
CPU implementer	: 0x41
CPU architecture: 7
CPU variant	: 0x0
CPU part	: 0xd03
CPU revision	: 4
uname -a shows

Code: Select all

pi@raspberrypi:/mnt/DISK1/build/ZoneMinder $ uname -a
Linux raspberrypi 4.9.34-v7+ #1013 SMP Sun Jun 25 17:06:40 BST 2017 armv7l GNU/Linux
Thanks,
Chris
User avatar
iconnor
Posts: 2862
Joined: Fri Oct 29, 2010 1:43 am
Location: Toronto
Contact:

Re: Raspberry pi3 and h264_mmal

Post by iconnor »

Good work.

I havn't tried to build master on a pi in a while, but I will try to do so today.
cmisip
Posts: 39
Joined: Sun Apr 30, 2017 10:09 pm

Re: Raspberry pi3 and h264_mmal

Post by cmisip »

There's a glimmer of hope that motion vector processing will work on h264 hardware decode as well. I managed to get a successful handoff from ffmpeg hardware decode to rpi mmal encoder in a test code. It is also quite possible to setup a native mmal decoder. I will try on the first and see where it leads.

Chris
cmisip
Posts: 39
Joined: Sun Apr 30, 2017 10:09 pm

Re: Raspberry pi3 and h264_mmal

Post by cmisip »

I got motion vectors from the hardware MMAL encoder. I just finished a testable version tonight. I am running tests on the system. Right now I am running 6 cameras running at 640x360 resolution with the first three using the hardware MMAL decoder and encoder ( for the vectors ) and the last three running software h264 decoding and using the motion vector from the side data.

There is no more a mvdect idle mode. I had to come up with a common motion_vector struct for both cases that saved more data that can be processed in monitor::analyse in case somebody wants to write a second tier (such as testing for vector clusters ) or third tier test ( such as testing for similar directions of vectors ). The size of the macroblock is saved in the motion_vector struct instead of being decomposed into 4x4 blocks in monitor::capture, so I cant find out how many 4x4 vectors there are until I actually monitor::analyse.

There seem to be a hardware limit of 3 cameras at 640x360. Probably less cameras with higher resolution.

Here are the stats as far as cpu utilization:

Code: Select all

PIDSTAT zmc : Reporting average of 12 readings at 5 seconds intervals

ZMC Process ID : 7442 ==> ( 16.40 + 16.60 + 15.80 + 16.40 + 16.20 + 16.00 + 15.60 + 16.60 + 16.40 + 16.60 + 16.20 + 16.20 ) / 12 
AVERAGE : 16.25

ZMC Process ID : 13493 ==> ( 16.60 + 16.20 + 16.60 + 17.00 + 16.80 + 16.60 + 16.80 + 16.00 + 17.40 + 17.00 + 17.00 + 16.80 ) / 12 
AVERAGE : 16.73

ZMC Process ID : 18485 ==> ( 10.00 + 10.20 + 10.20 + 10.00 + 10.20 + 10.00 + 10.60 + 10.20 + 10.20 + 10.80 + 9.60 + 10.00 ) / 12 
AVERAGE : 10.17

ZMC Process ID : 22481 ==> ( 20.40 + 22.00 + 20.80 + 22.20 + 20.40 + 22.20 + 20.60 + 19.20 + 22.40 + 21.60 + 21.40 + 22.00 ) / 12 
AVERAGE : 21.27

ZMC Process ID : 25491 ==> ( 21.40 + 22.20 + 20.40 + 20.60 + 22.80 + 21.60 + 21.40 + 21.40 + 22.60 + 21.20 + 22.20 + 21.40 ) / 12 
AVERAGE : 21.60

ZMC Process ID : 28308 ==> ( 10.20 + 10.40 + 10.20 + 9.80 + 10.60 + 9.78 + 10.62 + 9.60 + 10.38 + 10.00 + 10.40 + 9.80 ) / 12 
AVERAGE : 10.15

PIDSTAT zma : Reporting average of 12 readings at 5 seconds intervals

ZMA Process ID : 7305 ==> ( 8.00 + 8.00 + 8.00 + 7.80 + 8.20 + 7.80 + 8.20 + 8.20 + 8.20 + 8.60 + 8.00 + 8.60 ) / 12 
AVERAGE : 8.13

ZMA Process ID : 18899 ==> ( 8.37 + 8.60 + 8.00 + 7.80 + 8.40 + 8.20 + 8.40 + 8.40 + 8.20 + 8.00 + 8.40 + 8.60 ) / 12 
AVERAGE : 8.28

ZMA Process ID : 18924 ==> ( 14.77 + 13.60 + 14.20 + 13.80 + 14.40 + 14.80 + 14.40 + 14.00 + 14.60 + 15.20 + 14.80 + 13.60 ) / 12 
AVERAGE : 14.35

ZMA Process ID : 20888 ==> ( 8.20 + 8.40 + 8.40 + 7.80 + 8.20 + 8.20 + 8.00 + 8.40 + 8.60 + 8.00 + 8.40 + 8.00 ) / 12 
AVERAGE : 8.22

ZMA Process ID : 28340 ==> ( 14.37 + 13.80 + 14.00 + 14.00 + 14.00 + 14.00 + 14.20 + 14.00 + 14.00 + 14.40 + 14.20 + 14.20 ) / 12 
AVERAGE : 14.10

ZMA Process ID : 31602 ==> ( 8.40 + 8.00 + 7.60 + 8.20 + 8.00 + 8.00 + 8.20 + 8.20 + 8.00 + 8.00 + 7.80 + 8.40 ) / 12 
AVERAGE : 8.07

Here is a snapshot of the Log

Code: Select all

Jul 11 19:38:32 raspberrypi zma[7305]: INF [zma_m6] [Monitor-6: 14000 - Analysing at 14.29 fps]
Jul 11 19:38:37 raspberrypi zmc[13493]: INF [zmc_m2] [Monitor-2: 85000 - Capturing at 25.00 fps]
Jul 11 19:38:50 raspberrypi zma[18899]: INF [zma_m1] [Monitor-1: 62000 - Analysing at 14.29 fps]
Jul 11 19:38:55 raspberrypi zma[28340]: INF [zma_m2] [Monitor-2: 164000 - Analysing at 25.00 fps]
Jul 11 19:38:59 raspberrypi zmc[25491]: INF [zmc_m5] [Monitor-5: 114000 - Capturing at 25.00 fps]
Jul 11 19:38:59 raspberrypi zmc[18485]: INF [zmc_m3] [Monitor-3: 125000 - Capturing at 14.29 fps]
Jul 11 19:38:59 raspberrypi zmc[28308]: INF [zmc_m1] [Monitor-1: 155000 - Capturing at 14.29 fps]
Jul 11 19:39:00 raspberrypi zma[20888]: INF [zma_m3] [Monitor-3: 118000 - Analysing at 14.29 fps]
Jul 11 19:39:02 raspberrypi zma[18924]: INF [zma_m5] [Monitor-5: 70000 - Analysing at 25.64 fps]
Jul 11 19:39:04 raspberrypi zmc[22481]: INF [zmc_m4] [Monitor-4: 79000 - Capturing at 14.29 fps]
Jul 11 19:38:32 raspberrypi zma[7305]: INF [zma_m6] [Monitor-6: 14000 - Analysing at 14.29 fps]
Jul 11 19:38:37 raspberrypi zmc[13493]: INF [zmc_m2] [Monitor-2: 85000 - Capturing at 25.00 fps]
Jul 11 19:38:50 raspberrypi zma[18899]: INF [zma_m1] [Monitor-1: 62000 - Analysing at 14.29 fps]
Jul 11 19:38:55 raspberrypi zma[28340]: INF [zma_m2] [Monitor-2: 164000 - Analysing at 25.00 fps]
Jul 11 19:38:59 raspberrypi zmc[25491]: INF [zmc_m5] [Monitor-5: 114000 - Capturing at 25.00 fps]
Jul 11 19:38:59 raspberrypi zmc[18485]: INF [zmc_m3] [Monitor-3: 125000 - Capturing at 14.29 fps]
Jul 11 19:38:59 raspberrypi zmc[28308]: INF [zmc_m1] [Monitor-1: 155000 - Capturing at 14.29 fps]
Jul 11 19:39:00 raspberrypi zma[20888]: INF [zma_m3] [Monitor-3: 118000 - Analysing at 14.29 fps]
Jul 11 19:39:02 raspberrypi zma[18924]: INF [zma_m5] [Monitor-5: 70000 - Analysing at 25.64 fps]
Jul 11 19:39:04 raspberrypi zmc[22481]: INF [zmc_m4] [Monitor-4: 79000 - Capturing at 14.29 fps]
Jul 11 19:38:32 raspberrypi zma[7305]: INF [zma_m6] [Monitor-6: 14000 - Analysing at 14.29 fps]
Jul 11 19:38:37 raspberrypi zmc[13493]: INF [zmc_m2] [Monitor-2: 85000 - Capturing at 25.00 fps]
Jul 11 19:38:50 raspberrypi zma[18899]: INF [zma_m1] [Monitor-1: 62000 - Analysing at 14.29 fps]
Jul 11 19:38:55 raspberrypi zma[28340]: INF [zma_m2] [Monitor-2: 164000 - Analysing at 25.00 fps]
Jul 11 19:38:59 raspberrypi zmc[25491]: INF [zmc_m5] [Monitor-5: 114000 - Capturing at 25.00 fps]
Jul 11 19:38:59 raspberrypi zmc[18485]: INF [zmc_m3] [Monitor-3: 125000 - Capturing at 14.29 fps]
Jul 11 19:38:59 raspberrypi zmc[28308]: INF [zmc_m1] [Monitor-1: 155000 - Capturing at 14.29 fps]
Jul 11 19:39:00 raspberrypi zma[20888]: INF [zma_m3] [Monitor-3: 118000 - Analysing at 14.29 fps]
Jul 11 19:39:02 raspberrypi zma[18924]: INF [zma_m5] [Monitor-5: 70000 - Analysing at 25.64 fps]
Jul 11 19:39:04 raspberrypi zmc[22481]: INF [zmc_m4] [Monitor-4: 79000 - Capturing at 14.29 fps]
Jul 11 19:38:32 raspberrypi zma[7305]: INF [zma_m6] [Monitor-6: 14000 - Analysing at 14.29 fps]
Jul 11 19:38:37 raspberrypi zmc[13493]: INF [zmc_m2] [Monitor-2: 85000 - Capturing at 25.00 fps]
Jul 11 19:38:50 raspberrypi zma[18899]: INF [zma_m1] [Monitor-1: 62000 - Analysing at 14.29 fps]
Jul 11 19:38:55 raspberrypi zma[28340]: INF [zma_m2] [Monitor-2: 164000 - Analysing at 25.00 fps]
Jul 11 19:38:59 raspberrypi zmc[25491]: INF [zmc_m5] [Monitor-5: 114000 - Capturing at 25.00 fps]
Jul 11 19:38:59 raspberrypi zmc[18485]: INF [zmc_m3] [Monitor-3: 125000 - Capturing at 14.29 fps]
Jul 11 19:38:59 raspberrypi zmc[28308]: INF [zmc_m1] [Monitor-1: 155000 - Capturing at 14.29 fps]
Jul 11 19:39:00 raspberrypi zma[20888]: INF [zma_m3] [Monitor-3: 118000 - Analysing at 14.29 fps]
Jul 11 19:39:02 raspberrypi zma[18924]: INF [zma_m5] [Monitor-5: 70000 - Analysing at 25.64 fps]
Jul 11 19:39:04 raspberrypi zmc[22481]: INF [zmc_m4] [Monitor-4: 79000 - Capturing at 14.29 fps]
Jul 11 19:38:32 raspberrypi zma[7305]: INF [zma_m6] [Monitor-6: 14000 - Analysing at 14.29 fps]
Jul 11 19:38:37 raspberrypi zmc[13493]: INF [zmc_m2] [Monitor-2: 85000 - Capturing at 25.00 fps]
Jul 11 19:38:50 raspberrypi zma[18899]: INF [zma_m1] [Monitor-1: 62000 - Analysing at 14.29 fps]
Jul 11 19:38:55 raspberrypi zma[28340]: INF [zma_m2] [Monitor-2: 164000 - Analysing at 25.00 fps]
Jul 11 19:38:59 raspberrypi zmc[25491]: INF [zmc_m5] [Monitor-5: 114000 - Capturing at 25.00 fps]
Jul 11 19:38:59 raspberrypi zmc[18485]: INF [zmc_m3] [Monitor-3: 125000 - Capturing at 14.29 fps]
Jul 11 19:38:59 raspberrypi zmc[28308]: INF [zmc_m1] [Monitor-1: 155000 - Capturing at 14.29 fps]
Jul 11 19:39:00 raspberrypi zma[20888]: INF [zma_m3] [Monitor-3: 118000 - Analysing at 14.29 fps]
Jul 11 19:39:02 raspberrypi zma[18924]: INF [zma_m5] [Monitor-5: 70000 - Analysing at 25.64 fps]
Jul 11 19:39:04 raspberrypi zmc[22481]: INF [zmc_m4] [Monitor-4: 79000 - Capturing at 14.29 fps]
Jul 11 19:38:32 raspberrypi zma[7305]: INF [zma_m6] [Monitor-6: 14000 - Analysing at 14.29 fps]
Jul 11 19:38:37 raspberrypi zmc[13493]: INF [zmc_m2] [Monitor-2: 85000 - Capturing at 25.00 fps]
Jul 11 19:38:50 raspberrypi zma[18899]: INF [zma_m1] [Monitor-1: 62000 - Analysing at 14.29 fps]
Jul 11 19:38:55 raspberrypi zma[28340]: INF [zma_m2] [Monitor-2: 164000 - Analysing at 25.00 fps]
Jul 11 19:38:59 raspberrypi zmc[25491]: INF [zmc_m5] [Monitor-5: 114000 - Capturing at 25.00 fps]
Jul 11 19:38:59 raspberrypi zmc[18485]: INF [zmc_m3] [Monitor-3: 125000 - Capturing at 14.29 fps]
Jul 11 19:38:59 raspberrypi zmc[28308]: INF [zmc_m1] [Monitor-1: 155000 - Capturing at 14.29 fps]
Jul 11 19:39:00 raspberrypi zma[20888]: INF [zma_m3] [Monitor-3: 118000 - Analysing at 14.29 fps]
Jul 11 19:39:02 raspberrypi zma[18924]: INF [zma_m5] [Monitor-5: 70000 - Analysing at 25.64 fps]
Jul 11 19:39:04 raspberrypi zmc[22481]: INF [zmc_m4] [Monitor-4: 79000 - Capturing at 14.29 fps]
Jul 11 19:38:32 raspberrypi zma[7305]: INF [zma_m6] [Monitor-6: 14000 - Analysing at 14.29 fps]
Jul 11 19:38:37 raspberrypi zmc[13493]: INF [zmc_m2] [Monitor-2: 85000 - Capturing at 25.00 fps]
Jul 11 19:38:50 raspberrypi zma[18899]: INF [zma_m1] [Monitor-1: 62000 - Analysing at 14.29 fps]
Jul 11 19:38:55 raspberrypi zma[28340]: INF [zma_m2] [Monitor-2: 164000 - Analysing at 25.00 fps]
Jul 11 19:38:59 raspberrypi zmc[25491]: INF [zmc_m5] [Monitor-5: 114000 - Capturing at 25.00 fps]
Jul 11 19:38:59 raspberrypi zmc[18485]: INF [zmc_m3] [Monitor-3: 125000 - Capturing at 14.29 fps]
Jul 11 19:38:59 raspberrypi zmc[28308]: INF [zmc_m1] [Monitor-1: 155000 - Capturing at 14.29 fps]
Jul 11 19:39:00 raspberrypi zma[20888]: INF [zma_m3] [Monitor-3: 118000 - Analysing at 14.29 fps]
Jul 11 19:39:02 raspberrypi zma[18924]: INF [zma_m5] [Monitor-5: 70000 - Analysing at 25.64 fps]
Jul 11 19:39:04 raspberrypi zmc[22481]: INF [zmc_m4] [Monitor-4: 79000 - Capturing at 14.29 fps]
Jul 11 19:38:32 raspberrypi zma[7305]: INF [zma_m6] [Monitor-6: 14000 - Analysing at 14.29 fps]
Jul 11 19:38:37 raspberrypi zmc[13493]: INF [zmc_m2] [Monitor-2: 85000 - Capturing at 25.00 fps]
Jul 11 19:38:50 raspberrypi zma[18899]: INF [zma_m1] [Monitor-1: 62000 - Analysing at 14.29 fps]
Jul 11 19:38:55 raspberrypi zma[28340]: INF [zma_m2] [Monitor-2: 164000 - Analysing at 25.00 fps]
Jul 11 19:38:59 raspberrypi zmc[25491]: INF [zmc_m5] [Monitor-5: 114000 - Capturing at 25.00 fps]
Jul 11 19:38:59 raspberrypi zmc[18485]: INF [zmc_m3] [Monitor-3: 125000 - Capturing at 14.29 fps]
Jul 11 19:38:59 raspberrypi zmc[28308]: INF [zmc_m1] [Monitor-1: 155000 - Capturing at 14.29 fps]
Jul 11 19:39:00 raspberrypi zma[20888]: INF [zma_m3] [Monitor-3: 118000 - Analysing at 14.29 fps]
Jul 11 19:39:02 raspberrypi zma[18924]: INF [zma_m5] [Monitor-5: 70000 - Analysing at 25.64 fps]
Jul 11 19:39:04 raspberrypi zmc[22481]: INF [zmc_m4] [Monitor-4: 79000 - Capturing at 14.29 fps]
Jul 11 19:38:37 raspberrypi zmc[13493]: INF [zmc_m2] [Monitor-2: 85000 - Capturing at 25.00 fps]
Jul 11 19:38:50 raspberrypi zma[18899]: INF [zma_m1] [Monitor-1: 62000 - Analysing at 14.29 fps]
Jul 11 19:38:55 raspberrypi zma[28340]: INF [zma_m2] [Monitor-2: 164000 - Analysing at 25.00 fps]
Jul 11 19:38:59 raspberrypi zmc[25491]: INF [zmc_m5] [Monitor-5: 114000 - Capturing at 25.00 fps]
Jul 11 19:38:59 raspberrypi zmc[18485]: INF [zmc_m3] [Monitor-3: 125000 - Capturing at 14.29 fps]
Jul 11 19:38:59 raspberrypi zmc[28308]: INF [zmc_m1] [Monitor-1: 155000 - Capturing at 14.29 fps]
Jul 11 19:39:00 raspberrypi zma[20888]: INF [zma_m3] [Monitor-3: 118000 - Analysing at 14.29 fps]
Jul 11 19:39:02 raspberrypi zma[18924]: INF [zma_m5] [Monitor-5: 70000 - Analysing at 25.64 fps]
Jul 11 19:39:04 raspberrypi zmc[22481]: INF [zmc_m4] [Monitor-4: 79000 - Capturing at 14.29 fps]
Jul 11 19:39:17 raspberrypi zmc[13493]: INF [zmc_m2] [Monitor-2: 86000 - Capturing at 25.00 fps]
Thanks,
Chris
cmisip
Posts: 39
Joined: Sun Apr 30, 2017 10:09 pm

Re: Raspberry pi3 and h264_mmal

Post by cmisip »

Interesting results. The number of cameras that could use the hardware decoder increases with the GPU memory split. At 256 MB for the GPU and using total_mem=1024 in /boot/config.tx, I got 4 cameras at 704x480 working. At GPU mem of 144, I could only get three working. With each zma and zmc using up 10-13% of the memory at this resolution, this only leaves < 20% for the other processes. Gotta find the perfect balance.

Gotta figure out how to reduce the memory consumption of each zma running mvdect yet. Some statistical info would help to narrow down the memory requirement for zma mvdect mode as well as setting reasonable ranges for the scoring system.

Anybody interested in trying out the software? If there is interest and its ok with the devs, let me know and I could publish test packages. Just need a place to upload them. This is only for RPI3 running Raspbian Jessie and ffmpeg with h264_mmal support. This will not have neon support as that is currently broken on the RPI3 running Raspbian Jessie.

For those who know how to build a package, the github fork is at:
https://github.com/cmisip/ZoneMinder. branch motion_vectors

There are stumbling blocks to overcome which I could detail later for anyone interested.

Gotta warn everybody that there are no guarantees here. Been successful so far, but don't know If I have the know how to finnish it. I've learned a lot though.

Chris
User avatar
iconnor
Posts: 2862
Joined: Fri Oct 29, 2010 1:43 am
Location: Toronto
Contact:

Re: Raspberry pi3 and h264_mmal

Post by iconnor »

That is amazing. Decoding 4 streams in hardware.

I may have mentioned it already, but I am working on turning zma into a thread, which will get rid of our need for the large shared memory buffer. Which should DRASTICALLY reduce our memory use.

I'm starting to envision a pi computer cluster as the ideal video server...

I really
cmisip
Posts: 39
Joined: Sun Apr 30, 2017 10:09 pm

Re: Raspberry pi3 and h264_mmal

Post by cmisip »

Actually decoding 6 streams now in hardware at 640x360 each. I had to reduce the size of the motion_vector struct to 2 bytes. It can be modified in the future to add magnitude in x and y. For now, I just want to see what the hardware can do. The bottleneck now is the cumulative CPU utilization. I am going to use sysstat to monitor. The branch reduce_mem2 is the most stable branch right now. I did overlclock a little bit as per recommendation in the RPI forums by 6by9, one of the engineers there as multiple decodes are not detected as a "hard stream" and so is not aided by automatic overclocking. So far, sar reports:

Code: Select all

19:05:01      runq-sz  plist-sz   ldavg-1   ldavg-5  ldavg-15   blocked
19:15:01            7       257      5.04      5.45      3.69         0
19:25:01            0       255      5.91      5.26      4.40         1
19:35:02            5       255      4.51      5.10      4.82         0
19:45:01            8       255      4.87      5.17      4.99         0
19:55:02           10       256      5.03      5.16      5.06         0
20:05:01            1       256      5.55      4.58      4.68         1
20:15:01            2       257      3.46      4.10      4.45         0
20:25:01            3       257      2.50      3.54      3.98         0
20:35:01            0       256      1.72      2.17      3.02         0
20:45:01            1       257      2.11      2.08      2.53         0
20:55:01            1       258      3.89      2.48      2.48         2
21:05:01            2       258      5.42      4.27      3.19         0
Average:            3       256      4.17      4.11      3.94         0
Chris
User avatar
iconnor
Posts: 2862
Joined: Fri Oct 29, 2010 1:43 am
Location: Toronto
Contact:

Re: Raspberry pi3 and h264_mmal

Post by iconnor »

There is a ton of room still for optimising ZM code so maybe we can still free up a few cycles here and there.

Thanks for working on this, I'm sorry I havn't had time to follow it closer.
cmisip
Posts: 39
Joined: Sun Apr 30, 2017 10:09 pm

Re: Raspberry pi3 and h264_mmal

Post by cmisip »

I am running into the alignment trap maybe once every few hours in dmesg. Do you have time to look at the code to see where it might be coming from? I am trying to read up on this. I created a branch align_fix to test whether I could just save things byte by byte instead of using the motion_vector structure since uint8_t * buffer is byte aligned. I only reworked the hardware decode part in zm_ffmpeg_camera.cpp. My idea was to save a sequence of uint16_t by converting to an array of uint8_t and writing each pair of bytes onto mvect_buffer. I am still getting the alignment trap. The problem might be in zm_image.cpp where mvect_buffer is created.

Thanks,
Chris
cmisip
Posts: 39
Joined: Sun Apr 30, 2017 10:09 pm

Re: Raspberry pi3 and h264_mmal

Post by cmisip »

10 hours running without triggering the alignment trap. I think maybe that it is gone (fingers crossed). I will pull the changes back to the motion_vectors branch. Will be working on reducing the memory requirements of hardware decode since it really needs a smaller buffer than software decode. All hardware macroblocks are 16x16 with a fixed total number. In comparison, software macroblocks could be any size (could possibly be all 4x4).

Chris
ralimi
Posts: 1
Joined: Sat Jul 22, 2017 8:53 pm

Re: Raspberry pi3 and h264_mmal

Post by ralimi »

This is great work! I got a RPI3 a few weeks ago with the goal of trying out ZoneMinder with it, and it looks like you have made lots of progress. I'm very interested in giving this a try.
cmisip
Posts: 39
Joined: Sun Apr 30, 2017 10:09 pm

Re: Raspberry pi3 and h264_mmal

Post by cmisip »

I thought I fixed the alignment trap issue but I am not sure now if it even is related to the motion vector code. I have put in the latest changes that use memcpy instead of pointers for serial access to the uint8_t buffer in zm_ffmpeg_camera.cpp and zm_zone.cpp. The alignment trap is only triggered after running for hours. When it is triggered, the zma's and zmc's still show up in top as running processes with normal statistics. checking if zoneminder is running in systemctl shows that it is. However, the IP address in the web UI are red. A restart of zoneminder fixes the issue. Will have to do some more testing. So not quite ready.

On a positive note, I found out that the RPI has a hardware scaler that can do format conversions and hardware jpeg encoder component. This might be a suitable replacement for swscale. I will have to look into it. It's like a bag of toys.

Chris
cmisip
Posts: 39
Joined: Sun Apr 30, 2017 10:09 pm

Re: Raspberry pi3 and h264_mmal

Post by cmisip »

I think I discovered the bug. The mvect_buffer that is used to store the motion_vector data is being destroyed when a monitor is killed when the image_buffer array is deleted as a normal step in the termination of a monitor ( when manual shutdown or restart is done in systemctl or if the zone minder daemon thinks a monitor is stuck or hung and therefore decides to kill off a monitor). The mvect_buffer is a member of the Image object. When the shutdown happens during a read or write to the buffer, then the next read or write results in an alignment trap being triggered that the OS cannot handle. I hope this is it. I put a delay in the the Monitor destructor to wait until the first bit in mvect_buffer is set to 0 before deleting the image_buffer in the case that the function is MVDECT. Then each zma and zmc sets the mvect_buffer's first byte as a status bit in order to signal the system that there are ongoing reads and writes so in case of termination signal being reached, dont delete the image object yet until the status bit is zero. This seems to have gotten rid of the error and I have it running for 10 hours now without issues with testing with multiple restarts. Fingers crossed. Hoping to move past this into designing an swscale replacement.
Chris
User avatar
iconnor
Posts: 2862
Joined: Fri Oct 29, 2010 1:43 am
Location: Toronto
Contact:

Re: Raspberry pi3 and h264_mmal

Post by iconnor »

That would certainly do it. There should probably be some more formalised locking or even just a bit to flag when zmc is shutting down.

Glad you got it figured out.
Locked