I wrote an object-detection add-on for ZM

Forum for questions and support relating to the 1.30.x releases only.
tpcarmen
Posts: 17
Joined: Fri May 19, 2017 1:52 am

Re: I wrote an object-detection add-on for ZM

Post by tpcarmen »

you might have to fix the makefile to point to where your mysqsl libs are. It might be different than on my system.
User avatar
asker
Posts: 1553
Joined: Sun Mar 01, 2015 12:12 pm

Re: I wrote an object-detection add-on for ZM

Post by asker »

On OSX, I needed to modify the Makefile LDFLAGS:

Code: Select all

LDFLAGS= -lm -pthread -lconfuse `mysql_config --cflags --libs`
I no longer work on zmNinja, zmeventnotification, pyzm or mlapi. I may respond on occasion based on my available time/interest.

Please read before posting:
How to set up logging properly
How to troubleshoot and report - ES
How to troubleshoot and report - zmNinja
ES docs
zmNinja docs
rockedge
Posts: 1173
Joined: Fri Apr 04, 2014 1:46 pm
Location: Connecticut,USA

Re: I wrote an object-detection add-on for ZM

Post by rockedge »

@asker thanks for the LDFLAGS.... I used yours to see if it would compile.....and it worked. So now I have tried
and I get a :

Code: Select all

root#./zmdn
Segmentation fault
tpcarmen
Posts: 17
Joined: Fri May 19, 2017 1:52 am

Re: I wrote an object-detection add-on for ZM

Post by tpcarmen »

Set debug=1 in the /etc/zmdn.conf and see how far it gets and what the last thing is it says.
rockedge
Posts: 1173
Joined: Fri Apr 04, 2014 1:46 pm
Location: Connecticut,USA

Re: I wrote an object-detection add-on for ZM

Post by rockedge »

I have no /etc/zmdn.conf !! Should I create one and add debug=1? what does the config file look like? I compiled a version with debug=1 in the Makefile.

I still have a Segmentation fault. Getting closer
tpcarmen
Posts: 17
Joined: Fri May 19, 2017 1:52 am

Re: I wrote an object-detection add-on for ZM

Post by tpcarmen »

It's supposed to create the config file by itself.

Try getting the newest code from github.
rockedge
Posts: 1173
Joined: Fri Apr 04, 2014 1:46 pm
Location: Connecticut,USA

Re: I wrote an object-detection add-on for ZM

Post by rockedge »

OK it worked! I used the latest code from git, modified the Makefile,
on line 1->

Code: Select all

GPU=0
on line 21 changed:

Code: Select all

LDFLAGS= -lm -pthread
to->

Code: Select all

LDFLAGS= -lm -pthread -lconfuse `mysql_config --cflags --libs`
capture4354.png
capture4354.png (28.77 KiB) Viewed 11665 times
tpcarmen
Posts: 17
Joined: Fri May 19, 2017 1:52 am

Re: I wrote an object-detection add-on for ZM

Post by tpcarmen »

Very cool!

It's running and waiting for frames that have a score greater than "MinFrameScoreToCheck" from the config file.

When it finds one, it runs the detection code, and INSERTs records into the FrameMatches table, listing what it found in the frame.

All we need now is for someone to glue the FrameMatches back into ZoneMinder, so besides "Events" we can have "Stuff I found" 8-)

If you go into zmdn.conf. you can set debug to 0, and stop the messages, or leave them running for a while to keep an eye on it.

At some point soon (maybe tomorrow) I'll have it output to a log file or table.
rockedge
Posts: 1173
Joined: Fri Apr 04, 2014 1:46 pm
Location: Connecticut,USA

Re: I wrote an object-detection add-on for ZM

Post by rockedge »

looks great and I think I am getting results!
capture16282.png
capture16282.png (146.1 KiB) Viewed 11660 times
I will try to find a place to integrate for experimentation.
tpcarmen
Posts: 17
Joined: Fri May 19, 2017 1:52 am

Re: I wrote an object-detection add-on for ZM

Post by tpcarmen »

Yep! Those are results all right! 8-)

Very cool, thanks for playing with it!

If you start picking up invalid detections, you can raise "thresh" in zmdn.conf. Probabilities that are less than "thresh" aren't INSERTed into the DB.
User avatar
asker
Posts: 1553
Joined: Sun Mar 01, 2015 12:12 pm

Re: I wrote an object-detection add-on for ZM

Post by asker »

rockedge wrote: Mon Jun 05, 2017 5:23 pm Both looking at image data via the Frame table, or processing live streams should be options. A daemon process that can be started to achieve this along the path tpcarmen is suggesting, something like zmaudit.pl? The real-time analysis need only be applied maybe in the alarm frames and then at first by working with a sampling of some frames and then some logic to determine if that analysis frame rate should increase. So let ZM detect the initial motion then trigger real time analysis?
No, you can't use this approach for real-time detection without a very powerful GPU enabled system. I've played with YoLo before. It took 11 seconds to identify a person on a 65KB downsized image (https://pjreddie.com/darknet/yolo/) without a GPU (2.9GHz/i5/15GB DDR3, Iris Graphics - so no CUDA GPU). This is the key reason why we integrated with Dlib instead of Yolo - it was faster when we compared CNNs (and also offered non CNN models). The entire concept for real-time detection is for it to be real-time and detect before alarms are sent out. The #1 use case of image labeling for regular consumers is related to alerts - they don't want spam alerts but want alerts in time. For enterprises, this soft of passive analysis is useful because they record 24x7, and when a theft occurs, they want post-analysis (and time-shifting here is fine).Obviously non real time analysis is useful for consumers too, but its not something that is a primary use.

So bottom line, you can't run a daemon with this approach (CNN based) for real-time as a general purpose soln.
I no longer work on zmNinja, zmeventnotification, pyzm or mlapi. I may respond on occasion based on my available time/interest.

Please read before posting:
How to set up logging properly
How to troubleshoot and report - ES
How to troubleshoot and report - zmNinja
ES docs
zmNinja docs
User avatar
asker
Posts: 1553
Joined: Sun Mar 01, 2015 12:12 pm

Re: I wrote an object-detection add-on for ZM

Post by asker »

I hate to keep diverting this topic, but I've never tried tiny-yolo mentioned by tpcarmen. Its speed is amazing, but its prediction capability is quite low. But I do think tiny-yolo is very impressive and potentially a candidate for real-time analysis. I'll look into how its training is done. I suspect it can be tweaked - I ran tiny-yolo with low quality security cam footage of people in front of cars. In around half the cases it said "person" which is a great start. And without GPU it takes a second to half a second
I no longer work on zmNinja, zmeventnotification, pyzm or mlapi. I may respond on occasion based on my available time/interest.

Please read before posting:
How to set up logging properly
How to troubleshoot and report - ES
How to troubleshoot and report - zmNinja
ES docs
zmNinja docs
tpcarmen
Posts: 17
Joined: Fri May 19, 2017 1:52 am

Re: I wrote an object-detection add-on for ZM

Post by tpcarmen »

asker wrote: Tue Jun 06, 2017 5:58 pm I hate to keep diverting this topic, but I've never tried tiny-yolo mentioned by tpcarmen. Its speed is amazing, but its prediction capability is quite low. But I do think tiny-yolo is very impressive and potentially a candidate for real-time analysis. I'll look into how its training is done. I suspect it can be tweaked - I ran tiny-yolo with low quality security cam footage of people in front of cars. In around half the cases it said "person" which is a great start. And without GPU it takes a second to half a second
That would be great, because I really only care about cars, people, truck, dogs and maybe a few others.

The GPU is a huge performance boost even with a cheap gpu. I went from 10 seconds/image to about .25 seconds/image.

I wish I understood more about image processing. I read the darknet paper but don't even have the context to make sense out of it. Especially exactly what they mean by convolutional image processing. I understand about sharpening, blurring, etc, but have no idea how they get from there to "person".

FWIW, I'm not using tiny.yolo, I'm using the regular one. The tiny one wasn't very accurate for me.
rockedge
Posts: 1173
Joined: Fri Apr 04, 2014 1:46 pm
Location: Connecticut,USA

Re: I wrote an object-detection add-on for ZM

Post by rockedge »

asker wrote: Tue Jun 06, 2017 3:28 pm
No, you can't use this approach for real-time detection without a very powerful GPU enabled system.

So bottom line, you can't run a daemon with this approach (CNN based) for real-time as a general purpose soln.
Yes, well explained.....I am sending individual images via the Frame console to pjreddie's /darknet/yolo and it takes 6-12 seconds
on this machine :

Genuine Intel(R) CPU T2400 @ 1.83GHz
Socket Designation: Microprocessor
Manufacturer: Intel
Voltage: 3.3 V
External Clock: 166 MHz
Max Speed: 1833 MHz
Current Speed of Core 0:1000 MHz, 1:1833 MHz
Core Count: 2
Thread Count: 2

there is no way to even run a single web cam and run darknet/yolo, the system almost seems stopped. Real time detection on a zoneminder stream before zm sends alarm or in any other way is not possible on my set ups.

but for experimentation I would like to make a button or another column to select both views, one from zm and other after it runs through yolo. Also the FrameMatches table .... are the times synced with the event id and/or frame id? The FrameMatches table used in Filters as an added condition or reported in the notification email/message/push notification (in zmNinja). It all may not be practical, but fun to add and who knows?
tpcarmen
Posts: 17
Joined: Fri May 19, 2017 1:52 am

Re: I wrote an object-detection add-on for ZM

Post by tpcarmen »

rockedge wrote: Tue Jun 06, 2017 11:11 pm but for experimentation I would like to make a button or another column to select both views, one from zm and other after it runs through yolo.
Sounds great!

rockedge wrote: Tue Jun 06, 2017 11:11 pm Also the FrameMatches table .... are the times synced with the event id and/or frame id?
Neither. The FrameMatches table is linked to the Frames Table on the Frames.Id = FrameMatches.FrameId

FrameMatches.Added is the time that the row was created, which would be when zmdn finished analysing the image and figured out what it contained.

rockedge wrote: Tue Jun 06, 2017 11:11 pm The FrameMatches table used in Filters as an added condition or reported in the notification email/message/push notification (in zmNinja). It all may not be practical, but fun to add and who knows?
That would be great! It would be awesome to be able to set an alert for "Person at Back Door"

Terry
Locked