Machine learning video detection

Anything you want added or changed in future versions of ZoneMinder? Post here and there's a chance it will get in! Search to make sure it hasn't already been requested.
Post Reply
xxrb2010
Posts: 5
Joined: Tue Oct 03, 2017 2:51 pm

Machine learning video detection

Post by xxrb2010 » Thu Nov 30, 2017 11:14 pm

Hi,

With all the open source machine learning frameworks around there and the progress made on video detection With those frameworks I wonder why nobody already proposed to add a new detection method based on machine learning frameworks.

Take my input as a way to start the discussion.

Amazon just released a camera that will do exactly that based on "open source". As I see Amazon as a thief or a parasite in the open source world I am surprise that anybody in this forum never propose such a new method for event detection.

Looking forward for your inputs.

Please do not discuss your view on Amazon. This is not the goal of my post. I just put mine as I took Amazon as an example and want to be transparent with my post.

EpicThoughts
Posts: 6
Joined: Mon Sep 04, 2017 3:54 pm

Re: Machine learning video detection

Post by EpicThoughts » Sun Dec 03, 2017 5:52 pm

Machine learning is tricky business. If you ever used opencv you'll understand how hard it actually is just to get a computer to recognize a banana.

Whoever decides to attempts this would need a powerful computer. Not everyone has that. Especially with zm being an application that doesn't use hardware acceleration.

Requirements aside, let's assume you have all that. Even then it's hard because of how we teach the computer to understand objects. It still requires our intervention to tell it what is a banana and what isn't.

I don't know exactly how amazons camera works but I'd imagine there is something somewhere requiring us to tell it it's doing a good job vs a bad one.

I am going to guess, because it applies to other CCTV platform, that it hasn't been added yet simply because it's very tedious work. Amazon has hundreds if not thousands of people just sitting at computers and feeding the big machine positives and negatives of objects. People who work on FOSS normally don't get that luxury.

In short : I doubt zm will ever see such a feature. it's, simply, beyond the capabilities of the developer(s) to figure out.

User avatar
asker
Posts: 950
Joined: Sun Mar 01, 2015 12:12 pm

Re: Machine learning video detection

Post by asker » Mon Dec 04, 2017 2:31 pm

I don't think its hard at all. It most certainly isn't out of the range of skills for the ZM devs. It is essentially a matter of time. The reason I say this is machine learning based image classification has reached a point of commoditization for 'basic detection' as far as 'reasonable quality' goes. By that I mean there are a plethora of tools and frameworks, in C, in C++, in Python and in javascript that "just work" without devs having to figure out much. You basically only need to feed it new images (many of them) and it works reasonably well.

Neural nets do image classification differently from what we have been used to in Open CV. The simplest way to describe this is in OpenCV, you need to figure out the layers and steps involved in trying to break down an image into identifiable patters - that is, you figure out the filters, the transforms and the thresholds.

In neural nets, you supply the model with a whole bunch of images, mark what is "positive" (example, person) and what is not, and let the model start classifying randomly and then keep tuning its error parameters to match the original predictions. Also, interestingly, there is a concept called "transfer learning" which basically lets us use an already trained model (like inception) to learn new images and it works very well. For example, I did an experiment a year ago, where I used tensor flow to train my own new model with 1000 images of a particular classification that did not exist in inception. I made two models - one that I trained from scratch, and the other was based off inception (transfer learning). The inception model did much better.

So bottom line, doing ML image classification has pretty much wrapped itself into 'using libraries' unless you are a researcher trying to improve existing accuracy standards.

I don't think it will take more than 1 man month of time to get something like YOLO working with ZM, integrated into its core zm_monitor.cpp process. Doing it as an external trigger (OPT_TRIGGER) is even easier, and I think someone already did it.
--
My collection of ZoneMinder learnings:
https://wiki.zoneminder.com/Various_ZM_thoughts

xxrb2010
Posts: 5
Joined: Tue Oct 03, 2017 2:51 pm

Re: Machine learning video detection

Post by xxrb2010 » Mon Dec 04, 2017 2:54 pm

I think you both resume my understanding so far:

machine learning algorithms are not set and go, you have to train them by feeding them with inputs. I would argue the current zm detection method is also a try and error method to tune it correctly. At least this is what I did. But I am convince that in the long run the machine learning algorithm will beat the current manual try and error method.

Having said so, there are a lot of progress this day in the open source machine learning algorithm with on the shelf advanced libraries.

Looking forward to see if a developer might check in

montagdude
Posts: 46
Joined: Fri Nov 10, 2017 6:05 pm

Re: Machine learning video detection

Post by montagdude » Thu Jan 25, 2018 5:43 pm

After having run ZoneMinder for several months, I agree that this would be a really killer feature even if the current motion detection settings are retained. Imagine that when you view an event, there are buttons to "Mark real alarm" and "Mark false alarm," which if you click, ZoneMinder will add the alarm images from that event into a machine learning database and start retraining the algorithm. Eventually, after feeding the algorithm many events, false alarms should be almost completely eliminated. Of course, I have no idea how computationally expensive it would be for the algorithm to analyze potential alarm frames on the fly.

If I were a single man with lots of free time, I might start experimenting with this myself. :) Getting back to reality, I don't have any experience with imaging science, neural networks, the ZoneMinder source code at any deep level, or the time to learn all those things. But it seems like it would be a fun side project for one of the developers...

Jarod42
Posts: 1
Joined: Sat Jun 23, 2018 10:43 am

Re: Machine learning video detection

Post by Jarod42 » Sat Jun 23, 2018 11:57 am

I don't think that ZM really needs a machine based learning algorithm before it can handle motion detection reliably.

Right now ZM only calculates the number of pixels that differ from one picture to the other (please correct me if I'm wrong). Here at my setup this results in false alarms, when a cloud covers the sun or when the sun moves around the house and casts a shadow, or brightens certain spots. Even noise on the video (due to lack of light) causes an alarm.
Most parts of a surveillance image never change, in terms of their outline. So they remain pretty much static. So ZM needs something to distinguish the moving object form the more or less static background.
Searching the internet for "master thesis reliable motion detection" brings up a whole bunch of PDFs.
Even github seems to be a source of code from people who have done some work using OpenCV and other libs.

I'd say as soon as ZM offers reliable motion detection, it is time to think about machine based learning like recognizing faces, pets & cattle...

Just my 2 cents.

rockedge
Posts: 906
Joined: Fri Apr 04, 2014 1:46 pm
Location: Connecticut,USA
Contact:

Re: Machine learning video detection

Post by rockedge » Sat Jun 23, 2018 7:05 pm

you'll need a bigger machine.....most users of ZM do not have the machine power to do it real time detection with Opencv or any other framework. You could upload frames to google or amazon but it wouldn't be in real time.

User avatar
asker
Posts: 950
Joined: Sun Mar 01, 2015 12:12 pm

Re: Machine learning video detection

Post by asker » Sat Oct 13, 2018 11:22 am

An update on this:

1. I've got both YoloV3 and tiny yolo working on my ZM machine through OpenCV reporting alarms almost real-time via the event notification server (within ~3-4 seconds or less of the alarm being triggered) . My system doesn't have a GPU but is a workstation class linux machine running ubuntu. I now run this regularly at home and get incredible results instead of rubbish being pushed to my phone. I use the standard YoloV3 model not tiny yolo. I tried both and found YoloV3 to be more reliable (obviously). Given that the detection triggers after motion is detected, the load is not too much.

Read more here for a brief and here for a detailed explanation and with code here

Performance wise, I'm seeing acceptable performance on a non GPU machine. See performance notes here

I've been digging further into lower powered devices, and I do believe you can achieve usable performance on RPI 3 too using optimized builds like this one.

I'd strongly recommend folks to try it out, get things working and then look for optimized builds/models. Practical results may surprise you.
--
My collection of ZoneMinder learnings:
https://wiki.zoneminder.com/Various_ZM_thoughts

Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest