Instant notification on alarm + machine learning object detection

If you've made a patch to quick fix a bug or to add a new feature not yet in the main tree then post it here so others can try it out.
User avatar
iconnor
Posts: 2862
Joined: Fri Oct 29, 2010 1:43 am
Location: Toronto
Contact:

Re: Instant notification on alarm + machine learning object detection

Post by iconnor »

Hey do you want to take this development to gtihub? I can add you as ZoneMinder Members...
montagdude
Posts: 88
Joined: Fri Nov 10, 2017 6:05 pm

Re: Instant notification on alarm + machine learning object detection

Post by montagdude »

Sure, I'd be honored! What did you have in mind, though, integrating it into ZoneMinder itself in some way, or as a separate script the way it is now?
alabamatoy
Posts: 349
Joined: Sun Jun 05, 2016 2:53 pm

Re: Instant notification on alarm + machine learning object detection

Post by alabamatoy »

montagdude wrote: Fri Mar 01, 2019 3:42 pm Sure, I'd be honored! What did you have in mind, though, integrating it into ZoneMinder itself in some way, or as a separate script the way it is now?
I would like for it to be a maintained part of Zoneminder which could be enabled or disabled as desired. In many installs, I believe that the Zoneminder motion detection works well enough on its own, but in other situations (such as outside) it works not so well and this add-on would dramatically improve it.
montagdude
Posts: 88
Joined: Fri Nov 10, 2017 6:05 pm

Re: Instant notification on alarm + machine learning object detection

Post by montagdude »

alabamatoy wrote: Fri Mar 01, 2019 3:51 pm I would like for it to be a maintained part of Zoneminder which could be enabled or disabled as desired. In many installs, I believe that the Zoneminder motion detection works well enough on its own, but in other situations (such as outside) it works not so well and this add-on would dramatically improve it.
The main benefit of this tool, for me anyway, is still the instant notification part. For my usage, I have disabled object detection because it is still not reliable enough. However, all of my cameras are indoors, and I have to use low res images for the detection because my server is pretty underpowered for object detection. It would probably still work well for an outdoor camera in daylight and a fast server (especially if you can do the object detection on a GPU), so I will continue to test and maintain that functionality.

Anyway, I'm not sure I am the right person to do the heavy lifting of implementing object detection and/or instant notifications in the main ZoneMinder codebase. It would take a lot of learning and testing for me, and I just don't have the time for that right now. But I am more than happy to contribute where possible.
russellhq
Posts: 3
Joined: Sun Jul 07, 2019 8:56 pm

Re: Instant notification on alarm

Post by russellhq »

montagdude wrote: Fri Feb 01, 2019 4:55 am I have ditched my perl script and switched to a Python script that uses the API. Inspired by asker's zmeventserver, I have also added object detection with OpenCV + YOLO v3 to eliminate false notifications, which is working really well. This method also has the benefit of not reading the shared memory directly, so I no longer need to kill it when ZoneMinder stops to free the shared memory. The code is here - feel free to give it a try if so inclined:

https://github.com/montagdude/zoneminder-notifier
Thanks for sharing this. I gave it a try and got most of it to work but it always seems to fail at sending the message. I tested out using mutt to send a test message from the terminal and that worked fine. But when I use the notifier I get the following error message:

Code: Select all

[xxxx@localhost ~]$ sudo /usr/bin/zm_notifier
2019-07-07 21:40:33 Successful login
2019-07-07 21:40:33 ZoneMinder is now running.
2019-07-07 21:40:49 Appended monitor 3: Front Door
2019-07-07 21:40:50 Warning: no active monitor named 
2019-07-07 21:42:02 Connection error in getFrameImage
2019-07-07 21:42:02 Sending event alerts for event 156
Error sending message, child exited 127 (Exec error.).
Could not send the message.
2019-07-07 21:43:59 No events list present
2019-07-07 21:44:08 Downloaded image /tmp/zm_event_image.jpg after 4 attempts.
2019-07-07 21:44:17 Detected person in 8.60 seconds
2019-07-07 21:44:17 Sending event alerts for event 157
Error sending message, child exited 127 (Exec error.).
Could not send the message.
montagdude
Posts: 88
Joined: Fri Nov 10, 2017 6:05 pm

Re: Instant notification on alarm + machine learning object detection

Post by montagdude »

It looks like an error with mutt. Googling the error message turns up some hits (https://www.google.com/search?q=mutt+er ... essage+127), but I'm not sure if they are helpful. I would recommend taking a look and see if anything seems relevant to your situation.

When you sent a test message, did you send exactly the same thing that the script is sending? If you have set it up to attach an image, try:

Code: Select all

mutt -s "Subject line" -a /tmp/zm_event_image.jpg -- email_address < /tmp/zm_event_email.txt
where you replace email_address with your own. If not attaching an image, try:

Code: Select all

mutt -s "Subject line" email_address < /tmp/zm_event_email.txt
First make sure those files (/tmp/zm_event_email.txt and /tmp/zm_event_image.jpg) exist. I would expect it to fail in the same way as it does in the script if you run the same thing from the command line.
russellhq
Posts: 3
Joined: Sun Jul 07, 2019 8:56 pm

Re: Instant notification on alarm + machine learning object detection

Post by russellhq »

Thanks, I gave that a go and MUTT gave an error:

Code: Select all

Fcc to an IMAP mailbox is not supported in batch mode
I used this guide to configure mutt to use Gmail:
https://lifehacker.com/how-to-use-the-f ... th-5574557

Turns out the following configuration works (copied below) and now I get notification emails using the command line but not with notifier.
https://www.garron.me/en/go2linux/send- ... -mutt.html

Code: Select all

set from = "user@gmail.com"
set realname = "Guillermo Garron"
set imap_user = "user@gmail.com"
set imap_pass = "password"
set folder = "imaps://imap.gmail.com:993"
set spoolfile = "+INBOX"
set postponed ="+[Gmail]/Drafts"
set header_cache =~/.mutt/cache/headers
set message_cachedir =~/.mutt/cache/bodies
set certificate_file =~/.mutt/certificates
set smtp_url = "smtp://user@smtp.gmail.com:587/"
set smtp_pass = "password"
set move = no 
set imap_keepalive = 900
I googled a bit more and ran the following commands and now I don't get any errors but I don't get any notifications either (command line still works)

Code: Select all

sudo dnf install postfix
mkfifo /var/spool/postfix/public/pickup
sudo systemctl restart postfix
This is what I get now from zm-notifier:

Code: Select all

KeyboardInterrupt
[xxxx@localhost zoneminder-notifier]$ sudo /usr/bin/zm_notifier
2019-07-11 20:58:42 Successful login
2019-07-11 20:58:42 ZoneMinder is now running.
2019-07-11 20:58:58 Appended monitor 3: Front Door
2019-07-11 20:59:28 Sending event alerts for event 207
Contents of /etc/zmnotifier.cfg

Code: Select all

[ZoneMinderAPI]
# local_server_address is the address used for all ZoneMinder API queries
# (for example, could be http://localhost or other local network address, or
# an external address if desired). world_server_address is the address sent
# in image URLs. The two do not necessarily need to be different.
local_server_address: https://localhost
world_server_address: https://localhost

# ZoneMinder credentials. (Yes, stored in plain text. Please ensure this
# file is secured. For example, owned and only readable by root.)
username: ******
password: ******

# No is needed for self-signed certificate
verify_ssl: No

[Email]
# Email settings. Uses mutt to send the email. It is up to you to configure
# mutt as desired (e.g., setting it up to use smtp vs sendmail).

tmp_message_file: /tmp/zm_event_email.txt
tmp_image_file: /tmp/zm_event_image.jpg

# Enter email addresses to receive notifications delimited by commas. These
# can also be SMS/MMS gateways. The attach_image parameter corresponds to each
# address in the list. If No, a URL link to the image will be sent instead.
addresses: ******@gmail.com
attach_image: No

[Daemon]
# How long to pause when checking for new events
running_timeout: 3

# How long to pause when checking if running
stopped_timeout: 15

# Approx. how many seconds to wait between logins (to keep session alive).
# Should be at least every 60 minutes.
login_refresh: 1200

[Detection]
# Object detection settings. Uses OpenCV with to detect desired classes of
# objects (e.g., person, car) in the maxscore frame for each event. The
# primary purpose is to eliminate false alerts. However, depending on the
# settings, the detection may take up to a few seconds per frame.

detect_objects: No
detect_classes: person, car, dog

# Names of monitors to exclude from object detection. Can be blank if you want
# to do object detection with all of them.
exclude_monitors: Front Door

# Normally 320, 416, or 608. Doesn't have to match actual frame resolution.
# Lower is faster; higher is more accurate.
img_width: 416
img_height: 416

# Ignore objects with confidence less than this
conf_threshold: 0.25

# These files should be present if YOLO v3 is installed. Can also use
# YOLO v3 tiny for fast detection, but it doesn't work very well.
detection_config: /home/****/darknet/cfg/yolov3.cfg
detection_weights: /home/****/darknet/yolov3.weights
detection_names: /home/****/darknet/data/coco.names

# Once a requested class of objects is detected in a frame, you can
# use this to stop detecting for the desired number of seconds and just
# push notifications as quickly as possible. Set to 0 if you always want
# detection.
detection_suspend: 1
montagdude
Posts: 88
Joined: Fri Nov 10, 2017 6:05 pm

Re: Instant notification on alarm + machine learning object detection

Post by montagdude »

Well, I'm not really sure what's going on with Mutt for you then. Is there anything in your mail logs? (I'm not sure where that is for systems with systemd; I'm guessing in journalctl. On my system /var/log/maillog keeps track of emails that failed to send.)

Since you are not sending image attachments, another alternative if you can't get mutt to work is to just use a regular smtp client. I only have it set up to use mutt because it's easier to send attachments that way. For example, in the old version of this script I used msmtp. Translating it to python, you could replace the mutt call with this:

Code: Select all

subprocess.call(['msmtp', '-a', 'default', addr["address"]], stdin=stdin_file)
Of course, then you need to set up msmtp. My msmtprc file looks like this:

Code: Select all

# Example for a system wide configuration file

# A system wide configuration file is optional.
# If it exists, it usually defines a default account.
# This allows msmtp to be used like /usr/sbin/sendmail.
#
# Set default values for all following accounts.
defaults
auth           on
tls            on
tls_trust_file /etc/ssl/certs/ca-certificates.crt
logfile        /var/log/msmtp.log

# Gmail
account        gmail
host           smtp.gmail.com
port           587
from           user@gmail.com
user           *****
password       *****

# Set default account
account default : gmail

# Syslog logging with facility LOG_MAIL instead of the default LOG_USER.
syslog LOG_MAIL
However, I'd check the mail logs first. It's possible that whatever is causing your issue would also occur with a plain smtp client.
linuxhak
Posts: 1
Joined: Mon Jan 20, 2020 5:46 pm

Re: Instant notification on alarm + machine learning object detection

Post by linuxhak »

Hello. I don't really want to steal anyone's thunder or discount work that anyone else may have put into get this type of functionality but from casual observation this appears to be a process that polls and in order to get instant notification the polling needs to be rather frequent. Has anyone tried to do more of an interrupt driven method?

I'm a seasoned C developer who typically spends their time close to HW. For example, I've done a fair amount of work on the linux kernel and kernel drivers (back in the 2.2/2.4 days) and have spent the last several years working in u-boot and UEFI code. Yeah, I'm a C hard core coder but I've never done any web work and honestly don't do much in databases either.

That said, I was quickly and easily able to add one function to zm_monitor that spawns a child process right when an event is triggered and this child process runs, in my case, a bash script that captures a frame from the camera and emails it. WIthin a few seconds, say 3 to 5 from when the motion was first detected, I have a text message on my phone with a hi-res image from the camera. The function was written so that no zombies are created and the process that does the exec() has been dropped in priority to reduce the impact on zm itself.

Although I am running several cameras I personally really only have interest in getting notifications from one of the cameras at this point in time. So my vision is that the database can be used to define the command to run and the field would reside in the table for the monitors. (I'll be honest, I haven't looked at the database but I'm sure it would have a table for monitors.) This database field would of course be populated using the webui. This is the work that would take a long time for me because, as I've said, I don't do web and I'm not really a database guy. I'm sure I can figure it out but maybe someone here can do this work as quickly as I did the c++ portion.

So what I currently have in zm_monitor.cpp off of master is a hard coded hack to call my function only if the id matches the id I want events on. It is working great and has been 100% reliable for several days now with no issues. Again, there is no polling involved; at the point where the zm c++ code has detected a new event and it is going to create one it calls my function which spawns the notification process. I could easily modify this code to spawn a per-monitor database defined command where it only does so if a command is defined for the monitor. In other words, each monitor would be able to be independently configured to provide event notifications if desired and each monitor would be able to do something different.

Is there any interest in this methodology and particularly in collaborating to get the web/db aspect complete? Montagdude? I don't know your background or if you have experience doing this type of web/db work... Again, not trying to steal thunder or discount any previous work. I'm just trying to get a flexible, easy to use, low impact solution.
User avatar
asker
Posts: 1553
Joined: Sun Mar 01, 2015 12:12 pm

Re: Instant notification on alarm + machine learning object detection

Post by asker »

Yes there is interest. (Edited on Jan 22 to simplify)

The ZM Event Server currently polls SHM to retrieve state. I had wanted to write an extension to zm_monitor to do what you did but never got around to doing it (plus C++ is not a language I am very comfortable with). Give you are a seasoned C/C++ programmer, what would be actually nice is to write it the following way, so its not a single purpose hack:

a) We develop a broadcast server as part of the ZM ecosystem (lets call is zmbs.pl - love the name) that gets started as part of the daemons in zmdc.pl
b) Consumers (like the ZM Event Server, or others) register with zmbs over web sockets to get notifications on specific topic(s) (example topic = "new_alarm_start", field="monitor 9")
c) Producers (zmmonitor.cpp being one) use a named pipe to send events to zmbs.pl which then sends notifications as needed to interested consumers


That's my 2 cents anyway.
I no longer work on zmNinja, zmeventnotification, pyzm or mlapi. I may respond on occasion based on my available time/interest.

Please read before posting:
How to set up logging properly
How to troubleshoot and report - ES
How to troubleshoot and report - zmNinja
ES docs
zmNinja docs
russell_i_brown
Posts: 42
Joined: Wed Mar 18, 2009 9:46 am
Location: Peterborough, England

Re: Instant notification on alarm + machine learning object detection

Post by russell_i_brown »

Just a thought.... but wouldn't MQTT cover the functionality you describe and leverage all the other tools and systems (who just said Homeassistant?) that use MQTT?

PS - Just fired up your motion stuff today - Brilliant! TVVM.
montagdude
Posts: 88
Joined: Fri Nov 10, 2017 6:05 pm

Re: Instant notification on alarm + machine learning object detection

Post by montagdude »

@linuxhak,

Definitely don't worry about stealing anyone's thunder (certainly not mine). I agree it would be best for this to be a core ZoneMinder functionality, specifically because then there would be no need for polling. However, my Web/DB programming experience is pretty limited, and I don't have time for a big development effort anyway, so I'm not the right guy for the job.
User avatar
iconnor
Posts: 2862
Joined: Fri Oct 29, 2010 1:43 am
Location: Toronto
Contact:

Re: Instant notification on alarm + machine learning object detection

Post by iconnor »

Now that 1.34 is out and someday I'll start breaking things in 1.35... we can bring this in.. however getting rid of the polling action requires some serious functionality boosts in ZM core.

I'm wondering how this compares/overlaps with event server. Today I was talking with @pliablepixels about how maybe ES should be written in python instead of perl.

We have two separate efforts going on here, and while competition can be good, maybe working together is better.
mrd
Posts: 269
Joined: Wed Apr 26, 2006 12:39 am
Location: Boston USA

Re: Instant notification on alarm + machine learning object detection

Post by mrd »

I've been trying to catch up on what the latest release has in it these days and I'm wondering if this is part of it and built in to 1.34.7? Is there a doc somewhere that has all of the major features. I guess the release notes may have this but that is a long list to dig through.

Any hints would be great.
MRD
montagdude
Posts: 88
Joined: Fri Nov 10, 2017 6:05 pm

Re: Instant notification on alarm + machine learning object detection

Post by montagdude »

I'm finally getting around to updating these scripts for Python3 and ZoneMinder API 2.0 now that I have updated my system. Should have that done within a few days probably. If anyone here is using it, keep an eye on the Github repo for updates.

I've been away from any ZoneMinder news for a while, so I'm not sure what the status of object detection is in the main code. I know the event server has it, but I'm not sure of its development status. Just thought I'd throw in an idea. Lately I've been playing with SSD Mobilenet V3 and found that it runs much faster than Yolo V3, even real-time on the CPU if you don't try to analyze too many streams at once (or if have a powerful enough computer). My laptop can do full-time object detection on it with 2-3 streams at a time without fully pegging the CPU. However, rather than running object detection full-time, I think it could run sparingly as a way to filter out false alarms. For example, use the regular motion detection algorithms most of the time, but only initiate an event if a person is detected in the frame. This could make zones setup potentially a lot simpler, realistically removing the need for preclusive zones in many cases or any zones in the extreme case. Or with someone's existing setup, it could still add another layer of filtering to avoid false alarms.

Another option is Histogram of Oriented Gradients detection. It only does people, and I've found it a little more finicky and somewhat less reliable than SSD Mobilenet V3, but it runs comparably fast and has the advantage of being included in OpenCV.

By the way, I haven't found an "official" repository for SSD Mobilenet V3. I don't think the GitHub link above is it. Licensing would have to be figured out first before this is brought into any official ZoneMinder repository.
Post Reply