Instant notification on alarm + machine learning object detection

If you've made a patch to quick fix a bug or to add a new feature not yet in the main tree then post it here so others can try it out.
Autonomous
Posts: 11
Joined: Wed Dec 28, 2022 9:22 pm

Re: Instant notification on alarm + machine learning object detection

Post by Autonomous »

I modded pushapi_pushover.py to function as the hook end event. This script checks for "person" in event notes generated by objdetect and sends a notification immediately via pushover. The issue with push event processing is that DetectionJson isn't provided as a parameter to the notification script so you can't test for specific object types.

It takes about 12 seconds to complete object detection processing and receive an event via pushover when running ZM server on my Samsung S9 / Termux installation.

zm_event_end.py :

Code: Select all

#!/usr/bin/python3

version = 0.1

# This is a sample python script for sending notifications over pushover
# Write your own script to add a new push service, and modify 
# api_push_script in zmeventnotification.ini to invoke that script

# Example taken from https://support.pushover.net/i44-example-code-and-pushover-libraries#python

# Arguments passed
# ARG1 = event ID
# ARG2 = monitor ID
# ARG3 = monitor name
# ARG4 = Notes 
# ARG5 (Optional) = image path

# ===============================================================
# MODIFY THESE
# ===============================================================



# Look at https://pushover.net/api and put anything you want here
# just don't add image, title and message as it gets automatically
# populated later

param_dict = {
    'token': None, # Leave it as None to read from secrets or put a string value here
    'user' : None, # Leave it as None to read from secrets or put a string value here
    #'sound':'tugboat',
    #'priority': 0,
    # 'device': 'a specific device',
    # 'url':  'http://whateeveryouwant',
    # 'url_title': 'My URL title',
   
}




# ========== Don't change anything below here, unless you know what you are doing 

import sys
from datetime import datetime
import requests
import pyzm.ZMLog as zmlog
import os


# ES passes the image path, this routine figures out which image
# to use inside that path
def get_image(path, cause):
    # as of Mar 2020, pushover doesn't support
    # mp4
    if os.path.exists(path+'/objdetect.gif'):
        return path+'/objdetect.gif'
    elif os.path.exists(path+'/objdetect.jpg'):
        return path+'/objdetect.jpg'
    prefix = cause[0:2]
    if prefix == '[a]':
        return path+'/alarm.jpg'
    else:
        return path+'/snapshot.jpg'

# Simple function to read variables from secret file
def read_secrets(config='/etc/zm/secrets.ini'):
    from configparser import ConfigParser
    secrets_object = ConfigParser(interpolation=None, inline_comment_prefixes='#')
    secrets_object.optionxform=str
    zmlog.Debug(1,'eid:{} Reading secrets from {}'.format(eid,config))
    with open(config) as f:
        secrets_object.read_file(f)
    return secrets_object._sections['secrets']

# -------- MAIN ---------------
zmlog.init(name='zmeventnotification_pushapi')
zmlog.Info('--------| Pushover Plugin v{} |--------'.format(version))
if len(sys.argv) < 6:
    zmlog.Error ('Missing arguments, got {} arguments, was expecting at least 6: {}'.format(len(sys.argv)-1, sys.argv))
    zmlog.close()
    exit(1)

eid = sys.argv[1]
mid = sys.argv[2]
mname = sys.argv[3]
notes = sys.argv[4]
event_type = 'event_end'
image_path = None
files = None

if not "person" in notes:
#    print("no person found")
    quit()

#print("person found, sending notification")

if len(sys.argv) == 6:
    image_path =  sys.argv[5]
    fname=get_image(image_path, '')

    zmlog.Debug (1,'eid:{} Image to be used is: {}'.format(eid,fname))
    f,e=os.path.splitext(fname)
    if e.lower() == '.mp4':
        ctype = 'video/mp4'
    else:
        ctype = 'image/jpeg'
    zmlog.Debug (1,'Setting ctype to {} for extension {}'.format(ctype, e.lower()))
    files = {
         "attachment": ("image"+e.lower(), open(fname,"rb"), ctype)
    }


if not param_dict['token'] or param_dict['user']:
    # read from secrets
    secrets = read_secrets()
    if not param_dict['token']:
        param_dict['token'] = secrets.get('PUSHOVER_APP_TOKEN')
        zmlog.Debug(1, "eid:{} Reading token from secrets".format(eid))
    if not param_dict['user']:
        param_dict['user'] = secrets.get('PUSHOVER_USER_KEY'),
        zmlog.Debug(1, "eid:{} Reading user from secrets".format(eid))

param_dict['title'] = '{} Alarm ({})'.format(mname,eid)
param_dict['message'] = datetime.now().strftime(' at %I:%M %p, %b-%d')
if event_type == 'event_end':
    param_dict['title'] = 'Ended:' + param_dict['title']

disp_param_dict=param_dict.copy()
disp_param_dict['token']='<removed>'
disp_param_dict['user']='<removed>'
zmlog.Debug (1, "eid:{} Pushover payload: data={} files={}".format(eid,disp_param_dict,files))
#print ("finished processing: ")
#print(disp_param_dict)
#print(files)
r = requests.post("https://api.pushover.net/1/messages.json", data = param_dict, files = files)
zmlog.Debug(1,"eid:{} Pushover returned:{}".format(eid, r.text))
#print("Pushover returned:"+r.text)
zmlog.close()
Last edited by Autonomous on Tue Jan 10, 2023 3:41 am, edited 1 time in total.
montagdude
Posts: 88
Joined: Fri Nov 10, 2017 6:05 pm

Re: Instant notification on alarm + machine learning object detection

Post by montagdude »

Cool, I had never heard of Pushover. I may well end up using that too, since my carrier stopped supporting SMS gateway and never supported MMS gateway in the first place.

I'm doing a lot of experimenting with various detection models now. My notification code will support object detection on either the max score image or the entire video, kicking out once it detects one of the requested classes with enough confidence. The HOG and SSD models are fast enough to do the entire video if you have a decently fast CPU. The Darknet models are slower but more accurate, so they may be a better choice if just analyzing the single frame. Just as a quick benchmark, on my laptop with 8-core i7 2.4 GHz processors and no GPU detection I get about:
  • 1.5 FPS with Darknet (Yolo V3 & V4)
  • 8 FPS with SSD Inception V2
  • 15 FPS with SSD MobileNet V2 & V3
  • 18-25 FPS with HOG
The timings vary somewhat depending on the settings, especially how you scale the image for Darknet and HOG. HOG detects people only and isn't based on machine learning at all, but it works fairly well.
montagdude
Posts: 88
Joined: Fri Nov 10, 2017 6:05 pm

Re: Instant notification on alarm + machine learning object detection

Post by montagdude »

I just published a new release. See: https://github.com/montagdude/zoneminde ... es/tag/0.2

New in this release:
  • Support for latest ZoneMinder (tested with 1.36.32), Python 3, and ZoneMinder API 2.0
  • 4 object detection models included: Darknet (Yolo V4), SSD MobileNet V3, SSD Inception V2, and Histograms of Oriented Gradients
  • Ability to use different object detection settings for each monitor
  • Support for notifications via Pushover API (in addition to via email with Mutt that was included previously)
  • Image attachments are annotated with object detection results
  • Ability to do object detection on entire video instead of just max score frame, if desired (this is computationally expensive)
  • Improved reliability: you should now always get an image attachment if you asked for it
  • Under the hood, almost a complete rewrite of the code
If you try this and encounter any issues, please report them on Github. I've been testing it a lot in the last several days, but I'm sure I haven't tested every possible code path or scenario.
Magic919
Posts: 1381
Joined: Wed Sep 18, 2013 6:56 am

Re: Instant notification on alarm + machine learning object detection

Post by Magic919 »

Recently, there's work on the ZMES ML/AI being discussed on the Slack channel. You might find it interesting.
-
Autonomous
Posts: 11
Joined: Wed Dec 28, 2022 9:22 pm

Re: Instant notification on alarm + machine learning object detection

Post by Autonomous »

montagdude wrote: Mon Jan 16, 2023 3:02 pm I just published a new release. See: https://github.com/montagdude/zoneminde ... es/tag/0.2

New in this release:
  • 4 object detection models included: Darknet (Yolo V4), SSD MobileNet V3, SSD Inception V2, and Histograms of Oriented Gradients
Regarding running object detection on hardware with limited resources such as aarch64 with 4gb ram, here are two detection models of interest:
Claim: For person detection for security and surveillance purposes, running on hardware with limited resources, the best object detection model is MobileNet. MobileNet is an efficient neural network architecture that is designed specifically for mobile devices and other hardware with limited resources. It is capable of detecting objects with high accuracy, while also being relatively fast and efficient.
https://github.com/tensorflow/models/tr ... /mobilenet
Claim: YOLOv7 surpasses all known object detectors in both speed and accuracy in the range from 5 FPS to 160 FPS and has the highest accuracy 56.8%

The Yolov7 model is designed to be lightweight and relatively efficient, so it should be able to run on hardware with limited resources. However, it is important to note that the accuracy of the model may suffer as a result of the limited hardware resources.
https://github.com/WongKinYiu/yolov7

Any thoughts on implementing yolov7 or mobilenet?

Image

For my use case, which is running a ZM event notification server on android/termux, it is possible to send a stream of images obtained from ZM to TensorFlow Lite and use the GPU for acceleration. To do this, you will need to use the Android Neural Network API, which will allow you to access the GPU for accelerated inference. Additionally, you will need to ensure that your device supports the Android Neural Network API and that the termux app is running on the same device.
montagdude
Posts: 88
Joined: Fri Nov 10, 2017 6:05 pm

Re: Instant notification on alarm + machine learning object detection

Post by montagdude »

MobileNet is already one of the supported models. I looked at Yolo V7, but it doesn't look like anyone has converted it to a form that OpenCV can use directly yet, and I don't really want to deal with TensorFlow since it's a whole different toolkit.
Post Reply