Autodelete events without objects detected to dramatically reduce event count

If you've made a patch to quick fix a bug or to add a new feature not yet in the main tree then post it here so others can try it out.
Post Reply
russell_i_brown
Posts: 42
Joined: Wed Mar 18, 2009 9:46 am
Location: Peterborough, England
Contact:

Autodelete events without objects detected to dramatically reduce event count

Post by russell_i_brown » Mon Mar 02, 2020 10:14 am

I've been using Asker's most excellent motion detection stuff for ZM (https://github.com/pliablepixels/zmeventnotification) to 'declutter' ZM with dramatic effects. Of course, Asker's zmeventnotification extension's prime purpose is to filter events for actual objects of interest before sending you a notification; I'm not sending notifications but leveraging it to sanitise events on my ZM server.

Here is the console line for the same camera running on two ZM systems - one with Object Detection & Autodelete and one without. Same ZM motion detection settings for both.

Image

Image

As you can see, the system running Object Detection & Autodelete has only saved 20 events for the past week whereas the normal ZM system has 864. The storage is only halved because I'm saving analysis images and the .mp4 on the object-detection server.

The 20 events on the Object Detection ZM all feature 'interesting' things (people) whereas the 'traditional' Motion Detection system is full of events triggered by birds, bats and bugs.

All this means that I can glance at my ZM console (or see the event counters on my Homeassistant main page) and look at events of interest with some reasonable confidence that they will actually be... interesting :)


Now... the motion detection isn't perfect (I still get the occasional event where it's mis-identified something or, less often, missed something that I would be interested in) but it's very good and, let's face it, you're never going to review 864 events but 20 in a week is much more managable. I should also state at this point that I'm also recording the hi-res feeds from all my cameras 24/7 so if the Motion Detection Autodelete did get rid of something I can refer to those (I've been doing this for 'normal' motion detection since 2018 - viewtopic.php?f=9&t=27537).

Another useful side effect is that the balance of motion detection sensitivity can be tipped firmly towards the very sensitive (which would normally drive you mad with zillions of events) and the Autodelete will simply get rid of events that are not 'interesting'. You will see events appear in the ZM console and then disappear a few seconds later but I can live with that.

Asker's got all the instructions needed to install his EventNotification/Motion Detection stuff at the link show at the top of this post. Once that's installed and working, simply download the attached script (zm_event_end_autodelete.sh.txt - the .txt extension is only there to pacify the forum gods, get rid of it once downloaded) and follow the simple instructions contained therein.

Other notes.

Asker's Object detection doesn't need a GPU (although having one helps as it does for zmc). What does impact the overall load of doing Motion Detection is not having to reload the Yolov3 model for each event (this can take a couple of seconds). Asker's written an API gateway (MLAPI) to a standalone Object Detection routine (https://github.com/pliablepixels/mlapi) and running this, even on the same server as ZM, means that it loads the Yolov3 model once when it fires up and thereafter 'just' runs the images against the model each time it's called (milliseconds not seconds). You don't need to run the mlapi gateway but it smooths things out loadwise.

I fire up the MLAPI routine with a simple script and supervisord entry:

/usr/local/bin/run_mlapi

Code: Select all

#!/bin/bash
cd /home/russell/mlapi
python3 api.py -c config.ini
/etc/supervisor/conf.d/mlapi.conf

Code: Select all

#
#       MLAPI
[program:MLAPI]
command=/usr/local/bin/run_mlapi
autostart=yes
autorestart=yes
startretries=100
stderr_logfile=/var/log/zm/mlapi.err.log
stdout_logfile=/var/log/zm/mlapi.log
I hope this is useful and huge kudos to Asker (and indeed Isaac and all the other ZM contributors).
Attachments
zm_event_end_autodelete.sh.txt
(3.39 KiB) Downloaded 215 times

ejpb
Posts: 2
Joined: Thu Jan 16, 2020 10:11 am

Re: Autodelete events without objects detected to dramatically reduce event count

Post by ejpb » Fri Jul 24, 2020 11:27 am

Hello.
I have used this script for a while, it's great! Thanks.

I have, like many other it seems, a low-res monitor with Modect and a linked HD monitor with Nodect, which works good. I added a delete section for the linked events also to the script. It really deletes the latest event from the linked monitor, I couldn't find any good way to find the exact event, if there are difference in the recording lengths between the two, etc (see below). Best to keep their buffer length settings synced, I guess.
I made a python adaptation of it, the main benefits are being able to have OPT_USE_AUTH enabled and you get the logs within the zoneminder ecosystem. I had to "go low" and use the "private" function _make_request and also do a modification in the return handling. Perhaps that change is something Asker can verify (does it mess up other uses?). It works for this, as I don't use the return value. I also don't use the notifications, so the script exit values perhaps need to be verified by someone using them.

One issue regarding pyzm:
I have seen some weird bits in the event filtering in pyzm. I had som extra debug logging with the five input arguments to the script. The filter 'e_filter' (line 126) has m_id set to the linked monitor but the returned event is from the lowres monitor... I can't see how that could be, so I'm at a loss here. It usually works as intended. I had 0 post-event frames on the lowres and 12 seconds post-event on the HD, so I had two events versus one long, but I can't see how that would affect this, results from wrong monitor...

Code: Select all

$ journalctl -g hook_del_events -o cat
INF [hook_del_events] [args: 63594 - 8 - Office - Motion: door - /var/lib/zoneminder/events/8/2020-07-23/63594]
INF [hook_del_events] [Delete linked event 63596 @ m_24 (event from filter reports: monitor 8)]
INF [hook_del_events] [Delete event 63594 @ 8-Office: "Motion: door"]
INF [hook_del_events] [args: 63596 - 8 - Office - Motion: door - /var/lib/zoneminder/events/8/2020-07-23/63596]
INF [hook_del_events] [Delete linked event 63596 @ m_24 (event from filter reports: monitor 8)]
INF [hook_del_events] [Delete event 63596 @ 8-Office -: "Motion: door"]
And as intended:

Code: Select all

INF [hook_del_events] [args: 63616 - 8 - Office - Motion: door - /var/lib/zoneminder/events/8/2020-07-23/63616]
INF [hook_del_events] [Delete linked event 63617 @ m_24 (event from filter reports: monitor 24)]
INF [hook_del_events] [Delete event 63616 @ 8-Office: "Motion: door"]
Hope it helps someone.
Attachments
zm_event_end_autodelete.py.txt
please remove the .txt at the end
(5.92 KiB) Downloaded 52 times

Post Reply

Who is online

Users browsing this forum: No registered users and 3 guests