match_past_detections granularity?

Discussion topics related to mobile applications and ZoneMinder Event Server (including machine learning)
Post Reply
tsp84
Posts: 227
Joined: Thu Dec 24, 2020 4:04 am

match_past_detections granularity?

Post by tsp84 »

I haven't quite tested this yet but here is the scenario I am trying to have play out. I have a camera pointed at the driveway, I would like to be notified the first time a vehicle pulls up but not every time there is an alarm and the car has been sitting there parked for awhile. I also would like person detection and this is where my confusion begins. So if I set match_past_detections=yes and a person is seen in event 1, that means if a person is seen in event 2, that it will be ignored and I wont get a push? (Car gone so only person detected in event 1 and 2)
My understanding is to be notified in the 2nd event, there would need to be another (different type?) object detected alongside the person or a completely different object detected altogether. If this is true is it possible to add granularity to the match_past_detections and also add like a "jitter" counter? Example: It only matches past detections for "car" and/or "truck". Also instead of just not being seen in 1 past ml_sequence loop, can there be a threshold number for that? like 2 loops? Sometimes the way my camera angle and garage are my car isn't detected every time there is an event, It could be fine tuned a bit with a counter.
tsp84
Posts: 227
Joined: Thu Dec 24, 2020 4:04 am

Re: match_past_detections granularity?

Post by tsp84 »

So I've found a few thing related to how it works. It depends on the location of the detected object and in the settings you can set the % it moves by. I think that the bounding box part of the image is directly copied into a binary file and compared somehow. I'm not certain on how though because my programming knowledge is basic.
User avatar
asker
Posts: 1553
Joined: Sun Mar 01, 2015 12:12 pm

Re: match_past_detections granularity?

Post by asker »

match_past_detections parameters don't compare binary images.
Basically, when an objects are detected, this option saves the bounding boxes along with labels (person/car etc)
New detections are compared with this saved data, that is:
1. Make sure the box being checked has the same label (example, if in the past you detected a dog at the same location where you have now detected a person, it will not filter)
2. If the labels are the same, check the bounding boxes to see if they are "similar" - that is where the past_det_max_diff_area parameter is used. You will notice depending on lighting conditions, the same object can have different bounding boxes at the same spot. So this lets you add a variance in terms of area of difference (comments on top of this parameter explains more)
I no longer work on zmNinja, zmeventnotification, pyzm or mlapi. I may respond on occasion based on my available time/interest.

Please read before posting:
How to set up logging properly
How to troubleshoot and report - ES
How to troubleshoot and report - zmNinja
ES docs
zmNinja docs
tsp84
Posts: 227
Joined: Thu Dec 24, 2020 4:04 am

Re: match_past_detections granularity?

Post by tsp84 »

I've enabled it and it works very well, thanks for the explanation. I haven't poked around and looked at the pyzm stuff much.
tsp84
Posts: 227
Joined: Thu Dec 24, 2020 4:04 am

Re: match_past_detections granularity?

Post by tsp84 »

Is there any chance of adding a buffer type thing? Like if not detected in x tries then remove the prior matched data? Right now this does help but I still get alot of car detected notifications for garage/driveway. I think that something like this would be helpful to tune those extra notifications out.
tsp84
Posts: 227
Joined: Thu Dec 24, 2020 4:04 am

Re: match_past_detections granularity?

Post by tsp84 »

I'm not sure what's going on but it's hit and miss. I've been studying python so I can try and follow the scripts and maybe modify them for this use case. As my screenshot shows I am getting back to back events with same obj detected and need to figure out how to set up logging so I can see what pyzm past detections is doing.Image
Attachments
Screenshot_20210311-215133_Chrome.jpg
Screenshot_20210311-215133_Chrome.jpg (589.1 KiB) Viewed 5739 times
User avatar
asker
Posts: 1553
Joined: Sun Mar 01, 2015 12:12 pm

Re: match_past_detections granularity?

Post by asker »

Compare debug logs - pictures won’t help much.
Take a look at debug logs for 9658 and go through all steps and see what matching debug logs state
I no longer work on zmNinja, zmeventnotification, pyzm or mlapi. I may respond on occasion based on my available time/interest.

Please read before posting:
How to set up logging properly
How to troubleshoot and report - ES
How to troubleshoot and report - zmNinja
ES docs
zmNinja docs
tsp84
Posts: 227
Joined: Thu Dec 24, 2020 4:04 am

Re: match_past_detections granularity?

Post by tsp84 »

I cant seem to get verbose logging for pyzm to see what its doing when the past detections is called. I have enabled logging in zoneminder as described in the readthesocs for ES. I have restarted zoneminder and I have also stopped the ES and started it with --debug manually and the only message I ever see about past detections is the

Code: Select all

03/14/21 16:38:39 zmesdetect_m4[2049] INF zm_detect.py:441 [Removing matches to past detections]

I have a weird error im just noticing now as well with CUBLAS, I hadnt noticed it before. It is still detecting things at the moment on that monitor ->

Code: Select all

03/14/21 16:55:55 zmesdetect_m4[3179] ERR detect_sequence.py:461 [Error running model: OpenCV(4.4.0) /root/opencv-4.4.0/modules/dnn/src/cuda4dnn/csl/cublas.hpp:78: error: (-217:Gpu API call) CUBLAS_STATUS_NOT_INITIALIZED in function 'UniqueHandle'
]
I edited the debug levels to 9 in ZM -> options and in the ES server settings. Is there a seperate pyzm log somewhere? I havnt seen it in the /var/logs/zm folder.
tsp84
Posts: 227
Joined: Thu Dec 24, 2020 4:04 am

Re: match_past_detections granularity?

Post by tsp84 »

Here is a gist with the zmes_detect log and the console output from --debug
I cant see any info about the past detections logging. im guessing its pyzm logs but idk where those are or how to enable them properly.
https://gist.github.com/tylersprice84/c ... c2773ff0cb
User avatar
asker
Posts: 1553
Joined: Sun Mar 01, 2015 12:12 pm

Re: match_past_detections granularity?

Post by asker »

What happens when you invoke zm_detect manually using --debug as described in https://zmeventnotification.readthedocs ... g-problems ?

Code: Select all

sudo -u www-data /var/lib/zmeventnotification/bin/zm_detect.py --config /etc/zm/objectconfig.ini --debug --eventid <eid>  --monitorid <mid> --eventpath=/tmp
I no longer work on zmNinja, zmeventnotification, pyzm or mlapi. I may respond on occasion based on my available time/interest.

Please read before posting:
How to set up logging properly
How to troubleshoot and report - ES
How to troubleshoot and report - zmNinja
ES docs
zmNinja docs
tsp84
Posts: 227
Joined: Thu Dec 24, 2020 4:04 am

Re: match_past_detections granularity?

Post by tsp84 »

That did the trick, running them manually back to back removes the past matched car, ill play around with it and see how it goes.

Code: Select all

03/15/21 13:26:54 zmesdetect_m6[29338] DBG2 image_manip.py:174 [trying to load /var/lib/zmeventnotification/images/monitor-6-data.pkl]

03/15/21 13:26:54 zmesdetect_m6[29338] DBG1 image_manip.py:255 [past detection car@[(359, 138), (797, 138), (797, 370), (359, 370)] approximately matches car@[(358, 137), (800, 137), (800, 373), (358, 373)] removing]
When I run it back to back on the 2 events in the screenshot it doesnt remove it on the next event. I am guessing its the movement % setting so im trying playing with that. Here is log from the 2 events in screenshot. https://gist.github.com/tylersprice84/1 ... 2cdbf92b58

Does zmMagick have a place to ask questions? Ive blended some videos already but I havnt tried the search/find yet. For the image do we keep the image the same WxH and crop out the picture we want it to monitor and leave it in the same x,y it would be in its original WxH? Or do we just crop the part of the image we want it to monitor for in its own size and it figures it out on its own?
User avatar
asker
Posts: 1553
Joined: Sun Mar 01, 2015 12:12 pm

Re: match_past_detections granularity?

Post by asker »

That did the trick, running them manually back to back removes the past matched car, ill play around with it and see how it goes.
That may mean that you forgot to add _zm_detect to the list of LOG_DEBUG_TARGET entries in ZM log web option

I have:

Code: Select all

_zmesdetect|_zmeventnotification|_zm_mlapi
You can also add this to the [general] section of objectconfig.ini

Code: Select all

pyzm_overrides={'log_level_debug':5}
When I run it back to back on the 2 events in the screenshot it doesnt remove it on the next event. I am guessing its the movement % setting so im trying playing with that. Here is log from the 2 events in screenshot.
I've added some more logs to master that will give you some more clarity on what is going on
Does zmMagick have a place to ask questions? Ive blended some videos already but I havnt tried the search/find yet. For the image do we keep the image the same WxH and crop out the picture we want it to monitor and leave it in the same x,y it would be in its original WxH? Or do we just crop the part of the image we want it to monitor for in its own size and it figures it out on its own?
No :-) I've completely run out of time to support zmMagik and don't plan to continue it in the near future.
I no longer work on zmNinja, zmeventnotification, pyzm or mlapi. I may respond on occasion based on my available time/interest.

Please read before posting:
How to set up logging properly
How to troubleshoot and report - ES
How to troubleshoot and report - zmNinja
ES docs
zmNinja docs
tsp84
Posts: 227
Joined: Thu Dec 24, 2020 4:04 am

Re: match_past_detections granularity?

Post by tsp84 »

I figured so about zmMagick, very neat utility though. After rereading the README I see it specifies for same WxH and x,y of cropped image.

I am playing with the percentages for movement and its working well, my only request would be to see if it could be expanded a bit to only match certain objects at certain percentages? say just the car|truck = 30% and person = 8% and dog|cat = 5%. Something like that?
tsp84
Posts: 227
Joined: Thu Dec 24, 2020 4:04 am

Re: match_past_detections granularity?

Post by tsp84 »

I modified image_manip.py and common_params.py in order to implement a granularity system for "past_det_max_diff_area". Also required adding a few config options in object_config.ini. Per monitor overrides work as expected. Code might be a bit ugly but it works so far. Deff needs some optimization and is only set for car, truck, motorbike, boat, cat, dog right now. Is this something anyone else would be interested in?

When <object> is NOT overridden in objectconfig.ini

Code: Select all

03/21/21 20:21:17 zmesdetect_m4[2629] INF zm_detect.py:433 [Removing matches to past detections]

03/21/21 20:21:17 zmesdetect_m4[2629] DBG2 image_manip.py:211 [trying to load /var/lib/zmeventnotification/images/monitor-4-data.pkl]

03/21/21 20:21:17 zmesdetect_m4[2629] DBG4 image_manip.py:240 [There are NO overrides for the object detected: car, using past_det_max_diff_area: 5%]

03/21/21 20:21:17 zmesdetect_m4[2629] DBG4 image_manip.py:260 [process_past_detections: use_percent:True, max_diff_area:5]

03/21/21 20:21:17 zmesdetect_m4[2629] DBG1 image_manip.py:298 [past detection car@[(70, 90), (268, 90), (268, 304), (70, 304)] approximately matches car@[(70, 90), (268, 90), (268, 304), (70, 304)] removing]
when <object> is overridden via objectconfig.ini ->

Code: Select all

03/21/21 20:24:16 zmesdetect_m4[2737] INF zm_detect.py:433 [Removing matches to past detections]

03/21/21 20:24:16 zmesdetect_m4[2737] DBG2 image_manip.py:211 [trying to load /var/lib/zmeventnotification/images/monitor-4-data.pkl]

03/21/21 20:24:16 zmesdetect_m4[2737] DBG4 image_manip.py:237 [There ARE overrides for the object detected: car, using its value: 69420px]

03/21/21 20:24:16 zmesdetect_m4[2737] DBG4 image_manip.py:260 [process_past_detections: use_percent:False, max_diff_area:69420]

03/21/21 20:24:16 zmesdetect_m4[2737] DBG1 image_manip.py:298 [past detection car@[(70, 90), (268, 90), (268, 304), (70, 304)] approximately matches car@[(70, 90), (268, 90), (268, 304), (70, 304)] removing]

Code: Select all

03/21/21 20:26:50 zmesdetect_m4[2912] DBG4 utils.py:469 [[monitor-4] overrides key:object_detection_pattern with value:(car|truck|motorbike|person|dog|cat)]

03/21/21 20:26:50 zmesdetect_m4[2912] DBG4 utils.py:469 [[monitor-4] overrides key:match_past_detections with value:yes]

03/21/21 20:26:50 zmesdetect_m4[2912] DBG4 utils.py:469 [[monitor-4] overrides key:diff_area_car with value:69420px]

03/21/21 20:26:50 zmesdetect_m4[2912] DBG4 utils.py:469 [[monitor-4] overrides key:diff_area_truck with value:43%]

03/21/21 20:26:50 zmesdetect_m4[2912] DBG4 utils.py:469 [[monitor-4] overrides key:diff_area_motorbike with value:30%]

03/21/21 20:26:50 zmesdetect_m4[2912] DBG4 utils.py:469 [[monitor-4] overrides key:diff_area_boat with value:30%]

03/21/21 20:26:50 zmesdetect_m4[2912] DBG4 utils.py:469 [[monitor-4] overrides key:diff_area_person with value:5%]

03/21/21 20:26:50 zmesdetect_m4[2912] DBG4 utils.py:469 [[monitor-4] overrides key:diff_area_dog with value:5%]

03/21/21 20:26:50 zmesdetect_m4[2912] DBG4 utils.py:469 [[monitor-4] overrides key:diff_area_cat with value:5%]
tsp84
Posts: 227
Joined: Thu Dec 24, 2020 4:04 am

Re: match_past_detections granularity?

Post by tsp84 »

I created a pull request, here is the link to see what I changed if you want to play around/optimize.
https://github.com/pliablepixels/zmeven ... n/pull/377
Post Reply