neo-ZMES - Public Beta

Discussion topics related to mobile applications and ZoneMinder Event Server (including machine learning)
Post Reply
tsp84
Posts: 227
Joined: Thu Dec 24, 2020 4:04 am

neo-ZMES - Public Beta

Post by tsp84 »

Alright, I guess its time to open it up to the public! I am assuming IT WILL NOT be smooth sailing but I can't say for sure, everything might just work as intended for most people. I wrote up a detailed installation guide to install Zoneminder 1.36.9, Install Nvidia drivers with Cuda 10.2 and cuDNN 8.2.1 and the TPU libraries. There is a catch where you need an older version of GCC to compile Cuda 10.2 and that is all documented. I tried to also walk through some common errors and the remedies. The article also walks the user through building OpenCV 4.5.4 with GPU support, face_recognition (DLib) and openALPR with GPU support. It then walks you through installing and testing the new zmeventnotification, pyzm and mlapi (mlapi on the same host as ZMES). Mlapi gets installed as a service at the end and the new logrotation configuration is applied. I will be writing the other install guide for installing mlapi onto its own system without ZM or ZMES in the next day or two.

It shows an example of how to set up an encrypted ml_route, helps to install some Hacked powerline fonts, lsd (ls replacement) and bat (to make lof output pretty/different from mlapi/ZMES). I also include 2 shell functions to make running testing/past events easier -> es.baredebug.objdet <event ID> OR es.debug.objdet <event ID>. The difference is baredebug will not print out to console (if you are already watching the log files). Mlapi and ZMES capture stdout and stderr to the logfile so you wont miss out on errors!

Check it out, there's so much that I have changed and improved that I can't remember, the example configs have most of the examples. I will be writing new articles to go into detail on how to configure the new systems and use them effectively. the new pushover add-on in objectconfig.ini is hands down the fastest way to be notified of an object, the notification has a configurable clickable URL that will open a browser window and show the event that caused the notification!


I suggest reading through the detailed install guide, I split it up into 7 parts, 6 of 7 parts are ready for reading -> https://medium.com/@baudneo/install-zon ... fab7d7afe7

Repos:
https://github.com/baudneo/pyzm.git

https://github.com/baudneo/zmeventnotification.git

https://github.com/baudneo/mlapi.git


Here is a screenshot of the test LXC I used to install and test the new forked version. This has ZM, ZMES and mlapi all on the same system. The aliases I supplied, if you use them, and install 'bat'. Will show ZMES in the green/grey/orange and mlapi in the purple/white/red theme (and you can change themes too, they are built into bat). This is using the mlapi.logs alias and es.log.detect alias, the 'bat' version of the aliases.
Image
tsp84
Posts: 227
Joined: Thu Dec 24, 2020 4:04 am

Re: neo-ZMES - Public Beta

Post by tsp84 »

The GIF animation created for pushover is verging on the file size limit that pushover allows. I will be monitoring how many notifications are missed due to the animation being too large and work on the logic. I will probably add a check to send the jpg if the animation is to large instead of sending the file knowing pushover will reject the message altogether. At the moment it adds 4-8 frames of the labelled detected image object at the beginning of the animation. This has the effect of showing you what was detected and all the labels, polygon zones and filtered out detections (if you configure it that way) for the first 2 ish seconds of the animation and then plays the animation. I also added the ability to timestamp the animation frame by frame (and the timestamp is exactly when ZM recorded the frame, straight from the ZM DB). Timestamp size and format need to be configurable and that will happen in the future.

To test what the GIF looks like (or the objdetect.jpg - the labelled image of the detected objects) I use the MQTT add-on to send the jpg/GIF over to Home Assistant to display in the Lovelace GUI. Makes things super convenient to see without causing a hit to the pushover API. You can make an MQTT camera and subscribe that camera to the monitor's topic for JPG/GIF transport. The topic for JPG/GIF is zmes/picture/<MONITOR ID> . If you want to set up an MQTT camera in HA for monitor 6 you would subscribe the MQTT camera to the topic zmes/picture/6 , the detection data is sent to the topic zmes/data/6 . The MQTT data is a WIP, HA has a 255 character limit but it can be worked around using attributes, need to figure out a good structure with important information. Just sending the MQTT data to HA, HA logs it into its DB. You can use influxdb for HA and also use grafana or the influxdb web GUI to visualize the data and do all sorts of stuff.


Today I will be starting to finish the mlapi install guide for installing mlapi on a system that does not have ZM/ZMES installed. I will also start writing articles on how to configure the new add-ons effectively and some of the perks/nuances. I am pushing a bug fix shortly that will manage the ZMES side mlapi JWT tokens now that mlapi can be dynamic. Before, ZMES only had to worry about 1 JWT file for 1 mlapi instance. Now there can be many mlapi instances that you send requests to, hello high availability mlapi! (load balancing in the future, maybe?)
tsp84
Posts: 227
Joined: Thu Dec 24, 2020 4:04 am

Re: neo-ZMES - Public Beta

Post by tsp84 »

I am working on a new config system to make things a lot easier to work with. The thing about it is that it will move the config files (objectconfig and mlapiconfig) to YAML. Here is some testing output so far ->

Code: Select all

/home/tyler/PycharmProjects/my_mlapi/venv/bin/python /home/tyler/PycharmProjects/my_mlapi/pyzm/helpers/new_yaml.py
config:init: confirming config file exists and is a file -> '/shared/objectconfig.yml'
config:hash: the SHA-256 hexdigest for config file '/shared/objectconfig.yml' -> 1008afdb73f5ded9d6b147fa0b664d089cf1877b0b08ce699157857c985ba91d
config:zmes: added 27 default keys to the base config (these keys were missing) -> force_mpd, same_model_high_conf, skip_mons, basic_user, basic_password, max_detection_size, contained_area, hass_enabled, hass_notify, hass_cooldown, push_jpg, push_jpg_key, push_gif, push_gif_key, push_debug_device, push_cooldown, mqtt_topic, mqtt_port, tls_cert, tls_key, ml_enable, alpr_url, platerec_regions, openalpr_recognize_vehicle, openalpr_country, openalpr_state, openalpr_min_confidence!
config:zmes: confirming the configured secrets file exists and is a file -> '/shared/zm_secrets.yml'
config:zmes: secrets file confirmed! starting {[secrets]} substitution
config:zmes: successfully replaced 21 secrets in the base config -> AWS_SECRET_ACCESS_KEY, ZM_PORTAL, AWS_ACCESS_KEY_ID, MQTT_PASSWORD, ML_USER, HA_SERVER, HA_TOKEN, PUSHOVER_USER_KEY, ZM_PASSWORD, ZM_API_PORTAL, PUSHOVER_USER, PUSHOVER_PASS, ZM_USER, PLATEREC_ALPR_KEY, ZM_ML_USER, PUSHOVER_APP_TOKEN, mlapi_one_key, ZM_ML_PASSWORD, ML_PASSWORD, PUSHOVER_ERR_TOKEN, MQTT_USERNAME
config:zmes: substituting vars for the base config keys
config:zmes: successfully replaced 1 default sub var in the base config -> 'base_data_path'
config:zmes: starting {{var}} substitution
config:zmes: successfully replaced 51 sub vars in the base config -> platerec_stats, yolo4_object_framework, alpr_key, face_train_model, yolo4_object_weights, known_images_path, tpu_object_weights_mobilenetv2, unknown_images_path, save_unknown_faces, frame_strategy, same_model_sequence_strategy, face_detection_pattern, past_det_max_diff_area, platerec_min_score, gpu_max_processes, tpu_max_processes, object_min_confidence, yolo4_object_processor, model_sequence, tpu_face_weights_mobilenetv2, smart_fs_thresh, face_recog_dist_threshold, tpu_object_labels, max_detection_size, tpu_object_framework, match_past_detections, disable_locks, cpu_max_processes, openalpr_cmdline_min_confidence, fp16_target, contained_area, alpr_detection_pattern, face_num_jitters, tpu_object_weights_mobiledet, gpu_max_lock_wait, face_upsample_times, yolo4_object_config, frame_set, yolo4_object_labels, save_unknown_faces_leeway_pixels, resize, show_models, openalpr_cmdline_params, face_detection_framework, platerec_min_dscore, tpu_min_confidence, tpu_max_lock_wait, face_model, openalpr_cmdline_binary, cpu_max_lock_wait, object_detection_pattern
config:zmes: attempting to build an overrode config from monitor '1' overrides
config:zmes: monitor '1' overrides, starting {[secrets]} substitution
config:zmes: no secrets were replace during overrode config build
config:zmes: no vars were replace during overrode config build
config:zmes: found a polygon for defined zone 'front_yard' -> [(0, 877), (2170, 553), (3822, 1131), (3822, 2141), (0, 2159)]
config:zmes: detection pattern for monitor '1' defined zone 'front_yard' -> (person|dog|cat)
config:zmes: these are they keys that were overrode in the base config -> model_sequence, object_detection_pattern, frame_set, hass_notify, hass_cooldown
config:zmes: these are the keys from overrides that did not have a base value to override but are now in the config -> person_min_confidence, hass_person
ephemeral detection_patterns = {'front_yard': '(person|dog|cat)'}
self.polygons = {1: {'front_yard': [{'polygon': [(0, 877), (2170, 553), (3822, 1131), (3822, 2141), (0, 2159)], 'pattern': '(person|dog|cat)'}]}}
perf:config:zmes: 0.000497 seconds for monitor '1' overrode config build

config:zmes: attempting to build an overrode config from monitor '3' overrides
config:zmes: monitor '3' overrides, starting {[secrets]} substitution
config:zmes: no secrets were replace during overrode config build
config:zmes: no vars were replace during overrode config build
config:zmes: found a polygon for defined zone 'parking_pad' -> [(5, 408), (660, 175), (1695, 100), (1910, 275), (1910, 1070), (0, 1080)]
config:zmes: detection pattern for monitor '3' defined zone 'parking_pad' -> (person|dog|cat)
config:zmes: these are they keys that were overrode in the base config -> model_sequence, object_detection_pattern, frame_set, hass_notify, hass_cooldown
config:zmes: these are the keys from overrides that did not have a base value to override but are now in the config -> person_min_confidence
ephemeral detection_patterns = {'parking_pad': '(person|dog|cat)'}
self.polygons = {3: {'parking_pad': [{'polygon': [(5, 408), (660, 175), (1695, 100), (1910, 275), (1910, 1070), (0, 1080)], 'pattern': '(person|dog|cat)'}]}}
perf:config:zmes: 0.000253 seconds for monitor '3' overrode config build

config:zmes: attempting to build an overrode config from monitor '4' overrides
config:zmes: monitor '4' overrides, starting {[secrets]} substitution
config:zmes: no secrets were replace during overrode config build
config:zmes: no vars were replace during overrode config build
config:zmes: found a polygon for defined zone 'back_yard' -> [(0, 547), (824, 246), (1511, 161), (1706, 1070), (0, 1080)]
config:zmes: detection pattern for monitor '4' defined zone 'back_yard' -> (person)
config:zmes: these are they keys that were overrode in the base config -> model_sequence, object_detection_pattern, frame_set, hass_notify, hass_cooldown
config:zmes: these are the keys from overrides that did not have a base value to override but are now in the config -> person_min_confidence
ephemeral detection_patterns = {'back_yard': '(person)'}
self.polygons = {4: {'back_yard': [{'polygon': [(0, 547), (824, 246), (1511, 161), (1706, 1070), (0, 1080)], 'pattern': '(person)'}]}}
perf:config:zmes: 0.000228 seconds for monitor '4' overrode config build

config:zmes: attempting to build an overrode config from monitor '5' overrides
config:zmes: monitor '5' overrides, starting {[secrets]} substitution
config:zmes: no secrets were replace during overrode config build
config:zmes: no vars were replace during overrode config build
config:zmes: these are they keys that were overrode in the base config -> object_detection_pattern, model_sequence, push_cooldown, frame_strategy, frame_set, face_upsample_times, face_num_jitters
config:zmes: these are the keys from overrides that did not have a base value to override but are now in the config -> same_model_seq_strategy
ephemeral detection_patterns = {}
self.polygons = {}
perf:config:zmes: 0.000157 seconds for monitor '5' overrode config build




config:init: confirming config file exists and is a file -> '/shared/mlapiconfig.yml'
config:hash: the SHA-256 hexdigest for config file '/shared/mlapiconfig.yml' -> 78d495593def4be08bbf992e7b96ead8e625e2dc2a3e3e7c9aaf98985cbafbea
config:mlapi: added 20 default keys to the base config (these keys were missing) -> log_name, log_path, force_mpd, auth_enabled, api_portal, basic_auth_user, basic_auth_password, import_zm_zones, only_triggered_zm_zones, wait, max_detection_size, detection_sequence, detection_mode, object_framework, alpr_api_type, alpr_url, openalpr_recognize_vehicle, openalpr_country, openalpr_state, openalpr_min_confidence!
config:mlapi: confirming the configured secrets file exists and is a file -> '/shared/ml_secrets.yml'
config:mlapi: secrets file confirmed! starting {[secrets]} substitution
config:mlapi: successfully replaced 7 secrets in the base config -> MLAPI_SECRET_KEY, AWS_SECRET_ACCESS_KEY, ZM_PORTAL, AWS_ACCESS_KEY_ID, ZM_PASSWORD, ZM_USER, PLATEREC_ALPR_KEY
config:mlapi: substituting vars for the base config keys
config:mlapi: successfully replaced 1 default sub var in the base config -> 'base_data_path'
config:mlapi: starting {{var}} substitution
config:mlapi: successfully replaced 50 sub vars in the base config -> platerec_stats, yolo4_object_framework, alpr_key, face_train_model, yolo4_object_weights, known_images_path, tpu_object_weights_mobilenetv2, unknown_images_path, save_unknown_faces, frame_strategy, same_model_sequence_strategy, face_detection_pattern, past_det_max_diff_area, platerec_min_score, gpu_max_processes, tpu_max_processes, object_min_confidence, yolo4_object_processor, model_sequence, tpu_face_weights_mobilenetv2, smart_fs_thresh, face_recog_dist_threshold, tpu_object_labels, max_detection_size, tpu_object_framework, match_past_detections, disable_locks, cpu_max_processes, openalpr_cmdline_min_confidence, fp16_target, contained_area, alpr_detection_pattern, face_num_jitters, tpu_object_weights_mobiledet, gpu_max_lock_wait, face_upsample_times, yolo4_object_config, frame_set, yolo4_object_labels, save_unknown_faces_leeway_pixels, openalpr_cmdline_params, show_models, face_detection_framework, platerec_min_dscore, tpu_min_confidence, tpu_max_lock_wait, face_model, openalpr_cmdline_binary, cpu_max_lock_wait, object_detection_pattern
config:mlapi: attempting to build an overrode config from monitor '1' overrides
config:mlapi: monitor '1' overrides, starting {[secrets]} substitution
config:mlapi: no secrets were replace during overrode config build
config:mlapi: no vars were replace during overrode config build
config:mlapi: found a polygon for defined zone 'front_yard' -> [(0, 877), (2170, 553), (3822, 1131), (3822, 2141), (0, 2159)]
config:mlapi: detection pattern for monitor '1' defined zone 'front_yard' -> (person|dog|cat)
config:mlapi: these are they keys that were overrode in the base config -> object_detection_pattern, frame_set, model_sequence
config:mlapi: these are the keys from overrides that did not have a base value to override but are now in the config -> person_min_confidence
ephemeral detection_patterns = {'front_yard': '(person|dog|cat)'}
self.polygons = {1: {'front_yard': [{'polygon': [(0, 877), (2170, 553), (3822, 1131), (3822, 2141), (0, 2159)], 'pattern': '(person|dog|cat)'}]}}
perf:config:mlapi: 0.000297 seconds for monitor '1' overrode config build

config:mlapi: attempting to build an overrode config from monitor '3' overrides
config:mlapi: monitor '3' overrides, starting {[secrets]} substitution
config:mlapi: no secrets were replace during overrode config build
config:mlapi: no vars were replace during overrode config build
config:mlapi: found a polygon for defined zone 'parking_pad' -> [(5, 408), (660, 175), (1695, 100), (1910, 275), (1910, 1070), (0, 1080)]
config:mlapi: detection pattern for monitor '3' defined zone 'parking_pad' -> (person|dog|cat)
config:mlapi: these are they keys that were overrode in the base config -> frame_set, max_detection_size, object_detection_pattern
config:mlapi: these are the keys from overrides that did not have a base value to override but are now in the config -> person_min_confidence
ephemeral detection_patterns = {'parking_pad': '(person|dog|cat)'}
self.polygons = {3: {'parking_pad': [{'polygon': [(5, 408), (660, 175), (1695, 100), (1910, 275), (1910, 1070), (0, 1080)], 'pattern': '(person|dog|cat)'}]}}
perf:config:mlapi: 0.000216 seconds for monitor '3' overrode config build

config:mlapi: attempting to build an overrode config from monitor '4' overrides
config:mlapi: monitor '4' overrides, starting {[secrets]} substitution
config:mlapi: no secrets were replace during overrode config build
config:mlapi: no vars were replace during overrode config build
config:mlapi: found a polygon for defined zone 'back_yard' -> [(0, 547), (824, 246), (1511, 161), (1706, 1070), (0, 1080)]
config:mlapi: these are they keys that were overrode in the base config -> object_detection_pattern, frame_set
config:mlapi: these are the keys from overrides that did not have a base value to override but are now in the config -> person_min_confidence
ephemeral detection_patterns = {'back_yard': '(person)'}
self.polygons = {4: {'back_yard': [{'polygon': [(0, 547), (824, 246), (1511, 161), (1706, 1070), (0, 1080)], 'pattern': '(person)'}]}}
perf:config:mlapi: 0.000197 seconds for monitor '4' overrode config build

config:mlapi: attempting to build an overrode config from monitor '5' overrides
config:mlapi: monitor '5' overrides, starting {[secrets]} substitution
config:mlapi: no secrets were replace during overrode config build
config:mlapi: no vars were replace during overrode config build
config:mlapi: these are they keys that were overrode in the base config -> object_detection_pattern, model_sequence, frame_strategy, frame_set, face_upsample_times, face_num_jitters
config:mlapi: these are the keys from overrides that did not have a base value to override but are now in the config -> same_model_seq_strategy
ephemeral detection_patterns = {}
self.polygons = {}
perf:config:mlapi: 0.000148 seconds for monitor '5' overrode config build

perf: 0.288383 seconds -> total time to build both ZMES and MLAPI config objects with full configs for each overrode monitor
there is a copy of what the config file looked like at first read, a copy of the config with secrets and vars substituted before any monitor overrides
there is also a copy of a config for each monitor that is in the per monitor section that has been overridden. You can now override ml_sequence with {[secrets]} and {{vars}} embedded in substrings wherever you want! There are some keys that are off limits for per monitor overrides though, like base_data_path or other core configuration params that per monitors should not be touching.

there is also a hash function that saves the config and secrets file hash to check if the config or secrets file has changed on disk, if it has you can rebuild the config

Process finished with exit code 0
This will make the configuration an object that can be passed around freely and modified/updated easier. Just making this change will clean up a lot of the code base, make the code a lot easier to understand and more than likely optimize things a bit. I am starting to slowly rewrite the system piece by piece but will make improvements to the existing repos.

Edit: I have mlapi working with the new config system already, tmrw I will convert zmes over. It is disturbing how much code is removed using the new config system, I'm optimizing the best I can :D
juan11perez
Posts: 64
Joined: Tue Apr 27, 2021 3:41 am

Re: neo-ZMES - Public Beta

Post by juan11perez »

Good day, trying to build a docker image with zm + (your version of es + ml ).
I have a similar image ruing with the zoneminder version of these apps and it works fine now.

I've read through your files and modified to incorporate the changes to yaml for zmevent, mlapi, and objectconfig.
I've got the image running but first hiccup i have is creating the mlapi db user. I get the below error.

Code: Select all

root@54f94afd49af:/config# python3 /var/lib/zmeventnotification/mlapi_dbuser.py -u user -p password
Traceback (most recent call last):
  File "/var/lib/zmeventnotification/mlapi_dbuser.py", line 3, in <module>
    import pyzm.mlapi_helpers.mlapi_common_params as g
ModuleNotFoundError: No module named 'pyzm.mlapi_helpers'
Also there's still a number of places where your update config files point to previous config. For instance zemevents.ini has this

Code: Select all

secrets = /etc/zm/secrets.ini
Moreover perhaps it's possible to consolidate the secrets file. right now there's one for zmevent and one for mlapi.

Thank you
tsp84
Posts: 227
Joined: Thu Dec 24, 2020 4:04 am

Re: neo-ZMES - Public Beta

Post by tsp84 »

Sorry for some reasonthe notification that someone replied in this thread got lost. I am constantly updating the forked repos. I am ironing out all the YAML bugs and today will be a push that buttons up most of the errors. Yes you can consolidate the zm and mlapi secrets file as long as you change the names of the secrets to all be unique for each system. The secrets can be renamed to anything you like as long as you update where they are in the config file so they will be replaced. Example ->

Code: Select all

# zm_secrets.yml
ZM_PORTAL: https://zm.XXXX.com/zm
ZM_API_PORTAL: https://zm.XXXX.com/zm/api
ZM_USER: someDude
ZM_PASSWORD: 'somesuperlongcomplexpasswordwithsybols&*^%^%#%^$&^%*&(*^(*&('
The names of the keys are ZM_PORTAL, ZM_USER, etc. In the objectconfig.yml you can see where these secrets are embedded to be replaced as {[ZM_PORTAL]}. You could just as easily have this ->

Code: Select all

# zm_secrets.yml
HAN_sOlO: https://zm.XXXX.com/zm
ZM_API_PORTAL: https://zm.XXXX.com/zm/api
Darth_maul: someDude
ZM_PASSWORD: 'somesuperlongcomplexpasswordwithsybols&*^%^%#%^$&^%*&(*^(*&('
In your config the secret would now be {[HAN_sOlO]}, etc. So basically you can name a secret whatever. This doesn't work for variables as a lot of them are referenced by their specific name in the codebase BUT you can add your own and place them wherever to override built in ones ->

Code: Select all

# objectconfig.yml
frame_set: '{{my_variable_i_made}}'
model_sequence: face,alpr
my_variable_i_made: snapshot,233,300,370,snapshot,alarm


The new YAML syntax branch has been merged and I am ironing out bugs. I have rewrote how the configs are processed and its a lot more dynamic for MLAPI. MLAPI SHA_256 hashes the config file and secrets file and on every detection it checks to see if the hashes have changed. If the config/secrets have not changed it uses the cached configs (there are overrode configs for each monitor as well). If the files have changed it rebuilds the configs first. On average building the config object takes half a second while hashing takes 0.002 seconds. Also the pushover credentials are encrypted before being sent to MLAPI as well.

I have implemented a pushover sound system that you can see in the example .yml configs. If you upload some sounds to pushover like "A person was detected in the <front yard/back yard/driveway/porch>' and 'motion detected in the <front yard/back yard/driveway/porch>'. You can set the sound to play by default per monitor for pushover notifications. SO if monitor 1 is 'front yard' I can set it to play the 'motion detected in front yard' sound for every notification and then configure it to play 'person detected in front yard' when a person is in the detected labels for monitor 1!

Here is a list of current WIP.
- Pydantic models to validate data
- JWT error catching to enable IP logging for fail2ban jail creation (for MLAPI as ZMES uses ZM AUTH and the failed logins are recorded in web_php.log)
- stream.py has been fixed in MLAPI repo so you can use mlapi to run detections on webcam or video files / URLs (you can also run object,face,alpr all at the same time if you want)
- Fix usage of DNN_TARGET_CUDA_FP16 - This works and doesn't work..... FP32 trained models will sometimes throw 'NaN' errors.
- Better error catching and implementing a pushover error reporting system for ZMES/MLAPI
- match past detection - better logic and polish how the 'buffer' works (make it time aware)
- high confidence matching filter - polish the logic (if you look.... its WILD but it works!)
- default config build system for docker builds to customize ZMES / MLAPI pragmatically
- FastAPI/uvicorn(starlette) as framework instead of Flask/bjoern
- timestamps on objdetect.jpg and the animations are configurable. (format, show monitor name with id, text color, show bg (black rectangle text is written on), bg color and custom text scaling options dependant on dimensions of image)
- More in-depth configuration and customizations (think of things like a 'defined zones' nested dict into the per monitor settings which allows you to set the polygon of a defined zone based on the resolution of the source; polygon is different sizes for 1080p vs 4K)
- ADVANCED users options
juan11perez
Posts: 64
Joined: Tue Apr 27, 2021 3:41 am

Re: neo-ZMES - Public Beta

Post by juan11perez »

Thank you for your response.
With regards to the error i'm getting when creating the mlapi db user, would that be associated with your version of pyzm?
If yes, the problem would originate from your mlapi requirements txt which will pull the original pyzm? I'm installing your pyzm first, then mlapi, so that would overwrite your pyzm?

If this makes no sense my apologies, I'm just an enthusiast learning by trial and error.

I'd also like to share that I use gotify instead of pushover. host it myself and send as many messages as I want. I found a gotify.sh example and I just add it in the hooks folder instead of pushover. see below script

Code: Select all

#!/bin/bash
ZM_TOKEN='token'
EVENT_ID=`echo ${6} | awk -F'/' '{ print $8 }'`
MESSAGE=`echo ${4} | sed -e 's/.*] \(.*\)Motion.*/\1/'`
CAMERA=$3

if [[ "$MESSAGE" != "Motion: All" ]] && [[ "$3" == "doorbell" ]]; then
# if [[ "$MESSAGE" != "Motion: All" ]]; then

  curl --request POST \
    --url 'https://goti.mydomain/message?token=AsicUwGdOcr8If_' \
    --header 'content-type: application/json' \
    --data "{
        \"title\": \"ZM136 - ${CAMERA^} Camera\",
        \"message\": \"${MESSAGE^}\n\n ![Camera Image](https://zm.mydomain/zm/index.php?view=image&eid=${EVENT_ID}&fid=objdetect&popup=1&token=${ZM_TOKEN})\",
        \"priority\": 6,
          \"extras\": {
        \"client::display\": {
          \"contentType\": \"text/markdown\",
        \"client::notification\": {
          \"click\": \"{ 'url': 'https://zm.mydomain/zm/cgi-bin/nph-zms?mode=jpeg&frame=1&replay=none&source=event&event=${EVENT_ID}&connkey=77493&token=${ZM_TOKEN}'}\"
        }
      }
    }
  }"

fi
tsp84
Posts: 227
Joined: Thu Dec 24, 2020 4:04 am

Re: neo-ZMES - Public Beta

Post by tsp84 »

I messed up the requirents and fixed them but am not certain if I pushed that fox yet. I'm just setting things up for the day here and will review this error and reply with a proper response shortly.

I will add the provided script functionality into the base code and integrate it with the Hass sensors/ independent cooldown sensors.

The requirements got messed up by me exploring some.of the IDE options, there's some options you can configure for it to create requirements files and for some reason it added all sorts of stuff that wasn't there before.
tsp84
Posts: 227
Joined: Thu Dec 24, 2020 4:04 am

Re: neo-ZMES - Public Beta

Post by tsp84 »

Code: Select all

root@54f94afd49af:/config# python3 /var/lib/zmeventnotification/mlapi_dbuser.py -u user -p password
Traceback (most recent call last):
  File "/var/lib/zmeventnotification/mlapi_dbuser.py", line 3, in <module>
    import pyzm.mlapi_helpers.mlapi_common_params as g
ModuleNotFoundError: No module named 'pyzm.mlapi_helpers'
This happened because setup.py wasn't updated to create that module properly in the module structure. This change here moves the mlapi db.py and mlapi_common_params.py modules to the pyzm repos. MLAPI needs pyzm to function so no matter what, even if on a system without ZM, pyzm must be installed. Might as well have all the floating modules in a central location, all the ML is handled by pyzm anyways (This structure will change yet)

My version of ZMES is most deff not compatible with the source components its NEO or source, cant be mixed (API calls added/different behaviour than source). I am pushing to the repo now and it should solve those issues BUT I haven't tested installing the new codebase from scratch so there may be some hiccups.
juan11perez
Posts: 64
Joined: Tue Apr 27, 2021 3:41 am

Re: neo-ZMES - Public Beta

Post by juan11perez »

Thank you for your reply.
So pyzm wont be installed by mlapi. Will it be installed by zmeventnotifications?
Installing pyzm separately solves potential confusion.
tsp84
Posts: 227
Joined: Thu Dec 24, 2020 4:04 am

Re: neo-ZMES - Public Beta

Post by tsp84 »

I am now slowly crawling through the code and trying to optimize as much as I can. In the last few days I have gotten a detection that previously took me 20 ish seconds to do, down to 10-12..... which is incredible!

I now thread the logger completely and creating some of the objects is threaded. I updated logic so ZMES doesn't do a ZM API login or call unless it needs to. It now makes a threaded call to create the API object and a thread for the logger itself. There is possibly more optimizing to be done there but so far I am off to a roaring start.

I am most proud of this next optimization. Before when using mlapi and zmes, zmes would send mlapi the detection. Mlapi would then request images from the ZM API, run detections on them and reply with the labels, confidences and bounding box locations of the detections. ZMES would then grab the matching image from the ZM API and draw the bounding boxes and text onto the image to create objdetect.jpg. So mlapi hits ZM API for images, replies to ZMES and ZMES hitos the ZM API for the image that the detection matched on (Double hit).

Now, mlapi responds with a multipart encoded message that carries the detection data and the matching image. ZMES does not need to make an API call unless it is creating animations or some other things (which will be optimized soon). The speed is starting to get out of this world and I am running my stuff on older hardware. Suffice it to say I'm getting excited about the performance I am observing.

Onwards and upwards!
tsp84
Posts: 227
Joined: Thu Dec 24, 2020 4:04 am

Re: neo-ZMES - Public Beta

Post by tsp84 »

juan11perez wrote: Sun Nov 07, 2021 3:16 am Thank you for your reply.
So pyzm wont be installed by mlapi. Will it be installed by zmeventnotifications?
Installing pyzm separately solves potential confusion.
When I say pyzm and mlapi I am referring to the forked versions, I will specify 'source' if it is the source repo versions, just FYI to anyone reading any of these posts.


the reason pyzm is not a requirement for mlapi is that pyzm has packages in the pip repos from the source. The forked pyzm is not in the pip repos as of yet. The script in zmeventnotification does install the forked pyzm or you can clone the pyzm repo, cd into the pyzm repo and use pip to install the forked repo. I am staring work on the gotify addition and I will make a mlapi install script as well.

Code: Select all

git clone https://github.com/baudneo/pyzm.git
cd pyzm
pip3 install ./setup.py
I suppose I should make an official package once things are stable with a name change that is readily apparent so users do not get confused between this fork and source.

Edit: Gotify is pretty sweet, super simple. I am just reading up and testing its capabilities. Like does it support gifs or mp4s? sounds? The URL method seems to imply the clickable link is the whole notification but I haven't tested enough yet to know for sure. Shouldnt take too long to get it done.
Well it seems there is no image support for gotify at all internally? It seems you can display images via markdown. So I guess overriding the whole click on notification isn't such a crazy idea. When the notification is clicked it opens a browser window to view the event is what I am going to try and achieve. Same system as Pushover that will need a custom user in ZM API with view privs of events and stream only. Ill try and get the arkdown images working but that will depend on grabbing images externally I think, unless there is some /etc/hosts shack or something similar on the host to grab images locally from an internal flask web server that hosts the objdetect images.
juan11perez
Posts: 64
Joined: Tue Apr 27, 2021 3:41 am

Re: neo-ZMES - Public Beta

Post by juan11perez »

Ok, so If I install your zmeventnotification via your install.sh then I don't have to bother with installing pyzm. I install your mlapi, then your zmeventnotification and that's it.

With gotify I get the image on the notification via their android apk. when i click on the android notification shortcut it opens a web-browser where I can see the zoneminder video clip. It works really well for me.
tsp84
Posts: 227
Joined: Thu Dec 24, 2020 4:04 am

Re: neo-ZMES - Public Beta

Post by tsp84 »

juan11perez wrote: Mon Nov 08, 2021 1:09 pm Ok, so If I install your zmeventnotification via your install.sh then I don't have to bother with installing pyzm. I install your mlapi, then your zmeventnotification and that's it.

With gotify I get the image on the notification via their android apk. when i click on the android notification shortcut it opens a web-browser where I can see the zoneminder video clip. It works really well for me.


LOL, I keep changing things so much and switching between branches I overwrote some things. The zmeventnotificatiuon script does not Install neo-pyzm, neither does mlapi. I will fix that right now and push the new optimizations into the master branch.

Sorry about all the confusion :(

As for gotify, yes the script you provided does override the clickable notification and the markdown. I do see that gotify accepts markdown and with markdown it is possible to serve local images from the disk instead of the api. Therefore I should be able to embed the objdetect.jpg and possibly some other goodies into the main body of the message. I just installed a gotify container and am playing around.
juan11perez
Posts: 64
Joined: Tue Apr 27, 2021 3:41 am

Re: neo-ZMES - Public Beta

Post by juan11perez »

Thank you. So to confirm neo needs zmevent, mlapi and pyzm installations from your repo.
tsp84
Posts: 227
Joined: Thu Dec 24, 2020 4:04 am

Re: neo-ZMES - Public Beta

Post by tsp84 »

You don't need mlapi, you can run zmeventnotification and pyzm. Mlapi is recommended though (even when installing on same host as zmeventnotification) because mlapi keeps the models loaded on memory so the 2nd+ndetextions are way faster then the first.

For gotify, do you want the existing arguments to stay the same as how the pushapi_plugin args? I can also add some extra bits if it makes life easier for you (ZM TOKEN?)

# Arguments passed
# ARG1 = event ID
# ARG2 = monitor ID
# ARG3 = monitor name
# ARG4 = Alarm cause
# ARG5 = type of event (event_start or event_end)
# ARG6 (Optional) = image path
Post Reply