Pushed earlier this morning, mostly prepared for the public beta if anyone is interested. There are so many changes I can not list them all but I will try and make a list and organize it into something I can document. For the time being, this will not be 'new user' friendly, You must have some understanding of how this software system works and how to install and troubleshoot some things.
My end goal is to make ZM, ZMES, MLAPI more accessible to the general public and to have some sort of active development on this subsystem of ZM where enhancements can be made/requested. I am trying to write a mlapi install script to make installing mlapi easier for the public. I am also hoping to write a new python module to integrate ZM with Home Assistant better, Frigate is getting all the love there because it's so easy to install and run via HA, we should be focusing some attention on that area. The people doing home automation will most definitely financially support a good project (ZoneMinder), we just need to make it far more accessible to the point and click crowd.
There are configuration-breaking changes! Do not expect the beta testing to be super smooth! The core logic and structures of the system may still change drastically. I am considering changing configuration files (objectconfig.ini and mlapiconfig.ini) to YAML syntax
!SECRETS have been changed to {[secrets]}. Before there was a very small chance of errors (if a password started with ! for example). This lays the groundwork to allow substituting {[secrets]} when that secret is embedded in a substring (or nested into per monitor overrides, like a per-monitor override of ml_sequence)
The 'pattern' for each model has been changed -> ml_sequence:object:general:pattern used to have a substitution var of {{pattern}}. I have changed it to {{<model>_detection_pattern}}. So object would have {{object_detection_pattern}}, face would have {{face_detection_pattern}}.
The newest (and absolutely clutch) addition I have made to ZMES is how MLAPI and ZMES communicate with each other. In the original version, you would set up mlapi and hard configure that mlapi instance to only receive detections from 1 ZMES host. Using the new system, 'ml_routes' in objectconfig.ini and 'zmes_keys' in mlapiconfig.ini, ZMES encrypts all the configuration info that mlapi needs to reply to ZMES in the detection request using symmetrical key encryption. Let us analyze the example configuration for a quick breakdown.
The URLs and credentials are encrypted for transit, only the 'name' is unencrypted so mlapi can look in zmes_keys for the key to decrypt.
Snippit from objectconfig.ini ->
Code: Select all
ml_routes = [
# {[SECRETS]} work as long as they are the only value, they cannot be embedded in a substring ->
; CORRECT -> 'name': '{[secret_name]}',
; INCORRECT -> 'name': 'I have a really cool {[secret_name]}',
# {{VARS}} also work in here if needed, they can be in a substring and be correctly replaced.
{
# weight of route, lower is more important.
'weight': 1,
'name': 'mlapi_two',
'gateway': 'http://10.0.0.32:5000/api/v1',
'user': '{[ML_USER]}',
'pass': '{[ML_PASSWORD]}',
'zm_user': '{[ZM_ML_USER]}',
'zm_pass': '{[ZM_ML_PASSWORD]}',
'zm_basic_user': '',
'zm_basic_pass': '',
'enc_key': '{[mlapi_one_key]}',
},
{
# weight of route, lower is more important.
'weight': 0,
'name': 'mlapi_one',
'gateway': 'http://10.0.0.32:5000/api/v1',
# user/pass is the MLAPI user and pass
'user': '{[ML_USER]}',
'pass': '{[ML_PASSWORD]}',
# ZM user/pass is for token (API 2.0+) auth, these are the credentials passed to the mlapi host
# that mlapi will use to login to this ZM hosts API. This makes mlapi dynamic!
'zm_user': '{[ZM_ML_USER]}',
'zm_pass': '{[ZM_ML_PASSWORD]}',
# zm_basic is for using ZM basic auth
'zm_basic_user': '',
'zm_basic_pass': '',
# encryption key here must match the encryption key in mlapiconfig.ini zmes_keys
'enc_key': '{[mlapi_one_key]}',
},
] # IMPORTANT to keep ending brace indented
- weight: the importance of the route, lower numbers are the higher priority so 0 is the top priority. If 2+ routes have the same weight, it's a gamble which one will be chosen
- name: The name of the route ** THIS IS IMPORTANT, it must match the name of the key in zmes_keys so mlapi knows which key to use
- gateway: this is the URL of the mlapi host (ml_gateway basically)
- user: This is the ML username to log in to mlapi (the username and password you created when adding a user to the mlapi 'DB')
- pass: The password for mlapi user
- zm_user: The ZM API user you want mlapi to login with (This can be the same user that ZMES uses or a new user specifically for mlapi)
- zm_pass: The ZM API username password
- zm_basic_user: ZM API basic auth user if using basic auth
- zm_basic_pass: ZM API username password if using basic auth
- enc_key: The encryption key that 'get_encryption_key.py' output ** IMPORTANT: This key must be the same here and in zmes_keys otherwise decryption WILL FAIL
Snippet from mlapiconfig.ini ->
Code: Select all
zmes_keys = {
# Format -> 'name of key/route/zmes host': '<key>',
'mlapi_one': 'tlYoF3YT2vgrFPt8SHmEkw8bX0ISG4x0TwOVnbMS4oA=',
} # Always keep the closing brace indented
- 'mlapi_one': See how in ml_routes we named 1 route mlapi_one, this is how mlapi will know which encryption key to use
- 'NAME OF ROUTE': '<ENCRYPTION KEY>',
You can change the encryption key at any time by running get_encryption_key.py and copying the output key into ml_routes and zmes_keys for the specified route. Remember that this is symmetrical key encryption, the encryption key and decryption key are the same.
Here is some log output from the ZMES side ->
Code: Select all
10/19/21 13:46:56.961577 zmesdetect_m4[563468] DBG2 zm_detect:333 [**** 'http://10.0.0.32:5000/api/v1/detect/object?
type=object' using auth_header={'Authorization': 'Bearer eyJ0eXAiOiJKV1QiLCJhbGc...'}
*** params={'delete': True, 'response_format': 'zm_detect'}
****** JSON: {'version': '7.0.0', 'mid': '4', 'reason': 'Motion Back Fence,Walkway,Fire pit / Dog area', 'stream': '
40689', 'stream_options': {'frame_strategy': 'first', 'frame_set': 'snapshot,70,110,160,snapshot,alarm,210,280', 'co
ntig_frames_before_error': 2, 'max_attempts': 3, 'sleep_between_attempts': 2, 'sleep_between_frames': 0.04, 'sleep_b
etween_snapshots': 1, 'smart_fs_thresh': '5', 'resize': '800', 'api': '<REMOVED FOR HTTP TRANSFER>', 'polygons': [{'
name': 'back_yard', 'value': [(0, 547), (824, 246), (1511, 161), (1706, 1070), (0, 1080)], 'pattern': '(person)'}],
'mid': '4', 'reply_data': {'name': 'mlapi_one', 'auth_type': b'gAAAAABhbyCwZBOBvrdDh8FhRbcpHKo8pejg3ybwttzwp-2fzPKLb
q6xMgpATRuA1Iurx-m9pKkSq6VVUmp2uXbGU9UldzGYyg==', 'api_url': b'gAAAAABhbyCwsCpLKqhcBZVwA5FM2K-up7AhSVTVRIPss9TVMd-tc
NrNSH6Ohi2k0n-fFbLUC3i5ef4uoLlRA8ZT8tIwlyodwQiiA-om15AQElRSFnsJv54=', 'portal_url': b'gAAAAABhbyCw-xemxEkbKlf79q-amA
t1uFlDJ8aWEC-TKtM8jQNxglzYKbjc3s6rv_9DRxyUK_d-WjRLInEqdsad6BdBRoTv8wnsRPcOmdlaFgSlykdGa0U=', 'user': b'gAAAAABhbyCwC
o_KSkXiSgshEpLPc0yKQFl2xfhSrM5yCj6CLJIgEMrdaumAE4mndHOg5yF_gKKmGqCOOmp7uyT0SP6onKN1Kg==', 'pass': b'gAAAAABhbyCwQz0k
lECz-wvJTvfDAlm0G-npsOSGhI-MwigkQbjQCjHykXVEUOitVNtpDTTMQokQxEGVNm6aP5_L0Rb6JXiKTMkniDkLfXLzo36MhA0HamY2dLMGnbnbDCWq
3sRZUwPM1u6mK1sLIr7dnE5rQAAnxw==', 'b_user': None, 'b_pass': None}}, 'ml_overrides': {'enable': False, 'model_sequen
ce': 'object', 'object': {'pattern': '(person|car|motorbike|bus|truck|boat|dog|cat)'}, 'face': {'pattern': '.*'}, 'a
lpr': {'pattern': '.*'}}, 'sub_options': {'push_errors': 'yes', 'push_options': {'token': '<redacted>', 'user': <redacted>', 'sound': None}}}
]
And here are the logs from MLAPI side of that request ->
Code: Select all
10/19/21 13:46:57.057547 zm_mlapi[17070] DBG1 mlapi:165 [mlapi: ZMES sent encrypted data! Checking the keystore for '
mlapi_one']
10/19/21 13:46:57.060633 zm_mlapi[17070] DBG1 mlapi:180 [decrypted={'auth_type': 'token', 'api_url': 'https://<redacted>.com/zm/api', 'portal_url': 'https://<redacted>.com/zm', 'user': '<redacted>', 'pass': '<redacted>'}]
MLAPI uses that encrypted info to reply to that specific detection request and access that ZM API for information about the event!
I am currently spinning up LXC's to test installing ZoneMinder and then my forked ZMES and MLAPI (dubbed neo-zmeventnotfication, neo-mlapi, neo-pyzm-> neo_ZMES - to differentiate from source). Once I iron out the bugs there, things should be ready to beta test. I am writing up a tutorial on Medium for a full ZM 1.36.x branch install onto Ubuntu 21.04 using neo-zmeventnotification with GPU/TPU support and MLAPI. There will be a section on the new config options.
** I think I have found a change in OpenCV code that causes breaking changes when using GPU and yolov4 for detections. If using OpenCV 4.5.3-dev + yolo will throw a scalar error. There is a post on the forums here about it and the OP fixed it. I ran into this problem when testing the neo-ZMES system in the new LXC I am using for testing. Forum post ->
viewtopic.php?f=43&t=31129&sid=c987d358 ... 8f363c62b2
My stable LXC uses OpenCV 4.5.1 and the scalar error is not there, if I use the fix on the instance using OpenCV 4.5.1 it throws a different error all together -> only integer scalar arrays can be converted to a scalar index.
I have implemented a new fix for now that checks the OpenCV version and uses the fix based on if the version of opencv is 4.5.2 or above. I am unsure which version causes the breaking change so some testing will need to be done. Right now the fix is working for the LXC using OpenCV 4.5.4 and the 4.5.1 version instance uses the old code.