neo-ZMES - Public Beta

Discussion topics related to mobile applications and ZoneMinder Event Server (including machine learning)
tsp84
Posts: 227
Joined: Thu Dec 24, 2020 4:04 am

Re: neo-ZMES - Public Beta

Post by tsp84 »

I am assuming some sort of minor formatting error in the config or secrets file but am uncertain.

I've seen this happen also when a {{var}} or {[secret]} hasn't been quoted in the config or sometimes in a unquoted string if it contains certain characters.

I quote my passwords and weird strings that have special characters. Example ->

Code: Select all

Images-path: {{images_dir}}/testpath
# that would cause an error because the var or secret is unquoted

Images-path: "{{images_dir}}/testpath"
# correct

Some_pw: "a-+_]=7537[(+jfhsieugwh6936"
Also make sure you have the most recent copy of neo-pyzm (this shouldn't matter at the moment as the 9nly push I made yesterday would be to fix the BGR colored frames)

Worst case is sanitize your config and secrets and post them in a gist to double check.
tsp84
Posts: 227
Joined: Thu Dec 24, 2020 4:04 am

Re: neo-ZMES - Public Beta

Post by tsp84 »

Actually hold on a few the error isn't making sense to me. It seems like mlapi is trying to decrypt a string instead of an encrypted byte string. Did you update mlapi and pyzm within the last 2 days?

Edit: nvm this is on my end. git mismanagement, there's an old copy of zm_detect.py in there for some reason. Line 422 in repo is ancient lol, in my codebase line 422 isn't in the same function of zm_detect.py. Im pushing now.

Sorry I'm a straight up rookie with github and it shows.
tsp84
Posts: 227
Joined: Thu Dec 24, 2020 4:04 am

Re: neo-ZMES - Public Beta

Post by tsp84 »

IDK what happened but the code bases are messed right up. I pulled the repos and am getting errors running the code from the repos. Some of the code there is shit I fixed days ago and already pushed so I have no idea how there is code in there that doesn't match up to anything I'm working on atm.

Im going to delete the repos and refork and just do 1 ugly commit to get it to where it needs to be. Give me a few hours, Im not very impressed with whatever is happening here.
tsp84
Posts: 227
Joined: Thu Dec 24, 2020 4:04 am

Re: neo-ZMES - Public Beta

Post by tsp84 »

Things should be back up now. I pushed my current code base. Be aware you probably need to remove the ZM_ML_USER and ZM_ML_PASSWORD from your objectconfig.yml ml_routes. The script now sends the auth token encrypted for mlapi to use. This should save some time as mlapi will not need to login and request an auth tkn, it can just decrypt the token and its info and start making calls right away.

Other than that, things should be ok
juan11perez
Posts: 64
Joined: Tue Apr 27, 2021 3:41 am

Re: neo-ZMES - Public Beta

Post by juan11perez »

Thank you. I've rebuild with your latest push. Couple comments

in pyzm install.sh and make_package.sh are gone
in mlapi get_encryption_key.py is gone

Good thing i forked your repo the day before :D

I''m now running with your latest and the error persists

Code: Select all

root@511d36db3e78:/var/lib/zmeventnotification# sudo -u www-data /var/lib/zmeventnotification/bin/zm_detect.py --config /etc/zm/objectconfig.yml --debug --eventid 2469
11/15/21 16:28:00.083449 zmesdetect_m6[18653] DBG1 new_yaml.py:242->[config:init: the supplied config file exists -> '/etc/zm/objectconfig.yml']
11/15/21 16:28:00.16236 zmesdetect_m6[18653] DBG1 new_yaml.py:265->[config::init: default configuration built (no secrets or substitution vars replaced, yet!)]
11/15/21 16:28:00.163382 zmesdetect_m6[18653] DBG1 new_yaml.py:170->[config:zmes:proc: 36 built in default keys added to the 'base' config -> ['force_mpd', 'same_model_high_conf', 'skip_mons', 'force_live', 'sanitize_str', 'version', 'basic_user', 'basic_password', 'image_path', 'max_detection_size', 'hass_enabled', 'hass_people', 'hass_notify', 'hass_cooldown', 'push_err_device', 'push_jpg', 'push_jpg_key', 'push_gif', 'push_gif_key', 'push_debug_device', 'push_cooldown', 'mqtt_topic', 'mqtt_broker', 'mqtt_port', 'tls_cert', 'tls_key', 'fp16_target', 'alpr_api_type', 'alpr_url', 'openalpr_recognize_vehicle', 'openalpr_country', 'openalpr_state', 'openalpr_min_confidence', 'smart_fs_thresh', 'frame_strategy', 'same_model_sequence_strategy']]
11/15/21 16:28:00.163792 zmesdetect_m6[18653] DBG1 new_yaml.py:114->[config:zmes:proc: substituting '{{variables}}' for the 'base' config keys]
11/15/21 16:28:00.164661 zmesdetect_m6[18653] DBG1 new_yaml.py:136->[config:zmes: successfully replaced 3 default sub vars in the base config -> ['base_data_path', 'yolo4_models', 'coral_models']]
11/15/21 16:28:00.165133 zmesdetect_m6[18653] DBG1 new_yaml.py:285->[config:zmes: the configured secrets file exists and is a file -> '/etc/zm/zm_secrets.yml']
11/15/21 16:28:00.16549 zmesdetect_m6[18653] DBG1 new_yaml.py:291->[config:zmes: starting '{[secrets]}' substitution]
11/15/21 16:28:00.17145 zmesdetect_m6[18653] DBG1 new_yaml.py:324->[config:zmes: successfully replaced 17 secrets in the base config -> ['MQTT_PASSWORD', 'ML_USER', 'PUSHOVER_USER_KEY', 'PLATEREC_ALPR_KEY', 'HA_SERVER', 'HA_TOKEN', 'ZM_PORTAL', 'PUSHOVER_ERR_TOKEN', 'MQTT_USERNAME', 'PUSHOVER_USER', 'ZM_PASSWORD', 'ZM_USER', 'mlapi_one_key', 'ML_PASSWORD', 'ZM_API_PORTAL', 'PUSHOVER_PASS', 'PUSHOVER_APP_TOKEN']]
11/15/21 16:28:00.171812 zmesdetect_m6[18653] DBG1 new_yaml.py:329->[config:zmes: there are 2 secrets configured that have no substitution candidates in the base config or the secrets file -> ['AWS_ACCESS_KEY_ID', 'AWS_SECRET_ACCESS_KEY']]
11/15/21 16:28:00.173338 zmesdetect_m6[18653] DBG1 new_yaml.py:358->[config:zmes: starting '{{variable}}' substitution]
11/15/21 16:28:00.177684 zmesdetect_m6[18653] DBG1 new_yaml.py:375->[config:zmes: successfully replaced 51 sub vars in the base config -> ['face_detection_framework', 'object_detection_pattern', 'show_models', 'yolo4_object_labels', 'alpr_detection_pattern', 'yolo4_object_config', 'unknown_images_path', 'tpu_min_confidence', 'openalpr_cmdline_min_confidence', 'face_model', 'yolo4_object_framework', 'known_images_path', 'tpu_object_weights_mobiledet', 'save_unknown_faces_leeway_pixels', 'contained_area', 'tpu_object_framework', 'tpu_tf2_mobilenetv2', 'save_unknown_faces', 'disable_locks', 'platerec_min_score', 'tpu_max_lock_wait', 'yolo4_object_weights', 'face_num_jitters', 'object_min_confidence', 'platerec_stats', 'face_upsample_times', 'same_model_sequence_strategy', 'alpr_key', 'tpu_object_labels', 'openalpr_cmdline_params', 'frame_set', 'gpu_max_lock_wait', 'face_recog_dist_threshold', 'max_detection_size', 'smart_fps_thresh', 'tpu_max_processes', 'tpu_efficientdet_lite3', 'yolo4_object_processor', 'gpu_max_processes', 'past_det_max_diff_area', 'frame_strategy', 'cpu_max_processes', 'tpu_object_weights_mobilenetv2', 'openalpr_cmdline_binary', 'face_train_model', 'model_sequence', 'cpu_max_lock_wait', 'face_detection_pattern', 'match_past_detections', 'platerec_min_dscore', 'tpu_face_weights_mobilenetv2']]
11/15/21 16:28:00.17929 zmesdetect_m6[18653] DBG1 new_yaml.py:187->[config:zmes:proc: Base config has been built, all properly configured {[secrets]} and {{vars}} have been replaced! Remember you must manually build per monitor overrode configurations]
11/15/21 16:28:00.179608 zmesdetect_m6[18653] DBG1 zm_detect.py:551->[perf:zmes: building the intial config took 0.096178 ]
11/15/21 16:28:00.18001 zmesdetect_m6[18653] DBG1 new_yaml.py:747->[zmes:api create: building ZM API Session]
11/15/21 16:28:00.184804 zmesdetect_m6[18653] DBG1 api.py:114->[api: SSL certificate verification disabled (encryption enabled, vulnerable to MITM attacks)]
11/15/21 16:28:00.185503 zmesdetect_m6[18653] DBG1 api.py:283->[api: no token found, trying user/pass for login]
11/15/21 16:28:00.244716 zmesdetect_m6[18653] DBG1 api.py:320->[api: detected API ver 2.0+, using token system]
11/15/21 16:28:00.245195 zmesdetect_m6[18653] DBG1 api.py:335->[api: access token expires on: 2021-11-15 18:28:00.245179 (7200s)]
11/15/21 16:28:00.245569 zmesdetect_m6[18653] DBG1 api.py:347->[api: refresh token expires on: 2021-11-16 16:28:00.245560 (86400s)]
11/15/21 16:28:00.259318 zmesdetect_m6[18653] DBG1 new_yaml.py:780->[zmes:api create: ZM API created]
11/15/21 16:28:00.504115 zmesdetect_m6[18653] DBG1 ZMLog:336->[------|  FORKED NEO --- app->Hooks: 7.0.3 - pyzm: 1.0.3 - ES: 7.0.3 - OpenCV:4.5.4 |------]
11/15/21 16:28:00.504524 zmesdetect_m6[18653] DBG1 ZMLog:336->[zmes: Setting up signal handlers for log 'rotation' and log 'interrupt']
11/15/21 16:28:00.345426 zmesdetect_m6[18653] DBG1 zm_detect.py:663->[zmes: this is a 'PAST' (debugging?) event!]
11/15/21 16:28:00.505492 zmesdetect_m6[18653] DBG1 ZMLog:336->[|----------= Encrypted Route Name: 'mlapi_one' | Gateway URL: 'http://192.168.1.100:5003/api/v1' | Weight: 0 =----------|]
11/15/21 16:28:00.346951 zmesdetect_m6[18653] DBG1 zm_detect.py:103->[zmes:mlapi:login: No need to login, access token is valid for 2516.3268706798553 sec]
11/15/21 16:28:00.458468 zmesdetect_m6[18653] DBG1 zm_detect.py:327->[
** Gateway URL: 'http://192.168.1.100:5003/api/v1/detect/object?type=object' using auth_header={'Authorization': 'Bearer eyJ0eXAiOiJKV1QiLCJhbGc......'} 
**** params={'delete': True, 'response_format': 'zm_detect'}
****** JSON: stream: 2469 - mid: 6 - reason: None - stream options: {'frame_strategy': 'first', 'frame_set': 'snapshot, alarm, snapshot', 'contig_frames_before_error': 2, 'delay_between_attempts': 2.143256, 'max_attempts': 3, 'delay_between_snapshots': 1, 'smart_fps_thresh': '5', 'save_frames': 'no', 'save_frames_dir': None, 'PAST_EVENT': True} - files: {}
]
11/15/21 16:28:00.507088 zmesdetect_m6[18653] DBG1 ZMLog:336->[Connected to ZoneMinder Logging system with user 'www-data' -> '/var/log/zm/zmesdetect_m6.log']
11/15/21 16:28:00.482914 zmesdetect_m6[18653] DBG1 zm_detect.py:374->[zmes:mlapi: error during post to mlapi host-> keys must be str, int, float, bool or None, not bytes]
11/15/21 16:28:00.483731 zmesdetect_m6[18653] DBG1 zm_detect.py:774->[perf:zmes:mlapi: Monitor: office (6)->'Event': 2469 mlapi detection took: 0.137127]
11/15/21 16:28:00.484539 zmesdetect_m6[18653] DBG1 zm_detect.py:1542->[zmes: no predictions returned from detections]
11/15/21 16:28:00.486125 zmesdetect_m6[18653] DBG1 zm_detect.py:1610->[perf:zmes:FINAL: 'Monitor': office (6)->'Event': 2469 [PAST event] [total:0.40451] [detection:0.137127] [after core detection: 0.000886] ]
root@511d36db3e78:/var/lib/zmeventnotification# 
tsp84
Posts: 227
Joined: Thu Dec 24, 2020 4:04 am

Re: neo-ZMES - Public Beta

Post by tsp84 »

I'll push a debug copy that will print out the keys that it's complaining about. I need to see what the keys are before I can say what the issue is as i am not quite sure which key it could be. I've had this code base running 2 days straight with 0 issues so I'm thinking it's got to be a small oversight I made. I'll push a debug branch to the repo that will contain some print statements to help figure out what keys are malformed.
Last edited by tsp84 on Mon Nov 15, 2021 6:13 pm, edited 1 time in total.
tsp84
Posts: 227
Joined: Thu Dec 24, 2020 4:04 am

Re: neo-ZMES - Public Beta

Post by tsp84 »

The codebase is *almost* the exact same, just with a few more optimizations I made to test (mainly sending mlapi the auth token instead of login credentials).

The ugly commit history is gone :oops: instead its just 1 massive commit that completely changes everything about anything and is probably impossible to track the differences efficiently 8)

I'll add those few missing files to master and make a debug branch right away here. Im a bit stumped and interested in whats going on lol, boggled.

Edit: 0.0xxx seconds to build config..... you're moving at the speed of light, lol

Can you post the output from the mlapi logs as well? IDK if mlapi caught the error and logged it, it might have some clues. The error isn't clear enough for me to pinpoint exactly where this could be going on, my suspicion is something to do with decrypting the keys because that's the only time that mlapi should be seeing a bytestring.
juan11perez
Posts: 64
Joined: Tue Apr 27, 2021 3:41 am

Re: neo-ZMES - Public Beta

Post by juan11perez »

Where can i find that log? i checked /var/logs/zm and it's not theere

getting this error with the pyzm update just committed

Code: Select all

root@511d36db3e78:/var/lib/zmeventnotification# sudo -u www-data /var/lib/zmeventnotification/bin/zm_detect.py --config /etc/zm/objectconfig.yml --debug --eventid 2469
Traceback (most recent call last):
  File "/var/lib/zmeventnotification/bin/zm_detect.py", line 28, in <module>
    import pyzm.api
ModuleNotFoundError: No module named 'pyzm'
tsp84
Posts: 227
Joined: Thu Dec 24, 2020 4:04 am

Re: neo-ZMES - Public Beta

Post by tsp84 »

should be in the mlapi base_data_path/logs/zm_mlapi.log
Thats the default, ill change that to check if its installed on a ZM host and save to /var/log/zm. You can alsoset a log path and log name in the mlapiconfig.yml

Thats an odd error... IDK what is going on with github or pycharm, All I did was create a DEBUG branch a push the print statement versions to that branch. Im starting to miss using sublime and CLI git.

Ill spin up a new venv on a new LXC and work through the errors.
tsp84
Posts: 227
Joined: Thu Dec 24, 2020 4:04 am

Re: neo-ZMES - Public Beta

Post by tsp84 »

I made a new LXC, cloned the repos and found a couple of structure problems (missing dirs). That is all taken care of, I rm -rf'ed everything out of the new LXC, sudo pip3 uninstall neo-pyzm for a fresh slate. I recloned the three repos, I used the install.sh script in zmeventnotification/ to install neo-pyzm 1.0.3.

I set up the config files, only a couple of things needed filling out to proper settings and ran a 'local' detection and that worked. I then configured mlapi, I only needed to create a DB user and update the zm_secrets.yml for ML_USER, I left the default encryption key and most of everything else as it was. I ran a detection on CPU and things went ok until I hit the actual detection (I didn't properly install OpenCV in this LXC). To make sure that the OpenCV error was because of the LXC install, I reinstalled my production environment to the current codebase and it is running detections without issue.

Log output from the new test LXC

Code: Select all

tyler@ZMES-test:~/git/zmeventnotification$   11/15/21 14:46:26.556355 zm_mlapi[2868] DBG1 mlapi_db:90 [mlapi db: login SUCCEEDED for user -> 'test' (127.0.0.1) [correct password]]
  11/15/21 14:46:26.681706 zm_mlapi[2868] DBG1 mlapi:373 [mlapi:detect: The detection request is for MLAPI DB user 'test' using IP address -> 127.0.0.1]
  11/15/21 14:46:26.6933 zm_mlapi[2868] DBG1 mlapi:429 [mlapi:detect: the config file has not changed since it was last read!]
  11/15/21 14:46:26.703352 zm_mlapi[2868] DBG1 mlapi:432 [mlapi:detect: the secrets file has not changed since it was last read!]
  11/15/21 14:46:26.712687 zm_mlapi[2868] DBG1 mlapi:442 [perf:mlapi:detect: total time to hash config/secrets -> 0.019821]
  11/15/21 14:46:26.721215 zm_mlapi[2868] DBG1 mlapi:450 [mlapi:detect: monitor 1 has no overrode configuration built, using 'base' config...]
  11/15/21 14:46:26.730983 zm_mlapi[2868] DBG2 mlapi:461 [mlapi:detect: encrypted credentials received, checking keystore for 'mlapi_one']
  11/15/21 14:46:26.74659 zm_mlapi[2868] DBG2 mlapi:478 [mlapi:detect: credentials have been decrypted, attempting to login to the ZoneMinder API for 'mlapi_one']
  11/15/21 14:46:26.756408 zm_mlapi[2868] DBG2 mlapi:512 [api: SSL certificate verification disabled (encryption enabled, vulnerable to MITM attacks)]
  11/15/21 14:46:26.764758 zm_mlapi[2868] DBG2 mlapi:512 [api:kick start: ZMES sent the AUTH JWT, no need to re login to ZM API]
  11/15/21 14:46:26.776439 zm_mlapi[2868] DBG2 mlapi:516 [mlapi:detect: found 'stream_sequence' in './mlapiconfig.yml']
  11/15/21 14:46:26.786082 zm_mlapi[2868] DBG1 detect_sequence:855 [detect: this is a past event, optimizing settings and workflow for speed]
  11/15/21 14:46:26.7951 zm_mlapi[2868] DBG2 Media:117 [media: setting 'image' as type for event -> '21']
  11/15/21 14:46:26.804265 zm_mlapi[2868] DBG2 Media:149 [media: processing a maximum of 3 frames-> ['snapshot', ' alarm', ' snapshot']]
  11/15/21 14:46:26.816542 zm_mlapi[2868] DBG2 Media:180 [media: using API calls for stream -> 21]
  11/15/21 14:46:26.826033 zm_mlapi[2868] DBG2 detect_sequence:904 [detect:portalocker: using manual locking for single model -> 'object']
  11/15/21 14:46:26.836046 zm_mlapi[2868] DBG1 Media:453 [media:read:image: about to process first frame!]
  11/15/21 14:46:26.846189 zm_mlapi[2868] DBG2 Media:488 [media:read:image: Event: 21 - converting 'snapshot' to a frame ID -> s-11]
  11/15/21 14:46:26.855521 zm_mlapi[2868] DBG2 Media:618 [api:make_req: 'get'->http://10.0.0.6/zm/index.php?view=image&eid=21&fid=11 query={'token': 'eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJab25lTWluZGVyIi
  wiaWF0IjoxNjM3MDEyNzg1LCJleHAiOjE2MzcwMTk5ODUsInVzZXIiOiJ0ZXN0dXNlciIsInR5cGUiOiJhY2Nlc3MifQ.8fSVkbW9_IMmhyYJNelOzqKMSN2GeSglW3RKo1TSmTA...'}]
  11/15/21 14:46:26.930661 zm_mlapi[2868] DBG1 Media:679 [media:read:image: Image returned from ZM API dimensions: (1080, 1920)]
  11/15/21 14:46:26.947698 zm_mlapi[2868] DBG2 detect_sequence:945 [detect: no polygons/zones specified, adding 'full_image' as polygon @ [(0, 0), (1920, 0), (1920, 1080), (0, 1080)]]
  11/15/21 14:46:26.957902 zm_mlapi[2868] DBG2 detect_sequence:1045 [frame: s-11 [strategy:'first'] (1 of 3) - model: 'object' [strategy:'first'] (1 of 1) - sequence: 'DarkNet::v4 Pre-Trained' [strategy:'most'] (1
   of 1)]
  11/15/21 14:46:26.969994 zm_mlapi[2868] DBG2 object:34 [yolo:portalock: Waiting for 'pyzm_uid33_CPU_lock' portalock...]
  11/15/21 14:46:26.980018 zm_mlapi[2868] DBG2 object:37 [yolo:portalock: got 'pyzm_uid33_CPU_lock']
  11/15/21 14:46:26.989088 zm_mlapi[2868] DBG1 yolo:196 [yolo: 'DarkNet::v4 Pre-Trained' (CPU) - input image 1920*1080 - resized by  model_width/height to: 416*416]
  11/15/21 14:46:26.996243 zm_mlapi[2868] ERR yolo:206 [yolo: OpenCV 4.5.4+ 'getUnconnectedOutLayers()' API fix did not work! Please do some testing to see if you have installed the opencv-python pip/pip3 packages
   by mistake if you compiled OpenCV for GPU/CUDA! see  https://forums.zoneminder.com/viewtopic.php?f=33&t=31293
  invalid index to scalar variable.]
Log output from my production env, First block is ZMES as I have mlapi on its own host in production. Second block is mlapi logs

Code: Select all

 11/15/21 14:43:24.933558 zmesdetect_m1[219833] DBG1 ZMLog:336 [Connected to ZoneMinder Logging system with user 'www-data' -> '/var/log/zm/zmesdetect_m1.log']
  11/15/21 14:43:24.972704 zmesdetect_m1[219833] DBG2 zm_detect:142 [zmes:mlapi:login: writing MLAPI AUTH JWT to disk]
  11/15/21 14:43:25.054025 zmesdetect_m1[219833] DBG2 zm_detect:318 [
  ** Gateway URL: '<sanitized>' using auth_header={'Authorization': 'Bearer eyJ0eXAiOiJKV1QiLCJhbGc......'}
  **** params={'delete': True, 'response_format': 'zm_detect'}
  ****** JSON: stream: 46513 - mid: 1 - reason: None - stream options: {'frame_strategy': 'first', 'frame_set': 'snapshot,70,snapshot,120,150,alarm', 'contig_frames_before_error': 2, 'delay_between_attempts': 2, '
  max_attempts': 3, 'delay_between_snapshots': 1, 'smart_fps_thresh': '5', 'save_frames': 'no', 'save_frames_dir': None, 'polygons': [{'name': 'front_yard', 'value': [(0, 877), (2170, 553), (3822, 1131), (3822, 21
  41), (0, 2159)], 'pattern': '(person|dog|cat)'}], 'PAST_EVENT': True, 'resize': 800} - files: {}
  ]
  11/15/21 14:43:39.591915 zmesdetect_m1[219833] DBG1 zm_detect:383 [zmes:mlapi: parsed JSON detection data in multipart response]
  11/15/21 14:43:39.592877 zmesdetect_m1[219833] DBG1 zm_detect:379 [zmes:mlapi: parsed the matched image from bytes to a jpeg encoded array in multipart response]
  11/15/21 14:43:39.603399 zmesdetect_m1[219833] DBG1 zm_detect:765 [perf:zmes:mlapi: Monitor: Front (1)->'Event': 46513 mlapi detection took: 15.093031]
Mlapi production logs ->

Code: Select all

 11/15/21 14:55:05.528733 zm_mlapi[10504] DBG1 mlapi:373 [mlapi:detect: The detection request is for MLAPI DB user 'Tyler' using IP address -> 10.0.0.30]                                      10.0.0.32 ⭐ ─╯
  11/15/21 14:55:05.531548 zm_mlapi[10504] DBG1 mlapi:429 [mlapi:detect: the config file has not changed since it was last read!]
  11/15/21 14:55:05.533902 zm_mlapi[10504] DBG1 mlapi:432 [mlapi:detect: the secrets file has not changed since it was last read!]
  11/15/21 14:55:05.536082 zm_mlapi[10504] DBG1 mlapi:442 [perf:mlapi:detect: total time to hash config/secrets -> 0.005053]
  11/15/21 14:55:05.53829 zm_mlapi[10504] DBG1 mlapi:447 [mlapi:detect: monitor 1 has an overrode configuration built, switching to it...]
  11/15/21 14:55:05.540419 zm_mlapi[10504] DBG2 mlapi:461 [mlapi:detect: encrypted credentials received, checking keystore for 'mlapi_one']
  11/15/21 14:55:05.546448 zm_mlapi[10504] DBG2 mlapi:478 [mlapi:detect: credentials have been decrypted, attempting to login to the ZoneMinder API for 'mlapi_one']
  11/15/21 14:55:05.548788 zm_mlapi[10504] DBG2 mlapi:512 [api: SSL certificate verification disabled (encryption enabled, vulnerable to MITM attacks)]
  11/15/21 14:55:05.54896 zm_mlapi[10504] DBG2 mlapi:512 [api:kick start: ZMES sent the AUTH JWT, no need to re login to ZM API]
  11/15/21 14:55:05.551112 zm_mlapi[10504] DBG2 mlapi:516 [mlapi:detect: found 'stream_sequence' in '/shared/mlapiconfig.yml']
  11/15/21 14:55:05.553317 zm_mlapi[10504] DBG1 mlapi:534 [mlapi:detect: ZMES has resize=800 configured, propagating...]
  11/15/21 14:55:05.557249 zm_mlapi[10504] DBG1 detect_sequence:855 [detect: this is a past event, optimizing settings and workflow for speed]
  11/15/21 14:55:05.561324 zm_mlapi[10504] DBG2 Media:117 [media: setting 'image' as type for event -> '46513']
  11/15/21 14:55:05.565916 zm_mlapi[10504] DBG2 Media:150 [media: processing a maximum of 6 frames-> ['snapshot', '70', '120', 'snapshot', '150', 'alarm']]
  11/15/21 14:55:05.569509 zm_mlapi[10504] DBG2 Media:181 [media: using API calls for stream -> 46513]
  11/15/21 14:55:05.571965 zm_mlapi[10504] DBG2 detect_sequence:904 [detect:portalocker: using manual locking for single model -> 'object']
  11/15/21 14:55:05.574527 zm_mlapi[10504] DBG1 Media:454 [media:read:image: about to process first frame!]
  11/15/21 14:55:05.576986 zm_mlapi[10504] DBG2 Media:489 [media:read:image: Event: 46513 - converting 'snapshot' to a frame ID -> s-104]
  11/15/21 14:55:05.579569 zm_mlapi[10504] DBG2 Media:619 [api:make_req: 'get'-><sanitized>/index.php?view=image&eid=46513&fid=104 query={'token': 'eyJ0eXAiOiJKV1QiLCJhbGciO...'}]
  11/15/21 14:55:06.828111 zm_mlapi[10504] DBG2 pyzm_utils:385 [resize:img: success using resize=800.0 - original dimensions: 3840*2160 - resized dimensions: 450*800]
  11/15/21 14:55:06.834675 zm_mlapi[10504] DBG2 detect_sequence:283 [detect: rescaled polygons ('down') using factors: x=0.2083  y=0.2083
  *** OLD: [[{'name': 'front_yard', 'value': [(0, 877), (2170, 553), (3822, 1131), (3822, 2141), (0, 2159)], 'pattern': '(person|dog|cat)'}]]
  ****** NEW:[[{'name': 'front_yard', 'value': [(0, 182), (452, 115), (796, 235), (796, 446), (0, 449)], 'pattern': '(person|dog|cat)'}]]]
  11/15/21 14:55:06.837116 zm_mlapi[10504] DBG2 detect_sequence:1045 [frame: s-104 [strategy:'first'] (1 of 6) - model: 'object' [strategy:'first'] (1 of 1) - sequence: 'coral::MobileNETv2-SSD TensorFlow 2.0 300x3
  00' [strategy:'most'] (1 of 2)]
  11/15/21 14:55:06.839702 zm_mlapi[10504] DBG2 coral_edgetpu:197 [coral: model dimensions requested -> 300*300]
  11/15/21 14:55:06.847672 zm_mlapi[10504] DBG2 object:34 [coral:portalock: Waiting for 'pyzm_uid1000_TPU_lock' portalock...]
  11/15/21 14:55:06.850506 zm_mlapi[10504] DBG2 object:37 [coral:portalock: got 'pyzm_uid1000_TPU_lock']
  11/15/21 14:55:06.85283 zm_mlapi[10504] DBG1 coral_edgetpu:215 [coral: 'coral::MobileNETv2-SSD TensorFlow 2.0 300x300' input image (w*h): 800*450 resized by model_width/height to 300*300]
  11/15/21 14:55:07.055861 zm_mlapi[10504] DBG1 coral_edgetpu:232 [perf:coral: 'coral::MobileNETv2-SSD TensorFlow 2.0 300x300' detection took: 200.57 ms]
  11/15/21 14:55:07.058543 zm_mlapi[10504] DBG2 object:57 [coral:portalock: released 'pyzm_uid1000_TPU_lock']
  11/15/21 14:55:07.060874 zm_mlapi[10504] DBG2 coral_edgetpu:253 [coral: The image was resized before processing by the 'model width/height', scaling bounding boxes in image back up by factors of -> x=2.667 y=1.5
  ]
  11/15/21 14:55:07.063253 zm_mlapi[10504] DBG1 coral_edgetpu:262 [coral: returning ['car', 'car', 'car'] -- [[131, 52, 261, 108], [371, 56, 448, 90], [205, 39, 288, 68]] -- [0.8828125, 0.828125, 0.7890625]]
  11/15/21 14:55:07.065521 zm_mlapi[10504] DBG2 detect_sequence:1098 [detect: model: 'object' seq: 'coral::MobileNETv2-SSD TensorFlow 2.0 300x300' found 3 detections -> car, car, car]
  11/15/21 14:55:07.067897 zm_mlapi[10504] DBG1 detect_sequence:339 [>>> detected 'car (1/3)' confidence: 0.88]
  11/15/21 14:55:07.070428 zm_mlapi[10504] DBG1 detect_sequence:387 ['car (1/3)' minimum confidence found: (object_min_conf:sequence->coral::MobileNETv2-SSD TensorFlow 2.0 300x300) -> '0.6']
  11/15/21 14:55:07.072837 zm_mlapi[10504] DBG4 detect_sequence:403 ['car (1/3)' max area of detected object found (max_detection_size:sequence->coral::MobileNETv2-SSD TensorFlow 2.0 300x300) -> '90%']
  11/15/21 14:55:07.075185 zm_mlapi[10504] DBG2 detect_sequence:414 [max size: converted 90% of 800*450->360000 to 324000.0 pixels]
  11/15/21 14:55:07.077586 zm_mlapi[10504] DBG2 detect_sequence:433 [checking if 'car (1/3)' @ [(131, 52), (261, 52), (261, 108), (131, 108)] is inside polygon/zone 'front_yard' located at [(0, 182), (452, 115), (
  796, 235), (796, 446), (0, 449)]]
  11/15/21 14:55:07.079944 zm_mlapi[10504] DBG2 detect_sequence:525 [intersection: 'car (1/3)' does not intersect zone: front_yard, removing...]
  11/15/21 14:55:07.082291 zm_mlapi[10504] DBG2 detect_sequence:333 [detection: 'car (1/3)' has FAILED filtering]
  11/15/21 14:55:07.084618 zm_mlapi[10504] DBG1 detect_sequence:339 [>>> detected 'car (2/3)' confidence: 0.83]
  11/15/21 14:55:07.086992 zm_mlapi[10504] DBG1 detect_sequence:387 ['car (2/3)' minimum confidence found: (object_min_conf:sequence->coral::MobileNETv2-SSD TensorFlow 2.0 300x300) -> '0.6']
  11/15/21 14:55:07.089333 zm_mlapi[10504] DBG4 detect_sequence:403 ['car (2/3)' max area of detected object found (max_detection_size:sequence->coral::MobileNETv2-SSD TensorFlow 2.0 300x300) -> '90%']
  11/15/21 14:55:07.091663 zm_mlapi[10504] DBG2 detect_sequence:414 [max size: converted 90% of 800*450->360000 to 324000.0 pixels]
  11/15/21 14:55:07.094077 zm_mlapi[10504] DBG2 detect_sequence:433 [checking if 'car (2/3)' @ [(371, 56), (448, 56), (448, 90), (371, 90)] is inside polygon/zone 'front_yard' located at [(0, 182), (452, 115), (79
  6, 235), (796, 446), (0, 449)]]
  11/15/21 14:55:07.096453 zm_mlapi[10504] DBG2 detect_sequence:525 [intersection: 'car (2/3)' does not intersect zone: front_yard, removing...]
  11/15/21 14:55:07.098958 zm_mlapi[10504] DBG2 detect_sequence:333 [detection: 'car (2/3)' has FAILED filtering]
  11/15/21 14:55:07.101436 zm_mlapi[10504] DBG1 detect_sequence:339 [>>> detected 'car (3/3)' confidence: 0.79]
  11/15/21 14:55:07.104004 zm_mlapi[10504] DBG1 detect_sequence:387 ['car (3/3)' minimum confidence found: (object_min_conf:sequence->coral::MobileNETv2-SSD TensorFlow 2.0 300x300) -> '0.6']
  11/15/21 14:55:07.106541 zm_mlapi[10504] DBG4 detect_sequence:403 ['car (3/3)' max area of detected object found (max_detection_size:sequence->coral::MobileNETv2-SSD TensorFlow 2.0 300x300) -> '90%']
  11/15/21 14:55:07.108917 zm_mlapi[10504] DBG2 detect_sequence:414 [max size: converted 90% of 800*450->360000 to 324000.0 pixels]
  11/15/21 14:55:07.111414 zm_mlapi[10504] DBG2 detect_sequence:433 [checking if 'car (3/3)' @ [(205, 39), (288, 39), (288, 68), (205, 68)] is inside polygon/zone 'front_yard' located at [(0, 182), (452, 115), (79
  6, 235), (796, 446), (0, 449)]]
  11/15/21 14:55:07.113933 zm_mlapi[10504] DBG2 detect_sequence:525 [intersection: 'car (3/3)' does not intersect zone: front_yard, removing...]
  11/15/21 14:55:07.116302 zm_mlapi[10504] DBG2 detect_sequence:815 [detection: 'car (3/3)' has FAILED filtering]
  11/15/21 14:55:07.118546 zm_mlapi[10504] DBG2 detect_sequence:1045 [frame: s-104 [strategy:'first'] (1 of 6) - model: 'object' [strategy:'first'] (1 of 1) - sequence: 'DarkNet::v4 Pre-Trained' [strategy:'most']
  (2 of 2)]
  11/15/21 14:55:07.121413 zm_mlapi[10504] DBG2 object:34 [yolo:portalock: Waiting for 'pyzm_uid1000_GPU_lock' portalock...]
  11/15/21 14:55:07.124133 zm_mlapi[10504] DBG2 object:37 [yolo:portalock: got 'pyzm_uid1000_GPU_lock']
  11/15/21 14:55:07.126501 zm_mlapi[10504] DBG1 yolo:196 [yolo: 'DarkNet::v4 Pre-Trained' (GPU) - input image 800*450 - resized by  model_width/height to: 416*416]
  11/15/21 14:55:07.166633 zm_mlapi[10504] DBG2 object:57 [yolo:portalock: released 'pyzm_uid1000_GPU_lock']
  11/15/21 14:55:07.622198 zm_mlapi[10504] DBG2 yolo:309 [perf:yolo:GPU: 'DarkNet::v4 Pre-Trained' detection took: 493.22 ms]
  11/15/21 14:55:07.624731 zm_mlapi[10504] DBG1 yolo:318 [yolo: ['car', 'car', 'car'] -- [[133, 52, 259, 108], [371, 55, 445, 91], [208, 40, 288, 68]] -- [0.9894788861274719, 0.9852612018585205, 0.9837948679924011
  ]]
  11/15/21 14:55:07.628698 zm_mlapi[10504] DBG2 detect_sequence:1098 [detect: model: 'object' seq: 'DarkNet::v4 Pre-Trained' found 3 detections -> car, car, car]
  11/15/21 14:55:07.631284 zm_mlapi[10504] DBG1 detect_sequence:339 [>>> detected 'car (1/3)' confidence: 0.99]
  11/15/21 14:55:07.633889 zm_mlapi[10504] DBG1 detect_sequence:387 ['car (1/3)' minimum confidence found: (object_min_conf:sequence->DarkNet::v4 Pre-Trained) -> '0.6']
  11/15/21 14:55:07.636498 zm_mlapi[10504] DBG2 detect_sequence:433 [checking if 'car (1/3)' @ [(133, 52), (259, 52), (259, 108), (133, 108)] is inside polygon/zone 'front_yard' located at [(0, 182), (452, 115), (
  796, 235), (796, 446), (0, 449)]]
  11/15/21 14:55:07.638904 zm_mlapi[10504] DBG2 detect_sequence:525 [intersection: 'car (1/3)' does not intersect zone: front_yard, removing...]
  11/15/21 14:55:07.641297 zm_mlapi[10504] DBG2 detect_sequence:333 [detection: 'car (1/3)' has FAILED filtering]
  11/15/21 14:55:07.643818 zm_mlapi[10504] DBG1 detect_sequence:339 [>>> detected 'car (2/3)' confidence: 0.99]
  11/15/21 14:55:07.646342 zm_mlapi[10504] DBG1 detect_sequence:387 ['car (2/3)' minimum confidence found: (object_min_conf:sequence->DarkNet::v4 Pre-Trained) -> '0.6']
juan11perez
Posts: 64
Joined: Tue Apr 27, 2021 3:41 am

Re: neo-ZMES - Public Beta

Post by juan11perez »

Good day. Completed a new install and had the following issues.
This command is not creating the mlapi user
cd /config && python3 /var/lib/zmeventnotification/mlapi_dbuser.py -u mlapitest -p test123
It creates and empty db.json file

and it maybe related to the fact that latest pyzm is producing this error:

Code: Select all

root@d58573099cd9:~/pyzm# sudo -u www-data /var/lib/zmeventnotification/bin/zm_detect.py --config /etc/zm/objectconfig.yml --debug --eventid 2
Traceback (most recent call last):
  File "/var/lib/zmeventnotification/bin/zm_detect.py", line 28, in <module>
    import pyzm.api
ModuleNotFoundError: No module named 'pyzm'
root@d58573099cd9:~/pyzm# 
so installed your previous pyzm, which I forked before and it works for zmevent but produces the same error for mlapi login

Code: Select all

root@d58573099cd9:~/pyzm# sudo -u www-data /var/lib/zmeventnotification/bin/zm_detect.py --config /etc/zm/objectconfig.yml --debug --eventid 2

11/16/21 21:05:08.424519 zmesdetect_m3[1758] DBG1 new_yaml.py:242->[config:init: the supplied config file exists -> '/etc/zm/objectconfig.yml']

11/16/21 21:05:08.497158 zmesdetect_m3[1758] DBG1 new_yaml.py:265->[config::init: default configuration built (no secrets or substitution vars replaced, yet!)]

11/16/21 21:05:08.497942 zmesdetect_m3[1758] DBG1 new_yaml.py:170->[config:zmes:proc: 39 built in default keys added to the 'base' config -> ['force_mpd', 'same_model_high_conf', 'skip_mons', 'force_live', 'sanitize_str', 'version', 'basic_user', 'basic_password', 'image_path', 'max_detection_size', 'delete_after_analyze', 'write_debug_image', 'write_image_to_zm', 'hass_enabled', 'hass_people', 'hass_notify', 'hass_cooldown', 'push_err_device', 'push_jpg', 'push_jpg_key', 'push_gif', 'push_gif_key', 'push_debug_device', 'push_cooldown', 'mqtt_topic', 'mqtt_broker', 'mqtt_port', 'tls_cert', 'tls_key', 'fp16_target', 'alpr_api_type', 'alpr_url', 'openalpr_recognize_vehicle', 'openalpr_country', 'openalpr_state', 'openalpr_min_confidence', 'smart_fs_thresh', 'frame_strategy', 'same_model_sequence_strategy']]

11/16/21 21:05:08.498236 zmesdetect_m3[1758] DBG1 new_yaml.py:114->[config:zmes:proc: substituting '{{variables}}' for the 'base' config keys]

11/16/21 21:05:08.499031 zmesdetect_m3[1758] DBG1 new_yaml.py:136->[config:zmes: successfully replaced 3 default sub vars in the base config -> ['yolo4_models', 'coral_models', 'base_data_path']]

11/16/21 21:05:08.499395 zmesdetect_m3[1758] DBG1 new_yaml.py:285->[config:zmes: the configured secrets file exists and is a file -> '/etc/zm/zm_secrets.yml']

11/16/21 21:05:08.499723 zmesdetect_m3[1758] DBG1 new_yaml.py:291->[config:zmes: starting '{[secrets]}' substitution]

11/16/21 21:05:08.505495 zmesdetect_m3[1758] DBG1 new_yaml.py:324->[config:zmes: successfully replaced 17 secrets in the base config -> ['PUSHOVER_USER', 'ML_PASSWORD', 'ZM_API_PORTAL', 'ML_USER', 'PUSHOVER_PASS', 'HA_SERVER', 'ZM_PORTAL', 'PUSHOVER_APP_TOKEN', 'ZM_USER', 'PUSHOVER_USER_KEY', 'ZM_PASSWORD', 'MQTT_PASSWORD', 'PLATEREC_ALPR_KEY', 'MQTT_USERNAME', 'PUSHOVER_ERR_TOKEN', 'mlapi_one_key', 'HA_TOKEN']]

11/16/21 21:05:08.50584 zmesdetect_m3[1758] DBG1 new_yaml.py:329->[config:zmes: there are 2 secrets configured that have no substitution candidates in the base config or the secrets file -> ['AWS_SECRET_ACCESS_KEY', 'AWS_ACCESS_KEY_ID']]

11/16/21 21:05:08.507332 zmesdetect_m3[1758] DBG1 new_yaml.py:358->[config:zmes: starting '{{variable}}' substitution]

11/16/21 21:05:08.511712 zmesdetect_m3[1758] DBG1 new_yaml.py:375->[config:zmes: successfully replaced 51 sub vars in the base config -> ['gpu_max_processes', 'tpu_object_framework', 'face_num_jitters', 'face_recog_dist_threshold', 'contained_area', 'object_detection_pattern', 'yolo4_object_config', 'save_unknown_faces_leeway_pixels', 'same_model_sequence_strategy', 'tpu_tf2_mobilenetv2', 'frame_set', 'alpr_detection_pattern', 'openalpr_cmdline_params', 'model_sequence', 'openalpr_cmdline_binary', 'face_model', 'tpu_min_confidence', 'tpu_object_labels', 'openalpr_cmdline_min_confidence', 'tpu_max_lock_wait', 'yolo4_object_processor', 'face_train_model', 'face_detection_framework', 'smart_fps_thresh', 'disable_locks', 'tpu_max_processes', 'yolo4_object_framework', 'tpu_efficientdet_lite3', 'face_upsample_times', 'gpu_max_lock_wait', 'cpu_max_processes', 'show_models', 'frame_strategy', 'tpu_object_weights_mobilenetv2', 'yolo4_object_weights', 'alpr_key', 'save_unknown_faces', 'face_detection_pattern', 'platerec_min_dscore', 'platerec_min_score', 'known_images_path', 'past_det_max_diff_area', 'tpu_object_weights_mobiledet', 'max_detection_size', 'match_past_detections', 'object_min_confidence', 'tpu_face_weights_mobilenetv2', 'cpu_max_lock_wait', 'unknown_images_path', 'platerec_stats', 'yolo4_object_labels']]
11/16/21 21:05:08.513301 zmesdetect_m3[1758] DBG1 new_yaml.py:187->[config:zmes:proc: Base config has been built, all properly configured {[secrets]} and {{vars}} have been replaced! Remember you must manually build per monitor overrode configurations]
11/16/21 21:05:08.513605 zmesdetect_m3[1758] DBG1 zm_detect.py:542->[perf:zmes: building the intial config took 0.089105 ]
11/16/21 21:05:08.514035 zmesdetect_m3[1758] DBG1 new_yaml.py:747->[zmes:api create: building ZM API Session]
11/16/21 21:05:08.518527 zmesdetect_m3[1758] DBG1 api.py:114->[api: SSL certificate verification disabled (encryption enabled, vulnerable to MITM attacks)]
11/16/21 21:05:08.519165 zmesdetect_m3[1758] DBG1 api.py:283->[api: no token found, trying user/pass for login]
11/16/21 21:05:08.589037 zmesdetect_m3[1758] DBG1 api.py:320->[api: detected API ver 2.0+, using token system]
11/16/21 21:05:08.58953 zmesdetect_m3[1758] DBG1 api.py:335->[api: access token expires on: 2021-11-16 23:05:08.589511 (7200s)]
11/16/21 21:05:08.5899 zmesdetect_m3[1758] DBG1 api.py:347->[api: refresh token expires on: 2021-11-17 21:05:08.589891 (86400s)]
11/16/21 21:05:08.603434 zmesdetect_m3[1758] DBG1 new_yaml.py:780->[zmes:api create: ZM API created]
11/16/21 21:05:08.831121 zmesdetect_m3[1758] DBG1 ZMLog:336->[------|  FORKED NEO --- app->Hooks: 7.0.3 - pyzm: 1.0.3 - ES: 7.0.3 - OpenCV:4.5.4 |------]
11/16/21 21:05:08.831532 zmesdetect_m3[1758] DBG1 ZMLog:336->[zmes: Setting up signal handlers for log 'rotation' and log 'interrupt']
11/16/21 21:05:08.667982 zmesdetect_m3[1758] DBG1 zm_detect.py:654->[zmes: this is a 'PAST' (debugging?) event!]
11/16/21 21:05:08.832551 zmesdetect_m3[1758] DBG1 ZMLog:336->[|----------= Encrypted Route Name: 'mlapi_one' | Gateway URL: 'http://192.168.1.100:5003/api/v1' | Weight: 0 =----------|]
11/16/21 21:05:08.669418 zmesdetect_m3[1758] DBG1 zm_detect.py:103->[zmes:mlapi:login: No need to login, access token is valid for 473.60833048820496 sec]
11/16/21 21:05:08.795808 zmesdetect_m3[1758] DBG1 zm_detect.py:318->[
** Gateway URL: 'http://192.168.1.100:5003/api/v1/detect/object?type=object' using auth_header={'Authorization': 'Bearer eyJ0eXAiOiJKV1QiLCJhbGc......'} 
**** params={'delete': True, 'response_format': 'zm_detect'}
****** JSON: stream: 2 - mid: 3 - reason: None - stream options: {'frame_strategy': 'first', 'frame_set': 'snapshot, alarm, snapshot', 'contig_frames_before_error': 2, 'delay_between_attempts': 2.143256, 'max_attempts': 3, 'delay_between_snapshots': 1, 'smart_fps_thresh': '5', 'save_frames': 'no', 'save_frames_dir': None, 'PAST_EVENT': True} - files: {}
]
11/16/21 21:05:08.834301 zmesdetect_m3[1758] DBG1 ZMLog:336->[Connected to ZoneMinder Logging system with user 'www-data' -> '/var/log/zm/zmesdetect_m3.log']
11/16/21 21:05:08.808108 zmesdetect_m3[1758] DBG1 zm_detect.py:365->[zmes:mlapi: error during post to mlapi host-> keys must be str, int, float, bool or None, not bytes]
11/16/21 21:05:08.810038 zmesdetect_m3[1758] DBG1 zm_detect.py:368->[traceback-> Traceback (most recent call last):
  File "/var/lib/zmeventnotification/bin/zm_detect.py", line 328, in remote_detect
    r = requests.post(
  File "/usr/lib/python3/dist-packages/requests/api.py", line 116, in post
    return request('post', url, data=data, json=json, **kwargs)
  File "/usr/lib/python3/dist-packages/requests/api.py", line 60, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 519, in request
    prep = self.prepare_request(req)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 452, in prepare_request
    p.prepare(
  File "/usr/lib/python3/dist-packages/requests/models.py", line 316, in prepare
    self.prepare_body(data, files, json)
  File "/usr/lib/python3/dist-packages/requests/models.py", line 466, in prepare_body
    body = complexjson.dumps(json)
  File "/usr/lib/python3.8/json/__init__.py", line 231, in dumps
    return _default_encoder.encode(obj)
  File "/usr/lib/python3.8/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib/python3.8/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
TypeError: keys must be str, int, float, bool or None, not bytes
]
11/16/21 21:05:08.810698 zmesdetect_m3[1758] DBG1 zm_detect.py:765->[perf:zmes:mlapi: Monitor: sofia (3)->'Event': 2 mlapi detection took: 0.141558]
11/16/21 21:05:08.811555 zmesdetect_m3[1758] DBG1 zm_detect.py:1533->[zmes: no predictions returned from detections]
11/16/21 21:05:08.813199 zmesdetect_m3[1758] DBG1 zm_detect.py:1601->[perf:zmes:FINAL: 'Monitor': sofia (3)->'Event': 2 [PAST event] [total:0.390353] [detection:0.141558] [after core detection: 0.000822] ]
root@d58573099cd9:~/pyzm# 
tsp84
Posts: 227
Joined: Thu Dec 24, 2020 4:04 am

Re: neo-ZMES - Public Beta

Post by tsp84 »

I pushed DEBUG branches for mlapi and zmdetect that add some print statements that should print out more info on your issue. I'll fix the mlapi_dbuser.py I just use it without any flags ./mlapi_dbuser.py and follow the prompts.

You'll have to install from the DEBUG branch or edit the files yourself/copy and paste the new contents. There's not much difference besides print statements and trackbacks.

The no pyzm error is your system saying pyzm is not installed. Clone the neo pyzm repo, cd into the directory and sudo pip3 install -e .

I can't say for sure but the fact I nuked a whole lxc and reinstalled from.the current codebase and have been running for the last day with no issues makes me think there's something going on inside your docker env. I can't say for sure though.

Also another user installed and ran a detection a few days ago without the issue your having so there's something that's off.

Can you sanitize your configs and post them? I've tried messing my configs up to reproduce error and am not having any luck.
Last edited by tsp84 on Tue Nov 16, 2021 7:59 pm, edited 2 times in total.
juan11perez
Posts: 64
Joined: Tue Apr 27, 2021 3:41 am

Re: neo-ZMES - Public Beta

Post by juan11perez »

Understand the issue maybe on my side. The thing is zmeventnotification was working fine 2 days ago but mlapi would not connect. So zm and zmes were fine.
I'll try a new image with the debug.

The log I provided below already had more info after mlapi login failed. Not sure if you saw it.

Thank you
tsp84
Posts: 227
Joined: Thu Dec 24, 2020 4:04 am

Re: neo-ZMES - Public Beta

Post by tsp84 »

I am reviewing the second log output. It is when the request is being parsed through JSON. I'm just looking into that and will report back.
juan11perez
Posts: 64
Joined: Tue Apr 27, 2021 3:41 am

Re: neo-ZMES - Public Beta

Post by juan11perez »

Couple of days ago searched the error and came across a post saying it was associated with converting python to json.

I also read this https://pynative.com/python-json-dumps- ... -encoding/

But I'm afraid it's as much as I understand
Post Reply