Object/Face detection issues when using mlapi

Discussion topics related to mobile applications and ZoneMinder Event Server (including machine learning)
Post Reply
torkilt
Posts: 38
Joined: Thu Jul 06, 2017 10:56 am

Object/Face detection issues when using mlapi

Post by torkilt »

Admin note: I've split the discussion from the old thread because we are now talking about mlapi. Given its a new solution, I'd like to keep it separate so we can focus better.
asker wrote: Fri Jan 17, 2020 4:08 pm You also need to pass the event path if you want to test the event path functionality. Or just run it as a daemon and when an event is detected it will work.
Ah, thanks. It seems to be working now with local detection, but not with mlapi.

It seems it detects alright, but I get no bbox debug images nor images written to zoneminder. There's also the odd stack trace.

Snippets:

Code: Select all

01/17/20 18:17:15 zmesdetect_m4[3116] DBG zm_detect.py:271 [|--> model:yolo init took: 0.000195s]
01/17/20 18:17:15 zmesdetect_m4[3116] DBG zm_detect.py:300 [Using model: yolo with /var/lib/zmeventnotification/images/3189-alarm.jpg]
01/17/20 18:17:15 zmesdetect_m4[3116] INF zm_detect.py:40 [Detecting using remote API Gateway http://192.168.40.5:5000/api/v1]
01/17/20 18:17:15 zmesdetect_m4[3116] DBG zm_detect.py:51 [Found token file, checking if token has not expired]
01/17/20 18:17:15 zmesdetect_m4[3116] DBG zm_detect.py:63 [Access token is valid for 1461 more seconds]
01/17/20 18:17:16 zmesdetect_m4[3116] DBG zm_detect.py:308 [|--> model:yolo detection took: 1.385834s]
01/17/20 18:17:16 zmesdetect_m4[3116] DBG zm_detect.py:457 [ALPR not in use, no need for look aheads in processing]
01/17/20 18:17:16 zmesdetect_m4[3116] DBG zm_detect.py:480 [No match found in /var/lib/zmeventnotification/images/3189-alarm.jpg using model:yolo]
01/17/20 18:17:16 zmesdetect_m4[3116] DBG zm_detect.py:300 [Using model: yolo with /var/lib/zmeventnotification/images/3189-snapshot.jpg]
01/17/20 18:17:16 zmesdetect_m4[3116] INF zm_detect.py:40 [Detecting using remote API Gateway http://192.168.40.5:5000/api/v1]
01/17/20 18:17:16 zmesdetect_m4[3116] DBG zm_detect.py:51 [Found token file, checking if token has not expired]
01/17/20 18:17:16 zmesdetect_m4[3116] DBG zm_detect.py:63 [Access token is valid for 1463 more seconds]
01/17/20 18:17:17 zmesdetect_m4[3116] DBG zm_detect.py:308 [|--> model:yolo detection took: 1.344932s]
01/17/20 18:17:17 zmesdetect_m4[3116] DBG image_manip.py:127 [intersection: polygon in process=[(542, 356), (696, 356), (696, 670), (542, 670)]]
01/17/20 18:17:17 zmesdetect_m4[3116] DBG image_manip.py:133 [full_image intersects object:person[[(542, 356), (696, 356), (696, 670), (542, 670)]]]
01/17/20 18:17:17 zmesdetect_m4[3116] DBG zm_detect.py:457 [ALPR not in use, no need for look aheads in processing]
01/17/20 18:17:17 zmesdetect_m4[3116] INF zm_detect.py:473 [labels found: ['person']]
01/17/20 18:17:17 zmesdetect_m4[3116] DBG zm_detect.py:474 [match found in /var/lib/zmeventnotification/images/3189-snapshot.jpg, breaking file loop...]
01/17/20 18:17:17 zmesdetect_m4[3116] DBG zm_detect.py:271 [|--> model:face init took: 8e-06s]
01/17/20 18:17:17 zmesdetect_m4[3116] DBG zm_detect.py:298 [Skipping /var/lib/zmeventnotification/images/3189-alarm.jpg as we earlier matched /var/lib/zmeventnotification/images/3189-snapshot.jpg]
01/17/20 18:17:17 zmesdetect_m4[3116] DBG zm_detect.py:300 [Using model: face with /var/lib/zmeventnotification/images/3189-snapshot.jpg]
01/17/20 18:17:17 zmesdetect_m4[3116] INF zm_detect.py:40 [Detecting using remote API Gateway http://192.168.40.5:5000/api/v1]
01/17/20 18:17:17 zmesdetect_m4[3116] DBG zm_detect.py:51 [Found token file, checking if token has not expired]
01/17/20 18:17:17 zmesdetect_m4[3116] DBG zm_detect.py:63 [Access token is valid for 1464 more seconds]
01/17/20 18:17:18 zmesdetect_m4[3116] DBG zm_detect.py:308 [|--> model:face detection took: 0.250388s]
01/17/20 18:17:18 zmesdetect_m4[3116] DBG zm_detect.py:316 [Appending known faces to filter list]
01/17/20 18:17:18 zmesdetect_m4[3116] DBG zm_detect.py:323 [Found known faces list remote gateway supports. If you have trained new faces in the remote gateway, please delete this file]
01/17/20 18:17:18 zmesdetect_m4[3116] DBG zm_detect.py:326 [Read from existing names: ['20200115-231725_231_2_']]

Code: Select all

192.168.40.138 - - [17/Jan/2020 18:14:11] "POST /api/v1/detect/object?delete=True HTTP/1.1" 200 -
DEBUG: Object Recognition requested
DEBUG: get_file returned: images/b80793d0-8501-4997-b563-ac8d54db9359.jpg
192.168.40.138 - - [17/Jan/2020 18:14:12] "POST /api/v1/detect/object?delete=True HTTP/1.1" 200 -
DEBUG: Face Recognition requested
DEBUG: get_file returned: images/56f18195-761a-4c45-a400-3480b177e8b5.jpg
DEBUG: Face recognition: no faces found
192.168.40.138 - - [17/Jan/2020 18:14:12] "POST /api/v1/detect/object?type=face&delete=True HTTP/1.1" 200 -
DEBUG: Object Recognition requested
DEBUG: get_file returned: images/64a7bd1d-3d86-4a43-bdea-81047a39b6dc.jpg
DEBUG: Object Recognition requested
DEBUG: get_file returned: images/8e495f6b-bbd3-4228-b72e-d2440baf1b27.jpg
192.168.40.138 - - [17/Jan/2020 18:14:46] "POST /api/v1/detect/object?delete=True HTTP/1.1" 500 -
192.168.40.138 - - [17/Jan/2020 18:14:46] "POST /api/v1/detect/object?delete=True HTTP/1.1" 500 -
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 2463, in __call__
    return self.wsgi_app(environ, start_response)
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 2449, in wsgi_app
    response = self.handle_exception(e)
  File "/usr/local/lib/python3.6/site-packages/flask_restful/__init__.py", line 269, in error_router
    return original_handler(e)
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1866, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python3.6/site-packages/flask/_compat.py", line 38, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 2446, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1951, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python3.6/site-packages/flask_restful/__init__.py", line 269, in error_router
    return original_handler(e)
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1820, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python3.6/site-packages/flask/_compat.py", line 38, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1949, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1935, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/usr/local/lib/python3.6/site-packages/flask_restful/__init__.py", line 458, in wrapper
    resp = resource(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/flask/views.py", line 89, in view
    return self.dispatch_request(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/flask_restful/__init__.py", line 573, in dispatch_request
    resp = meth(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/flask_jwt_extended/view_decorators.py", line 108, in wrapper
    return fn(*args, **kwargs)
  File "/root/mlapi/api.py", line 123, in post
    detections = m.detect(image)
  File "/root/mlapi/modules/object.py", line 73, in detect
    w = int(detection[2] * Width)
OverflowError: cannot convert float infinity to integer
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 2463, in __call__
    return self.wsgi_app(environ, start_response)
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 2449, in wsgi_app
    response = self.handle_exception(e)
  File "/usr/local/lib/python3.6/site-packages/flask_restful/__init__.py", line 269, in error_router
    return original_handler(e)
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1866, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python3.6/site-packages/flask/_compat.py", line 38, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 2446, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1951, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python3.6/site-packages/flask_restful/__init__.py", line 269, in error_router
    return original_handler(e)
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1820, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python3.6/site-packages/flask/_compat.py", line 38, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1949, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1935, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/usr/local/lib/python3.6/site-packages/flask_restful/__init__.py", line 458, in wrapper
    resp = resource(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/flask/views.py", line 89, in view
    return self.dispatch_request(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/flask_restful/__init__.py", line 573, in dispatch_request
    resp = meth(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/flask_jwt_extended/view_decorators.py", line 108, in wrapper
    return fn(*args, **kwargs)
  File "/root/mlapi/api.py", line 123, in post
    detections = m.detect(image)
  File "/root/mlapi/modules/object.py", line 73, in detect
    w = int(detection[2] * Width)
OverflowError: cannot convert float infinity to integer
DEBUG: Object Recognition requested
DEBUG: get_file returned: images/b09c0bff-05da-4fa2-955c-b8857e04c128.jpg
192.168.40.138 - - [17/Jan/2020 18:15:06] "POST /api/v1/detect/object?delete=True HTTP/1.1" 200 -
DEBUG: Object Recognition requested
DEBUG: get_file returned: images/c6b195f0-cfa8-4669-999c-43f405d61bdc.jpg
INFO: object:person at [542, 356, 696, 670] has a acceptable confidence:0.9956036806106567 compared to min confidence of: 0.4, adding
192.168.40.138 - - [17/Jan/2020 18:15:07] "POST /api/v1/detect/object?delete=True HTTP/1.1" 200 -
DEBUG: Face Recognition requested
DEBUG: get_file returned: images/ea29ccd1-68bf-4ee8-827b-13edaa37ec4e.jpg
DEBUG: Face recognition: no faces found
192.168.40.138 - - [17/Jan/2020 18:15:08] "POST /api/v1/detect/object?type=face&delete=True HTTP/1.1" 200 -
DEBUG: Object Recognition requested
DEBUG: get_file returned: images/c8f71647-dacb-413d-baba-a095cbf36a96.jpg
192.168.40.138 - - [17/Jan/2020 18:15:21] "POST /api/v1/detect/object?delete=True HTTP/1.1" 200 -
DEBUG: Object Recognition requested
DEBUG: get_file returned: images/e726d74c-3f24-4617-aa9e-afdde6526019.jpg
INFO: object:person at [542, 356, 696, 670] has a acceptable confidence:0.9956036806106567 compared to min confidence of: 0.4, adding
192.168.40.138 - - [17/Jan/2020 18:15:22] "POST /api/v1/detect/object?delete=True HTTP/1.1" 200 -
DEBUG: Face Recognition requested
DEBUG: get_file returned: images/53faf76e-debc-4d73-a35e-dcf6f7621dbf.jpg
DEBUG: Face recognition: no faces found
192.168.40.138 - - [17/Jan/2020 18:15:23] "POST /api/v1/detect/object?type=face&delete=True HTTP/1.1" 200 -
DEBUG: Object Recognition requested
DEBUG: get_file returned: images/2f4290ca-9c6b-439a-b856-bd0a58fcbe05.jpg
192.168.40.138 - - [17/Jan/2020 18:16:01] "POST /api/v1/detect/object?delete=True HTTP/1.1" 200 -
DEBUG: Object Recognition requested
DEBUG: get_file returned: images/ee6e8c11-732d-4972-837a-e5e8a8594c42.jpg
192.168.40.138 - - [17/Jan/2020 18:16:02] "POST /api/v1/detect/object?delete=True HTTP/1.1" 200 -
DEBUG: Face Recognition requested
DEBUG: get_file returned: images/5229d490-f917-4ecb-a541-a5319fb83649.jpg
DEBUG: Face recognition: no faces found
192.168.40.138 - - [17/Jan/2020 18:16:03] "POST /api/v1/detect/object?type=face&delete=True HTTP/1.1" 200 -
DEBUG: Object Recognition requested
DEBUG: get_file returned: images/68004c2d-71a7-4615-aa2f-c9a12c1d7f8b.jpg
192.168.40.138 - - [17/Jan/2020 18:16:16] "POST /api/v1/detect/object?delete=True HTTP/1.1" 200 -
DEBUG: Object Recognition requested
DEBUG: get_file returned: images/58822545-581b-4ca5-bff8-087f2b395075.jpg
192.168.40.138 - - [17/Jan/2020 18:16:17] "POST /api/v1/detect/object?delete=True HTTP/1.1" 200 -
DEBUG: Face Recognition requested
DEBUG: get_file returned: images/744ad30c-4ef4-4961-ad68-b694ba917021.jpg
DEBUG: Face recognition: no faces found
192.168.40.138 - - [17/Jan/2020 18:16:17] "POST /api/v1/detect/object?type=face&delete=True HTTP/1.1" 200 -
DEBUG: Object Recognition requested
DEBUG: get_file returned: images/d1b8cbbe-1e8f-460d-af44-5589ea0773b1.jpg
192.168.40.138 - - [17/Jan/2020 18:17:06] "POST /api/v1/detect/object?delete=True HTTP/1.1" 200 -
DEBUG: Object Recognition requested
DEBUG: get_file returned: images/9d41563c-5c97-4acc-9f74-19a35a80fe2b.jpg
192.168.40.138 - - [17/Jan/2020 18:17:07] "POST /api/v1/detect/object?delete=True HTTP/1.1" 200 -
DEBUG: Face Recognition requested
DEBUG: get_file returned: images/a08de29b-f966-40a2-9d07-533faf3764a2.jpg
DEBUG: Face recognition: no faces found
192.168.40.138 - - [17/Jan/2020 18:17:07] "POST /api/v1/detect/object?type=face&delete=True HTTP/1.1" 200 -
DEBUG: Object Recognition requested
DEBUG: get_file returned: images/832448e0-ec57-4433-b907-4d479494b40f.jpg
192.168.40.138 - - [17/Jan/2020 18:17:16] "POST /api/v1/detect/object?delete=True HTTP/1.1" 200 -
DEBUG: Object Recognition requested
DEBUG: get_file returned: images/3f288e8c-d9a9-4fd7-93ff-d2d3cb318d63.jpg
INFO: object:person at [542, 356, 696, 670] has a acceptable confidence:0.9956036806106567 compared to min confidence of: 0.4, adding
192.168.40.138 - - [17/Jan/2020 18:17:17] "POST /api/v1/detect/object?delete=True HTTP/1.1" 200 -
DEBUG: Face Recognition requested
DEBUG: get_file returned: images/b416282d-12b1-44e6-8901-a44f8ec039b4.jpg
DEBUG: Face recognition: no faces found
192.168.40.138 - - [17/Jan/2020 18:17:18] "POST /api/v1/detect/object?type=face&delete=True HTTP/1.1" 200 -
DEBUG: Object Recognition requested
DEBUG: get_file returned: images/2779fed4-ae0c-4033-806a-3303a0800cfc.jpg
192.168.40.138 - - [17/Jan/2020 18:18:36] "POST /api/v1/detect/object?delete=True HTTP/1.1" 200 -
DEBUG: Object Recognition requested
DEBUG: get_file returned: images/d1d41519-7ef4-4401-b4bc-2fb907a8b5df.jpg
192.168.40.138 - - [17/Jan/2020 18:18:37] "POST /api/v1/detect/object?delete=True HTTP/1.1" 200 -
DEBUG: Face Recognition requested
DEBUG: get_file returned: images/8ff6672a-be9d-4aaa-b6c0-664648db5284.jpg
DEBUG: Face recognition: no faces found
192.168.40.138 - - [17/Jan/2020 18:18:38] "POST /api/v1/detect/object?type=face&delete=True HTTP/1.1" 200 -
DEBUG: Object Recognition requested
DEBUG: get_file returned: images/e42069b5-020b-4de4-a22f-a995fc5d076f.jpg
192.168.40.138 - - [17/Jan/2020 18:19:36] "POST /api/v1/detect/object?delete=True HTTP/1.1" 200 -
DEBUG: Object Recognition requested
DEBUG: get_file returned: images/048b9293-c898-4a96-9f50-1a1c63fb2e6c.jpg
192.168.40.138 - - [17/Jan/2020 18:19:37] "POST /api/v1/detect/object?delete=True HTTP/1.1" 200 -
DEBUG: Face Recognition requested
DEBUG: get_file returned: images/e52b2faa-9485-423e-8b0e-f264a9c525ec.jpg
DEBUG: Face recognition: no faces found
192.168.40.138 - - [17/Jan/2020 18:19:38] "POST /api/v1/detect/object?type=face&delete=True HTTP/1.1" 200 -
DEBUG: Object Recognition requested
DEBUG: get_file returned: images/09520112-3b79-4681-bd71-6e5f15b7334e.jpg
192.168.40.138 - - [17/Jan/2020 18:20:11] "POST /api/v1/detect/object?delete=True HTTP/1.1" 200 -
DEBUG: Object Recognition requested
DEBUG: get_file returned: images/ed38fd0d-2420-4a5b-a7c9-57a8ad1971b4.jpg
192.168.40.138 - - [17/Jan/2020 18:20:12] "POST /api/v1/detect/object?delete=True HTTP/1.1" 200 -
DEBUG: Face Recognition requested
DEBUG: get_file returned: images/d018225c-32f5-4b4e-b437-1f6763060572.jpg
DEBUG: Face recognition: no faces found
192.168.40.138 - - [17/Jan/2020 18:20:13] "POST /api/v1/detect/object?type=face&delete=True HTTP/1.1" 200 -
DEBUG: Object Recognition requested
DEBUG: get_file returned: images/7b109602-c6ed-46f3-8dca-8294226bc9b8.jpg
192.168.40.138 - - [17/Jan/2020 18:20:16] "POST /api/v1/detect/object?delete=True HTTP/1.1" 200 -
DEBUG: Object Recognition requested
DEBUG: get_file returned: images/bcacfdb4-157a-469b-bd0a-77290cd3b6e5.jpg
192.168.40.138 - - [17/Jan/2020 18:20:17] "POST /api/v1/detect/object?delete=True HTTP/1.1" 200 -
DEBUG: Face Recognition requested
DEBUG: get_file returned: images/7995cdd7-7fc5-443e-a7ee-e5dae65e6a33.jpg
DEBUG: Face recognition: no faces found
192.168.40.138 - - [17/Jan/2020 18:20:18] "POST /api/v1/detect/object?type=face&delete=True HTTP/1.1" 200 -
DEBUG: Object Recognition requested
DEBUG: get_file returned: images/82a3b17a-2fb2-45cd-a283-a9a2c6f28d30.jpg
192.168.40.138 - - [17/Jan/2020 18:21:17] "POST /api/v1/detect/object?delete=True HTTP/1.1" 200 -
DEBUG: Object Recognition requested
DEBUG: get_file returned: images/8ce0344a-d18d-441f-83e5-805d4a174137.j
Pretty cool with mlapi btw =)

Mvh.

Torkil
User avatar
asker
Posts: 1553
Joined: Sun Mar 01, 2015 12:12 pm

Re: Object detection working, kind of

Post by asker »

Okay, so it looks like in your situation, with mlapi:
a) You have object detection - which worked, returned person
b) You also have face detection - no faces were detected - that is where mlapi broke

I'll take a look. Would you mind creating a GH issue at https://github.com/pliablepixels/mlapi?

I am on travel at the moment till month end, so a fix may be in early feb (or earlier if I get time)

I'm happy to see folks use mlapi more.
I no longer work on zmNinja, zmeventnotification, pyzm or mlapi. I may respond on occasion based on my available time/interest.

Please read before posting:
How to set up logging properly
How to troubleshoot and report - ES
How to troubleshoot and report - zmNinja
ES docs
zmNinja docs
User avatar
asker
Posts: 1553
Joined: Sun Mar 01, 2015 12:12 pm

Re: Object detection working, kind of

Post by asker »

Actually, it looks like the issue many be of concurrent requests. Can you confirm if this problem only happens if there are multiple requests being processed at the same time?
I no longer work on zmNinja, zmeventnotification, pyzm or mlapi. I may respond on occasion based on my available time/interest.

Please read before posting:
How to set up logging properly
How to troubleshoot and report - ES
How to troubleshoot and report - zmNinja
ES docs
zmNinja docs
torkilt
Posts: 38
Joined: Thu Jul 06, 2017 10:56 am

Re: Object detection working, kind of

Post by torkilt »

asker wrote: Sun Jan 19, 2020 6:14 am Actually, it looks like the issue many be of concurrent requests. Can you confirm if this problem only happens if there are multiple requests being processed at the same time?
I can look at that tomorrow, but since I never got any bbox or debug images I'm guessing concurrency may be the cause of the stack track but not the cause of it not working at all?

I'll look at it and file a ticket.

Thanks,

Torkil
User avatar
asker
Posts: 1553
Joined: Sun Mar 01, 2015 12:12 pm

Re: Object detection working, kind of

Post by asker »

Please update to mlapi 1.0.1 and see if the issue persists.
I no longer work on zmNinja, zmeventnotification, pyzm or mlapi. I may respond on occasion based on my available time/interest.

Please read before posting:
How to set up logging properly
How to troubleshoot and report - ES
How to troubleshoot and report - zmNinja
ES docs
zmNinja docs
torkilt
Posts: 38
Joined: Thu Jul 06, 2017 10:56 am

Re: Object detection working, kind of

Post by torkilt »

asker wrote: Mon Jan 20, 2020 5:46 am Please update to mlapi 1.0.1 and see if the issue persists.
I think the new version is trying to do something untoward with my GPU:

Code: Select all

DEBUG: Object Recognition requested
DEBUG: get_file returned: images/52ac9970-d23f-4158-990d-ac65fee2a52e.jpg
DEBUG: Initializing Yolo
DEBUG: config:./models/yolov3/yolov3.cfg, weights:./models/yolov3/yolov3.weights
INFO: object:person at [462, 318, 600, 654] has a acceptable confidence:0.9877113103866577 compared to min confidence of: 0.4, adding
192.168.40.138 - - [20/Jan/2020 07:07:21] "POST /api/v1/detect/object?delete=True HTTP/1.1" 200 -
DEBUG: Face Recognition requested
DEBUG: get_file returned: images/bfe44a18-5ce6-4d08-a27b-6845c24e73ec.jpg
192.168.40.138 - - [20/Jan/2020 07:07:22] "POST /api/v1/detect/object?type=face&delete=True HTTP/1.1" 500 -
Error on request:
Traceback (most recent call last):
  File "/usr/local/lib64/python3.6/site-packages/werkzeug/serving.py", line 304, in run_wsgi
    execute(self.server.app)
  File "/usr/local/lib64/python3.6/site-packages/werkzeug/serving.py", line 292, in execute
    application_iter = app(environ, start_response)
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 2463, in __call__
    return self.wsgi_app(environ, start_response)
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 2449, in wsgi_app
    response = self.handle_exception(e)
  File "/usr/local/lib/python3.6/site-packages/flask_restful/__init__.py", line 269, in error_router
    return original_handler(e)
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1866, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python3.6/site-packages/flask/_compat.py", line 38, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 2446, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1951, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python3.6/site-packages/flask_restful/__init__.py", line 269, in error_router
    return original_handler(e)
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1820, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python3.6/site-packages/flask/_compat.py", line 38, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1949, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1935, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/usr/local/lib/python3.6/site-packages/flask_restful/__init__.py", line 458, in wrapper
    resp = resource(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/flask/views.py", line 89, in view
    return self.dispatch_request(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/flask_restful/__init__.py", line 573, in dispatch_request
    resp = meth(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/flask_jwt_extended/view_decorators.py", line 108, in wrapper
    return fn(*args, **kwargs)
  File "/root/mlapi/api.py", line 125, in post
    detections = m.detect(image)
  File "/root/mlapi/modules/face_recognition.py", line 68, in detect
    face_locations = face_recognition.face_locations(rgb_image, model=self.model, number_of_times_to_upsample=self.upsample_times)
  File "/usr/local/lib/python3.6/site-packages/face_recognition/api.py", line 116, in face_locations
    return [_trim_css_to_bounds(_rect_to_css(face.rect), img.shape) for face in _raw_face_locations(img, number_of_times_to_upsample, "cnn")]
  File "/usr/local/lib/python3.6/site-packages/face_recognition/api.py", line 100, in _raw_face_locations
    return cnn_face_detector(img, number_of_times_to_upsample)
RuntimeError: Error while calling cudaGetDevice(&the_device_id) in file /tmp/pip-install-28f7zm6b/dlib/dlib/cuda/gpu_data.cpp:201. code: 3, reason: initialization error
Using "processes = 1" seemingly work but I get no bbox images nor images written to zoneminder.

Mvh.

Torkil
User avatar
asker
Posts: 1553
Joined: Sun Mar 01, 2015 12:12 pm

Re: Object/Face detection issues when using mlapi

Post by asker »

So...

a) It looks like forked processes cannot access the GPU if the parent allocated resources for it. Spawned processes can (https://github.com/davisking/dlib/issues/617) which is why processes=1 works. I'll need to look into this more when I get back from my travels and am able to test with my GPU machine

b) Debug/bbox images - will test - I may be goofing up in zm_detect.py once it gets back data in the mlapi case.
I no longer work on zmNinja, zmeventnotification, pyzm or mlapi. I may respond on occasion based on my available time/interest.

Please read before posting:
How to set up logging properly
How to troubleshoot and report - ES
How to troubleshoot and report - zmNinja
ES docs
zmNinja docs
User avatar
asker
Posts: 1553
Joined: Sun Mar 01, 2015 12:12 pm

Re: Object/Face detection issues when using mlapi

Post by asker »

Can you post debug logs from zm_detect.py when you use mlapi with processes=1? I'm not sure why its not writing bbox images, if write_debug_images and write_image_to_zm are yes.
I no longer work on zmNinja, zmeventnotification, pyzm or mlapi. I may respond on occasion based on my available time/interest.

Please read before posting:
How to set up logging properly
How to troubleshoot and report - ES
How to troubleshoot and report - zmNinja
ES docs
zmNinja docs
torkilt
Posts: 38
Joined: Thu Jul 06, 2017 10:56 am

Re: Object/Face detection issues when using mlapi

Post by torkilt »

asker wrote: Mon Jan 20, 2020 9:00 am Can you post debug logs from zm_detect.py when you use mlapi with processes=1? I'm not sure why its not writing bbox images, if write_debug_images and write_image_to_zm are yes.
Seeing this when running foreground with mlapi trying to detect all objects:

Code: Select all

Traceback (most recent call last):
  File "/var/lib/zmeventnotification/bin/zm_detect.py", line 327, in <module>
    m.set_classes(data['names'])
No bbox etc. When run without mlapi it detects birds and benches in my garden and writes images.

I emailed you the full log (tee foreground with mlapi).

Mvh.

Torkil
User avatar
asker
Posts: 1553
Joined: Sun Mar 01, 2015 12:12 pm

Re: Object/Face detection issues when using mlapi

Post by asker »

torkilt wrote: Mon Jan 20, 2020 9:57 am

Code: Select all

Traceback (most recent call last):
  File "/var/lib/zmeventnotification/bin/zm_detect.py", line 327, in <module>
    m.set_classes(data['names'])
No bbox etc. When run without mlapi it detects birds and benches in my garden and writes images.

I emailed you the full log (tee foreground with mlapi).

Mvh.

Torkil
So that's why. zm_detect is crashing with mlapi in this scenario. You sent me the event server logs - zm_detect logs are in /var/log/zm/zmes_detect*.log
I no longer work on zmNinja, zmeventnotification, pyzm or mlapi. I may respond on occasion based on my available time/interest.

Please read before posting:
How to set up logging properly
How to troubleshoot and report - ES
How to troubleshoot and report - zmNinja
ES docs
zmNinja docs
torkilt
Posts: 38
Joined: Thu Jul 06, 2017 10:56 am

Re: Object/Face detection issues when using mlapi

Post by torkilt »

asker wrote: Mon Jan 20, 2020 11:15 am So that's why. zm_detect is crashing with mlapi in this scenario. You sent me the event server logs - zm_detect logs are in /var/log/zm/zmes_detect*.log
My bad, sent you a detect log just now. Here's some snippets:

MLAPI:

Code: Select all

DEBUG: Object Recognition requested
DEBUG: get_file returned: images/cc4c1c5e-96b6-4380-8445-7cf3b96efbf1.jpg
INFO: object:bench at [441, 304, 517, 338] has a acceptable confidence:0.5771249532699585 compared to min confidence of: 0.4, adding
INFO: object:bench at [378, 321, 470, 361] has a acceptable confidence:0.5321613550186157 compared to min confidence of: 0.4, adding
192.168.40.138 - - [20/Jan/2020 12:26:33] "POST /api/v1/detect/object?delete=True HTTP/1.1" 200 -
DEBUG: Face Recognition requested
DEBUG: get_file returned: images/00861825-f715-495c-9108-d512a9132b3f.jpg
DEBUG: Face recognition: no faces found
192.168.40.138 - - [20/Jan/2020 12:26:34] "POST /api/v1/detect/object?type=face&delete=True HTTP/1.1" 200 -
Detect log:

Code: Select all

01/20/20 12:26:31 zmesdetect_m2[20410] INF zm_detect.py:145 [---------| app version: 5.4.1 |------------]
01/20/20 12:26:31 zmesdetect_m2[20410] DBG utils.py:229 [secret filename: /etc/zm/secrets.ini]
01/20/20 12:26:31 zmesdetect_m2[20410] DBG utils.py:203 [Secret token found in config: !ZM_PORTAL]
01/20/20 12:26:31 zmesdetect_m2[20410] DBG utils.py:203 [Secret token found in config: !ZM_API_PORTAL]
01/20/20 12:26:31 zmesdetect_m2[20410] DBG utils.py:203 [Secret token found in config: !ML_USER]
01/20/20 12:26:31 zmesdetect_m2[20410] DBG utils.py:203 [Secret token found in config: !ML_PASSWORD]
01/20/20 12:26:31 zmesdetect_m2[20410] DBG utils.py:203 [Secret token found in config: !PLATEREC_ALPR_KEY]
01/20/20 12:26:31 zmesdetect_m2[20410] DBG utils.py:254 [allowing self-signed certs to work...]
01/20/20 12:26:31 zmesdetect_m2[20410] DBG utils.py:304 [key [config] is '/var/lib/zmeventnotification/models/yolov3/yolov3.cfg' after substitution]
01/20/20 12:26:31 zmesdetect_m2[20410] DBG utils.py:304 [key [weights] is '/var/lib/zmeventnotification/models/yolov3/yolov3.weights' after substitution]
01/20/20 12:26:31 zmesdetect_m2[20410] DBG utils.py:304 [key [labels] is '/var/lib/zmeventnotification/models/yolov3/yolov3_classes.txt' after substitution]
01/20/20 12:26:31 zmesdetect_m2[20410] DBG utils.py:304 [key [tiny_config] is '/var/lib/zmeventnotification/models/tinyyolo/yolov3-tiny.cfg' after substitution
]
01/20/20 12:26:31 zmesdetect_m2[20410] DBG utils.py:304 [key [tiny_weights] is '/var/lib/zmeventnotification/models/tinyyolo/yolov3-tiny.weights' after substit
ution]
01/20/20 12:26:31 zmesdetect_m2[20410] DBG utils.py:304 [key [tiny_labels] is '/var/lib/zmeventnotification/models/tinyyolo/yolov3-tiny.txt' after substitution
]
01/20/20 12:26:31 zmesdetect_m2[20410] DBG utils.py:304 [key [known_images_path] is '/var/lib/zmeventnotification/known_faces' after substitution]
01/20/20 12:26:31 zmesdetect_m2[20410] INF zm_detect.py:171 [Importing remote shim classes for Yolo/Face]
01/20/20 12:26:31 zmesdetect_m2[20410] DBG utils.py:128 [Trying to download https://zoneminder.localdomain/zm/index.php?view=image&eid=5333&fid=alarm]
01/20/20 12:26:31 zmesdetect_m2[20410] DBG utils.py:142 [Trying to download https://zoneminder.localdomain/zm/index.php?view=image&eid=5333&fid=snapshot]
01/20/20 12:26:31 zmesdetect_m2[20410] DBG zm_detect.py:209 [No polygon area specfied, so adding a full image polygon:[{'name': 'full_image', 'value': [(0, 0), (1920, 0), (1920, 1080), (0, 1080)]}]]
01/20/20 12:26:31 zmesdetect_m2[20410] DBG zm_detect.py:211 [resizing to 1200 before analysis...]
01/20/20 12:26:31 zmesdetect_m2[20410] DBG utils.py:37 [resized polygons x=0.625/y=0.625: [{'name': 'full_image', 'value': [(0, 0), (1200, 0), (1200, 675), (0, 675)]}]]
01/20/20 12:26:31 zmesdetect_m2[20410] DBG zm_detect.py:229 [User ALPR if vehicle found: False]
01/20/20 12:26:31 zmesdetect_m2[20410] DBG zm_detect.py:271 [|--> model:yolo init took: 0.000201s]
01/20/20 12:26:31 zmesdetect_m2[20410] DBG zm_detect.py:300 [Using model: yolo with /var/lib/zmeventnotification/images/5333-alarm.jpg]
01/20/20 12:26:32 zmesdetect_m2[20410] INF zm_detect.py:40 [Detecting using remote API Gateway http://192.168.40.5:5000/api/v1]
01/20/20 12:26:32 zmesdetect_m2[20410] DBG zm_detect.py:51 [Found token file, checking if token has not expired]
01/20/20 12:26:32 zmesdetect_m2[20410] DBG zm_detect.py:63 [Access token is valid for 3512 more seconds]
01/20/20 12:26:33 zmesdetect_m2[20410] DBG zm_detect.py:308 [|--> model:yolo detection took: 1.762923s]
01/20/20 12:26:33 zmesdetect_m2[20410] DBG image_manip.py:127 [intersection: polygon in process=[(441, 304), (517, 304), (517, 338), (441, 338)]]
01/20/20 12:26:33 zmesdetect_m2[20410] DBG image_manip.py:133 [full_image intersects object:bench[[(441, 304), (517, 304), (517, 338), (441, 338)]]]
01/20/20 12:26:33 zmesdetect_m2[20410] DBG image_manip.py:127 [intersection: polygon in process=[(378, 321), (470, 321), (470, 361), (378, 361)]]
01/20/20 12:26:33 zmesdetect_m2[20410] DBG image_manip.py:133 [full_image intersects object:bench[[(378, 321), (470, 321), (470, 361), (378, 361)]]]
01/20/20 12:26:33 zmesdetect_m2[20410] DBG zm_detect.py:457 [ALPR not in use, no need for look aheads in processing]
01/20/20 12:26:33 zmesdetect_m2[20410] INF zm_detect.py:473 [labels found: ['bench', 'bench']]
01/20/20 12:26:33 zmesdetect_m2[20410] DBG zm_detect.py:474 [match found in /var/lib/zmeventnotification/images/5333-alarm.jpg, breaking file loop...]
01/20/20 12:26:33 zmesdetect_m2[20410] DBG zm_detect.py:271 [|--> model:face init took: 1e-05s]
01/20/20 12:26:33 zmesdetect_m2[20410] DBG zm_detect.py:300 [Using model: face with /var/lib/zmeventnotification/images/5333-alarm.jpg]
01/20/20 12:26:33 zmesdetect_m2[20410] INF zm_detect.py:40 [Detecting using remote API Gateway http://192.168.40.5:5000/api/v1]
01/20/20 12:26:33 zmesdetect_m2[20410] DBG zm_detect.py:51 [Found token file, checking if token has not expired]
01/20/20 12:26:33 zmesdetect_m2[20410] DBG zm_detect.py:63 [Access token is valid for 3514 more seconds]
01/20/20 12:26:34 zmesdetect_m2[20410] DBG zm_detect.py:308 [|--> model:face detection took: 0.311108s]
01/20/20 12:26:34 zmesdetect_m2[20410] DBG zm_detect.py:316 [Appending known faces to filter list]
01/20/20 12:26:34 zmesdetect_m2[20410] DBG zm_detect.py:323 [Found known faces list remote gateway supports. If you have trained new faces in the remote gateway, please delete this file]
01/20/20 12:26:34 zmesdetect_m2[20410] DBG zm_detect.py:326 [Read from existing names: ['20200115-231725_231_2_']]
Poking around I also found this in zmdc.log:

Code: Select all

" Back door" "/zoneminder//2/2020-01-20/5006"
Traceback (most recent call last):
  File "/var/lib/zmeventnotification/bin/zm_detect.py", line 327, in <module>
    m.set_classes(data['names'])
TypeError: set_classes() takes 1 positional argument but 2 were given
Mvh.

Torkil
User avatar
asker
Posts: 1553
Joined: Sun Mar 01, 2015 12:12 pm

Re: Object/Face detection issues when using mlapi

Post by asker »

I’m on the road but:

https://github.com/pliablepixels/zmeven ... igw.py#L24

Change
def set_classes(classes):

To

def set_classes(self,classes):

And make sure you reinstall the file (via pip or just copy over)
I no longer work on zmNinja, zmeventnotification, pyzm or mlapi. I may respond on occasion based on my available time/interest.

Please read before posting:
How to set up logging properly
How to troubleshoot and report - ES
How to troubleshoot and report - zmNinja
ES docs
zmNinja docs
torkilt
Posts: 38
Joined: Thu Jul 06, 2017 10:56 am

Re: Object/Face detection issues when using mlapi

Post by torkilt »

asker wrote: Mon Jan 20, 2020 12:04 pm I’m on the road but:

https://github.com/pliablepixels/zmeven ... igw.py#L24

Change
def set_classes(classes):

To

def set_classes(self,classes):

And make sure you reinstall the file (via pip or just copy over)
Thanks, that fixed it!

Torkil
Post Reply