Coral edgetpu on ubuntu 20.04.1 LTS zm 1.35.10

Previous development branch now released as 1.36
Locked
goatman64
Posts: 31
Joined: Wed Oct 21, 2015 7:41 pm

Coral edgetpu on ubuntu 20.04.1 LTS zm 1.35.10

Post by goatman64 »

I have a functional zm system 1.35.10 on Ubuntu 20.04.1. LTS Opencv 4.5 from source. Using Yolo4 and working very well. Wanted to try the Coral edgetpu so added a usb 3.1 card and the Google Coral edgetpu.

I can run the classify_image test without issue but zm fails trying to open the device.

From my zmesdetect logs

File "/usr/lib/python3/dist-packages/edgetpu/basic/basic_engine.py", line 92, in __init__
self._engine = BasicEnginePythonWrapper.CreateFromFile(model_path)
RuntimeError: Error in device opening (/sys/bus/usb/devices/10-2)!

Anyone else using edgetpu on 1.35.x without issue? I see some posts about edgetpu on 1.34.x

Thanks
Magic919
Posts: 1381
Joined: Wed Sep 18, 2013 6:56 am

Re: Coral edgetpu on ubuntu 20.04.1 LTS zm 1.35.10

Post by Magic919 »

I run that ZM and Ubuntu version and it works.

Looks a lot like your install of the Coral is the place to start.
-
goatman64
Posts: 31
Joined: Wed Oct 21, 2015 7:41 pm

Re: Coral edgetpu on ubuntu 20.04.1 LTS zm 1.35.10

Post by goatman64 »

Thanks for the confirmation ...
goatman64
Posts: 31
Joined: Wed Oct 21, 2015 7:41 pm

Re: Coral edgetpu on ubuntu 20.04.1 LTS zm 1.35.10

Post by goatman64 »

Missed the step to grant the www-data user access

sudo usermod -a -G plugdev www-data

Works! Huge time improvement over CPU based detection.
Magic919
Posts: 1381
Joined: Wed Sep 18, 2013 6:56 am

Re: Coral edgetpu on ubuntu 20.04.1 LTS zm 1.35.10

Post by Magic919 »

Well done spotting it. Glad it's all ok now.
-
coyote386
Posts: 5
Joined: Sun Nov 08, 2020 6:34 pm

Re: Coral edgetpu on ubuntu 20.04.1 LTS zm 1.35.10

Post by coyote386 »

Hi, yesterday i've installed my Coral Edge tpu.
It's faster than before, but i would like to know, how long is the time between alarm start and sending notification?!
Mine is about 13-15 seconds between alarm start and sending notification, is this normal? I hoped, that this is just a few seconds or realtime
User avatar
asker
Posts: 1553
Joined: Sun Mar 01, 2015 12:12 pm

Re: Coral edgetpu on ubuntu 20.04.1 LTS zm 1.35.10

Post by asker »

You'll have to read the logs.
There are many items:

1. Time the ES takes to detect the event
2. time in wait (specified by wait parameter)
3. time to download images
4. time to detect
5. time to send
6. time your phone receives it

1 is around 5 seconds. You can change it
2. is immediate unless there is an explict wait statement
3. Depends - how much ever it takes to download
4. is when your TPU kicks in. If you are not using mlapi, the model will be loaded each time (longer - around 2-3 secs or more)
5. usually immediate
6. depends on FCM servers and your phone - not under my control.

Bottom line, please read logs, see where the delay is. It will also tell you other things like whether TPU is used, some other model/cpu is used etc.
I no longer work on zmNinja, zmeventnotification, pyzm or mlapi. I may respond on occasion based on my available time/interest.

Please read before posting:
How to set up logging properly
How to troubleshoot and report - ES
How to troubleshoot and report - zmNinja
ES docs
zmNinja docs
coyote386
Posts: 5
Joined: Sun Nov 08, 2020 6:34 pm

Re: Coral edgetpu on ubuntu 20.04.1 LTS zm 1.35.10

Post by coyote386 »

@asker

thanks for your reply.
Ok, first step i have changed to 1 sec, so now i'm on ~5-7sec from event start to send notification.

Now, i installed mlapi on the same VM. If i try to start the server with "python3 ./mlapi.py -c mlapiconfig.ini" it ends in an error like this:

Code: Select all

coyote@zoneminder:~/mlapi$ python3 ./mlapi.py -c mlapiconfig.ini
Initializing log
DEBUG: secret filename: ./secrets.ini
DEBUG: Secret token found in config: !MLAPI_SECRET_KEY
DEBUG: Secret token found in config: !PLATEREC_ALPR_KEY
DEBUG: Opening DB at ./db/db.json
DEBUG: DB engine ready
Dlib was compiled to use SSE41 instructions, but these aren't available on your machine.
Nov 09 2020 15:27:22.570703 [INF] Using simple log output (default)
Nov 09 2020 15:27:22.570836 [DBG 1] Initializing face recognition with model:hog upsample:1, jitters:0
Nov 09 2020 15:27:22.570938 [DBG 1] trained file not found, reading from images and doing training...
Nov 09 2020 15:27:22.570975 [DBG 1] If you are using a GPU and run out of memory, do the training using zm_train_faces.py. In this case, other models like yolo may already take up a lot of GPU memory
Nov 09 2020 15:27:22.571028 [INF] Using simple log output (default)
Nov 09 2020 15:27:22.571075 [DBG 1] Face Recognition library load time took: 0.002 milliseconds
Nov 09 2020 15:27:22.571290 [ERR] No known faces found to train, encoding file not created
Nov 09 2020 15:27:22.571347 [DBG 1] Face Recognition training took: 0.226 milliseconds
Nov 09 2020 15:27:22.571417 [ERR] Error loading KNN model: [Errno 2] No such file or directory: './known_faces/faces.dat'
Nov 09 2020 15:27:22.571496 [INF] Using simple log output (default)
Nov 09 2020 15:27:22.596498 [INF] Using simple log output (default)
Nov 09 2020 15:27:22.596609 [DBG 2] Semaphore: max:1, name:pyzm_tpu_lock, timeout:120
Nov 09 2020 15:27:22.596664 [DBG 1] Waiting for TPU lock...
Traceback (most recent call last):
  File "./mlapi.py", line 218, in <module>
    od_obj = ObjectDetect.Object(options=g.config)
  File "/usr/local/lib/python3.8/dist-packages/pyzm/ml/object.py", line 31, in __init__
    self.model = tpu.Tpu(options=options, logger=logger)
  File "/usr/local/lib/python3.8/dist-packages/pyzm/ml/coral_edgetpu.py", line 37, in __init__
    self.lock.acquire()
  File "/usr/local/lib/python3.8/dist-packages/portalocker/utils.py", line 330, in acquire
    if self.try_lock(filenames):
  File "/usr/local/lib/python3.8/dist-packages/portalocker/utils.py", line 349, in try_lock
    self.lock.acquire()
  File "/usr/local/lib/python3.8/dist-packages/portalocker/utils.py", line 152, in acquire
    fh = self._get_fh()
  File "/usr/local/lib/python3.8/dist-packages/portalocker/utils.py", line 199, in _get_fh
    return open(self.filename, self.mode, **self.file_open_kwargs)
PermissionError: [Errno 13] Permission denied: '/tmp/pyzm_tpu_lock.00.lock'
I found out, it is because "pyzm_tpu_lock.00.lock" owner is www-data, so if i start the server with my user it ends in permission error.
I changed owner to user "coyote" (just to test, if mlapi works) and now it starts:

Code: Select all

coyote@zoneminder:~/mlapi$ python3 ./mlapi.py -c mlapiconfig.ini
Initializing log
DEBUG: secret filename: ./secrets.ini
DEBUG: Secret token found in config: !MLAPI_SECRET_KEY
DEBUG: Secret token found in config: !PLATEREC_ALPR_KEY
DEBUG: Opening DB at ./db/db.json
DEBUG: DB engine ready
Dlib was compiled to use SSE41 instructions, but these aren't available on your machine.
Nov 09 2020 15:28:21.639711 [INF] Using simple log output (default)
Nov 09 2020 15:28:21.639864 [DBG 1] Initializing face recognition with model:hog upsample:1, jitters:0
Nov 09 2020 15:28:21.639992 [DBG 1] trained file not found, reading from images and doing training...
Nov 09 2020 15:28:21.640038 [DBG 1] If you are using a GPU and run out of memory, do the training using zm_train_faces.py. In this case, other models like yolo may already take up a lot of GPU memory
Nov 09 2020 15:28:21.640102 [INF] Using simple log output (default)
Nov 09 2020 15:28:21.640168 [DBG 1] Face Recognition library load time took: 0.002 milliseconds
Nov 09 2020 15:28:21.640306 [ERR] No known faces found to train, encoding file not created
Nov 09 2020 15:28:21.640360 [DBG 1] Face Recognition training took: 0.135 milliseconds
Nov 09 2020 15:28:21.640435 [ERR] Error loading KNN model: [Errno 2] No such file or directory: './known_faces/faces.dat'
Nov 09 2020 15:28:21.640489 [INF] Using simple log output (default)
Nov 09 2020 15:28:21.665235 [INF] Using simple log output (default)
Nov 09 2020 15:28:21.665342 [DBG 2] Semaphore: max:1, name:pyzm_tpu_lock, timeout:120
Nov 09 2020 15:28:21.665408 [DBG 1] Waiting for TPU lock...
Nov 09 2020 15:28:21.665723 [DBG 1] Lock acquired, TPU loading ./models/coral_edgetpu/ssd_mobilenet_v2_coco_quant_postprocess_edgetpu.tflite
Nov 09 2020 15:28:24.611096 [DBG 1] init TPU lock released
Nov 09 2020 15:28:24.611259 [DBG 1] TPU initialization (loading model from disk) took: 945.466 milliseconds
Nov 09 2020 15:28:24.612306 [INF] Using simple log output (default)
Nov 09 2020 15:28:24.612393 [INF] Using simple log output (default)
Nov 09 2020 15:28:24.612449 [DBG 1] PlateRecognizer ALPR initialized with url: https://api.platerecognizer.com/v1
INFO: --------| mlapi version:2.0.0 |--------
INFO: Starting server with max:1 processes
 * Serving Flask app "mlapi" (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
 * Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)
Now it works so far, i get an access token and your example
curl -H "Content-Type:application/json" -H "Authorization: Bearer ${ACCESS_TOKEN}" -XPOST -d "{\"url\":\"https://upload.wikimedia.org/wikipedia/ ... ngbird.jpg\"}" http://localhost:5000/api/v1/detect/object
works fine.



But does MLAPI start, if i boot up zoneminder? Unfortunately I found nothing in the documentation, I know this one will follow.

EDIT
Forget the sentence above....
Found the solution in mlapi.service. Works now.



Many many thanks for your works, i love zmeventnotification. It took me a long time to look through everything, but it's getting slow :lol:
coyote386
Posts: 5
Joined: Sun Nov 08, 2020 6:34 pm

Re: Coral edgetpu on ubuntu 20.04.1 LTS zm 1.35.10

Post by coyote386 »

@asker hopeyou can help me...

MLAPI doesn't work:

Code: Select all

2020-11-09 23:23:47	zmesdetect_m2		6120	INF	Falling back to local execution...	zm_detect.py	410
2020-11-09 23:23:47	zmesdetect_m2		6120	ERR	Error executing remote API: HTTPConnectionPool(host='192.168.66.248', port=5000): Max retries exceeded with url: /api/v1/login (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused'))	zm_detect.py	408
2020-11-09 23:23:47	zmesdetect_m2		6120	INF	Detecting using remote API Gateway http://192.168.66.248:5000/api/v1	zm_detect.py	46
systemctl status mlapi.service says running..

mlapi_error.log:

Code: Select all

coyote@zoneminder:/var/log/zm$ cat mlapi_error.log
Dlib was compiled to use SSE41 instructions, but these aren't available on your machine.
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/pyzm/ml/coral_edgetpu.py", line 37, in __init__
    self.lock.acquire()
  File "/usr/local/lib/python3.8/dist-packages/portalocker/utils.py", line 343, in acquire
    raise exceptions.AlreadyLocked()
portalocker.exceptions.AlreadyLocked

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "mlapi.py", line 218, in <module>
    od_obj = ObjectDetect.Object(options=g.config)
  File "/usr/local/lib/python3.8/dist-packages/pyzm/ml/object.py", line 31, in __init__
    self.model = tpu.Tpu(options=options, logger=logger)
  File "/usr/local/lib/python3.8/dist-packages/pyzm/ml/coral_edgetpu.py", line 45, in __init__
    raise ValueError ('Timeout waiting for TPU lock for {} seconds'.format(self.lock_timeout))
ValueError: Timeout waiting for TPU lock for 120 seconds
User avatar
asker
Posts: 1553
Joined: Sun Mar 01, 2015 12:12 pm

Re: Coral edgetpu on ubuntu 20.04.1 LTS zm 1.35.10

Post by asker »

Try updating pyzm to the latest version.
I no longer work on zmNinja, zmeventnotification, pyzm or mlapi. I may respond on occasion based on my available time/interest.

Please read before posting:
How to set up logging properly
How to troubleshoot and report - ES
How to troubleshoot and report - zmNinja
ES docs
zmNinja docs
coyote386
Posts: 5
Joined: Sun Nov 08, 2020 6:34 pm

Re: Coral edgetpu on ubuntu 20.04.1 LTS zm 1.35.10

Post by coyote386 »

Thank you, MLAPI seems to be working now.
At least for a while, until the Coral TPU gets out.
It doesn't seem to work properly when I run Zoneminder in an Ubuntu VM on proxmox with the USB Coral Stick passed through. I get this error after a while:

[1458.709460] usb 3-1: reset SuperSpeed ​​Gen 1 USB device number 2 using xhci_hcd
[1458.730320] usb 3-1: LPM exit latency is zeroed, disabling LPM.
[1549.400848] xhci_hcd 0000: 01: 1b.0: ERROR Transfer event TRB DMA ptr not part of current TD ep_index 2 comp_code 1
[1549.400953] xhci_hcd 0000: 01: 1b.0: Looking for event-dma 000000012afb8de0 trb-start 000000012afb86d0 trb-end 000000012afb8dd0 seg-start 000000012afb8000 seg-end 000000012afb8ff0
[1549.400955] xhci_hcd 0000: 01: 1b.0: ERROR Transfer event TRB DMA ptr not part of current TD ep_index 2 comp_code 1
[1549.401000] xhci_hcd 0000: 01: 1b.0: Looking for event-dma 000000012afb8df0 trb-start 000000012afb86d0 trb-end 000000012afb8dd0 seg-start 000000012afb8000 seg-end 000000012afb8ff0
[1549.401002] xhci_hcd 0000: 01: 1b.0: ERROR Transfer event TRB DMA ptr not part of current TD ep_index 2 comp_code 1
[1549.401047] xhci_hcd 0000: 01: 1b.0: Looking for event-dma 000000012afb8e00 trb-start 000000012afb86d0 trb-end 000000012afb8dd0 seg-start 000000012afb8000 seg-end 000000012afb8ff0
[1549.401048] xhci_hcd 0000: 01: 1b.0: ERROR Transfer event TRB DMA ptr not part of current TD ep_index 2 comp_code 1
[1549.401093] xhci_hcd 0000: 01: 1b.0: Looking for event-dma 000000012afb8e10 trb-start 000000012afb86d0 trb-end 000000012afb8dd0 seg-start 000000012afb8000 seg-end 000000012afb8ff0
[1549.401094] xhci_hcd 0000: 01: 1b.0: ERROR Transfer event TRB DMA ptr not part of current TD ep_index 2 comp_code 1
[1549.401139] xhci_hcd 0000: 01: 1b.0: Looking for event-dma 000000012afb8e20 trb-start 000000012afb86d0 trb-end 000000012afb8dd0 seg-start 000000012afb8000 seg-end 000000012afb8ff0
[1549.401140] xhci_hcd 0000: 01: 1b.0: ERROR Transfer event TRB DMA ptr not part of current TD ep_index 2 comp_code 1
[1549.401185] xhci_hcd 0000: 01: 1b.0: Looking for event-dma 000000012afb8e30 trb-start 000000012afb86d0 trb-end 000000012afb8dd0 seg-start 000000012afb8000 seg-end 000000012afb8ff0
[1549.401186] xhci_hcd 0000: 01: 1b.0: ERROR Transfer event TRB DMA ptr not part of current TD ep_index 2 comp_code 1
[1549.401231] xhci_hcd 0000: 01: 1b.0: Looking for event-dma 000000012afb8e40 trb-start 000000012afb86d0 trb-end 000000012afb8dd0 seg-start 000000012afb8000 seg-end 000000012afb8ff0
[1549.992776] xhci_hcd 0000: 01: 1b.0: ERROR Transfer event TRB DMA ptr not part of current TD ep_index 2 comp_code 13
[1549.992841] xhci_hcd 0000: 01: 1b.0: Looking for event-dma 000000012afb8e50 trb-start 000000012afb86d0 trb-end 000000012afb8de0 seg-start 000000012afb8000 seg-end 000000012afb8ff0
User avatar
asker
Posts: 1553
Joined: Sun Mar 01, 2015 12:12 pm

Re: Coral edgetpu on ubuntu 20.04.1 LTS zm 1.35.10

Post by asker »

Unfortunately, that is not an area of I would have any idea about (VM pass throughs and associated challenges). Note that the google coral github repo is very helpful as well. There are docker related issues there as well https://github.com/google-coral/edgetpu/issues/166
I read someone else post about similar challenges, but its not something I can help with.
I no longer work on zmNinja, zmeventnotification, pyzm or mlapi. I may respond on occasion based on my available time/interest.

Please read before posting:
How to set up logging properly
How to troubleshoot and report - ES
How to troubleshoot and report - zmNinja
ES docs
zmNinja docs
Locked