new coral models

[VicThor]

Hey all,

On the coral model page there are quite a few new models, like EfficientDet-Lite3 with a better precision.
Anyone got the new models running?

https://coral.ai/models/object-detection/

I've tried but no go. Doesn't seem to do detection on it via TPU.

[tsp84]

If it's a simple coral model already converted to the coral model framework it should literally be as easy as switching the absolute path from the original model to the new model. I'll take a look later today and see what's up.

[tsp84]

It worked for me, all I did was change the .tflite absolute path ->

Code: Select all

10/01/21 14:59:21.79152 zm_mlapi[7619] DBG2 detect_sequence:296 [ml: loading 'object' sequence 'coral::EfficientDet_lite3' (1 of 5) with options-> {'name': 'coral::EfficientDet_lite3', 'object_weights': '/home/t
  yler/PycharmProjects/my_mlapi/mlapi/models/coral_edgetpu/efficientdet_lite3_512_ptq_edgetpu.tflite', 'object_labels': '/home/tyler/PycharmProjects/my_mlapi/mlapi/models/coral_edgetpu/coco_indexed.names', 'object
  _min_confidence': 0.6, 'object_framework': 'coral_edgetpu', 'tpu_max_processes': 3, 'tpu_max_lock_wait': 30, 'max_detection_size': '100%', 'disable_locks': 'no'}]

10/01/21 14:59:21.907395 zm_mlapi[7619] DBG2 yolo:36 [portalock: [max: 3] - [timeout: 60s] - [name: pyzm_uid1000_GPU_lock]]
  10/01/21 14:59:21.909423 zm_mlapi[7619] DBG2 detect_sequence:285 [ml: skipping sequence 'yolo v5::Coral Ultralytics' (4 of 5) as it is disabled]
  10/01/21 14:59:21.911449 zm_mlapi[7619] DBG2 detect_sequence:285 [ml: skipping sequence 'AWS rekognition (PAID)' (5 of 5) as it is disabled]
  10/01/21 14:59:21.913584 zm_mlapi[7619] DBG2 coral_edgetpu:52 [Waiting for pyzm_uid1000_TPU_lock portalock...]
  10/01/21 14:59:21.915994 zm_mlapi[7619] DBG2 coral_edgetpu:54 [portalock: got 'pyzm_uid1000_TPU_lock']
  10/01/21 14:59:21.918123 zm_mlapi[7619] DBG2 coral_edgetpu:52 [Waiting for pyzm_uid1000_TPU_lock portalock...]
  10/01/21 14:59:21.920536 zm_mlapi[7619] DBG2 coral_edgetpu:54 [portalock: got 'pyzm_uid1000_TPU_lock']
  10/01/21 14:59:21.92285 zm_mlapi[7619] DBG2 yolo:76 [portalock: acquired -> 'pyzm_uid1000_GPU_lock']
  10/01/21 14:59:21.924751 zm_mlapi[7619] DBG2 detect_sequence:1227 [frame: s-112 [strategy:'first'] (1 of 8) - model: 'object' [strategy:'first'] (1 of 1) - sequence: 'coral::EfficientDet_lite3' [strategy:'most']
   (1 of 3)]
  10/01/21 14:59:21.926872 zm_mlapi[7619] DBG2 coral_edgetpu:206 [coral: pre processing resize for coral model requested from 800*450 -> 320*320]
  10/01/21 14:59:21.933409 zm_mlapi[7619] DBG2 coral_edgetpu:49 [pyzm_uid1000_TPU_lock portalock already acquired]
  10/01/21 14:59:21.93575 zm_mlapi[7619] DBG1 coral_edgetpu:98 [coral: loading model data from sequence 'coral::EfficientDet_lite3' ]
  10/01/21 14:59:24.637519 zm_mlapi[7619] DBG1 coral_edgetpu:146 [perf:coral: initialization -> loading 'efficientdet_lite3_512_ptq_edgetpu.tflite' took: 2699.28 ms]
  10/01/21 14:59:24.639813 zm_mlapi[7619] DBG1 coral_edgetpu:222 [coral: 'coral::EfficientDet_lite3' input image: 320w*320h]
  10/01/21 14:59:24.879413 zm_mlapi[7619] DBG2 coral_edgetpu:71 [portalock: released 'pyzm_uid1000_TPU_lock']
  10/01/21 14:59:24.881915 zm_mlapi[7619] DBG1 coral_edgetpu:240 [perf:coral: 'coral::EfficientDet_lite3' detection took: 236.83 ms]
  10/01/21 14:59:24.885298 zm_mlapi[7619] DBG2 coral_edgetpu:259 [coral: the image was resized before processing, scaling bounding boxes in image back up by factors of -> x=2.5 y=1.406]
  10/01/21 14:59:24.889065 zm_mlapi[7619] DBG1 coral_edgetpu:279 [coral returning [] -- [] -- []]
  10/01/21 14:59:24.891126 zm_mlapi[7619] DBG2 detect_sequence:1304 [ml: model: 'object' seq: 'coral::EfficientDet_lite3' found 0 detections -> ]

Ill keep this model loaded and compare it to the other coral models. So far in my experiance though, yolov4 with the pretrained weights is far superior to the TPU stuff for accuracy. I am working on training a yolo model for each of my monitors and that should be the end all. 2 - 3 models per monitor summer/winter/edge cases that are trained with both day (full color) and night (IR colorwash) images. False positives/negatives shouldnt be an issue after the training.

How are you finding the TPU models to be?

[VicThor]

Wonder why mine did not work, will have another shot, maybe i need more processes for the TPU (currently only 1 on mine). Did you try when it is detecting something? think you need a resize as well:
coral_edgetpu:206 [coral: pre processing resize for coral model requested from 800*450 -> 320*320] to 512x512 ?

I too find TPU to be less precise than yolo4 but it is faster.

[tsp84]

Yes i do need a resize, thats for the mobiledet_ssd lite series and I only swapped the model out of the sequence and left all the params for mobiledet. Yes you must have more than 1 tpu process max if you have more than 1 tpu sequence BUT it should of took the 120 seconds to timeout before throwing the timeout exception. Ill run it on a past event with previous deteections and post the output ->

Code: Select all

10/01/21 15:53:42.137438 zm_mlapi[7619] DBG2 detect_sequence:1227 [frame: s-132 [strategy:'first'] (1 of 8) - model: 'object' [strategy:'first'] (1 of 1) - sequence: 'coral::EfficientDet_lite3' [strategy:'most']
   (1 of 3)]
  10/01/21 15:53:42.139672 zm_mlapi[7619] DBG2 coral_edgetpu:206 [coral: pre processing resize for coral model requested from 800*450 -> 320*320]
  10/01/21 15:53:42.145814 zm_mlapi[7619] DBG2 coral_edgetpu:52 [Waiting for pyzm_uid1000_TPU_lock portalock...]
  10/01/21 15:53:42.14853 zm_mlapi[7619] DBG2 coral_edgetpu:54 [portalock: got 'pyzm_uid1000_TPU_lock']
  10/01/21 15:53:42.15078 zm_mlapi[7619] DBG1 coral_edgetpu:222 [coral: 'coral::EfficientDet_lite3' input image: 320w*320h]
  10/01/21 15:53:42.390982 zm_mlapi[7619] DBG2 coral_edgetpu:71 [portalock: released 'pyzm_uid1000_TPU_lock']
  10/01/21 15:53:42.393339 zm_mlapi[7619] DBG1 coral_edgetpu:240 [perf:coral: 'coral::EfficientDet_lite3' detection took: 237.38 ms]
  10/01/21 15:53:42.395568 zm_mlapi[7619] DBG2 coral_edgetpu:259 [coral: the image was resized before processing, scaling bounding boxes in image back up by factors of -> x=2.5 y=1.406]
  10/01/21 15:53:42.397758 zm_mlapi[7619] DBG1 coral_edgetpu:279 [coral returning ['car'] -- [[175, 53, 298, 104]] -- [0.875]]
  10/01/21 15:53:42.399715 zm_mlapi[7619] DBG2 detect_sequence:1304 [ml: model: 'object' seq: 'coral::EfficientDet_lite3' found 1 detection -> car]
  10/01/21 15:53:42.401781 zm_mlapi[7619] DBG1 detect_sequence:490 [>>> detected 'car (1/1)' confidence: 0.88]
  10/01/21 15:53:42.404372 zm_mlapi[7619] DBG1 detect_sequence:539 ['car (1/1)' minimum confidence found: (object_min_conf:sequence->coral::EfficientDet_lite3) -> '0.6']
  10/01/21 15:53:42.406505 zm_mlapi[7619] DBG2 detect_sequence:587 [checking if 'car (1/1)' @ [(175, 53), (298, 53), (298, 104), (175, 104)] is inside polygon/zone 'front_yard' located at [(0, 182), (452, 115), (7
  96, 235), (796, 446), (0, 449)]]
  10/01/21 15:53:42.408648 zm_mlapi[7619] DBG2 detect_sequence:680 [intersection: 'car (1/1)' does not intersect zone: front_yard, removing...]
  10/01/21 15:53:42.41071 zm_mlapi[7619] DBG2 detect_sequence:970 [detection: 'car (1/1)' has FAILED filtering]
  10/01/21 15:53:42.412767 zm_mlapi[7619] DBG2 detect_sequence:1227 [frame: s-132 [strategy:'first'] (1 of 8) - model: 'object' [strategy:'first'] (1 of 1) - sequence: 'coral::MobileNETv2-SSD' [strategy:'most'] (2
   of 3)]

here is mobiledetv2 output ->

Code: Select all

[ml: model: 'object' seq: 'coral::MobileNETv2-SSD' found 1 detection -> person]

and yolo output of same frame ->

Code: Select all

[ml: model: 'object' seq: 'DarkNet::v4 Pre-Trained' found 4 detections -> car, person, car, truck]

[VicThor]

I feel like an idiot, i had the path wrong to the tflite file

It is defo an improvement over SSDLite MobileDet.
Ran it manually over a previously detected car at 70% (by tpu) and and I got a 76%.
Ran it manually over a previously detected car 99% (by yolo, coral did not detect or bellow 60%) and I got: detected:(coral) car:82%

I'd say pretty sweet.

[juan11perez]

Good day. So these are the models currently in mlapi

Code: Select all

https://dl.google.com/coral/canned_models/coco_labels.txt
https://github.com/google-coral/edgetpu/raw/master/test_data/ssd_mobilenet_v2_coco_quant_postprocess_edgetpu.tflite
https://github.com/google-coral/test_data/raw/master/ssdlite_mobiledet_coco_qat_postprocess_edgetpu.tflite
https://github.com/google-coral/test_data/raw/master/ssd_mobilenet_v2_face_quant_postprocess_edgetpu.tflite

to test EfficientDet-Lite3 we need only substitute this model:

Code: Select all

https://github.com/google-coral/test_data/raw/master/ssdlite_mobiledet_coco_qat_postprocess_edgetpu.tflite

for this?

Code: Select all

https://github.com/google-coral/test_data/raw/master/efficientdet_lite3_512_ptq_edgetpu.tflite

rest unchanged?

thank you.

[VicThor]

Pretty much, and update mlapiconfig.ini to reflect that

[tsp84]

Download the coral tflite model and place its absolute path into the tpu 'object_weights' variable.

Example:
the efficentdet3.tflite file is in /home/me/Downloads

The following is a 'sequence' inside of ml_sequence -> object -> sequences

Code: Select all

    - name: 'coral::EfficientDet v3'
        #      enabled: 'no'
        object_weights: '/home/me/Downloads/efficientdet3.tflite'
        # Labels are the same as mobiledet
        object_labels: '{{tpu_object_labels}}'
        # adjust these if you want
        object_min_confidence: '{{tpu_min_confidence}}'
        object_framework: '{{tpu_object_framework}}'
        tpu_max_processes: '{{tpu_max_processes}}'
        tpu_max_lock_wait: '{{tpu_max_lock_wait}}'
        max_detection_size: '{{max_detection_size}}'

The syntax will be different for you as your config files will be in the .ini format but this should give you the general idea.

** make sure that the tflite file is readable by the user running mlapi

[juan11perez]

Thank you all for your reply. Much appreciated