Eventnotification Issue (timeout)

Discussion topics related to mobile applications and ZoneMinder Event Server (including machine learning)
cafftab
Posts: 48
Joined: Mon May 06, 2019 6:03 pm

Eventnotification Issue (timeout)

Post by cafftab »

I had my hard drive crash and reinstalled everything to a new drive using ubuntu 20.04 and zoneminder 1.34.25 I have everything working with a letsencript certificate I can view monitors and events on zmninga. but I seem to have an issue with detection hooks, things pause at the username/password the after a minute or so the error follows. I followed installation and step by step but I am stuck here.



==> /var/log/zm/zmesdetect_m1.log <==
04/18/21 12:41:29 zmesdetect_m1[5417] DBG2 utils.py:383 [Secret token found in config: !ZM_PASSWORD]
04/18/21 12:41:29 zmesdetect_m1[5417] DBG2 utils.py:383 [Secret token found in config: !ZM_API_PORTAL]
04/18/21 12:41:29 zmesdetect_m1[5417] DBG2 utils.py:383 [Secret token found in config: !PLATEREC_ALPR_KEY]
04/18/21 12:41:29 zmesdetect_m1[5417] DBG1 utils.py:445 [allowing self-signed certs to work...]
04/18/21 12:41:29 zmesdetect_m1[5417] DBG4 utils.py:455 [Now checking for monitor overrides]
04/18/21 12:41:29 zmesdetect_m1[5417] DBG4 utils.py:523 [Finally, doing parameter substitution]
04/18/21 12:41:29 zmesdetect_m1[5417] INF zm_detect.py:296 [Importing local classes for Object/Face]
04/18/21 12:41:29 zmesdetect_m1[5417] INF zm_detect.py:321 [Connecting with ZM APIs]
04/18/21 12:41:29 zmesdetect_m1[5417] DBG2 api.py:67 [API SSL certificate check has been disbled]
04/18/21 12:41:29 zmesdetect_m1[5417] DBG1 api.py:176 [using username/password for login]

==> /var/log/zm/zmesdetect_m1.log <==
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 581, in post
return self.request('POST', url, data=data, json=json, **kwargs)
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 533, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 646, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python3/dist-packages/requests/adapters.py", line 516, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='myserver.com', port=443): Max retries exceeded with url: /zm/api/host/login.json (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f943b8c9ac0>: Failed to establish a new connection: [Errno 110] Connection timed out'))
]
User avatar
asker
Posts: 1553
Joined: Sun Mar 01, 2015 12:12 pm

Re: Eventnotification Issue (timeout)

Post by asker »

The detection scripts can't access ZM

requests.exceptions.ConnectionError: HTTPSConnectionPool(host='myserver.com', port=443): Max retries exceeded with url:

Check your secrets.ini and make sure the server names are correct (and unless you changed the hostname to myserver.com for the post, it is unlikely that is your real ZM server)
I no longer work on zmNinja, zmeventnotification, pyzm or mlapi. I may respond on occasion based on my available time/interest.

Please read before posting:
How to set up logging properly
How to troubleshoot and report - ES
How to troubleshoot and report - zmNinja
ES docs
zmNinja docs
cafftab
Posts: 48
Joined: Mon May 06, 2019 6:03 pm

Re: Eventnotification Issue (timeout)

Post by cafftab »

sorry it did take the actual server name out
cafftab
Posts: 48
Joined: Mon May 06, 2019 6:03 pm

Re: Eventnotification Issue (timeout)

Post by cafftab »

Update, I disabled hooks and now my zmninja notifications come to my phone. It seems with hooks on, I can't get past the username/password line. I assume that is a zoneminder username and password that I have in my secrets.ini.

The DBG2 line saying "API SSL certificate check has been disbled" is coming from where? I don't have self signed certificates and I have SSl turned on in the configuration file as well as on zmninja
User avatar
asker
Posts: 1553
Joined: Sun Mar 01, 2015 12:12 pm

Re: Eventnotification Issue (timeout)

Post by asker »

No idea. Post full debug logs of zm_detect from the start. You may want to read the documentation on how to manually run zm_detect with --debug
I no longer work on zmNinja, zmeventnotification, pyzm or mlapi. I may respond on occasion based on my available time/interest.

Please read before posting:
How to set up logging properly
How to troubleshoot and report - ES
How to troubleshoot and report - zmNinja
ES docs
zmNinja docs
cafftab
Posts: 48
Joined: Mon May 06, 2019 6:03 pm

Re: Eventnotification Issue (timeout)

Post by cafftab »

This is the complete log when a detection takes place. Any help pointing me in the right direction would be appreciated


04/20/21 09:01:03 zmesdetect_m1[10116] DBG2 utils.py:383 [Secret token found in config: !ZM_USER]
04/20/21 09:01:03 zmesdetect_m1[10116] DBG2 utils.py:383 [Secret token found in config: !ZM_PASSWORD]
04/20/21 09:01:03 zmesdetect_m1[10116] DBG2 utils.py:383 [Secret token found in config: !ZM_API_PORTAL]
04/20/21 09:01:03 zmesdetect_m1[10116] DBG2 utils.py:383 [Secret token found in config: !PLATEREC_ALPR_KEY]
04/20/21 09:01:03 zmesdetect_m1[10116] DBG1 utils.py:447 [strict SSL cert checking is on...]
04/20/21 09:01:03 zmesdetect_m1[10116] DBG4 utils.py:455 [Now checking for monitor overrides]
04/20/21 09:01:03 zmesdetect_m1[10116] DBG4 utils.py:523 [Finally, doing parameter substitution]
04/20/21 09:01:03 zmesdetect_m1[10116] INF zm_detect.py:296 [Importing local classes for Object/Face]
04/20/21 09:01:04 zmesdetect_m1[10116] INF zm_detect.py:321 [Connecting with ZM APIs]
04/20/21 09:01:04 zmesdetect_m1[10116] DBG1 api.py:176 [using username/password for login]
04/20/21 09:03:14 zmesdetect_m1[10116] FAT zm_detect.py:554 [Unrecoverable error:HTTPSConnectionPool(host='centinelview.chickenkiller.com', port=443): Max retries exceeded with url: /zm/api/host/login.json (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f1470498b20>: Failed to establish a new connection: [Errno 110] Connection timed out')) Traceback:Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 159, in _new_conn
conn = connection.create_connection(
File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 84, in create_connection
raise err
File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 74, in create_connection
sock.connect(sa)
TimeoutError: [Errno 110] Connection timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 665, in urlopen
httplib_response = self._make_request(
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 376, in _make_request
self._validate_conn(conn)
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 996, in _validate_conn
conn.connect()
File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 314, in connect
conn = self._new_conn()
File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 171, in _new_conn
raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.VerifiedHTTPSConnection object at 0x7f1470498b20>: Failed to establish a new connection: [Errno 110] Connection timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/requests/adapters.py", line 439, in send
resp = conn.urlopen(
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 719, in urlopen
retries = retries.increment(
File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 436, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='my.real.servername.com, port=443): Max retries exceeded with url: /zm/api/host/login.json (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f1470498b20>: Failed to establish a new connection: [Errno 110] Connection timed out'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/var/lib/zmeventnotification/bin/zm_detect.py", line 549, in <module>
main_handler()
File "/var/lib/zmeventnotification/bin/zm_detect.py", line 322, in main_handler
zmapi = zmapi.ZMApi(options=api_options)
File "/usr/local/lib/python3.8/dist-packages/pyzm/api.py", line 75, in __init__
self._login()
File "/usr/local/lib/python3.8/dist-packages/pyzm/api.py", line 188, in _login
r = self.session.post(url, data=data)
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 581, in post
return self.request('POST', url, data=data, json=json, **kwargs)
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 533, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 646, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python3/dist-packages/requests/adapters.py", line 516, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='my.real.servername.com', port=443): Max retries exceeded with url: /zm/api/host/login.json (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f1470498b20>: Failed to establish a new connection: [Errno 110] Connection timed out'))
]
Magic919
Posts: 1381
Joined: Wed Sep 18, 2013 6:56 am

Re: Eventnotification Issue (timeout)

Post by Magic919 »

The API responds when I test it (albeit not authenticated). Don't know why Python requests is having trouble with it. Have you tested this from the machine that runs ZMES? I'm wondering about DNS, for example.
-
cafftab
Posts: 48
Joined: Mon May 06, 2019 6:03 pm

Re: Eventnotification Issue (timeout)

Post by cafftab »

The logs were from the machine that was running zmes.
Magic919
Posts: 1381
Joined: Wed Sep 18, 2013 6:56 am

Re: Eventnotification Issue (timeout)

Post by Magic919 »

Yes, clearly.
-
User avatar
asker
Posts: 1553
Joined: Sun Mar 01, 2015 12:12 pm

Re: Eventnotification Issue (timeout)

Post by asker »

Bottom line, zm_detect cannot reach your ZM server. Things to look at:

a) Check the value of ZM_PORTAL in /etc/zm/secrets.ini and make sure it is accurate
b) Make sure the portal keyword in objectconfig.ini points to !ZM_PORTAL
c) Make sure you are indeed using HTTPS as your logs show
d) From a terminal try to do a "telnet <the servername you see in the logs> 443". Are you able to connect? That is where zm_detect is trying to connect to.

If you still don't know what is going on:

a) Modify ZM_PORTAL value in /etc/zm/secrets.ini to "BADNAME.BAD" or something like that and see if its being picked up in logs. Then at least you know the files you think have the right values are the files that are being picked up.
I no longer work on zmNinja, zmeventnotification, pyzm or mlapi. I may respond on occasion based on my available time/interest.

Please read before posting:
How to set up logging properly
How to troubleshoot and report - ES
How to troubleshoot and report - zmNinja
ES docs
zmNinja docs
cafftab
Posts: 48
Joined: Mon May 06, 2019 6:03 pm

Re: Eventnotification Issue (timeout)

Post by cafftab »

Thanks. Still having issues but I changed the portal to my localhost and I got past this step but stopped again returning the image using the servername. I must have something configured wrong in apache or a file permission issue because I also noticed I can't reach my server using the domain name as I could before my new hard drive was installed. Can't even display the apache default page unless I use the local server address.
User avatar
asker
Posts: 1553
Joined: Sun Mar 01, 2015 12:12 pm

Re: Eventnotification Issue (timeout)

Post by asker »

I'd encourage you to read the documents properly...
This is covered in both key principles and the FAQ

Your images are controlled by the value in picture_url in zmeventnotification.ini which usually uses a value specified in secrets.ini (ZMES_PICTURE_URL)

However, doing localhost for the picture won't work anyway. You'll need to resolve your domain/DNS/firewall issues.
I no longer work on zmNinja, zmeventnotification, pyzm or mlapi. I may respond on occasion based on my available time/interest.

Please read before posting:
How to set up logging properly
How to troubleshoot and report - ES
How to troubleshoot and report - zmNinja
ES docs
zmNinja docs
Magic919
Posts: 1381
Joined: Wed Sep 18, 2013 6:56 am

Re: Eventnotification Issue (timeout)

Post by Magic919 »

As I said before, I can reach the API from here, which says it's working. I could also see the default Apache page.

The fact the dynamic DNS hostname I used is in the logs, says that bit works too.
-
User avatar
asker
Posts: 1553
Joined: Sun Mar 01, 2015 12:12 pm

Re: Eventnotification Issue (timeout)

Post by asker »

@Magic919 not sure who you are responding to. This post is in response to the OP who started the thread.
Thanks. Still having issues but I changed the portal to my localhost and I got past this step but stopped again returning the image using the servername. I must have something configured wrong in apache or a file permission issue because I also noticed I can't reach my server using the domain name as I could before my new hard drive was installed. Can't even display the apache default page unless I use the local server address.
I no longer work on zmNinja, zmeventnotification, pyzm or mlapi. I may respond on occasion based on my available time/interest.

Please read before posting:
How to set up logging properly
How to troubleshoot and report - ES
How to troubleshoot and report - zmNinja
ES docs
zmNinja docs
Magic919
Posts: 1381
Joined: Wed Sep 18, 2013 6:56 am

Re: Eventnotification Issue (timeout)

Post by Magic919 »

Mostly the OP, and as an FYI for you, so you can see what works.
-
Post Reply