Page 1 of 2
zmeventnotification: Error parsing config:/etc/zm/objectconfig.ini
Posted: Mon Jun 15, 2020 4:28 am
by SkippyDo
I'm trying to debug but cannot find where to enable additional debugging. This is what gets logged in ZM's logs (zmesdetect_m1.log):
Code: Select all
2020-06-14 19:29:25 zmesdetect_m1 26865 ERR Error was:<urlopen error [Errno -2] Name or service not known> utils.py 328
2020-06-14 19:29:24 zmesdetect_m1 26865 ERR Error parsing config:/etc/zm/objectconfig.ini utils.py 327
I've scoured secrets.ini and objectconfig.ini and cannot figure out where there could be any sort of formatting/syntax errors.
It appears that the value of ZMES_PICTURE_URL from the secrets.ini file isn't being recognized.
Per FAQ testing suggestions I've run zm_event_start.sh manually. I modified it so I could see it's processing. Here's the relevant output:
Code: Select all
+ CONFIG_FILE=/etc/zm/objectconfig.ini
+ EVENT_PATH=
+ DETECTION_SCRIPT=(/var/lib/zmeventnotification/bin/zm_detect.py --monitorid $2 --eventid $1 --config "${CONFIG_FILE}" --eventpath "${EVENT_PATH}")
++ /var/lib/zmeventnotification/bin/zm_detect.py --monitorid 1 --eventid 837 --config /etc/zm/objectconfig.ini --eventpath ''
++ grep detected:
+ RESULTS=
+ _RETVAL=1
+ [[ ! -z '' ]]
+ echo
+ exit 1
I'm not sure what "eventpath" ought to be. Just figured that there's a default value somewhere.
Re: zmeventnotification: Error parsing config:/etc/zm/objectconfig.ini
Posted: Mon Jun 15, 2020 8:42 pm
by asker
Turn on debug logs for the ES
Code: Select all
2020-06-14 19:29:25 zmesdetect_m1 26865 ERR Error was:<urlopen error [Errno -2] Name or service not known> utils.py 328
means that your URL is unreadable/wrong. Debug logs will tell you if it read secrets or not.
Re: zmeventnotification: Error parsing config:/etc/zm/objectconfig.ini
Posted: Tue Jun 16, 2020 12:35 am
by SkippyDo
I can copy and paste the ZMES_PICTURE_URL string in to a browser, changing EVENTID to a valid event number, and it works perfectly well.
OK. Turned on debug for zmesdetect_m1. Here's what I got:
Code: Select all
animation: Traceback:Traceback (most recent call last): File "/usr/lib/python3.7/urllib/request.py", line 1317, in do_open encode_chunked=req.has_header('Transfer-encoding')) File "/usr/lib/python3.7/http/client.py", line 1244, in request self._send_request(method, url, body, headers, encode_chunked) File "/usr/lib/python3.7/http/client.py", line 1290, in _send_request self.endheaders(body, encode_chunked=encode_chunked) File "/usr/lib/python3.7/http/client.py", line 1239, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "/usr/lib/python3.7/http/client.py", line 1026, in _send_output self.send(msg) File "/usr/lib/python3.7/http/client.py", line 966, in send self.connect() File "/usr/lib/python3.7/http/client.py", line 1399, in connect super().connect() File "/usr/lib/python3.7/http/client.py", line 938, in connect (self.host,self.port), self.timeout, self.source_address) File "/usr/lib/python3.7/socket.py", line 707, in create_connection for res in getaddrinfo(host, port, 0, SOCK_STREAM): File "/usr/lib/python3.7/socket.py", line 748, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags):socket.gaierror: [Errno -2] Name or service not knownDuring handling of the above exception, another exception occurred:Traceback (most recent call last): File "/var/lib/zmeventnotification/bin/zm_detect.py", line 219, in <module> args) File "/usr/local/lib/python3.7/dist-packages/zmes_hook_helpers/utils.py", line 190, in download_files input_file = opener.open(url) File "/usr/lib/python3.7/urllib/request.py", line 525, in open response = self._open(req, data) File "/usr/lib/python3.7/urllib/request.py", line 543, in _open '_open', req) File "/usr/lib/python3.7/urllib/request.py", line 503, in _call_chain result = func(*args) File "/usr/lib/python3.7/urllib/request.py", line 1360, in https_open context=self._context, check_hostname=self._check_hostname) File "/usr/lib/python3.7/urllib/request.py", line 1319, in do_open raise URLError(err)urllib.error.URLError: <urlopen error [Errno -2] Name or service not known>
URL path-related stuff from secrets.ini:
Code: Select all
ZMES_PICTURE_URL=https://<DELETED - my server's FQDN>/zm/index.php?view=image&eid=EVENTID&fid=objdetect&width=600
ZM_PORTAL=https://<DELETED - my server's FQDN>/zm
ZM_API_PORTAL=https://<DELETED - my server's FQDN>/zm/api
URL path-related stuff from objectconfig.ini:
Code: Select all
portal=!ZM_PORTAL
api_portal=!ZM_API_PORTAL
URL path-related stuff from zmeventnotification.ini:
Re: zmeventnotification: Error parsing config:/etc/zm/objectconfig.ini
Posted: Tue Jun 16, 2020 2:19 pm
by asker
Please post full debug logs of zmesdetect_m1 from the time it starts to the error. Hard to understand context with just 1-2 lines of logs.
Re: zmeventnotification: Error parsing config:/etc/zm/objectconfig.ini
Posted: Wed Jun 17, 2020 1:06 am
by SkippyDo
OK [sorry]. I failed to catch the extra debugging output (which clearly shows that secrets.ini was being parsed): doing lots of different debugging and was getting a bit punch-drunk.
Code: Select all
06/15/20 19:09:31 zmesdetect_m1[15750] INF zm_detect.py:178 [---------| hook version: 5.15.4, ES version: 5.15 , OpenCV version: 4.3.0|------------]
06/15/20 19:09:31 zmesdetect_m1[15750] DBG-1 utils.py:267 [secret filename: /etc/zm/secrets.ini]
06/15/20 19:09:31 zmesdetect_m1[15750] DBG-2 utils.py:240 [Secret token found in config: !ZM_PORTAL]
06/15/20 19:09:31 zmesdetect_m1[15750] DBG-2 utils.py:240 [Secret token found in config: !ZM_API_PORTAL]
06/15/20 19:09:31 zmesdetect_m1[15750] DBG-2 utils.py:240 [Secret token found in config: !ZM_USER]
06/15/20 19:09:31 zmesdetect_m1[15750] DBG-2 utils.py:240 [Secret token found in config: !ZM_PASSWORD]
06/15/20 19:09:31 zmesdetect_m1[15750] DBG-2 utils.py:240 [Secret token found in config: !ML_USER]
06/15/20 19:09:31 zmesdetect_m1[15750] DBG-2 utils.py:240 [Secret token found in config: !ML_PASSWORD]
06/15/20 19:09:31 zmesdetect_m1[15750] DBG-2 utils.py:240 [Secret token found in config: !PLATEREC_ALPR_KEY]
06/15/20 19:09:31 zmesdetect_m1[15750] DBG-1 utils.py:290 [allowing self-signed certs to work...]
06/15/20 19:09:31 zmesdetect_m1[15750] DBG-2 utils.py:309 [[monitor-1] overrides key:match_past_detections with value:no]
06/15/20 19:09:31 zmesdetect_m1[15750] DBG-2 utils.py:309 [[monitor-1] overrides key:wait with value:5]
06/15/20 19:09:31 zmesdetect_m1[15750] DBG-2 utils.py:309 [[monitor-1] overrides key:detect_pattern with value:(person|car|motorbike|bus|truck|boat)]
06/15/20 19:09:31 zmesdetect_m1[15750] DBG-2 utils.py:315 [adding polygon: my_driveway_perimeter [0,0 480,88 1276,580 0,719]]
06/15/20 19:09:31 zmesdetect_m1[15750] DBG-2 utils.py:309 [[monitor-1] overrides key:resize with value:no]
06/15/20 19:09:31 zmesdetect_m1[15750] DBG-2 utils.py:309 [[monitor-1] overrides key:models with value:yolo]
06/15/20 19:09:31 zmesdetect_m1[15750] DBG-2 utils.py:345 [key [config] is '/var/lib/zmeventnotification/models/yolov3/yolov3.cfg' after substitution]
06/15/20 19:09:31 zmesdetect_m1[15750] DBG-2 utils.py:345 [key [weights] is '/var/lib/zmeventnotification/models/yolov3/yolov3.weights' after substitution]
06/15/20 19:09:31 zmesdetect_m1[15750] DBG-2 utils.py:345 [key [labels] is '/var/lib/zmeventnotification/models/yolov3/yolov3_classes.txt' after substitution]
06/15/20 19:09:31 zmesdetect_m1[15750] DBG-2 utils.py:345 [key [tiny_config] is '/var/lib/zmeventnotification/models/tinyyolo/yolov3-tiny.cfg' after substitution]
06/15/20 19:09:31 zmesdetect_m1[15750] DBG-2 utils.py:345 [key [tiny_weights] is '/var/lib/zmeventnotification/models/tinyyolo/yolov3-tiny.weights' after substitution]
06/15/20 19:09:31 zmesdetect_m1[15750] DBG-2 utils.py:345 [key [tiny_labels] is '/var/lib/zmeventnotification/models/tinyyolo/yolov3-tiny.txt' after substitution]
06/15/20 19:09:31 zmesdetect_m1[15750] DBG-2 utils.py:345 [key [known_images_path] is '/var/lib/zmeventnotification/known_faces' after substitution]
06/15/20 19:09:31 zmesdetect_m1[15750] DBG-2 utils.py:345 [key [unknown_images_path] is '/var/lib/zmeventnotification/unknown_faces' after substitution]
06/15/20 19:09:31 zmesdetect_m1[15750] INF zm_detect.py:207 [Importing local classes for Yolo/Face]
06/15/20 19:09:31 zmesdetect_m1[15750] INF utils.py:101 [Sleeping for 5 seconds before downloading]
06/15/20 19:09:36 zmesdetect_m1[15750] DBG-1 utils.py:189 [Trying to download https://<DELETED - my server's FQDN>/zm/index.php?view=image&eid=870&fid=snapshot&username=<DELETED - my ZM user name>&password=*
****]
06/15/20 19:09:46 zmesdetect_m1[15750] ERR zm_detect.py:221 [Error downloading files: <urlopen error [Errno -2] Name or service not known>]
06/15/20 19:09:46 zmesdetect_m1[15750] FAT zm_detect.py:222 [animation: Traceback:Traceback (most recent call last):
File "/usr/lib/python3.7/urllib/request.py", line 1317, in do_open
encode_chunked=req.has_header('Transfer-encoding'))
File "/usr/lib/python3.7/http/client.py", line 1244, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/lib/python3.7/http/client.py", line 1290, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/lib/python3.7/http/client.py", line 1239, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/lib/python3.7/http/client.py", line 1026, in _send_output
self.send(msg)
File "/usr/lib/python3.7/http/client.py", line 966, in send
self.connect()
File "/usr/lib/python3.7/http/client.py", line 1399, in connect
super().connect()
File "/usr/lib/python3.7/http/client.py", line 938, in connect
(self.host,self.port), self.timeout, self.source_address)
File "/usr/lib/python3.7/socket.py", line 707, in create_connection
for res in getaddrinfo(host, port, 0, SOCK_STREAM):
File "/usr/lib/python3.7/socket.py", line 748, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/var/lib/zmeventnotification/bin/zm_detect.py", line 219, in <module>
args)
File "/usr/local/lib/python3.7/dist-packages/zmes_hook_helpers/utils.py", line 190, in download_files
input_file = opener.open(url)
File "/usr/lib/python3.7/urllib/request.py", line 525, in open
response = self._open(req, data)
File "/usr/lib/python3.7/urllib/request.py", line 543, in _open
'_open', req)
File "/usr/lib/python3.7/urllib/request.py", line 503, in _call_chain
result = func(*args)
File "/usr/lib/python3.7/urllib/request.py", line 1360, in https_open
context=self._context, check_hostname=self._check_hostname)
File "/usr/lib/python3.7/urllib/request.py", line 1319, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [Errno -2] Name or service not known>
]
Looks like secrets.ini IS fine.
My intent is to use ZM_zones, but set to "no" for testing just to eliminate one call to the DB server.
I looked over the permissions for my ZM user (thinking that this might be a DB user permission issue) and found that I didn't have API enabled or View for Monitors. I enabled API and enabled View. No change in behavior.
Re: zmeventnotification: Error parsing config:/etc/zm/objectconfig.ini
Posted: Wed Jun 17, 2020 11:30 am
by asker
The sequence of events points to an incorrect value either in your url, username or password. zm_detect cannot download the image.
Are you able to access the URL https://<DELETED - my server's FQDN>/zm/index.php?view=image&eid=870&fid=snapshot&username=<DELETED - my ZM user name>&password=<yourpassword> in a browser? Please make sure you copy paste the log URL (replacing the placeholders) - that way if there are any errant characters, it will be reproduced.
The "name or service not known" usually occurs if:
- You don't specify the url correctly
- The scheme "http/https" is missing or malformed
- The URL cannot be resolved by DNS
** Note: This has nothing to do with ZMES_PICTURE_URL. This is related to ZM_PORTAL, ZM_USER and ZM_PASS
Re: zmeventnotification: Error parsing config:/etc/zm/objectconfig.ini
Posted: Wed Jun 17, 2020 11:50 pm
by SkippyDo
Yes, pasting the constructed URL (same as logged above but with a different/new EID [one I just triggered] and the username and password [verified correct]) and it pops up just fine.
Somewhere along the line I realized that ZMES_PICTURE_URL played no part in this.
The ZM user I'm using only has View permissions on Events and Monitors, and APIEnabled IS enabled. The web browser call works fine (when I'm also logged in the to ZM console as the same user).
It's driving me nuts as I cannot see where there's a problem anywhere. But, obviously there is. Here's more stuff that hit the ZM logs (I ran using an account with full privileges [modified secrets.ini and stopped and started zmeventserver]):
Code: Select all
animation: Traceback:Traceback (most recent call last): File "/usr/lib/python3.7/urllib/request.py", line 1317, in do_open encode_chunked=req.has_header('Transfer-encoding')) File "/usr/lib/python3.7/http/client.py", line 1244, in request self._send_request(method, url, body, headers, encode_chunked) File "/usr/lib/python3.7/http/client.py", line 1290, in _send_request self.endheaders(body, encode_chunked=encode_chunked) File "/usr/lib/python3.7/http/client.py", line 1239, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "/usr/lib/python3.7/http/client.py", line 1026, in _send_output self.send(msg) File "/usr/lib/python3.7/http/client.py", line 966, in send self.connect() File "/usr/lib/python3.7/http/client.py", line 1399, in connect super().connect() File "/usr/lib/python3.7/http/client.py", line 938, in connect (self.host,self.port), self.timeout, self.source_address) File "/usr/lib/python3.7/socket.py", line 707, in create_connection for res in getaddrinfo(host, port, 0, SOCK_STREAM): File "/usr/lib/python3.7/socket.py", line 748, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags):socket.gaierror: [Errno -2] Name or service not knownDuring handling of the above exception, another exception occurred:Traceback (most recent call last): File "/var/lib/zmeventnotification/bin/zm_detect.py", line 219, in <module> args) File "/usr/local/lib/python3.7/dist-packages/zmes_hook_helpers/utils.py", line 190, in download_files input_file = opener.open(url) File "/usr/lib/python3.7/urllib/request.py", line 525, in open response = self._open(req, data) File "/usr/lib/python3.7/urllib/request.py", line 543, in _open '_open', req) File "/usr/lib/python3.7/urllib/request.py", line 503, in _call_chain result = func(*args) File "/usr/lib/python3.7/urllib/request.py", line 1360, in https_open context=self._context, check_hostname=self._check_hostname) File "/usr/lib/python3.7/urllib/request.py", line 1319, in do_open raise URLError(err)urllib.error.URLError: <urlopen error [Errno -2] Name or service not known>
No characters in password that would trip up handling the password.
I ran with debug turned off (I'd turned it off yesterday after testing), so this is all that's available.
Re: zmeventnotification: Error parsing config:/etc/zm/objectconfig.ini
Posted: Fri Jun 19, 2020 1:31 am
by SkippyDo
I also fed the same URL construct to 'lynx' from a command line on the server and it is able to fetch the snapshot.jpg file.
URL vs authentication scheme issue?
Code: Select all
if g.config['basic_user']:
g.logger.debug('Basic auth config found, associating handlers',level=2)
password_mgr = urllib.request.HTTPPasswordMgrWithDefaultRealm()
top_level_url = g.config['portal']
password_mgr.add_password(None, top_level_url, g.config['basic_user'],
g.config['basic_password'])
handler = urllib.request.HTTPBasicAuthHandler(password_mgr)
opener = urllib.request.build_opener(handler, main_handler)
else:
opener = urllib.request.build_opener(main_handler)
In my objects.ini file:
Code: Select all
# If you need basic auth to access ZM
#basic_user=user
#basic_password=password
I'm confused as to what authentication mechanism I am supposed to be using. Within ZM I have OPT_USE_AUTH enabled, AUTH_TYPE set to builtin and AUTH_RELAY set to hashed (with string provided for AUTH_HASH_SECRET).
Re: zmeventnotification: Error parsing config:/etc/zm/objectconfig.ini
Posted: Fri Jun 19, 2020 12:40 pm
by asker
A few thoughts:
1. Stop the event server (using zmdc)
2. Run it in
manual mode
3. See if the error repeats. If it does:
3.1: Stop the ES again
3.2: Add a "wait" for, say, 5 seconds, for that monitor in objectconfig.ini (see ini file for an example of how to use wait)
3.3: See if error repeats. If it does:
3.3.1: Try and run zmdetect on a past event that has already finished (see
here)
Lets go this far and see.
On your note of auth etc, what you have done is correct. Leave basic auth commented out. Only put in ZM_USER and ZM_PASSWORD.
Re: zmeventnotification: Error parsing config:/etc/zm/objectconfig.ini
Posted: Sat Jun 20, 2020 12:29 am
by SkippyDo
Stepped through your instructions. No change, same errors. (I'd already had a 5 second wait on the monitor that I'm using to test with: I'm guessing that you thought that perhaps the files weren't written yet).
Need to trace what is happening right here in utils.py line 190:
input_file = opener.open(url)
No idea what opener is (library? from where?).
Re: zmeventnotification: Error parsing config:/etc/zm/objectconfig.ini
Posted: Sat Jun 20, 2020 3:09 pm
by asker
1. Modify
https://github.com/pliablepixels/zmeven ... ls.py#L189 to
Code: Select all
g.logger.debug('Trying to download {}'.format(url))
(We've changed durl to url). Note that this file will be in your python site libraries. You could also change it locally in your cloned repo and re-install.
2. Now repeat step 3.3.1 above
3. I assume it will fail. Please post the exact output (mask the password). you should see the real URL, with real password.
4. Now in your command line do a
Code: Select all
wget "<that same url you saw in the logs>"
(Make sure to wrap the url in quotes)
Please post output
Re: zmeventnotification: Error parsing config:/etc/zm/objectconfig.ini
Posted: Mon Jun 22, 2020 5:07 am
by SkippyDo
Here's the output (from the command line command):
Code: Select all
--2020-06-21 21:48:12-- https://<DELETED - my server's FQDN>/zm/index.php?view=image&eid=1407&fid=snapshot&username=<DELETED - username>&password=<DELETED - password>
Resolving <DELETED - my server's FQDN> (<DELETED - my server's FQDN>)... <DELETED - my server's IP>
Connecting to <DELETED - my server's FQDN> (<DELETED - my server's FQDN>)|<DELETED - my server's IP>|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [image/jpeg]
index.php?view=image&eid=1407&fid=snapshot&username=<DELETED - username>&password=<DELETED - password>: Permission denied
Cannot write to ‘index.php?view=image&eid=1407&fid=snapshot&username=<DELETED - username>&password=<DELETED - password>’ (Permission denied).
I used my administrative account to ensure that I have full privileges.
Verified same string (minus wget and quotes) works from a web browser (remote computer).
Re: zmeventnotification: Error parsing config:/etc/zm/objectconfig.ini
Posted: Mon Jun 22, 2020 5:29 am
by SkippyDo
OK...
I changed my working directory on the server (where I'm running the command) to /tmp/zm, which is writeable by www-data, the user I'm invoking as, and it created the file by the name of (incl quotes):
Code: Select all
'index.php?view=image&eid=1407&fid=snapshot&username=<DELETED - username>&password=<DELETED - password>'
I verified that the ZM events directory has the correct permissions. While I was verifying permissions I noticed that I had a file called
snapshot-48x27.jpg. It was written 48 minutes after the alarm for that event, approximately the same time I ran the wget command.
Re: zmeventnotification: Error parsing config:/etc/zm/objectconfig.ini
Posted: Mon Jun 22, 2020 1:26 pm
by asker
If wget is successfully downloading the image, but zm_detect cannot when run manually, I'm out of ideas. The only thing I can think of is there is some config setting that is not right
Re: zmeventnotification: Error parsing config:/etc/zm/objectconfig.ini
Posted: Mon Jun 22, 2020 3:53 pm
by SkippyDo
The network pieces and the construction of the URL all seem to be working. It then comes down to the downloading of the image as being the point of failure.
What directory are these images expected to end up in? Seems that it's /var/lib/zmeventnotification/images? Any directory would be written to as user www-data (user and group)?
Here are the permissions for the only places that I know of where files could get written to
Code: Select all
ls -lsa /var/lib/zmeventnotification/
total 44
4 drwxr-xr-x 11 www-data www-data 4096 Jun 9 11:29 .
4 drwxr-xr-x 59 root root 4096 Jun 9 11:29 ..
4 drwxr-xr-x 2 www-data www-data 4096 Jun 19 19:02 bin
4 drwxr-xr-x 2 www-data www-data 4096 Jun 9 11:52 contrib
4 drwxr-xr-x 2 www-data www-data 4096 Jun 9 11:29 images
4 drwxr-xr-x 2 www-data www-data 4096 Jun 9 11:29 known_faces
4 drwxr-xr-x 2 www-data www-data 4096 Jun 9 11:29 misc
4 drwxr-xr-x 2 www-data www-data 4096 Jun 9 11:29 mlapi
4 drwxr-xr-x 5 www-data www-data 4096 Jun 9 11:29 models
4 drwxr-xr-x 2 www-data www-data 4096 Jun 9 12:53 push
4 drwxr-xr-x 2 www-data www-data 4096 Jun 9 11:29 unknown_faces
ls -lsa /var/cache/zoneminder
total 44
4 drwxr-xr-x 6 www-data www-data 4096 May 12 21:53 .
4 drwxr-xr-x 18 root root 4096 May 10 22:56 ..
24 drwxr-xr-x 2 www-data www-data 20480 Jun 18 18:50 cache
4 drwxr-xr-x 6 www-data www-data 4096 May 11 21:28 events
4 drwxr-xr-x 2 www-data www-data 4096 May 2 17:33 images
4 drwxr-xr-x 2 www-data www-data 4096 May 2 17:33 temp
ls -lsa /tmp/zm
total 8
4 drwxr-xr-x 2 www-data www-data 4096 Jun 21 22:26 .
4 drwxrwxrwt 14 root root 4096 Jun 22 08:46 ..