Zm 1.3 is loosing one WLAN camera

[Pulsar]

I have 3 cameras (2xAxis M1025 (LAN), Axis M1004-W (WLAN)) and Zoneminder (1.30.0-4.el7.centos) running on Centos 7. System is configured and running.
After 1 or 2 days Zoneminder losses the connection to the Axis M1004-W (WLAN). Nothing is recorded. No live view is available.
If I switch on the monitor configuration of this camera and save it without any changes, the zoneminder shows live view and recording is working.
In the log file I can see the following for this monitor:
2016-12-18 10:26:12.966853 zmc_m6 54269 ERR Unable to get response, disconnecting zm_remote_camera_http.cpp 1141
2016-12-18 10:26:12.956764 zmc_m6 54269 ERR Unable to read content zm_remote_camera_http.cpp 1026
2016-12-18 10:26:12.946792 zmc_m6 54269 ERR Select error: Interrupted system call zm_remote_camera_http.cpp 175
2016-12-18 09:32:49.817220 zmdc 5362 ERR 'zmc -m 6' exited abnormally, exit status 255 zmdc.pl
2016-12-18 09:32:49.789340 zmc_m6 54219 ERR Failed to pre-capture monitor 6 -453590268 (1/1) zmc.cpp 315
2016-12-18 09:32:49.779274 zmc_m6 54219 ERR Unable to connect to camera zm_remote_camera_http.cpp 1113
2016-12-18 09:32:49.769211 zmc_m6 54219 ERR Unable to connect to the remote camera, aborting zm_remote_camera_http.cpp 120
2016-12-18 09:32:49.754373 zmc_m6 54219 WAR Can't connect to remote camera: Connection timed out zm_remote_camera_http.cpp 111
2016-12-18 09:30:41.955414 zmc_m6 53397 ERR Unable to get response, disconnecting zm_remote_camera_http.cpp 1141
2016-12-18 09:30:41.945427 zmc_m6 53397 ERR Unable to read content zm_remote_camera_http.cpp 1026
2016-12-18 09:30:41.935409 zmc_m6 53397 ERR Select error: Interrupted system call zm_remote_camera_http.cpp 175
2016-12-18 08:49:28.608380 zmdc 5362 ERR 'zmc -m 6' exited abnormally, exit status 255 zmdc.pl
2016-12-18 08:49:28.579528 zmc_m6 53365 ERR Failed to pre-capture monitor 6 1424085764 (1/1) zmc.cpp 315
2016-12-18 08:49:28.569471 zmc_m6 53365 ERR Unable to connect to camera zm_remote_camera_http.cpp 1113
2016-12-18 08:49:28.559556 zmc_m6 53365 ERR Unable to connect to the remote camera, aborting zm_remote_camera_http.cpp 120
2016-12-18 08:49:28.538404 zmc_m6 53365 WAR Can't connect to remote camera: Connection timed out zm_remote_camera_http.cpp 111
2016-12-18 08:47:20.926277 zmc_m6 25117 ERR Unable to get response, disconnecting zm_remote_camera_http.cpp 1141
2016-12-18 08:47:20.916202 zmc_m6 25117 ERR Unable to read content zm_remote_camera_http.cpp 1026
2016-12-18 08:47:20.906185 zmc_m6 25117 ERR Select error: Interrupted system call zm_remote_camera_http.cpp 175
2016-12-17 22:22:33.787715 zmc_m6 25117 WAR Buffer overrun at index 17, image 128117, slow down capture, speed up analysis or increase ring buffer size zm_monitor.cpp 3085

Any idea, what is going wrong ?


By the way, I get the following message when I am crawling in the Zoneminder log:
2016-12-22 11:13:22.137464 web_php 58067 FAT SQL-ERR 'SQLSTATE[42000]: Syntax error or access violation: 1064 You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near '' order by TimeKey asc' at line 1', statement was 'select * from Logs where TimeKey >= ? and TimeKey <= ? and Component = ?' and Level <= ? order by TimeKey asc' /usr/share/zoneminder/www/includes/database.php 115

[Baylink]

Looks to me like your capture daemon is puking and dying, and the spawned replacement isn't finding the camera.

Those are two separate problems.

I can't speak to why it's dying, but as to why it can't open the camera again... is it possible you have a static IP assigned to the camera and that address is inadvertantly inside an active DHCP range on the network... and something's taken it, causing a conflict that's keeping the camera from being accessible.

Alternatively, is the camera's grasp on the wireless at all tenuous?

[ Oh, and remember: trailing 0's are significant in version numbers: 1.28 - 1.29 - 1.30. ]

[px03afk]

I've just started getting the same type of problem. System running perfectly OK and then I can no longer see one, or maybe two cameras. In my case the two that just did are both wired and have fixed IP addresses which are not in a DHCP range. If I kill the zmc process the camera recovers.

The zm logfile shows a number of can't connect messages and then the last process which started seems to get hung up.

[Baylink]

Px: is your server dedicated to ZM? Any changes on your LAN lately?

In both cases: is your WLAN router locked on a single channel? If you have a scanner, do you know how many other networks are in your area, on that channel?

[bbunge]

How is the memory in the ZM server? Could you be running out of tmpfs? If so, lower the camera resolution and frame rate at the camera.

[Baylink]

Would zmc not log that separately, bbunge?

[px03afk]

Shouldn't the watchdog somehow spot that this process isn't running correctly and kill it?

[Baylink]

px: unless I'm mistaken, the process *itself* is noticing that and dying.

The question is:

1) Why is it dying?
2) Why won't it restart reliably?

[snake]

I'm having a similar problem. Ubuntu PPA 1.30.
I did not have this problem with 1.28.

Cameras did drop out in 1.28 but would come back on their own. The drop outs may be due to poor wiring causing the camera to restart. One camera drops out in 1.3, but does not come back. Killing the zmc process of the monitor brings it back, as reported above. As does a service zm restart.

The logs have something like the following below for a period between 14:49:25 - 14:55:16 from first error to last, but after 14:55:16, the attempts to connect stop, and the monitor is silent in the logs. No activity. The zma_m16 and zmc_m15 processes are still running. Zombies at that point.

Here's some notable log errors from that time.

Code: Select all

zmc_m15[10214]: ERR [Unrecognised content type 'text/plain']
zmc_m15[10214]: ERR [Unable to get response, disconnecting]
zmwatch[2458]: INF [Restarting capture daemon for Fs15_OutsideAboveDock, t
zmdc[2061]: INF ['zmc -m 15' sending stop to pid 10214 at 17/01/04 14:50:5
zmc_m15[10214]: INF [Got signal 15 (Terminated), exiting]
zmc_m15[10214]: ERR [Select error: Interrupted system call]
zmc_m15[10214]: ERR [Unable to read header]
zmc_m15[10214]: ERR [Unable to get response, disconnecting]
zmdc[2061]: INF ['zmc -m 15' exited normally]
zmwatch[2458]: INF [Restarting capture daemon for Fs15_OutsideAboveDock, shared data not valid]
zmdc[2061]: INF ['zmc -m 15' starting at 17/01/04 14:55:16, pid = 10404]
zmdc[10404]: INF ['zmc -m 15' started at 17/01/04 14:55:16]
zmc_m15[10404]: INF [Starting Capture version 1.30.0]

[Baylink]

Yeah: it appears to be sucessfully killing off the zmc that's not working anymore -- meaning it's noticed -- but it's not restarting one for that camera.

[onehitwonder]

Having the same issue. For now I just have a cron job restarting the server everyone night and that seems to be doing a decent job.

[alabamatoy]

I am getting this same error. I have 1.30 on a Ubuntu 16.04 that has been running for months, and recently I am experiencing this same problem: "Buffer overrun at index 46, image 845046, slow down capture, speed up analysis or increase ring buffer size" I also seem to see "Unable to parse ffmpeg option 0 '', expecting key=value" sometimes shortly before the cam goes away.

Nothing on the box has changed. Its not always the same camera. The immediate fix is to disable the monitor, save it, re-enable, save it and it will start working again, sometimes for a few days, sometimes a few hours. I really would like to find a fix.

Is there some recent Ubuntu patch that's boinking something in ZM? This box has been reliable for months, and recently this seems to be happening a lot, several times a day. I can cron something to disable and re-enable all the cameras, but that's just applying a bandaid.

[rockedge]

Here is what you are experiencing? https://github.com/ZoneMinder/ZoneMinder/pull/1717

rockedge wrote:
I am running into a problem I first noticed working with the master development version 1.30.1 where the /dev/shm creeps upwards until it hits 100% and cameras begin to shut down and the log indicates

Code: Select all

disk_free_space returned false for /dev/shm

the first camera to shutdown is prone to disconnect then reconnect which seems to be WiFi related.

knnniggett wrote:
This is happening when a camera is restarted w/o restarting zoneminder. One or more of the perl daemons are keeping open file handles on the mmap files when they shouldn't. It'll be fixed by the next release.

You can see this happening with a "sudo lsof /dev/shm"

This has been fixed in master:
https://github.com/ZoneMinder/ZoneMinder/pull/1717

[alabamatoy]

You can see this happening with a "sudo lsof /dev/shm"

How many is too many?

[alabamatoy]

This is killin me...

I have one camera monitor that goes into some ridiculous frame rate (the camera is set to 10FPS) and then I get the buffer overrun. First I will see in the logs "analyzing at 45.67 FPS" or some equally nonsensical frame rate, then shortly after that I get "Approaching buffer overrun, consider slowing capture, simplifying analysis or increasing ring buffer size" and it goes to hell from there. Im doing nothing but record, no analysis. The feed goes offline, video screen turns blue (sometimes black) and the only solution I have found is a reboot of the box.

Is this the same problem reported above in this thread?