Terminating, last frame sent time 1430120263.528179 secs

Forum for questions and support relating to the 1.28.x releases only.
lowprofile
Posts: 19
Joined: Mon Apr 27, 2015 8:01 am

Terminating, last frame sent time 1430120263.528179 secs

Post by lowprofile »

Hi.

Can anyone shed any light on this message?

Terminating, last frame sent time 1430120263.528179 secs more than maximum of 10.000000

I have 2 cameras, an old DLink configure as remote/HTTP/Simple, and an Escam configure RTSP/LibVlc.
The DLink occasionally loses signal but recovers. Both cameras are conneceted wirelessly

The Escam randomly disappears (sometimes after an hour, sometimes after 12 or more) with the message above. I can also connect using ffmpeg, doesn't seem to make any difference. The camera is dual stream, 1280x720 and
704x576. Makes no difference which one I use, or the framerate it's set to (tried everything from 1 to 20 fps). When the signal is lost I can connect to the camera directly via its web interface and the image is fine.
Both cameras are set on modect, and until the signal vanishes, everything works fine.

The machine has plenty of horsepower, dual core AMD A6 3.9G with 16GB of memory. With that in mind I adjusted the shared memory settings:

kernel.shmall = 536870912
kernel.shmmax = 4294967296

Made no difference. Other than that it's an out of the box install v1.28.1 on Xubuntu 14.04

Any ideas? Thanks.
User avatar
asker
Posts: 1553
Joined: Sun Mar 01, 2015 12:12 pm

Re: Terminating, last frame sent time 1430120263.528179 secs

Post by asker »

Please double check if you have sufficient mapped memory.
Do a "df -h" from terminal and please post the results.

If you need to bump up mapped memory then please see this http://www.zoneminder.com/wiki/index.ph ... o_optimize

(Though with just 2 cameras and 16GB of memory, I doubt this is your problem, but always good to check. If that's not your problem then it may be something specific to VLC, which I don't use)
I no longer work on zmNinja, zmeventnotification, pyzm or mlapi. I may respond on occasion based on my available time/interest.

Please read before posting:
How to set up logging properly
How to troubleshoot and report - ES
How to troubleshoot and report - zmNinja
ES docs
zmNinja docs
lowprofile
Posts: 19
Joined: Mon Apr 27, 2015 8:01 am

Re: Terminating, last frame sent time 1430120263.528179 secs

Post by lowprofile »

Hi asker. Was it really filesystem usage you wanted?

Filesystem Size Used Avail Use% Mounted on
/dev/sda1 49G 4.5G 42G 10% /
none 4.0K 0 4.0K 0% /sys/fs/cgroup
udev 7.3G 4.0K 7.3G 1% /dev
tmpfs 1.5G 1.8M 1.5G 1% /run
none 5.0M 0 5.0M 0% /run/lock
none 7.3G 234M 7.1G 4% /run/shm
none 100M 12K 100M 1% /run/user
/dev/md0 3.6T 1.1T 2.4T 32% /home/lowprofile/Data

I did forget to mention that my events and images directories are on md0, as detailed here http://www.zoneminder.com/wiki/index.ph ... Hard_Drive

This is because the system runs on a 60GB SSD, all the data goes to the 4TB raid.
User avatar
asker
Posts: 1553
Joined: Sun Mar 01, 2015 12:12 pm

Re: Terminating, last frame sent time 1430120263.528179 secs

Post by asker »

Hi lowprofile, yes that was what I was looking for, specifically the "tmpfs" entry - that's where ZM stores its mapped memory files, which is a filesystem but actually maps to your memory. That's at 1% which means the issue is not mapped memory space. I suspect it is something to do with the VLC or camera settings, but I don't know what since I don't use VLC at all (I found it took up a lot of CPU).

While you wait for someone more knowledgeable to suggest options, I found this thread http://steamforge.net/wiki/index.php/Ho ... .04_Server --> apparently the error you saw _also_ pops up if there is a problem with the image size. Not saying that is your problem (since it works for a while), but worth a try till you get other suggestions.
lowprofile wrote:Hi asker. Was it really filesystem usage you wanted?

Filesystem Size Used Avail Use% Mounted on
/dev/sda1 49G 4.5G 42G 10% /
none 4.0K 0 4.0K 0% /sys/fs/cgroup
udev 7.3G 4.0K 7.3G 1% /dev
tmpfs 1.5G 1.8M 1.5G 1% /run
none 5.0M 0 5.0M 0% /run/lock
none 7.3G 234M 7.1G 4% /run/shm
none 100M 12K 100M 1% /run/user
/dev/md0 3.6T 1.1T 2.4T 32% /home/lowprofile/Data

I did forget to mention that my events and images directories are on md0, as detailed here http://www.zoneminder.com/wiki/index.ph ... Hard_Drive

This is because the system runs on a 60GB SSD, all the data goes to the 4TB raid.
I no longer work on zmNinja, zmeventnotification, pyzm or mlapi. I may respond on occasion based on my available time/interest.

Please read before posting:
How to set up logging properly
How to troubleshoot and report - ES
How to troubleshoot and report - zmNinja
ES docs
zmNinja docs
lowprofile
Posts: 19
Joined: Mon Apr 27, 2015 8:01 am

Re: Terminating, last frame sent time 1430120263.528179 secs

Post by lowprofile »

Hmm, when I was researching this on the net I found several such posts, where the 'last frame sent' was 10.011587 secs or a similar amount. I can believe that's a genuine timeout - however my log shows that it waited
1430120263.528179 secs (which it clearly didn't since that is 45 years or so).
lowprofile
Posts: 19
Joined: Mon Apr 27, 2015 8:01 am

Re: Terminating, last frame sent time 1430120263.528179 secs

Post by lowprofile »

Not sure if it's significant (or even just a symptom of the previous error) but the last time it occurred -

Terminating, last frame sent time 1430145421.371676 secs more than maximum of 10.000000

was followed by

socket_sendto( /var/run/zm/zms-926792s.sock ) failed: No such file or directory

As ever, it requires a Zoneminder restart to get it going again.
lowprofile
Posts: 19
Joined: Mon Apr 27, 2015 8:01 am

Re: Terminating, last frame sent time 1430120263.528179 secs

Post by lowprofile »

Well I've tried this on two machines now with three different cameras (including a d-link one that works fine as MJPEG/HTTP) and I simply cant get a reliable RTSP connection.

I can't believe I'm the only person having this problem!
codabiz
Posts: 59
Joined: Sun Jan 04, 2009 10:16 am
Location: London, UK

Re: Terminating, last frame sent time 1430120263.528179 secs

Post by codabiz »

You are not the only one. Running 14.04 LTS -zm 1.28.1 - dual core and 4Gb ram. This problem I found infuriating. Streaming simple http - remote, picture freezes when viewing monitor. From logs:

Terminating, last frame sent time 1430301155.783266 secs more than maximum of 10.000000

followed by,

socket_sendto( /var/run/zm/zms-681918s.sock ) failed: No such file or directory

also got a lot of

getStreamCmdResponse stream error: socket_sendto( /var/run/zm/zms-464665s.sock ) failed: No such file or directory - checkStreamForErrors()

'one example':. Frame send time 1758 msec too slow, throttling maxfps to 3.33

I made many changes to fps setting, and many restarts of zm, before it seemed to resolve itself. I will try to replicate it again. I have never had this problem with any previous versions of zm, so was wondering if it is a bug?
User avatar
asker
Posts: 1553
Joined: Sun Mar 01, 2015 12:12 pm

Re: Terminating, last frame sent time 1430120263.528179 secs

Post by asker »

This send_sock problem is a nightmare when it happens. I've seen it happen in various situations (with me):

1) You decided to install from source and not go the package way - and along the way read/write permissions were not set properly in the mapped memory areas

2) You are running out of shared memory space

3) There is a goof up on Apache ScriptAlias

4) You woke up in the morning and forgot to hug your zone minder server and wish it had an eventful day

5) You are trying to set an FPS in ZM that is _less_ than the in-camera FPS setting

6) Some process in ZM keeps crashing and restarting

Seriously, there are many issues where this can (eventually) happen! I've always managed to fix them, but after all sorts of frustrating tinkering.

Incidentally, point 6) above is quite often a reason. For example, there was this one time that I saw this error and noticed zmc kept crashing. The reason was it could not "sudo www-data" - I found that out by chance and fixed it (forgot how I fixed it)


I plan to read through every line of code one fine day and get a better understanding on what is really going on.
I no longer work on zmNinja, zmeventnotification, pyzm or mlapi. I may respond on occasion based on my available time/interest.

Please read before posting:
How to set up logging properly
How to troubleshoot and report - ES
How to troubleshoot and report - zmNinja
ES docs
zmNinja docs
lowprofile
Posts: 19
Joined: Mon Apr 27, 2015 8:01 am

Re: Terminating, last frame sent time 1430120263.528179 secs

Post by lowprofile »

I must admit it bugs me that I can't get this to work, but I simply don't have the time. Switched to Xeoma, it's rock solid with all my cameras. Cost 50 quid, but cheaper than replacing cameras and/or messing about for hours on end trawling through logs trying to make some sense out of them. It's a tool, after all.

Nn all.
User avatar
asker
Posts: 1553
Joined: Sun Mar 01, 2015 12:12 pm

Re: Terminating, last frame sent time 1430120263.528179 secs

Post by asker »

lowprofile wrote:I must admit it bugs me that I can't get this to work, but I simply don't have the time. Switched to Xeoma, it's rock solid with all my cameras. Cost 50 quid, but cheaper than replacing cameras and/or messing about for hours on end trawling through logs trying to make some sense out of them. It's a tool, after all.

Nn all.
I've never heard of Xeoma. Chinese company?
I had first evaluated security spy (for mac) --> pretty good, except they all charge different rates on a per-camera basis. Xeoma seems similar. Glad it worked for you.

If you are struggling with ZM, just use one of its packages - its really simple - I face pain because I compile from source and don't use packages.

I love my ZM system
a) I have an incredible mobile app https://github.com/arjunroychowdhury/zmNinja :D
b) it switches recording on an off depending on whether I am in or out of the house (tracks iPhone location): http://www.zoneminder.com/wiki/index.ph ... _recording
c) Its timeline function is superb
d) I don't pay on a per camera basis
e) infinitely customizable - I can connect it to all sorts of home appliances (including NEST)


But yes, its for geeks the way it stands today
Last edited by asker on Mon May 04, 2015 3:49 pm, edited 1 time in total.
I no longer work on zmNinja, zmeventnotification, pyzm or mlapi. I may respond on occasion based on my available time/interest.

Please read before posting:
How to set up logging properly
How to troubleshoot and report - ES
How to troubleshoot and report - zmNinja
ES docs
zmNinja docs
codabiz
Posts: 59
Joined: Sun Jan 04, 2009 10:16 am
Location: London, UK

Re: Terminating, last frame sent time 1430120263.528179 secs

Post by codabiz »

I have replicated this error!

'Terminating, last frame sent time 1430301155.783266 secs more than maximum of 10.000000'

'socket_sendto( /var/run/zm/zms-681918s.sock ) failed: No such file or directory'

I am streaming a Logitech webcam via Raspberry Pi with MotionPie software to zm 1.28.1 (Http - Simple). Under certain fps frame rate/video streaming combo I get the above 'Terminating' message.

Other general socket errors were being caused by MotionPie software streaming failures. However, ZM does not like 'Terminating, last frame sent time 1430301155.783266 secs more than maximum of 10.000000', as it also seems to hang the logging for a time. Perhaps there is a need for an inbuilt timeout limit or auto reset?
User avatar
asker
Posts: 1553
Joined: Sun Mar 01, 2015 12:12 pm

Re: Terminating, last frame sent time 1430120263.528179 secs

Post by asker »

codabiz wrote:I have replicated this error!

'Terminating, last frame sent time 1430301155.783266 secs more than maximum of 10.000000'

'socket_sendto( /var/run/zm/zms-681918s.sock ) failed: No such file or directory'

I am streaming a Logitech webcam via Raspberry Pi with MotionPie software to zm 1.28.1 (Http - Simple). Under certain fps frame rate/video streaming combo I get the above 'Terminating' message.

Other general socket errors were being caused by MotionPie software streaming failures. However, ZM does not like 'Terminating, last frame sent time 1430301155.783266 secs more than maximum of 10.000000', as it also seems to hang the logging for a time. Perhaps there is a need for an inbuilt timeout limit or auto reset?
Excellent. Can you be specific on what is the fps/streaming combo and did you fix it?
I no longer work on zmNinja, zmeventnotification, pyzm or mlapi. I may respond on occasion based on my available time/interest.

Please read before posting:
How to set up logging properly
How to troubleshoot and report - ES
How to troubleshoot and report - zmNinja
ES docs
zmNinja docs
codabiz
Posts: 59
Joined: Sun Jan 04, 2009 10:16 am
Location: London, UK

Re: Terminating, last frame sent time 1430120263.528179 secs

Post by codabiz »

'Under certain fps frame rate/video streaming combo I get the above 'Terminating' message.'

Example: MotionPie software set frame capture at 5 fps (640x480), and MotionPie video streaming at 7fps caused the 'Terminating... 1430301155.783266 secs' problem, with ZM 1.28.1 hanging for a while. ZM fps was set blank.

To recover: reset MotionPie (in gui) to 5fps/5fps and restarted MotionPie. Opened up Monitor source tab and saved ( no changes made), then refreshed ZoneMinder gui.

General socket errors also occured with other non-streaming parameters.

Generally: Raspberry Pi with MotionPie is not streaming well, terminating about every 1 minute or so (blue screen), but it auto recovers with a working set of parameters.
Last edited by codabiz on Sun May 03, 2015 2:28 pm, edited 1 time in total.
User avatar
asker
Posts: 1553
Joined: Sun Mar 01, 2015 12:12 pm

Re: Terminating, last frame sent time 1430120263.528179 secs

Post by asker »

Thanks - so to summarize, you set a lower FPS inside ZM and a higher FPS in your camera software correct (or was it the other way around)? This indeed is a bane of many problems
http://www.zoneminder.com/wiki/index.ph ... _inside_ZM
I no longer work on zmNinja, zmeventnotification, pyzm or mlapi. I may respond on occasion based on my available time/interest.

Please read before posting:
How to set up logging properly
How to troubleshoot and report - ES
How to troubleshoot and report - zmNinja
ES docs
zmNinja docs
Locked