[SOLVED] /dev/shm keeps filling up when ZoneMinder is stopped and started
-
- Posts: 88
- Joined: Fri Nov 10, 2017 6:05 pm
[SOLVED] /dev/shm keeps filling up when ZoneMinder is stopped and started
Ugh, more issues with my new server. These ones are actually serious, unlike the load and disk usage display issues in my other thread. I noticed that /dev/shm kept getting more and more full every time I started and stopped ZoneMinder (changing the run state). After about 5-10 of these, it filled up completely and of course caused all sorts of problems. Doing an ls in /dev/shm showed 0 files, but then I'm not sure if it is supposed to show anything.
Here are my system details:
ZoneMinder 1.30.4 on Slackware
Linux kernel 4.4.88
Zotac ZBox BI325 with quad-core Intel Celeron N3160 processor
4 GB ram
Any idea where to look? This processor was released in Q1 2016, which is prior to the initial release of the 4.4 kernel, but maybe I should try a newer one?
Here are my system details:
ZoneMinder 1.30.4 on Slackware
Linux kernel 4.4.88
Zotac ZBox BI325 with quad-core Intel Celeron N3160 processor
4 GB ram
Any idea where to look? This processor was released in Q1 2016, which is prior to the initial release of the 4.4 kernel, but maybe I should try a newer one?
Last edited by montagdude on Fri Dec 01, 2017 11:48 pm, edited 1 time in total.
Re: /dev/shm keeps filling up when ZoneMinder is stopped and started
With 4 gig of RAM that would make the tmpfs 2 gig if Slackware is like other distros. May not be enough if you are pushing high res cameras. The files Zoneminder creates should be deleated on a restart. As a test you could relocate the settings that use tmpfs to the hard drive but you will likely take a performance hit. Rebooting is another option.
Re: /dev/shm keeps filling up when ZoneMinder is stopped and started
Another option is to switch to Ubuntu or Centos... just saying! Still trymore RAM. Your processor is not the issue unless you can only run a 32 bit OS.
-
- Posts: 88
- Joined: Fri Nov 10, 2017 6:05 pm
Re: /dev/shm keeps filling up when ZoneMinder is stopped and started
I'm certain that this isn't simply an issue of not having enough RAM. I am using a single 1280x960 camera at 3FPS, with the standard 50 frame buffer. The /dev/shm usage increases predictably by about 12% each time I restart ZoneMinder. You are correct that the total size of /dev/shm is 2 GB. I am going to try a new kernel first before switching to a different distro.
Something that I forgot to mention in the original post is that I had been testing the exact same setup on my Core i5 laptop for weeks before migrating to the new server. It has 8 GB of RAM instead of 4, but I certainly would have filled up /dev/shm within that time if it had the same problem of increasing the 240 MB every time ZoneMinder was started. Anyway, thank you for the advice.
Edit: nevertheless, I will definitely try running it on the laptop again to make sure /dev/shm is not growing and report back.
Something that I forgot to mention in the original post is that I had been testing the exact same setup on my Core i5 laptop for weeks before migrating to the new server. It has 8 GB of RAM instead of 4, but I certainly would have filled up /dev/shm within that time if it had the same problem of increasing the 240 MB every time ZoneMinder was started. Anyway, thank you for the advice.
Edit: nevertheless, I will definitely try running it on the laptop again to make sure /dev/shm is not growing and report back.
-
- Posts: 88
- Joined: Fri Nov 10, 2017 6:05 pm
Re: /dev/shm keeps filling up when ZoneMinder is stopped and started
Well, I believe I figured out my problem with /dev/shm. I have a script that sends automatic notifications when motion is detected. (I know there is a native way to do this in ZM, but it is not instantaneous. See my post in the User Contributions forum.) This keeps running even when ZoneMinder itself is stopped. Apparently, this process holds onto the memory map, and then a new one is created when ZoneMinder starts again. Output of `lsof | grep /dev/shm` after stopping ZoneMinder:
My script already checks for when ZoneMinder is stopped, so I just have to figure out how to make it release the memory map. Just for informational purposes, after rebooting and starting ZoneMinder, /dev/shm is only 235 MB.
I will mark this as solved. Sorry for the noise.
Code: Select all
zm_event_ 1367 root DEL REG 0,17 114697 /dev/shm/zm.mmap.4
I will mark this as solved. Sorry for the noise.
Re: [SOLVED] /dev/shm keeps filling up when ZoneMinder is stopped and started
question.... are you using a script similar to zm-alarm.pl to detect events and send the email?? I do the same thing but I do not have the memory map being retained issue that you mention. My scripts for ZM event detection are based on zm-alarm.pl
-
- Posts: 88
- Joined: Fri Nov 10, 2017 6:05 pm
Re: [SOLVED] /dev/shm keeps filling up when ZoneMinder is stopped and started
Yes, it's similar to that but a bit more complicated. I'm not sure what is causing it to hold onto the memory map files. Here is my script:rockedge wrote: ↑Sat Dec 02, 2017 4:06 pm question.... are you using a script similar to zm-alarm.pl to detect events and send the email?? I do the same thing but I do not have the memory map being retained issue that you mention. My scripts for ZM event detection are based on zm-alarm.pl
Code: Select all
#!/usr/bin/perl -w
# This script has been adapted from utils/zm-alarm.pl to send a notification
# within 3 seconds of a new event being generated for a monitor, which occurs
# as soon as motion is detected. The notification includes a link to the frame
# with the highest score, and filters out events caused by signal loss. For
# reference, see:
# scripts/ZoneMinder/lib/ZoneMinder/Memory.pm
# and
# http://zoneminder.readthedocs.io/en/latest/faq.html#how-can-i-use-zoneminder-to-trigger-something-else-when-there-is-an-alarm
# and
# https://forums.zoneminder.com/viewtopic.php?t=21781
use strict;
use warnings;
use ZoneMinder;
use DBI;
require MIME::Entity;
use List::Util qw[min max];
$| = 1;
# Interval between monitor checking cycles
my $timeout = 3;
# Pause after sending a message (to reduce spam)
our $message_timeout = 1;
# Min number of cycles after signal loss to send event message. Goal is to
# filter out any events caused by signal loss.
my $signal_overload_count = 5;
# Email parameters. Fill these in! Note that "@" has to be escaped like "\@"
our @toaddrs = ("");
our $fromaddr = "";
our $url = "";
my $driver = "mysql";
my $database = "zm";
my $user = "zmuser";
my $password = "zmpass";
my @monitors;
my $rebuild_monitors = 1;
while (1) {
sleep $timeout;
# Have to exit when ZoneMinder stops running to make sure mapped memory is freed
my $status = runCommand( "zmdc.pl check" );
if ( $status ne "running" ) {
Info( "zm_event_alert.pl is exiting" );
last;
}
# Initialize or re-initialize array of monitors
if ( $rebuild_monitors ) {
my $dbh = DBI->connect("DBI:$driver:$database", $user,
$password) or die $DBI::errstr;
my $sql = "select M.*, max(E.Id) as LastEventId from Monitors as M left join Events as E on M.Id = E.MonitorId where M.Function != 'None' group by (M.Id)";
my $sth = $dbh->prepare_cached( $sql ) or die( "Can't prepare '$sql': ".$dbh->errstr() );
my $res = $sth->execute() or die( "Can't execute '$sql': ".$sth->errstr() );
@monitors = ();
while ( my $monitor = $sth->fetchrow_hashref() ) {
$monitor->{LastEventId} = zmGetLastEvent( $monitor );
$monitor->{LastSignal} = $signal_overload_count + 1;
push( @monitors, $monitor );
}
}
$rebuild_monitors = 0;
# Loop over monitors
foreach my $monitor ( @monitors ) {
if ( !zmMemVerify( $monitor ) ) {
$rebuild_monitors = 1;
next;
}
# Skip any monitor that is inactive
if ( !getMonitorActive( $monitor ) ) {
next;
}
# Skip any monitor that is not receiving a signal
my $signal = getMonitorSignal( $monitor );
if ( !$signal ) {
$monitor->{LastSignal} = 1;
Info( "$monitor->{Name} is not sending a signal." );
next;
} else {
$monitor->{LastSignal} = min( $monitor->{LastSignal} + 1,
$signal_overload_count + 1 );
}
# Send a message if a new event is available. Try to filter out
# any events caused by signal loss.
my $last_event_id = zmGetLastEvent( $monitor );
if ( $last_event_id != $monitor->{LastEventId} ) {
if ( $monitor->{LastSignal} <= $signal_overload_count ) {
Info( "$monitor->{Name} signal overload count ".
"$monitor->{LastSignal}" );
} else {
notifyNewEvent( $monitor->{Name},
$last_event_id );
Info( "Sent new event $last_event_id alert ".
"for $monitor->{Name}" );
}
$monitor->{LastEventId} = $last_event_id;
}
}
}
sub getMonitorActive {
my $monitor = shift;
return( zmMemRead( $monitor, 'shared_data:active' ) );
}
sub getMonitorSignal {
my $monitor = shift;
return( zmMemRead( $monitor, 'shared_data:signal' ) );
}
sub notifyNewEvent {
my $monitor_name = shift;
my $last_event_id = shift;
my $subject = "ZoneMinder alarm";
my $maxscore_url = $url . "/index.php?view=frame&eid=".
$last_event_id . "&fid=0";
my $message = "New event $last_event_id for monitor $monitor_name.\n".
"URL: $maxscore_url";
foreach my $toaddr ( @toaddrs ) {
sendMessage($toaddr, $subject, $message);
}
}
sub sendMessage {
my $toaddr = shift;
my $subject = shift;
my $message = shift;
my $mail = MIME::Entity->build(
From => $fromaddr,
To => $toaddr,
Subject => $subject,
Type => (($message=~/<html>/)?'text/html':'text/plain'),
Data => $message
);
$mail->smtpsend( Host => "localhost",
MailFrom => $fromaddr,
);
sleep $message_timeout;
}
viewtopic.php?f=9&t=26758
For now, I've just modified the script to stop when ZoneMinder stops, and I have a second "watcher" script that starts it again when ZoneMinder starts.
Re: [SOLVED] /dev/shm keeps filling up when ZoneMinder is stopped and started
I see....nice. I made a daemon using the zm-alarm.php script which I use as a base to modify too do various functions according to the state of the monitors. This example is to send one email only for a set period of time.....
viewtopic.php?p=103057#p103057
Can I try out your version?
viewtopic.php?p=103057#p103057
Can I try out your version?
Re: [SOLVED] /dev/shm keeps filling up when ZoneMinder is stopped and started
I need to test out if the daemon in some cases does the same thing as you found out when ZM stops and the daemon continues to run...I haven't noticed anything but I did not look specifically....
-
- Posts: 88
- Joined: Fri Nov 10, 2017 6:05 pm
Re: [SOLVED] /dev/shm keeps filling up when ZoneMinder is stopped and started
Feel free to. If you want to test out the old behavior that was causing problems for me, you have to change the script so it doesn't exit when ZoneMinder is stopped. Change these lines:rockedge wrote: ↑Sat Dec 02, 2017 10:33 pm I see....nice. I made a daemon using the zm-alarm.php script which I use as a base to modify too do various functions according to the state of the monitors. This example is to send one email only for a set period of time.....
viewtopic.php?p=103057#p103057
Can I try out your version?
Code: Select all
# If zm is not running, have to stop to clear memory
my $status = runCommand( "zmdc.pl check" );
if ( $status ne "running" ) {
Info( "zm_event_alert.pl is exiting" );
last;
}
Code: Select all
# If zm is not running, just go to the next cycle
my $status = runCommand( "zmdc.pl check" );
if ( $status ne "running" ) {
if ( !$rebuild_monitors ) {
$rebuild_monitors = 1;
}
next;
}
Re: [SOLVED] /dev/shm keeps filling up when ZoneMinder is stopped and started
Have you run across this error at all using these types of queries to ZM database?? (before the API itself or the zmTrigger.pl this was the "API").
or
these errors occur running zm-alarm.pl based scripts with ZM 1.31.1 +
thanks! I am messing around with your code and it works nicely on my ZM 1.30.4 running on Puppy Linux Xenial 7.0.6 32 bit operating system (ubuntu 16.04 binaries, built with Woof-CE )
Linux Kernel: 4.9.13 (i686)
Kernel Version: #1 SMP PREEMPT Thu Mar 9 09:27:40 GMT 2017
PAE Enabled: No
PHP 7.0
mariadb
Apache 2.4
development:
Bash: 4.3.29
Gtkdialog: 0.8.4
Perl: 5.22.1
Python: 2.7.12
Yad: 0.12.4
Code: Select all
no memkey in zmMemInvalidate
Code: Select all
Can't read from mapped memory for monitor '1', gone away?
thanks! I am messing around with your code and it works nicely on my ZM 1.30.4 running on Puppy Linux Xenial 7.0.6 32 bit operating system (ubuntu 16.04 binaries, built with Woof-CE )
Linux Kernel: 4.9.13 (i686)
Kernel Version: #1 SMP PREEMPT Thu Mar 9 09:27:40 GMT 2017
PAE Enabled: No
PHP 7.0
mariadb
Apache 2.4
development:
Bash: 4.3.29
Gtkdialog: 0.8.4
Perl: 5.22.1
Python: 2.7.12
Yad: 0.12.4
-
- Posts: 88
- Joined: Fri Nov 10, 2017 6:05 pm
Re: [SOLVED] /dev/shm keeps filling up when ZoneMinder is stopped and started
Hm, no, I haven't seen errors like that. I am running 1.30.4, though. It would be worth submitting a bug report if the same script worked fine on 1.30.x. If you haven't done so already, also check whether zm-alarm.pl has been changed in 1.31.x.
Re: [SOLVED] /dev/shm keeps filling up when ZoneMinder is stopped and started
zm-alarm.php looks the same in v 1.31.x..... I will test again to see if I can track down the problem.