No live video...

[nuck]

Foxtroop11

To stream video Apache for looking for zms and nph-zms in /usr/lib/cgi-bin.
Whereas ZM had them in /var/www/cgi-bin.

Tim - Have a look and see how your httpd.conf is set up. There is no good reason in the world that you should have to be installing web apps (or any parts of them) anywhere in your /usr tree.

I now have the problem that after about 10 minutes of streaming video Apache crashes.

I've increased MaxClients to 256 as recommended elsewhere - but still no success.

Apache only seems to crash when streaming.

What version of Ubuntu are you playing with again? Is it a stable release or development? Not that I've ever played with Ubuntu (or intend to do so), someone did mention in a previous thread not long back that the more recent Ubuntu has several "issues".

[nuck]

[Sat Jan 03 03:02:55 2009] [notice] caught SIGTERM, shutting down

The log files are being generated although I have no clue as to how complete they may be.

What I am seeing in /tmp/zmdc.log that is somewhat confusing is -

no connection to syslog available
- /dev/log is not a socket
- stream /dev/conslog is not writable
- console is not writable at /usr/lib/perl5/site_perl/5.8.8/ZoneMinder/Debug.pm line 322
01/03/09 03:02:59.993283 zmdc[2747].INF ['zmfilter.pl ' crashed, signal 14]

My bad. This is on shutdown when the socket has been killed.

[timf]

I'm using Ubuntu 8.10 with a fresh install of everything and this time including --enable-mmap=yes like this:

./configure --with-webdir=/var/www/html/zm --with-cgidir=/var/www/cgi-bin --with-ffmpeg=/usr/local/bin ZM_DB_HOST=localhost ZM_DB_NAME=zm ZM_DB_USER=zmuser ZM_DB_PASS=zmpass --enable-debug=yes --with-webgroup=www-data --with-webuser=www-data ZM_SSL_LIB=openssl --enable-mmap=yes

No error messages during config or make etc.

I thought I had a shared memory problem as running ipcs -m didn't show any memory segments being shared to zm or apache. I had hoped --enable-mmap=yes might fix the problem.

It didn't - ie ZM runs fine until I start to stream, then it crashes apache and sometimes the whole machine.

ipcs -m still returns this:

------ Shared Memory Segments --------
key shmid owner perms bytes nattch status
0x00000000 0 tim 600 393216 2 dest
0x00000000 32769 tim 600 393216 2 dest
0x00000000 65538 tim 600 393216 2 dest
0x00000000 98307 tim 600 393216 2 dest
0x00000000 131076 tim 600 393216 2 dest
0x00000000 163845 tim 600 393216 2 dest
0x00000000 196614 tim 600 393216 2 dest
0x00000000 229383 tim 600 393216 2 dest
0x00000000 262152 tim 600 393216 2 dest
0x00000000 294921 tim 600 393216 2 dest
0x00000000 327690 tim 600 393216 2 dest
0x00000000 360459 tim 600 393216 2 dest
0x00000000 393228 tim 600 393216 2 dest
0x00000000 655373 tim 600 393216 2 dest
0x00000000 688142 tim 600 393216 2 dest
0x00000000 491535 tim 600 393216 2 dest
0x00000000 524304 tim 600 393216 2 dest
0x00000000 720913 tim 600 393216 2 dest

I'm wondering if it's because zm still wants to get nph-zms and zms from /usr/lib/cgi-bin even though I had zm configured for /var/www/cgi-bin.

I've googled about apache2 crashing but not found anything yet.......

But have found this in the debug log:

01/04/09 18:22:17.804058 zma_m1[10024].DB1-zm_monitor.cpp/2124 [Got 1 monitors]
01/04/09 18:22:17.806711 zma_m1[10024].DB1-zm_monitor.cpp/335 [monitor purpose=2]
01/04/09 18:22:17.806741 zma_m1[10024].DB1-zm_monitor.cpp/342 [mem.size=36865192]
01/04/09 18:22:17.806794 zma_m1[10024].DB1-zm_monitor.cpp/367 [Unable to map file /dev/shm/.zm.mmap.1 (36865192 bytes) to locked memory, trying unlocked]

and this:

01/04/09 19:01:44.440818 zms[6439].DB1-zm_monitor.cpp/342 [mem.size=36865192]
01/04/09 19:01:44.440869 zms[6439].DB1-zm_monitor.cpp/367 [Unable to map file /dev/shm/.zm.mmap.1 (36865192 bytes) to locked memory, trying unlocked]

/dev/shm permissions:

tim@dualp4:~$ ls -la /dev/shm
total 36468
drwxrwxrwt 2 root root 80 2009-01-04 18:46 .
drwxr-xr-x 14 root root 14300 2009-01-04 18:44 ..
-r-------- 1 tim tim 2097176 2009-01-04 18:44 pulse-shm-3290677580
-rw------- 1 www-data www-data 36865192 2009-01-04 19:01 .zm.mmap.1

I've also edited sysctl.conf as required:

kernel.shmall = 134217728
kernel.shmmax = 134217728

and just increased shmmax to 1073741824 as the machine has 4 GM of memory installed.

[nuck]

I've also edited sysctl.conf as required:

kernel.shmall = 134217728
kernel.shmmax = 134217728

and just increased shmmax to 1073741824 as the machine has 4 GM of memory installed.

If it helps any, I have my shared memory settings at 536870912 (with 2 gig ram and 5 cameras)

Are apache's logs giving you any indication of why it is crashing? Perhaps you could try turning on debug logging with the LogLevel directive in httpd.conf if you haven't already.

[timf]

given the consistency of apache crashing first and the fact the crash can bring down the entire machine I'm thinking it's a ZMS memory issue.

Maybe apache crashing first is just a symptom ?

So close to success - but running out of ideas, at least for now.

[cordel]

Mind allowing me into the machine?

[foxtroop11]

That's strange, I've had my zoneminder install running for almost a week now with 3 usb cam's going and one remote one. I have an older desktop with a p4 and 3 gigs of ram. I haven't had it crash yet and I use apache for other things as well. I can display all streams at once with no problems either locally or from work.

Wish I could provide help, but unfortunately I'll have to sit back and listen to what your problem might be. I've documented everything I've done so far and the base I'm using is Kubuntu 8.04.

[timf]

Hi Cordel,

So I've got the shared memory issue resolved - I think, but still suffering from major crashes when ZMS starts up.

Nothing in the logs indicate its a ZMS or ZM issue - I've reverted to thinking its an apache problem.

To recap - ZM runs fine and records events which can be viewed later.

But if I stream live video for any length of time Apache either freezes and or the whole PC crashes and resets.

I suspect if I were to view an event for any length of time apache it too would crash the machine.

As root when I issue ipcs -m I can see shared memory www-data for each camera.
When not logged in as root ipcs -m shows nothing related to www-data.

I still think this is a clue:

[Sat Jan 03 02:59:25 2009] [error] [client 127.0.0.1] socket_sendto() failed: No such file or directory, referer: http://localhost/zm/index.php?view=montage&group=0
[Sat Jan 03 02:59:25 2009] [error] [client 127.0.0.1] array (\n 0 => \n array (\n 'file' => '/var/www/htdocs/zm/ajax/stream.php',\n 'line' => 47,\n 'function' => 'ajaxError',\n 'args' => \n array (\n 0 => 'socket_sendto() failed: No such file or directory',\n ),\n ),\n 1 => \n array (\n 'file' => '/var/www/htdocs/zm/index.php',\n 'line' => 116,\n 'args' => \n array (\n 0 => '/var/www/htdocs/zm/ajax/stream.php',\n ),\n 'function' => 'require_once',\n ),\n), referer: http://localhost/zm/index.php?view=montage&group=0
[Sat Jan 03 03:02:55 2009] [notice] caught SIGTERM, shutting down

and it could be related to group permissions somewhere.

Learning a lot about Linux generally and ZM

Only too pleased to give you access when I get home tonight (I'm in California) - how might I go about this ?

Cheers Tim

[nuck]

I still think this is a clue:

[Sat Jan 03 02:59:25 2009] [error] [client 127.0.0.1] socket_sendto() failed: No such file or directory, referer: http://localhost/zm/index.php?view=montage&group=0
[Sat Jan 03 02:59:25 2009] [error] [client 127.0.0.1] array (\n 0 => \n array (\n 'file' => '/var/www/htdocs/zm/ajax/stream.php',\n 'line' => 47,\n 'function' => 'ajaxError',\n 'args' => \n array (\n 0 => 'socket_sendto() failed: No such file or directory',\n ),\n ),\n 1 => \n array (\n 'file' => '/var/www/htdocs/zm/index.php',\n 'line' => 116,\n 'args' => \n array (\n 0 => '/var/www/htdocs/zm/ajax/stream.php',\n ),\n 'function' => 'require_once',\n ),\n), referer: http://localhost/zm/index.php?view=montage&group=0
[Sat Jan 03 03:02:55 2009] [notice] caught SIGTERM, shutting down

and it could be related to group permissions somewhere.

Maybe.... I posted something to this effect a few days back. In "my" case, as soon as I got cambazola straightened out that specific error (again, in my case) went away.

Learning a lot about Linux generally and ZM

Ain't Lnux fun? Trick is, don't try to fix what ain't broken unless you know for sure it is. You'll give yourself fits doing the trial and error thing.

Only too pleased to give you access when I get home tonight (I'm in California) - how might I go about this ?

Now you're talking!!

[timf]

started up ZM so that the console was showing and did nothing else. Came back 1 hour later and the PC had crashed - didn't even run ZMS or do anything.
Nothing in the logs - so just for kicks have changed the memory sticks.

No idea if that'll do any good, I doubt it will as the box runs fine when not running ZM - but I'm getting desperate.

[nuck]

started up ZM so that the console was showing and did nothing else. Came back 1 hour later and the PC had crashed - didn't even run ZMS or do anything.
Nothing in the logs - so just for kicks have changed the memory sticks.

No idea if that'll do any good, I doubt it will as the box runs fine when not running ZM - but I'm getting desperate.

If you have nay doubt about your ram have a look for a copy of memtest86. It's included with a lot of distros as a part of the install disk. You can also get it as a single bootable cd and my own fave is the ultimate boot cd.

[timf]

and no crash yet - maybe it was the memory afterall.

Dual cpu motherboard board is running both cpu's at about 50% each and memory usage is steady at 490MB.

I'm going to leave it streaming the montage over night and see what happens - then I'll rum memtest to find out what's going on

[nuck]

and no crash yet - maybe it was the memory afterall.

Dual cpu motherboard board is running both cpu's at about 50% each and memory usage is steady at 490MB.

I'm going to leave it streaming the montage over night and see what happens - then I'll rum memtest to find out what's going on

Why not just save yourself some time and run memtest... If nothing else, it'll tell you that the hardware is stable.

BTW. Since you installed your system with memory that you now suspect may have been corrupted, you should expect that the entire file system (install) is likewise suspect. Take the time to verify your memory first and if it is clean, do a new install. I speak from hard learned experience, don't trust your current file system to be reliable.

[timf]

Zm continues to stream and capture events without any issue now that I've reduced the RAM in the pc from 4G to 2G.
I'll leave it running and check performance for a few days before I declare it fixed !

I'll write up my recipe and the 'gotcha's' for the benefit of others once I'm sure it's ok.

Thanks for all the help guys !

[nuck]

Glad to hear you got it sorted out. Memory issues can be a royal pain in the CPU.