out of memory - Ubuntu 7.10 server - ZM 1.23.2

Support and queries relating to all previous versions of ZoneMinder
helvetica
Posts: 10
Joined: Tue Jun 03, 2008 9:26 pm

out of memory - Ubuntu 7.10 server - ZM 1.23.2

Post by helvetica »

I've tried everything I could find in the forum, wiki and google but still have the same problem. I've got 2 GB ram, after a few hours it is full and my ubuntu, server, no x crash.

What can I do now?
haus
Posts: 213
Joined: Thu Oct 11, 2007 5:10 am

Post by haus »

Are you viewing a montage stream in Firefox on that box? FF eats up memory forever in that mode - I have to switch to "stills" and back again a couple times a day or memory usage goes up forever.
helvetica
Posts: 10
Joined: Tue Jun 03, 2008 9:26 pm

Post by helvetica »

No, I'm using a Ubuntu/Server without GUI (gnome) and connecting over my network. The server memory is growing up until it is full 2GB. After that my server ist very slow or I have to reboot.

Everything else is working:
- recording from 1 cam 420x288 b/w PAL
- ffmpeg
haus
Posts: 213
Joined: Thu Oct 11, 2007 5:10 am

Post by haus »

Hm. We have exactly the same setup; headless 7.10 for me. If you apt-get install htop and watch it for awhile, is there anything jumping out at you that appears to be hogging memory? I've run my box for months (on 7.04) without so much as a reboot.
helvetica
Posts: 10
Joined: Tue Jun 03, 2008 9:26 pm

Post by helvetica »

I'm watching the result of "top" (I think its the same like "htop") and can see how the memory is getting bigger and bigger .... until my whole memory 2GB.
It happens every time the cam is recording an event (function: Modect). Because I've placed the cam on the street = have 100 events per hour, after 2 or 3 houres the memory is full.
haus
Posts: 213
Joined: Thu Oct 11, 2007 5:10 am

Post by haus »

I like htop because you can sort (you probably can with top, but I don't know how) and see what is taking up memory. Plus you get a nice graph at the top. But the bottom line is, you want to see what is using more and more memory in order to figure out where to start debugging (or searching these forums for more details).

Here's a little guide to the columns:

http://syscon.seartipy.com/use-htop-to- ... processes/

I guess mem% is the important one.
helvetica
Posts: 10
Joined: Tue Jun 03, 2008 9:26 pm

Post by helvetica »

I have installed htop. Please see the results on:
http://www.you2.ch/htop.png

Because you can not see on the picture:htop.png the whole line, here is the htop field Command of the marked line:

Code: Select all

/usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-external-locking --port=3306 --socket=/var/run/mysqld/mysqld.sock
I've got many of those lines, so I think this is the problem? Can you help me to resolve my memory problem now?
haus
Posts: 213
Joined: Thu Oct 11, 2007 5:10 am

Post by haus »

I can try, but your memory isn't full yet. As of that picture, you're only using 222MB out of 2GB of RAM, and no swap. That's great so far. Can you snap another shot when you're actually out of memory? Your box looks healthy to me - load avg of 0.00, very low CPU usage.
haus
Posts: 213
Joined: Thu Oct 11, 2007 5:10 am

Post by haus »

You could try skip-innoDB in mysql:

Code: Select all

cd /etc/mysql

sudo nano my.cnf
uncomment "skip-innodb".

Restart mysql or your machine and see what happens.
helvetica
Posts: 10
Joined: Tue Jun 03, 2008 9:26 pm

Post by helvetica »

hmm... so in this way, what are the line ¦¦¦¦¦ before 222/2018MB - what is it showing?
And after starting the command: "top" I can read:

Code: Select all

Mem: 2067268k total, 2027268k used, 58696 free
When I have top or htop open, after every motion detection I see ca. +10MB more of memory used and this is going until its full. After that I have a very slow system.

.... uncomment "skip-innodb". -> no changes.

I hope you've got another idea?
haus
Posts: 213
Joined: Thu Oct 11, 2007 5:10 am

Post by haus »

I'm not a linux memory expert - all I know is, "it's complicated". Maybe top reports shared memory allocated for use? I just don't know. I'm guessing there are threads about this here, though - I've seen posts about memory usage problems before.

When I have a montage window open on a networked machine, I have just as many MySQL lines as you do (about 40 right now), though my "VIRT" is 53936 rather than 127M, which I think means that my mysql processes are only taking half the memory of yours. The important thing, I suspect, is to see what changes as your machine becomes slow. Does anything in the MEM% column go way up? You can sort on that column by pressing "M" (has to be capital M) when htop is running.
haus
Posts: 213
Joined: Thu Oct 11, 2007 5:10 am

Post by haus »

Oh, as far as I know, the bars are just a visual display. They turn yellow and then red as you approach limits.

What are your shared memory settings in /etc/sysctl.conf?
helvetica
Posts: 10
Joined: Tue Jun 03, 2008 9:26 pm

Post by helvetica »

like I've been read in many threads and guides in this forum:

Code: Select all

kernel.shmall = 134217728
kernel.shmmax = 134217728
at the moment I have one cam: 420x288, b/w, PAL
at the end I will have one more with the same attributes.

Are they the right memory settings for this configuration?
helvetica
Posts: 10
Joined: Tue Jun 03, 2008 9:26 pm

Post by helvetica »

If I see, with the command "htop" or "top" that my computer is beginning to swaping, so in this case I tought it is because out of memory, isn't it ?
haus
Posts: 213
Joined: Thu Oct 11, 2007 5:10 am

Post by haus »

According to your picture above, htop says you aren't using any swap (0/1608MB). That's why I'd like to see a pic when things start to slow down; I think it might provide more information. My old ZM box with 512MB RAM always used *some* swap. Now with 1.5GB RAM I don't use any swap at all. The machine is noticeably faster.

Your shared memory settings look fine and are certainly adequate for 1 cam! I have 10 cams going with the same settings. Though I'll admit I've never completely understood the shared memory settings - there are some conflicting posts/wiki entries that seem to indicate one of those values should be divided by 4096 or something similar, but most posts just leave them the same.

Bottom line, on a headless Ubuntu box you shouldn't be losing memory every time there is an event. Do you have any filters running executables or something like that that might have a memory leak?
Locked