Memory usage

[RonRN18]

I just built a new machine and installed 32 GB of RAM... thinking that should be sufficient. I just noticed that I'm solidly using 26.0 GB of RAM, thankfully no swap, but only 4.6 GB is listed as available. While I'm glad I'm not at 100% and 0% swap, I'm curious what is using all the RAM. I'm not sure how much is due to the camera count or overhead with my ZFS RAIDz array, where I store my video files. Should I just "bite the bullet" and double my RAM to 64 GB? On the motherboard, I have four DIMM slots and populating two of them with 16 GB DDR4 DIMMs with two vacant. Is this the situation where the system is using all but a certain percentage of RAM, no matter how much RAM you have, or is it truly needing all that RAM? My overall CPU utilization is relatively low. Linux is showing 20 CPU cores although it is a 12th generation Core i7 with 8P+4E cores. I have four "CPUs" listing ~30% usage and the rest are less than 10%. My GPU, an Nvidia GTX 1050 Ti is also not breaking a sweat with about 10% usage. I basically wish to add a few more cameras without taxing the system too much. I'm looking at adding six new cameras with both main and sub feeds to my existing nine cameras with both main and sub feeds. I analyze with the sub-feeds and review recordings from the main feeds. I seldom look at sub-feed recordings and have a rolling deletion after two days of sub-feeds.

[bbunge]

I just built a new machine and installed 32 GB of RAM... thinking that should be sufficient. I just noticed that I'm solidly using 26.0 GB of RAM, thankfully no swap, but only 4.6 GB is listed as available. While I'm glad I'm not at 100% and 0% swap, I'm curious what is using all the RAM. I'm not sure how much is due to the camera count or overhead with my ZFS RAIDz array, where I store my video files. Should I just "bite the bullet" and double my RAM to 64 GB? On the motherboard, I have four DIMM slots and populating two of them with 16 GB DDR4 DIMMs with two vacant. Is this the situation where the system is using all but a certain percentage of RAM, no matter how much RAM you have, or is it truly needing all that RAM? My overall CPU utilization is relatively low. Linux is showing 20 CPU cores although it is a 12th generation Core i7 with 8P+4E cores. I have four "CPUs" listing ~30% usage and the rest are less than 10%. My GPU, an Nvidia GTX 1050 Ti is also not breaking a sweat with about 10% usage. I basically wish to add a few more cameras without taxing the system too much. I'm looking at adding six new cameras with both main and sub feeds to my existing nine cameras with both main and sub feeds. I analyze with the sub-feeds and review recordings from the main feeds. I seldom look at sub-feed recordings and have a rolling deletion after two days of sub-feeds.

Did you check the database (MySQL or Mariadb) to see if it is eating up the RAM? Running mysqltuner should help. You did not say how many cams you have but I have found using video passthrough generally uses less RAM. Also, not using the JPEG writing and using video uses less RAM.

Keep in mind that Linux will use the RAM that it needs and sometimes that may seem high to us. Your server is a beast compared to my old i3 dual core with 8 GB RAM. With my five cams it uses 28% RAM and 30% CPU. I am running Bullseye with Webmin to manage. OS and storage are on separate SSD's. Small SSD's to boot!

[RonRN18]

I currently have nine cameras, with 18 monitors set up; a main and a sub from each camera. All subs are used for monitoring activity with MoDetect and all mains are set to NoDetect but use the subs as triggers. I found that my memory was constantly creeping up day by day but when I rebooted the system, the memory dropped away to using around 17-18 GB of RAM when it had grown up to 30.4 GB of used RAM before reboot.

I did optimize my database and got it tuned up a bit. I used the MySQLTuner perl script to get help in tuning. I was trying to determine WHAT was using the majority of the memory, so I exported a sudo'd "ps aux" to determine what user and command were using how much memory. I discovered that MySQL/MariaDB only consumes about 383 MB of RAM. About 1.5-2 GB is "wasted" due to the fact I installed the desktop version instead of the server version of Ubuntu. The biggest portion is used by www-data. There are 18 instances of the process "zmc" and take up a combined 14.3 GB of RAM. As there are 18 instances and I know I have set up 18 monitors, I have deductively reasoned that each instance is for each monitor. 9 of the 18 zmc instances use significantly more memory and CPU usage than the other 9, so this differentiates the main feeds from the sub feeds. I do plan on adding up to 6 more cameras, but I think I should be OK for about 4 or so more cameras before I need to increase my RAM. I may need to setup a planned reboot at some point if RAM usage increases throughout the day. The reboot time is less than a minute, so I wouldn't miss much with a reboot set daily.

[Magic919]

Just a ZM restart will likely 'reset' the RAM use. I also don't save JPEGs and have turned off ZM logging to the DB. Worth a try.

[RonRN18]

Just a ZM restart will likely 'reset' the RAM use. I also don't save JPEGs and have turned off ZM logging to the DB. Worth a try.

I restarted ZM while looking at the resource monitor.

SystemResources.jpg (426.27 KiB) Viewed 2384 times

You should be able to see where the restart took place. It was a difference of about 5 GB of RAM freed up after the restart of ZM. I had stored the process information after a restart yesterday and took another look at processes just before that restart and see that all of the changes revolve around the processes "zmc" or as I assume the monitors.

RAM Difference.jpg (367.02 KiB) Viewed 2384 times

I have aligned the processes in order of the differences between the two times. Some of the processes grew by well over a gigabyte and a couple actually got smaller. None of the monitors save JPGs All are set for H.264 passthrough.

I know that much of this is just me learning more about the system. By learning, I can optimize.

[Magic919]

There's only so much you can do. Hopefully someone can identify it and fix in the code.

[parky]

Yesterday I built up an i3 with 4GB of Ram and it's running 8 x 1080p cameras on Mocord. It's sitting at a Load of 2-3 and 39% ram usage with an shm of 10%
Main box is running the same along with a bunch of virtuals and a has a lot more grunt, but is running the same 186mb of the 6gb so not much resource is needed
It's just a straight build using the Wiki guide on 20.04 Ubuntu Server as it doesn't need to waste resources on a GUI

[swimik]

Dropping the framerate drops the ram usage, on my machine 3 cameras at 1080p and 30fps use over 16GB ram and about 14GB at 25fps. I found out the hard way because I only have 16GB at the moment.
I think decreasing the buffer sizes will decrease ram usage. It used to be more obvious that would work because you could see it filling up /dev/shm but something changed recently and I think its all kept in memory and not shared memory anymore.

I wish there was a way to offload more to the GPU, I have a 1060 and it barely does anything even though I have the DecoderHWAccelName set to cuda and have cuda installed.

[robzio]

I'll reheat the topic. I use ZM for one camera (Hikvision IPCAM-B4 / DS-2CD1041G0-I
13152507455).
1. High RAM usage helps only reboot.

2. I attach the log.
3. I reduced knocks every FPS.
4. Changing the recording location does not help
Any suggestions ?

links to logs:
https://drive.google.com/drive/folders/ ... share_link