ZMC-process: constantly increasing CPU load

Forum for questions and support relating to the 1.28.x releases only.
Locked
Markymark
Posts: 50
Joined: Fri Jan 03, 2014 7:43 am

ZMC-process: constantly increasing CPU load

Post by Markymark »

Hi everyone,

this is a follow-up of my previous post:

viewtopic.php?f=32&t=23521

Because some time has passed and I did some further investigation I thought it might be useful to start a new post.

I am still struggling with constantly increasing CPU load. The following chart was generated with only one camera active in monitor-mode. You can see steps to a higher load niveau every morning. This happens even if the automatic IR filter is switched of. The camera is pointed to a quiet surrounding with only a few movements a day.
processlog (wo ir).jpg
processlog (wo ir).jpg (67.72 KiB) Viewed 6574 times
I do not have the slightest idea of what is going on here. Any help is apprechiated.

@bbunge: I still did not follow you hint regarding MySQL because of my ongoing lack of deeper understanding. From my point of view so far the above effect does not seem to be a MySQL issue. I might be terribly wrong. If so, please do not hesitate to call me ignorant and I will start >now< to dig into MySQL.

Best regards,

Mark
User avatar
asker
Posts: 1553
Joined: Sun Mar 01, 2015 12:12 pm

Re: ZMC-process: constantly increasing CPU load

Post by asker »

Some more questions to establish a baseline (my apologies if you have already answered them somewhere else)

1. How much RAM does your system have?
2. How many cameras, what are the FPS settings for each camera, what resolution?
3. What is the output of df -h ?
4. How are you accessing your camera(s) - ffmpeg, libvlc, remote etc?

thx
I no longer work on zmNinja, zmeventnotification, pyzm or mlapi. I may respond on occasion based on my available time/interest.

Please read before posting:
How to set up logging properly
How to troubleshoot and report - ES
How to troubleshoot and report - zmNinja
ES docs
zmNinja docs
Markymark
Posts: 50
Joined: Fri Jan 03, 2014 7:43 am

Re: ZMC-process: constantly increasing CPU load

Post by Markymark »

Hi asker,

I am happy to answer any question that helps to get this odd issue solved. ;-)

1. 2GB RAM (Cubietruck)

2. Four cameras at all - but the logging was done with just one camera active as 'Monitor'. The function of the other cameras were set to 'None'. Setting is 2 FPS with a resolution of 720 x 840 greyscale.

3.

Filesystem Size Used Avail Use% Mounted on
rootfs 116G 22G 89G 20% /
/dev/root 116G 22G 89G 20% /
devtmpfs 1000M 0 1000M 0% /dev
tmpfs 128M 252K 128M 1% /run
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 1.0G 50M 975M 5% /run/shm
tmpfs 1.0G 0 1.0G 0% /tmp

4. cURL (HTTP) / MJPEG

Let me know if any additional information could be useful.

Greetings,

Mark
User avatar
asker
Posts: 1553
Joined: Sun Mar 01, 2015 12:12 pm

Re: ZMC-process: constantly increasing CPU load

Post by asker »

Thanks - for the sake of a 'base state', would you mind deleting all but 1 camera and then lets debug.

_after_ you delete the 3 cameras, keep only one, and restart ZM:

1. Try changing the access mode to that camera and monitor load (switch to ffmpeg if it supports it, for example, or using simple HTTP - if you tell me which camera you have I can help)

2. is the load spike happening only when you view a feed or even when you don't have a browser window open with live feed?

3. When you say 2FPS - make sure its in-camera setting - not in ZM. Remove any FPS setting in ZM

(As an aside 2GB ram is not sufficient for 4 cameras at your resolution- but lets get there when we get there - for 1 camera this should be fine)
I no longer work on zmNinja, zmeventnotification, pyzm or mlapi. I may respond on occasion based on my available time/interest.

Please read before posting:
How to set up logging properly
How to troubleshoot and report - ES
How to troubleshoot and report - zmNinja
ES docs
zmNinja docs
Markymark
Posts: 50
Joined: Fri Jan 03, 2014 7:43 am

Re: ZMC-process: constantly increasing CPU load

Post by Markymark »

OK, there is just cam left - all other deleted and ZM restarted.

1. cURL was the only setting that worked so far. I stopped to capture single JPEG because that was very slow - I could not get FPS > 1. The cams are AVTech AVN257.

2. The chart you see was generated with no browser window open.

3. The 2FPS were set in-camera. But they were also set in the 'General' tap of ZM under 'Maximum FPS' and 'Alarm Maximum FPS'. I deleted both settings in the general tap now and left them blank. I started my logger to see if there is already any change in behaviour - but this will take a while to figure out (tomorrow afternoon should be fine).

>As an aside 2GB ram is not sufficient for 4 cameras at your resolution

Yes, that came to my mind as well.

Thanks so far.
User avatar
asker
Posts: 1553
Joined: Sun Mar 01, 2015 12:12 pm

Re: ZMC-process: constantly increasing CPU load

Post by asker »

sounds good - look forward to an update after you monitor for a while (and also keep an eye on syslog for any odd errors or recurring warnings)

Also, you can experiment with the other formats of your camera too
https://www.ispyconnect.com/man.aspx?n=AVTech
I no longer work on zmNinja, zmeventnotification, pyzm or mlapi. I may respond on occasion based on my available time/interest.

Please read before posting:
How to set up logging properly
How to troubleshoot and report - ES
How to troubleshoot and report - zmNinja
ES docs
zmNinja docs
Markymark
Posts: 50
Joined: Fri Jan 03, 2014 7:43 am

Re: ZMC-process: constantly increasing CPU load

Post by Markymark »

@asker,

I just checked the log and I could not find any noticable behaviour so far - but I would like to keep it going at least until tomorrow afternoon. To early to jump to conclusions.

Mark
Markymark
Posts: 50
Joined: Fri Jan 03, 2014 7:43 am

Re: ZMC-process: constantly increasing CPU load

Post by Markymark »

@asker,

the logging is stable so far. CPU 8%, MEM 2% - no peaks or slopes.

Two factors have been changed - deleting 3 cameras and blanking the FPS settings in ZM. I just re-entered the FPS settings in ZM to test if this leads back zu the crazy chart above.

Meanwhile I found the option 'anonymous viewer login' deep down in the frontend of the camera. I will test if this helps to get rid of the unusual cURL-method for capturing.

EDIT:

Re-entering the FPS-settings in ZM leads to the same high load behaviour. I can't believe that the solution was so simple. I read about issues with setting the FPS on 'both sides' - but strangely this does not came to my mind. Many thanks for sorting this out.

I still have problems to get rid of the cURL method using anonymous login but I will start a new post to not mix the topics.

Mark
User avatar
asker
Posts: 1553
Joined: Sun Mar 01, 2015 12:12 pm

Re: ZMC-process: constantly increasing CPU load

Post by asker »

Glad you found out the root cause. What value did you enter inside ZM and what was the value in-camera for this load spike to occur?
Markymark wrote:@asker,

Re-entering the FPS-settings in ZM leads to the same high load behaviour. I can't believe that the solution was so simple. I read about issues with setting the FPS on 'both sides' - but strangely this does not came to my mind. Many thanks for sorting this out.
Mark
I no longer work on zmNinja, zmeventnotification, pyzm or mlapi. I may respond on occasion based on my available time/interest.

Please read before posting:
How to set up logging properly
How to troubleshoot and report - ES
How to troubleshoot and report - zmNinja
ES docs
zmNinja docs
Markymark
Posts: 50
Joined: Fri Jan 03, 2014 7:43 am

Re: ZMC-process: constantly increasing CPU load

Post by Markymark »

The value in the camera and the values in ZM were both set to 2 FPS.
Locked