Storage usage is wrong

Forum for questions and support relating to the 1.31.x releases only.
Locked
DerGraf
Posts: 35
Joined: Fri Aug 17, 2018 7:33 pm

Storage usage is wrong

Post by DerGraf »

I use a own empty 100GB logical volume for event storage (/data/video/).

ZM tell me i use 5% (5.07GB) of storage but "du -sh /data/video/" give me

Code: Select all

root@server:~# du -sh /data/video/
64K     /data/video/
All events are deleted.
Whats wrong? Any ideas?
bbunge
Posts: 2930
Joined: Mon Mar 26, 2012 11:40 am
Location: Pennsylvania

Re: Storage usage is wrong

Post by bbunge »

Linux distro and version? Zoneminder version?
jwarfin
Posts: 41
Joined: Mon Jul 23, 2018 4:36 am

Re: Storage usage is wrong

Post by jwarfin »

I think this has been discussed before: https://forums.zoneminder.com/viewtopi ... on#p103584

The issue is due to the php functions used to calc disk usage. I've looked into this too for 1.30.4 because I saw 4% disk usage when the events partition was empty. What I believe is going on:

When a partition is formatted, blocks are utilized for the formatting (including journal, depending on the fs). This consumes available blocks but it's not easily visible through tools like df. Anyway, this is partly why you lose some disk capacity when formatting. I think another reason for the issue discussed here is because the php functions used to determine disk usage return floating point numbers, which are usually rounded to some degree in the scripts. In early releases of ZM, folks were using low rez analog cams with smaller drives and much smaller event partitions - so the rounded php numbers for disk usage were not apparent to most people. But for today's larger drives the difference is more noticeable.
User avatar
iconnor
Posts: 2879
Joined: Fri Oct 29, 2010 1:43 am
Location: Toronto
Contact:

Re: Storage usage is wrong

Post by iconnor »

There can be another reason. 1.31.* now store the value of a du for each event in the db. (We do this for efficiency). If something alters the contents of the event dir then these values can get out of date. Also, if you are using the FAST_DELETES option, then it only deletes the db entry and not the files on disk so the #'s will get out of sync.

So there are a couple of places to look. Make sure you have run zmaudit which will make sure the db and fs agree.
DerGraf
Posts: 35
Joined: Fri Aug 17, 2018 7:33 pm

Re: Storage usage is wrong

Post by DerGraf »

bbunge wrote: Wed Aug 29, 2018 9:55 pm Linux distro and version? Zoneminder version?
Debian 9.5, ZM 1.31.45
iconnor wrote: Thu Aug 30, 2018 12:28 pm Also, if you are using the FAST_DELETES option
I don´t use FAST_DELETE. It shows 5% usage since starting up ZM on clean system. zmaudit ist running every 15 minutes.
DerGraf
Posts: 35
Joined: Fri Aug 17, 2018 7:33 pm

Re: Storage usage is wrong

Post by DerGraf »

I´m back to 1.30.4 for testing. It shows the same 5% usage on clean install. I think 5% of 100G is a little bit to much for journaling, etc as jwarfin mean.

One thing more for the devs :wink:
User avatar
iconnor
Posts: 2879
Joined: Fri Oct 29, 2010 1:43 am
Location: Toronto
Contact:

Re: Storage usage is wrong

Post by iconnor »

I wonder if the 5% is due to being reserved for root.
jwarfin
Posts: 41
Joined: Mon Jul 23, 2018 4:36 am

Re: Storage usage is wrong

Post by jwarfin »

DerGraf wrote: Mon Sep 10, 2018 5:52 pm ... I think 5% of 100G is a little bit to much for journaling, etc as jwarfin mean.
Yeah, sorry about that. But I finally remembered what this storage issue is really about...

When you format a Unix/Linux partition, a number of reserved blocks are allocated during formatting. The default amount of reserved blocks for ext2/3/4 is 5% (you can use tune2fs to change the %). This "reserved blocks" idea has been a feature of Unix/Linux releases just about forever. In essence, it is implemented so superuser can recover a system with a completely full root filesystem. But, "full" actually means when the partition hits 95% full - the extra hidden 5% gives some extra space for superuser to run commands to recover the system. This is all well and good for a root file system, but it can be a pain for other types of partitions. Heck, a default 5% reserved block allocation for a 1tb non-root partition wastes a lot of space. This is why sysadmins may choose to use tune2fs to reduce the reserved blocks for a partition.

Needless to say, the reserved blocks can lead to confusion because common user commands - like df - don't directly take the reserved blocks into account. However, tune2fs has an option that lets you see what the reserved block/count is for a partition.

The php function(s) that get disk capacity apparently include the reserved blocks in the usage calculation somehow. Kind of unfortunate.

Google "linux partition reserved blocks" for more details.

For example: https://odzangba.wordpress.com/2010/02/ ... artitions/.
Locked