The challenge as many have realized is one disk is just not enough to store any significant volume.
Options then include raid systems, LVM system and the like. With low end systems, they add to cost or complexity. Any non-mirrored or redundant systems suffer the risk of an entire volume failure if single disk fails. Not an option I wanted.
Using a couple of scripts, I think I've addressed it...
I've added each disk in the system mounted under /mnt as follows:
/mnt/zm1 /dev/sda1 (this is the same as my / mount - here for a couple of reasons which are used in the script below)
/mnt/zm2 /dev/sdb1
/mnt/zm3 /dev/sdc1
add as many as you wish...
Here's the first script...
Code: Select all
#!/bin/bash
#variables
eventlocation="/var/www/zm/events"
bigdisk=`df |grep zm|sort -n -r -k 4|head -1|awk '{print $6}'`
newpath=`date -d "+1 hour" +/%y/%m/%d/%H`
echo $bigdisk
echo $newpath
if [ "$bigdisk" != "/mnt/zm1" ] ; then
# find paths to work in
cd $eventlocation
for d in `find . -maxdepth 1 -mindepth 1 -type d|sort`
do
camera=`echo $d|awk '{print substr($0,2,99)}'`
totalnewpath=`echo $bigdisk$eventlocation$camera$newpath`
totallnpath=`echo $eventlocation$camera$newpath`
mkdir -p $totalnewpath
ln -s $totalnewpath $totallnpath
chown www-data:www-data $totallnpath
done
fiI've cron'd this to run at 50 minutes on the hour.
next, the cleanup script...
Code: Select all
#!/bin/bash
#variables
eventlocation="/var/www/zm/events"
disklimit=25000000
# calc if any disks need cleaning.
df |grep zm|sort -n -k 4|head -1|awk '$4 < '$disklimit' {print $6}'>/tmp/diskover.tmp
wcdiskover=`cat /tmp/diskover.tmp|wc -l`
cat /tmp/diskover.tmp
echo "wcdiskover is " $wcdiskover
while [ "$wcdiskover" -gt "0" ]
do
cd $eventlocation
for d in `find . -mindepth 1 -maxdepth 1 -type d`;do cd $d;find . -mindepth 4 -maxdepth 4 -type d |sort|head -1 ;cd ..;done|sort -u |head -1 >/tmp/pathtoclean.tmp
pathtoclean=`cat /tmp/pathtoclean.tmp|awk '{print substr($0,2,999)}'`
echo $pathtoclean
for d in `find . -maxdepth 1 -mindepth 1 -type d|sort`
do
camera=`echo $d|awk '{print substr($0,2,99)}'`
totalpathclean=`echo $eventlocation$camera$pathtoclean`
rm -rf $totalpathclean
done
df |grep zm|sort -n -k 4|head -1|awk '$4 < '$disklimit' {print $6}'>/tmp/diskover.tmp
wcdiskover=`cat /tmp/diskover.tmp|wc -l`
cat /tmp/diskover.tmp
echo "wcdiskover is " $wcdiskover
df
done
find -L . -maxdepth 5 -type l -deleteThe audit process could run at this point or on it's own. I've left mine at this point.
This will leave 25GB free on each disk.
The first program will balance filling the disks by simply creating links to the disk with the most free space. As the disks fill, they will use the space to the volume specified in the cleanup file.
Note the path used on both programs to your current installation.
The only other gotcha I've seen is different distributions do the date differently. Check your date formatting to ensure the date command in the program works for you. the other option is a "-d" instead of as shown. For the record this is on Ubuntu 10.04 LTS
As stated earlier, this is a new program running for a few days now on my system. I'll probably tweak it as it goes. That said, feel free to comment or make suggestions.
Finally, USE AT YOUR OWN RISK. I am not responsible should this mess your system up. Try it on a test system first if you want.
With that, enjoy and hopefully you find it useful.