How to create image of ZM computer?

Forum for questions and support relating to the 1.30.x releases only.
Locked
sunriseblvd
Posts: 21
Joined: Fri Jun 08, 2018 4:24 am

How to create image of ZM computer?

Post by sunriseblvd »

It wasn't easy but I got my ZM Ubuntu Server up and running. I have it running on a very old HD so with the constant HD use, it'll probably die soon. ZM has been running non-stop so HD usage is almost maxed out.

How do I create an image of the HD but excluding the ZM temp files and recordings? The image should only have the OS, ZM, LAMP, SSH, etc...

Thanks
mikb
Posts: 586
Joined: Mon Mar 25, 2013 12:34 pm

Re: How to create image of ZM computer?

Post by mikb »

This is one of the better cases for using partitions. Some people will sneer that it's "old school", and everything should just go in one big partition :)

My ZM-running system is split between 3 partitions. 1) Tiny non-encrypted "boot" partition, 2) encrypted system partition, 3) encrypted data partition.

Ignore that there is encryption and LVM further dividing 2) and 3) ...

PING (and more recently, Clonezilla) image backups take in 1) and 2) and are manageable size. If 3) is lost, tough -- it's mostly transitory data that would be pointless to backup at all, let alone image off. Some of the slightly more useful stuff is occasionally mirrored elsewhere. Never imaged.

Without splitting it this way, "imaging" the drive would be hard, you can't easily pick and choose. Images are usually all or nothing at the partition (or drive) level.

There's probably other ways to tackle this, but as I probably count as "old school" (Slackware since 1993) ...
kennbr34
Posts: 48
Joined: Sat Jan 14, 2017 6:43 pm

Re: How to create image of ZM computer?

Post by kennbr34 »

Are you trying to make an image you can put on a new disk and have it boot right up with everything as you've installed? If I were you I would just clone to a new disk now versus waiting for failure, drive failures may not be catastrophic and can cause a lot of headscrathing when they begin to act erratically. You can also use the current drive for whatever life it has remaining as strictly storage to avoid those headaches and not waste a disk that's workinf, if even for the time being.

In any case, just delete all your zoneminder events, ensuring the option to delete the files in the options is checked, and then boot up with both drives installed using a live cd. No events, no temp files, and the right environment to do what you need. Then it's just a matter of creating the image or cloning the disk, which is pretty simple even if it seems like a lot of information to take in at once.

The biggest problem you'll have is your uuid (a unique string for each disk generated by ubuntu) will have to be changed. You can clone the entire disk onto a new one (or restore an image), but grub and your /etc/fstab are expecting thay uuid string which uniquely identifies to one particular drive, so they will stay pointing to the old drive'a uuid. Without changing it, grub won't know where to find your kernel, and your kernel won't know what to mount as root. Grub is easy, you just update it and it will find the os on the new drive itself, but you'll need to edit /etc/fstab to point to the uuid of your new drive manually. If you dont, and you remove the old drive, grub will not find the os on the new drive left inserted. However if you leave the old drive in (wishing to utilize it as spare storage), and forget to change the uuids, then it will simply boot and mount root using the old drive, which of course would defeat the purpose.

So read all three of these without doing anything at all. Then clone your drive or make your image with 1. Edit your /etc/fstab with both 1. and 2. Then take the old hdd out, and update grub with 3.

https://help.ubuntu.com/community/DriveImaging
https://help.ubuntu.com/community/UsingUUID
https://howtoubuntu.org/how-to-repair-r ... tu-live-cd

Keep in mind that if you upgrade other parts of the computer, then the current installation and collection of drivers you have may not work with different configurations, so in other words this is not a universal image, it should be used only for the computer it's created on.

Plus unless you really create a backup often, it will not reflect important software updates, and may leave your system in such a deprecated state it's unable to update via aptitude or .deb packages, and that's an ugly spot to be in. I would strongly suggest you just clone to a new drive as quickly as possible, otherwise you'll need to stay vigilant in updating and backing up.
bbunge
Posts: 2930
Joined: Mon Mar 26, 2012 11:40 am
Location: Pennsylvania

Re: How to create image of ZM computer?

Post by bbunge »

Much easier to make a backup of your Zoneminder database then import it into a new system. I use Webmin to manage the server and use its built in backup. Then when you have a new system up and running with Zoneminder installed just drop the zm database and import the backup. Voila, all your cameras and users are set up! Of course you will not have the events from the old server but Zoneminder will clean up the database. Creating individual partitions is old school.
mikb
Posts: 586
Joined: Mon Mar 25, 2013 12:34 pm

Re: How to create image of ZM computer?

Post by mikb »

@bbunge I knew I could count on someone to sneer :)

Note that the OP was asking to back up the whole system, OS and all, not just the ZM database. It's no good having the full ZM database and blank hard drive. There could be a lot of configuring and installing to do before you get to restore the ZM database! I think you answered a different question.

Hence imaging off the working system, MINUS the database/events.
bbunge
Posts: 2930
Joined: Mon Mar 26, 2012 11:40 am
Location: Pennsylvania

Re: How to create image of ZM computer?

Post by bbunge »

mikb wrote: Fri Aug 03, 2018 4:33 pm @bbunge I knew I could count on someone to sneer :)

Note that the OP was asking to back up the whole system, OS and all, not just the ZM database. It's no good having the full ZM database and blank hard drive. There could be a lot of configuring and installing to do before you get to restore the ZM database! I think you answered a different question.

Hence imaging off the working system, MINUS the database/events.
Nope. Just a different approach. Takes too much time to save 800 GIG of images from a 1 TB drive. So, I waste the event images and backup the database which is much smaller. Setting up a new Zoneminder server on a fresh Linux install is quick and importing the old database gets all the camera settings, configuration and users.
sunriseblvd
Posts: 21
Joined: Fri Jun 08, 2018 4:24 am

Re: How to create image of ZM computer?

Post by sunriseblvd »

mikb wrote: Fri Aug 03, 2018 4:33 pm @bbunge I knew I could count on someone to sneer :)

Note that the OP was asking to back up the whole system, OS and all, not just the ZM database. It's no good having the full ZM database and blank hard drive. There could be a lot of configuring and installing to do before you get to restore the ZM database! I think you answered a different question.

Hence imaging off the working system, MINUS the database/events.
Yes thank you for reiterating what I originally asked. The reason I want to image the system now is because of the huge pain it was just to get it going. It literally took me 4 weeks!! The big problem is that documentation is really poor. Various FAQs bounce between elementary school level explanations all the way to PhD level explanations. I actually counted the number of FAQs that I looked at ... 23 in total to get a Ubuntu Linux server running with LAMP, a self signed certificate, Webmin, Apache virtual server config, and SSH. Zoneminder itself wasn't all that easy either as the FAQ author had to modify the instructions. Not a single set of instructions worked without some degree of complication. Btw, I've even been writing software for years and I found it to be quite complex. I come from a Windows background so there is quite a learning curve.

The best resource hands down is Digital Ocean. I just wish every linux documentation were that good.

So I really don't want to have to go through all that again :D
sunriseblvd
Posts: 21
Joined: Fri Jun 08, 2018 4:24 am

Re: How to create image of ZM computer?

Post by sunriseblvd »

kennbr34 wrote: Thu Aug 02, 2018 11:44 pm Are you trying to make an image you can put on a new disk and have it boot right up with everything as you've installed? If I were you I would just clone to a new disk now versus waiting for failure, drive failures may not be catastrophic and can cause a lot of headscrathing when they begin to act erratically. You can also use the current drive for whatever life it has remaining as strictly storage to avoid those headaches and not waste a disk that's workinf, if even for the time being.

In any case, just delete all your zoneminder events, ensuring the option to delete the files in the options is checked, and then boot up with both drives installed using a live cd. No events, no temp files, and the right environment to do what you need. Then it's just a matter of creating the image or cloning the disk, which is pretty simple even if it seems like a lot of information to take in at once.

The biggest problem you'll have is your uuid (a unique string for each disk generated by ubuntu) will have to be changed. You can clone the entire disk onto a new one (or restore an image), but grub and your /etc/fstab are expecting thay uuid string which uniquely identifies to one particular drive, so they will stay pointing to the old drive'a uuid. Without changing it, grub won't know where to find your kernel, and your kernel won't know what to mount as root. Grub is easy, you just update it and it will find the os on the new drive itself, but you'll need to edit /etc/fstab to point to the uuid of your new drive manually. If you dont, and you remove the old drive, grub will not find the os on the new drive left inserted. However if you leave the old drive in (wishing to utilize it as spare storage), and forget to change the uuids, then it will simply boot and mount root using the old drive, which of course would defeat the purpose.

So read all three of these without doing anything at all. Then clone your drive or make your image with 1. Edit your /etc/fstab with both 1. and 2. Then take the old hdd out, and update grub with 3.

https://help.ubuntu.com/community/DriveImaging
https://help.ubuntu.com/community/UsingUUID
https://howtoubuntu.org/how-to-repair-r ... tu-live-cd

Keep in mind that if you upgrade other parts of the computer, then the current installation and collection of drivers you have may not work with different configurations, so in other words this is not a universal image, it should be used only for the computer it's created on.

Plus unless you really create a backup often, it will not reflect important software updates, and may leave your system in such a deprecated state it's unable to update via aptitude or .deb packages, and that's an ugly spot to be in. I would strongly suggest you just clone to a new drive as quickly as possible, otherwise you'll need to stay vigilant in updating and backing up.
Thanks for the detailed reply. That is exactly what I am trying to do. Image it early on in the event of a hard drive failure. The image would be restored back to a different hard drive but on the same computer.

I am a little bit confused about your last paragraph. If I take a snapshot image using a live CD, will there not be a way to catch up with all updates? Doesn't the aptget updates commandline prompt take care of all of that? I come from the windows background so updating is relatively trivial with either service packs or update selected via GUI. If I have to make frequent back ups using a live CD after deleting all data, it is somewhat self-defeating to use Linux with zone minder with such a high maintenance burden. There wouldn't be some automated way for Linux server to create a disk image of itself with everything installed minus the zone mind or data files? The intent would be to restore it to the same computer but a different hard drive since it all assumes a hard drive failure.

Edit: I just read the links you provided... that appears to be what I needed!!

Maybe doing this offline procedure once a year will suffice?

Now I need to figure out how to create the disk image over to a Windows network share, which is what I use as a NAS (a Win7 machine with a ton of HDs). I have never been able to make linux operate with Windows network shares. No matter what Samba libraries i install, there is always something that has prevented linux from working with windows shares. I've actually tried to switch away from windows several times trying to use Ubuntu Linux as a desktop operating system replacement but I have always given up year after year because of the windows network share problem. Running Ubuntu server with Zoneminder has actually been my first and only success.
jwarfin
Posts: 41
Joined: Mon Jul 23, 2018 4:36 am

Re: How to create image of ZM computer?

Post by jwarfin »

I've tried and used several solutions for imaging Linux and Windows disks over the years. I always end up sticking with Clonezilla. I can do just about everything I need with it, including imaging to files on a central NAS (via samba/CIFS). Performance wise, Clonezilla is way faster than dd and more reliable (backups/images are actually verified). Plus you can image a whole disk or just specific partitions.

dd is (usually) incredibly slow by comparison - especially on modern high capacity drives. To use dd for imaging, you really need to spend some time googling for the typical parameters to use and also to understand the limitations.

It is possible to image Linux drives with products like Macrim or even Acronis - but you'll have to create their boot CD/DVD & boot from that. Capabilities are limited and performance can be slow. I can't remember which, but Macrim or Acronis has distinct limitations with ext3 or ext4, causing imaging to unexpectedly revert to a very slow block-by-block method (similar to dd).

Modern UEFI motherboards can complicate things when doing a restore because UEFI can interfere with GRUB's boot process (googling for this will give you info on how to fix this if it happens). Best way to mitigate is to configure your motherboard to be compatible with "legacy" (bios) or "hybrid" (bios+uefi) boot.

If you decide to try clonezilla, get it from the official site AND get the latest stable Unbuntu-based version(currently "artful"). They also provide a Debian version, but it's compatibility with hardware isn't as robust. Not a criticism of Debian, just my experience.

If you need to image a drive that is failing or has known errors, check out ddrescue. It's a fairly popular tool for data recovery, and it can be used for imaging too. But, like dd, performance is relatively slow. It's available in the standard Ubuntu repos.
rockedge
Posts: 1173
Joined: Fri Apr 04, 2014 1:46 pm
Location: Connecticut,USA

Re: How to create image of ZM computer?

Post by rockedge »

I use ZM with Puppy Linux built with Ubuntu binaries and I remaster the OS when it is running the way I want. Which spits out a finished bootable ISO of the operating system and a complete ZM and LAMP which I could run as a LiveCD or install again in various ways. I have also made and image of ZM 1.30.4 and the LAMP stack, made as a PET package for a one-click install on a Puppy Linux Tahr...which also was an interesting exercise...It is basically a snapshot of a ZM LAMP system as freshly installed, database included.
sunriseblvd
Posts: 21
Joined: Fri Jun 08, 2018 4:24 am

Re: How to create image of ZM computer?

Post by sunriseblvd »

These instructions might help someone in the future... this is what I'll be trying when I get the chance.

https://www.raymond.cc/blog/guide-for-m ... lonezilla/
fatboy
Posts: 47
Joined: Wed Jul 02, 2008 11:42 am
Location: Country of Texas

Re: How to create image of ZM computer?

Post by fatboy »

If you're gonna rebuild...

I use Debian Wheezy Xen 4.1 (putting together another using Stretch - both 4 core 16G RAM, hardware raid1) "Cloud Machine".

Setting up Xen is not to hard once you have your Debian box running (which doesn't take long).

You want to set up a Stretch DomU Zoneminder "virtual machine" whichever flavor Dom0 you go with. Get fancy and use LVM and you can take instant snapshots that will run on any Xen box you drop the image on with a couple small tweaks). Or simply use file based Zoneminder Virtual Machine, shutdown Zoneminder Virtual Machine, backup the disk.img and /etc/xen/zoneminder.cfg.

Keep you backup handy and if you get another box of feel the need to rebuild, Zoneminder is ready to go.

I've got a post around here somewhere on installing Zoneminder... :roll:
Locked