Page 1 of 2

Convert stand alone to multi server

Posted: Thu Mar 29, 2018 7:14 pm
by Pedulla
Hi All,
zm 1.30.4 on Ubu 16.04, maria, Apache, NFS to FreeNas for event storage.

Has anyone ever converted a stand-alone ZM to multi-server without blowing away the existing events?

I get the concept listed here http://zoneminder.readthedocs.io/en/sta ... erver.html.

I'm think the following:
ZM1 = existing server and eventual db server & some existing monitors
ZM2 = additional zm server pointing to same event storage and pointing to ZM1 for maria/mysql.

Proposed Order of operations:
1-Stop all ZM servers
2-Configure all zm.conf with appropriate identification ZM_SERVER_HOST=zmX (where X=1 or 2 accordingly)
3-Configure ZM1 ZM_DB_HOST=localhost (default)
4-Configure ZM2 ZM_DB_HOST=ipaddr of ZM1
5-Start ZM1 and configure Options.Server tab with all server names from previous step
6-Configure each monitor with a server name (ZM1 or ZM2)
7-Stop ZM1
8-Do zmupdate.pl -f on each ZM server
9-Start ZM1
10-Start ZM2

and taadaa !

Let me know if you think this will work, or am I going to just have to bite the bullet and blow away 7TB of events.

Thanks

Re: Convert stand alone to multi server

Posted: Thu Mar 29, 2018 11:20 pm
by bbunge

Re: Convert stand alone to multi server

Posted: Thu Mar 29, 2018 11:58 pm
by Pedulla
frik'n brilliant

Re: Convert stand alone to multi server

Posted: Fri Mar 30, 2018 11:58 pm
by Pedulla
Any issues doing this with NGINX instead of Apache?

Re: Convert stand alone to multi server

Posted: Sat Mar 31, 2018 1:43 am
by bbunge
Likely not. Most of the issues could be with the storage and database server. Do not think web server config will matter.

Re: Convert stand alone to multi server

Posted: Mon May 07, 2018 2:49 pm
by rkaule
I just tried this. Everything went well. But as soon a I put a value in for ZM_SERVER_HOST and restart zoneminder I can no longer view the video. When logged into ZM1, I change the monitor to use a server it's ip address turns red if on ZM2 and green if it's on ZM1. If I log into ZM2 I see the reverse ( ZM1 monitors ip in red, ZM2 monitors ip in green). However if I click on the monitor name it opens up a new window, but no video. Every thing else is displayed (monitor size, events list, etc) but no video.

Any ideas?

Thanks,

Re: Convert stand alone to multi server

Posted: Mon May 07, 2018 3:09 pm
by bbunge
rkaule wrote: Mon May 07, 2018 2:49 pm I just tried this. Everything went well. But as soon a I put a value in for ZM_SERVER_HOST and restart zoneminder I can no longer view the video. When logged into ZM1, I change the monitor to use a server it's ip address turns red if on ZM2 and green if it's on ZM1. If I log into ZM2 I see the reverse ( ZM1 monitors ip in red, ZM2 monitors ip in green). However if I click on the monitor name it opens up a new window, but no video. Every thing else is displayed (monitor size, events list, etc) but no video.

Any ideas?

Thanks,
Likely a permissions error with the database or storage or both.

Re: Convert stand alone to multi server

Posted: Mon May 07, 2018 6:42 pm
by rkaule
Which of the log files would show this?

Re: Convert stand alone to multi server

Posted: Wed May 09, 2018 3:40 pm
by rkaule
I have checked all permissions on both servers, both the file system and the database. As soon as I enter a ZM_SERVER_HOST the video stops. I have been going thru the source (web, perl and cpp) to try to get a handle on what happens when you click on the monitor name on the home page. Anybody have a little insight into where I should be looking?

Thanks.

Re: Convert stand alone to multi server

Posted: Thu May 10, 2018 2:22 am
by bbunge
rkaule wrote: Wed May 09, 2018 3:40 pm I have checked all permissions on both servers, both the file system and the database. As soon as I enter a ZM_SERVER_HOST the video stops. I have been going thru the source (web, perl and cpp) to try to get a handle on what happens when you click on the monitor name on the home page. Anybody have a little insight into where I should be looking?

Thanks.
Review the instructions I wrote in the WIKI. I remember running those at least six times to be sure they worked. If you did something different let us know so we can come up with a fix. If needed I can set up two machines to test what you want to do. Might take me a few days as family matters are getting in the way of my computer work.

Re: Convert stand alone to multi server

Posted: Thu May 10, 2018 3:50 pm
by rkaule
I have reviewed the WiKI. The only differences are as follows:

I have the original server (I'll refer to as ZM1) running Centos 7.3 with 64Gb of ram dual Xeon processors (top shows 32 cpus). We had 12Tb of storage on an IBM storage array (dual 10Gb iScsi controllers). The server has 2 10Gb iScsi interfaces ( multipathed) to the storage array. We have 32 Axis cameras ( all 1080P) connect on their own network. Things were/are working fine, the load average on the server hangs around 25. We decided to bump the storage up to 32Tb and add a second server. The second server (ZM2) is about the same as ZM1 ( 64Gb, dual Xeon ) running Centos 7.4 with the same connections to the storage array and network.

The first thing I did was move the Mysql (mariadb) database to a faster bigger drive array on the first server. If anybody else wants to move their database to a non standard place, make sure you leave the mysql socket in it's original location ( pearl and python do not like it moved). Once I got that straight I restarted ZoneMinder to make sure it was still working, it was.

Then, I added the additional disk drives to the storage array. I added a second pool, attached it to the host (ZM1), on ZM1 now the events directory now has 32Tb. I restarted ZoneMinder to make sure it was still working, it was.

Then, on ZM2 I got the events driectory mounted from the storage array. There was an issue with ZM2 writing to the directory but after using xfs_repair I was able to resolve the problem. At this point I have both servers mounted to the same 32Tb of storage.

Next was getting ZM2 access to the database. I created several new users ( one for each host that a request may come in on), I did not want to use a '%' for the zmuser because of other newtorks that have access to our common firewall. I have SELinux disabled, and no firewall on both servers.

At this point I have both servers with a common events directory and access to the database. I started ZM1, added two entries to the server area in the options page. Then I click on each monitor's IP address and change the Server field to one or the other server. I had 6 cameras going to ZM1 and the rest going to ZM2. I stopped ZM1. Made the changes to both zm.conf files for the ZM_SERVER_HOST and ZM_DB_HOST settings. ZM1 is still using localhost for the database host and ZM2 is using the IP address of ZM1. With both servers stopped I run /usr/bin/zmupdate -f on each server.

Now I start ZM1, then ZM2. I log in to ZM1 and any monitor assigned to ZM2 is red and the ones assigned to ZM1 are green. When I log into ZM2 I get the same results except flipped, ZM1 monitors are red and ZM2 monitors are green.

Everything looks like its working, but when I click on the monitor's name it pops up a new window with all the controls to review and resize but no video window. The control that is normally under the video window to start and stop are at the top of the page with the events list below it.

An additional note all monitors are in Nodect.

I will try anything you want me to do.

Thanks

Re: Convert stand alone to multi server

Posted: Fri May 11, 2018 1:23 am
by bbunge
Took some time to set up two test "servers" with Ubuntu 18.04 and ran the Multi-server install from the WIKI.
https://wiki.zoneminder.com/Common_Issu ... _on_Ubuntu

And the camera I had added would not work... Took a break and walked away for a bit then read the last line of the instructions:

"When you add a camera make sure you select a server you want to add it to (Source - General - Server dropdown) All of your cameras can be managed and viewed from one server Zoneminder screen."

Of course I had deleted the camera I had set up but added the cameras I had connected to the servers (USB) and selected the correct server in the Source setup and voila...it worked!

So,all your existing cameras on ZM1 should work if you change the Source-Server for each camera.

Re: Convert stand alone to multi server

Posted: Fri May 11, 2018 11:17 am
by rkaule
I did change the Source-Server value for each camera. Are you saying that I need to delete the camera and re-add it? Maybe it is an issue with changing the Source-Server value of an existing monitor? I will set the servers to multi server, then delete the monitor and re-add it to see if that works.

Re: Convert stand alone to multi server

Posted: Fri May 11, 2018 2:40 pm
by bbunge
Found an image for Ubuntu 16.04 with ZM 1.30.4 so I loaded that on my two test machines. 16.04 had LAMP with MySQL and I was able to successfully add the server to the existing camera on my "main server-database" PC. Did have issues with SQL errors at first that I was able to fix. Existing camera on the remote server was lost as the database was not used.

Edit.
Just reran the test with Ubuntu 18.04, Mariadb, ZM 1.30.4. My main server was set up with a USB and an IP camera. After conversion to Multi-server I was able to add the Server "name" (u2 from the WIKI example) to the existing cameras and see the events recorded before conversion.
Make sure the permissions to the database server works and the file system export to the storage server works. The example I made does not use complicated or very secure connections.

As you are using Centos, which has the firewall enabled by default, you will need to open ports to allow database and file system access in the database/storage and remote servers.

Re: Convert stand alone to multi server

Posted: Mon May 14, 2018 7:33 pm
by rkaule
No firewalls on either server. The one mistake that I made was with the storage array. I attached the same lun's on both servers. It will let you do this. It will even look like they are shared. But without something like GlusterFS they are not shared. So I mounted the storage on one server and created an NFS share. Then from the other server I mounted said share and they are now able to share the space. So I though that I found the problem and proceeded to try the multi server again. I stopped the single server updated the database to point the monitors to the correct server, added the ZM_SERVER_HOST on ZM1, updated both servers with /usr/bin/zmupdate -f then started both servers. This time I opened a browser to each server to the ZM login page. I logged into both servers. Each monitor displayed which server it was connected to and the IP address for each monitor was green if connect to the current server and red if connected to the other server. If I click on a monitor on the ZM2 server it starts up the review window and the video was displayed along with the events and other controls. It didn't matter which server the monitor was connected to, it would display the video from the camera. However on ZM1 no matter which monitor i clicked on the window opens and no video.