Multi Server Setup

Forum for questions and support relating to the 1.34.x releases only.
Post Reply
tra
Posts: 19
Joined: Tue Jan 10, 2017 10:56 am

Multi Server Setup

Post by tra »

I have changed single server setup to multi server by two different hosts (hardware).
All works well in local network, I can use all cameras, see videos etc. but from internet I can access only the cameras connected to the main server. Problem with internal server is with both live videos and recordings. I can configure them, but I can't see any video from cameras of other server.

I wonder what is the most elegant way to fix the problem, tuning apache configuration?

My setup:
Main server has name, let's say pub95, on internet is has public name, let's say, pub95.foo.bar. in LAN ip is 192.168.0.95
I have also in /etc/hosts (local machines) line:

Code: Select all

192.168.0.95   pub95.foo.bar pub95
so I can access this server by pub95.foo.bar anywhere (SSL & Lets encrypt certificate).
This server has mysgl (mariadb) of zoneminder.
Other server is not public. LAN name is (let's say) int99 and ip 192.168.0.99

Zoneminder is 1.34.5, distro Fedora 31 using apache.

--
- TRa
bbunge
Posts: 2930
Joined: Mon Mar 26, 2012 11:40 am
Location: Pennsylvania

Re: Multi Server Setup

Post by bbunge »

The servers need to see each other and share data. I wrote a procedure for Ubuntu a while back that might help you:

https://wiki.zoneminder.com/Common_Issu ... _on_Ubuntu
tra
Posts: 19
Joined: Tue Jan 10, 2017 10:56 am

Re: Multi Server Setup

Post by tra »

Good instructions, thanks, but I have done everything needed there. Servers see each other perfectly and share data in local net. Also mariadb is configured so that both zoneminders can use it. But from internet, I think I need to build reverse proxy to apache so that also my internal server int99 is visible from internet. I have never done that but I will try it when I have little more time.

--
- TRa
bbunge
Posts: 2930
Joined: Mon Mar 26, 2012 11:40 am
Location: Pennsylvania

Re: Multi Server Setup

Post by bbunge »

A simple port forward through your router shoud do it. Just set up Apache on your prime ZM server with https.
tra
Posts: 19
Joined: Tue Jan 10, 2017 10:56 am

Re: Multi Server Setup

Post by tra »

Port forward the other server to different port and open that port to internet works, thanks. Also need to change options of server to that port of course.
But then you need open extra port to the internet.
Because of that I tried alternative solution (no need touch existing firewall rules), config apache to use reserve proxy.
It almost works but web interface is odd, because all the time it kicks me off to login screen.

My httpd config:

Code: Select all

<IfModule mod_proxy.c>
SSLProxyEngine on
SSLProxyVerify none
SSLProxyCheckPeerCN off
SSLProxyCheckPeerExpire off
</IfModule>

ProxyPass "/zm99""https://192.168.0.99/zm"
ProxyPassReverse "/zm99""https://192.168.0.99/zm"
ProxyPass        "/cgi-bin-zm99" "https://192.168.0.99/cgi-bin-zm"
ProxyPassReverse "/cgi-bin-zm99" "https://192.168.0.99/cgi-bin-zm"
From zoneminder also changed Options -> Servers the other server setup paths like there (zm --> zm99)

It almost works well. I can live stream cameras of both servers, also recordings.
But always after watching live stream or recoding of the other server (ip 192.168.0.99) I will be log out.
This does not happen with main server (where mysql database is). I can also watch all cameras from
Montage Reviev and then I will be logged in still after that. But after watching same live video from one camera
of other server.. I will be kicked out after that.

I wonder what can cause that? Am I missing something in my config? I don't see anything from logs according to this, no errors from web_php.
Is this bug? Can I debug this somehow more?
--
- TRa
tra
Posts: 19
Joined: Tue Jan 10, 2017 10:56 am

Re: Multi Server Setup

Post by tra »

Oh damn... not this problem starts to be worse. Even I do not have now reverse proxy setup in apache, I use port forwarding / firewall. Earlier it seems to work. But now I will be kicked out to login screen randomly and very often. Usually middle of watching video or almost doing anything could cause the kick out.

From log I found errors like:
Bogus number of lines return from query, 0 returned for query SELECT Id FROM States WHERE IsActive=1.
But time of these errors are not at same time when I'm kicked out to login screen, about 30s earlier, at least. Could the messages still be related to this?

Other error messages from log:
'zma -m 8' exited abnormally, exit status 255
Got empty memory map file size 0, is the zmc process for this monitor running?


I have never anything problems like this in single serever setup. How this multi server setup could be so harder?

I really appreciative any help, this is very annoying, these happens now very often. I can not watch some recordings (of secondary server) more than one minutes, etc. Everything seems to work well with main server. Could this be something wrong with my setup / database?

--
- TRa
tra
Posts: 19
Joined: Tue Jan 10, 2017 10:56 am

Re: Multi Server Setup

Post by tra »

I had some time to debug this, but I couldn't fix the problem. So I need help, please.

Sorry about long text, and long debug messages,I can not explain this shortly. Sorry about my bad English too :oops:
Distro is Fedora 31, Zoneminder version v1.34.7 with httpd (apache).

I have two servers, I call them later server1 = server which has mariadb and the name is visible as public xxx.dy.fi.
server1 has most of the cameras and storage area (visible also to other server).
Other server I call server2 internally named xxx99 and it is not visible outside, only via reserve proxy setup by server1.
server2 has few cameras and storage area, which is visible by nfs also to the server1.
Not public IPs are 192.168.0.95 = server1 and 192.168.0.99 = server2.

I put debugging log on for web_php interface and checked what happens there. I tried also add some debug code for auth.php but I'm not so familiar with PHP, or authenticatrion, so it does not helped much. I just know that somehow it loses username and authention information from $_SESSION . Then the session does not have user anymore and zoneminder kicks user to the login screen. I do not know why. After login again I can continue doing what I was doing. This happens only with live stream or events (video recordings) of server2. Anything related to server1 has no problems.

Here is example from log what happens when I try to visit one event from server2.
At 20:53 I select events of server2. At 20:54 I select one event I want to watch, it kicks me to login screen. 20:55 I login again, kick out, I login again, I can select one event and watch a video. At 20:56 during watching the video I click console and again, kicking out. I login in and everything is normal again.
04/10/20 20:53:03.502402 web_php[1002357].DBG [80.186.161.122] [Setting cookie parameters to lifetime(3600) path(/) domain () secure() httpOnly(1) name:ZMSESSID] at /usr/share/zoneminder/www/includes/session.php line 20
04/10/20 20:53:03.503625 web_php[1002357].DBG [80.186.161.122] [View: events Request: Action: User: testuser] at /usr/share/zoneminder/www/index.php line 210
04/10/20 20:54:04.856223 web_php[1064493].DBG [80.186.161.122] [Setting cookie parameters to lifetime(3600) path(/) domain () secure() httpOnly(1) name:ZMSESSID] at /usr/share/zoneminder/www/includes/session.php line 20
04/10/20 20:54:04.857042 web_php[1064493].DBG [80.186.161.122] [No username in session] at /usr/share/zoneminder/www/includes/auth.php line 258
04/10/20 20:54:04.857287 web_php[1064493].DBG [80.186.161.122] [View: events Request: Action: User: none] at /usr/share/zoneminder/www/index.php line 210
04/10/20 20:54:04.857392 web_php[1064493].DBG [80.186.161.122] [Redirecting to login] at /usr/share/zoneminder/www/index.php line 243
04/10/20 20:54:04.857486 web_php[1064493].DBG [80.186.161.122] [Setting cookie parameters to lifetime(3600) path(/) domain () secure() httpOnly(1) name:ZMSESSID] at /usr/share/zoneminder/www/includes/session.php line 20
04/10/20 20:54:04.857640 web_php[1064493].DBG [80.186.161.122] [Redirecting to /zm/index.php?view=login] at /usr/share/zoneminder/www/index.php line 261
04/10/20 20:54:05.091135 web_php[1002358].DBG [80.186.161.122] [Setting cookie parameters to lifetime(3600) path(/) domain () secure() httpOnly(1) name:ZMSESSID] at /usr/share/zoneminder/www/includes/session.php line 20
04/10/20 20:54:05.091604 web_php[1002358].DBG [80.186.161.122] [No username in session] at /usr/share/zoneminder/www/includes/auth.php line 258
04/10/20 20:54:05.091783 web_php[1002358].DBG [80.186.161.122] [View: login Request: Action: User: none] at /usr/share/zoneminder/www/index.php line 210
04/10/20 20:55:05.779259 web_php[1024046].DBG [80.186.161.122] [Setting cookie parameters to lifetime(3600) path(/) domain () secure() httpOnly(1) name:ZMSESSID] at /usr/share/zoneminder/www/includes/session.php line 20
04/10/20 20:55:05.780065 web_php[1024046].DBG [80.186.161.122] [Setting Access-Control-Allow-Origin from https://xxx.dy.fi] at /usr/share/zoneminder/www/includes/functions.php line 113
04/10/20 20:55:05.780773 web_php[1024046].DBG [80.186.161.122] [bcrypt signature found, assumed bcrypt password] at /usr/share/zoneminder/www/includes/auth.php line 80
04/10/20 20:55:05.867826 web_php[1024046].DBG [80.186.161.122] [View: login Request: Action: login User: testuser] at /usr/share/zoneminder/www/index.php line 210
04/10/20 20:55:05.867949 web_php[1024046].DBG [80.186.161.122] [Including includes/actions/login.php] at /usr/share/zoneminder/www/index.php line 228
04/10/20 20:55:05.868030 web_php[1024046].DBG [80.186.161.122] [Setting cookie parameters to lifetime(3600) path(/) domain () secure() httpOnly(1) name:ZMSESSID] at /usr/share/zoneminder/www/includes/session.php line 20
04/10/20 20:55:05.868160 web_php[1024046].INF [80.186.161.122] [Login successful for user "testuser"] at /usr/share/zoneminder/www/includes/actions/login.php line 87
04/10/20 20:55:05.870049 web_php[1024046].DBG [80.186.161.122] [Setting cookie parameters to lifetime(3600) path(/) domain () secure() httpOnly(1) name:ZMSESSID] at /usr/share/zoneminder/www/includes/session.php line 20
04/10/20 20:55:06.876686 web_php[1002359].DBG [80.186.161.122] [Setting cookie parameters to lifetime(3600) path(/) domain () secure() httpOnly(1) name:ZMSESSID] at /usr/share/zoneminder/www/includes/session.php line 20
04/10/20 20:55:06.877908 web_php[1002359].DBG [80.186.161.122] [View: events Request: Action: User: testuser] at /usr/share/zoneminder/www/index.php line 210
04/10/20 20:55:18.265575 web_php[1080061].DBG [80.186.161.122] [Setting cookie parameters to lifetime(3600) path(/) domain () secure() httpOnly(1) name:ZMSESSID] at /usr/share/zoneminder/www/includes/session.php line 20
04/10/20 20:55:18.266183 web_php[1080061].DBG [80.186.161.122] [No username in session] at /usr/share/zoneminder/www/includes/auth.php line 258
04/10/20 20:55:18.266425 web_php[1080061].DBG [80.186.161.122] [View: event Request: Action: User: none] at /usr/share/zoneminder/www/index.php line 210
04/10/20 20:55:18.266478 web_php[1080061].DBG [80.186.161.122] [Redirecting to login] at /usr/share/zoneminder/www/index.php line 243
04/10/20 20:55:18.266524 web_php[1080061].DBG [80.186.161.122] [Setting cookie parameters to lifetime(3600) path(/) domain () secure() httpOnly(1) name:ZMSESSID] at /usr/share/zoneminder/www/includes/session.php line 20
04/10/20 20:55:18.266620 web_php[1080061].DBG [80.186.161.122] [Redirecting to /zm/index.php?view=login] at /usr/share/zoneminder/www/index.php line 261
04/10/20 20:55:18.334341 web_php[1002355].DBG [80.186.161.122] [Setting cookie parameters to lifetime(3600) path(/) domain () secure() httpOnly(1) name:ZMSESSID] at /usr/share/zoneminder/www/includes/session.php line 20
04/10/20 20:55:18.334789 web_php[1002355].DBG [80.186.161.122] [No username in session] at /usr/share/zoneminder/www/includes/auth.php line 258
04/10/20 20:55:18.334943 web_php[1002355].DBG [80.186.161.122] [View: login Request: Action: User: none] at /usr/share/zoneminder/www/index.php line 210
04/10/20 20:55:23.445040 web_php[1002356].DBG [80.186.161.122] [Setting cookie parameters to lifetime(3600) path(/) domain () secure() httpOnly(1) name:ZMSESSID] at /usr/share/zoneminder/www/includes/session.php line 20
04/10/20 20:55:23.445620 web_php[1002356].DBG [80.186.161.122] [Setting Access-Control-Allow-Origin from https://xxx.dy.fi] at /usr/share/zoneminder/www/includes/functions.php line 113
04/10/20 20:55:23.446090 web_php[1002356].DBG [80.186.161.122] [bcrypt signature found, assumed bcrypt password] at /usr/share/zoneminder/www/includes/auth.php line 80
04/10/20 20:55:23.529840 web_php[1002356].DBG [80.186.161.122] [View: login Request: Action: login User: testuser] at /usr/share/zoneminder/www/index.php line 210
04/10/20 20:55:23.529936 web_php[1002356].DBG [80.186.161.122] [Including includes/actions/login.php] at /usr/share/zoneminder/www/index.php line 228
04/10/20 20:55:23.529999 web_php[1002356].DBG [80.186.161.122] [Setting cookie parameters to lifetime(3600) path(/) domain () secure() httpOnly(1) name:ZMSESSID] at /usr/share/zoneminder/www/includes/session.php line 20
04/10/20 20:55:23.530089 web_php[1002356].INF [80.186.161.122] [Login successful for user "testuser"] at /usr/share/zoneminder/www/includes/actions/login.php line 87
04/10/20 20:55:23.532085 web_php[1002356].DBG [80.186.161.122] [Setting cookie parameters to lifetime(3600) path(/) domain () secure() httpOnly(1) name:ZMSESSID] at /usr/share/zoneminder/www/includes/session.php line 20
04/10/20 20:55:24.515595 web_php[1074386].DBG [80.186.161.122] [Setting cookie parameters to lifetime(3600) path(/) domain () secure() httpOnly(1) name:ZMSESSID] at /usr/share/zoneminder/www/includes/session.php line 20
04/10/20 20:55:24.516896 web_php[1074386].DBG [80.186.161.122] [View: event Request: Action: User: testuser] at /usr/share/zoneminder/www/index.php line 210
04/10/20 20:55:25.992673 web_php[1074734].DBG [80.186.161.122] [Setting cookie parameters to lifetime(3600) path(/) domain () secure() httpOnly(1) name:ZMSESSID] at /usr/share/zoneminder/www/includes/session.php line 20
04/10/20 20:55:25.993507 web_php[1074734].DBG [80.186.161.122] [Setting Access-Control-Allow-Origin from https://xxx.dy.fi] at /usr/share/zoneminder/www/includes/functions.php line 113
04/10/20 20:55:25.994280 web_php[1074734].DBG [80.186.161.122] [View: request Request: log Action: User: testuser] at /usr/share/zoneminder/www/index.php line 210
04/10/20 20:55:26.117769 web_php[1002379].DBG [80.186.161.122] [Setting cookie parameters to lifetime(3600) path(/) domain () secure() httpOnly(1) name:ZMSESSID] at /usr/share/zoneminder/www/includes/session.php line 20
04/10/20 20:55:26.118943 web_php[1002379].DBG [80.186.161.122] [View: request Request: status Action: User: testuser] at /usr/share/zoneminder/www/index.php line 210
04/10/20 20:55:26.131821 web_php[1002357].DBG [80.186.161.122] [Setting cookie parameters to lifetime(3600) path(/) domain () secure() httpOnly(1) name:ZMSESSID] at /usr/share/zoneminder/www/includes/session.php line 20
04/10/20 20:55:26.132631 web_php[1002357].DBG [80.186.161.122] [View: request Request: status Action: User: testuser] at /usr/share/zoneminder/www/index.php line 210
04/10/20 20:56:21.133724 web_php[1064493].DBG [80.186.161.122] [Setting cookie parameters to lifetime(3600) path(/) domain () secure() httpOnly(1) name:ZMSESSID] at /usr/share/zoneminder/www/includes/session.php line 20
04/10/20 20:56:21.134305 web_php[1064493].DBG [80.186.161.122] [No username in session] at /usr/share/zoneminder/www/includes/auth.php line 258
04/10/20 20:56:21.138828 web_php[1064493].DBG [80.186.161.122] [View: console Request: Action: User: none] at /usr/share/zoneminder/www/index.php line 210
04/10/20 20:56:21.138935 web_php[1064493].DBG [80.186.161.122] [Redirecting to login] at /usr/share/zoneminder/www/index.php line 243
04/10/20 20:56:21.139008 web_php[1064493].DBG [80.186.161.122] [Setting cookie parameters to lifetime(3600) path(/) domain () secure() httpOnly(1) name:ZMSESSID] at /usr/share/zoneminder/www/includes/session.php line 20
04/10/20 20:56:21.139130 web_php[1064493].DBG [80.186.161.122] [Redirecting to /zm/index.php?view=login] at /usr/share/zoneminder/www/index.php line 261
04/10/20 20:56:21.226478 web_php[1002358].DBG [80.186.161.122] [Setting cookie parameters to lifetime(3600) path(/) domain () secure() httpOnly(1) name:ZMSESSID] at /usr/share/zoneminder/www/includes/session.php line 20
04/10/20 20:56:21.226881 web_php[1002358].DBG [80.186.161.122] [No username in session] at /usr/share/zoneminder/www/includes/auth.php line 258
04/10/20 20:56:21.227066 web_php[1002358].DBG [80.186.161.122] [View: login Request: Action: User: none] at /usr/share/zoneminder/www/index.php line 210
04/10/20 20:56:39.425930 web_php[1024046].DBG [80.186.161.122] [Setting cookie parameters to lifetime(3600) path(/) domain () secure() httpOnly(1) name:ZMSESSID] at /usr/share/zoneminder/www/includes/session.php line 20
04/10/20 20:56:39.426826 web_php[1024046].DBG [80.186.161.122] [Setting Access-Control-Allow-Origin from https://xxx.dy.fi] at /usr/share/zoneminder/www/includes/functions.php line 113
04/10/20 20:56:39.427580 web_php[1024046].DBG [80.186.161.122] [bcrypt signature found, assumed bcrypt password] at /usr/share/zoneminder/www/includes/auth.php line 80
04/10/20 20:56:39.514693 web_php[1024046].DBG [80.186.161.122] [View: login Request: Action: login User: testuser] at /usr/share/zoneminder/www/index.php line 210
04/10/20 20:56:39.514816 web_php[1024046].DBG [80.186.161.122] [Including includes/actions/login.php] at /usr/share/zoneminder/www/index.php line 228
04/10/20 20:56:39.514889 web_php[1024046].DBG [80.186.161.122] [Setting cookie parameters to lifetime(3600) path(/) domain () secure() httpOnly(1) name:ZMSESSID] at /usr/share/zoneminder/www/includes/session.php line 20
04/10/20 20:56:39.515005 web_php[1024046].INF [80.186.161.122] [Login successful for user "testuser"] at /usr/share/zoneminder/www/includes/actions/login.php line 87
04/10/20 20:56:39.516721 web_php[1024046].DBG [80.186.161.122] [Setting cookie parameters to lifetime(3600) path(/) domain () secure() httpOnly(1) name:ZMSESSID] at /usr/share/zoneminder/www/includes/session.php line 20
04/10/20 20:56:40.564937 web_php[1002359].DBG [80.186.161.122] [Setting cookie parameters to lifetime(3600) path(/) domain () secure() httpOnly(1) name:ZMSESSID] at /usr/share/zoneminder/www/includes/session.php line 20
04/10/20 20:56:40.569591 web_php[1002359].DBG [80.186.161.122] [View: console Request: Action: User: testuser] at /usr/share/zoneminder/www/index.php line 210
04/10/20 20:57:41.561682 web_php[1080061].DBG [80.186.161.122] [Setting cookie parameters to lifetime(3600) path(/) domain () secure() httpOnly(1) name:ZMSESSID] at /usr/share/zoneminder/www/includes/session.php line 20
04/10/20 20:57:41.565937 web_php[1080061].DBG [80.186.161.122] [View: console Request: Action: User: testuser] at /usr/share/zoneminder/www/index.php line 210
After this I can select events of server1, live stream of server1, anything related with server1. No problem, no kicking out.

I think it could be something to do with hashing when the serves has different IP:s, but visible name outside is same (only paths to index/ZMS are different) I also noticed if I change from options AUTH_HASH_IPS on, I can not see the events from server2 at all. Even after login again. It says video is corrupted. Something is really wrong then. Same happens also if I change AUTH_RELAY to plain.
Only working combination is AUTH_RELAY is hashed, AUTH_HASH_IPS is off. Then I can see videos and live stream from server2, but kicking out continously.

I can give any other information, if needed, I do not have ideas anymore. Continously kicking out from web-interface drives me grazy :(

I noticed also other problem when I restarted zoneminder often. It crashed ofter after about 30s by core-dump. Also after start it again, it need to start about 3-6 times before it start to be up. After running more than 40s it never crash by core-dump anymore. This never happened for me in single server setup, so I'm confused with this, why this happens with multiserver setup so often. However I can live with it because it happens only when starting zoneminder, not after that.
I did not noticed any errors or reason why core-dump.
systemd says:
% systemctl status zoneminder
● zoneminder.service - ZoneMinder CCTV recording and security system
Loaded: loaded (/usr/lib/systemd/system/zoneminder.service; enabled; vendor preset: disabled)
Drop-In: /usr/lib/systemd/system/zoneminder.service.d
└─zm-httpd.conf
Active: failed (Result: core-dump) since Fri 2020-04-10 21:30:12 EEST; 4min 22s ago
Process: 683812 ExecStart=/usr/bin/zmpkg.pl start (code=exited, status=0/SUCCESS)
Main PID: 683823 (code=dumped, signal=SEGV)
CPU: 1min 18.590s
--
- TRa
User avatar
iconnor
Posts: 2880
Joined: Fri Oct 29, 2010 1:43 am
Location: Toronto
Contact:

Re: Multi Server Setup

Post by iconnor »

zmwatch is probably killing it. Check your Options -> System -> WATCH_MAX_DELAY. I keep mine around 45.
User avatar
iconnor
Posts: 2880
Joined: Fri Oct 29, 2010 1:43 am
Location: Toronto
Contact:

Re: Multi Server Setup

Post by iconnor »

As to the login problems, these are usually due to timezone problems.
tra
Posts: 19
Joined: Tue Jan 10, 2017 10:56 am

Re: Multi Server Setup

Post by tra »

WATCH_MAX_DELAY was very short as default, only 5. I changed it to 45 but did not help. Still core-dump after restart. Now it needed 3 times to start before ZM is now again up and running on server2. I did simple restart daemon to handle this, so I can not accidentally forget ZM to be down for a long time. It just check periodically if zoneminder is not running and try to start if needed.

Code: Select all

#!/bin/sh 
# How often (in seconds) check if zoneminder needs restarting 
period=55 
while true ; do 
    sleep $period 
    systemctl status zoneminder  > /dev/null 
    if [ $? -eq 3 ] ; then # need start 
        date  >> /var/log/zoneminder/restart-daemon.log 2>&1 
        echo "ZM is not up, starting..." >> /var/log/zoneminder/restart-daemon.log 2>&1 
        systemctl start zoneminder   >> /var/log/zoneminder/restart-daemon.log 2>&1 
    fi 
done 
I do not understand what could be problem with timezone? Kick off because of wrong timezone?

However I have correct in options. GMT +03:00 Europe, Helsinki

Little more information. I changed my multi server system to not use reverse proxy setup by httpd. Instead of that I use port forwarding with firewall config, server2 introduces https as in port 2443, (server1 default 443) so these are in servers config.

Now it works little better. No problem with live stream of both servers. But same kick out to login screen when I try to watch any event from server2. Or if I'm in events view by camera of server2 and refresh happens or I request refresh from browser.
I also noticed that if I look debug log from server2, I never see there messages like "[No username in session] at ...." like in server1 even when watch request fails and redirect to login screen happens. In that case there are simply no log at all.

What that could tell about the problem?

--
- TRa
tra
Posts: 19
Joined: Tue Jan 10, 2017 10:56 am

Re: Multi Server Setup

Post by tra »

More investigating. Little bit hard because I do not know much how multi server is implemented in ZM. I can say something very stupid, please correct.

Again server1 (mariadb there) is connected to internet (https 443), server2 connected to server1, and firewall setup is so that server2 is visible internet as https port 2443.

Looks like with events ZM somehow confuses at first time which server is valid. I mean, again examples from log.
At first I select events view from one camera from server2. Correctly this log is generated to server2 log, everything is still OK (at 14:00):
04/12/20 14:00:08.452417 web_php[1685].DBG [192.168.0.95] [Setting cookie parameters to lifetime(3600) path(/) domain () secure() httpOnly(1) name:ZMSESSID] at /usr/share/zoneminder/www/includes/session
.php line 20
.. similar messages ....
04/12/20 14:00:08.453441 web_php[59928].DBG [192.168.0.95] [Setting cookie parameters to lifetime(3600) path(/) domain () secure() httpOnly(1) name:ZMSESSID] at /usr/share/zoneminder/www/includes/session.php line 20
04/12/20 14:00:08.454065 web_php[1685].DBG [192.168.0.95] [View: image Request: Action: User: testuser] at /usr/share/zoneminder/www/index.php line 213
.....
But then I click one event. All logs are going to server1 log now, why! And it fails like this and kicks me to the login screen:
04/12/20 14:00:24.007796 web_php[1064493].DBG [80.186.161.122] [Setting cookie parameters to lifetime(3600) path(/) domain () secure() httpOnly(1) name:ZMSESSID] at /usr/share/zoneminder/www/includes/session.php line 20
04/12/20 14:00:24.008302 web_php[1064493].DBG [80.186.161.122] [No username in session] at /usr/share/zoneminder/www/includes/auth.php line 258
04/12/20 14:00:24.008508 web_php[1064493].DBG [80.186.161.122] [View: event Request: Action: User: none] at /usr/share/zoneminder/www/index.php line 213
04/12/20 14:00:24.008561 web_php[1064493].DBG [80.186.161.122] [Redirecting to login] at /usr/share/zoneminder/www/index.php line 246
04/12/20 14:00:24.008610 web_php[1064493].DBG [80.186.161.122] [Setting cookie parameters to lifetime(3600) path(/) domain () secure() httpOnly(1) name:ZMSESSID] at /usr/share/zoneminder/www/includes/session.php line 20
04/12/20 14:00:24.008706 web_php[1064493].DBG [80.186.161.122] [Redirecting to /zm/index.php?view=login] at /usr/share/zoneminder/www/index.php line 264
04/12/20 14:00:24.109010 web_php[1002355].DBG [80.186.161.122] [Setting cookie parameters to lifetime(3600) path(/) domain () secure() httpOnly(1) name:ZMSESSID] at /usr/share/zoneminder/www/includes/session.php line 20
04/12/20 14:00:24.109498 web_php[1002355].DBG [80.186.161.122] [No username in session] at /usr/share/zoneminder/www/includes/auth.php line 258
04/12/20 14:00:24.109867 web_php[1002355].DBG [80.186.161.122] [View: login Request: Action: User: none] at /usr/share/zoneminder/www/index.php line 213
.. etc....
When this happens, here are no logs at all on server2 logs!

Then I login again (about 14:01) and then I can get correct log to server2 web_php.log. Looks like ZM knows now that this event comes from server2 correctly. And I can see correctly the video. Logs then from server2:
04/12/20 14:01:06.472651 web_php[59928].DBG [192.168.0.95] [Setting cookie parameters to lifetime(3600) path(/) domain () secure() httpOnly(1) name:ZMSESSID] at /usr/share/zoneminder/www/includes/session.php line 20
04/12/20 14:01:06.474774 web_php[59928].DBG [192.168.0.95] [View: view_video Request: Action: User: testuser] at /usr/share/zoneminder/www/index.php line 213
04/12/20 14:01:06.476767 web_php[59928].DBG [192.168.0.95] [Path: /net/xxx99/s2/zoneminder/events/20/2020-04-12/14502/14502-video.mp4] at /usr/share/zoneminder/www/views/view_video.php line 43
04/12/20 14:01:06.476894 web_php[59928].DBG [192.168.0.95] [Using Range bytes=0-] at /usr/share/zoneminder/www/views/view_video.php line 71
04/12/20 14:01:06.476975 web_php[59928].DBG [192.168.0.95] [Using Range 0 66418080 size: 66418081, length: 66418081] at /usr/share/zoneminder/www/views/view_video.php line 78
04/12/20 14:01:06.746234 web_php[123904].DBG [192.168.0.95] [Setting cookie parameters to lifetime(3600) path(/) domain () secure() httpOnly(1) name:ZMSESSID] at /usr/share/zoneminder/www/includes/session.php line 20
04/12/20 14:01:06.748031 web_php[123904].DBG [192.168.0.95] [View: view_video Request: Action: User: testuser] at /usr/share/zoneminder/www/index.php line 213
04/12/20 14:01:06.750011 web_php[123904].DBG [192.168.0.95] [Path: /net/xxx99/s2/zoneminder/events/20/2020-04-12/14502/14502-video.mp4] at /usr/share/zoneminder/www/views/view_video.php line 43
04/12/20 14:01:06.750139 web_php[123904].DBG [192.168.0.95] [Using Range bytes=66224128-] at /usr/share/zoneminder/www/views/view_video.php line 71
04/12/20 14:01:06.750220 web_php[123904].DBG [192.168.0.95] [Using Range 66224128 66418080 size: 66418081, length: 193953] at /usr/share/zoneminder/www/views/view_video.php line 78
04/12/20 14:01:07.235374 web_php[127290].DBG [192.168.0.95] [Setting cookie parameters to lifetime(3600) path(/) domain () secure() httpOnly(1) name:ZMSESSID] at /usr/share/zoneminder/www/includes/session.php line 20
04/12/20 14:01:07.237238 web_php[127290].DBG [192.168.0.95] [View: view_video Request: Action: User: testuser] at /usr/share/zoneminder/www/index.php line 213
04/12/20 14:01:07.239404 web_php[127290].DBG [192.168.0.95] [Path: /net/xxx99/s2/zoneminder/events/20/2020-04-12/14502/14502-video.mp4] at /usr/share/zoneminder/www/views/view_video.php line 43
04/12/20 14:01:07.239611 web_php[127290].DBG [192.168.0.95] [Using Range bytes=32768-] at /usr/share/zoneminder/www/views/view_video.php line 71
.. ... etc....
You can see that server1 has in log public IP, server2 logs has internal IP-address of server1. I do not know, if this is problematic for ZM when they sure are not the same.

--
- TRa
Maximo1970
Posts: 97
Joined: Sun May 28, 2017 4:29 pm

Re: Multi Server Setup

Post by Maximo1970 »

I'm running a multi-server setup with 4 servers and using a reverse proxy to access. Here's my nginx config below, it may help you or hopefully give you something else to try.

Code: Select all

server {
        server_name [i][b]EXTERNAL FQDN[/b][/i];
        server_name [b][i]INTERNAL FQDN[/i][/b];

        location / {
                add_header Last-Modified $date_gmt;
                add_header Cache-Control 'no-store, no-cache, must-revalidate, proxy-revalidate, max-age=0';
                if_modified_since off;
                expires off;
                etag off;
                proxy_pass                              http://[b][i]IP ADDRESS OF ZONEMINDER SERVER[/i][/b];
                proxy_set_header Host                   $http_host;
                proxy_set_header X-Real-IP              $remote_addr;
                proxy_set_header X-Forwarded-For        $proxy_add_x_forwarded_for;
                proxy_set_header X-Forwarded-Proto      $scheme;
                proxy_set_header X-Forwarded-Port       $server_port;

        }
        location /zm/cgi-bin {
        auth_basic off;
                alias /usr/libexec/zoneminder/cgi-bin;
                include fastcgi_params;
                fastcgi_param SCRIPT_FILENAME $request_filename;
                fastcgi_param HTTP_PROXY "";
                fastcgi_pass unix:/var/run/fcgiwrap.socket;
        }
        location ~ /zm/api/(css|img|ico) {
                auth_basic off;
                rewrite ^/zm/api(.+)$ /api/app/webroot/$1 break;
                try_files $uri $uri/ =404;
        }
        listen 80;
}
tra
Posts: 19
Joined: Tue Jan 10, 2017 10:56 am

Re: Multi Server Setup

Post by tra »

Thanks! I have httpd, so nginx is not so familiar to me at all. So I need some time to study this, what are these all with httpd.
I do not fully understand the configuration yet. Interesting that there are 2 statements for server_name.

Do I understand this correctly, is this setup for main server which is available to internet? The other servers do not need any special?

What will change if there are (as you said) totally 4 different servers?

BTW, Does anybody use reverse proxy setup for httpd (apache)? I liked to see configuration :)

--
- TRa
Maximo1970
Posts: 97
Joined: Sun May 28, 2017 4:29 pm

Re: Multi Server Setup

Post by Maximo1970 »

The server_name is simply how you identify the response. Or put it another, it's the host header or virtual host (Apache term I believe) name.

My Multi-Server configuration has a total of 4 servers. The reverse proxy points to only one of these servers, in my instance this server collates all of the lower resolution feeds. I use the zmNinja app so there is no need for high bandwidth streams. When using the web interface over the Internet I'm usually just reviewing the captures from the previous day while I'm at work. I capture the Hi-Resolution streams via the other three servers.
tra
Posts: 19
Joined: Tue Jan 10, 2017 10:56 am

Re: Multi Server Setup

Post by tra »

It was much harder than I thought but now it works without kicking me to the login screen all the time :)
Expecially thanks to Maximo for pointing me good direction to use proxy pass setup.
I think problem is that when I have same hostnames in options --> servers setup for both servers. Then ZM is confused in some circumstances which server is correct. Now I have dyn DNS setup so that I have two different FQDNs to both machines, even the public IP of them is same.
I have also let's encrypt certificates and setup for both FQDNs, Then the problems went away.

So the setup, if somebody want to do it with apache, that should work :) :
server1 is main server, has mariadb and cameras. Internal ip is 192.168.0.95 FQDN is xxx.foo.bar
server2 is second server has only cameras, no database, only connected to internet via server1. Internal ip is 192.168.0.99 FQDN is xxx99.foo.bar

Apache setup in server1, my-proxy.conf
<IfModule mod_ssl.c>
<VirtualHost _default_:443>
ServerName xxx99.foo.bar

############################################
# Multiserver setup for zoneminder)
<IfModule mod_proxy.c>
SSLProxyEngine on
SSLProxyVerify none
SSLProxyCheckPeerCN off
SSLProxyCheckPeerExpire off
</IfModule>

ProxyPass "/" "https://192.168.0.99/"
ProxyPassReverse "/" "https://192.168.0.99/"
########################################################################
SSLCertificateFile /etc/letsencrypt/live/xxx.foo.bar/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/xxx.foo.bar/privkey.pem
Include /etc/letsencrypt/options-ssl-apache.conf
</VirtualHost>
</IfModule>
--
- TRa
Post Reply