Turvaserveri klasterdamine
Allpool olev juhend on konkreetne samm-sammuline kokkuvõte üldjuhendist: https://github.com/nordic-institute/X-Road/blob/master/doc/Manuals/LoadBalancing/ig-xlb_x-road_external_load_balancer_installation_guide.md
Pilt NIISi juhendist master-slave klastrist.
MASTER (osa käske vaja ka SLAVE peal teha; vaikimisi kõik on MASTER käsud):
- Install the X-Road security server packages using the normal installation procedure or use an existing standalone node. (Soovitus kohe SLAVE peal ka X-Road tarkvara paigaldada, initsialiseerima ei pea)
- Stop the xroad services.
- service xroad-* stop
Create a separate PostgreSQL instance for the serverconf database (see section 4. Database replication setup for details in GITHUB MANUAL, link above; use the newest PostgreSQL version if possible):
Create replication keys for every node (on master node):
openssl req -new -x509 -days 7300 -nodes -sha256 -out ca.crt -keyout ca.key -subj '/O=cluster/CN=CA' openssl req -new -nodes -days 7300 -keyout server.key -out server.csr -subj "/O=cluster/CN=master" openssl req -new -nodes -days 7300 -keyout server_slave1.key -out server_slave1.csr -subj "/O=cluster/CN=slave1" openssl x509 -req -in server.csr -CAcreateserial -CA ca.crt -CAkey ca.key -days 7300 -out server.crt openssl x509 -req -in server_slave1.csr -CAcreateserial -CA ca.crt -CAkey ca.key -days 7300 -out server_slave1.crt
CODEkõigi node'de peal:
sudo mkdir -p -m 0755 /etc/xroad/postgresql; sudo chmod o+x /etc/xroad
CODECopy ca.crt server.crt server.key to /etc/xroad/postgresql/
# masteris cp ca.crt server.crt server.key /etc/xroad/postgresql/ # igasse slave node-i masteri pealt: scp ca.crt server_slave1.crt server_slave1.key domain-user@slave-ss-server:/tmp/ # slave peal: cd /tmp; cp ca.crt server_slave1.crt server_slave1.key /etc/xroad/postgresql/ # kõigis masinates sudo chown postgres:postgres /etc/xroad/postgresql/*; sudo chmod 400 /etc/xroad/postgresql/*
CODEMaster masinas:
sudo -u postgres pg_createcluster -p 5433 14 serverconf # muuta alljärgnevad read failis: vi /etc/postgresql/14/serverconf/postgresql.conf ssl = on ssl_ca_file = '/etc/xroad/postgresql/ca.crt' ssl_cert_file = '/etc/xroad/postgresql/server.crt' ssl_key_file = '/etc/xroad/postgresql/server.key' listen_addresses = '*' wal_level = replica max_wal_senders = 4 wal_keep_size = 8 # wal_keep_segments = 10 (on Postgre 10 instead) # vi /etc/postgresql/14/serverconf/pg_hba.conf #lisada lõppu rida hostssl replication +slavenode samenet cert # käivitab uue eraldatud serverconf baasi ja loob sinna kasutaja (kasutajale vaja parool määrata): systemctl start postgresql@14-serverconf sudo -u postgres psql -p 5433 -c "CREATE ROLE slavenode NOLOGIN"; sudo -u postgres psql -p 5433 -c "CREATE USER slave1 REPLICATION PASSWORD NULL IN ROLE slavenode"; sudo -u postgres psql -p 5433 -c "CREATE USER serverconf PASSWORD '<parool tuleb /etc/xroad/db.properties failist>'"; # ainult remote baasi korral on vaja hoolitseda, et baasi loomiseks vajalik serverconf_admin kasutaja oleks olemas /etc/xroad.properties failis. sudo -u postgres psql -p 5433 -c "CREATE USER serverconf_admin PASSWORD '<parool tuleb /etc/xroad.properties failist>'"; # jätkame uue serveconfi baasi loomisega sudo -u postgres pg_dump -C serverconf | sudo -u postgres psql -p 5433 -f - sudo -u postgres psql -p 5432 -c "ALTER DATABASE serverconf RENAME TO serverconf_old";
CODESLAVE peal:
SLAVE: # sisesta käsud (viimane käsk on kaherealine): sudo -u postgres pg_createcluster -p 5433 14 serverconf; cd /var/lib/postgresql/14/serverconf/; rm -rf *; sudo -u postgres PGSSLMODE=verify-ca PGSSLROOTCERT=/etc/xroad/postgresql/ca.crt PGSSLCERT=/etc/xroad/postgresql/server_slave1.crt PGSSLKEY=/etc/xroad/postgresql/server_slave1.key pg_basebackup -h master-ss-server-IP-or-hostname -p 5433 -U slave1 -D .; # Vaja luua replikeerimise jaoks stanby.signal: touch /var/lib/postgresql/14/serverconf/standby.signal chown postgres:postgres /var/lib/postgresql/14/serverconf/standby.signal chmod 0600 /var/lib/postgresql/14/standby.signal # muuta alljärgnevad read failis: vi /etc/postgresql/14/serverconf/postgresql.conf listen_addresses = 'localhost' ssl = on ssl_ca_file = '/etc/xroad/postgresql/ca.crt' ssl_cert_file = '/etc/xroad/postgresql/server_slave1.crt' ssl_key_file = '/etc/xroad/postgresql/server_slave1.key' # leave all WAL settings be default primary_conninfo = 'host=<master-ss-server-IP-or-hostname> port=5433 user=slave1 sslmode=verify-ca sslcert=/etc/xroad/postgresql/server.crt sslkey=/etc/xroad/postgresql/server.key sslrootcert=/etc/xroad/postgresql/ca.crt' hot_standby = on hot_standby_feedback = on # käivitada uus baas systemctl start postgresql@14-serverconf
CODE
- Change /etc/xroad/db.properties to point to the separate database instance (MASTER):
- vi /etc/xroad/db.properties
- serverconf.hibernate.connection.url : Change the url port number from 5432 to 5433
- If you are using an already configured server as the master, the existing configuration was replicated to the slaves in step 3. Otherwise, proceed to configure the master server: install the configuration anchor, set up basic information, create authentication and signing keys and so on. See the security server installation guide [IG-SS] for help with the basic setup.
Set up the configuration file replication:
adduser --system --shell /bin/bash --ingroup xroad xroad-slave sudo mkdir -m 755 -p /home/xroad-slave/.ssh && sudo touch /home/xroad-slave/.ssh/authorized_keys # SLAVE peal: su xroad ssh-keygen less ~/.ssh/id_rsa.pub # MASTER peal: vi /home/xroad-slave/.ssh/authorized_keys # kopeeri avanenud faili SLAVE-s olev id_rsa.pub sisu # SLAVE (ikka xroad kasutaja õigustes): #connect to the master host using ssh and accept the host key. ssh -i /var/lib/xroad/.ssh/id_rsa xroad-slave@master-ss-server-IP-or-hostname exit exit (tagasi root õigustesse) # SLAVE peal kopeeri allpool olevad read faili (sisesta õige master serveri aadress) vi /etc/systemd/system/xroad-sync.service [Unit] Description=X-Road Sync Task After=network.target Before=xroad-proxy.service Before=xroad-signer.service Before=xroad-confclient.service Before=xroad-jetty.service [Service] User=xroad Group=xroad Type=oneshot Environment=XROAD_USER=xroad-slave Environment=MASTER=master-ss-server-IP-or-hostname ExecStartPre=/usr/bin/test ! -f /var/tmp/xroad/sync-disabled ExecStart=/usr/bin/rsync -e "ssh -o ConnectTimeout=5 " -aqz --timeout=10 --delete-delay --exclude db.properties --exclude "/conf.d/node.ini" --exclude "/gpghome" --exclude "*.tmp" --exclude "/postgresql" --exclude "/nginx" --exclude "/globalconf" --exclude "/jetty" --delay-updates --log-file=/var/log/xroad/slave-sync.log ${XROAD_USER}@${MASTER}:/etc/xroad/ /etc/xroad/ [Install] WantedBy=multi-user.target WantedBy=xroad-proxy.service # SLAVE peal tee samamoodi veel failiga vi /etc/systemd/system/xroad-sync.timer [Unit] Description=Sync X-Road configuration [Timer] OnBootSec=60 OnUnitActiveSec=60 [Install] WantedBy=timers.target # SLAVE peal taustatöö käivitamiseks: systemctl enable xroad-sync.timer xroad-sync.service systemctl start xroad-sync.timer # SLAVE peal loo fail /etc/logrotate.d/xroad-slave-sync alljärgneva sisuga: vi /etc/logrotate.d/xroad-slave-sync /var/log/xroad/slave-sync.log { daily rotate 7 missingok compress su xroad xroad nocreate }
CODEConfigure the node type as master in /etc/xroad/conf.d/node.ini (MASTER)
# tekita fail /etc/xroad/conf.d/node.ini sisuga: [node] type=master #muuda faili õigused chown xroad:xroad /etc/xroad/conf.d/node.ini
CODEDisable support for client-side pooled connections (HTTP connection persistence) in /etc/xroad/conf.d/local.ini
# vi /etc/xroad/conf.d/local.ini [proxy] server-support-clients-pooled-connections=false
CODESeadista op-monitori teenus töötama ühes klastri masinas või eraldiseisvas masinas (https://www.x-tee.ee/docs/live/xroad/ug-ss_x-road_7_security_server_user_guide.html#1524-installing-an-external-operational-monitoring-daemon)
Op-monitori masteriks valin hetkel master masina:
# vi /etc/xroad/conf.d/local.ini [op-monitor] host = <masteri host aadress>
CODEPeatan teenuse teistes masinates (minu näite puhul SLAVE-s):
# on other (e.g. SLAVE) machines: service xroad-opmonitor stop systemctl disable xroad-opmonitor
CODE
- Start the X-Road services.
- service xroad-* start
SLAVE osa lisaks eespool tehtule:
- Install security server packages using the normal installation procedure.
- Stop the xroad services.
- service xroad-* stop
- Create a separate PostgreSQL instance for the serverconf database (juba tehtud eespool)
Change /etc/xroad/db.properties to point to the separate database instance and change password to match the one defined in the master database
# vi /etc/xroad/db.properties serverconf.hibernate.connection.url : Change the url port number from 5432 to 5433 serverconf.hibernate.connection.password: Change to match the master db's password (vaata master masinast samast failist).
CODESet up SSH between the master and the slave (juba tehtud)
Set up state synchronization using rsync+ssh (enamus tehtud, kuid lisaks):
rsync -e "ssh -i /var/lib/xroad/.ssh/id_rsa" -avz --delete --exclude db.properties --exclude "/postgresql" --exclude "/conf.d/node.ini" --exclude "/gpghome" --exclude "/nginx" xroad-slave@master-ss-server-IP-or-hostname:/etc/xroad/ /etc/xroad/ # if got an error "rsync: opendir "/etc/xroad/signer" failed: Permission denied (13)" then: chmod 750 /etc/xroad/signer/
CODEConfigure the node type as slave in /etc/xroad/conf.d/node.ini
# vi /etc/xroad/conf.d/node.ini [node] type=slave # muuda õigused chown xroad:xroad /etc/xroad/conf.d/node.ini
CODEStart the X-Road services.
service xroad-* start
Selleks, et igaöine autobackup töötaks vaja slave õlal genereerida gpgvõtmed:
sudo -i -u xroad /usr/share/xroad/scripts/generate_gpg_keypair.sh /etc/xroad/gpghome 'turvaserveri_pikk_id'#to get a Security Server's ID:/usr/share/xroad/scripts/get_security_server_id.sh
CODE
Remote baasi korral on vaja hoolitseda, et baasi loomiseks vajalikud admin kontod (just serverconf_admin) oleksid määratud /etc/xroad.properties failis. Vastav õpetus juhendis: https://x-tee.ee/docs/live/xroad/ig-ss_x-road_v7_security_server_installation_guide.html#26-remote-database-setup-optional
Klasterdatud turviku õlas hooldusrežiimi aktiveerimine:
curl http://localhost:5566/maintenance?targetState=true
PS!
Kui muutub master või slave masina hostname, siis vaja genereerida uued ssh võtmed xroad-slave kasutaja jaoks. Muidu ei tööta xroad-sync.service sees olev rsync enam. (käsitsi rsync ssh -i töötab edasi)
Hostinime vahetus vaja teha ka /var/lib/postgresql/10/serverconf/recovery.conf sees.