GitLab Geo configuration

Note: This is the documentation for installations from source. For installations using the Omnibus GitLab packages, follow the Omnibus GitLab Geo nodes configuration guide.

Configuring a new secondary node

Note: This is the final step in setting up a secondary Geo node. Stages of the setup process must be completed in the documented order. Before attempting the steps in this stage, complete all prior stages.

The basic steps of configuring a secondary node are:

replicate required configurations between the primary and the secondaries;
configure a second, tracking database on each secondary;
start GitLab on the secondary node's machine.

You are encouraged to first read through all the steps before executing them in your testing/production environment.

Notes:

Do not setup any custom authentication in the secondary nodes, this will be handled by the primary node.

Do not add anything in the secondaries Geo nodes admin area (Admin Area ➔ Geo Nodes). This is handled solely by the primary node.

Step 1. Copying the database encryption key

GitLab stores a unique encryption key on disk that is used to encrypt sensitive data stored in the database. All secondary nodes must have the exact same value for db_key_base as defined on the primary node.

SSH into the primary node, and execute the command below to display the current encryption key:
```
sudo -u git -H bundle exec rake geo:db:show_encryption_key RAILS_ENV=production
```

Copy the encryption key to bring it to the secondary node in the following steps.

SSH into the secondary, and execute the command below to open the secrets.yml file:
```
sudo -u git -H editor config/secrets.yml
```
Change the value of db_key_base to the output from the primary node. Then save and close the file.
Restart GitLab for the changes to take effect:
```
service gitlab restart
```

The secondary will start automatically replicating missing data from the primary in a process known as backfill. Meanwhile, the primary node will start to notify changes to the secondary, which will act on those notifications immediately. Make sure the secondary instance is running and accessible.

Step 2. (Optional) Enabling hashed storage

Once restarted, the secondary will automatically start replicating missing data from the primary in a process known as backfill. Meanwhile, the primary node will start to notify the secondary of any changes, so that the secondary can act on those notifications immediately.

Make sure the secondary instance is running and accessible. You can login to the secondary node with the same credentials as used in the primary.

Step 2. (Optional) Enabling hashed storage (from GitLab 10.0)

Read Enabling Hashed Storage

Step 3. (Optional) Configuring the secondary to trust the primary

You can safely skip this step if your primary uses a CA-issued HTTPS certificate.

If your primary is using a self-signed certificate for HTTPS support, you will need to add that certificate to the secondary's trust store. Retrieve the certificate from the primary and follow your distribution's instructions for adding it to the secondary's trust store. In Debian/Ubuntu, for example, with a certificate file of primary.geo.example.com.crt, you would follow these steps:

sudo -i
cp primary.geo.example.com.crt /usr/local/share/ca-certificates
update-ca-certificates

Step 4. Managing the secondary GitLab node

Congratulations! Your secondary geo node is now configured!

The initial replication, or 'backfill', will probably still be in progress. You can monitor the synchronization process on each geo node from the primary node's Geo Nodes dashboard (Admin Area ➔ Geo Nodes, /admin/geo_nodes) in your browser.

After the backfill is completed you can continue to monitor geo node health and replication delays from the dashboard.

The two most obvious issues that can become apparent in the dashboard are:

Database replication not working well
Instance to instance notification not working. In that case, it can be something of the following:
- You are using a custom certificate or custom CA (see the ### Step 4. Enable Git access over HTTP/HTTPS
- Instance is firewalled (check your firewall rules)

Please note that disabling a secondary node will stop the sync process.

Please note that if git_data_dirs is customized on the primary for multiple repository shards you must duplicate the same configuration on the secondary.

GitLab Geo synchronizes repositories over HTTP/HTTPS, and so requires this clone method to be enabled. Navigate to Admin Area ➔ Settings (/admin/application_settings) on the primary node, and set Enabled Git access protocols to Both SSH and HTTP(S) or Only HTTP(S).

Verify proper functioning of the secondary node

Read Verify proper functioning of the secondary node.

Selective replication

Read Selective replication.

Troubleshooting

Read the troubleshooting document.