Upgrade Graylog Server from 4.x to 5

: Travis Nuske; 09 Mar 2023

Guide for upgrading Graylog server and required components from version 4.x to 5 on Ubuntu 20.04.5.

There are 4 main components that need to be upgraded to to run Graylog Server version 5.x - the database (MongoDB), the java environment (JDK), the search engine (ElasticSearch), and the Graylog server itself.

Upgrade MongoDB

Graylog Server 5.x requires at least version 5.x of MongoDB. To check your current version of MongoDB, log into your server and run this command:

 $ mongo

 > db.version()

In my environment, this command returned 4.0.28, so I need to upgrade it to version 4.2, then to 4.4 and finally to version 5. Note there is no direct upgrade from version 4.0 to 5; you must upgrade to 4.2 and then to 4.4 before upgrading to 5.0.

Increase Open Files ulimit Setting

Starting from Mongo 4.2, you need to increase the open files limit in the operating system. To check your current ulimit setting, log into your server and run this command:

$ ulimit -n

If the returned value 1024, I'd suggest increasing this value. If it's unlimited or a very large value, skip this step. To increase the ulimit value in Ubuntu:

Edit the following file:
/etc/security/limits.conf

Add following lines to it:

* soft nproc 65535 
* hard nproc 65535 
* soft nofile 65535 
* hard nofile 65535
root soft nproc 65535 
root hard nproc 65535 
root soft nofile 65535 
root hard nofile 65535

Edit the following file:

/etc/pam.d/common-session

Add this line to it

session required pam_limits.so

logout, log back in and run the following command:

$ ulimit -n

The open files limit should now be set to 65535

Upgrade MongoDB from 4.0 to 4.2

Open the mongo console:

$ mongo

Run this command to check the current feature level:

> db.adminCommand( { getParameter: 1, featureCompatibilityVersion: 1 } )

If the featureCompatibilityVersion displayed in the output is not 4.0, run this command to set the featureCompibilityVersion to 4.0:

> db.adminCommand( { setFeatureCompatibilityVersion: "4.0" } )

Exit from the mongo console:

$ exit

Download and install the Mongo 4.2 server keys:

wget -qO - https://www.mongodb.org/static/pgp/server-4.2.asc |sudo apt-key add -

This command will return 'OK' when completed.

Add the MongoDB 4.2 repo:

$ sudo echo "deb [ arch=amd64,arm64 ] https://repo.mongodb.org/apt/ubuntu bionic/mongodb-org/4.2 multiverse"|sudo tee /etc/apt/sources.list.d/mongodb-org-4.2.list

Install MongoDB 4.2.18 by running this command:

$ sudo apt-get install -y mongodb-org=4.2.18 mongodb-org-server=4.2.18 mongodb-org-shell=4.2.18 mongodb-org-mongos=4.2.18 mongodb-org-tools=4.2.18

Once the install is complete, reboot the server.

$ sudo reboot

Log back in and check the MongoDB version.

$ mongo

> db.version()

You should now see the server is running version 4.2.18.

At this point I'd suggest logging back into your Graylog instance and check that everything is working correctly.

Set the FeatureCompatibilityVersion to 4.2:

> db.adminCommand( { setFeatureCompatibilityVersion: "4.2" } )

Upgrade MongoDB from 4.2 to 4.4

Download and install the Mongo 4.4 server keys:

$ wget -qO - https://www.mongodb.org/static/pgp/server-4.4.asc | sudo apt-key add -

This command will return 'OK' when completed.

Add the MongoDB 4.4 repo:

$ sudo echo "deb [ arch=amd64,arm64 ] https://repo.mongodb.org/apt/ubuntu focal/mongodb-org/4.4 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-4.4.list
$ sudo apt-get update

Install MongoDB 4.4.18 by running this command:

$ sudo apt-get install -y mongodb-org=4.4.18 mongodb-org-server=4.4.18 mongodb-org-shell=4.4.18 mongodb-org-mongos=4.4.18 mongodb-org-tools=4.4.18
$ sudo apt-get install -y mongodb-org-database-tools-extra

Once the install is complete, reboot the server.

$ sudo reboot

Log back in and check the MongoDB version.

$ mongo

> db.version()

You should now see the server is running version 4.4.18.

At this point I'd suggest logging back into your Graylog instance and check that everything is working correctly.

Set the FeatureCompatibilityVersion to 4.4:

> db.adminCommand( { setFeatureCompatibilityVersion: "4.4" } )

Upgrade MongoDB from 4.4 to 5.0

Download and install the Mongo 5.0 server keys:

$ wget -qO - https://www.mongodb.org/static/pgp/server-5.0.asc | sudo apt-key add -

This command will return 'OK' when completed.

Add the MongoDB 5.0 repo:

$ sudo echo "deb [ arch=amd64,arm64 ] https://repo.mongodb.org/apt/ubuntu focal/mongodb-org/5.0 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-5.0.list
$ sudo apt-get update

Install MongoDB 4.4.18 by running this command:

$ sudo apt-get install -y mongodb-org=5.0.14 mongodb-org-database=5.0.14 mongodb-org-server=5.0.14 mongodb-org-shell=5.0.14 mongodb-org-mongos=5.0.14 mongodb-org-tools=5.0.14sudo

Once the install is complete, reboot the server.

$ sudo reboot

Log back in and check the MongoDB version.

$ mongo

> db.version()

You should now see the server is running version 5.0.14.

At this point I'd suggest logging back into your Graylog instance and check that everything is working correctly.

Set the FeatureCompatibilityVersion to 5.0:

> db.adminCommand( { setFeatureCompatibilityVersion: "5.0" } )

Update The Java Environment (JDK)

From the command line, run this command:

$ apt install -y openjdk-17-jdk openjdk-17-jre

Check the latest java version is being used:

$ java -version

You should see the version is now "17.0.6"

ElasticSearch Update

Run this command to check your current version of ElasticSearch:

$ curl -XGET 'http://localhost:9200'

Check the version - number section of the output. Note that the latest supported version of ElasticSearch is 7.10.2.

If you need to update to this specific version of ElasticSearch, Run these commands:

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.10.2-amd64.deb
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.10.2-amd64.deb.sha512
shasum -a 512 -c elasticsearch-7.10.2-amd64.deb.sha512 
sudo dpkg -i elasticsearch-7.10.2-amd64.deb