Analytical Stack Server Configuration
- Víctor Enrique Mendoza Yanes
- Ricardo Gomez
On this page:
The following are the specs where the eLearning analytical stack has been installed and tested. Please try to recreate the following environment as close as possible to ensure the correct functionality of all components.
Hardware Specs
These specs belong to a t3.large EC2 machine.
Item | Value | Notes |
|---|---|---|
vCPUs | 2 |
|
RAM | 8 GiB |
|
SSD Storage | 30 GiB | If on AWS: gp3 volume |
Software Specs
Item | Value | Notes |
|---|---|---|
Operating System | ubuntu-noble-24.04-amd64-server-20240801 |
|
Web Server | nginx/1.24.0 (Ubuntu) |
|
Virtualization Software | Docker/27.2.1 (build 9e34c9b) |
|
SSL Certificate | Certbot/2.11.0 | Generates and automatically renews SSL certificates. |
Open ports
Item | Notes |
|---|---|
80 | Certbot needs this port open for generating and the renewal of SSL certificates. |
443 | HTTPS port |
22 | SSH access |
Make sure to install nginx before generating an SSL certificate using Certbot, as Certbot can automatically install the certificate and make all the required configurations on nginx. This guide makes use of that functionality.
Certbot - SSL Certificate
Having an SSL certificate ensures a secured connection between users and the server, and that no data is compromised while it is traveling over the internet. This allows users to connect using HTTPS protocol over the port 443.
Before starting this guide, it is required that the server has nginx installed. To install nginx run the following commands:
sudo apt update
sudo apt install nginxOpen ports 80 and 443, belonging to HTTP and HTTPS respectively. Port 22 is also required.
Connect via SSH to the server with a user with sudo privileges.
Install Certbot
sudo snap install --classic certbotPrepare the Certbot command
sudo ln -s /snap/bin/certbot /usr/bin/certbotGenerate the certificate, this will also automatically edit the nginx configuration to serve it.
Certbot will ask some questions, like an email to send notifications about certificate renewals.
After the initial questions, Certbot will ask for the domain names to issue the certificate. It will try to access the server over port 80 using the domain name, so it’s imperative the DNS records are already configured.
Test that Certbot is capable of renewing the certificate, otherwise after a couple of months it will expire and users will lose access to the services.
Renewal succeeded output.
Official installation guide: Certbot Instructions
Official documentation: Welcome to the Certbot documentation! — Certbot 3.1.0 documentation
Nginx Configuration
NiFi by default runs on port 8443, similarly, Superset runs on port 8088. However, to have a secured connection users should only be able to connect through port 443, where the SSL certificate is served.
Nginx will act as a reverse-proxy and redirect users requests that come through port 443 to the correct destinations.
Nginx configuration file can be edited using the following command:
Search for the server bracket that is listening to port 443. Certbot configuration can be found here.
Edit and add the locations to redirect traffic. For superset:
For NiFi:
The above configuration will make Superset accessible by using the base domain (https://exampledomain.com) and NiFi accesible by adding /nifi as a path (https://exampledomain.com/nifi)
The following is a complete example of the file:
Additionally, it is necessary to add one more parameter to the nginx main configuration file:
Edit the nginx.conf file.
On the http bracket, add the following line.
This ensures there will be no problems when uploading the NiFi pipelines or the Superset dashboards.
Docker Configuration
create Docker network so containers can see eachother
docker network create my-network
docker network connect my-network container1
docker network ls