This guide provides an overview of the hardware, software and communication requirements for an on-premises installation of Projectal.  It also contains instructions how to install and maintain your Projectal.

Projectal is a modern, web-based application that can be easily scaled to support 100 users to 100,000 users. This allows you to scale it to fit the size and needs of your company.

This guide is intended for experienced IT professionals at a company that has licensed Projectal to install Projectal onto their company's IT infrastructure.  It is required that you have good Linux system administration skills to complete a successful Projectal installation.

Tip: The Projectal Support team are available to complete the installation of Projectal for you onto your company's IT infrastructure as part of your Projectal license.  This ensures that your Projectal is running correctly with little inconvenience.  Contact the Projectal Support team via the helpdesk if you would like our help.

1. Projectal License Key & Software Repository

When your company licenses Projectal and your company requests that it wishes to run Projectal as on-premises installation on your company's IT infrastructure, then the Projectal Support team will provide two important items:

• PROJECTAL-LICENSE-KEY - This is your company's unique license key for running Projectal.  Without it, you cannot run Projectal.
• PROJECTAL-REPOSITORY - This is the download location for accessing the installation files for Projectal to complete a Projectal install on your company's IT infrastructure.

These two items are referenced below in the installation instructions.  If you do not have them, then contact the Projectal Support team.

2. Architecture

The basic server architecture that is required for running Projectal on-premises within your IT infrastructure is as follows:

3. Load Balancer

The load balancer sits in front of the API servers and routes user client requests to the API servers.

If you have an existing load balancer within your IT infrastructure, then you may be able to use it for Projectal’s load balancing requirements.

If you do not have an existing load balancer, then it is recommended using HAProxy.  See https://www.haproxy.org/.

API servers will listen for incoming connections on port 8443. The load balancer should direct incoming requests to Projectal to this port on the API servers.

4. API Server

The API servers process each incoming user request. The more users, then the more API servers are required.

The typical starting Projectal configuration recommended is to have 3 x API servers. For small use cases, then a minimum of 2 x API servers is recommended.

Each API server should be running Ubuntu 20.04 server.

The API Server should have the following minimum specification:

• RAM: 16GB
• Storage: 8GB
• CPUs: 4

5. DB Server

The Database server stores all information contained in Projectal. The storage database used is Cassandra and it can be clustered if required.

1 x Database server is required.

The Database server should be running Ubuntu 20.04 server.

The Database server should have the following minimum specification:

• RAM: 32GB
• Storage: 1TB
• CPUs: 8

The Database server will need a static IP address.

6. DB Server - Preparation

Open a terminal connection to the Projectal DB server.  Before beginning the Projectal installation, make sure all software on the server is up to date.

sudo apt-get update

sudo apt-get upgrade -y

7. DB Server - Java

The Projectal components require version 8 of the open JDK to be installed.

sudo apt install openjdk-8-jdk -y

8. DB Server - Cassandra

Install the Cassandra component by adding the Cassandra source.

echo "deb https://debian.cassandra.apache.org 40x main" | sudo tee -a /etc/apt/sources.list.d/cassandra.sources.list

Add the Cassandra key to verify the installation sources.

curl https://downloads.apache.org/cassandra/KEYS | sudo apt-key add

sudo apt-get update

sudo apt-get install cassandra -y

9. DB Server - Elasticsearch

Download the Elasticsearch 8.12.0 package.

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-8.12.0-amd64.deb

Install the package.

sudo dpkg -i elasticsearch-8.12.0-amd64.deb

Enable and start the service.

sudo systemctl enable elasticsearch

sudo systemctl start elasticsearch

10. DB Server - Janusgraph

The Janusgraph component requires a user account to be created.

sudo useradd -M janusgraph

To extract the Janusgraph components, the unzip tool is needed.

sudo apt-get install zip unzip -y

Download the Janusgraph component.

wget https://github.com/JanusGraph/janusgraph/releases/download/v1.0.0/janusgraph-1.0.0.zip

Extract the components from the compressed zip file.

unzip janusgraph-1.0.0.zip

Transfer ownership of the files to the user account previously created.

sudo chown -R janusgraph:janusgraph janusgraph-1.0.0

Move the files to a system location and create a symbolic link.

sudo mv janusgraph-1.0.0 /opt

sudo ln -s /opt/janusgraph-1.0.0/ /usr/share/janusgraph

11. DB Server - Service Files

Projectal connects to Janusgraph using a web socket.  Projectal needs the following service and helper scripts to be installed.

Download the Janusgraph service files from the provided PROJECTAL-REPOSITORY.

wget https://PROJECTAL-REPOSITORY/projectal-janusgraph-config.zip -O projectal-janusgraph-config.zip

Extract the compressed archive.

unzip projectal-janusgraph-config.zip

Change the upgrade script to be executable.

sudo chmod a+x projectal-janusgraph-config/upgrade_db_schema.sh

Change the service script to be executable.

sudo chmod a+x projectal-janusgraph-config/janusgraph-service.sh

Move the scripts and config files to the Janusgraph installation location.

sudo mv projectal-janusgraph-config/gremlin-server-websoc.sh /usr/share/janusgraph/bin/

sudo mv projectal-janusgraph-config/gremlin-server-websoc.conf /usr/share/janusgraph/bin/

sudo mv projectal-janusgraph-config/gremlin-server-websoc.yaml /usr/share/janusgraph/conf/gremlin-server/

sudo mv projectal-janusgraph-config/janusgraph-cql-es-server-websoc.properties /usr/share/janusgraph/conf/gremlin-server/

sudo mv projectal-janusgraph-config/janusgraph-websoc.service /etc/systemd/system/

sudo mv projectal-janusgraph-config/log4j-server.properties /opt/janusgraph-1.0.0/conf/gremlin-server/log4j-server.properties

sudo mv projectal-janusgraph-config/remote-objects.yaml /opt/janusgraph-1.0.0/conf/remote-objects.yaml

sudo mv projectal-janusgraph-config/ex-open-janusks.groovy /opt/janusgraph-1.0.0/bin/ex-open-janusks.groovy

Change ownership and permissions of the scripts and files.

sudo chown root:root /etc/systemd/system/janusgraph-websoc.service

sudo chmod 664 /etc/systemd/system/janusgraph-websoc.service

sudo chown -R janusgraph:janusgraph /usr/share/janusgraph/

sudo chmod 755 /usr/share/janusgraph/bin/gremlin-server-websoc.sh

sudo chmod 777 /opt/janusgraph-1.0.0/conf/gremlin-server/log4j-server.properties

sudo chmod 777 /opt/janusgraph-1.0.0/bin/ex*

Enable the service and start it.

sudo systemctl enable janusgraph-websoc

sudo systemctl start janusgraph-websoc

12. DB Server - Projectal

Download the Projectal components from the provided PROJECTAL-REPOSITORY.

wget https://PROJECTAL-REPOSITORY/projectal-janusgraph-jars.zip -O projectal-janusgraph-jars.zip

Unzip the compressed archive.

unzip projectal-janusgraph-jars.zip

Create the Projectal schema.  Replace <version> with the version of the janusks-schema jar (see unzipped files).

sudo java -cp projectal-janusgraph-jars/janusks-schema-<version>.jar com.janusks.schema.SchemaManager --conf default --drop schema --note study --overwrite

Create the Projectal database entries for company and the super user account.  Replace <company-name> with the name of the company licensing Projectal.   Replace <version> with the version of the janusks-gremlin jar (see unzipped files).  Replace <email-address> with your email address.

sudo java -cp projectal-janusgraph-jars/janusks-gremlin-<version>.jar com.janusks.gremlin.ProjectalInstall -a clean_install -u <email-address> -c "<company-name>" -f application.properties

13. DB Server - Kafka

Download the Kafka package.

wget https://downloads.apache.org/kafka/3.6.2/kafka_2.13-3.6.2.tgz

Extract the compressed files.

tar -vxf kafka_2.13-3.6.2.tgz

Add the Kafka user account.

sudo useradd -M kafka

Transfer the ownership of the files to the new Kafka account.

sudo chown -R kafka:kafka kafka_2.13-3.6.2

Move the files to a system location and create a symbolic link.

sudo mv kafka_2.13-3.6.2 /opt

sudo ln -s /opt/kafka_2.13-3.6.2/ /usr/share/kafka

Download the Projectal Kafka service and config files from the provided PROJECTAL-REPOSITORY.

wget https://PROJECT-REPOSITORY/projectal-kafka-config.zip -O projectal-kafka-config.zip

Extract the compressed files.

unzip projectal-kafka-config.zip

Set the IP address of the server in the config file. Replace <ip-address> with the IP address of the server.

sed -i "s/kafka_host_name/<ip-address>/g" projectal-kafka-config/server.properties

Move the files to the Kafka installation location.

sudo mv projectal-kafka-config/kafka_server_jaas.conf /usr/share/kafka/config/

sudo mv projectal-kafka-config/zookeeper_jaas.conf /usr/share/kafka/config/

sudo mv projectal-kafka-config/server.properties /usr/share/kafka/config/

sudo mv projectal-kafka-config/zookeeper.properties /usr/share/kafka/config/

sudo mv projectal-kafka-config/kafka.service /etc/systemd/system/

sudo mv projectal-kafka-config/zookeeper.service /etc/systemd/system/

Transfer ownership of the service files to the root account.

sudo chown root:root /etc/systemd/system/kafka.service

sudo chmod 664 /etc/systemd/system/kafka.service

sudo chown root:root /etc/systemd/system/zookeeper.service

sudo chmod 664 /etc/systemd/system/zookeeper.service

Transfer ownership of the Kafka installation files to the Kafka account.

sudo chown -R kafka:kafka /usr/share/kafka/

Enable and start the services.

sudo systemctl enable zookeeper

sudo systemctl start zookeeper

sudo systemctl enable kafka

sudo systemctl start kafka

Create the topics for Kafka notifications.

/usr/share/kafka/bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic webhook_event

/usr/share/kafka/bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic webhook_refresh

14. DB Server - MySQL

Install the MySQL component.

sudo apt-get install mysql-server -y

Create the Kafka database.

sudo mysql -u root < projectal-kafka-config/kafka.sql

Connections from other servers to MySQL need to be permitted.  Copy the MySQL config file.

cp /etc/mysql/mysql.conf.d/mysqld.cnf ./mysqld.cnf.orig

Change the bind address of MySQL to allow external connections.

awk -F'=' -v OFS='=' '/bind-address/{$2="'0.0.0.0'";print;next}1'   mysqld.cnf.orig > mysqld.cnf

Copy the modified config file back to the MySQL config folder.

sudo cp mysqld.cnf /etc/mysql/mysql.conf.d

Restart the MySQL service to apply the changes.

sudo systemctl restart mysql.service

15. DB Server - Projectal Kafka

Download the Projectal Kafka components from the provided PROJECTAL-REPOSITORY.

wget https://PROJECTAL-REPOSITORY/projectal-kafka-jars.zip -O projectal-kafka-jars.zip

Extract the compressed archive.

unzip projectal-kafka-jars.zip

Rename the janusks-kafka jar file to be 'janusks-kafka.jar'.  Replace <version> with the version of janusks-kafka you are installing (see unzipped files).

mv projectal-kafka-jars/janusks-kafka-<version>.jar janusks-kafka.jar

Write the IP address of the server to the Kafka config file.  Replace <ip-address> with the IP address of your server.

sed -i "s/localhost/<ip-address>/g" projectal-kafka-config/application-trial.properties

Change permissions on the log files to allow the Kafka process to write to the log files.

sudo chmod -R 777 /var/log/janusks

Edit the 'janusks-kafka.service' file in a text editor and change the user name from 'ubuntu' to the Linux user account under which you are installing Projectal.  

sudo nano projectal-kafka-config/janusks-kafka.service

User=ubuntu
Group=ubuntu

Edit the 'janusks-kafka.service' file in a text editor and change the path to the 'janusks-kafka.jar' file to replace 'ubuntu' with the Linux user account  under which you are installing Projectal.

/home/ubuntu/janusks-kafka.jar

Edit the 'janusks-kafka.service' file in a text editor and replace the working directory with the full path to janusks-kafka-config.  Replace 'ubuntu' with the Linux user account  under which you are installing Projectal.

WorkingDirectory=/home/ubuntu/projectal-kafka-config/

Move the Projectal Kafka service to the system service folder

sudo mv projectal-kafka-config/janusks-kafka.service /etc/systemd/system/

Change ownership and permissions on the service file

sudo chown root:root /etc/systemd/system/janusks-kafka.service

sudo chmod 664 /etc/systemd/system/janusks-kafka.service

Enable and start the service

sudo systemctl enable janusks-kafka.service

sudo systemctl start janusks-kafka.service

16. DB Server - Uploaded User Files

If multiple API servers are being installed for your company's Projectal, then the API servers will need a common file location so that each API server has access to uploaded user files.

Ensure NFS is installed.

sudo apt-get install nfs-kernel-server

sudo ufw allow nfs

Create the directory /data/file_storage and create an entry in the '/etc/exports' file.

sudo mkdir -p /data/file_storage

sudo nano /etc/exports

Change <api-server> to the IP address (or name) of your API server and add this to the exports file.  Repeat this entry for each API server.  Ensure there is no whitespace at the end of each line to avoid errors.

/data/file_storage <api-server>(rw,no_root_squash,sync,no_subtree_check)

Tip: Ensure that the file_storage folder is only used for storing uploaded user files.  Do not store other non-Projectal related files into this folder since it is backed up by Projectal's backup procedures. 

17. API Server (x N) - Preparation

For each API server in your Projectal, open a terminal connection to the API server and make sure all software on the server is up to date.

sudo apt-get update

sudo apt-get upgrade -y

18. API Server (x N) - Java

The Projectal components require version 8 of the open JDK to be installed.

sudo apt install openjdk-8-jdk -y

19. API Server (x N) - Uploaded User Files

For each API server,  install NFS.

sudo apt-get install nfs-common

On each API server, create the 'data/file_storage' folder.

sudo mkdir -p /data/file_storage

Change the ownership of the '/data/file_storage' folder.  Replace <user> with the Linux use name you are installing Projectal under.

sudo chown -R <user>:<user> /data

On each API server, create the '/var/log/janusks' folder

sudo mkdir /var/log/janusks

sudo chmod 777 /var/log/janusks

Mount the data file_storage folder.  Replace <db-server> with the IP address (or name) of your Projectal DB server.

sudo mount -t nfs <db-server>:/data/file_storage /data/file_storage

Tip: Ensure that the file_storage folder is only used for storing uploaded user files.  Do not store other non-Projectal related files into this folder since it is backed up by Projectal's backup procedures. 

20. API Server (x N) - Projectal API

For each API server, download the 'projectal-api config' compressed archive from the provided PROJECTAL-REPOSITORY.

wget https://PROJECTAL-REPOSITORY/projectal-api-config.zip -O projectal-api-config.zip

Install the unzip tool.

sudo apt-get install zip unzip -y

Unzip the compressed archive.

unzip projectal-api-config.zip

Create a '/data/projectal' folder for Projectal.

sudo mkdir -p /data/projectal

Set the IP address of the Projectal DB server.

sed -i "s/db_host/<db-server>/g" projectal-api-config/application.properties

Set the license key for Projectal that has been provided to you by the Projectal Support team.

sed -i "s/license_key/PROJECTAL-LICENSE-KEY/g" projectal-api-config/application.properties

Move the files to the '/data/projectal' folder and set the permissions.

sudo mv projectal-api-config/* /data/projectal

sudo chmod 755 /data/projectal/*

Set the version in the 'applicationstatus.properties' file.   Replace <version> with the version of Projectal you are installing.

sudo sed -i "s/projectal_version/<version>/g" /data/projectal/applicationstatus.properties

Write a random value into the Projectal service file for a unique ID for the API server.

sudo sed -i "s/random_value/$RANDOM/g" /data/projectal/projectal-web-api.service

Edit the 'projectal-web-api.service' file and change the user name from 'ubuntu' to the Linux user name you are installing Projectal under.

sudo nano /data/projectal/projectal-web-api.service

User=ubuntu
Group=ubuntu

Download the Projectal API compressed archive from the provided PROJECTAL-REPOSITORY.

wget https://PROJECTAL-REPOSITORY/projectal-web-api-jar.zip -O projectal-web-api-jar.zip

Unzip the compressed archive.

unzip projectal-web-api-jar.zip

Move the Projectal jar file to the '/data/projectal' folder.   Replace <version> with the version of Projectal you are are installing.

sudo mv projectal-web-api-jar/janusks-web-api-<version>.jar /data/projectal/janusks-web-api.jar

Move the service file to the system service folder.

sudo mv /data/projectal/projectal-web-api.service /etc/systemd/system/

Change ownership and permissions on the file and enable then start the service.

sudo chown root:root /etc/systemd/system/projectal-web-api.service

sudo chmod 664 /etc/systemd/system/projectal-web-api.service

sudo systemctl enable projectal-web-api.service

sudo systemctl start projectal-web-api.service

Allow port 8443 through the firewall.

sudo ufw allow 8443

Tip: Your Projectal should now be available in a web browser at https://<ip-address>:8443, where <ip-address> is the IP address of the API server. 

21. API Server (x N) - Email

Projectal sends emails when activating new users and when resetting user passwords. 

Projectal will need access to an SMTP mail server for sending these emails. These email settings are configured in the application properties file on each API server.

sudo nano /data/projectal/application.properties

# Mail
 smtp.enabled=true
 spring.mail.host=
 spring.mail.port=
 spring.mail.username=
 spring.mail.password=

If your company does not have access to an SMTP mail server or has a security policy to not use SMTP mail servers, then you can configure Projectal to not use email in its workflow when creating new user accounts or when resetting user passwords.

sudo nano /data/projectal/application.properties

# Mail
 smtp.enabled=false
 spring.mail.host=
 spring.mail.port=
 spring.mail.username=
 spring.mail.password=

In this case, when the SMTP mail server setting is turned off, Projectal will prompt for a password immediately after creating a new account.  The user account will not be activated until a password is provided.  And, when resetting a user's password, Projectal will prompt immediately for a new password.

22. DB Server - Backups

It is important for on-premise installations to have a backup strategy so that databases and files containing important information in Projectal are backed up on a regular basis.

It is recommended to have a backup retention policy that runs a daily backup with a 30 day cycle.

Scripts are provided that perform these daily backups and retain these daily backups for 30 days by deleting the oldest backup each day.  Backups are stored at your nominated storage location.  

It is recommended to have 1TB storage space be allocated for backup purposes for typical Projectal usage.

To create backups of Elasticsearch, a directory must be registered in its config file. Create the directory in the backup location (change the path to your preferred backup location).

mkdir -p /home/ubuntu/janusks/backup/elasticsearch

chmod 644 /home/ubuntu/janusks/backup/elasticsearch

Write the backup location into the Elasticsearch config file.

echo 'path.repo: ["/home/ubuntu/janusks/backup/elasticsearch"]' |  sudo tee -a /etc/elasticsearch/elasticsearch.yml

Restart the Elasticsearch service to apply the changes.

sudo systemctl restart elasticsearch

Download the Projectal backup scripts from the provided PROJECTAL-REPOSITORY.

wget https://PROJECTAL-REPOSITORY/projectal-backup.zip -O projectal-backup.zip

Extract the backup scripts from the compressed archive.

unzip projectal-backup.zip

Edit the 'backup.sh' file to replace the value for USER= to the Linux user account you want to create the backups.  You can create a specific user account for backups such as projectal_backup or use an existing account.

sudo nano projectal-backup/backup.sh

Create a cron job to run the backup daily.  The backup script will remove backups older than 30 days.

crontab -e

Copy and paste the line below to run a cron job at 3:00PM each day.  Replace <path-to-backup> with the path to the 'backup.sh' file.

00 15 * * * <path-to-backup>/backup.sh

Tip: Do not set the backup folder location to store daily Projectal backups to be the file_storage folder, since this folder is intended for uploaded user files and is also included in the backup script. 

23. Load Balancer - Preparation

If you have multiple API servers in your Projectal, then you will need to have a load balancer to handle multiple incoming user requests to your Projectal.  These steps assume you are using HAProxy, which is our recommended load balancer.

Open a terminal connection to the load balancer server and make sure all software on the server is up to date.

sudo apt-get update

sudo apt-get upgrade -y

24. Load Balancer - Configuration

To install HAProxy on Ubuntu run the following command.

sudo sudo apt install haproxy

Once installation is complete, you can confirm the installation by running the following command.

haproxy -v

You should see the HAProxy version displayed.

Note: You can skip steps 25, 26 and 27 if you have an SSL certificate already prepared.

25. Load Balancer - Certificate Signing Request (CSR)

Using a text editor, create a file called 'config_ssl.cnf' and insert this text into it.  Change the country, state, locality, organization, email address and domain to match your company's details. Note: If copying the following text, then ensure that there are no spaces on empty lines.

[ req ]
default_bits = 2048

prompt = no
distinguished_name=req_distinguished_name
req_extensions = v3_req

[ req_distinguished_name ]
countryName=US
stateOrProvinceName=NY
localityName=New York
organizationName=Example LLC
organizationalUnitName=Example IT Department
commonName=example.localhost
emailAddress=email@example.com

[ alternate_names ]
DNS.1 = example.com
DNS.2 = www.example.com
DNS.5 = *.example.com

[ v3_req ]
keyUsage=digitalSignature
basicConstraints=CA:false
subjectAltName = @alternate_names
subjectKeyIdentifier = hash

Run the following command to create a Projectal CSR file.

openssl genrsa -out projectal.key 2048

openssl req -new -sha256 -key projectal.key -config config_ssl.cnf -out projectal.csr

26. Load Balancer - Certificate Authority

If the IT infrastructure where Projectal is installed is locked down by your company for security and privacy reasons, and it does not allow external access to the Internet, then it may not be possible to use an external Certificate Authority (CA) for signing SSL certificates.  In this case, you will need to create a CA on your load balancer to sign the SSL certificate so that your company staff do not get warnings displayed about Projectal being unsecured in their web browsers.

Tip: If the IT infrastructure where Projectal is installed does have access to an external Certificate Authority (CA), then you do not need to create a CA and can skip this step.  Instead, you should proceed to create a CRT using your external CA.

Using a text editor, create a file called 'config_ssl_ca.cnf'.  Change the country, state, locality, organization, email address and domain to match your company's details. Note: If copying the following text ensure that there are no spaces on empty lines.

[ req ]
default_bits = 2048

prompt = no
distinguished_name=req_distinguished_name
req_extensions = v3_req

[ req_distinguished_name ]
countryName=US
stateOrProvinceName=NY
localityName=New York
organizationName=Example(localhost)
organizationalUnitName=Example IT Department
commonName=example.localhost
emailAddress=root_email@root.localhost

[ alternate_names ]
DNS.1 = example.localhost
DNS.2 = www.example.localhost
DNS.3 = mail.example.localhost
DNS.4 = ftp.example.localhost
DNS.5 = *.example.localhost

[ v3_req ]
keyUsage=digitalSignature
basicConstraints=CA:true
subjectKeyIdentifier = hash
subjectAltName = @alternate_names

Run the command to create your Root key and certificates

openssl genrsa -out ca.key 2048

openssl req -new -x509 -key ca.key -out ca.crt -days 365 -config config_ssl_ca.cnf

27. Load Balancer - Signing Your Certificate

Using the .CRT created using your external CA or using the CA created on your load balancer (based on previous step), then make sure the .CRT file is located in the current folder and is called 'ca.crt'.

The 'ca.srl' text file contains the next serial number to use in hex. This is mandatory. This file must be present and contain a valid serial number.  Run the following commands to create a serial number for your certificate.

echo 00 > ca.srl

touch index.txt

Using a text editor, create a file called 'config_ca.cnf' and insert this text into it.  Replace <domain.com> with your intranet domain.  Note: If copying the following text ensure that there are no spaces on empty lines.

# we use 'ca' as the default section because we're using the ca command
[ ca ]
default_ca = my_ca

[ my_ca ]
#  a text file containing the next serial number to use in hex. Mandatory.
#  This file must be present and contain a valid serial number.
serial = ./ca.srl

# the text database file to use. Mandatory. This file must be present though
# initially it will be empty.
database = ./index.txt

# specifies the directory where new certificates will be placed. Mandatory.
new_certs_dir = ./

# the file containing the CA certificate. Mandatory
certificate = ./ca.crt

# the file contaning the CA private key. Mandatory
private_key = ./ca.key

# the message digest algorithm. Remember to not use MD5
default_md = sha256

# for how many days will the signed certificate be valid
default_days = 365

# a section with a set of variables corresponding to DN fields
policy = my_policy

# MOST IMPORTANT PART OF THIS CONFIG
copy_extensions = copy

[ my_policy ]
# if the value is "match" then the field value must match the same field in the
# CA certificate. If the value is "supplied" then it must be present.
# Optional means it may be present. Any fields not mentioned are silently
# deleted.
countryName = match
stateOrProvinceName = supplied
organizationName = supplied
commonName = <domain.com>
organizationalUnitName = optional
commonName = supplied

Using the CSR file previously created, sign the certificate using the openssl command.

openssl ca -config config_ca.cnf -out projectal.crt -in projectal.csr

After signing the certificate, a .pem file needs to be created for HAProxy. 

bash -c 'cat projectal.key projectal.crt >> prj.pem'

Move the .pem file to the SSL folder and restart HAProxy so it uses the new certificate:

sudo mkdir /etc/ssl/prj

sudo mv prj.pem /etc/ssl/prj/

28. Load Balancer - Final Configuration

Edit the HAProxy config file 'haproxy.cfg'.

sudo nano /etc/haproxy/haproxy.cfg

Add the following text to the end of the config file.  Replace <lb-ip-address> with the IP address of the server running your load balancer.  Replace <domain.pem> with the file name of your SSL certificate. Replace <api-ip-address> with the IP addresses of your set of API servers. 

frontend web-frontend
   bind <lb-ip-address>:443 ssl crt /etc/ssl/prj/<domain.pem>
   mode http
   default_backend web-backend
backend web-backend
   balance roundrobin
   server api-srv-01 <api-ip-address-1>:8443
   server api-srv-02 <api-ip-address-2>:8443
   server api-srv-03 <api-ip-address-3>:8443

Save the config file and then restart HSProxy.

sudo systemctl restart haproxy

In order for Projectal users to see Projectal as trusted and not see an error about an invalid certificate authority in their web browsers, the ROOT CA cert created will need to be imported into the domain's Trusted Root Certification Authorities, or imported into the Trusted Root Certification Authorities within each web browser that is going to use Projectal.  In the latter case, go to the Certificate Settings section in your web browser to import the ROOT CA.

29. API Server (x N) - SSL

For each API server, either SSL Termination must be used or each API server must operate with SSL enabled.

If SSL Termination is used, then SSL can be disabled in the application properties file on each API server.

sudo nano /data/projectal/application.properties

server.ssl.enabled=false

If SSL Termination is not used, then an SSL certificate must be stored in a keystore file.  It can be created using the following commands.

openssl pkcs12 -export -out projectal-keystore.p12 -inkey project_private.key -in projectal_com.crt -certfile projectal_com_interCA.crt

projectal_private.key > private key

projectal_com.crt > certificate body

projectal_com_interCA.crt > Intermediate certificate

Use password: janusks

The path to the keystore and the password for the keystore is configured in the application properties file on each API server.

sudo nano /data/projectal/application.properties

You will also need to import the certificate using the Java cacert keytool command on each API server.

sudo keytool -import -noprompt -trustcacerts -alias 1 -file /data/projectal/projectal-com.pem -keystore /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/security/cacerts -storepass changeit 

30. API Server (x N) & DB Server - Time Synchronization

Each API server and the DB server must have the time set to UTC.

sudo timedatectl set-timezone UTC

The time on each API server must be synchronized with the time on the DB server.   This is done using chrony.  On each API server and the DB server you will need to install chrony.  This will get the current time from Internet-based NTP servers.  Note: This assumes your IT infrastructure has access to the Internet.

sudo apt install chrony

If your IT infrastructure is locked down and does not have Internet access, then you will need to run an NTP server on the DB server.  On the DB server, edit the '/etc/chrony/chrony.conf' file and add the following.  Replace 192.168.0.0 with your subnet.

# Allow NTP client access from local network.
allow 192.168.0.0/16
 
# Serve time even if not synchronized to a time source.
local stratum 10

Save the file and restart chrony.

sudo systemctl restart chrony

On each API server, run the following commands.  Replace <db-server-ip> with the IP address of the DB server.

sudo sed -i '1 i\server <db-server-ip> prefer iburst minpoll 4 maxpoll 4' /etc/chrony/chrony.conf

sudo systemctl restart chrony

Test the time synchronization for each API server with the following command.  Replace <db-server-ip> with the IP address of the db server.

sudo chronyd -q 'server <db-server-ip> iburst'

Example output:
2023-04-27T02:54:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC +PRIVDROP +SCFILTER +SIGND +ASYNCDNS +SECHASH +IPV6 -DEBUG)
2023-04-27T02:54:29Z Initial frequency -24.714 ppm
2023-04-27T02:54:33Z System clock wrong by 0.000162 seconds (step)
2023-04-27T02:54:33Z chronyd exiting

Run the following command on each API server to make sure they are updating from the correct server.

chronyc tracking

It should output similar to the following. 

Reference ID    : 0ADF59DC (db-prj-01)
Stratum         : 12
Ref time (UTC)  : Mon Dec 12 22:45:17 2022
System time     : 0.000000005 seconds fast of NTP time
Last offset     : -0.000083039 seconds
RMS offset      : 0.000083039 seconds
Frequency       : 37.959 ppm slow
Residual freq   : -12.043 ppm
Skew            : 0.005 ppm
Root delay      : 0.000350792 seconds
Root dispersion : 0.536345661 seconds
Update interval : 5.1 seconds
Leap status     : Normal

31. External URL Access

If your company's firewall is configured to have a discrete set of URLs and IP addresses that are only allowed external access, then the firewall will need to allow external access to the following URLs for Projectal to operate in an on-premises installation.

32. Health Check

To verify that Projectal is running correctly, go to the IP address (or domain name) of the load balancer in a web browser and see if the Projectal login screen appears.   

Try logging into Projectal using the user login account you have created.  

Important: The initial password for your login account is Disclaimer!Cva2021

Tip: On the Login screen, click Forgot Password to reset your login account's password.

If login is successful, then enter the path in your web browser.  Replace <projectal-url> with the web address of your Projectal.

<projectal-url>/management/health 

This displays JSON showing the status of various components in Projectal.  For example, you can see whether Projectal can connect to MySQL and also whether mail is available.

The logs for each API server are located at /var/log/janusks/janusks-web-api.log.  Look for any error messages in the log files to verify if Projectal is running correctly.

Check the status of the service running on each API server.

sudo systemctl status projectal-web-api

On the DB server, the logs are located at /var/log/janusks/janusks-gremlin.log.  Look for any error messages in the log files to verify that Janusgraph is running correctly.

Check the status of each of the services running on the DB server.

sudo systemctl status cassandra

sudo systemctl status elasticsearch

sudo systemctl status kafka

sudo systemctl status janusgraph-websoc

Tip: If you have any errors or suspect Projectal is not operating correctly, then please contact the Projectal Support team immediately and they will be able to provide diagnostic advice.

33. Restarting Projectal Services

On each API server, to restart the Projectal web api service, enter this command.

sudo systemctl restart projectal-web-api

To restart the services on the DB server, enter these commands.  Note: make sure to follow this order.

sudo systemctl restart cassandra

sudo systemctl restart elasticsearch

sudo systemctl restart janusgraph-websoc

34. Exporting the Projectal Database as a JSON file

When applying updates or making system changes to Projectal, it is recommended to export your Projectal database as a JSON file so that your database can be easily recovered if anything goes wrong.  Replace <path-to-export-json> with the path where you want the export file to be written.  The folder must exist where the export file is to be written.

java -cp ./projectal-janusgraph-jars/janusks-schema-<version>.jar com.janusks.schema.SchemaManager --conf default --export <path-to-export-json> --note brief --overwrite

Note: The export will create two files - a .json file and a .var file.  Both files are required to import the database.

35. Importing a JSON Projectal Database

If you have a problem during update or you need to revert to a previously exported database, you can import the JSON file with the following command.  Replace <path-to-export-json> with the path of your JSON file.

java -cp ./projectal-janusgraph-jars/janusks-schema-<version>.jar com.janusks.schema.SchemaManager --conf default --drop entries --import <path-to-export-json> --note brief --overwrite

After importing has completed, the services on the DB server need to be restarted.

sudo systemctl restart cassandra

sudo systemctl restart elasticsearch

sudo systemctl restart janusgraph-websoc

On each API server, the Projectal web api service needs to be restarted.

sudo systemctl restart projectal-web-api

36. Updating Projectal

When a new version of Projectal is released, your company will be notified so that you can update your Projectal to the latest version. 

Important: It is strongly recommended to apply all new releases to your Projectal so that you can benefit from the new features, improvements, bug fixes, security patches and performance tweaks.

When you wish to apply a new release, you will be provided you with updated jar components.   These should be extracted onto the DB server or API servers next to their corresponding jar files.  

Before applying an update to your Projectal, the recommended first step is to export your Projectal database as a JSON file so that your database can be recovered if anything goes wrong. See preceding section to do this.

If a DB update is required, you can run it using the following command.  Replace <version> with the version number of the updated jar.

java -cp ./janusks-schema-<version>.jar com.janusks.schema.SchemaManager --conf default --drop upgrade --note brief –overwrite

After running a DB upgrade, you must restart the cassandra, elasticsearch and janusgraph-websoc services on the DB server.

sudo systemctl restart cassandra

sudo systemctl restart elasticsearch

sudo systemctl restart janusgraph-websoc

After running a DB upgrade,  you must change the DB version in the 'application.properties' file on each API server.  The DB version specified in the 'application.properties' file must match the version of the DB running on the DB server.  The DB version is reported at the end of the upgrade process, or you can get it by running the following command.

java -cp ./janusks-schema-<version>.jar com.janusks.schema.SchemaManager --conf default --note brief

On each API server, replace <db-version> in the 'application.properties' file with the version reported.  Note: <db-version> should be entered as an integer.  For example, 0.56 becomes 56.

sudo nano /data/projectal/application.properties

janusgraph.schema.version=<db-version>

On each API server, unzip the updated jar component and move it to the '/data/projectal' folder.  Replace <version> with the version of the updated jar component.

sudo mv janusks-web-api-<version>.jar /data/projectal/janusks-web-api.jar

On each API server, set the version in the 'applicationstatus.properties' file. Replace <version> with the version of the updated jar component.

sudo nano /data/projectal/applicationstatus.properties

application.status.version=<version>

After copying the updated jar files to each API server, updating the DB version in the 'application.properties' file and updating the 'applicationstatus.properties' file, the Projectal web api service must be restarted on each API server.

sudo systemctl restart projectal-web-api

Note: After applying an update, Projectal may return a 503 error in your web browser. To fix this, clear the cached storage for Projectal in your web browser and then refresh the page.

37. Restoring Backups

If you wish to restore a backups of your Projectal, then the files archive must be extracted to the same location from which it was backed up ('/data/file_storage').

The backup database can then be restored using the 'restore-db.sh' script. This script must be run on the DB server with sudo so that it has privileges to modify the Cassandra folder. The script will prompt for the Cassandra folder '/var/lib/cassandra' and the commitlog folder '/var/lib/cassandra/commitlog' and the location of the archive containing the backup. The backup will then be written into the Cassandra folder to restore the snapshot of the db. Once completed, Cassandra will be restarted. You should also restart Elasticsearch and 'janusgraph-websoc' service on the DB server, followed by the projectal-web-api service on each API server.

sudo ./restore-db.sh 

To restore Elasticsearch, extract the backup to the backup folder configured in Elasticsearch (e.g. '/home/ubuntu/janusks/backup/elasticsearch').  Delete the existing indexes.

./restore_es.sh

To restore the user uploaded files, you can extract the files archive using the following command.

tar xvf archive.tar

Remove the existing files in the '/data/file_storage' folder on the DB server and copy the extracted files into this folder.

Restart the 'janusgraph-websoc' service then restart each of the 'projectal-web-api' services on each API server.