​
Prerequisites

For a smooth sailing make sure you pass this checklist before you begin the installation:
  • The target machine is clean of any other software that can interfere with it. That means no docker, no kubernetes, http servers, no Nvidia drivers, etc.
  • The machine has stable and strong Internet connection
  • You can reboot the machine if needed
  • Your firewall is not blocking connections to any of these websites
  • huggingface.co
  • sentry.io (if you opt-in to send us crash reports)
  • *.grafana.net (if you opt-in to send us anonymous & aggregated observability metrics)
  • The machine has curl installed
  • You have a domain for Zylon ready (ex: zylon.your-company.com), and a valid SSL certificate for that domain
Kernel upgrades (like Ubuntu unattended upgrades) will break Nvidia drivers on every change!.To test if the drivers are working use nvidia-smi utility, it will display the status of your GPU(s) or fail if the drivers are not working:If drivers are not working properly they MUST be recompiled for the new kernel version, to do so run:
sudo zylon-cli install-drivers --force
An implicit kernel update might also happen after a restart of the machine when running in cloud providers like AWS or Azure.
Once you have fulfilled the hardware requirements and have a domain ready it is time to install Zylon! For single instance setups Zylon it’s installed using it’s CLI zylon-cli on the machine where you want to run it. To start the install process make sure that:
  • At install time, the machine has internet connection (After Zylon has been installed internet connection it is not a requirement)
  • You have ssh access to the machine
  • Firewall rules will not block the CLI to download any Zylon container images or certificate generation

​
Step 1: Install the Zylon CLI

The CLI will speed up the time it takes to get everything ready, it will take care of installing drivers, updating Zylon and generating template configuration for you. To install it, SSH into the machine where you want to install Zylon and run 
# ssh user@zylon.company.com
sudo curl -sL get.zylon.ai | sh
Root access is required at multiple steps into the installation, if you wish to inspect the script simply run 
curl -sL get.zylon.ai
Run sudo zylon-cli help to get a list of all the commands!

​
Step 2: Generate a configuration file for your Zylon installation

Zylon is configured with a yaml file which needs to be located in /etc/zylon/zylon-conf.yaml You will need to generate this file by yourself. To generate the template for the configuration file use the CLI: 
sudo zylon-cli config
It will output a template you must fill with your client information in the right location. Open your editor of choice and replace all placeholders with the right values. Some items, like your license ID, and some tokens should have been provided by the Zylon team ahead of time, just paste them in place. An example configuration file looks like the following yaml file 
# Machine configuration, this is the only supported option
node:
  distro: "ubuntu22"
  arch: "x86_64"
  drivers: "generic"

# Cluster type, this is the only
# supported option for single instance installations
cluster:
  type: "k0s"

# unstable channel also available under request
channel: "stable"

license:
  email: "champion@company.com"
  id: "2oq..." # A license id a Zylon employee shared with you via email

# AI configuration, detailed information below.
ai:
  preset: "baseline-24g"

# We will use this to identify your company in our anonymous metrics
companyIdentifier: "your-company"

# Enable anonymous metrics and crash reporting
observability:
  crashReporting: true
  usageMetrics: true

# Your network configuration, the host users will use to access Zylon
ingress:
  host: "zylon.your-company.com"
  enabled: true
  tlsEnabled: true
  certManagerAnnotations:
    # If you plan to open the machine to the internet, SSL certificate
    # will be automatically generated using letsencrypt
    cert-manager.io/cluster-issuer: "letsencrypt-prod"
For detailed Zylon instance configuration, refer to the Configuration page.

​
Step 3: Prepare the machine

In order to install Zylon the machine needs to be ready to do so, and during this step, the drivers will be installed, an embedded kubernetes cluster, and other dependencies will be setup. This process will take between 5 to 15 minutes, it’s normal to see the output freeze while some installers are running, simply wait until it’s done. 
sudo zylon-cli setup
Generally, you only need to run the setup once. Running it multiple times has no effect unless an update by Zylon team added a new dependency.
It’s highly recommended to reboot the machine after this step is completed to ensure GPU drivers are properly loaded.

​
Step 4: Install Zylon

This step (install / update) will make Zylon available in your machine assuming the configuration file in /etc/zylon/zylon-conf.yamlis correct. 
sudo zylon-cli update
To update Zylon to a newer version, simply run this command again. You can do so as frequently as you need. If you modify your Zylon configuration file you will also need to run the update command again.
The initial installation of Zylon will take around 15 minutes to complete after the update command itself is completed.
This is what is happening in the background:
  • Zylon images are being downloaded
  • AI models are being downloaded
  • GPU configuration is getting applied, several validations are running to ensure compatibility
  • SSL certificate is being generated
You can watch the process running. When all pods are at 1/1 ready status you can proceed. 
watch sudo k0s kubectl get pods -n zylon
# You can also watch nvidia GPU validation process
watch sudo k0s kubectl get pods -n nvidia
If some service gets stuck in a ContainerCrashLoopBackOff for more than one hour contact us, the installation has failed.

​
Step 5: Prepare your organization

To start using Zylon you will need to create your organization in it, as well as adding a root admin that will serve as your entry point for other management tasks. You will need:
  • A name for your organization (can be changed later)
  • A password for the root administrator (can be changed later).
  • Due to some encoding limitations in the CLI some special symbols in password don’t work correctly, prefer using a long alphanumeric password.
  • An email regex to allow users to join automatically. If your usual work email is name@company.com your email regex should be .*@company\.com
# Put a secure password!
sudo zylon-cli seed --org-name "My Org" --admin-password "admin" --email-regex ".*@yourcompany\.com"

# At seed time, if Zylon is not ready,
# this command will wait until it’s completed.
Running the seed command will WIPE ALL DATA. Only do this once during the Zylon lifecycle
The credentials to login to Zylon will be: 
user: admin@zylon.ai
password: the_password_you_set_up_previously
The root admin email is fixed and will always be admin@zylon.ai

​
Step 6 (optional): Manually installing SSL certificates

If you opted in to use Let’s Encrypt certificates you can skip this section. In case your Zylon installation is behind a VPN / private cloud and it’s not reachable through the internet you will need to configure SSL yourself. To do that you must upload your own SSL certificates and make them available to the server where Zylon is installed.

​
Configuration file

First, REMOVE from the configuration file located in /etc/zylon/zylon-conf.yaml the following lines 
certManagerAnnotations:
  cert-manager.io/cluster-issuer: "letsencrypt-prod"
Since we are using Kubernetes we need to have the private key (*.key ) and the certificate file in two files (*.crt ) to store them as a secret.

​
I have a .pfx file (ignore otherwise)

In case you have a .pfx file you can get the desired files with the following commands: 
openssl pkcs12 -in <your-certificate.pfx> -clcerts -nokeys -out tls.crt
openssl pkcs12 -in <your-certificate.pfx> -nocerts -out tls.key

​
Create secret


# Import or refresh the certificates
sudo k0s kubectl delete secret tls zylon-tls --ignore-not-found -n zylon
sudo k0s kubectl create secret tls zylon-tls --key tls.key --cert tls.crt -n zylon

# Check if it's correctly created
sudo k0s kubectl describe secret zylon-tls -n zylon
To reload the changes done to the configuration file, restart Zylon 
sudo zylon-cli update
Finally, go to your Zylon instance and confirm that the connection is using HTTPS.