Skip to main content
Version: 4.x

Manual Deployment

Note

We recommend the Docker install for a consistent experience.

Requirements​

  • Python 3.8 or later
  • MySQL 5.7 / MariaDB 10.3 or later
  • Redis 5.0 or later
  • Git 2.17.0 or later (needed to create standard deploy requests)
  • rsync, sshfs (file distribution and pipeline data transfer), ping (monitoring ping checks)
  • A modern browser

Steps​

The steps assume CentOS 7 and the directory /data/spug; on Ubuntu / Debian only the package installation command differs.

1. Clone the repository​

git clone https://github.com/openspug/spug /data/spug
cd /data/spug
git checkout 4.0

2. Download the pre-built front-end​

Extract the bundle into spug_web (it creates the build directory):

curl -o /tmp/web_v4.0.0.tar.gz https://cdn.spug.cc/spug/web_v4.0.0.tar.gz
tar xf /tmp/web_v4.0.0.tar.gz -C /data/spug/spug_web/
Tip

You can also build it yourself with npm run build as described in Development setup.

3. Create the runtime environment​

# CentOS dependencies
yum install -y mariadb-devel python3-devel gcc openldap-devel redis nginx supervisor rsync fuse-sshfs iputils git

# Ubuntu / Debian dependencies
# apt install -y libmariadb-dev python3-dev python3-venv gcc libldap2-dev libsasl2-dev redis-server nginx supervisor rsync sshfs iputils-ping git

# virtual environment
cd /data/spug/spug_api
python3 -m venv venv
source venv/bin/activate

# Python packages
pip install -U pip setuptools -i https://pypi.tuna.tsinghua.edu.cn/simple/
pip install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple/
pip install gunicorn mysqlclient -i https://pypi.tuna.tsinghua.edu.cn/simple/

4. Configure the back-end​

The back-end uses SQLite by default; switch it to MySQL with the settings below. Using SQL Server?

Note

Create overrides.py in spug_api/spug/; it overrides the defaults when the services start. Do not edit settings.py, so that future updates apply cleanly.

spug_api/spug/overrides.py
DEBUG = False
# replace with a random string, used to sign sessions
SECRET_KEY = 'replace-me-with-a-random-string'

DATABASES = {
    'default': {
        'ATOMIC_REQUESTS': True,
        'ENGINE': 'django.db.backends.mysql',
        'NAME': 'spug',             # database name, create it with the utf8mb4 charset first
        'USER': 'spug_user',        # database user
        'PASSWORD': 'spug_passwd',  # database password
        'HOST': '127.0.0.1',        # database host
        'OPTIONS': {
            'charset': 'utf8mb4',
            'sql_mode': 'STRICT_TRANS_TABLES',
            #'unix_socket': '/opt/mysql/mysql.sock' # socket path for a local, non-default MySQL installation
        }
    }
}

5. Initialize the database​

cd /data/spug/spug_api
python manage.py updatedb

6. Create the administrator account​

python manage.py user add -u admin -p spug.cc -s -n Administrator

# -u user name
# -p password
# -s super administrator
# -n display name

7. Create the service definitions​

/etc/supervisord.d/spug.ini
[program:spug-api]
command = bash /data/spug/spug_api/tools/start-api.sh
autostart = true
stdout_logfile = /data/spug/spug_api/logs/api.log
redirect_stderr = true

[program:spug-ws]
command = bash /data/spug/spug_api/tools/start-ws.sh
autostart = true
stdout_logfile = /data/spug/spug_api/logs/ws.log
redirect_stderr = true

[program:spug-worker]
command = bash /data/spug/spug_api/tools/start-worker.sh
autostart = true
stdout_logfile = /data/spug/spug_api/logs/worker.log
redirect_stderr = true

[program:spug-monitor]
command = bash /data/spug/spug_api/tools/start-monitor.sh
autostart = true
stdout_logfile = /data/spug/spug_api/logs/monitor.log
redirect_stderr = true

[program:spug-scheduler]
command = bash /data/spug/spug_api/tools/start-scheduler.sh
autostart = true
stdout_logfile = /data/spug/spug_api/logs/scheduler.log
redirect_stderr = true

The services: spug-api is the HTTP API (gunicorn on 127.0.0.1:9001), spug-ws the WebSocket service (daphne on 127.0.0.1:9002), spug-worker runs batch executions, deployments and pipelines, and spug-monitor / spug-scheduler are the monitoring and scheduling processes. On Ubuntu / Debian the supervisor directory is /etc/supervisor/conf.d/ and files end with .conf.

8. Create the Nginx configuration​

/etc/nginx/conf.d/spug.conf
server {
        listen 80;
        server_name _;     # set your domain
        root /data/spug/spug_web/build/;
        client_max_body_size 20m;   # limits uploads in the file manager, adjust as needed

        gzip  on;
        gzip_min_length  1k;
        gzip_buffers     4 16k;
        gzip_http_version 1.1;
        gzip_comp_level 7;
        gzip_types       text/plain text/css text/javascript application/javascript application/json;
        gzip_vary on;

        location ^~ /api/ {
                rewrite ^/api(.*) $1 break;
                proxy_pass http://127.0.0.1:9001;
                proxy_read_timeout 180s;
                proxy_redirect off;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        }

        location ^~ /api/ws/ {
                rewrite ^/api(.*) $1 break;
                proxy_pass http://127.0.0.1:9002;
                proxy_http_version 1.1;
                proxy_set_header Upgrade $http_upgrade;
                proxy_set_header Connection "Upgrade";
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        }

        location / {
                try_files $uri /index.html;
        }
}
Note

If you do not set a server_name in spug.conf, comment out or delete the default server block in /etc/nginx/nginx.conf, otherwise the Nginx welcome page is shown.

9. Start the services​

# enable on boot
systemctl enable nginx
systemctl enable redis
systemctl enable supervisord

# start
systemctl restart nginx
systemctl restart redis
systemctl restart supervisord

10. Open Spug​

Open the site in the browser and log in with the account created in step 6.

11. Security recommendations​

  • Make sure Redis listens on 127.0.0.1 only. For a password-protected Redis see Using a Redis server with a password.
  • When adding another proxy layer, forward the X-Forwarded-For header correctly; Spug uses it for security (tokens expire when the client IP changes), see Best practices.
  • Enable login MFA under Settings / Security Settings and restrict access with a firewall.