Deploy Configuration

Overview

A deploy configuration defines how an application is released in one environment. Two modes are available: Standard deploy and Custom deploy.

Standard deploy

Spug pulls the code, builds and packages it, transfers it to the hosts and switches versions, with hook scripts you can customize at every stage. It suits most Git-based applications.

Note

Standard deploy borrows ideas from the open-source project walle, thanks to its authors. If you have used walle, you can migrate to standard deploy directly.

Basic settings

• Environment: the environment this configuration applies to, e.g. test / production. Several environments let the same application use different flows.
• Target hosts: one or more hosts the configuration deploys to; a deploy request can pick a subset of them.
• Git repository: the repository URL. For private repositories click Private repository? and choose Key authentication (add the public key of Spug as a deploy key and use the ssh URL) or Account / password authentication (http/https URL).
• Deploy mode: Parallel deploys all hosts independently at the same time; Serial deploys one host after another and stops on the first failure.
• Review: when enabled, deploy requests must be approved by a user with review permission before they can be deployed.
• Notification: deploy result notifications via DingTalk, Feishu, WeCom bots or a custom Webhook; enter the bot URL.

Build settings

• File filter: Include or Exclude paths relative to the project root (one per line); only matching files are packaged or excluded, e.g. exclude .git and node_modules.
• Before checkout: runs on the server (or container) that hosts Spug in the repository directory. Avoid modifying tracked files here, otherwise the checkout may fail.
• After checkout: runs on the Spug server in the checked-out source directory. This is where builds such as npm run build or mvn package usually happen.

Deploy settings

• Deploy path: the final path of the application on the hosts. The directory must not exist beforehand; Spug creates it and manages it as a symlink to the current version. Global variables can be used, e.g. /www/$SPUG_APP_KEY.
• Storage path: the directory on the hosts that keeps historical versions, e.g. /data/repos/$SPUG_APP_KEY.
• Versions: how many versions to keep; older build records and versions are deleted to free disk space, and this also limits how far you can roll back.
• Before deploy on host: runs on the target host in the directory of the new version before it goes live, for preparation steps.
• After deploy on host: runs on the target host in the live application directory, for example to restart services.

Tip

• All four hooks can use the global variables below and the configuration center values (exposed as _SPUG_<identifier>_<key> environment variables); run env | grep SPUG to list everything available.
• The page keeps spinning after starting a background process with nohup or &?
• A deployment reports command not found?

Flow

1. Build stage (server): pull the repository → Before checkout → check out the branch / tag / commit → After checkout → package according to the file filter; the artifact is recorded in the build repository.
2. Deploy stage (hosts): remove versions beyond the configured count → transfer the artifact to the storage path → Before deploy on host → switch the deploy path symlink to the new version → After deploy on host.

One artifact can be released several times (for example to test hosts first and to production later); choosing a Build repository version in the deploy request skips the build stage.

Custom deploy

In this mode Spug simply runs the recorded actions in order and does not care where the code comes from. It suits unusual flows or artifacts produced by an external CI system.

Fields

• Environment / Target hosts / Deploy mode / Review / Notification: same as standard deploy.
• Local actions: any number of actions executed on the Spug server, always before the host actions.
• Host actions: any number of actions executed on the target hosts in order. A host action is either Execute command or Data transfer.

Data transfer

A data transfer action copies files from the container or server running Spug to the target hosts:

• Data source
  • Local path: a file or directory on the Spug server, as an absolute path.
  • Upload on deploy: the file is uploaded when the deploy request is created.
• Filter: only for local paths; Include and Exclude rules with paths relative to the local path, several paths separated by commas. Ignored when the source is a file.
• Target path: for uploads the target must be a file path (e.g. /tmp/upload.tar.gz); for local paths the type must match the source (a file source needs a file target, a directory source a directory target), see the examples below.

For uploads note the following:

Note

1. The target can be a file path such as /data/upload.jar or an existing directory on the host; in the latter case the file keeps its original name inside that directory. In most cases another action is needed to process the uploaded file.
2. When the target is a file path that does not exist yet, Spug creates the parent directory (/data in the example above) automatically.
3. The action overwrites an existing file every time it runs.

The examples below use the source code of Spug at /data/spug to illustrate local-path sources.

• Local path /data/spug, target /www/spug, filter off

  Result

  The whole /data/spug directory is copied to /www/spug on the hosts. If /www/spug already exists there, it is deleted first.

• Local path /data/spug, target /www/spug, filter Include spug_api,spug_web/dist

  Result

  Only spug_api and spug_web/dist are copied, so /www/spug on the hosts contains just these two directories afterwards. Exclude works the same way but copies everything except the listed paths.

• Local path /data/spug/README.md, target /www/spug/README.md, filter Include spug_api

  Result

  Only README.md is copied; the filter is ignored because the source is a file.

• Local path /data/spug/README.md, target /www/spug, filter off

  Result

  README.md replaces /www/spug, which becomes a file with the content of README.md. This is a counter-example: when copying a file, give the target a full file path.

• Local path /data/spug, target /www/spug/README.md, filter off

  Result

  The whole /data/spug directory is copied and named README.md, so /www/spug/README.md becomes a directory. Another counter-example: when copying a directory, give the target a directory path.

Global variables

• SPUG_APP_NAME application name
• SPUG_APP_KEY application identifier
• SPUG_APP_ID application ID
• SPUG_REQUEST_ID deploy request ID
• SPUG_REQUEST_NAME deploy request title
• SPUG_VERSION deploy request version
• SPUG_BUILD_VERSION internal version number of the request
• SPUG_ENV_ID environment ID
• SPUG_ENV_KEY environment identifier
• SPUG_DEPLOY_ID deploy configuration ID
• SPUG_DEPLOY_TYPE deploy type ("1" normal, "2" rollback, "3" webhook-triggered)
• SPUG_API_TOKEN token for reading the configuration center
• SPUG_HOST_ID ID of the current host (host stage only)
• SPUG_HOST_NAME IP / domain of the current host (host stage only)

Standard deploy variables

• SPUG_REPOS_DIR source storage directory, injected for standard and custom deploys ($SPUG_REPOS_DIR/$SPUG_DEPLOY_ID is the source directory of this deploy)
• SPUG_BUILD_ID ID of the build record used by this deploy
• SPUG_DST_DIR deploy path on the target host
• SPUG_GIT_BRANCH selected Git branch (branch-based deploys)
• SPUG_GIT_COMMIT_ID selected Git commit (branch-based deploys)
• SPUG_GIT_TAG selected Git tag (tag-based deploys)

Custom deploy variables

• SPUG_RELEASE the SPUG_RELEASE value entered in the deploy request

  Tip

  SPUG_RELEASE is split on spaces into numbered variables; abc 123 def yields four variables:

  SPUG_RELEASE = abc 123 def
  SPUG_RELEASE_1 = abc
  SPUG_RELEASE_2 = 123
  SPUG_RELEASE_3 = def

Configuration center variables

The configuration of the application in the deploy environment (including dependent applications and services) is injected as _SPUG_ plus the upper-cased key, e.g. api_order_db_host becomes _SPUG_API_ORDER_DB_HOST. See Configuration API.

Passing variables between stages and updating configuration

Two special environment variables exported by server-side scripts (Before / After checkout or local actions of a custom deploy) are captured by Spug:

• SPUG_GEV_*: variables with the SPUG_GEV_ prefix are passed on to the host stage, so values computed during the build (an image tag, for instance) reach the host scripts:

  # After checkout
  export SPUG_GEV_IMAGE_TAG=$(git rev-parse --short HEAD)

  # After deploy on host
  echo "deploying image $SPUG_GEV_IMAGE_TAG"

• SPUG_SET: in the form app-or-service-identifier:environment-identifier:key=value, creates or updates a configuration center value during the deploy; the change is recorded in the configuration history:

  export SPUG_SET="api_order:prod:release_version=$SPUG_VERSION"