Continuous integration and deployment
This guide will help you set up continuous integration/continuous deployment (CI/CD) workflows for a DSS deployment on Google Cloud Platform (GCP).
Add the Cloud SQL Auth Proxy client to your
brew install cloud-sql-proxy
Check your GCP metadata:
gcloud organizations list
gcloud alpha billing accounts list
Clone the Econia repository and configure project variables using the values from above:
git clone https://github.com/econia-labs/econia.git
cp runner/template.tfvars runner/terraform.tfvars
Initialize the CI/CD project runner, which on startup will run a script that installs dependencies:
After the runner has been created, you can pull the startup script logs via:
After 5–10 minutes, the startup script logs should show something like:
... google_metadata_script_runner: startup-script exit status 0
... google_metadata_script_runner: Finished running startup scripts.
... systemd: google-startup-scripts.service: Deactivated successfully.
... systemd: Finished google-startup-scripts.service - Google Compute Engine Startup Scripts.
... systemd: google-startup-scripts.service: Consumed 7min 17.688s CPU time.
Archive Terraform state for the runner on the runner itself, now that it has been bootstrapped:
This will upload
/econia/src/terraform/dss-ci-cd/runneron the runner. Other relevant data is stored on the runner under
- Terraform state for the DSS (generated upon deployment, the next step)
- GCP service account credentials.
terraform.tfvars, used by both the DSS and runner Terraform projects.
Now you can remotely execute commands on the runner. For example, to deploy the DSS:
source scripts/run.sh "terraform -chdir=dss apply -parallelism 50"
And, if you'd like to connect to the runner for an interactive session as root user:
Starting a new project
If you'd like to start a new project, make sure you've archived the runner state as described above. Then, before running the init-project script again, clear local project state:
This will leave behind
runner/terraform.tfvars so you can manually edit it, then initialize a new project.
After the DSS has been deployed and the runner state has been archived, you can change your
gcloud CLI config options to work on other GCP projects.
If you'd like to resume operations, or if someone else would like to get involved (and they have permissions for the relevant GCP project), then all that is required is the GCP project ID:
source scripts/engage-dss-project GCP_PROJECT_ID
This command will simply update the local
gcloud CLI config to the same project, region, and zone as the runner.
Updating the config enables local development, for example, like downloading Terraform state files to simulate a runner on the local machine in order to test out modifications to Terraform configuration files.
If you'd like to engage in this or other complex development operations, see the scripts directory.
If you'd like to redeploy your DSS after a release without having to sync data from chain tip, you can perform a "hot upgrade", which involves:
- Shutting off the aggregator and processor.
- Running the latest migrations.
- Redeploying the aggregator and processor.
To perform a hot upgrade, you'll need to pick two Git revisions (e.g. a tag like
- A revision for the DSS source code (including migrations and Docker image source).
- A revision for the Terraform project.
source scripts/hot-upgrade.sh $DSS_SOURCE_REV $TERRAFORM_PROJECT_REV