CiviForm Docs
HomepageAboutContact
Search
⌃K
Links

Terraform deploy system

Terraform is an infrastructure-as-code tool that allows you to define both cloud and on-prem resources in human-readable configuration files that you can version, reuse, and share. CiviForm provides Terraform configuration files that allow you to deploy CiviForm on Azure and AWS. Knowledge of Terraform is not required to run them, but reading the high-level Terraform overview may be useful.

Setup

Outside configuration

You will need some values that are configured outside of CiviForm before you start the setup. Some of the steps are optional, meaning that you can bring up a staging environment and get the app working without them, but they will need to be completed for production setup.

Steps to run

  1. 1.
    Fork the civiform-deploy repo to your organization via the GitHub webpage.
  2. 2.
    Clone the repo onto the machine you are deploying from. Ideally, this would be a shared instance that multiple people can log onto.
  3. 3.
    Find the version that you want to deploy on Github.
  4. 4.
    Copy the civiform_config.example.sh into civiform_config.sh and fill out the missing values. You can get a sense of required values depending on your cloud provider by looking at staging-azure or staging-aws configs.
  5. 5.
    Run bin/doctor and install the dependencies.
  6. 6.
    Run bin/setup. What to expect:
    • Takes 5-10 minutes to run.
    • Terraform brings up resources in the cloud (database, network, server, etc).
    • Asks confirmation a few times before creating resources, listing everything that will be created.
    • Safe to re-run the script if it fails. There is a known issue where bin/setup fails on the first run.
    • The configuration values in civiform_config.sh represent the desired state of your CiviForm deployment. The bin/setup and bin/deploy commands work to make your cloud environment match the desired state. If a command fails, your cloud environment may not match the desired state. These commands are safe to retry if they fail. If a command is persistently failing, you can work with our on-call engineer to resolve the issue. Our on-call engineer responds to new issues in the CiviForm issue tracker.

Deploy

  1. 1.
    Find the version that you want to deploy on Github.
  2. 2.
    Update the CIVIFORM_VERSION value in civiform_config.sh.
  3. 3.
    Run bin/deploy.

Troubleshooting

Terraform fails with error "Provider produced inconsistent final plan"

This error can happen when running bin/setup for the first time. If you see it, re-run bin/setup. This is a known Terraform bug.

Terraform fails with other errors

The deploy command is idempotent, so if it fails, try running it again. The setup command can also be re-run, but it resets a lot of variables which are kind of a pain to continually set up when you run it for Azure.
If changes were made upstream, you can change the code in the checkout env, but will need to commit PRs to fix in the main repo.

No such file or directory

If you see error like "no such file or directory"
./db-connection: line 2: cloud/azure/bin/lib.sh: No such file or directory
./db-connection: line 21: out::error: command not found
The scripts expect you to be in specific directories. You probably need to cd into the checkout directory or the top level directory. If you are running setup/deploy/revert, you will need to be in the top level directory. If you are running a script like db-connection, you need to be in the checkout directory.

Terraform fails with Error acquiring the state lock

This situation can happen when exiting deployment scripts using "Ctrl-C". Terraform acquires a lock every time you run ./bin/deploy or ./bin/setup, and releases the lock at the end of the script. This helps to prevent concurrent infrastructure changes. If the deploy process exited outside of Terraform, the lock remains and needs to be force removed in order to run deploy again. To do it, run the following command (assuming you deploy on AWS):
terraform -chdir=checkout/cloud/aws/templates/aws_oidc force-unlock $LOCK_ID

Helpful resources

Requesting AWS certificate

Follow official documentation. After your certificate is ready and validated, copy the ARN from your console and put it into the SSL_CERTIFICATE_ARN variable in your civiform_config.sh
ARN has format of arn:aws:acm::<user_id>:certificate/<identifier>
Screen Shot 2022-08-10 at 3 50 39 PM

Setting up Azure AD

The setup script prompts you to set up Azure AD. There are a few additional steps.
In Azure Portal:
  • Go to Azure Active Directory -> App registrations
  • Click the tab for "All applications"
  • Find your app (staging-dynamic-heron for CiviForm Azure Staging)
    image
    ​
Use the menu on the left:
  • Authentication: setup the Redirect URI to be what the app expects: https://<custom_hostname>/callback/AdClient.
  • You will also need an admin group which creates CiviForm admins
  • Token configuration: To allow for CiviForm admins, you need to have Azure AD return the groups claim. Add the security groups claim (you can verify the groups claim is being returned by decoding the base64 token from the token you get back from Azure AD on the website-- if you preserve the log in the Chrome Dev Tool window it should be from https://<custom_hostname>/callback/AdClient)

Access the database for emergency repair

AWS

We support on-demand deployment of the pgadmin web UI to access the CiviForm database. We require explicit IP allow-listing via a list of CIDR blocks. Only these IPs will be able to access pgadmin. The public IP of the host running the web browser used to access the pgadmin web UI (like your work laptop/desktop) must be covered by a block in the list. To detect the public IP of a host running a web browser, visit https://checkip.amazonaws.com.
  1. 1.
    Run bin/run and pass in pgadmin for the command.
  2. 2.
    The deploy tool will auto-detect the public IP of the host it is running on and ask if you would like to add the IP to the allow-list. If you want the deploy tool to wait until pgadmin is available to print out connection information, enter 'y'.
  3. 3.
    Enter in a CIDR block that covers the IP of the host that will access the pgadmin web UI. If the host's IP is '127.0.0.1', enter in '127.0.0.1/32' to allow-list just that IP.
  4. 4.
    Either accept or reject the allow-list. If you previously chose to add the deploy tool's IP to the allow-list, and then reject the list, the deploy tool's IP will not be automatically added to the list again.
  5. 5.
    After accepting a list, Terraform will run to bring up the pgadmin resources. When it asks "Do you want to perform these actions?", enter "yes".
  6. 6.
    The deploy tool will attempt to connect to pgadmin every 10 seconds. When a connection is successful, the pgadmin URL and authentication information will be printed. Press Ctrl-c to shortcut this wait.
  7. 7.
    Open the pgadmin URL. Log in using the 'login email' and 'login password' printed by the deploy tool.
  8. 8.
    Expand the 'CiviForm (1)' item in the left navigation pane. You should be prompted to enter in the password to the database. Enter in the 'database password' printed by the deploy tool.
  9. 9.
    Expand the 'Databases (2)' item under the 'CiviForm (1)' item.
  10. 10.
    The 'postgres' item under the 'Databases (2)' item is the CiviForm database. Right click on the 'postgres' item and select 'Query Tool' to send commands to the database.

Azure

  1. 1.
    cd checkout
  2. 2.
    cloud/azure/bin/db-connection -g sgdev -d civiform-artistic-halibut -v sgdev-civiform-kv

Restore data to the database from a dump file (only Azure)

  1. 1.
    If on WSL, figure out what the location of the dump file is (possibly /mnt/c/..)
  2. 2.
    cd checkout
  3. 3.
    cloud/azure/bin/pg-restore -g sgdev -d civiform-artistic-halibut -v sgdev-civiform-kv -f /mnt/c/pg\_dump.dump -b testData.dump

Clear data from the database (only Azure)

  1. 1.
    cd checkout
  2. 2.
    From checkout directory run cloud/azure/bin/db-connection -d <database_name> -g <resource_group> -v <keyvault> Note that the database_name does not include the .postgres.database
  3. 3.
    Wait for the application to say you can run sql (try running \dt to make sure you can see the data)
  4. 4.
    Run the following commands (has to be in order)
DELETE FROM applications;
DELETE FROM applicants;
DELETE FROM accounts;
DELETE FROM programs;
DELETE FROM questions;
DELETE FROM ti_organizations;
DELETE FROM versions;
DELETE FROM versions_programs;
DELETE FROM versions_questions;
Previous
Initial Deployment
Next
AWS Terraform deployment
Last modified 13d ago