CiviForm Docs
HomeAboutContactNewsFAQ
  • CiviForm Docs
  • Overview
    • What is CiviForm?
    • How does CiviForm work?
    • Glossary
  • User Manual
    • CiviForm Admin Guide
      • CiviForm Admin training overview
      • How to navigate CiviForm
      • Working with programs
        • Create a program
        • Edit a program
        • Show or hide questions based on inputs
        • Manage program eligibility
        • Manage address & service area validation
        • Manage notifications
        • How to publish programs
        • Set a pre-screener
      • Working with questions
        • Manage questions
        • Question export settings
        • Universal and Primary Applicant Information questions
        • Using enumerator questions & screens in a program
      • Manage translations for programs & questions
      • Manage versions for programs & questions
      • Working with applications
        • Add statuses to a program
        • Download exported data
      • Role management
        • Manage Program Admins
        • Manage Trusted Intermediaries
      • Manage API keys
      • Using Markdown
      • Migrating programs between environments
    • Program Admin Guide
      • How to become a Program Admin
      • Review completed applications
    • Trusted Intermediary Guide
      • Apply to a program
    • Onboarding Guide
      • Organization assessment
      • Program assessment
      • Getting started with service design
      • Journey mapping
      • Discovery, eligibility, and intake
      • Consolidating questions across programs
      • Working with existing tools and processes
      • Working across jurisdictions
      • Data reporting and other integrations
      • Security and privacy considerations
      • Staffing overview
  • IT Manual
    • Technical Deployment Guide
      • Initial Deployment
        • Terraform deploy system
          • AWS Terraform deployment
        • Authentication setup
        • Email configuration
        • GIS Service configuration
      • Upgrading to a New Release
        • CiviForm server environment variables
          • v1.20.0
          • v1.20.1
          • v1.21.0
          • v1.22.0
          • v1.23.0
          • v1.23.1
          • v1.24.0
          • v1.24.1
          • v1.24.2
          • v1.25.0
          • v1.26.0
          • v1.27.0
          • v1.28.0
          • v1.29.0
          • v1.30.0
          • v1.30.1
          • v1.31.0
          • v1.33.0
          • v1.34.0
          • v1.34.1
          • v1.34.2
          • v1.35.0
          • v1.36.0
          • v1.37.0
          • v1.38.0
          • v1.38.1
          • v1.38.2
          • v1.39.0
          • v1.40.0
          • v1.41.0
          • v1.42.0
          • v1.43.0
          • v1.44.0
          • v1.45.0
          • v1.46.0
          • v1.47.0
          • v1.48.0
          • v1.49.0
          • v1.50.0
          • v1.51.0
          • v1.52.0
          • v1.53.0
          • v1.54.0
          • v1.55.0
          • v1.56.0
          • v1.56.1
          • v1.57.0
          • v1.58.0
          • v1.59.0
          • v1.60.0
          • v1.61.0
          • v1.62.0
          • v1.63.0
          • v2.0.0
          • v2.0.1
          • v2.0.2
          • v2.1.0
          • v2.10.0
          • v2.11.0
          • v2.12.0
          • v2.13.0
          • v2.14.0
          • v2.15.0
          • v2.16.0
          • v2.17.0
          • v2.18.0
          • v2.19.0
          • v2.2.0
          • v2.20.0
          • v2.21.0
          • v2.22.0
          • v2.23.0
          • v2.24.0
          • v2.25.0
          • v2.26.0
          • v2.27.0
          • v2.28.0
          • v2.29.0
          • v2.3.0
          • v2.30.0
          • v2.31.0
          • v2.32.0
          • v2.33.0
          • v2.34.0
          • v2.35.0
          • v2.36.0
          • v2.37.0
          • v2.38.0
          • v2.39.0
          • v2.4.0
          • v2.4.1
          • v2.4.2
          • v2.4.3
          • v2.5.0
          • v2.6.0
          • v2.7.0
          • v2.8.0
          • v2.9.0
      • Monitoring
      • Troubleshooting Production
      • Disaster Recovery
      • Database Disaster Recovery
      • Production Database Access
    • Infrastructure Requirements
    • Existing deployments
    • API Integration
      • Authentication
      • List applications
    • Testing & QA
      • Testing resources
      • SQL queries to look for missing questions
  • Governance & Management
    • Project Management
      • On Call Guide
    • Governance
      • Roles, Committees, & Responsibilities
      • Governance Processes
      • Development Principles
      • Communication
Powered by GitBook
On this page
  • Setup
  • Outside configuration
  • Steps to run
  • Deploy
  • Rotating the application secret
  • Troubleshooting
  • Helpful resources
  • Requesting AWS certificate
  • AWS deployment setup options
  • Setting up Azure AD
  • Access the database for emergency repair

Was this helpful?

Edit on GitHub
Export as PDF
  1. IT Manual
  2. Technical Deployment Guide
  3. Initial Deployment

Terraform deploy system

PreviousInitial DeploymentNextAWS Terraform deployment

Last updated 1 year ago

Was this helpful?

Terraform is an infrastructure-as-code tool that allows you to define both cloud and on-prem resources in human-readable configuration files that you can version, reuse, and share. CiviForm provides Terraform configuration files that allow you to deploy CiviForm on AWS. Knowledge of Terraform is not required to run them, but reading the high-level may be useful.

More detailed information on our deploy system can be found in our .

Setup

Outside configuration

You will need some values that are configured outside of CiviForm before you start the setup. Some of the steps are optional, meaning that you can bring up a staging environment and get the app working without them, but they will need to be completed for production setup.

  • (Optional) Admin auth client_id, client_secret, and discovery_uri. See

  • (Optional) Applicant auth client_id, client_secret, and discovery_uri. See setting up the

  • Domain name for your deployment. For example civiform.mycity.gov

  • (AWS) ARN of an SSL certificate for load balancer. See

  • (AWS) Decision around where deployments will live. See

Steps to run

  1. Fork the repo to your organization via the GitHub webpage.

  2. Clone the repo onto the machine you are deploying from. Ideally, this would be a shared instance that multiple people can log onto.

  3. Find the version that you want to deploy on .

  4. Copy the into civiform_config.sh and fill out the missing values. You can get a sense of required values depending on your cloud provider by looking at configs.

  5. Run bin/doctor and install the dependencies.

  6. Run bin/setup. What to expect:

    • Takes up to 20 minutes to run. Make sure you have time to allow the script to run to completion to avoid errors.

    • Terraform brings up resources in the cloud (database, network, server, etc).

    • Asks confirmation a few times before creating resources, listing everything that will be created.

    • Safe to re-run the script if it fails (re-runs will take longer because resources must be destroyed before being re-created).

    • The configuration values in civiform_config.sh represent the desired state of your CiviForm deployment. The bin/setup and bin/deploy commands work to make your cloud environment match the desired state. If a command fails, your cloud environment may not match the desired state. These commands are safe to retry if they fail. If a command is persistently failing, you can work with our on-call engineer to resolve the issue. Our on-call engineer .

Note: If you are running bin/doctor or another command for a config file other than civiform_config.sh, you can specify that with the config flag (i.e. bin/doctor --config=civiform_staging_config.sh)

Deploy

  1. Update the CIVIFORM_VERSION value in civiform_config.sh.

  2. Run bin/deploy.

Rotating the application secret

IMPORTANT: When the secret is rotated, all user sessions will be invalidated. This means that any guest users in the middle of an application that have not submitted it yet will lose that application, and any logged in users or admins will get logged out. It is recommended that this rotation happen at a low traffic time of day. You may also want to give users a warning before it happens.

To rotate the secret, run bin/run, and enter the rotate_app_secret command. This will redeploy CiviForm, changing the secret key to a new, random value.

Additionally, you should add export RANDOM_PASSWORD_LENGTH=64 to your civiform_config.sh file. The secret length was originally only 16 characters, and when CiviForm moves to using version 2.9 or later of the Play Framework, 32 will be the minimum.

Troubleshooting

Terraform fails with error "Provider produced inconsistent final plan"

Terraform fails with other errors

The deploy command is idempotent, so if it fails, try running it again. The setup command can also be re-run, but it may fail to destroy resources if the resources Terraform uses to track created resources are corrupted. If the script is failing to destroy resources, try running bin/run destroy on its own. If this fails, you may need to manually delete resources in your cloud provider's console. Once all resources are cleaned up, run bin/run destroy_backend_state_resources to cleanup up the corrupted backend state resources and then try running bin/setup again.

If changes were made upstream, you can change the code in the checkout env, but will need to commit PRs to fix in the main repo.

No such file or directory

If you see error like "no such file or directory"

./db-connection: line 2: cloud/aws/bin/lib.sh: No such file or directory
./db-connection: line 21: out::error: command not found

The scripts expect you to be in specific directories. You probably need to cd into the checkout directory or the top level directory. If you are running setup/deploy/revert, you will need to be in the top level directory. If you are running a script like db-connection, you need to be in the checkout directory.

Terraform fails with Error acquiring the state lock

This situation can happen when exiting deployment scripts using "Ctrl-C". Terraform acquires a lock every time you run ./bin/deploy or ./bin/setup, and releases the lock at the end of the script. This helps to prevent concurrent infrastructure changes. If the deploy process exited outside of Terraform, the lock remains and needs to be force removed in order to run deploy again. The end of the error message will contain the LOCK_ID. To unlock, re-run either bin/deploy or bin/setup with FORCE_UNLOCK_ID set like this:

FORCE_UNLOCK_ID="$LOCK_ID" ./bin/deploy

Alternatively, it can be done by manually calling terraform like this:

terraform -chdir=checkout/cloud/aws/templates/aws_oidc force-unlock $LOCK_ID

(AWS) Notes on using CloudShell for deployment

It is an option to use CloudShell for doing deployments, but you should be aware that installations aren't persisted (see (FAQs)[https://aws.amazon.com/cloudshell/faqs/]), so it may be easier to use the AWS CLI.

If you do choose to use CloudShell, you likely will need to install Java and Terraform.

Helpful resources

Requesting AWS certificate

ARN has format of arn:aws:acm::<user_id>:certificate/<identifier>

AWS deployment setup options

Before going through the AWS deployment, it is helpful to understand how you plan to manage various deployments, if you plan to create multiple instances of CiviForm (i.e. staging, prod, etc.).

Some benefits of using AWS environments are:

  • There is more separation between environments, which makes it easier to remove resources without worrying about causing any issues to another deployment environment

  • It is easier to see cost breakdowns by deployment

A drawback to be aware of is that you'll have to create profiles in the AWS CLI and switch between the different profiles when doing deployments or, if using CloudShell, you'll have to do this in separate CloudShell for each environment.

Setting up Azure AD

The setup script prompts you to set up Azure AD. There are a few additional steps.

In Azure Portal:

  • Go to Azure Active Directory -> App registrations

  • Click the tab for "All applications"

Use the menu on the left:

  • Authentication: setup the Redirect URI to be what the app expects: https://<custom_hostname>/callback/AdClient.

  • You will also need an admin group which creates CiviForm admins

  • Token configuration: To allow for CiviForm admins, you need to have Azure AD return the groups claim. Add the security groups claim (you can verify the groups claim is being returned by decoding the base64 token from the token you get back from Azure AD on the website-- if you preserve the log in the Chrome Dev Tool window it should be from https://<custom_hostname>/callback/AdClient)

Access the database for emergency repair

Find the version that you want to deploy on .

The Play Framework used by CiviForm utilizes an . This key is used for signing session cookies and CSRF tokens, among other things. While this secret is secured storely by the deployment system, it's a good idea to rotate it periodically in order to mitigate the risk of a leaked secret.

This error can happen when running bin/setup for the first time. If you see it, re-run bin/setup. This is a known Terraform .

Java can be installed with sudo yum install java and terraform can be installed by following the under Linux / Amazon Linux.

Follow . After your certificate is ready and validated, copy the ARN from your console and put it into the SSL_CERTIFICATE_ARN variable in your civiform_config.sh

Screen Shot 2022-08-10 at 3 50 39 PM

By default, each deployment will have its own prefix, which allows you to distinguish the different ones from each other, but is a way of keeping deployments more separate from each other.

Find your app (staging-dynamic-heron for CiviForm Azure Staging)

The CiviForm deployment system provides a mechanism for temporary and secure direct access to the production database via the

For details on how to access the database with pgadmin visit the page.

Github
application secret key
bug
installation commands
official documentation
AWS Organizations
pgadmin
Production Database Access
Terraform overview
developer docs
Authentication Providers
civiform-deploy
Github
civiform_config.example.sh
staging-aws
setting up Azure AD for an example
requesting AWS certificate
AWS deployment setup options
responds to new issues in the CiviForm issue tracker