CiviForm Docs
HomeAboutContactNewsFAQ
  • CiviForm Docs
  • Overview
    • What is CiviForm?
    • How does CiviForm work?
    • Glossary
  • User Manual
    • CiviForm Admin Guide
      • CiviForm Admin training overview
      • How to navigate CiviForm
      • Working with programs
        • Create a program
        • Edit a program
        • Show or hide questions based on inputs
        • Manage program eligibility
        • Manage address & service area validation
        • Manage notifications
        • How to publish programs
        • Set a pre-screener
      • Working with questions
        • Manage questions
        • Question export settings
        • Universal and Primary Applicant Information questions
        • Using enumerator questions & screens in a program
      • Manage translations for programs & questions
      • Manage versions for programs & questions
      • Working with applications
        • Add statuses to a program
        • Download exported data
      • Role management
        • Manage Program Admins
        • Manage Trusted Intermediaries
      • Manage API keys
      • Using Markdown
      • Migrating programs between environments
    • Program Admin Guide
      • How to become a Program Admin
      • Review completed applications
    • Trusted Intermediary Guide
      • Apply to a program
    • Onboarding Guide
      • Organization assessment
      • Program assessment
      • Getting started with service design
      • Journey mapping
      • Discovery, eligibility, and intake
      • Consolidating questions across programs
      • Working with existing tools and processes
      • Working across jurisdictions
      • Data reporting and other integrations
      • Security and privacy considerations
      • Staffing overview
  • IT Manual
    • Technical Deployment Guide
      • Initial Deployment
        • Terraform deploy system
          • AWS Terraform deployment
        • Authentication setup
        • Email configuration
        • GIS Service configuration
      • Upgrading to a New Release
        • CiviForm server environment variables
          • v1.20.0
          • v1.20.1
          • v1.21.0
          • v1.22.0
          • v1.23.0
          • v1.23.1
          • v1.24.0
          • v1.24.1
          • v1.24.2
          • v1.25.0
          • v1.26.0
          • v1.27.0
          • v1.28.0
          • v1.29.0
          • v1.30.0
          • v1.30.1
          • v1.31.0
          • v1.33.0
          • v1.34.0
          • v1.34.1
          • v1.34.2
          • v1.35.0
          • v1.36.0
          • v1.37.0
          • v1.38.0
          • v1.38.1
          • v1.38.2
          • v1.39.0
          • v1.40.0
          • v1.41.0
          • v1.42.0
          • v1.43.0
          • v1.44.0
          • v1.45.0
          • v1.46.0
          • v1.47.0
          • v1.48.0
          • v1.49.0
          • v1.50.0
          • v1.51.0
          • v1.52.0
          • v1.53.0
          • v1.54.0
          • v1.55.0
          • v1.56.0
          • v1.56.1
          • v1.57.0
          • v1.58.0
          • v1.59.0
          • v1.60.0
          • v1.61.0
          • v1.62.0
          • v1.63.0
          • v2.0.0
          • v2.0.1
          • v2.0.2
          • v2.1.0
          • v2.10.0
          • v2.11.0
          • v2.12.0
          • v2.13.0
          • v2.14.0
          • v2.15.0
          • v2.16.0
          • v2.17.0
          • v2.18.0
          • v2.19.0
          • v2.2.0
          • v2.20.0
          • v2.21.0
          • v2.22.0
          • v2.23.0
          • v2.24.0
          • v2.25.0
          • v2.26.0
          • v2.27.0
          • v2.28.0
          • v2.29.0
          • v2.3.0
          • v2.30.0
          • v2.31.0
          • v2.32.0
          • v2.33.0
          • v2.34.0
          • v2.35.0
          • v2.36.0
          • v2.37.0
          • v2.38.0
          • v2.39.0
          • v2.4.0
          • v2.4.1
          • v2.4.2
          • v2.4.3
          • v2.5.0
          • v2.6.0
          • v2.7.0
          • v2.8.0
          • v2.9.0
      • Monitoring
      • Troubleshooting Production
      • Disaster Recovery
      • Database Disaster Recovery
      • Production Database Access
    • Infrastructure Requirements
    • Existing deployments
    • API Integration
      • Authentication
      • List applications
    • Testing & QA
      • Testing resources
      • SQL queries to look for missing questions
  • Governance & Management
    • Project Management
      • On Call Guide
    • Governance
      • Roles, Committees, & Responsibilities
      • Governance Processes
      • Development Principles
      • Communication
Powered by GitBook
On this page
  • The Subsystems
  • Why Cloud Deployment is Preferred
  • What Cloud deployment looks like

Was this helpful?

Edit on GitHub
Export as PDF
  1. IT Manual

Infrastructure Requirements

PreviousProduction Database AccessNextExisting deployments

Last updated 2 years ago

Was this helpful?

This document describes the infrastructure required to deploy CiviForm into a new jurisdiction.

CiviForm is a classic web-based client-server architecture which stores applicant information in a database. The application server typically runs in a Docker container, a type of virtualized environment which contains most of the pieces described below.

The Subsystems

Basic architecture

Basic architecture diagram

Note that there usually is not direct traffic between the CiviForm server and identity providers (dotted lines). Rather, users authenticate by visiting CiviForm, which redirects their browser to an identity provider, which in turn redirects them back to CiviForm.

Load Balancer

Software-defined load balancer to route incoming client traffic to an array of backends; required to ensure the software scales up gracefully with increased load.

If running in a cloud-service, an example would be AWS ELB or Azure Load Balancer.

If running on-prem, examples would be SeeSaw, nginx, HA Proxy, Kubernetes load balancer, etc.

Database

CiviForm uses the PostgreSQL database.

  • Only a single instance is necessary, with regular snapshot backups.

  • Estimated 2 vCPU, 8GB memory, 50GB storage.

Application Servers

These are the servers receiving & processing client requests.

  • Minimum 2 instances , each with 2 vCPU, 8GB memory

File Storage

File storage needs are for applicant-uploaded files and depend entirely upon the civic entity's configuration and usage. We recommend limiting applicant file upload size to 10MB per file.

Email Sending Service

For a cloud deployment, examples would be AWS SNS or Azure SendGrid.

For a local deployment, one would need to interface with the government's own email systems.

Authentication Systems

Two systems are needed: one which is configured to authenticate applicants, and one configured to authenticate administrators. Both systems must be OIDC compliant, and the adminstrator-systems must have configurable claims.

Why Cloud Deployment is Preferred

CiviForm is designed to run reliably and with minimal ongoing effort in a cloud-native environment. This is accomplished through the use of standardized infrastructure-as-code tools that automatically provision and deploy the necessary infrastucture.

The core application and database can be run on-premises with additional effort if desired, though service dependencies such as email, file storage, and monitoring are often easier to manage through a cloud provider. Deploying on-premises means these services would need to be separately developed, deployed, and maintained internally without full use of CiviForm's included deployment system. Such services include:

  • provisioning

  • deployment

  • monitoring

  • file storage and file system scaling

  • email notifications

  • database backups and recovery and storage scaling

  • application server scaling

On-prem deployment:

Pros

  • Total control of all infrastructure and data.

  • Can use existing pre-approved and familiar software and infrastructure.

Cons

  • Hardware capex & opex becomes expensive at scale

  • Given the limited size and time-availability of civic IT departments, on-prem is much more work to scale up, and likely less reliable (more outages).

  • Requires building bespoke systems that are much more laborious for new contributors/maintainers to work with. In contrast, major cloud providers have extensive training and certification offerings making it straightforward to train existing personnel and identify qualified job applicants.

Cloud deployment:

Pros

  • Instant scalability (just "add" more capacity)

  • Likely more reliable (fewer outages)

  • Opex cost is likely less than on-prem machine maintenance

  • More efficient use of IT staff's time

  • Dramatically less technical effort put towards infrastructure, meaning more time can go towards other efforts.

Cons

  • Requires careful audit to ensure cloud service and architecture is meeting government security & privacy requirements.

  • May require multiple levels of agency approval to deploy.

What Cloud deployment looks like

In general, a cloud provider refers to a business like AWS, Azure, or GCP.

Similarly, a cloud service refers to a service within the provider (such as file storage or container orchestration.)

From the IT department's point of view:

  • The CiviForm system is comprised of a set of resources (e.g. application servers, PostgreSQL database, file storage) will be running in a virtual private cloud (VPC).

    • A virtual private cloud ensures the different components of the CiviForm systems are only accessible by the entities that should have access. E.g. the database is only accessible from the application servers, the application servers are only accessible from the load balancer, etc

  • Terraform then deploys the resources to a commercial cloud provider – whether it be AWS, GCP, or Azure.

  • CiviForm's Java application servers run in a tool called Docker, all cloud providers understand (natively) how to run a Docker container in groups, using container orchestration services.

    • AWS has a container service called ECS; Azure has a container service called ACI. These services run the server code in a Docker container, but are using the native cloud app-services systems under the hood.

    • Under development, a local Docker instance contains a private PostgreSQL database within the same container as the app servers. In production, multiple Docker containers are deployed to the cloud, which then access a shared SQL database (e.g. RDS on AWS, or AzureSQL on Azure).

Cloud providers also have:

  • ...their own load balancing systems needed by CiviForm – they will properly route and balance incoming requests to the containers.

  • ...file storage systems (e.g. to allocate to the DB and store resident-uploaded files)

  • ...email sending systems

Costs of deploying into the Cloud:

  • Training: IT staff will need to learn basic proficiency with the cloud provider.

    • How to manage cloud accounts

    • How to monitor the health / traffic / resource consumption of their containers

    • How to modify the quantity of resources consumed

  • Operating Expense

    • Cloud providers will charge a monthly bill for resource consumption.

For CiviForm, we have designed the system's resources and deployment to be controlled in a set of text files – these are configuration files. IT staff has the freedom to modify these files and change how cloud resources deploy, how much disk/ram/cpu to give each, etc.

Terraform