Breaking Glass on Database
Breaking glass on the production or staging database is a way to save production or staging outages if something extreme happens that cannot be patched with software patches. Staging is the wild-west, so we can just
curl -X POST http://staging.seattle.civiform.com/dev/seed/clearif we need to truncate all of the tables in staging.
To break glass,
- 1.Sign into the AWS management console and open up cloudshell
- 2.In cloudshell, clone civiform:
git clone https://github.com/civiform/civiform.git
- 3.In cloudshell, run civiform's
bin/breakglass-db-access prodscript to break glass into prod. Or just
bin/breakglass-db-accessto break glass into staging. The
breakglass-db-accessscript puts you directly in psql shell connected to the production or staging environment.
- 4.Make sure to exit out of the postgresql shell by running
ctrl + dto terminate the ec2 instance and delete the emergency DB security group, or you may need to manually clean things up afterwards.
- 5.To manually clean up any instances, go to EC2 console and terminate any remaining instances and delete any keys.