Breaking Glass on Database
Breaking glass on the production or staging database is a way to save production or staging outages if something extreme happens that cannot be patched with software patches. Staging is the wild-west, so we can just
curl -X POST http://staging.seattle.civiform.com/dev/seed/clear
if we need to truncate all of the tables in staging.To break glass,
- 1.Sign into the AWS management console and open up cloudshell
- 2.In cloudshell, clone civiform:
git clone https://github.com/civiform/civiform.git
- 3.In cloudshell, run civiform's
bin/breakglass-db-access prod
script to break glass into prod. Or justbin/breakglass-db-access
to break glass into staging. Thebreakglass-db-access
script puts you directly in psql shell connected to the production or staging environment. - 4.Make sure to exit out of the postgresql shell by running
\q
or pressingctrl + d
to terminate the ec2 instance and delete the emergency DB security group, or you may need to manually clean things up afterwards. - 5.To manually clean up any instances, go to EC2 console and terminate any remaining instances and delete any keys.
TODO
Last modified 6mo ago