Troubleshooting
If something goes wrong with this process, you may see an error in tests at the time that the database connection is being initiated. It will use the words "not enhanced", and contain a list of troubleshooting steps. The error will tell you to install a plugin for IDEA or Eclipse. This is a red herring! We do ebeans enhancement at build time, and that plugin is for projects which don't use a project management tool (like
sbt or mvn). The plugin performs a small portion of what sbt does, to enable you to click the green "build" button in your IDE - but none of the rest of our tooling will work with that, so there's no benefit, no need to do this.The error will also tell you to check
ebeans.mf. We don't have that - that file is automatically generated by the play integration. Our equivalent of that is in conf/application.conf, near the bottom, ebeans.default = "models.*". This step is worth doing - is your model outside of models.*? If so, add your model to the list. This is just one of the ways that the framework is highly dependent on our particular package structure, which should not be changed without good reason - you might find that strange things start happening.The most likely cause of this issue is actually an apparently-benign issue loading the ebeans code. The
sbt test / sbt run will continue, but at the top of the output, you might see java.lang.UnsupportedClassVersionError: Unsupported major version 59. This means that you've compiled the code with a higher version of the jdk than you're running it with - at some point. It's not guaranteed that you're currently trying to do that - sbt tries to be really smart about incremental compilation, so you may have compiled it by, e.g. accidentally hitting the "build" button in Eclipse from muscle memory, possibly quite some time ago.The best solution to this is to run
sbt clean cleanFiles. This is not quite a full clean - you must then delete every directory named target under the project root (at minimum target/, project/target/, and project/project/target/), then re-run a docker build / test.Examples: it's not running tests, or it's telling you there are compiler warnings even though there aren't.
SBT has a lot of caching built in, and it doesn't always clean itself up quite right. There is no single way to clean up the entire cache.
Try
bin/sbt clean, and if that doesn't work you might try rm -rf server/target/ server/project/project server/project/target server/logs/.You might also try
docker-compose down --remove-orphans or docker system prune, if you suspect the problem might be a rogue docker container locking the sbt directory.If the problem persists even after trying the tips above, you can try increasing Docker memory and CPU usage limit.
I don't know why this happens. I have a lot of info on the symptoms and a solution that seems to work. Solution first - run this on your host machine (not in the container).
rm -rf server/target/*. Seems scary but this is just output cache stuff. Everything works fine once you do this.The bind mount in docker just doesn't copy subdirectories of
target/streams/. I can't figure out why to save my life, it makes no sense, it copies files in the target directory just fine, picks up changes to that directory fine. There's no symlinks or hardlinks that it's failing to follow, as far as I can tell. But if you delete the directory it works okay. Google is no help - "bind mount missing subdirectory" and related searches are full of people confused about the wrong thing. Please edit this if you figure out why - I hate mysteries! But at least there's a fix.bin/run-dev pulls the latest dev image from the public registry. If for any reason, you cannot download the image, you can build it yourself locally. To build the container that runs the app, type docker build -t civiform . Running this takes ~5 minutes, but it bakes in most of the dependencies you will need to download, so if you make a significant change to the dependencies you may want to re-build. Once the image is built, set environment variable USE_LOCAL_CIVIFORM=1 so that bin/run-dev will use the image you just built.The docker container no longer requires the volume mount, so it could be that you ran the container without the volume mapping. Try adding
-v $PWD/server:/usr/src/server. If you're running with docker-compose or bin/run-dev, check that you're running the command from the correct directory (the root of the git repo, one directory above the build.sbt file).The tests run a postgres docker container for full integration testing. We used to use an in memory database (which play supports quite well), but we stopped doing that because the in-memory database doesn't support the json types we are using as the core of our application, and we would have had to mock out the majority of our database interactions even in the integration tests. However, this means that the tests (which run inside a docker container) need to be able to start another docker container. This situation is called "docker in docker", and everyone recommends against doing it (the author of that article is the one who invented the tools to make it possible).
Instead, we cheat a little. The docker client communicates with the docker daemon using a unix socket located at
/var/run/docker.sock. So, we just mount the host machine's /var/run/docker.sock inside the container's /var/run, the container's docker client communicates with the host system's docker daemon, and everything works fine. Add the magic string -v /var/run/docker.sock:/var/run/docker.sock to the docker run invocation.If you find yourself needing to do this during a
docker build command - for instance because you'd like to run unit tests during container build, as we used to be able to do - you're not likely to succeed and I'd recommend finding another way to do what you need to do. Volume mounts (which allow this end-running of docker-in-docker) are not available at container build time.The error message says expecting
"[email protected]" to be contained in the html content. This is likely you have a stale oidc-provider docker image. Simply run docker pull public.ecr.aws/t1q6b4h2/oidc-provider:latest to get the latest image and try again.Unit tests run against a single Postgres instance. The maximum number of concurrent Postgres connections is configured via the
fixedConnectionPool Play configuration variable. If a unit test makes use of play.inject.guice.GuiceApplicationBuilder or play.test.Helpers.fakeApplication, a new Postgres connection will be created as part of applying database evolutions. If no call is made to play.test.Helpers.stop after the test completes, the Postgres connection will be leaked and potentially cause unrelated tests to fail with the above error.If you see this error and a bunch error like the below, running bin/sbt clean seem to have it fixed for many. [error] GET /support/unconfirmedIdcsEmail controllers.SupportController.handleUnconfirmedIdcsEmail(request: Request)
If you're running Docker on an M1 Mac and getting a weird error like this:
civiform | # A fatal error has been detected by the Java Runtime Environment:
civiform | #
civiform | # SIGILL (0x4) at pc=0x000000401ddb6205, pid=22, tid=236
civiform | #
civiform | # JRE version: OpenJDK Runtime Environment AdoptOpenJDK (11.0.10+9) (build 11.0.10+9)
civiform | # Java VM: OpenJDK 64-Bit Server VM AdoptOpenJDK (11.0.10+9, mixed mode, tiered, compressed oops, g1 gc, linux-amd64)
civiform | # Problematic frame:
civiform | # J 3941 c1 sbt.Scope.hashCode()I (8 bytes) @ 0x000000401ddb6205 [0x000000401ddb61e0+0x0000000000000025]
civiform | #
civiform | # No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
civiform | #
civiform | # An error report file with more information is saved as:
civiform | # /tmp/hs_err_pid22.log
civiform | Compiled method (c1) 29645 4267 3 sbt.internal.util.Init::$anonfun$delegateForKey$2 (48 bytes)
civiform | total in heap [0x000000401de69410,0x000000401de69b10] = 1792
civiform | relocation [0x000000401de69588,0x000000401de695e0] = 88
civiform | main code [0x000000401de695e0,0x000000401de69980] = 928
civiform | stub code [0x000000401de69980,0x000000401de699d0] = 80
civiform | oops [0x000000401de699d0,0x000000401de699d8] = 8
civiform | metadata [0x000000401de699d8,0x000000401de699f0] = 24
civiform | scopes data [0x000000401de699f0,0x000000401de69a60] = 112
civiform | scopes pcs [0x000000401de69a60,0x000000401de69ae0] = 128
civiform | dependencies [0x000000401de69ae0,0x000000401de69af0] = 16
civiform | nul chk table [0x000000401de69af0,0x000000401de69b10] = 32
civiform | Compiled method (c2) 29656 4125 4 scala.util.hashing.MurmurHash3::productHash (70 bytes)
civiform | total in heap [0x000000401de1d110,0x000000401de1d9b8] = 2216
civiform | relocation [0x000000401de1d288,0x000000401de1d2f8] = 112
civiform | main code [0x000000401de1d300,0x000000401de1d680] = 896
civiform | stub code [0x000000401de1d680,0x000000401de1d6c8] = 72
civiform | oops [0x000000401de1d6c8,0x000000401de1d6d0] = 8
civiform | metadata [0x000000401de1d6d0,0x000000401de1d740] = 112
civiform | scopes data [0x000000401de1d740,0x000000401de1d7d8] = 152
civiform | scopes pcs [0x000000401de1d7d8,0x000000401de1d8e8] = 272
civiform | dependencies [0x000000401de1d8e8,0x000000401de1d900] = 24
civiform | handler table [0x000000401de1d900,0x000000401de1d9a8] = 168
civiform | nul chk table [0x000000401de1d9a8,0x000000401de1d9b8] = 16
civiform | Could not load hsdis-amd64.so; library not loadable; PrintAssembly is disabled
civiform | #
civiform | # If you would like to submit a bug report, please visit:
civiform | # https://github.com/AdoptOpenJDK/openjdk-support/issues
civiform | #
civiform | qemu: uncaught target signal 6 (Aborted) - core dumped
civiform | /entrypoint.sh: line 4: 22 Aborted sbt "[email protected]"
It's because the AdoptOpenJDK image is incompatible with ARM architecture. M1 Mac users will need to use the
USE_LOCAL_CIVIFORM=1 environment variable, and build the local docker image using docker build --build-arg PLATFORM=arm64 -t civiform-dev .If you see an error when running
bin/run-dev like [CompletionException: javax.persistence.EntityNotFoundException: Bean not found during lazy load or refresh. id[1] type[class models.Account]], you probably have a cookie set in the browser from the last time you loaded the site, but the associated account doesn't exist in the fresh local database. To fix, simply clear your browser cookie(s) for localhost (in Chrome, this can be accomplished by clicking the info/lock icon next to the URL, then selecting "cookies" - click "Remove" to remove all associated cookies for localhost).Sometimes you have partial db information. A fix is to fully clear the db with
docker-compose down --remove-orphansIf you see an error when running
bin/run-dev such as [Guice/ErrorInjectingConstructor]: RuntimeException: S3 exception: The specified bucket does not exist (Service: S3, Status Code: 404), this is due to the localstack S3 bucket is not created successfully. If you have AWS CLI installed, please make sure it is configured. You can verify or reconfigure by running aws configure. Enter test to any questions that have none as default.Try dropping it completely with
docker rm postgres, and then it'll be recreated automatically when you run bin/run-devTry these steps to clear caches, docker, etc. - hopefully one will get you into a better state. They are roughly ordered by which have been most helpful in fixing past issues.
- 1.Clean up docker containers:
docker-compose down --remove-orphans - 2.Run
docker system prune - 3.Clear the sbt cache (run from
universal-application-tool/directory):rm -rf server/target/ server/project/project server/project/target server/logs/ - 4.Remove all docker images - you can do this in Docker Desktop by going to "Images", then clicking the "Clean up" button in the top right
- 5.Remove all docker volumes - run
docker volume lsto see a list, and thendocker volume rm ${volume_name}to remove them - 6.Remove home directory cache folders:
rm -rf ~/.ivy ~/.ivy2 ~/.sbt ~/.coursier
If after trying everything, you are entering an infinite compile loop... try deleting your Civiform repo and re-cloning. This is a very last resort if the above steps (clearing cache, bin/sbt-clean, everything else) don't help you escape the infinite SBT loop!
Last modified 2mo ago