For me I think the fix was changing my Dockerfile to disable automatic resolution. NOTE: that I also deleted my derived data directory.
RUN swift build \
		--enable-test-discovery \
		--disable-automatic-resolution \
		-c release \
		-Xswiftc -g
Not sure if this had a material impact but while I was in there... I upgraded my docker build:
#FROM swift:5.2.3-bionic as build
FROM swift:5.3-bionic as build
Topic:
Developer Tools & Services
SubTopic:
Xcode
Tags: