Forgive me as I'm just sort of spitballing here... the short answer is no I don't think so.
The Apple Documentation on the colorSchemeContrast environment variable tells us if the user is using increased contrast setting on their device.
What you could do is increase the contrast on the color using the .contrast modifier to that specific views color. However it is important to test the edge case that if a user has the A11Y setting which is to increase the contrast enabled, how will that effect your view?
Maybe you use a combination of the environment variable, and the contrast modifier to only apply the increased contrast if the user does not have increased contrast enabled?
I do hope that this helped guide you in some manner.