Network Framework Revocation Checks

Posting the code snippet above here as well as I reached word limit above:

sec_protocol_options_set_verify_block(sec_options, ^(sec_protocol_metadata_t metadata, sec_trust_t trust_ref, sec_protocol_verify_complete_t complete){

    if (ignoreCertValidation)
    {
        logger->Info("TLS certificate validation errors will be ignored.");
        complete(true);
    }
    else
    {
        SecTrustRef peerTrust = NULL;
        if (!(peerTrust = sec_trust_copy_ref(trust_ref)))
        {
            logger->Error("Unable to copy SecTrustRef for SSL verification.");
        }

        CFArrayRef trustPolicy = NULL;
        SecPolicyRef revocationPolicy = NULL;
        OSStatus copyPolicyStatus = SecTrustCopyPolicies(peerTrust, &trustPolicy);
        if (copyPolicyStatus != errSecSuccess)
        {
            string errMessage = StringUtils::CFStringRefToString(SecCopyErrorMessageString(copyPolicyStatus, NULL));
            logger->Error("Unable to create additional policies for TLS trust evaluation. Reason: %s", errMessage.c_str());
        }
        else
        {
            CFMutableArrayRef mPolicies = CFArrayCreateMutableCopy(kCFAllocatorDefault, (CFArrayGetCount(trustPolicy) + 1), trustPolicy);
            CFRelease(trustPolicy);
            SecPolicyRef sslPolicy = (SecPolicyRef)CFArrayGetValueAtIndex(mPolicies, 0);
            CFDictionaryRef sslPolicyProperties = SecPolicyCopyProperties(sslPolicy);
            CFStringRef policyType = (CFStringRef)CFDictionaryGetValue(sslPolicyProperties, kSecPolicyOid);

            if (policyType && CFEqual(kSecPolicyAppleSSL, policyType)) // Only apply revocation to SSL validation
            {
                if (crlChecks == CRLChecks::Hard)
                {
                    revocationPolicy = SecPolicyCreateRevocation(kSecRevocationCRLMethod | kSecRevocationRequirePositiveResponse);
                }
                else
                {
                    revocationPolicy = SecPolicyCreateRevocation(kSecRevocationCRLMethod);
                }

                if (revocationPolicy)
                {
                    CFArrayAppendValue(mPolicies, revocationPolicy);
                    OSStatus setPolicyStatus = SecTrustSetPolicies(peerTrust, mPolicies);
                    if (setPolicyStatus != errSecSuccess)
                    {
                        string errMessage = StringUtils::CFStringRefToString(SecCopyErrorMessageString(setPolicyStatus, NULL));
                        logger->Error("Unable to add additional policies for TLS evaluation. Reason: %s", errMessage.c_str());
                    }
                }
            }

            if (revocationPolicy)
            {
                CFRelease(revocationPolicy);
            }

            if (sslPolicyProperties)
            {
                CFRelease(sslPolicyProperties);
            }

            if (mPolicies)
            {
                CFRelease(mPolicies);
            }
        }
        

        CFErrorRef trustError = NULL;
        if (SecTrustEvaluateWithError(peerTrust, &trustError))
        {
            logger->Debug("TLS trust evaluation successful");
            complete(true);
        }
        else
        {
            SecTrustResultType trustResult;
            SecTrustGetTrustResult(peerTrust, &trustResult);
            switch(trustResult)
            {
                case kSecTrustResultUnspecified:
                    logger->Error("Trust evaluation result - kSecTrustResultUnspecified");
                    break;
                case kSecTrustResultProceed:
                    logger->Error("Trust evaluation result - kSecTrustResultProceed");
                    break;
                case kSecTrustResultDeny:
                    logger->Error("Trust evaluation result - kSecTrustResultDeny");
                    break;
                case kSecTrustResultRecoverableTrustFailure:
                    logger->Error("Trust evaluation result - kSecTrustResultRecoverableTrustFailure");
                    break;
                case kSecTrustResultFatalTrustFailure:
                    logger->Error("Trust evaluation result - kSecTrustResultFatalTrustFailure");
                    break;
                case kSecTrustResultOtherError:
                    logger->Error("Trust evaluation result - kSecTrustResultOtherError");
                    break;
                case kSecTrustResultInvalid:
                    logger->Error("Trust evaluation result - kSecTrustResultInvalid");
                    break;
            }

            CFDictionaryRef trust_results = NULL;
            trust_results = SecTrustCopyResult(peerTrust);
            CFBooleanRef revoChecked = (CFBooleanRef)CFDictionaryGetValue(trust_results, kSecTrustRevocationChecked);
            if(revoChecked == kCFBooleanTrue)
            {
                logger->Error("Revocation check returned TRUE");
            }
            else
            {
                logger->Error("Revocation check returned FALSE");
            }

            if(trust_results)
            {
                CFRelease(trust_results);
            }

            // Call to CFErrorCopyDescription can sometimes return the wrong error message strings.
            // Check the Security.framework error codes at Security.framework/Headers/SecBase.h  
            CFIndex errCode = CFErrorGetCode(trustError);
            const string errMessage = StringUtils::CFStringRefToString(CFErrorCopyDescription(trustError));
            logger->Error("TLS trust evaluation failed. Error: %ld '%s'", errCode, errMessage.c_str());

            complete(false);
        }

        if (trustError)
        {
            CFRelease(trustError);
        }

        if (peerTrust)
        {
            CFRelease(peerTrust);
        }
    }
}, m_connectionQueue);

If I run WireShark traces on the server machine where the CRL distribution point is also located, I do not see any HTTP requests coming in for the CRL list. I have checked the CRL DP URL in a browser and it is reachable.

Is there something wrong with the policy creation process? Why is it not at least trying to access the CRL DP?

Performing revocation checks for a certification via CRLs has never been supported in iOS. In macOS 10.15, support was removed in favor of OCSP across iOS and macOS. My recommendation would be to move towards OCSP, or if you need to use CRLs, you will need to do this by hand and act upon the response yourself.

One way to run a few tests would be to try your OCSP checks with something like this:

// Obtain the certificate chain

let certArray: [SecCertificate] = [caCertificate, intermediateCertificate, leafCertificate]

let secSSLPolicy = SecPolicyCreateSSL(true, "www.myDomain.com" as CFString)
let ocspPolicy = SecPolicyCreateRevocation(kSecRevocationOCSPMethod)!
let policyArray: [SecPolicy] = [ocspPolicy, secSSLPolicy]
var secTrust: SecTrust?

let status = SecTrustCreateWithCertificates(certArray as CFArray, policyArray as CFArray, &secTrust)
os_log("SecTrustCreateWithCertificates status: %d", status)

guard let trust = secTrust else {
    os_log("Could not create SecCertificates from data")
    return
}
// Note that this should hit the network
DispatchQueue.global().async {
    SecTrustEvaluateAsyncWithError(trust, .global()) {
        trust, result, error in
        if result {
            print("Trusted: \(result)! - error: \(error)")
        } else {
            print("Trust failed: \(error!.localizedDescription)")
        }
    }
}

Now, the code above should be refactored to fit your situation, but it should get your going at least. Note, make sure to use SecTrustEvaluateAsyncWithError if you are doing OCSP checks.

Thanks meaton. I did not come across any documentation saying CRL checks had been removed from 10.15. I set up an OCSP responder setup using a Windows server and I can see OCSP requests coming in for my Windows clients but the macOS client does not send any such request. Still getting the error code -67635 corresponding to errSecIncompleteCertRevocationCheck. I prefer to make this whole operation synchronously, so was continuing to use SecTrustEvaluateWithError but when that continued to give me the same errors, I switched over to SecTrustEvaluateAsyncWithError. I was blocking the main thread using a std::promise and std::future setup like this:

std::future<void> barrier_future = m_barrier.get_future();
					
std::thread workThread(TrustEval);
					
barrier_future.wait();
workThread.join();
					
if(trustEvalResult)
{
    fprintf(stderr,"COMPLETE = TRUE \n");
    complete(true);
}
else
{
    fprintf(stderr,"COMPLETE = FALSE \n");
    complete(false);
}

where m_barrier and trustEvalResult are global variables. This code is still inside the sec_protocol_options_set_verify_block.

The TrustEval function running in a separate thread namely workThread is:

void TrustEval()
{
	fprintf(stderr,"Starting TRUST EVAL \n");
	
	dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
		SecTrustEvaluateAsyncWithError(peerTrust, dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^(SecTrustRef evaluatedTrust, bool trustResult, CFErrorRef error) {
				if (trustResult) {
					fprintf(stderr,"TRUST EVAL SUCCESSFUL \n");
					// Evaluation succeeded!
					trustEvalResult = true;
	 
				} else {
					fprintf(stderr,"TRUST EVAL FAILED \n");
					// Evaluation failed: Check the error
					trustEvalResult = false;
				}
	 
				// Finally, release the trust and error.
				if (evaluatedTrust) { CFRelease(evaluatedTrust); }
				if (error)          { CFRelease(error); }
				
				
				fprintf(stderr,"Setting m_barrier \n");
				m_barrier.set_value();
			}
		);
	});
}

But still see the revocation check fail from the SecTrustWithErrorCallback block. And no OCSP requests from the macOS machine. I should mention going to the browser I can access the crl and ocsp endpoint urls.

I did not come across any documentation saying CRL checks had been removed from 10.15.

Right, I filled an internal bug about this back in March (r. 89708330), but I would encourage you to open another bug regarding this matter. Please respond back with the Feedback ID.

Regarding the inconsistencies of that you are seeing, if you extract the code into a macOS Command Line test app and test it with a valid chain and with an invalid chain, are you able to sort out what is happening?

Note:

 % security error -67635
Error: 0xFFFEF7CD -67635 An incomplete certificate revocation check occurred.

I have submitted another feedback report (FB10033659) for your reference. I can download the CRL and access OCSP endpoint from my macOS machine. I also verified the server certificate revocation status using an OpenSSL command:

 openssl ocsp -issuer caCert.cer -cert switchCert.cer -text -url http://<address>/ocsp

I am able to get the response back like this for a revoked certificate:

Response Verify Failure
4571629228:error:27FFF065:OCSP routines:CRYPTO_internal:certificate verify error:/AppleInternal/Library/BuildRoots/b6051351-c030-11ec-96e9-3e7866fcf3a1/Library/Caches/com.apple.xbs/Sources/libressl/libressl-2.8/crypto/ocsp/ocsp_vfy.c:141:Verify error:unable to get local issuer certificate
switchCert.cer: revoked
        This Update: May 31 16:14:42 2022 GMT
        Next Update: Jun  2 04:34:42 2022 GMT
        Reason: certificateHold
        Revocation Time: May 31 16:24:00 2022 GMT

This is all expected, but my macOS Command Line test app as you suggested I make still makes no requests to the OCSP endpoint where I am running WireShark traces to monitor incoming traffic. This is both with a valid or invalid chain. To expand that a bit more, the chain the server sends to the macOS machine is the leaf certificate and the root CA. This root CA I have added to the keychain and marked as "Always Trust" for everything in Keychain Access.

Weirdly both Chrome and Safari mark this revoked server certificate as valid, when I try to make a HTTPS request to the server. Also not making any requests to the CRL or OCSP endpoints. On my Windows machines, this certificate is marked correctly as revoked in chrome when I try to make HTTPS requests to the same server. Is this something related?

I have submitted another feedback report (FB10033659) for your reference.

Thank you!

Regarding:

This root CA I have added to the keychain and marked as "Always Trust" for everything in Keychain Access.

If you try this with a certificate chain that the root is in the trust store, but the leaf has not been seen before on the machine, does an OCSP call go out to the server at that point?

Also, double check to make sure that your server is not doing the OCSP check for your and then vending the response back in the handshake negotiation.

The server is not responding with an OCSP check in the handshake negotiation. OCSP stapling is not enabled on the server. I tried two leaf certificate with my new root CA and still no OCSP messages on the WireShark traces.

Weirdly when I try to visit websites hosted by digicert.com which server revoked certificates from their CAs, I can see the OCSP messages going to http://ocsp.digicert.com. The application correctly recognizes the revoked certificate and fails the TLS trust evaluation. These test websites can be found here: DigiCert Root Certificates - Download & Test | DigiCert.com

Weirdly when I try to visit websites hosted by digicert.com which server revoked certificates from their CAs, I can see the OCSP messages going to http://ocsp.digicert.com. The application correctly recognizes the revoked certificate and fails the TLS trust evaluation.

Good. Now we have a baseline for the situation and we know that it does work.

Regarding:

I tried two leaf certificate with my new root CA and still no OCSP messages on the WireShark traces.

Here is my theory on what is happening in this case; since the CA certificates are not in the system trust store then trust evaluation can be done locally as opposed to going to the server to see if the certificate chain has been revoked. For example, OCSP is most likely being done on a chain that exists in the trust store to see if since adding to trust store that a certificate in the chain has been revoked. In this case though, there is no need to go to the OCSP server because the system can see if your chain is trusted manually by the user because the user would have to override the trust settings locally to ensure a custom CA certificate is trusted. Then, going to the OCSP server would be an extra network call if the user already manually set the chain as trusted. This is my interpretation on what is happening, if you do see something else happening I would encourage you to open a bug report.

I finally found a solution to the OCSP requests not being sent. The problem was caused by the /etc/hosts file. I have my server and my OCSP responder hosted on the same machine. That is why I had a line in the hosts file like:

192.168.x.x server1.local.com server2.local.com ocsp.responder.com

These are just placeholder names. With this line I could not see any OCSP requests on my WireShark trace.

As soon as I decoupled those lines in the /etc/hosts file like:

192.168.x.x server1.local.com server2.local.com ocsp.responder.com
192.168.x.x ocsp.responder.com

I could see OCSP requests from the macOS machine to my OCSP responder. Everything now works pretty well. This macOS machines was running Monterey(12.4) , but I do compile my project with the target OS being 10.15.