sendto() system call - Nondeterministic "No route to host" due to local network restrictions

Please consider this trivial C code which deals with BSD sockets. This will illustrate an issue with sendto() which seems to be impacted by the recent "Local Network" restrictions on 15.3.1 macos.

#include <stdio.h>
#include <stdlib.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include "sys/socket.h"
#include <string.h>
#include <unistd.h>
#include <ifaddrs.h>
#include <net/if.h>

// prints out the sockaddr_in6
void print_addr(const char *msg_prefix, struct sockaddr_in6 sa6) {
    char addr_text[INET6_ADDRSTRLEN] = {0};
    printf("%s%s:%d, addr family=%u\n",
           msg_prefix,
           inet_ntop(AF_INET6, &sa6.sin6_addr, (char *) &addr_text, INET6_ADDRSTRLEN),
           sa6.sin6_port,
           sa6.sin6_family);
}

// creates a datagram socket
int create_dgram_socket() {
    const int fd = socket(AF_INET6, SOCK_DGRAM, 0);
    if (fd < 0) {
        perror("Socket creation failed");
        return -1;
    }
    return fd;
}

// returns a string representing the current local time
char *current_time() {
    time_t seconds_since_epoch;
    time(&seconds_since_epoch);
    char *res = ctime(&seconds_since_epoch);
    const size_t len = strlen(res);
    // strip off the newline character that's at the end of the ctime() output
    res[len - 1] = '\0';
    return res;
}

// Creates a datagram socket and then sends a messages (through sendto()) to a valid
// multicast address. This it does two times, to the exact same destination address from
// the exact same socket.
//
// Between the first and the second attempt to sendto(), there is
// a sleep of 1 second.
//
// The first time, the sendto() succeeds and claims to have sent the expected number of bytes.
// However system logs (generated through "log collect") seem to indicate that the message isn't
// actually sent (there's a "cfil_service_inject_queue:4466 CFIL: sosend() failed 65" in the logs).
//
// The second time the sendto() returns a EHOSTUNREACH ("No route to host") error.
//
// If the sleep between these two sendto() attempts is removed then both the attempts "succeed".
// However, the system logs still suggest that the message isn't actually sent.
int main() {
    printf("current process id:%ld parent process id: %ld\n", (long) getpid(), (long) getppid());
    // valid multicast address as specified in
    // https://www.iana.org/assignments/ipv6-multicast-addresses/ipv6-multicast-addresses.xhtml
    const char *ip6_addr_str = "ff01::1";
    struct in6_addr ip6_addr;
    int rv = inet_pton(AF_INET6, ip6_addr_str, &ip6_addr);
    if (rv != 1) {
        fprintf(stderr, "failed to parse ipv6 addr %s\n", ip6_addr_str);
        exit(EXIT_FAILURE);
    }

    // create a AF_INET6 SOCK_DGRAM socket
    const int sock_fd = create_dgram_socket();
    if (sock_fd < 0) {
        exit(EXIT_FAILURE);
    }
    printf("created a socket, descriptor=%d\n", sock_fd);

    const int dest_port = 12345; // arbitrary port
    struct sockaddr_in6 dest_sock_addr;
    memset((char *) &dest_sock_addr, 0, sizeof(struct sockaddr_in6));
    dest_sock_addr.sin6_addr = ip6_addr; // the target multicast address
    dest_sock_addr.sin6_port = htons(dest_port);
    dest_sock_addr.sin6_family = AF_INET6;
    print_addr("test will attempt to sendto() to destination host:port -> ", dest_sock_addr);

    const char *msg = "hello";
    const size_t msg_len = strlen(msg) + 1;
    for (int i = 1; i <= 2; i++) {
        if (i != 1) {
            // if not the first attempt, then sleep a while before attempting to sendto() again
            int num_sleep_seconds = 1;
            printf("sleeping for %d second(s) before calling sendto()\n", num_sleep_seconds);
            sleep(num_sleep_seconds);
        }
        printf("%s attempt %d to sendto() %lu bytes\n", current_time(), i, msg_len);

        const size_t num_sent = sendto(sock_fd, msg, msg_len, 0, (struct sockaddr *) &dest_sock_addr,
                                       sizeof(dest_sock_addr));
        if (num_sent == -1) {
            fprintf(stderr, "%s ", current_time());
            perror("sendto() failed");
            close(sock_fd);
            exit(EXIT_FAILURE);
        }
        printf("%s attempt %d of sendto() succeeded, sent %lu bytes\n", current_time(), i, num_sent);
    }
    return 0;
}

What this program does is, it uses the sendto() system call to send a message over a datagram socket to a (valid) multicast address. It does this twice, from the same socket to the same target address. There is a sleep() of 1 second between these two sendto() attempts.

Copy that code into noroutetohost.c and compile:

clang noroutetohost.c

Then run:

./a.out

This generates the following output:

current process id:58597 parent process id: 21614
created a socket, descriptor=3
test will attempt to sendto() to destination host:port ->ff01::1:14640, addr family=30
Fri Mar 14 20:34:09 2025 attempt 1 to sendto() 6 bytes
Fri Mar 14 20:34:09 2025 attempt 1 of sendto() succeeded, sent 6 bytes
sleeping for 1 second(s) before calling sendto()
Fri Mar 14 20:34:10 2025 attempt 2 to sendto() 6 bytes
Fri Mar 14 20:34:10 2025 sendto() failed: No route to host

Notice how the first call to sendto() "succeeds", even the return value (that represents the number of bytes sent) matches the number of bytes that were supposed to be sent. Then notice how the second attempt fails with a EHOSTUNREACH ("No route to host") error. Looking through the system logs, it appears that the first attempt itself has failed:

2025-03-14 20:34:09.474797	default	kernel	cfil_hash_entry_log:6082 <CFIL: Error: sosend_reinject() failed>: [58597 a.out] <UDP(17) out so 891be95f3a70c605 22558774573152560 22558774573152560 age 0> lport 0 fport 12345 laddr :: faddr ff01::1 hash 1003930
2025-03-14 20:34:09.474806	default	kernel	cfil_service_inject_queue:4466 CFIL: sosend() failed 65

(notice the time on that log messages, they match the first attempt from the program's output log)

So even though the first attempt failed, it never got reported back to the application. Then after sleeping for (an arbitrary amount of) 1 second, the second call fails with the EHOSTUNREACH. The system logs don't show any error (at least not the one similar to that previous one) for the second call.

If I remove that sleep() between those two attempts, then both the sendto() calls "succeed" (and return the expected value for the number of bytes sent). However, the system logs show that the first call (and very likely even the second) has failed with the exact same log message from the kernel like before.

If I'm not wrong then this appears to be some kind of a bug in the "local network" restrictions. Should this be reported? I can share the captured logs but I would prefer to do it privately for this one.

Another interesting thing in all this is that there's absolutely no notification to the end user (I ran this program from the Terminal) about any of the "Local Network" restrictions.

I see that 15.3.2 of macos has been released. So I gave this a try after upgrading to that version and this issue continues to reproduce there too.

And no notification/pop-ups either.

Any suggestions on how to address this?

The C program which has been provided as a reproducer in this thread was hand crafted to demonstrate more easily what the issue is.

The real world code which reproduces this in Java program is as trivial as the following:

import java.net.*;

public class LocalNetworkTest {
    public static void main(final String[] args) throws Exception {
        final byte[] hello = "hello".getBytes();
        try (final DatagramSocket ds = new DatagramSocket()) {
            final int arbitraryDestPort = 12345;
            final InetAddress destAddr = InetAddress.getByName("ff01::1");
            final DatagramPacket packet = new DatagramPacket(hello, hello.length, destAddr, arbitraryDestPort);
            System.out.println("attempting to send a packet to " + destAddr);
            ds.send(packet);
            System.out.println("successfully sent packet to " + destAddr);
            System.out.println("application will now do some unrelated work for a second");
            doSomeOtherWork();
            // send again
            System.out.println("application will now again attempt to send a packet to " + destAddr);
            ds.send(packet);
            System.out.println("(again) successfully sent packet to " + destAddr);

        }
    }

    private static void doSomeOtherWork() throws Exception {
        Thread.sleep(1000);
    }
}

Save this code to a LocalNetworkTest.java file and then go to the Terminal (i.e. macos Terminal app) and just do:

java LocalNetworkTest.java

You should see the output as follows

attempting to send a packet to /ff01:0:0:0:0:0:0:1
successfully sent packet to /ff01:0:0:0:0:0:0:1
application will now do some unrelated work for a second
application will now again attempt to send a packet to /ff01:0:0:0:0:0:0:1
Exception in thread "main" java.net.NoRouteToHostException: No route to host
	at java.base/sun.nio.ch.DatagramChannelImpl.send0(Native Method)
	at java.base/sun.nio.ch.DatagramChannelImpl.sendFromNativeBuffer(DatagramChannelImpl.java:914)
	at java.base/sun.nio.ch.DatagramChannelImpl.send(DatagramChannelImpl.java:871)
	at java.base/sun.nio.ch.DatagramChannelImpl.send(DatagramChannelImpl.java:798)
	at java.base/sun.nio.ch.DatagramChannelImpl.blockingSend(DatagramChannelImpl.java:857)
	at java.base/sun.nio.ch.DatagramSocketAdaptor.send(DatagramSocketAdaptor.java:178)
	at java.base/java.net.DatagramSocket.send(DatagramSocket.java:593)
	at LocalNetworkTest.main(LocalNetworkTest.java:17)

Notice how the first attempt gives an impression that the send has succeeded then the second attempt (after a delay of 1 second within that program) ends up with that "no route to host" exception.

Like I noted earlier, there are no pop-ups or notifications asking for permission to allow local network access. Plus, this was launched from the Terminal app, yet the local network restrictions seem to be applied which goes against what the documentation here https://developer.apple.com/documentation/technotes/tn3179-understanding-local-network-privacy states:

macOS automatically allows local network access by:

... Command-line tools run from Terminal ...

While this seemingly is a bug somewhere in the macOS network stack, UDP apps absolutely need to be coded to expect packets to vanish with no errors reported to the sending app.

TCP connections will report an error here, but that certainty of feedback is not the case with UDP.

And I'd expect EHOSTUNREACH only after some packet tried the path.

Put differently, I'd re-code the app to implement its own recovery logic. Don't get the expected feedback? Resend, or give up, as app appropriate. Or apps using UDP multicast usually assume the next multicast will catch up any clients that missed the earlier multicast packet.

Hello Hoffman,

While this seemingly is a bug somewhere in the macOS network stack, UDP apps absolutely need to be coded to expect packets to vanish with no errors reported to the sending app.

Right, that part about UDP is understood.

The reason I opened this thread is to have this behaviour analyzed to be certain that this isn't due to bug(s) in the local network restriction implementation, which is new to 15.x macos. If it indeed is a bug, it would be good to have it addressed to prevent hard to debug issues (like we are currently having).

sendto() system call - Nondeterministic "No route to host" due to local network restrictions
 
 
Q