How to increase "receive buffer size"

Question

Created Jul ’24

Replies 1

Boosts 0

Views 1.1k

Participants 2

I am trying to host a Flask API on docker/ Podman (tried on both). It works as expected when directly run on macOS using python3 command. But when I try to dockerize it, it throws an error and some logs like

2024-07-30 03:33:20,661 - INFO - Tunnel Output: 2024-07-30T03:32:58Z INF Starting metrics server on 127.0.0.1:46585/metrics
2024-07-30 03:33:21,669 - INFO - Tunnel Output: 2024/07/30 03:32:58 failed to sufficiently increase receive buffer size (was: 208 kiB, wanted: 7168 kiB, got: 416 kiB). See https://github.com/quic-go/quic-go/wiki/UDP-Buffer-Sizes for details.
2024-07-30 03:33:21,671 - INFO - New Cloudflare URL: https://github.com/quic-go/quic-go/wiki/UDP-Buffer-Sizes
2024-07-30 03:33:21,672 - INFO - My API URL: http://127.0.0.1:5000
2024-07-30 03:33:21,672 - INFO - My Cloudflare URL: https://github.com/quic-go/quic-go/wiki/UDP-Buffer-Sizes

I tried following but no success (by changing "/etc/sysctl.conf":

kern.ipc.maxsockbuf=16777216
net.inet.tcp.win_scale_factor=8
net.inet.tcp.autorcvbufmax=33554432
net.inet.tcp.autosndbufmax=33554432
net.inet.udp.recvspace=8388608
net.inet.udp.maxdgram=8388608

I tried

def set_socket_buffers(sock, recv_buf_size, send_buf_size):
    # Set the receive buffer size
    sock.setsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF, recv_buf_size)
    # Set the send buffer size
    sock.setsockopt(socket.SOL_SOCKET, socket.SO_SNDBUF, send_buf_size)

    # Verify the buffer sizes
    actual_recv_buf_size = sock.getsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF)
    actual_send_buf_size = sock.getsockopt(socket.SOL_SOCKET, socket.SO_SNDBUF)

    logging.info(f"Requested receive buffer size: {recv_buf_size} bytes")
    logging.info(f"Actual receive buffer size: {actual_recv_buf_size} bytes")
    logging.info(f"Requested send buffer size: {send_buf_size} bytes")
    logging.info(f"Actual send buffer size: {actual_send_buf_size} bytes")

if __name__ == "__main__":
    print("Starting the main process...")

    # Define the path to the log file
    log_file_path = "logs.log"

    # Clear the log file before starting logging
    clear_log_file(log_file_path)

    # Configure logging
    logging.basicConfig(
        filename=log_file_path,
        level=logging.DEBUG,
        format="%(asctime)s - %(levelname)s - %(message)s",
    )
    logging.info("Starting the main process...")

    # Define the desired buffer sizes
    recv_buf_size = 8388608  # 8 MB
    send_buf_size = 8388608  # 8 MB

    # Create a UDP socket
    udp_sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)

    # Set the socket buffers
    set_socket_buffers(udp_sock, recv_buf_size, send_buf_size)

    main()

On directly running (using python3 command), the values set as expected but when running on Docker logs come out as:

2024-07-30 03:32:50,987 - INFO - Requested receive buffer size: 8388608 bytes
2024-07-30 03:32:50,987 - INFO - Actual receive buffer size: 425984 bytes
2024-07-30 03:32:50,987 - INFO - Requested send buffer size: 8388608 bytes
2024-07-30 03:32:50,987 - INFO - Actual send buffer size: 425984 bytes

So, I want to know how to increase UDP Receive buffer size for dockerized applications.

Thanks!

Boost

Answer 1

DTS Engineer OP

Apple

Jul ’24

Recommended

It works as expected when directly run on macOS using python3 command.

Cool.

But when I try to dockerize it, it throws an error and some logs

Ah, um, that’s a Linux problem, right?

I don’t have a lot of experience with Docker, so the following is based on me literally reading the Wikipedia page O-: However, I see it breaking down in two ways:

If you’re running Docker on your Mac then it runs the container as a virtual machine. That means that the container is running its own networking stack, so you have to configure it using techniques appropriate for the container’s OS. And presumable that means Linux.
If you’re running Docker on Linux, the mechanics change a little but, regardless, it’s Linux both inside the container and in the host machine.

I think you might have more luck asking this in a forums more focused on Docker, or just Linux itself.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

1