What Actually Happens When You Open a TCP Socket in Linux

Most developers interact with networking code every day. You spin up an Express server, write a Go HTTP handler, or configure Nginx. Somewhere near the base of the call stack, a library executes a system call that looks like this:

int fd = socket(AF_INET, SOCK_STREAM, 0);

To user-space code, a socket looks like a standard Unix file descriptor. You write bytes into it, and you read bytes out of it.

Underneath that simple integer, the Linux kernel is juggling complex data structures: slab-allocated protocol control blocks, dual-queue connection backlogs, sliding-window packet buffers, and red-black tree event pollers.

Here is the exact lifecycle of a TCP socket in Linux, from the initial system call to connection termination.

1. The Syscall: struct socket vs struct sock

When user space issues socket(AF_INET, SOCK_STREAM, 0), control traps into the kernel via sys_socket().

Linux does not represent a socket with a single struct. It separates the virtual filesystem interface from the networking protocol engine across two distinct layers:

[ User Space File Descriptor (e.g. fd = 3) ]
                     │
                     ▼
[ VFS Layer: struct socket (BSD socket, inode in sockfs) ]
                     │
                     ▼ (sk pointer)
[ Network Layer: struct sock / struct tcp_sock (Protocol state, buffers, timers) ]

The VFS Layer: struct socket

The VFS layer manages the generic file abstraction. The kernel creates an anonymous inode mounted on an internal pseudo-filesystem called sockfs. It allocates an entry in the process's file descriptor table (current->files->fdt) and binds the integer file descriptor to a struct file where f_op = &socket_file_ops.

The Network Layer: struct sock and struct tcp_sock

Beneath the generic VFS wrapper sits the protocol control block. For a TCP socket, Linux allocates a struct tcp_sock (which wraps struct inet_sock and struct sock).

This struct holds:

• Protocol state machine status (TCP_CLOSE, TCP_LISTEN, TCP_ESTABLISHED).
• Monotonic sequence numbers: snd_nxt (next byte to send), rcv_nxt (next byte expected).
• Sliding window variables: Advertised receive window, congestion window (snd_cwnd), slow start threshold (snd_ssthresh).
• Kernel timers: Retransmission timer (RTO), keepalive timer, delayed ACK timer.
• Pluggable congestion control state pointing to tcp_congestion_ops (such as CUBIC or BBR).

The integer fd returned to your application is just an index in your process table. The heavy networking logic lives entirely inside struct tcp_sock.

2. Socket Buffers: Why Sockets Don't Store Flat Bytes

A common misconception is that a socket allocates a simple flat byte buffer where write() appends data and read() consumes data.

Linux networking revolves around struct sk_buff (Socket Buffer, or "skb"). An sk_buff is a metadata envelope wrapping raw packet memory.

+-------------------------------------------------------+
| struct sk_buff (head, data, tail, end pointers)       |
+-------------------------------------------------------+
   |          |                     |             |
   v          v                     v             v
[ Headroom | Protocol Headers | Data Payload | Tailroom ]

By manipulating data and tail pointers (skb_push, skb_pull, skb_put), the kernel can prepend TCP, IP, and Ethernet headers or strip them away without copying byte payloads in memory.

Inside struct sock, there are two linked queues of sk_buff structs:

1. Send Queue (write_queue): Holds outbound segments waiting to be serialized, passed to the network interface card (NIC) ring buffer, or retained until the remote peer sends an ACK.
2. Receive Queue (receive_queue): Holds incoming, in-order packets that have cleared checksum and protocol checks, ready for user-space recv() or read().

Kernel Memory Limits and Autotuning

The memory allocated to these queues is bounded by kernel sysctl parameters:

sysctl net.ipv4.tcp_rmem
# Output: 4096  131072  6291456 (min, default, max in bytes)

sysctl net.ipv4.tcp_wmem
# Output: 4096  16384   4194304 (min, default, max in bytes)

Linux does not lock receive buffers to static sizes. Through dynamic buffer autotuning (tcp_rcv_space_adjust()), the kernel monitors connection throughput and latency, scaling the receive window to match the Bandwidth-Delay Product (BDP) of the link.

3. Server Setup: bind(), listen(), and the Dual Queues

When building a server, creating the socket is only the first step. You then invoke bind() and listen().

bind(listen_fd, (struct sockaddr *)&addr, sizeof(addr));
listen(listen_fd, 128);

bind() registers the socket inside the kernel's hash table (inet_hashinfo.listening_hash), reserving the port on the specified IP address.

listen() transitions the socket state to TCP_LISTEN and allocates two separate kernel queues:

                      Client SYN
                          │
                          ▼
             +---------------------------+
             |        SYN Queue          |  <-- tcp_max_syn_backlog
             |   (Half-Open / SYN_RECV)  |      (Holds struct request_sock)
             +---------------------------+
                          │
                   Client Final ACK
                          │
                          ▼
             +---------------------------+
             |       Accept Queue        |  <-- min(backlog, somaxconn)
             | (Completed / ESTABLISHED) |      (Ready for accept())
             +---------------------------+
                          │
                    user accept()
                          │
                          ▼
                 New client_fd (FD 4)

1. The SYN Queue (Half-Open Connection Queue)

When an incoming SYN packet arrives from a client, the kernel creates a lightweight struct request_sock and puts it in the SYN Queue. The server sends back a SYN-ACK and waits in TCP_SYN_RECV.

• Queue limit: Controlled by net.ipv4.tcp_max_syn_backlog.
• SYN Flood Protection: If malicious actors flood the server with SYNs without sending final ACKs, this queue fills up. If net.ipv4.tcp_syncookies = 1, Linux stops allocating memory in the SYN Queue entirely. Instead, it encodes the connection state, MSS, and timestamp directly into the 32-bit initial sequence number of the SYN-ACK. When the legitimate client replies with an ACK containing ack_seq = cookie + 1, the kernel reconstructs the socket on the fly without state memory.

2. The Accept Queue (Completed Connection Queue)

Once the client returns the final ACK, the three-way handshake is complete. The kernel moves the connection to TCP_ESTABLISHED and places it in the Accept Queue.

• Queue limit: Bounded by min(backlog_parameter, net.core.somaxconn).
• Queue Overflow: If your application is blocked and fails to call accept() fast enough, the Accept Queue fills up. By default, Linux silently drops incoming final ACKs. The client assumes the ACK was lost and retransmits it, giving the application a grace period to catch up. If you set net.ipv4.tcp_abort_on_overflow = 1, the kernel immediately sends a RST to the client instead.

4. What accept() Actually Does: The 4-Tuple Demuxing Illusion

Beginners often wonder: if a server listens on port 8080 and handles 10,000 connected clients, does it open 10,000 ports?

No. A network port is not a physical doorway or hardware channel. It is simply a numerical field inside the TCP header.

When your application executes:

int client_fd = accept(listen_fd, (struct sockaddr *)&client_addr, &addr_len);

The kernel performs three distinct actions:

1. It dequeues the oldest connection from the Accept Queue.
2. It promotes the lightweight request_sock into a full-fledged struct tcp_sock.
3. It allocates a new file descriptor (for example, client_fd = 4) and points it to this new socket.

The listening socket (listen_fd = 3) remains untouched on port 8080, continuing to process incoming handshakes.

NIC Layer: Ingress Packet
        │
        ▼
Extract 4-Tuple: (Src IP, Src Port, Dst IP, Dst Port)
        │
        ├─► Lookup in inet_hashinfo.ehash (Established Sockets)
        │         └─► Match found: Dispatch skb to client_fd (FD 4)
        │
        └─► No match: Lookup in inet_hashinfo.listening_hash
                  └─► Match found: Dispatch SYN to listen_fd (FD 3)

When an Ethernet frame hits the network card:

1. The NIC triggers an interrupt, and the kernel SoftIRQ (NET_RX_SOFTIRQ) extracts the TCP 4-tuple: (Source IP, Source Port, Destination IP, Destination Port).
2. Linux computes a hash of this 4-tuple and checks the established socket hash table (inet_hashinfo.ehash).
3. If an established connection matches, the packet is pushed directly to client_fd's receive queue.
4. If no established match exists, it checks listening_hash to see if a listening socket wants the SYN.

5. Why epoll Scales: O(1) Event Delivery

If you have 50,000 active connections, how does the kernel tell your program which sockets have data ready to read?

The Legacy Problem: select() and poll()

In older systems, you passed an array of 50,000 file descriptors to select() or poll(). The kernel had to iterate through all 50,000 descriptors, register callbacks on each socket's wait queue, and scan them all again when waking up. This O(N) scanning caused severe CPU bottlenecks under high concurrency.

The epoll Architecture

epoll solves this by maintaining persistent state inside the kernel across two data structures:

1. Red-Black Tree (rbr): Stores all file descriptors being monitored. Adding, modifying, or removing a watched descriptor with epoll_ctl() takes O(log N) time.
2. Ready List (rdllist): A doubly-linked list containing only the descriptors that have unhandled I/O events.

[ NIC Packet Arrives ]
         │
         ▼
[ SoftIRQ processes TCP Packet ]
         │
         ▼
[ sk_buff added to socket receive queue ]
         │
         ▼
[ Socket Wait Queue triggers callback: ep_poll_callback() ]
         │
         ▼
[ Adds epitem to epoll Ready List (rdllist) ]
         │
         ▼
[ epoll_wait() unblocks and returns ONLY active descriptors: O(Events) ]

When data arrives for a socket, the kernel network stack places the sk_buff into the socket's receive queue and calls the socket's wait queue callback: ep_poll_callback().

This callback immediately adds the corresponding descriptor item (struct epitem) to the epoll instance's Ready List.

When user space calls epoll_wait(), the kernel does not scan 50,000 descriptors. It simply transfers the contents of the Ready List to the user's event buffer. If 5 out of 50,000 sockets have data, epoll_wait() does work proportional to 5, achieving O(1) event delivery relative to total idle connections.

6. Teardown, TIME_WAIT, and Port Reuse Traps

When an application closes a TCP connection, the handshake does not instantly evaporate.

Endpoint A (Active Close)                   Endpoint B (Passive Close)
         │                                               │
         │──────────────── FIN ─────────────────────────►│  (FIN_WAIT_1 -> CLOSE_WAIT)
         │◄─────────────── ACK ──────────────────────────│  (FIN_WAIT_2)
         │                                               │
         │◄─────────────── FIN ──────────────────────────│  (LAST_ACK)
         │──────────────── ACK ─────────────────────────►│  (TIME_WAIT -> CLOSED)
         │
     [ 2 * MSL = 60s ]
         │
     (CLOSED)

The side that initiates the close (the active closer) transitions through FIN_WAIT_1 and FIN_WAIT_2, landing in TIME_WAIT after sending the final ACK.

In Linux, a socket remains in TIME_WAIT for 60 seconds (calculated as 2 * MSL, where Maximum Segment Lifetime is 30 seconds).

Why TIME_WAIT Is Mandatory

1. Preventing Stray Duplicate Segments: If old, delayed packets from the closed connection wander through the internet and arrive later, TIME_WAIT ensures they are discarded rather than corrupting a new connection reusing the exact same 4-tuple.
2. Guaranteed Final ACK Delivery: If the final ACK sent by Endpoint A is dropped in transit, Endpoint B will retransmit its FIN. Because Endpoint A is in TIME_WAIT, it can resend the ACK. If Endpoint A had immediately closed and freed the port, it would respond to the retransmitted FIN with an unexpected RST.

The SO_REUSEADDR and SO_REUSEPORT Flags

When restarting a server, you might encounter bind: Address already in use. This happens because old connection endpoints are still lingering in TIME_WAIT on that port.

• SO_REUSEADDR: Tells the kernel that a new listening socket can bind to a local port even if existing sockets on that port are in TIME_WAIT. This is essential for rapid server restarts.
• SO_REUSEPORT: Added in Linux 3.9, this allows multiple completely independent processes or threads to bind to the exact same IP and port combination. The Linux kernel hashes incoming connection 4-tuples and distributes them evenly across the listening sockets, eliminating multi-threaded accept lock contention (the classic thundering herd problem).

7. Kernel Observability: How to Inspect Your Sockets

To inspect socket health and queue depths on a live Linux system, use modern tools like ss (Socket Statistics) instead of obsolete netstat.

1. Inspecting Listening Queues

ss -lnt

Output format for listening sockets:

• Send-Q: The maximum Accept Queue limit (backlog).
• Recv-Q: The current number of established connections waiting to be accept()ed.

If Recv-Q is consistently close to Send-Q, your application is CPU-bound or blocking, and incoming connections are risking packet drops.

2. Checking Dropped SYN and Accept Packets

netstat -s | grep -i listen

Look for lines like:

• times the listen queue of a socket overflowed (Accept Queue full).
• SYNs to LISTEN sockets dropped (SYN Queue or Accept Queue full).

Summary Checklist

Knowing what happens inside the kernel transforms networking from an abstract black box into predictable systems engineering. When latencies spike or connection queues overflow, the answers are written directly in the kernel data structures.