Routines Excluded
OpenBSD
Bijan — Fri 15 December 2017

OpenBSD's Network stack (PART 1)

What I learned from reading the OpenBSD's network stack source code

The OpenBSD Network Stack Graph

Since OpenBSD Network Stack Internals by Claudio Jeker at AsiaBSDCon 2008, there's a noticeable changes in OpenBSD network stack. obviously the removal of using softnet task queue for forwarding IP packets which is now done by almost a complete run (passing packets from Incoming Interface to outgoing Interface). For me to understand the processing of incoming packets, I did a quick glance of OpenBSD's network stack and did a little graph below:

The journey

Stern Warning: This is my understanding of OpenBSD's network stack internals (by reading the source code) and this is by no mean a case study. So please feel free to fact check and let me know where I miss-understood.

The above graph, shows the input of the packet to network stack by the ethernet device driver to the network stack. This is done at the lowest level and is the lowest layer which is Layer 1. At the next Layer (Layer 2), ether_input validates the ethernet header and by the value of it's type, passes the packet to the Layer 3. This used to be done by enqueuing the packet (in our case, IP Packet) into a queue and schedule a software interrupt to process the packet later, but apparently with the new changes in OpenBSD, the queuing process is removed, letting forwarding the packets without scheduling any interrupts.

Now in ip_input_if, the IP packet's header is ispected to decide if it's destined to us or should be forwared. if it's not destined locally, ip_forward will do the forwarding the packet. if the destincation is not reachable, it produces an ICMP Unreachable message and sends it to the sender, otherwise the ip_output will send the packet to the right interface. Now the packet enter's the Layer 2 again and will be enqueued into outgoing interface to be send out later.

But, if the IP packet is destined for us, it will be enqueued by niq_enqueue. This process is done at the Layer 3 and later by scheduling an interrupt, the packet will be dequeued by the if_netisr and be passed to the propper internet protocol by ip_deliver. This is where the packet leaves the Layer 3 and enters the Layer 4 of our network stack.

Now, deppending on the payload and opened sockets on the user land and a few other conditions (like if the socket is spliced), the packet is processed differently. But typically it is done by copying the mbuf packet into the right socket's received buffer (if it's not FULL) and will be notified by sorwakeup.

Other related blog posts

If you find these topics interesting or you're a newbie like me, you should read the followings as well. Hope to be useful: