Skip to content
Published on

Seeing TCP Congestion Control — CUBIC, BBRv3, and a gVisor Netstack Running in Your Browser

Share
Authors

Introduction — Two TCP Stacks Running in a Browser Tab

On July 28, 2026, a link titled Simulating TCP loss and congestion in browser using Go/WASM went up on Hacker News. Open it and you land on a page called ccsim, and what runs there is not a demo animation. The description in the source repository is precise — it stands up two copies of gVisor's userspace TCP/IP stack (gvisor.dev/gvisor/pkg/tcpip), one as the sender and one as the receiver, wedges a token-bucket bottleneck link model between them, and builds that entire event-driven simulator to WebAssembly to run it inside a browser worker.

The motivation is written up on the authors' engineering blog. They use the gVisor netstack to inject packets in their tunneling infrastructure, and probes kept dying on a co-founder's flaky home line. They suspected CUBIC does badly on a lossy link, and trying to confirm that led them to implement BBRv3 on top of netstack themselves and to build a verification harness for it. The simulator is a by-product of that harness.

The reason not to dismiss this as a browser demo is in the repository's verification document. The CUBIC window growth curve fits RFC 9438's cubic function with C=0.410 and R²=0.9999, the loss response function lands within 0.68 to 0.95 times the spec value, and RED's marking curve is compared against the exact ramp probability mass function using a chi-squared test. On top of that, the sample output streams from the native build and the WASM build are byte-for-byte identical. It is a teaching aid and a regression test at the same time.

This post uses the simulator as a hook but puts the real value in the algorithms underneath it. Congestion control has been solving the same problem for more than 30 years, and what we ultimately need to know is which symptoms that sawtooth on the screen turns into on a real server.

The Skeleton — slow start, congestion avoidance, fast retransmit

A TCP sender watches two windows at once. One is the receive window (rwnd) the receiver advertises, "how much the far side can handle"; the other is the congestion window (cwnd), "how much the sender estimates the network can handle." What it can actually put on the wire is the smaller of the two. A congestion control algorithm is, in the end, a set of rules about when and by how much to raise and lower cwnd.

Slow start, despite the name, is exponentially fast. When a connection opens, cwnd is set to the initial window (on modern Linux, 10 MSS per RFC 6928), and every arriving ACK raises cwnd by 1 MSS. Since one ACK releases two new segments, cwnd doubles every RTT. It is a search meant to approach path capacity in close to logarithmic time from a state of knowing nothing about it. This phase continues until ssthresh is reached or a loss is detected.

Congestion avoidance comes next. Now that the sender considers itself near capacity, the growth rate drops to roughly 1 MSS per RTT. On a loss, cwnd is cut multiplicatively. This AIMD (additive increase, multiplicative decrease) structure is the core of what saved the internet from collapse — when several flows share the same bottleneck, AIMD converges toward fairness, whereas multiplicative increase does not.

Fast retransmit is the mechanism for catching a loss far sooner than the retransmission timeout (RTO) would. When a receiver gets an out-of-order segment, it repeatedly ACKs the last contiguously received point. When the sender gets three duplicates of the same ACK, it retransmits that segment immediately instead of waiting for the RTO, enters fast recovery, cuts cwnd roughly in half, and keeps sending. Waiting for the RTO would have burned hundreds of milliseconds at minimum.

One thing has to be flagged here. The paragraphs above are the textbook Reno account, and what Linux actually does for loss detection today is RACK-TLP (RFC 8985). Instead of counting duplicate ACKs, it compares each segment's transmit time against the arrival order confirmed by SACK, and declares a loss when something sent after this segment has already arrived and the reordering window has passed as well. The three-duplicate-ACK rule was weak against reordering and helpless against tail loss (the last segment disappearing, so no duplicate ACKs are generated at all). ccsim runs with RACK/TLP enabled too.

What CUBIC Optimizes For

CUBIC, the default in the Linux, Windows, and Apple stacks, was standardized as RFC 9438 in August 2023 (obsoleting RFC 8312). As the name says, it defines window growth as a cubic function.

W_cubic(t) = C * (t - K)^3 + W_max

  t     : elapsed time since the last congestion event (seconds)
  W_max : window size just before the congestion event
  C     : scaling constant (default 0.4)
  K     : cbrt(W_max * (1 - beta) / C),   beta = 0.7

The shape of this curve is the entire design intent. Right after a loss it recovers concavely and fast up to near W_max, flattens out almost completely once it reaches W_max and lingers around there probing, and if nothing happens it accelerates convexly above that point to go looking for new capacity. The decrease factor is 0.7 rather than Reno's 0.5, so a single loss also costs less.

As for what it optimized, there are two things.

First, utilization on high-BDP links. Reno's 1-MSS-per-RTT increase is hopelessly slow on a path with large bandwidth and long delay. CUBIC's increase is a function of elapsed time, so it is not tied directly to RTT.

Second, and for exactly that reason, RTT fairness. In the Reno family, a flow with a shorter RTT increases more over the same span of time and takes more bandwidth. Being a function of time, CUBIC has a much smaller bias here.

What CUBIC gave up in exchange is clear. CUBIC's congestion signal is still packet loss and nothing else. It does not look at how full the queue is or how much delay has grown. It pushes up until there is a loss, and cuts when there is one. Both of the symptoms below are derived from that one sentence.

What BBR Looks At Instead — a Path Model, Not Loss

BBR changes the question. Instead of "when does loss happen," it continuously estimates "what are this path's bottleneck bandwidth and minimum round-trip delay," and tries to hold near their product (BDP). The on-screen explanation in ccsim compresses the contrast well — 1 BDP is the optimum, and CUBIC finds that point by crossing over into loss, while BBR estimates the path and stays near it.

This optimum is also something Kleinrock worked out back in 1979. If in-flight is below BDP the link idles; if it is above BDP, the excess all piles up in the bottleneck queue and only adds delay while throughput stays the same. The problem is that you cannot measure bandwidth and minimum delay at the same time — measuring peak bandwidth requires building a queue, and measuring minimum delay requires draining one. So BBR measures the two values in alternation.

The BBRv3 that ccsim implements follows the state machine in draft-ietf-ccwg-bbr (currently revision 06, status Experimental). Carrying over the constants listed in the repository's docs, it looks like this.

  • Startup — pacing gain 710/256 (about 2.77), cwnd gain 2.0. Pushes up until the bandwidth estimate stops increasing.
  • Drain — pacing gain 88/256 (about 0.34). Drains the queue that Startup built up.
  • ProbeBW — a cycle of DOWN (232/256) → CRUISE (1.0) → REFILL (1.0) → UP (1.25). Most of the time is spent here, and only during the UP phase does it push a little harder to check whether bandwidth has grown.
  • ProbeRTT — cwnd gain 0.5 for about 200ms. Scheduled at least 5 seconds apart, it drains the queue to refresh the min_rtt filter. min_rtt itself is kept over a 10-second window.

On top of this, what BBRv3 adds relative to v1 is explicit response bounds for loss and ECN. Ceilings such as inflight_hi, inflight_lo, and bw_lo mean that when the path actually returns losses or CE marks, it backs off instead of pushing its model through regardless. This is the answer to the criticism that BBRv1 was excessively aggressive when coexisting with CUBIC.

And the thing that inevitably comes along if you want to run BBR for real is pacing. Opening cwnd and blasting bursts blows up the bottleneck queue instantaneously, so packets have to be spread evenly over the time axis at the estimated bandwidth. On Linux the fq qdisc or in-kernel pacing does this job. ccsim puts it not inside the CC but in a send integration layer, releasing packets through a virtual clock timer in quanta of min(pacing_rate * 1ms, 64KB).

Three Symptoms Engineers Actually See

That was the theory; what follows is the shape these take when they arrive as a ticket. Carrying the scenario measurements from the ccsim repository over as they are makes the three symptoms visible as numbers.

ScenarioConditionsResult
cubic-single100Mbps, no loss96.5 Mbps, 3 cwnd reductions, 0 RTOs
random-loss1% random lossCUBIC 3.0 Mbps vs BBRv3 57 Mbps
bufferbloatdeep taildrop buffer, base RTT 30msCUBIC steady-state srtt 1,070ms vs BBRv3 32ms
ecn-codelCoDel + ECN654 CE marks, 0 drops and retransmits, srtt at most 2.1x of base
rate-stepstepped bandwidth change24 Mbps delivered immediately, 1.07x the new BDP two cycles after the max-bw filter
fairnessCUBIC and BBR coexisting on one 100Mbps linklink 96% utilized, split about 33% vs 67%

Symptom 1 — throughput collapse on a lossy link. At 1% random loss, CUBIC slumps to 3.0 Mbps. On a 100Mbps link that is 3%. This is not a bug but a direct consequence of a loss-based design, and the Mathis approximation predicts it exactly.

throughput  ~  MSS / (RTT * sqrt(p))  * C

  MSS 1448B, RTT 40ms, p = 0.01, C ~ 1.22
  -> 1.22 * 1448 * 8 / (0.04 * 0.1)  ~  3.5 Mbps

What matters in the formula is that the loss rate enters under a square root. When loss rises tenfold from 0.1% to 1%, throughput falls not to a third but to about one 3.16th. And CUBIC cannot distinguish whether that loss came from congestion or from wireless interference or line quality. That is the classic reason a flow crossing a wireless segment or a long-haul circuit is inexplicably slow, and it is also why BBR gets 57 Mbps — BBR does not shake its model much on loss as long as the delivery rate estimate holds up.

Symptom 2 — bufferbloat. On a bottleneck with a deep buffer, CUBIC's steady-state srtt is 1,070ms. The path's actual round-trip delay is 30ms. A full second of that is queueing time. And here losses barely happen at all — because the buffer is deep, the excess is absorbed as delay rather than as drops. So CUBIC concludes "nothing is wrong" and stabilizes in that state. Under the same conditions BBRv3 holds 32ms. With a base RTT of 30ms, that means the queue is effectively empty.

Symptom 3 — making the buffer bigger makes it worse. This falls out automatically from combining the two paragraphs above. The steady-state queue occupancy of a loss-based CC is decided by the buffer size. Double the buffer and steady-state delay roughly doubles too. The intuition that "we see packet drops, so let's add buffer" only converts drops into delay, and from the point of view of interactive traffic that is a worse trade. Router buffers getting cheap and this mistake spreading across the whole internet is the history of the bufferbloat problem.

The prescription is queue discipline, not buffer size. The ecn-codel row of the table is the picture of it — CoDel watches queueing delay and marks CE preemptively, so there are zero drops and zero retransmits while srtt is pressed down to at most 2.1x of base. It replaced the loss signal with a delay signal, and on Linux fq_codel or cake does this job. On a consumer router this is effectively the only practical solution.

The fairness number in the last row also matters in practice. When CUBIC and BBR share a link, it splits 33% to 67%. The link is 96% full so nothing is wasted, but it is not even either. Even granting that BBRv3 is gentler than v1, this means converting only part of a server fleet to BBR leaves the remaining CUBIC flows worse off. Better to switch by domain than in fragments.

How to See cwnd and Retransmits on a Real Host

Every value the simulator draws so nicely can also be seen on a Linux host. No instrumentation code required.

# Current default algorithm and the list of available ones
sysctl net.ipv4.tcp_congestion_control
sysctl net.ipv4.tcp_available_congestion_control

# To use BBR you need the module and a pacing qdisc together
sudo modprobe tcp_bbr
sudo sysctl -w net.ipv4.tcp_congestion_control=bbr
tc qdisc show dev eth0        # check whether it is fq or fq_codel

# Per-socket state: cwnd, ssthresh, rtt, retransmits, delivery rate, pacing rate all show up here
ss -tin state established
#   cubic wscale:7,7 rto:236 rtt:35.6/2.1 mss:1448 cwnd:73 ssthresh:52
#   bytes_sent:... retrans:0/14 delivery_rate:76.4Mbps busy:... pacing_rate:...

# Only a specific destination
ss -tin dst 203.0.113.10

# Host-wide retransmit counters (deltas only)
nstat -az | grep -Ei 'TcpRetransSegs|TcpExtTCPLostRetransmit|TcpExtTCPTimeouts|TcpExtTCPFastRetrans'

The three most practically important values in ss -tin output are these.

  • cwnd — in packets. Divide cwnd * mss by rtt and you get the ceiling throughput at that instant. Compare it against delivery_rate and you can separate the application failing to fill the pipe (app-limited) from the network holding it back.
  • retrans:X/Y — the first is currently outstanding retransmits, the second is cumulative. What percentage the second number is of bytes_sent is the loss rate p from the previous section.
  • rtt:A/B — A is the smoothed RTT, B is the variance (mdev). If A is far above the path's minimum RTT while retransmits are near zero, that is the signature of bufferbloat.

If you want to sample periodically and watch the cwnd sawtooth with your own eyes, this much is enough.

# Pull out only cwnd/rtt/retrans every 200ms and keep it as a time series
while :; do
  date +%s.%N | tr -d '\n'
  ss -tin dst 203.0.113.10 | tr '\n' ' ' \
    | grep -oE 'cwnd:[0-9]+|rtt:[0-9.]+|retrans:[0-9]+/[0-9]+' | tr '\n' ' '
  echo
  sleep 0.2
done

To go down to the kernel event level, the tracepoints are already in place.

# Watch congestion state transitions and retransmits as events
sudo bpftrace -e '
tracepoint:tcp:tcp_retransmit_skb { @retrans[comm] = count(); }
tracepoint:tcp:tcp_cong_state_set { @state[args->cong_state] = count(); }'

# Building a reproduction: 1% loss + 20ms delay + shallow queue
sudo tc qdisc replace dev eth0 root netem loss 1% delay 20ms limit 100
# Reproducing bufferbloat: make the queue very deep
sudo tc qdisc replace dev eth0 root netem delay 20ms limit 10000
# Confirming the prescription: swap in AQM
sudo tc qdisc replace dev eth0 root fq_codel

Set up loss with netem and send the same file over CUBIC and over BBR in turn, and within 30 seconds you will feel what kind of difference the 3.0 Mbps versus 57 Mbps in the previous section's table is. The overall internal structure of the Linux network stack is written up separately in Linux networking internals, and TCP's own behavior in TCP deep dive.

What the Simulator Cannot Show You

As a teaching aid, ccsim's strengths are clear. It runs a real protocol implementation, so the SACK scoreboard, RACK reordering decisions, and ECN echo are the real thing rather than approximations, and it is deterministic, so the same seed produces the same result. Move a slider and the parameter changes while the simulation is still running, so how differently the two algorithms react when bandwidth suddenly drops is visible on a single screen.

At the same time the boundaries of the simulation are just as clear.

  • There are one or two flows. A real bottleneck has thousands of flows, and most of them are short enough that they never even leave slow start. Steady-state behavior accounts for a smaller share of internet traffic than you would think.
  • There is one bottleneck and it is fixed. On a real path the bottleneck moves, and the capacity of a wireless segment fluctuates second by second.
  • There are no middleboxes. Gear that strips ECN bits, firewalls that break SACK, and proxies that terminate TCP on your behalf are all common on the real internet. A large share of the cases where BBR is turned on but the expected effect never appears live right here.
  • There is no CPU or interrupt handling. GRO, TSO, NIC queues, and which core handles softirq shift the delay distribution substantially in reality.

So what you should take from the simulator is not "BBR gets N Mbps in our environment" but which signals this algorithm reacts to and which ones it does not. With that sense in hand, what you can read out of a single line of ss -tin output changes noticeably.

Closing — Congestion Control Is Still an Estimation Problem

To summarize.

  • cwnd is the sender's estimate of network capacity. The difference between algorithms is a difference in what they take as evidence for updating that estimate.
  • CUBIC uses loss and only loss as evidence. So it mistakes wireless loss for congestion, and in front of a deep buffer it never receives the evidence and stabilizes with the queue full.
  • BBR builds a path model out of delivery rate and minimum RTT and aims near BDP. BBRv3 adds loss and ECN response bounds on top of that and improves coexistence, but the split when sharing with CUBIC is still not even.
  • Growing the buffer is a trade that converts drops into delay. The answer is not queue size but AQM such as fq_codel.
  • All of this can be reproduced and observed on your own host with three tools: ss -tin, nstat, and tc netem.

When you get a "it's slow" ticket after not having looked at congestion control in a long time, the instinct is usually to suspect bandwidth. In practice it is more often that cwnd is pinned by the loss rate, or that RTT is inflated by a queue. In both cases adding bandwidth improves nothing.

References