In the Linux kernel, the following vulnerability has been resolved:
ipv6: Fix soft lockups in fib6selectpath under high next hop churn
Soft lockups have been observed on a cluster of Linux-based edge routers
located in a highly dynamic environment. Using the bird
service, these
routers continuously update BGP-advertised routes due to frequently
changing nexthop destinations, while also managing significant IPv6
traffic. The lockups occur during the traversal of the multipath
circular linked-list in the fib6_select_path
function, particularly
while iterating through the siblings in the list. The issue typically
arises when the nodes of the linked list are unexpectedly deleted
concurrently on a different core—indicated by their 'next' and
'previous' elements pointing back to the node itself and their reference
count dropping to zero. This results in an infinite loop, leading to a
soft lockup that triggers a system panic via the watchdog timer.
Apply RCU primitives in the problematic code sections to resolve the issue. Where necessary, update the references to fib6_siblings to annotate or use the RCU APIs.
Include a test script that reproduces the issue. The script periodically updates the routing table while generating a heavy load of outgoing IPv6 traffic through multiple iperf3 clients. It consistently induces infinite soft lockups within a couple of minutes.
Kernel log:
0 [ffffbd13003e8d30] machinekexec at ffffffff8ceaf3eb 1 [ffffbd13003e8d90] _crashkexec at ffffffff8d0120e3 2 [ffffbd13003e8e58] panic at ffffffff8cef65d4 3 [ffffbd13003e8ed8] watchdogtimerfn at ffffffff8d05cb03 4 [ffffbd13003e8f08] _hrtimerrunqueues at ffffffff8cfec62f 5 [ffffbd13003e8f70] hrtimerinterrupt at ffffffff8cfed756 6 [ffffbd13003e8fd0] _sysvecapictimerinterrupt at ffffffff8cea01af 7 [ffffbd13003e8ff0] sysvecapictimerinterrupt at ffffffff8df1b83d -- <IRQ stack> -- 8 [ffffbd13003d3708] asmsysvecapictimerinterrupt at ffffffff8e000ecb [exception RIP: fib6selectpath+299] RIP: ffffffff8ddafe7b RSP: ffffbd13003d37b8 RFLAGS: 00000287 RAX: ffff975850b43600 RBX: ffff975850b40200 RCX: 0000000000000000 RDX: 000000003fffffff RSI: 0000000051d383e4 RDI: ffff975850b43618 RBP: ffffbd13003d3800 R8: 0000000000000000 R9: ffff975850b40200 R10: 0000000000000000 R11: 0000000000000000 R12: ffffbd13003d3830 R13: ffff975850b436a8 R14: ffff975850b43600 R15: 0000000000000007 ORIGRAX: ffffffffffffffff CS: 0010 SS: 0018 9 [ffffbd13003d3808] ip6polroute at ffffffff8ddb030c 10 [ffffbd13003d3888] ip6polrouteinput at ffffffff8ddb068c 11 [ffffbd13003d3898] fib6rulelookup at ffffffff8ddf02b5 12 [ffffbd13003d3928] ip6routeinput at ffffffff8ddb0f47 13 [ffffbd13003d3a18] ip6rcvfinishcore.constprop.0 at ffffffff8dd950d0 14 [ffffbd13003d3a30] ip6listrcvfinish.constprop.0 at ffffffff8dd96274 15 [ffffbd13003d3a98] ip6sublistrcv at ffffffff8dd96474 16 [ffffbd13003d3af8] ipv6listrcv at ffffffff8dd96615 17 [ffffbd13003d3b60] _netifreceiveskblistcore at ffffffff8dc16fec 18 [ffffbd13003d3be0] netifreceiveskblistinternal at ffffffff8dc176b3 19 [ffffbd13003d3c50] napigroreceive at ffffffff8dc565b9 20 [ffffbd13003d3c80] icereceiveskb at ffffffffc087e4f5 [ice] 21 [ffffbd13003d3c90] icecleanrxirq at ffffffffc0881b80 [ice] 22 [ffffbd13003d3d20] icenapipoll at ffffffffc088232f [ice] 23 [ffffbd13003d3d80] _napipoll at ffffffff8dc18000 24 [ffffbd13003d3db8] netrxaction at ffffffff8dc18581 25 [ffffbd13003d3e40] _dosoftirq at ffffffff8df352e9 26 [ffffbd13003d3eb0] runksoftirqd at ffffffff8ceffe47 27 [ffffbd13003d3ec0] smpbootthreadfn at ffffffff8cf36a30 28 [ffffbd13003d3ee8] kthread at ffffffff8cf2b39f 29 [ffffbd13003d3f28] retfromfork at ffffffff8ce5fa64 30 [ffffbd13003d3f50] retfromforkasm at ffffffff8ce03cbb