How can ebpf programs be attached to TC classes/qdiscs?

[vincentmli]

Hi,

Open this to preserve some background knowledge about why cilium/ebpf missing TC loader, anyone with insight could share the knowledge here.

[vincentmli]

from what I understand, there are two options:

1 netlink based TC loader
2, bpf_link based TC loader

according to https://lore.kernel.org/bpf/[email protected]/, bpf_link based TC is still missing in kernel.

[vincentmli]

here are examples of netlink based:

https://pkg.go.dev/github.com/florianl/go-tc

https://github.com/cilium/cilium/blob/master/pkg/datapath/loader/netlink.go#L155-L192

[vincentmli]

according to https://lore.kernel.org/bpf/[email protected]/, bpf_link based TC is still missing in kernel.

seems not going to be true anymore https://github.com/cilium/linux/commits/pr/meta5

[ti-mo]

Long story short: use https://pkg.go.dev/github.com/florianl/go-tc#example-package-EBPF.

TC is a completely separate kernel subsystem from eBPF that can currently only be interacted with using netlink sockets and its corresponding protocol. As a comparison, on older versions of the Linux kernel, attaching XDP programs also required netlink. However, due to its simplicity (XDP associates a prog FD with an ifindex, which can be retrieved by name using the Go stdlib), XDP has since received a bpf_link interface, and cilium/ebpf was easily extended to support it.

TC, however, is significantly more complex in both its design and its implementation. Assuming the user sets up a class/qdisc hierarchy using the tc tool or using one of the existing Go libraries, we'd still need to teach cilium/ebpf to send a netlink message to associate a node in the tc hierarchy with a tc bpf filter.

While this is relatively trivial nowadays due to the availability of decent Go libraries, it does require us to add either florianl/go-tc or vishvananda/netlink as a dependency, which kind of goes against our 'minimal dependencies' policy. If we choose either, projects using the other will now import (and vendor) both, even if they're not using bpf+tc.

Then there's also the question of API design. At minimum, the caller would have to specify the qdisc to attach the program to, and configure some fields in the created tc filter. Assuming we're using go-tc, we could take a tc.Object, forcing the caller to describe the qdisc using go-tc structs, while they might already be using vishvananda/netlink for everything else (like Cilium). This wouldn't create a pleasant experience and invites bugs. To avoid lock-in, we could provide our own descriptors and convert to the format of whatever library we'd use internally, but now we're suddenly reinventing the wheel.

In conclusion, until tc bpf_link comes around, a dedicated tc library is always going to be the best tool for the job. 🙂 Perhaps we can highlight this more prominently by pointing to external examples for this?

[unikzforce]

@vincentmli
I'm not sure what is the status of TC in cilium/ebpf , but i think the api is not released yet,

as i was searching i came across libbpfgo library , They have not yet released their first version, but they have presented bpf_link tc hook in their APIs.

this is an example : https://github.com/aquasecurity/libbpfgo/blob/main/selftest/tc/main.go

I guess for the sake of TC one can use that side by side cilium/ebpf and let cilium/ebpf deal with other things and libpfgo deal with TC, although i'm not sure what would the risks of having 2 libraries to interface the same subject.

but anyway it worth a try for the time being until cilium/ebpf realise an api for dealing with tc using bpf_link.

[lmb]

You can now use https://pkg.go.dev/github.com/cilium/ebpf/link#AttachTCX on very new kernels.

[unikzforce]

great, thanks