struct inode ctime fields changed again with Kernel >= 6.11

[agalauner-r7]

Description

Running tracee on a kernel with version >= 6.11 fails with the below error.

Reason for this is that the ctime fields in the inode struct changed again: torvalds/linux@3aa63a5

[...]
; if (bpf_core_field_exists(inode->__i_ctime)) { // Version >= 6.6 @ filesystem.h:61
3941: (15) if r1 == 0x0 goto pc+2     ; R1_w=0
3944: <invalid CO-RE relocation>
failed to resolve CO-RE relocation <byte_off> [620] struct inode___older_v66.i_ctime (0:0 @ offset 0)
processed 3273 insns (limit 1000000) max_states_per_insn 0 total_states 226 peak_states 226 mark_read 173
-- END PROG LOAD LOG --
{"level":"warn","ts":1732264697.5150177,"msg":"libbpf: prog 'tracepoint__sched__sched_process_exec': failed to load: -22"}
{"level":"warn","ts":1732264697.518446,"msg":"libbpf: failed to load object ''"}
{"level":"fatal","ts":1732264697.5203826,"msg":"Tracee runner failed","error":"cmd.Runner.Run: error initializing Tracee: ebpf.(*Tracee).Init: ebpf.(*Tracee).initBPF: failed to load BPF object: invalid argument"}

Output of tracee version:

Tracee version: v0.22.4

Output of uname -a:

Linux 500576e7ccce 6.11.6-arch1-1 #1 SMP PREEMPT_DYNAMIC Fri, 01 Nov 2024 03:30:41 +0000 x86_64 GNU/Linux

Additional details

I would fix it myself and submit a PR, but there are multiple ways to do it and I am not sure what's the preferred way.

Right now you check using the CO-RE framework if a new field for kernels between 6.6 and 6.10 is present. If so, read that, otherwise cast the struct into an older mock version and use CO-RE to read the old field.

This doesn't work if we have three different versions now.

There is a way to check for the current linux kernel version by declaring an external variable:

extern int LINUX_KERNEL_VERSION __kconfig;

if (LINUX_KERNEL_VERSION >= KERNEL_VERSION(6, 11, 0)) {
    /* we are on v6.11+ */
}

So, naively, I ended up with code like this:

statfunc u64 get_ctime_nanosec_from_inode(struct inode *inode)
{
    struct timespec64 ts;
    if (LINUX_KERNEL_VERSION >= KERNEL_VERSION(6, 11, 0)) {
        ts.tv_sec = BPF_CORE_READ(inode, i_ctime_sec);
        ts.tv_nsec = BPF_CORE_READ(inode, i_ctime_nsec);
    } else if (LINUX_KERNEL_VERSION >= KERNEL_VERSION(6, 6, 0)) {
        ts = BPF_CORE_READ(inode, __i_ctime);
    } else {
        struct inode___older_v66 *old_inode = (void *) inode;
        ts = BPF_CORE_READ(old_inode, i_ctime);
    }

    return get_time_nanosec_timespec(&ts);
}

This doesn't work though, because vmlinux.h doesn't contain the new definition of struct inode.

Now there are two tways to solve this:

1. Introduce a struct inode___newer_v611 which contains the two new fields and use it like the else case:

statfunc u64 get_ctime_nanosec_from_inode(struct inode *inode)
{
    struct timespec64 ts;

    if (LINUX_KERNEL_VERSION >= KERNEL_VERSION(6, 11, 0)) {
        struct inode___newer_v611 *new_inode = (void *) inode;
        ts.tv_sec = BPF_CORE_READ(new_inode, i_ctime_sec);
        ts.tv_nsec = BPF_CORE_READ(new_inode, i_ctime_nsec);
    } else if (LINUX_KERNEL_VERSION >= KERNEL_VERSION(6, 6, 0)) {
        ts = BPF_CORE_READ(inode, __i_ctime);
    } else {
        struct inode___older_v66 *old_inode = (void *) inode;
        ts = BPF_CORE_READ(old_inode, i_ctime);
    }
}

2. Regenerate vmlinux.h so we have a current struct inode and introduce a struct inode___older_v611 which contains the old fields and use that in the else if branch like you do now in the else branch:

statfunc u64 get_ctime_nanosec_from_inode(struct inode *inode)
{
    struct timespec64 ts;

    if (LINUX_KERNEL_VERSION >= KERNEL_VERSION(6, 11, 0)) {
        ts.tv_sec = BPF_CORE_READ(inode, i_ctime_sec);
        ts.tv_nsec = BPF_CORE_READ(inode, i_ctime_nsec);
    } else if (LINUX_KERNEL_VERSION >= KERNEL_VERSION(6, 6, 0)) {
        ts = BPF_CORE_READ(inode, __i_ctime);
        struct inode___older_v611 *old_inode = (void *) inode;
        ts = BPF_CORE_READ(old_inode, __i_ctime);
    } else {
        struct inode___older_v66 *old_inode = (void *) inode;
        ts = BPF_CORE_READ(old_inode, i_ctime);
    }
}

So what's preferred? And if it's solution number 2, how do I regenerate vmlinux.h? On my local machine using bpftool? Do you have another process for that?

I personally would prefer the second case, because it feels better to work with more "current" code and have the special cases present for older kernels.

[rscampos]

Hello @agalauner-r7,

Thank you for using Tracee and for providing such a detailed issue report.

I was able to reproduce the issue in my environment as well and will take a closer look at it.

; if (bpf_core_field_exists(inode->__i_ctime)) { // Version >= 6.6 @ filesystem.h:61
3941: (15) if r1 == 0x0 goto pc+2     ; R1_w=0
3944: <invalid CO-RE relocation>
failed to resolve CO-RE relocation <byte_off> [620] struct inode___older_v66.i_ctime (0:0 @ offset 0)
processed 3273 insns (limit 1000000) max_states_per_insn 0 total_states 226 peak_states 226 mark_read 173
-- END PROG LOAD LOG --
{"level":"warn","ts":1732305951.2560215,"msg":"libbpf: prog 'tracepoint__sched__sched_process_exec': failed to load: -22"}
{"level":"warn","ts":1732305951.257808,"msg":"libbpf: failed to load object ''"}
{"level":"fatal","ts":1732305951.2585351,"msg":"Tracee runner failed","error":"cmd.Runner.Run: error initializing Tracee: ebpf.(*Tracee).Init: ebpf.(*Tracee).initBPF: failed to load BPF object: invalid argument"}

Kernel version:

Linux ip-172-31-3-75 6.11.0-061100-generic #202409151536 SMP PREEMPT_DYNAMIC Sun Sep 15 16:01:12 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux