Skip to content

Commit

Permalink
pmdabpf: prototype eBPF Kelper module (sustainable-computing.io)
Browse files Browse the repository at this point in the history
  • Loading branch information
natoscott committed Nov 21, 2023
1 parent 1a78625 commit 0748162
Show file tree
Hide file tree
Showing 9 changed files with 929 additions and 40 deletions.
35 changes: 20 additions & 15 deletions src/pmdas/bpf/README
Original file line number Diff line number Diff line change
@@ -1,37 +1,42 @@
PCP PMDA to load linux BPF modules
==================================

This PMDA is capable of collecting and generating arbitrary metrics from kernel-side code running as pre-compiled
ELF BPF/eBPF modules.
This PMDA is capable of collecting and generating arbitrary metrics from
kernel-side code running as pre-compiled ELF BPF/eBPF modules.

Comparison to other PCP PMDAS
=============================

- pmdabcc runs as python and is a little easier to develop and maintain, has more advanced configuration files,
however it requires significantly more runtime memory. It loads and compiles the BCC code through LLVM which
has a heavy footprint to work with the fact that kernel structures might change. By comparison, pmdabpf uses
pre-compiled ELF-based BPF CO-RE modules to avoid relocation constraints.
- pmdabcc runs as python and is a little easier to develop and maintain,
however it requires significantly more runtime memory. It loads and
compiles the BCC code through LLVM which has a heavy footprint to
work with the fact that kernel structures might change. By comparison,
pmdabpf uses pre-compiled ELF-based BPF CO-RE modules to avoid
relocation constraints.

- pmdabpftrace can load and run arbitrary bpftrace code, so it is very flexible, however it does this by shelling
out to bpftrace executable which again requires quite a lot of memory.
- pmdabpftrace can load and run arbitrary bpftrace code, so it is very
flexible, however it does this by shelling out to bpftrace executable
which again requires quite a lot of memory.

Deployment
==========

The file `bpf.conf` lists modules, one per line, that will be started.

A relatively new version of libbpf is required on the system. libbpf 0.1.0 is unlikely to work, however, libbpf 0.4.0
has been shown to work.
A relatively new version of libbpf is required on the system. libbpf
1.0.0 is known to work.

Development
===========

To develop additional modules:
- Follow the examples provided (runqlat and biolatency) to create a new module (which will output a .so). The entry
point in your .so will be a load_module() call that should return a newly allocated `struct module` object.
- Create your bpf code (this will become a .bpf.o). Use the various `_helpers` headers.
- Ensure `module.h` has correct unique setup for your cluster and metric ids.
- Add details to `pmns` and `help` files to ensure they match the `module.h` changes.
- Follow the examples provided (runqlat and biolatency) to create a new
module (which will output a .so). The entry point in your .so will
be a load_module() call that should return a newly allocated `struct
module` object.
- Create your bpf code (this will become a .bpf.o). Use the various
`_helpers` headers.
- Add your module name to the global known-modules list in `module.h`.

TODO
====
Expand Down
9 changes: 9 additions & 0 deletions src/pmdas/bpf/bpf.conf
Original file line number Diff line number Diff line change
Expand Up @@ -135,3 +135,12 @@ enabled = false
# cgroup - string - unset : Trace process in cgroup path
[biosnoop.so]
enabled = false

# This tool traces scheduling activity for use in power management.
#
# Configuration options:
# Name - type - default
#
# process_count - int - 20 : number of processes to keep in cache
[kepler.so]
enabled = false
13 changes: 11 additions & 2 deletions src/pmdas/bpf/modules/GNUmakefile
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,15 @@ APPS_BPF = \
tcpconnlat.bpf.c \
vfsstat.bpf.c \

# non-libbpf header files.
LOCAL_H = \
kepler.bpf.h \

# non-libbpf *.bpf.c files.
LOCAL_BPF = \
kepler.bpf.c \

PMDABPF_MODULES += kepler.so

default_pcp default: $(PMDABPF_MODULES)

Expand Down Expand Up @@ -99,7 +108,7 @@ $(APPS_BPF):
vmlinux.h: $(PMDABPF_VMLINUXH)
$(LN_S) -f $< $@

%.bpf.o: %.bpf.c vmlinux.h $(APPS_H) $(APPS_BPF)
%.bpf.o: %.bpf.c vmlinux.h $(APPS_H) $(APPS_BPF) $(LOCAL_H) $(LOCAL_BPF)
$(CLANG) -Wall -g -O2 -target bpf -D__TARGET_ARCH_$(PMDABPF_ARCH) \
-I. -c $< -o $@
$(LLVM_STRIP) -g $@
Expand All @@ -108,7 +117,7 @@ vmlinux.h: $(PMDABPF_VMLINUXH)
$(BPFTOOL) gen skeleton $< > $@

%.o: %.c
%.o: %.c %.skel.h $(HELPERS_H) $(APPS_H)
%.o: %.c %.skel.h $(HELPERS_H) $(APPS_H) $(LOCAL_H)
$(CC) -c $(CFLAGS) $(CPPFLAGS) $< -o $@

%_helpers.o: %_helpers.c
Expand Down
Loading

0 comments on commit 0748162

Please sign in to comment.