Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ERROR: testmod_drv.o: .return_sites section header details differ from .return_sites #1300

Open
he7850 opened this issue Sep 15, 2022 · 20 comments
Assignees

Comments

@he7850
Copy link

he7850 commented Sep 15, 2022

recently kernel introduces .return_sites section, and kpatch has added support for it.
but this section causes some testcases failure.

failed testcase:

  • original module code
#define pr_fmt(fmt) "testmod_ext: " fmt
  
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/kobject.h>
#include <linux/sysfs.h>
#include <linux/kthread.h>

static struct kobject *testmod_kobj;
int value = 2;

int test_child(void)
{
        printk("I am child\n");
        if(value == 0)
                printk("haha\n");
        return 0;
}

static ssize_t value_store(struct kobject *kobj, struct kobj_attribute *attr,
                         const char *buf, size_t count)
{
        int ret = 0;
        ret = sscanf(buf, "%d", &value);

        return count;
}

ssize_t value_show(struct kobject *kobj,
                          struct kobj_attribute *attr, char *buf)
{
        printk("I am caller\n");
        test_child();
        return sprintf(buf, "%d\n", value);
}
static struct kobj_attribute testmod_value_attr = __ATTR(value,0644,value_show,value_store);

static int testmod_init(void)
{
        int ret;

        testmod_kobj = kobject_create_and_add("testmod_ext", kernel_kobj);
        if (!testmod_kobj)
                return -ENOMEM;
test_child();
        ret = sysfs_create_file(testmod_kobj, &testmod_value_attr.attr);
        if (ret) {
                kobject_put(testmod_kobj);
                return ret;
        }

        return 0;
}

static void testmod_exit(void)
{
        sysfs_remove_file(testmod_kobj, &testmod_value_attr.attr);
        kobject_put(testmod_kobj);
}

module_init(testmod_init);
module_exit(testmod_exit);
MODULE_LICENSE("GPL");
  • patch to the module
localhost:~/testmod_ext # git diff testmod_drv.c{,.003}
diff --git a/testmod_drv.c b/testmod_drv.c.003
index e6ace31..8de2c9d 100644
--- a/testmod_drv.c
+++ b/testmod_drv.c.003
@@ -17,6 +17,13 @@ int test_child(void)
        return 0;
 }
 
+int new_export_testfun(void)
+{
+       printk("I am new_export_testfun\n");
+       return 0;
+}
+EXPORT_SYMBOL(new_export_testfun);
+
 static ssize_t value_store(struct kobject *kobj, struct kobj_attribute *attr,
                          const char *buf, size_t count)
 {
  • log of /root/.kpatch/build.log
checking file testmod_drv.c
patching file testmod_drv.c
make: Entering directory '/root/testmod_ext'
[ -e /root/testmod_ext ] || mkdir -p /root/testmod_ext
make -C /lib/modules/5.10.0-60.18.0.50.h485.x86_64/build M=/root/testmod_ext  modules
make[1]: Entering directory '/usr/src/kernels/5.10.0-60.18.0.50.h485.x86_64'
  CC [M]  /root/testmod_ext/testmod_drv.o
  LD [M]  /root/testmod_ext/testmod_ext.o
  MODPOST /root/testmod_ext/Module.symvers
  CC [M]  /root/testmod_ext/testmod_ext.mod.o
  LD [M]  /root/testmod_ext/testmod_ext.ko
make[1]: Leaving directory '/usr/src/kernels/5.10.0-60.18.0.50.h485.x86_64'
make: Leaving directory '/root/testmod_ext'
checking file testmod_drv.c
patching file testmod_drv.c
make: Entering directory '/root/testmod_ext'
[ -e /root/testmod_ext ] || mkdir -p /root/testmod_ext
make -C /lib/modules/5.10.0-60.18.0.50.h485.x86_64/build M=/root/testmod_ext  modules
make[1]: Entering directory '/usr/src/kernels/5.10.0-60.18.0.50.h485.x86_64'
  CC [M]  /root/testmod_ext/testmod_drv.o
  LD [M]  /root/testmod_ext/testmod_ext.o
  MODPOST /root/testmod_ext/Module.symvers
  CC [M]  /root/testmod_ext/testmod_ext.mod.o
  LD [M]  /root/testmod_ext/testmod_ext.ko
make[1]: Leaving directory '/usr/src/kernels/5.10.0-60.18.0.50.h485.x86_64'
make: Leaving directory '/root/testmod_ext'
make: Entering directory '/root/testmod_ext'
[ -e /root/testmod_ext ] || mkdir -p /root/testmod_ext
make -C /lib/modules/5.10.0-60.18.0.50.h485.x86_64/build M=/root/testmod_ext  modules
make[1]: Entering directory '/usr/src/kernels/5.10.0-60.18.0.50.h485.x86_64'
  CC [M]  /root/testmod_ext/testmod_drv.o
  LD [M]  /root/testmod_ext/testmod_ext.o
  MODPOST /root/testmod_ext/Module.symvers
  CC [M]  /root/testmod_ext/testmod_ext.mod.o
  LD [M]  /root/testmod_ext/testmod_ext.ko
make[1]: Leaving directory '/usr/src/kernels/5.10.0-60.18.0.50.h485.x86_64'
make: Leaving directory '/root/testmod_ext'
checking file testmod_drv.c
patching file testmod_drv.c
ERROR: testmod_drv.o: .return_sites section header details differ from .return_sites
/usr/libexec/kpatch/create-diff-object: unreconcilable difference
@he7850
Copy link
Author

he7850 commented Sep 15, 2022

have debugged kpatch, found that the entsize of .return_sites section changed from 4 to 0, thus kpatch reported this error.

@he7850
Copy link
Author

he7850 commented Sep 15, 2022

after adding V=1 to module Makefile, log output shows:

make -f ./scripts/Makefile.build obj=/root/testmod_ext \
single-build= \
need-builtin=1 need-modorder=1

  gcc -Wp,-MMD,/root/testmod_ext/.testmod_drv.o.d -nostdinc -isystem /usr/lib/gcc/x86_64-linux-gnu/10.3.1/include -I./arch/x86/include -I./arch/x86/include/generated  -I./include -I./arch/x86/include/uapi -I./arch/x86/include/generated/uapi -I./include/uapi -I./include/generated/uapi -include ./include/linux/kconfig.h -include ./include/linux/compiler_types.h -D__KERNEL__ -fmacro-prefix-map=./= -Wall -Wundef -Werror=strict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -fshort-wchar -fno-PIE -Werror=implicit-function-declaration -Werror=implicit-int -Werror=return-type -Wno-format-security -std=gnu89 -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -mno-avx -fcf-protection=none -m64 -falign-jumps=1 -falign-loops=1 -mno-80387 -mno-fp-ret-in-387 -mpreferred-stack-boundary=3 -mskip-rax-setup -mtune=generic -mno-red-zone -mcmodel=kernel -Wno-sign-compare -fno-asynchronous-unwind-tables -mindirect-branch=thunk-extern -mindirect-branch-register -mfunction-return=thunk-extern -fno-jump-tables -fno-delete-null-pointer-checks -Wno-frame-address -Wno-format-truncation -Wno-format-overflow -Wno-address-of-packed-member -O2 -fno-allow-store-data-races -Wframe-larger-than=2048 -fstack-protector-strong -Wimplicit-fallthrough -Wno-unused-but-set-variable -Wno-unused-const-variable -g -gdwarf-4 -pg -mrecord-mcount -mfentry -DCC_USING_FENTRY -fno-inline-functions-called-once -Wdeclaration-after-statement -Wvla -Wno-pointer-sign -Wno-stringop-truncation -Wno-zero-length-bounds -Wno-array-bounds -Wno-stringop-overflow -Wno-restrict -Wno-maybe-uninitialized -fno-strict-overflow -fno-stack-check -fconserve-stack -Werror=date-time -Werror=incompatible-pointer-types -Werror=designated-init -Wno-packed-not-aligned  -DMODULE  -DKBUILD_BASENAME='"testmod_drv"' -DKBUILD_MODNAME='"testmod_ext"' -c -o /root/testmod_ext/testmod_drv.o /root/testmod_ext/testmod_drv.c

  ./tools/objtool/objtool orc generate  --module --no-fp --retpoline --rethunk --uaccess /root/testmod_ext/testmod_drv.o

  if objdump -h /root/testmod_ext/testmod_drv.o | grep -q __ksymtab; then 
  gcc -E -D__GENKSYMS__ -Wp,-MMD,/root/testmod_ext/.testmod_drv.o.d -nostdinc -isystem /usr/lib/gcc/x86_64-linux-gnu/10.3.1/include -I./arch/x86/include -I./arch/x86/include/generated  -I./include -I./arch/x86/include/uapi -I./arch/x86/include/generated/uapi -I./include/uapi -I./include/generated/uapi -include ./include/linux/kconfig.h -include ./include/linux/compiler_types.h -D__KERNEL__ -fmacro-prefix-map=./= -Wall -Wundef -Werror=strict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -fshort-wchar -fno-PIE -Werror=implicit-function-declaration -Werror=implicit-int -Werror=return-type -Wno-format-security -std=gnu89 -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -mno-avx -fcf-protection=none -m64 -falign-jumps=1 -falign-loops=1 -mno-80387 -mno-fp-ret-in-387 -mpreferred-stack-boundary=3 -mskip-rax-setup -mtune=generic -mno-red-zone -mcmodel=kernel -Wno-sign-compare -fno-asynchronous-unwind-tables -mindirect-branch=thunk-extern -mindirect-branch-register -mfunction-return=thunk-extern -fno-jump-tables -fno-delete-null-pointer-checks -Wno-frame-address -Wno-format-truncation -Wno-format-overflow -Wno-address-of-packed-member -O2 -fno-allow-store-data-races -Wframe-larger-than=2048 -fstack-protector-strong -Wimplicit-fallthrough -Wno-unused-but-set-variable -Wno-unused-const-variable -g -gdwarf-4 -pg -mrecord-mcount -mfentry -DCC_USING_FENTRY -fno-inline-functions-called-once -Wdeclaration-after-statement -Wvla -Wno-pointer-sign -Wno-stringop-truncation -Wno-zero-length-bounds -Wno-array-bounds -Wno-stringop-overflow -Wno-restrict -Wno-maybe-uninitialized -fno-strict-overflow -fno-stack-check -fconserve-stack -Werror=date-time -Werror=incompatible-pointer-types -Werror=designated-init -Wno-packed-not-aligned  -DMODULE  -DKBUILD_BASENAME='"testmod_drv"' -DKBUILD_MODNAME='"testmod_ext"' /root/testmod_ext/testmod_drv.c | scripts/genksyms/genksyms    -r /dev/null > /root/testmod_ext/.tmp_testmod_drv.ver; 
  ### this ld changed the entsize
  ld -m elf_x86_64 -r -o /root/testmod_ext/.tmp_testmod_drv.o /root/testmod_ext/testmod_drv.o -T /root/testmod_ext/.tmp_testmod_drv.ver; 
  mv -f /root/testmod_ext/.tmp_testmod_drv.o /root/testmod_ext/testmod_drv.o; 
  rm -f /root/testmod_ext/.tmp_testmod_drv.ver; 
  fi

  ld -m elf_x86_64   -r -o /root/testmod_ext/testmod_ext.o /root/testmod_ext/testmod_drv.o
  { echo  /root/testmod_ext/testmod_drv.o;  echo; } > /root/testmod_ext/testmod_ext.mod
  {   echo /root/testmod_ext/testmod_ext.ko; :; } | awk '!x[$0]++' - > /root/testmod_ext/modules.order
make -f ./scripts/Makefile.modpost
  sed 's/ko$/o/' /root/testmod_ext/modules.order | scripts/mod/modpost -m -a   -o /root/testmod_ext/Module.symvers -e -i Module.symvers   -T -
make -f ./scripts/Makefile.modfinal
  gcc -Wp,-MMD,/root/testmod_ext/.testmod_ext.mod.o.d -nostdinc -isystem /usr/lib/gcc/x86_64-linux-gnu/10.3.1/include -I./arch/x86/include -I./arch/x86/include/generated  -I./include -I./arch/x86/include/uapi -I./arch/x86/include/generated/uapi -I./include/uapi -I./include/generated/uapi -include ./include/linux/kconfig.h -include ./include/linux/compiler_types.h -D__KERNEL__ -fmacro-prefix-map=./= -Wall -Wundef -Werror=strict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -fshort-wchar -fno-PIE -Werror=implicit-function-declaration -Werror=implicit-int -Werror=return-type -Wno-format-security -std=gnu89 -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -mno-avx -fcf-protection=none -m64 -falign-jumps=1 -falign-loops=1 -mno-80387 -mno-fp-ret-in-387 -mpreferred-stack-boundary=3 -mskip-rax-setup -mtune=generic -mno-red-zone -mcmodel=kernel -Wno-sign-compare -fno-asynchronous-unwind-tables -mindirect-branch=thunk-extern -mindirect-branch-register -mfunction-return=thunk-extern -fno-jump-tables -fno-delete-null-pointer-checks -Wno-frame-address -Wno-format-truncation -Wno-format-overflow -Wno-address-of-packed-member -O2 -fno-allow-store-data-races -Wframe-larger-than=2048 -fstack-protector-strong -Wimplicit-fallthrough -Wno-unused-but-set-variable -Wno-unused-const-variable -g -gdwarf-4 -pg -mrecord-mcount -mfentry -DCC_USING_FENTRY -fno-inline-functions-called-once -Wdeclaration-after-statement -Wvla -Wno-pointer-sign -Wno-stringop-truncation -Wno-zero-length-bounds -Wno-array-bounds -Wno-stringop-overflow -Wno-restrict -Wno-maybe-uninitialized -fno-strict-overflow -fno-stack-check -fconserve-stack -Werror=date-time -Werror=incompatible-pointer-types -Werror=designated-init -Wno-packed-not-aligned  -DMODULE  -DKBUILD_BASENAME='"testmod_ext.mod"' -DKBUILD_MODNAME='"testmod_ext"' -c -o /root/testmod_ext/testmod_ext.mod.o /root/testmod_ext/testmod_ext.mod.c
  ld -r -m elf_x86_64 --build-id=sha1  -T scripts/module.lds -o /root/testmod_ext/testmod_ext.ko /root/testmod_ext/testmod_ext.o /root/testmod_ext/testmod_ext.mod.o;  true
make[1]: Leaving directory '/usr/src/kernels/5.10.0-60.18.0.50.h485.x86_64'

there is one 'ld' step after objtool generates .return_sites section, and it changed entsize of .return_sites section of testmod_drv.o from 4 to 0.
the reason why 'ld' step happens seems to be that patched module contains __ksymtab section due to exporting a function.

@he7850
Copy link
Author

he7850 commented Sep 15, 2022

localhost:~/testmod_ext # readelf -SW /root/.kpatch/tmp/orig/testmod_drv.o | grep return_sites
  [46] .return_sites     PROGBITS        0000000000000000 0147aa 000010 04   A  0   0  1
  [47] .rela.return_sites RELA            0000000000000000 0147c0 000060 18   I 43  46  8
localhost:~/testmod_ext # readelf -SW /root/.kpatch/tmp/patched/testmod_drv.o | grep return_sites
  [28] .return_sites     PROGBITS        0000000000000000 000239 000014 00   A  0   0  1
  [29] .rela.return_sites RELA            0000000000000000 00b848 000078 18   I 58  28  8

localhost:~/testmod_ext # readelf -SW /root/.kpatch/tmp/orig/testmod_drv.o | grep symtab
  [43] .symtab           SYMTAB          0000000000000000 013f00 0004f8 18     44  38  8
localhost:~/testmod_ext # readelf -SW /root/.kpatch/tmp/patched/testmod_drv.o | grep symtab
  [16] __ksymtab_strings PROGBITS        0000000000000000 00018a 000014 01 AMS  0   0  1
  [17] ___ksymtab+new_export_testfun PROGBITS        0000000000000000 0001a0 00000c 00   A  0   0  4
  [18] .rela___ksymtab+new_export_testfun RELA            0000000000000000 00b770 000048 18   I 58  17  8
  [58] .symtab           SYMTAB          0000000000000000 00ab20 000660 18     59  51  8

@he7850
Copy link
Author

he7850 commented Sep 15, 2022

Not sure should kpatch deal with this situation, or it is kernel's problem.
Or ld should keep testmod_drv.o's entsize not changed?

@yhcote
Copy link
Contributor

yhcote commented Sep 16, 2022

Hi @he7850,

I am not getting the error you mention while trying to reproduce your issue. Could you please attach a tarball including the original test module C file + the patch file you applied?

Could you also provide the "kpatch-build" line you used to build and get that error?
Thanks

@he7850
Copy link
Author

he7850 commented Sep 16, 2022

Hi @he7850,

I am not getting the error you mention while trying to reproduce your issue. Could you please attach a tarball including the original test module C file + the patch file you applied?

Could you also provide the "kpatch-build" line you used to build and get that error? Thanks

I cannot upload files to github due to the firewall of my company, I will copy the code to my PC and upload it, but it may take some time.

@yhcote
Copy link
Contributor

yhcote commented Sep 16, 2022

@he7850 Maybe just the kpatch-build ... command line that failed used to build the kpatch-patch and if available, the link to the kernel source tree you use should be a good start.

@he7850
Copy link
Author

he7850 commented Sep 20, 2022

sorry for the late reply. here is the command:

localhost:~/testmod_ext # kpatch-build -e testmod_ext.ko -s /root/testmod_ext -t all -c /opt/patch_workspace/.config -v /opt/patch_workspace/vmlinux -n klp_testexport003 /root/testmod_ext/testexport003.patch -R
Using source directory at /root/testmod_ext
Testing patch file(s)
Reading special section data
Building original source
Building patched source
Extracting new and modified ELF sections
ERROR: testmod_drv.o: .return_sites section header details differ from .return_sites
/usr/libexec/kpatch/create-diff-object: unreconcilable difference
ERROR: 1 error(s) encountered. Check /root/.kpatch/build.log for more details.

@he7850
Copy link
Author

he7850 commented Sep 20, 2022

  • kernel source tree
    https://gitee.com/openeuler/kernel/tree/openEuler-22.03-LTS/
  • testmod_ext module C file
    it has been pasted in the fisrt comment, its file name is testmod_drv.c:
  • testmod_ext module patch file
    it has been pasted in the fisrt comment, its file name is testexport003.patch:
  • testmod_ext module Makefile
######## modify the name to the module name ########
name := testmod_ext

######## do not modify the following content ###########
CURDIR := ${shell pwd}
BINDIR=$(shell echo $(CURDIR) | sed 's/testcases\//bin\//g')

ifeq ("$(ARCH)", "arm64")
       EXTRA_CFLAGS += -DCONFIG_ARM64=1
endif

SOURCES_FILE := $(wildcard *.c)

ifneq ($(KERNELRELEASE),)
        obj-m:=$(name).o
        $(name)-objs := testmod_drv.o
else
        KVER := $(shell uname -r)
        KDIR := /lib/modules/$(KVER)/build
        PWD  := $(shell pwd)

all:
        [ -e $(BINDIR) ] || mkdir -p $(BINDIR)
        $(MAKE) V=1 -C $(KDIR) M=$(PWD)  modules

.PHONY: install
install:
        cp *.ko $(BINDIR)

.PHONY: clean
clean:
        make -C $(KDIR) M=$(PWD) clean
        rm -rf $(BINDIR)/*.ko

endif

@joe-lawrence
Copy link
Contributor

Can the problem be reproduced without adding a new symbol export? This is something that kpatch-build does not support (for example, see #1178)

@joe-lawrence
Copy link
Contributor

Also, not sure what the -e command line option does, as the upstream kpatch-build only processes:

options="$(getopt -o ha:r:s:c:v:j:t:n:o:dR ...

@joe-lawrence
Copy link
Contributor

@jpoimboe: (symbol exports aside) I think this is another variant of [0] objtool creating elf sections with entry size while the rest of the toolchain (ie, ld) not bothering? If we're otherwise ignoring this section, should we relax the section header check to allow for N to 0 entry size changes?

[0] https://lore.kernel.org/lkml/[email protected]/T/#mea3ba10046d6a72b4ed07556f63cc537da767e15

@he7850
Copy link
Author

he7850 commented Sep 21, 2022

Also, not sure what the -e command line option does, as the upstream kpatch-build only processes:

options="$(getopt -o ha:r:s:c:v:j:t:n:o:dR ...

-e means oot module.
I'm using Kpatch 0.9.5, kpatch-build option has not been refactored:

localhost:~/testmod_ext # kpatch-build -h
usage: kpatch-build [options] <patch1 ... patchN>
		patchN                  Input patchfile(s)
		-h, --help              Show this help message
		-a, --archversion       Specify the kernel arch version
		-r, --sourcerpm         Specify kernel source RPM
		-s, --sourcedir         Specify kernel source directory
		-c, --config            Specify kernel config file
		-v, --vmlinux           Specify original vmlinux
		-j, --jobs              Specify the number of make jobs
		-t, --target            Specify custom kernel build targets
		-n, --name              Specify the name of the kpatch module
		-o, --output            Specify output folder
		-d, --debug             Enable 'xtrace' and keep scratch files
		                        in <CACHEDIR>/tmp
		                        (can be specified multiple times)
		-e, --oot-module        Enable patching out-of-tree module,
		                        specify current version of module
		-R, --non-replace       Disable replace patch (replace is on by default)
		--skip-cleanup          Skip post-build cleanup
		--skip-compiler-check   Skip compiler version matching check
		                        (not recommended)

@he7850
Copy link
Author

he7850 commented Sep 21, 2022

lol, just found my version of kpatch does support exporting symbol in patch module. (there is a patch allowing this)
and header difference problem only happens when the patched module contains __symtab section and original module does not.
bcz ld to textmod_drv.o only gets executed when there is __symtab section and ld changes the entsize of .return_sites section.

@he7850
Copy link
Author

he7850 commented Sep 21, 2022

I cannot figure why ld will change the entsize of .return_sites section, but I have tried one possible solution:
add SHF_MERGE to .return_sites section when creating it in objtool, then ld will keep the entsize not changed in the ouput testmod_drv.o

localhost:/opt/patch_workspace/kernel-source # git diff tools/objtool/check.c*
diff --git a/tools/objtool/check.c b/tools/objtool/check.c.new
index c7b6e6b..1162bf7 100644
--- a/tools/objtool/check.c
+++ b/tools/objtool/check.c.new
@@ -672,7 +672,7 @@ static int create_return_sites_sections(struct objtool_file *file)
        if (!idx)
                return 0;
 
-       sec = elf_create_section(file->elf, ".return_sites", 0,
+       sec = elf_create_section(file->elf, ".return_sites", 0x10, // SHF_MERGE
                                 sizeof(int), idx);
        if (!sec) {
                WARN("elf_create_section: .return_sites");

but I am not sure if this flag can be added to the section.

currently I temporarily solve this problem by kpatch ignoring the section header change. maybe setting the entsize to 0 in objtool is better, if the entsize attrubute of .return_sites section is not actually used.

@joe-lawrence
Copy link
Contributor

just found my version of kpatch does support exporting symbol in patch module. (there is a patch allowing this)
Is this a local patch, or from a forked kpatch project?

As for the entry size changing, I'll leave it up to Josh to whether objtool should mimic the rest of the toolchain or not.

@he7850
Copy link
Author

he7850 commented Sep 22, 2022

just found my version of kpatch does support exporting symbol in patch module. (there is a patch allowing this)
Is this a local patch, or from a forked kpatch project?

the patch is from openEuler's forked kpatch repository: https://gitee.com/src-openeuler/kpatch

@github-actions
Copy link

This issue has been open for 30 days with no activity and no assignee. It will be closed in 7 days unless a comment is added.

@github-actions github-actions bot added the stale label Aug 16, 2023
@jpoimboe jpoimboe removed the stale label Aug 16, 2023
@github-actions
Copy link

This issue has been open for 30 days with no activity and no assignee. It will be closed in 7 days unless a comment is added.

@github-actions github-actions bot added the stale label Sep 16, 2023
@github-actions
Copy link

This issue was closed because it was inactive for 7 days after being marked stale.

@jpoimboe jpoimboe self-assigned this Sep 25, 2023
@jpoimboe jpoimboe removed the stale label Sep 25, 2023
@jpoimboe jpoimboe reopened this Sep 25, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants