Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bootstrapping fails on OpenMPTCProuter v0.59 (a build of OpenWRT) #130

Open
sskras opened this issue Mar 14, 2024 · 13 comments
Open

Bootstrapping fails on OpenMPTCProuter v0.59 (a build of OpenWRT) #130

sskras opened this issue Mar 14, 2024 · 13 comments

Comments

@sskras
Copy link

sskras commented Mar 14, 2024

To run the bootstrap I needed to fake groups and whoami executables. The related (sub-) issue: #137

Then it failed due to missing nawk:

root@OpenMPTCProuter:~/debug/src/github.com/NetBSD/pkgsrc.git# bootstrap/bootstrap
  ...
===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -c -o root -g root -m 755 /root/debug/src/github.com/NetBSD/pkgsrc.git/work/pkg_install/add/pkg_add /root/debug/src/github.
com/NetBSD/pkgsrc.git/work/sbin/pkg_add
===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -c -o root -g root -m 755 /root/debug/src/github.com/NetBSD/pkgsrc.git/work/pkg_install/admin/pkg_admin /root/debug/src/git
hub.com/NetBSD/pkgsrc.git/work/sbin/pkg_admin
===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -c -o root -g root -m 755 /root/debug/src/github.com/NetBSD/pkgsrc.git/work/pkg_install/create/pkg_create /root/debug/src/g
ithub.com/NetBSD/pkgsrc.git/work/sbin/pkg_create
===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -c -o root -g root -m 755 /root/debug/src/github.com/NetBSD/pkgsrc.git/work/pkg_install/info/pkg_info /root/debug/src/githu
b.com/NetBSD/pkgsrc.git/work/sbin/pkg_info
sh: -f: not found
bmake[1]: "/root/debug/src/github.com/NetBSD/pkgsrc.git/lang/gcc6/options.mk" line 77: warning: "echo  6.5.0 |   -f ../../mk/scripts/larger_symbol_version.awk" returned non-zero status
sh: -f: not found
bmake[1]: "/root/debug/src/github.com/NetBSD/pkgsrc.git/lang/gcc6/options.mk" line 77: warning: "echo  6.5.0 |   -f ../../mk/scripts/larger_symbol_version.awk" returned non-zero status
sh: {print $3; exit}: not found
bmake[1]: "/root/debug/src/github.com/NetBSD/pkgsrc.git/mk/tools/gmake.mk" line 43: warning: "/usr/bin/make --version |                           '{print $3; exit}'" returned non-zero status
===> running: (cd /root/debug/src/github.com/NetBSD/pkgsrc.git/pkgtools/cwrappers && /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/bmake  MAKE_JOBS=1 PKG_COMPRESSION=none PKGSRC_KEEP_BIN_PKGS=no MAKECON
F=/root/debug/src/github.com/NetBSD/pkgsrc.git/work/mk.conf install)
sh: -f: not found
bmake[1]: "/root/debug/src/github.com/NetBSD/pkgsrc.git/lang/gcc6/options.mk" line 77: warning: "echo  6.5.0 |   -f ../../mk/scripts/larger_symbol_version.awk" returned non-zero status
sh: -f: not found
bmake[1]: "/root/debug/src/github.com/NetBSD/pkgsrc.git/lang/gcc6/options.mk" line 77: warning: "echo  6.5.0 |   -f ../../mk/scripts/larger_symbol_version.awk" returned non-zero status
sh: {print $3; exit}: not found
bmake[1]: "/root/debug/src/github.com/NetBSD/pkgsrc.git/mk/tools/gmake.mk" line 43: warning: "/usr/bin/make --version |                           '{print $3; exit}'" returned non-zero status
/root/debug/src/github.com/NetBSD/pkgsrc.git/mk/pkgformat/pkg/list-dependencies: line 47: /usr/pkg/bin/nawk: not found
===> Installing dependencies for cwrappers-20220403
/root/debug/src/github.com/NetBSD/pkgsrc.git/mk/pkgformat/pkg/list-dependencies: line 47: /usr/pkg/bin/nawk: not found
*** Error code 1

Stop.
bmake: stopped in /root/debug/src/github.com/NetBSD/pkgsrc.git/pkgtools/cwrappers
===> exited with status 1
aborted.

The OS:

root@OpenMPTCProuter:~/debug/src/github.com/NetBSD/pkgsrc.git# uname -a
Linux OpenMPTCProuter 5.4.194 #0 SMP Tue May 17 22:11:28 2022 x86_64 GNU/Linux

root@OpenMPTCProuter:~/debug/src/github.com/NetBSD/pkgsrc.git# cat /etc/os-release
NAME="openmptcprouter"
VERSION="v0.59.1-5.4"
ID="openmptcprouter"
ID_LIKE="lede openwrt"
PRETTY_NAME="openmptcprouter v0.59.1-5.4"
VERSION_ID="v0.59.1-5.4"
HOME_URL="https://openwrt.org/"
BUG_URL="https://bugs.openwrt.org/"
SUPPORT_URL="https://forum.openwrt.org/"
BUILD_ID="r0+16594-ce92de8c8c"
OPENWRT_BOARD="x86/64"
OPENWRT_ARCH="x86_64"
OPENWRT_TAINTS="busybox override"
OPENWRT_DEVICE_MANUFACTURER="OpenWrt"
OPENWRT_DEVICE_MANUFACTURER_URL="https://openwrt.org/"
OPENWRT_DEVICE_PRODUCT="Generic"
OPENWRT_DEVICE_REVISION="v0"
OPENWRT_RELEASE="openmptcprouter v0.59.1-5.4 r0+16594-ce92de8c8c"
@dhgutteridge
Copy link
Member

I don't have an OpenMPTCProuter or OpenWrt environment to test with,
but searches suggest there is a BusyBox non-GNU implementation of awk
included. That wouldn't get picked up with the present assumptions that
some pkgsrc tooling makes. I don't know if that BusyBox version would
be adequate anyway, as various others don't meet pkgsrc expectations.
It's probably simplest to force use of nawk here. Please try this
patch.

--- bootstrap.orig
+++ bootstrap
@@ -743,6 +743,8 @@
 	elif grep -sq '^CHROMEOS_RELEASE_NAME' /etc/lsb-release; then
 		need_awk=yes
 		need_sed=yes
+	elif grep -sq openwrt /etc/os-release; then
+		need_awk=yes
 	fi
 	machine_arch=`uname -m`
 	# Override machine_arch where required.

(Now, I'm also not sure if it's best to just handle OpenWrt and
derivates, or search for a BusyBox indicator instead, etc., but this
should be a starting point. According to BusyBox documentation, their
grep should handle the options I applied.)

You may also find the BusyBox sed implementation is inadequate (though,
from browsing the documentation, it seems it may be fine), in which
case you'd also need to add another line with "need_sed=yes" as well.
(Depending on when you last pulled from GitHub, you'd also need to
update textproc/nbsed first, as it may not build for you without a fix
I applied on the 20th.)

Dave

@jwbowen
Copy link

jwbowen commented Mar 24, 2024 via email

@sskras
Copy link
Author

sskras commented Mar 24, 2024

@dhgutteridge commented 2 days ago:

searches suggest there is a BusyBox non-GNU implementation of awk included.

True:

root@OpenMPTCProuter:~/debug/src/github.com/NetBSD/pkgsrc.git# command -v gawk
root@OpenMPTCProuter:~/debug/src/github.com/NetBSD/pkgsrc.git# command -v nawk
root@OpenMPTCProuter:~/debug/src/github.com/NetBSD/pkgsrc.git# command -v mawk
root@OpenMPTCProuter:~/debug/src/github.com/NetBSD/pkgsrc.git# command -v awk
/usr/bin/awk

That wouldn't get picked up with the present assumptions that some pkgsrc tooling makes.

Can you be more specific, please? I tried looking into mk/tools/tools.Linux.mk:

root@OpenMPTCProuter:~/debug/src/github.com/NetBSD/pkgsrc.git# grep -iC1 awk mk/tools/tools.Linux.mk
.  endif
.  if exists(${_path}/gawk)
TOOLS_PLATFORM.gawk?=           ${_path}/gawk
.  endif
--

# assuming grep/sed/tar/awk are usually the GNU versions, is this safe?

--

TOOLS_PLATFORM.awk?=            ${TOOLS_PLATFORM.gawk}

... but I don't understand a thing yet. :)

I don't know if that BusyBox version would be adequate anyway, as various others don't meet pkgsrc expectations.

The BusyBox version seems to be a mix of features of nawk, gawk and whatnot:
https://wiki.alpinelinux.org/wiki/Awk

It's probably simplest to force use of nawk here. Please try this patch.

Thanks, but before that I would like to understand the picking algorithm. Especially:

  1. Where does the expectation that /usr/pkg/bin/nawk will be present come from?
  2. Why is AWK variable left uninitialized?
    (Which leads to several sh: -f: not found errors near the end)
--- bootstrap.orig
+++ bootstrap
@@ -743,6 +743,8 @@
 	elif grep -sq '^CHROMEOS_RELEASE_NAME' /etc/lsb-release; then
 		need_awk=yes
 		need_sed=yes
+	elif grep -sq openwrt /etc/os-release; then
+		need_awk=yes
 	fi
 	machine_arch=`uname -m`
 	# Override machine_arch where required.

(Now, I'm also not sure if it's best to just handle OpenWrt and derivates, or search for a BusyBox indicator instead, etc., but this should be a starting point. According to BusyBox documentation, their grep should handle the options I applied.)

BTW, grep handles this fine.

Tried going with -A d for bmake:

--- a/bootstrap/bootstrap
+++ b/bootstrap/bootstrap
@@ -1471,13 +1471,13 @@ build_package_nopreserve() {
 # Special packages that we don't want marked with BOOTSTRAP_PKG, but must be
 # built (if required) without -DPKG_PRESERVE set so that they can be deleted.
 #
-use_cwrappers=`(cd $pkgsrcdir/devel/bmake && $bmake show-var VARNAME=_USE_CWRAPPERS)`
+use_cwrappers=`(cd $pkgsrcdir/devel/bmake && $bmake -d A show-var VARNAME=_USE_CWRAPPERS)`
 case "$use_cwrappers" in
 yes)
        build_package_nopreserve "pkgtools/cwrappers"
        ;;
 esac
-use_mktools=`(cd $pkgsrcdir/devel/bmake && $bmake show-var VARNAME=_PKGSRC_USE_MKTOOLS)`
+use_mktools=`(cd $pkgsrcdir/devel/bmake && $bmake -d A show-var VARNAME=_PKGSRC_USE_MKTOOLS)`
 case "$use_mktools" in
 yes)
        build_package_nopreserve "pkgtools/mktools"

... but that left me overwhelmed with the output (which increased from 79k to 30M in size).

Any ideas on how to proceed with finding the root causes?

@dhgutteridge
Copy link
Member

Sorry, I think we're coming with different ideas of what was intended. My assumption was your primary interest was bootstrapping and using pkgsrc ASAP. It's probable you will encounter further issues as you go, so it could turn into a more protracted process to walk through every question that arises. The simple answer without getting into a lot of details is that "stuff will break" if bootstrapping goes awry. You may find make components behave inconsistently, as you see here (${AWK} evaluating to nothing, or to an unexpected default). This is code written by different people at different times.

Yes, some of this could be handled better; I'm not arguing otherwise. If you'd like to have a detailed technical review, that's a perfectly fine thing to request, but it's not something I can prioritize, I'm afraid.

@sskras
Copy link
Author

sskras commented Mar 25, 2024

@dhgutteridge, thanks for making things clear!

It's probable you will encounter further issues as you go, so it could turn into a more protracted process to walk through every question that arises.

I thought about progressing ASAP at first. But the delay in responses made me to consider doing an RCA. Which I think is still a good thing to do in most cases.

The simple answer without getting into a lot of details is that "stuff will break" if bootstrapping goes awry.

[...]

If you'd like to have a detailed technical review, that's a perfectly fine thing to request, but it's not something I can prioritize, I'm afraid.

Fair enough. I am willing to do the research myself, but I need some (at least initial) guidance. It's hard to notice the moment where bootstrapping starts missing something and diverges from the plan.

I'll file a new issue to walk through that in parallel. Or maybe discussing it online like on IRC would be better? I already started a talk on that with @jperkin.

Now back to testing your fix.

@sskras
Copy link
Author

sskras commented Mar 25, 2024

I tried to take the hint from @jwbowen into account:

--- a/bootstrap/bootstrap
+++ b/bootstrap/bootstrap
@@ -748,6 +748,9 @@ Linux)
        elif grep -sq '^CHROMEOS_RELEASE_NAME' /etc/lsb-release; then
                need_awk=yes
                need_sed=yes
+       # Busybox distros have some things broken.
+       elif command -v awk | xargs ls -l | grep -sq busybox; then
+               need_awk=yes
        fi
        machine_arch=`uname -m`
        # Override machine_arch where required.

... and the error changed:

===> bootstrap command: bootstrap/bootstrap 
===> bootstrap started: Mon Mar 25 13:00:12 UTC 2024
===> running: /bin/sed -e 's|@DEFAULT_INSTALL_MODE@|'0755'|' /root/debug/src/github.com/NetBSD/pkgsrc.git/sysutils/install-sh/files/install-sh.in > /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh
===> running: /bin/chmod +x /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh
===> Creating default mk.conf in /root/debug/src/github.com/NetBSD/pkgsrc.git/work
===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -d -o root -g root /root/debug/src/github.com/NetBSD/pkgsrc.git/work/sbin
===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -d -o root -g root /root/debug/src/github.com/NetBSD/pkgsrc.git/work/share/mk
===> Bootstrapping mk-files
===> running: (cd /root/debug/src/github.com/NetBSD/pkgsrc.git/pkgtools/bootstrap-mk-files/files && env CP=/bin/cp  OPSYS=Linux MK_DST=/root/debug/src/github.com/NetBSD/pkgsrc.git/work/share/mk ROOT_GROUP=root ROOT_USER=root SED=/bin/sed SYSCONFDIR=/usr/pkg/etc /bin/sh ./bootstrap.sh)
===> Bootstrapping bmake
===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -d -o root -g root /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bmake
===> running: (cd /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bmake && /bin/sh configure  --prefix=/root/debug/src/github.com/NetBSD/pkgsrc.git/work --with-default-sys-path=/root/debug/src/github.com/NetBSD/pkgsrc.git/work/share/mk --with-machine-arch=x86_64 )
===> running: (cd /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bmake && /bin/sh make-bootstrap.sh)
===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -c -o root -g root -m 755 /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bmake/bmake /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/bmake
===> Building libnbcompat
===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -d -o root -g root /root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat
===> running: (cd /root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat; /bin/sh ./configure  -C --prefix=/usr/pkg --infodir=/usr/pkg/info --mandir=/usr/pkg/man --sysconfdir=/usr/pkg/etc --enable-bsd-getopt --enable-db && /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/bmake  -j1)
===> Bootstrapping awk
===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -d -o root -g root /root/debug/src/github.com/NetBSD/pkgsrc.git/work/awk
===> running: (cd /root/debug/src/github.com/NetBSD/pkgsrc.git/work/awk && /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/bmake  -j1 -f Makefile CC="cc" CFLAGS="")
cc   -c awkgram.tab.c
sh: cc: not found
*** [awkgram.tab.o] Error code 127

bmake: stopped in /root/debug/src/github.com/NetBSD/pkgsrc.git/work/awk
1 error

bmake: stopped in /root/debug/src/github.com/NetBSD/pkgsrc.git/work/awk
===> exited with status 2
aborted.

Now it misses cc, although the above steps like Bootstrapping bmake and Building libnbcompat dealt with no cc just fine.

Is having a cc or ${CC} defined a requirement of pkgsrc? Maybe just another, minor bug?

@sskras
Copy link
Author

sskras commented Mar 25, 2024

Now after defining CC=gcc pkgsrc gets into some circular dependency:

===> Bootstrapping pkgtools
  ...
gcc -L/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -L../lib -o pkg_admin check.o main.o -linstall  -lnbcompat
gcc -DHAVE_CONFIG_H -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/libarchive -I. -I. -I../lib -DBOOTSTRAP -g -O2 -c main.c
gcc -DHAVE_CONFIG_H -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/libarchive -I. -I. -I../lib -DBOOTSTRAP -g -O2 -c perform.c
gcc -DHAVE_CONFIG_H -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/libarchive -I. -I. -I../lib -DBOOTSTRAP -g -O2 -c pl.c
gcc -DHAVE_CONFIG_H -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/libarchive -I. -I. -I../lib -DBOOTSTRAP -g -O2 -c util.c
gcc -DHAVE_CONFIG_H -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/libarchive -I. -I. -I../lib -DBOOTSTRAP -g -O2 -c build.c
gcc -L/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -L../lib -o pkg_create main.o perform.o pl.o util.o build.o -linstall /root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/.libs/libarchive.a   -lnbcompat
gcc -DHAVE_CONFIG_H -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/libarchive -I. -I. -I../lib -DBOOTSTRAP -g -O2 -c main.c
gcc -DHAVE_CONFIG_H -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/libarchive -I. -I. -I../lib -DBOOTSTRAP -g -O2 -c perform.c
gcc -DHAVE_CONFIG_H -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/libarchive -I. -I. -I../lib -DBOOTSTRAP -g -O2 -c show.c
gcc -L/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -L../lib -o pkg_info main.o perform.o show.o -linstall  -lnbcompat
===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -c -o root -g root -m 755 /root/debug/src/github.com/NetBSD/pkgsrc.git/work/pkg_install/add/pkg_add /root/debug/src/github.com/NetBSD/pkgsrc.git/work/sbin/pkg_add
===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -c -o root -g root -m 755 /root/debug/src/github.com/NetBSD/pkgsrc.git/work/pkg_install/admin/pkg_admin /root/debug/src/github.com/NetBSD/pkgsrc.git/work/sbin/pkg_admin
===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -c -o root -g root -m 755 /root/debug/src/github.com/NetBSD/pkgsrc.git/work/pkg_install/create/pkg_create /root/debug/src/github.com/NetBSD/pkgsrc.git/work/sbin/pkg_create
===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -c -o root -g root -m 755 /root/debug/src/github.com/NetBSD/pkgsrc.git/work/pkg_install/info/pkg_info /root/debug/src/github.com/NetBSD/pkgsrc.git/work/sbin/pkg_info
===> running: (cd /root/debug/src/github.com/NetBSD/pkgsrc.git/pkgtools/cwrappers && /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/bmake  MAKE_JOBS=1 PKG_COMPRESSION=none PKGSRC_KEEP_BIN_PKGS=no MAKECONF=/root/debug/src/github.com/NetBSD/pkgsrc.git/work/mk.conf install)
===> Installing dependencies for cwrappers-20220403
=> Tool dependency coreutils>=5.2.1: NOT found
=> Verifying reinstall for ../../sysutils/coreutils
=> Bootstrap dependency digest>=20211023: NOT found
=> Verifying reinstall for ../../pkgtools/digest
===> Installing dependencies for digest-20220214
=> Tool dependency mktools-[0-9]*: NOT found
=> Verifying reinstall for ../../pkgtools/mktools
===> Installing dependencies for mktools-20220614
=> Tool dependency cwrappers>=20150314: NOT found
=> Verifying reinstall for ../../pkgtools/cwrappers
ERROR: This package has set PKG_FAIL_REASON:
ERROR: Circular dependency detected
*** Error code 1

Stop.
bmake[4]: stopped in /root/debug/src/github.com/NetBSD/pkgsrc.git/pkgtools/cwrappers
*** Error code 1

Stop.
bmake[3]: stopped in /root/debug/src/github.com/NetBSD/pkgsrc.git/pkgtools/mktools
*** Error code 1

Stop.
bmake[2]: stopped in /root/debug/src/github.com/NetBSD/pkgsrc.git/pkgtools/digest
*** Error code 1

Stop.
bmake[1]: stopped in /root/debug/src/github.com/NetBSD/pkgsrc.git/sysutils/coreutils
*** Error code 1

Stop.
bmake: stopped in /root/debug/src/github.com/NetBSD/pkgsrc.git/pkgtools/cwrappers
===> exited with status 1
aborted.

@dhgutteridge
Copy link
Member

Now it misses cc, although the above steps like Bootstrapping bmake and Building libnbcompat dealt with no cc just fine.

Is having a cc or ${CC} defined a requirement of pkgsrc? Maybe just another, minor bug?

You haven't shared a complete log, so it's hard to know what's going on in entirety (e.g., with the other two packages that already built). This implies the individual packages have different expectations, but I haven't walked through them.

@dhgutteridge
Copy link
Member

Now after defining CC=gcc pkgsrc gets into some circular dependency:

===> Bootstrapping pkgtools
  ...
gcc -L/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -L../lib -o pkg_admin check.o main.o -linstall  -lnbcompat
gcc -DHAVE_CONFIG_H -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/libarchive -I. -I. -I../lib -DBOOTSTRAP -g -O2 -c main.c
gcc -DHAVE_CONFIG_H -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/libarchive -I. -I. -I../lib -DBOOTSTRAP -g -O2 -c perform.c
gcc -DHAVE_CONFIG_H -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/libarchive -I. -I. -I../lib -DBOOTSTRAP -g -O2 -c pl.c
gcc -DHAVE_CONFIG_H -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/libarchive -I. -I. -I../lib -DBOOTSTRAP -g -O2 -c util.c
gcc -DHAVE_CONFIG_H -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/libarchive -I. -I. -I../lib -DBOOTSTRAP -g -O2 -c build.c
gcc -L/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -L../lib -o pkg_create main.o perform.o pl.o util.o build.o -linstall /root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/.libs/libarchive.a   -lnbcompat
gcc -DHAVE_CONFIG_H -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/libarchive -I. -I. -I../lib -DBOOTSTRAP -g -O2 -c main.c
gcc -DHAVE_CONFIG_H -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/libarchive -I. -I. -I../lib -DBOOTSTRAP -g -O2 -c perform.c
gcc -DHAVE_CONFIG_H -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/libarchive -I. -I. -I../lib -DBOOTSTRAP -g -O2 -c show.c
gcc -L/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -L../lib -o pkg_info main.o perform.o show.o -linstall  -lnbcompat
===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -c -o root -g root -m 755 /root/debug/src/github.com/NetBSD/pkgsrc.git/work/pkg_install/add/pkg_add /root/debug/src/github.com/NetBSD/pkgsrc.git/work/sbin/pkg_add
===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -c -o root -g root -m 755 /root/debug/src/github.com/NetBSD/pkgsrc.git/work/pkg_install/admin/pkg_admin /root/debug/src/github.com/NetBSD/pkgsrc.git/work/sbin/pkg_admin
===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -c -o root -g root -m 755 /root/debug/src/github.com/NetBSD/pkgsrc.git/work/pkg_install/create/pkg_create /root/debug/src/github.com/NetBSD/pkgsrc.git/work/sbin/pkg_create
===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -c -o root -g root -m 755 /root/debug/src/github.com/NetBSD/pkgsrc.git/work/pkg_install/info/pkg_info /root/debug/src/github.com/NetBSD/pkgsrc.git/work/sbin/pkg_info
===> running: (cd /root/debug/src/github.com/NetBSD/pkgsrc.git/pkgtools/cwrappers && /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/bmake  MAKE_JOBS=1 PKG_COMPRESSION=none PKGSRC_KEEP_BIN_PKGS=no MAKECONF=/root/debug/src/github.com/NetBSD/pkgsrc.git/work/mk.conf install)
===> Installing dependencies for cwrappers-20220403
=> Tool dependency coreutils>=5.2.1: NOT found
=> Verifying reinstall for ../../sysutils/coreutils
=> Bootstrap dependency digest>=20211023: NOT found
=> Verifying reinstall for ../../pkgtools/digest
===> Installing dependencies for digest-20220214
=> Tool dependency mktools-[0-9]*: NOT found
=> Verifying reinstall for ../../pkgtools/mktools
===> Installing dependencies for mktools-20220614
=> Tool dependency cwrappers>=20150314: NOT found
=> Verifying reinstall for ../../pkgtools/cwrappers
ERROR: This package has set PKG_FAIL_REASON:
ERROR: Circular dependency detected
*** Error code 1

Stop.
bmake[4]: stopped in /root/debug/src/github.com/NetBSD/pkgsrc.git/pkgtools/cwrappers
*** Error code 1

Stop.
bmake[3]: stopped in /root/debug/src/github.com/NetBSD/pkgsrc.git/pkgtools/mktools
*** Error code 1

Stop.
bmake[2]: stopped in /root/debug/src/github.com/NetBSD/pkgsrc.git/pkgtools/digest
*** Error code 1

Stop.
bmake[1]: stopped in /root/debug/src/github.com/NetBSD/pkgsrc.git/sysutils/coreutils
*** Error code 1

Stop.
bmake: stopped in /root/debug/src/github.com/NetBSD/pkgsrc.git/pkgtools/cwrappers
===> exited with status 1
aborted.

This is a different issue entirely, and one I'm pretty sure isn't specific to your platform/setup. I think it could be reproducible elsewhere. I will look into that aspect.

@dhgutteridge
Copy link
Member

Now after defining CC=gcc pkgsrc gets into some circular dependency:

===> Bootstrapping pkgtools
  ...
gcc -L/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -L../lib -o pkg_admin check.o main.o -linstall  -lnbcompat
gcc -DHAVE_CONFIG_H -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/libarchive -I. -I. -I../lib -DBOOTSTRAP -g -O2 -c main.c
gcc -DHAVE_CONFIG_H -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/libarchive -I. -I. -I../lib -DBOOTSTRAP -g -O2 -c perform.c
gcc -DHAVE_CONFIG_H -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/libarchive -I. -I. -I../lib -DBOOTSTRAP -g -O2 -c pl.c
gcc -DHAVE_CONFIG_H -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/libarchive -I. -I. -I../lib -DBOOTSTRAP -g -O2 -c util.c
gcc -DHAVE_CONFIG_H -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/libarchive -I. -I. -I../lib -DBOOTSTRAP -g -O2 -c build.c
gcc -L/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -L../lib -o pkg_create main.o perform.o pl.o util.o build.o -linstall /root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/.libs/libarchive.a   -lnbcompat
gcc -DHAVE_CONFIG_H -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/libarchive -I. -I. -I../lib -DBOOTSTRAP -g -O2 -c main.c
gcc -DHAVE_CONFIG_H -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/libarchive -I. -I. -I../lib -DBOOTSTRAP -g -O2 -c perform.c
gcc -DHAVE_CONFIG_H -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -I/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libarchive/libarchive -I. -I. -I../lib -DBOOTSTRAP -g -O2 -c show.c
gcc -L/root/debug/src/github.com/NetBSD/pkgsrc.git/work/libnbcompat -L../lib -o pkg_info main.o perform.o show.o -linstall  -lnbcompat
===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -c -o root -g root -m 755 /root/debug/src/github.com/NetBSD/pkgsrc.git/work/pkg_install/add/pkg_add /root/debug/src/github.com/NetBSD/pkgsrc.git/work/sbin/pkg_add
===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -c -o root -g root -m 755 /root/debug/src/github.com/NetBSD/pkgsrc.git/work/pkg_install/admin/pkg_admin /root/debug/src/github.com/NetBSD/pkgsrc.git/work/sbin/pkg_admin
===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -c -o root -g root -m 755 /root/debug/src/github.com/NetBSD/pkgsrc.git/work/pkg_install/create/pkg_create /root/debug/src/github.com/NetBSD/pkgsrc.git/work/sbin/pkg_create
===> running: /bin/sh /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/install-sh -c -o root -g root -m 755 /root/debug/src/github.com/NetBSD/pkgsrc.git/work/pkg_install/info/pkg_info /root/debug/src/github.com/NetBSD/pkgsrc.git/work/sbin/pkg_info
===> running: (cd /root/debug/src/github.com/NetBSD/pkgsrc.git/pkgtools/cwrappers && /root/debug/src/github.com/NetBSD/pkgsrc.git/work/bin/bmake  MAKE_JOBS=1 PKG_COMPRESSION=none PKGSRC_KEEP_BIN_PKGS=no MAKECONF=/root/debug/src/github.com/NetBSD/pkgsrc.git/work/mk.conf install)
===> Installing dependencies for cwrappers-20220403
=> Tool dependency coreutils>=5.2.1: NOT found
=> Verifying reinstall for ../../sysutils/coreutils
=> Bootstrap dependency digest>=20211023: NOT found
=> Verifying reinstall for ../../pkgtools/digest
===> Installing dependencies for digest-20220214
=> Tool dependency mktools-[0-9]*: NOT found
=> Verifying reinstall for ../../pkgtools/mktools
===> Installing dependencies for mktools-20220614
=> Tool dependency cwrappers>=20150314: NOT found
=> Verifying reinstall for ../../pkgtools/cwrappers
ERROR: This package has set PKG_FAIL_REASON:
ERROR: Circular dependency detected
*** Error code 1

Stop.
bmake[4]: stopped in /root/debug/src/github.com/NetBSD/pkgsrc.git/pkgtools/cwrappers
*** Error code 1

Stop.
bmake[3]: stopped in /root/debug/src/github.com/NetBSD/pkgsrc.git/pkgtools/mktools
*** Error code 1

Stop.
bmake[2]: stopped in /root/debug/src/github.com/NetBSD/pkgsrc.git/pkgtools/digest
*** Error code 1

Stop.
bmake[1]: stopped in /root/debug/src/github.com/NetBSD/pkgsrc.git/sysutils/coreutils
*** Error code 1

Stop.
bmake: stopped in /root/debug/src/github.com/NetBSD/pkgsrc.git/pkgtools/cwrappers
===> exited with status 1
aborted.

This could also relate to #124.

@dhgutteridge
Copy link
Member

Was the solution in #124 any help to you?

I wasn't able to reproduce this. I thought a relatively recent change might have created a condition where this could occur (on NetBSD), but it seems not. I'm familiar with situations where this pattern can occur, but I doubt they are relevant to you. (E.g., if someone confuses overriding WRKDIR when they really want WRKOBJDIR, they would get this circular dependency.)

@sskras
Copy link
Author

sskras commented Apr 12, 2024

IIRC it helped. I wanted to file a related bug about pkgsrc not indicating that hostname is missing (and trying to build coreutils instead). IMO it's fair check to do.

Is that (detection) doable – what do you think, David?

PS. Where did you try reproducing it BTW?

@dhgutteridge
Copy link
Member

IIRC it helped. I wanted to file a related bug about pkgsrc not indicating that hostname is missing (and trying to build coreutils instead). IMO it's fair check to do.

Is that (detection) doable – what do you think, David?

Detection is certainly possible. I'm not sure that's the best solution, would need understand it better. Evidently some Linux distros don't include hostname by default. Hard to support N number of Linux distros, each with their own ideas of things. (Doesn't scale well with the small number of pkgsrc developers.) "uname -n" would perhaps be a better choice (i.e. patch the problem software instead of introducing another dependency), but there may be conflicting historical reasons why that wasn't done.

In general, IMO it's not realistic to expect pkgsrc to just work out of the box on any Linux distro without installing certain things in the base system (as opposed to requiring them to build during a bootstrap). Some of this is documented in README.Linux.

PS. Where did you try reproducing it BTW?

NetBSD 9.3_STABLE, which isn't a true "bootstrap" at all (under typical usage), but will pull in that dependency chain when someone starts from scratch. Thought I'd encountered a cyclical problem there back in February, but couldn't reproduce it now.

@sskras sskras changed the title Bootstrapping misses nawk on OpenMPTCProuter v0.59 (a build of OpenWRT) Bootstrapping fails on OpenMPTCProuter v0.59 (a build of OpenWRT) Apr 19, 2024
netbsd-srcmastr pushed a commit that referenced this issue Apr 29, 2024
0.2.6
* Make list tightness match the reference implementation closer (#150,
  Michael Howell). This solves the problem where blank lines in the middle
  of a list are attributed to the list itself instead of the item, making
  its parent list become spuriously loose.
* Fix bug with entities inside link destinations (#149). The bug affects
  cases like this: [link](\!); the backslash escape was being ignored
  here.
* Commonmark.Entity: export pEntity [API change].

0.2.5.1
* Replace source with search in list of block tags. This is a spec 0.31
  change that was forgotten in the last release.

0.2.5
* Fix HTML comment parser to conform to 0.31.2 spec.
* Update spec.txt tests to commonmark-spec 0.31.2.
* Match HTML declaration blocks with lowercase letters (Michael Howell).
* Specifically track the position where enders end (Michael Howell).

0.2.4.1
* Commonmark.Html: Add aria-hidden, d, and viewBox to allowed attributes
  list.
* Correctly merge list blanks with non-list blanks (#133, Michael Howell).
* Do not look for backslashed hard breaks in link titles (#130, Michael
  Howell).
* Work around ghc bug with -K RTS options, to set the stack space properly
  for tests (#129). See https://gitlab.haskell.org/ghc/ghc/-/issues/10445.
* Revert block state completely if lazy line (#126). This fixes an issue
  with lazily-wrapped footnotes.
* Avoid adding trailing newline to list block if it's already there
  (Michael Howell). This fixes tight/loose classification in a few cases.
* Fix incorrectly parsing links with nested [] (Michael Howell).
netbsd-srcmastr pushed a commit that referenced this issue Aug 12, 2024
1.65 -- 2024-08-06
	[BUGFIXES]
	 * Update exotic names test to avoid single quote package separator
	   (GH #128)
	 * Don't loop forever in uniqnum.t (GH #130)
netbsd-srcmastr pushed a commit that referenced this issue Sep 1, 2024
# ragg 1.3.2

* Fix a bug in how groups are defined and recalled

# ragg 1.3.1

* Make sure the linejoin/linemitre setting is honored when drawing rectangles
  (#162)
* Fix a bug with resetting clipping during group and pattern rendering
* Fix scaling of raster glyphs in drawGlyph

# ragg 1.3.0

* Added supported for new graphics enigine features: Groups, paths, luminance
  masks, and glyphs
* Add a switch (defaults to on) to snapping rectangles to the pixel grid when
  they are only drawn with fill (no stroke)
* Fixed a bug causing repeated warnings when using a font at 0 size (#130)
* Silence a bug in clang-ASAN that incorrectly reported sanitiser issues
netbsd-srcmastr pushed a commit that referenced this issue Oct 21, 2024
1.8.2 (2024-09-24)

What's Changed

* Drop commented-out line by @olleolleolle in #108
* Add Ruby 3.1 & 3.2 to CI matrix by @tricknotes in #109
* Fix/redos by @ooooooo-q in #114
* Raise HTTPStatus::BadRequest for requests with invalid/duplicate
  content-length headers by @jeremyevans in #120
* Bump actions/checkout from 3 to 4 by @dependabot in #121
* Improve CI by @hsbt in #123
* Fix WEBrick::TestFileHandler#test_short_filename test not working on mswin
  by @KJTsanaktsidis in #128
* Fix bug chunk extension detection by @jeremyevans in #125
* Fix CI. by @ioquatix in #131
* Merge multiple cookie headers, preserving semantic correctness. by
  @ioquatix in #130
* Test on macos-latest by @byroot in #132
* Require CRLF line endings in request line and headers by @jeremyevans in
  #138
* Prefer squigly heredocs. by @ioquatix in #143
* Only strip space and horizontal tab in headers by @jeremyevans in #141
* Treat missing CRLF separator after headers as an EOFError by @jeremyevans
  in #142
* Return 400 response for chunked requests with unexpected data after chunk
  by @jeremyevans in #136
* Fix reference to URI::REGEXP::PATTERN::HOST by @casperisfine in #144
* Prevent request smuggling by @jeremyevans in #146

New Contributors

* @tricknotes made their first contribution in #109
* @ooooooo-q made their first contribution in #114
* @KJTsanaktsidis made their first contribution in #128
* @byroot made their first contribution in #132
* @casperisfine made their first contribution in #144
netbsd-srcmastr pushed a commit that referenced this issue Oct 26, 2024
(inst/NEWS)
Version 0.7-7
    o Replace Calloc and Free with R_Calloc and R_Free, respectively,
    to conform STRICT.

Version 0.7-6
    o   Fix bugs found by warnings and suppress other nonsesical warnings
    o   Fix partial match warning #132
    o   exclude dropped column from column listing #130
netbsd-srcmastr pushed a commit that referenced this issue Dec 2, 2024
# wk 0.9.4

- Ensure package tests pass against sf 1.0-18 (#224, #225).

# wk 0.9.3

- Ensure package compiles with `STRICT_R_HEADERS=1` (#222).

# wk 0.9.2

- Add `wk_crs()` and `wk_set_crs()` methods for `bbox` (#213)
- Fix wk_trans inconsistent meta flags handling (#217)
- Ensure package builds on arm64 for Windows (#220)

# wk 0.9.1

- Fix format strings/arguments for R-devel (#209).

# wk 0.9.0

## Breaking changes

- The common well-known binary representation of POINT EMPTY (i.e.,
  POINT (nan nan)) is now handled as POINT EMPTY allowing empty points
  to roundtrip through `wkb()` vectors (#196, #204).
- `xy(NA, NA)` is now read as a null feature instead of POINT EMPTY. This
  preserves the invariant that null features can also be identified using
  `is.na()` (#205).
- `xy(NaN, NaN)` is now read as POINT EMPTY and `is.na(xy(NaN, NaN))`
  now returns `FALSE`. This means that both EMPTY and null points can roundtrip
  through `xy()` (#205).

## Bugfixes and improvements

- `wk_meta()` now contains a new column `is_empty`, which is `TRUE`
  for any feature that contains at least one non-empty coordinate. This allows
  more efficient detection of features with zero coordinates (#197, #199).
- Updated PROJ data to use the latest pull of the database packaged with
  PROJ 9.3.0 (#201).
- The wk package now compiles once again on gcc 4.8 (#203, #206).
- Fixed `sfc_writer()` to correctly attach the `classes` attribute to
  sfc output with mixed geometry types (#195).
- Function `sfc_writer()` now has an argument `promote_multi` to write any
  input as the MULTI variant. This makes it more likely that an input vector
  will be read as a single geometry type (#198).
- The `wk_collection_filter()` now correctly increments the `part_id` when
  calling the child handler (@brownag, #194).

# wk 0.8.0

* Added `wkb_to_hex()` (@anthonynorth, #183).
* Implemented `vctrs::vec_proxy_equal()` for `wkb()` vctrs
  (@anthonynorth, #183).
* Fixed `sfc_writer()`, which had returned NULL for some inputs
  (e.g., via `wk_collection()`) (@anthonynorth, #182, #186).
* Added `wk_clockwise()` and `wk_counterclockwise()` to re-wind polygon rings
  (@anthonynorth, #188).
* New replacement-function mode for `wk_coords<-()` for in-place modification
  of coordinates (@mdsumner, #187).
* New function `wk_trans_explicit()` migrated from crs2crs (@mdsumner, #187).

# wk 0.7.3

* Fix tests for updated waldo package (#178).

# wk 0.7.2

* Fix use-after-free warnings.

# wk 0.7.1

* Fix implicit reliance on error `as.data.frame.default()`,
  which no longer occurs in r-devel (#166).

# wk 0.7.0

* Remove legacy headers that are no longer used by any downstream package
  (#146).
* `validate_wk_wkt()` now errors for an object that does not inherit from
  'wk_wkt' (#123, #146).
* Added `wk_crs_projjson()` to get a JSON representation of a CRS object.
  To make lookup possible based on shortcut-style CRS objects (e.g.,
  `"EPSG:4326"` or `4326`), added data objects `wk_proj_crs_view` and
  `wk_proj_crs_json` that contain cached versions of rendered PROJJSON
  based on the latest PROJ version (#147).
* Added a `wk_crs_proj_definition()` method for `wk_crs_inherit()` (#136,
  #147).
* Conversion to sf now uses the `sfc_writer()` for all wk classes, making
  conversions faster and fixing at least one issue with conversion of NA
  geometries to sf (#135).
* `wk_plot()` now plots `NULL`/`NA` geometries and mixed geometry types
  more reliably (#142, #143, #149).
* Exported EMPTY geometries to well-known text now include dimension
  (e.g., `POINT Z EMPTY`) (#141, #150).
* Fixed bug where `wk_polygon()` doubled some points when the input contained
  closed rings (#134, #151).
* Fixed bug where `wk_count()` exposed uninitialized values for empty input
  (#139, #153).
* The `xy_writer()` now opportunistically avoids allocating vectors for Z
  or M values unless they are actually needed (#131, #154).
* Added example WKT for all geometry types and dimensions plus helper
  `wk_example()` to access them and set various properties (#155).
* Fixes warnings when compiling with `-Wstrict-prototypes` (#157, #158).
* Removed `wk_chunk_map_feature()` in favour of using chunking strategies
  directly (#132, #159).
* Optimized `wk_coords()` for `xy()` objects (#138, #160).
* Added accessor methods for record-style vectors: `rct_xmin()`, `rct_xmax()`,
  `rct_ymin()`, `rct_ymax()`, `rct_width()`, `rct_height()`, `crc_center()`,
  `crc_x()`, `crc_y()`, `crc_r()`, `xy_x()`, `xy_y()`, `xy_z()`, and `xy_m()`
  (#144, #161).
* Added rectangle operators `rct_intersects()`, `rct_contains()`,
  and `rct_intersection()` (#161).

# wk 0.6.0

* Fixed `wk_affine_rescale()` to apply the translate and scale
  operations in the correct order (#94).
* Add `wk_handle_slice()` and `wk_chunk_map_feature()` to support
  a chunk + apply workflow when working with large vectors (#101, #107).
* C and R code was rewritten to avoid materializing ALTREP vectors
  (#103, #109).
* Added a `wk_crs_proj_definition()` generic for foreign CRS objects
  (#110, #112).
* Added `wk_crs_longlat()` helper to help promote authority-compliant
  CRS choices (#112).
* Added `wk_is_geodesic()`, `wk_set_geodesic()`, and argument `geodesic`
  in `wkt()` and `wkb()` as a flag for objects whose edges must
  be interpolated along a spherical/ellipsoidal trajectory (#112).
* Added `sf::st_geometry()` and `sf::st_sfc()` methods for wk geometry
  vectors for better integration with sf (#113, #114).
* Refactored well-known text parser to be more reusable and faster
  (#115, #104).
* Minor performance enhancement for `is.na()` and `validate_wk_wkb()`
  when called on a very long `wkb()` vector (#117).
* Fixed issue with `validate_wk_wkb()` and `validate_wk_wkt()`, which failed
  for most valid objects (#119).
* Added `wk_envelope()` and `wk_envelope_handler()` to compute feature-wise
  bounding boxes (#120, #122).
* Fixed headers and tests to pass on big endian systems (#105, #122).
* Incorporated the geodesic attribute into vctrs methods, data frame
  columns, and bbox/envelope calculation (#124, #125).
* Fix `as_xy()` for nested data frames and geodesic objects (#126, #128).
* Remove deprecated `wkb_problems()`, `wkt_problems()`, `wkb_format()`,
  and `wkt_format()` (#129).
* `wk_plot()` is now an S3 generic (#130).

# wk 0.5.0

* Fixed bugs relating to the behaviour of wk classes as
  vectors (#64, #65, #67, #70).
* `crc()` objects are now correctly exported as polygons
  with a closed loop (#66, #70).
* Added `wk_vertices()` and `wk_coords()` to extract individual
  coordinate values from geometries with optional identifying
  information. For advanced users, the `wk_vertex_filter()`
  can be used as part of a pipeline to export coordinates
  as point geometries to another handler (#69, #71).
* Added `wk_flatten()` to extract geometries from collections.
  For advanced users, the `wk_flatten_filter()` can be used as
  part of a pipeline (#75, #78).
* `options("max.print")` is now respected by all vector classes
  (#72, #74).
* Moved implementation of plot methods from wkutils to wk to
  simplify the dependency structure of both packages (#80, #76).
* Added `wk_polygon()`, `wk_linestring()`, and `wk_collection()`
  to construct polygons, lines, and collections. For advanced
  users, `wk_polygon_filter()`, `wk_linestring_filter()`, and
  `wk_collection_filter()` can be used as part of a pipeline
  (#77, #84).
* Added a C-level transform struct that can be used to simplify
  the the common pattern of transforming coordinates. These
  structs can be created by other packages; however, the
  `wk_trans_affine()` and `wk_trans_set()` transforms are
  also built using this feature. These are run using the
  new `wk_transform()` function and power the new
  `wk_set_z()`, `wk_set_m()`, `wk_drop_z()`, `wk_drop_m()`,
  functions (#87, #88, #89).

# wk 0.4.1

* Fix LTO and MacOS 3.6.2 check errors (#61).

# wk 0.4.0

* Removed `wksxp()` in favour of improved `sf::st_sfc()` support
  (#21).
* Rewrite existing readers, writers, and handlers, using
  a new C API (#13).
* Use new C API in favour of header-only approach for all
  wk functions (#19, #22).
* Use cpp11 to manage safe use of callables that may longjmp
  from C++.
* Vector classes now propagate `attr(, "crs")`, and check
  that operations that involve more than one vector have
  compatable CRS objects as determined by `wk_crs_equal()`.
* Added an R-level framework for other packages to implement
  wk readers and handlers: `wk_handle()`, `wk_translate()`,
  and `wk_writer()` (#37).
* Added a native reader and writer for `sf::st_sfc()` objects
  and implemented R-level generics for sfc, sfg, sf, and bbox
  objects (#28, #29, #38, #45).
* Implement `crc()` vector class to represent circles (#40).
* Added a 2D cartesian bounding box handler (`wk_bbox()`) (#42).
* Refactored unit tests reflecting use of the new API and
  for improved test coverage (#44, #45, #46).
* Added `wk_meta()`, `wk_vector_meta()`, and `wk_count()` to
  inspect properties of vectors (#53).
* Modified all internal handlers such that they work with vectors
  of unknown length (#54).

# wk 0.3.4

* Fixed reference to `wkutils::plot.wk_wksxp()`, which
  no longer exists.

# wk 0.3.3

* Fixed WKB import of ZM geometries that do not use EWKB.
* Added `xy()`, `xyz()`, `xym()` and `xyzm()` classes
  to efficiently store point geometries.
* Added the `rct()` vector class to efficiently store
  two-dimensional rectangles.
* Fixed the CRAN check  failure caused by a circular
  dependency with  the wkutils package.
* Added S3 methods to coerce sf objects to and from
  `wkt()`, `wkb()` and `wksxp()`.

# wk 0.3.2

* Fixed EWKB output for collections and multi-geometries
  that included SRID (#3).
* Fixed CRAN check errors related to exception handling on
  MacOS/R 3.6.2.

# wk 0.3.1

* Added a `NEWS.md` file to track changes to the package.
netbsd-srcmastr pushed a commit that referenced this issue Dec 15, 2024
# sass 0.4.9

- Closed #138: font_google(local = TRUE) now uses woff2 (instead of
  woff) for a font file type. (#139)

# sass 0.4.8

- Closed #132: Fix R CMD check warning re error() format strings (for
  r-devel). (#133)

# sass 0.4.7

- Closed #129: Fixed a compilation warning on latest Apple Clang (15). (#130)
- Closed #125: Installation now (correctly) requires `{fs}` >= 1.2.4.
netbsd-srcmastr pushed a commit that referenced this issue Dec 17, 2024
What's Changed

    Fix typo, add links, and formatting by @grahammiln in #128
    README change binary name in usage example by @mikelolasagasti in #126
    gha: update golangci-lint to v1.62.2 by @thaJeztah in #131
    Preserve compatibility with go <= 1.17 by @thaJeztah in #130
    update minimum go version to go1.12, and test in gh by @thaJeztah in #129
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants