SmallVector.h: avoid casting away constness (#821) #1440

Job	Run time
linux-coverage	19m 36s
conan	10m 8s
bindings	25m 18s
build (clang-sanitize, ubuntu-22.04)	1h 18m 16s
build (gcc-release, ubuntu-22.04)	20m 48s
build (gcc-10-release, ubuntu-22.04)	17m 38s
build (gcc-debug-shared, ubuntu-22.04)	13m 59s
build (gcc-debug-noexcept, ubuntu-22.04)	14m 52s
build (macos-release, macos-13)	7m 48s
build (win64-release, windows-latest)	19m 59s
build (win64-debug-shared, windows-latest)	19m 5s
build (win64-debug-noexcept, windows-latest)	15m 21s
	4h 22m 48s

Provide feedback