Remove pandas<2.1.0 pin #961

Job	Run time
build (false, ubuntu-latest, 4.0.1, 3.8)	22m 15s
build (false, ubuntu-latest, nightly, 3.8)	4m 36s
build (true, ubuntu-latest, 4.0.1, 3.10)	3m 59s
build (false, ubuntu-latest, 1.5.3, 4.0.1, 3.11)	3m 55s
build (false, ubuntu-latest, 1.5.3, 13.0.0, 3.11)	4m 15s
build (false, ubuntu-latest, 5.0.0, 3.9)	4m 41s
build (false, ubuntu-latest, 6.0.1, 3.9)	5m 27s
build (false, ubuntu-latest, 7.0.0, 3.10)	4m 0s
build (false, ubuntu-latest, 8.0.1, 3.10)	4m 35s
build (false, ubuntu-latest, 9.0.0, 3.10)	4m 50s
build (false, ubuntu-latest, 10.0.1, 3.11)	3m 47s
build (false, ubuntu-latest, 11.0.0, 3.11)	4m 21s
build (false, ubuntu-latest, 12.0.0, 3.11)	3m 43s
build (false, ubuntu-latest, 13.0.0, 3.11)	3m 28s
build (false, macos-latest, 4.0.1, 3.8)	4m 39s
	1h 22m 31s

Provide feedback