fix: Add libdir to library search path

[shs96c]

We discovered when dealing with libraries such as psycopg2 that the wheel would attempt to link against libpython.a. This fix points the linker at the correct python version being used.

[aignas]

Because this is gated by a feature toggle, it seems to be low risk to include.

[aignas]

NOTE: this is enabled for both, WORKSPACE and bzlmod: https://rules-python--2476.org.readthedocs.build/en/2476/api/rules_python/python/extensions/pip.html#pip.parse.add_libdir_to_library_search_path

However, bzlmod will not forward this value to underlying whl_library instances, so extension.bzl needs to be updated. I think we should only forward this value to whl_libraries that are following the legacy pip way of downloading deps or downloading an sdist.

[shs96c]

I'm not sure how to do this. Can you give me a pointer in the right direction, please?

[aignas]

Should we instead extend the LDFLAGS instead of replacing the value?

[shs96c]

On line 151 we've checked to make sure that env["LDFLAGS"] hasn't been set. We know we're not replacing anything at this point

[rickeylev]

Just want to respond to something the comment says:

# [calling the python interpreter in repo-phase]
# is super hacky, but working out something nice is tricky.
# This is in particular needed for psycopg2 which attempts to link libpython.a,
# in order to point the linker at the correct python intepreter.

It's not possible to reliably get the correct Python interpreter. This is because the repo phase doesn't know, and can't know, what the build phase is going to do (i.e. what python interpreter will be used). The basic change in this PR is somewhat an improvement, but yeah -- just wanted to point out it'll still be prone to issues. To really fix this, the sdist building needs to move into the build phase.

[aignas]

I'll add some ideas about this in #2410.

[shs96c]

Is there anything else you'd like me to do for this PR?