Skip to content

Commit

Permalink
Fixing "externally-managed-environment" problem with pip (#9)
Browse files Browse the repository at this point in the history
* Using a python venv by default for groundlight python SDK stuff.

* Fixing cv2 issue, and readme install.

* Fixup file location.
  • Loading branch information
robotrapta authored Dec 13, 2023
1 parent 92449ab commit b384791
Show file tree
Hide file tree
Showing 5 changed files with 50 additions and 16 deletions.
3 changes: 1 addition & 2 deletions GL-README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,7 @@ Note that we're tracking the `arm64` branch, not main. (If we build off the mai

## Building Images

You should really build this on an ARM system (e.g. an m7g instance in ec2).
You _can_ build this on an x86 instance, but it will take significantly longer, and it's not fast to start with.
We recommend building on an ARM machine (like an m7g instance in ec2). You can build on an x86 machine, but it will take significantly longer, and it's not fast to start with. Building on ARM-powered Macs seems like a good idea, and "should" work, but isn't tested.

### Building directly

Expand Down
20 changes: 12 additions & 8 deletions stage-gl1/00-install-packages/00-packages
Original file line number Diff line number Diff line change
@@ -1,11 +1,15 @@
# Absolute basics
python3 python3-pip
python3 python-is-python3

# Install python libraries
# (We'd rather install this stuff with pip or poetry, but pip struggles inside chroot for anything
# complicated. So for now this works.)
python3-opencv python3-numpy
python3-pydantic python3-pil python3-yaml
# we don't install things like python3-pip or many libraries here, because pip and apt don't
# get along, so we'll set up a venv later.

# Nice-to-have utilities
iotop nethogs net-tools telnet
# We'll set up a venv to get around this.
python3-venv

# We do install OS's opencv to make sure all the .so files are available.
python3-opencv

# Some nice-to-have utilities
iotop nethogs net-tools telnet
git curl wget
17 changes: 11 additions & 6 deletions stage-gl1/01-install-python-libraries/00-run-chroot.sh
Original file line number Diff line number Diff line change
@@ -1,9 +1,14 @@
#!/bin/bash -e

pipit() {
/usr/bin/pip3 $@ --break-system-packages
}
# set up a groundlight virtual environment for python
python3 -m venv /opt/groundlight/gl-py
chmod a+rw -R /opt/groundlight/gl-py
source /opt/groundlight/gl-py/bin/activate

pipit install groundlight
# Framegrab will try to re-install opencv, which won't go well.
pipit install --no-deps framegrab
# Now install the groundlight python libraries
pip install groundlight
# framegrab will install opencv-python, numpy, and pillow
pip install framegrab

# add a .bashrc entry to activate the groundlight virtual environment
echo "source /opt/groundlight/gl-py/bin/activate" >> /home/${FIRST_USER_NAME}/.bashrc
5 changes: 5 additions & 0 deletions stage-gl1/01-install-python-libraries/01-run.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
#!/bin/bash -e

GL_DIR="${ROOTFS_DIR}/opt/groundlight"

install -v files/gl-py-README.md "${GL_DIR}/gl-py/README.md"
21 changes: 21 additions & 0 deletions stage-gl1/01-install-python-libraries/files/gl-py-README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Groundlight Python Libraries
We have set up a python virtual environment (`venv`) for you to use. To activate it, run:

```
source /opt/groundlight/gl-py/bin/activate
```

(This line is already in your .bashrc file, so you probably don't need to run it yourself.)

Some might think this is unnecessarily complicated, or not how they'd like to manage their
python systems. We understand. We have chosen this approach because it is the most
natural for many users, especially those who "just want to use pip thank you very much."
This attitude is extremely understandable, however modern python (3.11) and debian/ubuntu
systems have made this difficult with the "externally-managed-environment" feature
which basically prevents you from using pip without a virtual environment.

If you like `conda`, `mamba`, `poetry`, `pyenv`, or some other python environment manager,
so do we! Please go ahead and use them. Hopefully this default has not gotten in your way.
If it is, please [get in touch](https://github.com/groundlight/groundlight-pi-gen/issues).

Happy visual understanding!

0 comments on commit b384791

Please sign in to comment.