Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Installing emsdk overwrites node on PATH #1142

Closed
romatthe opened this issue Jan 10, 2017 · 48 comments · May be fixed by #1141
Closed

Installing emsdk overwrites node on PATH #1142

romatthe opened this issue Jan 10, 2017 · 48 comments · May be fixed by #1141

Comments

@romatthe
Copy link

romatthe commented Jan 10, 2017

This is a small but annoying issue. When installing the emscripten sdk, or whenever you install a new version of the toolchain, you're told to execute source ./emsdk_env.sh.

This is rather annoying, as the toolchain comes packaged with its own specific version of node, and this conflicts with all other node tools we're used to using. For example, it put node 4.1.1 or my path while I previously had 7.4.0. I've noticed the same goes for clang, but it doesn't bother my that much. Even when relying on a tool like nvm, it's annoying because switching to another version would essentially break emscripten/emsdk?

If emscripten is reliant on a specific version of node, should it not be its own responsibility to make sure its scripts run with its own internally packaged version? This just seems a tad messy.

(Note: One possible temporary workaround is to simply avoid emsdk_env.sh in this case and just manually add emsdk/upstream/emscripten to your path. That is all that should be needed to get emscripten working on many systems).

@stale
Copy link

stale bot commented Aug 30, 2019

This issue has been automatically marked as stale because there has been no activity in the past 2 years. It will be closed automatically if no further activity occurs in the next 7 days. Feel free to re-open at any time if this issue is still relevant.

@stale stale bot added the wontfix label Aug 30, 2019
@stale stale bot closed this as completed Sep 6, 2019
@umutzd
Copy link

umutzd commented Nov 13, 2019

Any updates?

@kripken
Copy link
Member

kripken commented Nov 13, 2019

Reopening, as this does sound like it would be nice to fix. I think just not adding those things to the path would be fine, as emscripten does have direct paths to them in the config file for its own use. Perhaps someone has time to test that and see if it works? If so let me know if you have any questions about how to modify the emsdk for this!

@theoparis
Copy link

theoparis commented Sep 30, 2021

Any updates again? It's been 2 years and I can't use emscripten without this being fixed as I need node.js 16 and emscripten overwrites my nvm version with 14.

Update: I found https://github.com/Sxxov/emscripten-sdk-npm and it seems to be working, node -v still reports v16.10.0.
Second update: Nevermind, once I ran a source command it started using the wrong node.js version again...

@sbc100
Copy link
Collaborator

sbc100 commented Sep 30, 2021

We no longer inject clang or llvm into the PATH, but we do still inject our node in to the PATH.

I tried to make a change where node would only be added to the PATH if it wasn't already there:
#714

But we decided against landed it.

Do you really need to have emsdk and your own version of node active at the same time in the same shell? Can you explain a little about your use case? If its really probelem maybe we can re-open #714

In #714 it was mentioned that as a workaround you should be able to install and activate just the sub-components of the SDK without also installing node. e.g.:

./emsdk activate releases-upstream-c69458f1bbf3ef5b8da4e934de210659cc9bca04-64bit

@kleisauke
Copy link
Collaborator

kleisauke commented Oct 1, 2021

Note that you could also do:

# Prefer the default system-installed version of Node.js
echo "NODE_JS = '$(which node)'" >> .emscripten

Just before:

# Activate PATH and other environment variables in the current terminal
source ./emsdk_env.sh

To ensure that the Node.js from emsdk doesn't clobber the PATH environment variable. FWIW, in previous versions of emsdk it was also possible to do this:

# Prefer the default system-installed version of Node.js
./emsdk uninstall $(./emsdk list | grep INSTALLED | tr -d \(\*\) | grep node | awk '{print $1}')

(from: #96 (comment))

Just before:

./emsdk activate latest

But that doesn't seems to work anymore (error: tool is not installed and therefore cannot be activated: 'node-14.15.5-64bit').

@david-fong
Copy link

Does the emsdk_env.sh script do anything other than modify the PATH? And will the emscripten tools be okay if its node executable isn't found on the path?

If so, I'm just going to settle for manually adding its stuff to the end of my path and excluding its node executable. I found it surprising (in an annoying way) that my snap installation of node wasn't the one found on my PATH.

@sbc100
Copy link
Collaborator

sbc100 commented Apr 12, 2022

@david-fong that solution should work fine, yes. All you should really need is emcc in your PATH make emsdk work. You don't even need that if you call it directly.

emcc will find emsdk/.emscripten relative to itself and node based on the configured value in emsdk/.emscripten.. so node doesn't need to be in your PATH at all to use emsdk.

@arsnyder16
Copy link

arsnyder16 commented May 4, 2022

@sbc100 I am running into a similar problem. With the release of firefox 100 wasm exceptions seems to now be supported by all major browsers by default. So i was switching my build process to use wasm exceptions, what i found was when using node 14 i was hitting the following error Aborted(CompileError: WebAssembly.instantiate(): Compiling function emscripten-core/emscripten#37 failed: Invalid opcode @+5216). Switching to node 16 works fine.

It seems that emsdk_env.sh is still overwriting node even if it exists in the path contrary to what #714

~# which node
/usr/bin/node
~# nvm use 16
Now using node v16.13.0 (npm v8.1.0)
~# which node
/root/.nvm/versions/node/v16.13.0/bin/node
~# source ~/emsdk/emsdk_env.sh
Adding directories to PATH:
PATH += /root/emsdk
PATH += /root/emsdk/upstream/emscripten
PATH += /root/emsdk/node/14.18.2_64bit/bin

Setting environment variables:
PATH = /root/emsdk:/root/emsdk/upstream/emscripten:/root/emsdk/node/14.18.2_64bit/bin:/root/.nvm/versions/node/v16.13.0/bin:/root/.wasmtime/bin:/root/.wasmer/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
EMSDK = /root/emsdk
EM_CONFIG = /root/emsdk/.emscripten
EMSDK_NODE = /root/emsdk/node/14.18.2_64bit/bin/node
~# which node
/root/emsdk/node/14.18.2_64bit/bin/node

@sbc100
Copy link
Collaborator

sbc100 commented May 4, 2022

We go push back on #714 so it didn't land. I'll see if we can revist.

@juj
Copy link
Collaborator

juj commented May 10, 2022

Originally Emsdk was designed to allow people to control what tools they want to have in their PATH. If a user wanted to use all of Emscripten's provided tools (the most common scenario), they would do e.g.

emsdk activate sdk-upstream-main-64bit

(or some other sdk target, depending on the version they choose)

The SDKs are defined as convenience shorthands to collections of SDK Tools to use at the same time. For example, sdk-upstream-main-64bit is a collection of the following Tools:

  {
    "version": "upstream-main",
    "bitness": 64,
    "uses": ["python-3.9.2-nuget-64bit", "llvm-git-main-64bit", "node-14.18.2-64bit", "emscripten-main-64bit", "binaryen-main-64bit"],
    "os": "win"
  },

This means that typing

emsdk install/activate sdk-upstream-main-64bit

is identical to typing

emsdk install/activate python-3.9.2-nuget-64bit llvm-git-main-64bit node-14.18.2-64bit emscripten-main-64bit binaryen-main-64bit

If I wanted to omit one of the Tools, I can issue an Emsdk install/activate step which skips that tool altogether. E.g.

emsdk install/activate python-3.9.2-nuget-64bit llvm-git-main-64bit emscripten-main-64bit binaryen-main-64bit

does an installation that does not bring Emsdk node into the mix. So if you wanted to drive Emscripten with your own node and not an emsdk provided node, you could do an installation/activation like that.


One should also recognize the behavior of the Emsdk --global option? (this is a Windows specific option) That will definitely globally install node and python to Windows registry PATH that would replace user's own. In such a case, the solution is the same as above. If user has been calling

emsdk activate --global sdk-upstream-main-64bit

it is the same as typing

emsdk activate --global  python-3.9.2-nuget-64bit llvm-git-main-64bit node-14.18.2-64bit emscripten-main-64bit binaryen-main-64bit

i.e. the user is explicitly requesting a global registry PATH installation of python and node. In this case, if it is undesirable, the recommended solution would be not to run that command, but instead do a activate --global with only the tools that one wants to add to Windows PATH registry, e.g.

emsdk activate --global  llvm-git-main-64bit emscripten-main-64bit binaryen-main-64bit

The command prompt emsdk_env intends to enter a prompt environment that contains all the tools that are activated at the time, so it is intentional that this prompt has the Node and Python etc. tools explicitly and up front in PATH (if user installed and activated those tools via emsdk). The intent is to provide developers with a method of entering the exact environment where the same tools are available to the prompt that Emscripten uses.

If the issue is that emsdk_env adds those tools to PATH that clobbers the use of some other commands, then maybe the root realization is that those other commands would not be appropriate to be run under the emsdk_env provided environment? Nothing should force developers to run their own tool command lines under emsdk_env if they so choose.


Emscripten command invocations do not need or care about any of the emsdk provided node and python tools being in the PATH. They only care about the EM_CONFIG file. This ensures that it is possible to invoke Emscripten tools outside emsdk_env environment (using emsdk_env is optional), as long as you provide the appropriate config file, either by using the EM_CONFIG environment variable, or by specifying the configuration using --em-config command line flag. You can see how emsdk_env sets that env. var by checking the print output when entering emsdk_env. So one solution might be to manually set this environment variable.

For example, to run emcc outside the emsdk prompt while pointing it to the configuration that emsdk has provided, one can run

EM_CONFIG=/Users/clb/emsdk/.emscripten emcc tests/hello_world.c -o hello_world.js

or

emcc --em-config /Users/clb/emsdk/.emscripten tests/hello_world.c -o hello_world.js

This kind of use can ensure that people don't get any adjustments to their PATH if not desired.


Overall, if Emsdk did not add the needed tools to PATH, my impression is that there would be more users who would be confused of not getting the tools readily available. E.g. the getting started instructions

./emcc tests/hello_world.c
node a.out.js

would not work out of the box if emsdk did not provide all the tools for the users as necessary.

Also I think the idea of pushing the tools to the PATH last is not great, as it would weaken the reproducibility of emsdk_env itself, and could cause subtleties in reproducing the same builds across systems. Given the above methods of avoiding this PATH pollution, e.g. via manually choosing the Tools to activate, or explicitly specifying --em-config / EM_CONFIG options, I think the current behavior is the most sensible one to have that can cater to all use cases.

@arsnyder16
Copy link

@juj Fair enough i will work around it.

I would rather not have to try and keep up to date with all the dependencies of emscripten because emsdk install/activate sdk-upstream-main-64bit could change

Trying to keep this aligned seems painful as well
If something new is added, that will need tracked.
Some of these include specific version numbers, so any upgrade on the emscripten side would cause churn downstream

emsdk install/activate python-3.9.1-nuget-64bit llvm-git-main-64bit emscripten-main-64bit binaryen-main-64bit

Could there be an omit/exclude flag added instead?

emsdk activate "3.1.10" --exclude node

Let me ask a another question, and more specific to why i need to exclude node. What is the cadence that emscripten will update nodejs? For example emscripten is producing wasm that the node version it includes does not support. That's really the only reason i need to use a different node. I would happily use the bundled nodejs, unfortunately i can't.

For dependencies in the toolchain like clang emscripten point to trunk, are there any plan to keep nodejs more inline with active versions?

@sbc100
Copy link
Collaborator

sbc100 commented May 10, 2022

I can see @juj pointer that changing the defaults might cause more confusion that it solves.

I do want to make a change to allow the behaviour that @arsnyder16 want though. I kind of want the same thing since I have version of node that I like to use in my PATH already.

I kind of like the idea of something like emsdk_env --exclude=node or maybe emsdk_env --minimal. The minimal mode would just be emcc on the PATH and nothing more. No need for EM_CONFIG since emcc will find the config relative to itself. This minimal mode would do minimal necessary to put a working emcc in your PATH.

(BTW in your example above emcc --em-config /Users/clb/emsdk/.emscripten tests/hello_world.c -o hello_world.js, emcc is still required to be in the PATH).

@disarticulate
Copy link

Dont overwrite shell's node. use an environment variable or something else.

@sbc100
Copy link
Collaborator

sbc100 commented Jun 13, 2022

Dont overwrite shell's node. use an environment variable or something else.

@juj makes the opposite argument in #1142. Its not clear which approach is better/easier.

@disarticulate
Copy link

disarticulate commented Jun 13, 2022 via email

@bakkot
Copy link

bakkot commented Jul 17, 2022

@juj:

The command prompt emsdk_env intends to enter a prompt environment that contains all the tools that are activated at the time, so it is intentional that this prompt has the Node and Python etc. tools explicitly and up front in PATH (if user installed and activated those tools via emsdk). The intent is to provide developers with a method of entering the exact environment where the same tools are available to the prompt that Emscripten uses.

If the issue is that emsdk_env adds those tools to PATH that clobbers the use of some other commands, then maybe the root realization is that those other commands would not be appropriate to be run under the emsdk_env provided environment? Nothing should force developers to run their own tool command lines under emsdk_env if they so choose.

I think the main issue here is that the documentation suggests that you should always be in emsdk_env. Specifically, running emsdk activate latest will prompt you to do

Next steps:
- To conveniently access emsdk tools from the command line,
  consider adding the following directories to your PATH:
    /emsdk
    /emsdk/node/14.18.2_64bit/bin
    /emsdk/upstream/emscripten
- This can be done for the current shell by running:
    source "/emsdk/emsdk_env.sh"
- Configure emsdk in your shell startup scripts by running:
    echo 'source "/emsdk/emsdk_env.sh"' >> $HOME/.zprofile`

and it is not at all clear that doing the "Configure emsdk in your shell startup scripts by running [...]" step is also going to clobber the system version of node for any future shell you open. That was very surprising to me.

It sounds like emsdk_env is kind of a specialist thing. Most of the time, I just want to invoke emcc in my shell, not mess around with all the other tools exactly as emcc sees them. So if, as mentioned above, it's possible for emcc to work correctly outside of an emsdk_env, I think that should be the thing suggested by the documentation, and the tools should users in reaching that state, with emsdk_env being reserved for more specialized applications instead of being the default.

If you want to clobber the system version of node when specifically entering an emsdk_env, that's fine, as long as the documentation does not suggest always entering an emsdk_env and it is possible to run emcc without being in emsdk_env.


To put it another way, here's the process I went through:

  • google "install emscripten"
  • docs suggest emsdk, I'll clone that
  • ./emsdk install latest
  • ./emsdk activate latest
  • Output says "Next steps: [...] Configure emsdk in your shell startup scripts by running [...]", ok, I'll do that
  • open a new shell
  • emcc works, nice!
  • node has been downgraded, WTF?

I would like to go through exactly the same process, including the "emcc works in a new shell part", except without the last step.

@sbc100
Copy link
Collaborator

sbc100 commented Jul 17, 2022

Currently we do recommend using emsdk_env whenever you want to run emcc. As it happens you can get away with running emcc without emsdk_env if you manually put emcc in your PATH.. but the recommented way to put emcc in your path is to run emsdk_env. I doubt that recommendation will change.

I think the expression "clobber the system version node" is a little strong. What we are doing is putting the emsdk version of node first in the PATH when emsdk is active, the system version is still there, its just no longer first in the PATH.

@sbc100 sbc100 changed the title Installing emsdk overwrites node/clang on PATH Installing emsdk overwrites node on PATH Jul 17, 2022
@sbc100 sbc100 reopened this Jul 17, 2022
@sbc100
Copy link
Collaborator

sbc100 commented Sep 17, 2022

I sent a message to the list to try to see who out there is depending on this: https://groups.google.com/g/emscripten-discuss/c/KnG7io3zu2Q

@juj
Copy link
Collaborator

juj commented Sep 17, 2022

@juj Fair enough i will work around it.

I would rather not have to try and keep up to date with all the dependencies of emscripten because emsdk install/activate sdk-upstream-main-64bit could change

Agree, this problem makes sense - in order to be able to get "emsdk activate everything - node", one will currently need to first learn and track what "everything" is in order to arrive to that everything minus node. That is a shortcoming.

Could there be an omit/exclude flag added instead?

emsdk activate "3.1.10" --exclude node

This would be a clean feature to add. Name bikeshedding, I'd prefer --skip=node, e.g.

emsdk activate foo bar xyz --skip=node

would match to skip node-*-*bit (i.e. all tools with name node, any version, any bitness)

However, reading further about the confusion, I think this would probably not be a best of all worlds solution, because it would be solving the problem by layering more concepts on people (rather than simplifying). That being said, as a general feature, I think such a --skip= feature would be good to have. (note I think the equals sign there would be preferable, since a command line emsdk install emscripten --skip node python is a bit troublesome for the reader, would prefer that to raise an ambiguity error)

Let me ask a another question, and more specific to why i need to exclude node. What is the cadence that emscripten will update nodejs? For example emscripten is producing wasm that the node version it includes does not support. That's really the only reason i need to use a different node. I would happily use the bundled nodejs, unfortunately i can't.

If/when Emscripten is producing wasm content that Emsdk-bundled node is unable to run, that is by itself an issue with Emsdk, and I think we should address that issue as well. If Emsdk has fallen badly behind, then updating is certainly in order.

Though using a relatively old version in Emsdk to ensure that our CI tests against the "minimum supported version" is also important - but I think here we can explicitly instruct the CI to download the min spec version, and default users to download a new version (in case we get into these types of testing trouble). Maybe straddle different testing targets to download different nodes.

I kind of like the idea of something like emsdk_env --exclude=node or maybe emsdk_env --minimal. The minimal mode would just be emcc on the PATH and nothing more.

I'd recommend against emsdk --minimal, because such a name is introducing a new concept, and requires one to learn what meaning it carries.

Also, if we do a minimal mode that only has Emscripten in PATH and nothing else, then we can still get other users complaining why they don't have a way to say they want Emscripten-provided Python or Clang in PATH, but not Node or Java. The permutations are large, so a general option like --exclude=node seems like it could reach more use cases. (although now we'd be mixing emsdk activate --skip vs emsdk_env --skip uses.. but maybe in general those might apply equally for easier cognitive load)

I think the main issue here is that the documentation suggests that you should always be in emsdk_env.

It sounds like emsdk_env is kind of a specialist thing.

I agree with this characterisation. emsdk_env has intended to be the "full solution in isolation", that would enable the simplest getting started scenarios (just get me everything) and the CI/distribution scenarios (make sure I have the 100% exact same bits across systems). We can not lose these features.

But like people have mentioned in the above comments, they are running into a problem that the node that Emsdk provides is not enough to run their Emscripten produced wasm features. If that is the case, then that issue should specifically be addressed - sounds like Emsdk bundled node has fallen too far behind(?). Please raise specific issues about those (sorry if you did already, my attention bandwidth may be struggling)

Most of the time, I just want to invoke emcc in my shell, not mess around with all the other tools exactly as emcc sees them.

In this scenario, do you want Emscripten to invoke your supplied node, or Emsdk-bundled node? (I presume this?) The issue Emsdk intends to solve is to avoid the possibility that Emsdk gives the users a partial installation of the toolchain where they are unable to compile code, since python or node or java or clang or some other dependency the user has installed would not be up to what Emscripten would expect. I.e. to be able to source the 100% same tools that the Emscripten CIs also are testing.

If #714 (comment), that's fine, as long as the documentation does not suggest always entering an emsdk_env and it is possible to run emcc without being in emsdk_env.

Output says "Next steps: [...] Configure emsdk in your shell startup scripts by running [...]", ok, I'll do that

node has been downgraded, WTF?

I would like to go through exactly the same process, including the "emcc works in a new shell part", except without the last step.

Thanks, now I understand. I agree this is to a great extent a documentation problem. emsdk_env indeed is intended to be entered specifically when one wants to do Emscripten compilation activities - having the getting started docs tell people to put it in their profile does go beyond that usage, and that instruction is what is guiding people astray when they want to do non-Emscripten things in their terminals.

it sounds like @juj is thinking of emsdk_env as being something the user does explicitly when they want to enter the emsdk environment, which means they shouldn't be surprised by having the emsdk-provided versions of tools like node, rather than being something which is done by default in every shell, which is how the documentation currently suggests using it.

I agree with this characterization.

To solve these issues, I recommend that we modify the behavior of emsdk_env by making it also accept a list of tool (/command) names as a parameter, which will tell emsdk_env which tools to add to PATH. For example, if one writes

source ./emsdk_env.sh emcc clang

then emsdk_env would enter an environment where these two tools only will be brought into the user's PATH. The earlier emsdk activate step that wrote Emscripten .config file will still also persist, and will dictate which tools Emscripten compiler itself uses. (so the above command will not redirect Emscripten compiler to utilize user's custom Node.js when compiling)

Then, let's create an alias *, i.e. if one writes

source ./emsdk_env.sh *

then all the tools that were previously activated with emsdk activate will be provided in PATH.

If no parameters are specified, i.e.

source ./emsdk_env.sh

then it will be the same as if one had written

source ./emsdk_env.sh emcc

(this will be a breaking change. For a transition period, we can issue a hint if this produces no node at all in user's PATH)

This way, we can have the getting started docs recommend people to run source ./emsdk_env.sh * in their initial command line on terminal to be able to right after run node hello_world.js, but when we recommend people to add a thing in their ~/.profile, we can recommend them to write source ./emsdk_env.sh in there so that Emsdk won't override node for them (and as the other option, recommend them to write *, or explicitly list out the Emscripten toolchain tools they want to have available 24/7 from emsdk).

Hopefully that would make it easy for people to target all scenarios they wish for, with minimal pain?

@bakkot
Copy link

bakkot commented Sep 17, 2022

To solve these issues, I recommend that we modify the behavior of emsdk_env by making it also accept a list of tool (/command) names as a parameter, which will tell emsdk_env which tools to add to PATH.

That sounds good to me!

This way, we can have the getting started docs recommend people to run source ./emsdk_env.sh * in their initial command line on terminal to be able to right after run node hello_world.js, but when we recommend people to add a thing in their ~/.profile, we can recommend them to write source ./emsdk_env.sh in there so that Emsdk won't override node for them

I would suggest instead testing if node is already available, and then prompting the user to run either source ./emsdk_env.sh emcc or source ./emsdk_env.sh emcc node depending on if it is or is not respectively (i.e., only suggest they use emsdk to set up node in the PATH if they do not already have it). And making the same suggestion for modifying the shell profile. That way the experience is the same during first setup as on subsequent use.

And then source ./emsdk_env.sh * can be reserved only for CI environments.

@curiousdannii
Copy link
Contributor

The above seems more complicated than it should be. While a few people may want to expose emsdk's node, most won't. IMO exposing node should be opt-in not opt-out.

@bakkot
Copy link

bakkot commented Sep 17, 2022

@curiousdannii with the above proposal it would be opt-in, as I read it?

@sbc100
Copy link
Collaborator

sbc100 commented Sep 17, 2022

Modifying emsdk_env like this SGTM. (And yes my reading is that node would then be opt in.. either via writing node or * on the command line.. since he default would be just emcc).

@curiousdannii
Copy link
Contributor

curiousdannii commented Sep 17, 2022

Yes it's opt-in.

But I don't think the script should be looking to see if node already exists in path. The docs shouldn't encourage exposing emsdk's node, but instead should recommend people install node for their OS as normal.

I don't see the wisdom in a * option either. What if a future version adds some new component? Exporting unspecified internal dependencies to the user's path is unsafe and should not be encouraged. It shouldn't be in the docs even as a first step. Even continuing to have a way to expose node is a concession I wish didn't need to happen.

But then I'm not part of the project team, so you decide what you want to do :)

@sbc100
Copy link
Collaborator

sbc100 commented Sep 18, 2022

Sure, I think whether and how we recommend using * is quite a minor detail

The main point is that there exists some way to preserve the old behaviour for those what do want emsdk to be a completely solution (some folks I think do want emsdk to install all dependencies without having to go use apt-get for some of the stuff).

@ymoreau
Copy link

ymoreau commented Oct 6, 2022

It would be interesting to know how many developers using emscripten do not install their own node on their system, it's not a small unknown dependency of emscripten, it's a common package for every dev I know.
It would seem much more natural to me that emscripten requires having your own node on your system, but at least if it has to be included it should not replace the "native" node, that's very counter-intuitive.

@bromagosa
Copy link

Is there no workaround for this issue as of today? I am working both on a project that uses emscripten and an unrelated project that needs Node v18, and I don't want to be forced to manually call source emsdk_env.sh every time I work on the emscripten project. I also would rather not have to modify emsdk.py because (asides from it being really bad practice) I'll totally forget I did so the next time I install emscripten, and I'll again be puzzled by my system Node version being silently trumped by someone else.

Is my best shot sed-ing the emscripten version of Node out of my PATH in my .zshrc? Is there a more elegant solution?

@sbc100
Copy link
Collaborator

sbc100 commented Nov 29, 2022

The simplest solution might just be to avoid using emsdk_env.sh completely. Instead you can simply add emsdk/upstream/emscripten to your PATH.. that alone should be enough to use emcc (in that past emsdk_env.sh did a lot more but these days its not really doing more than just that).

@kleisauke
Copy link
Collaborator

Is there no workaround for this issue as of today?

See #1142 (comment).

@sbc100
Copy link
Collaborator

sbc100 commented Dec 1, 2022

Is there no workaround for this issue as of today?

See #1142 (comment).

IIUC the ideal solution doesn't avoid installing the emsdk version of node, or have emscripten use an unexpected/system version. The goal is to avoid having emsdk's override system node in the PATH, right? Avoiding emsdk node, even within emsdk itself, seems like overkill and could cause other issues.

@kleisauke
Copy link
Collaborator

IIUC the ideal solution doesn't avoid installing the emsdk version of node

As mentioned in that comment, you cannot activate Emscripten without installing the bundled Node.js provided by emsdk.

or have emscripten use an unexpected/system version.

This assumes that the globally-installed Node.js version is newer than what emsdk provides. emsdk currently provides Node.js 14.x, which active support ended one year ago. Security updates for this version ends in 5 months. So, I would argue to at least update the bundled Node.js version to the latest LTS version, which would also fix issues like emscripten-core/emscripten#18084.

FWIW, this workaround is currently used in at least libjxl and wasm-vips to avoid compatibility issues with Wasm SIMD, which requires Node.js 16.4.0 or later to match the final SIMD opcodes.
https://github.com/libjxl/libjxl/blob/a58dd9de85f512d6b8870d2ab40caa8423ae0e51/.github/workflows/build_test.yml#L401
https://github.com/kleisauke/wasm-vips/blob/e89ea44d7376c0bbd35cc3ff28d79d97a6340104/Dockerfile#L23-L24

The goal is to avoid having emsdk's override system node in the PATH, right?

I agree that would be the ideal solution. I only pointed out that comment since he was looking for a workaround.

Avoiding emsdk node, even within emsdk itself, seems like overkill and could cause other issues.

Just wondering, what do you mean with other issues? With the same assumption as above, at least for Emscripten, using a globally-installed Node.js should be safe after PR emscripten-core/emscripten#18070.

Another solution would be that emsdk only installs the bundled Node.js, if a globally-installed Node.js is not available, or if the globally-installed Node.js version is lower than the minimum required Node.js version (or the one provided by emsdk). This would also fix this PATH clobbering issue in most cases.

@sbc100
Copy link
Collaborator

sbc100 commented Dec 1, 2022

Just wondering, what do you mean with other issues? With the same assumption as above, at least for Emscripten, using a globally-installed Node.js should be safe after PR emscripten-core/emscripten#18070.

We only test emcc itself with a specific version of node, so using any other version is vearing into untested teritory. Its probably fine, but we only test our internal compiler tools against that fixed version.

Another solution would be that emsdk only installs the bundled Node.js, if a globally-installed Node.js is not available, or if the globally-installed Node.js version is lower than the minimum required Node.js version (or the one provided by emsdk). This would also fix this PATH clobbering issue in most cases.

I tried to land that solution in #714 but it was deemed to magical. We could try to re-litigate that one. I think its quite a nice solution myself.

@kleisauke
Copy link
Collaborator

We only test emcc itself with a specific version of node, so using any other version is vearing into untested teritory. Its probably fine, but we only test our internal compiler tools against that fixed version.

I understand. There's a test-latest-node CI job in Emscripten, but that currently lacks some coverage as it only runs two tests.

I tried to land that solution in #714 but it was deemed to magical. We could try to re-litigate that one. I think its quite a nice solution myself.

Great, I had forgotten about that PR. IIUC, that only affects the emsdk activate phase and still downloads and extracts the Node.js tarball (or zipball on Windows)?

If so, perhaps a better solution would be to do this check in the emsdk install phase? We should probably only do this if Emscripten is installed (i.e. only for the tags/aliases listed in emscripten-releases-tags.json). This way, users can still install and activate the bundled Node.js provided by emsdk via:

$ ./emsdk install node-14.18.2-64bit
$ ./emsdk active node-14.18.2-64bit

@sbc100
Copy link
Collaborator

sbc100 commented Dec 1, 2022

We only test emcc itself with a specific version of node, so using any other version is vearing into untested teritory. Its probably fine, but we only test our internal compiler tools against that fixed version.

I understand. There's a test-latest-node CI job in Emscripten, but that currently lacks some coverage as it only runs two tests.

I tried to land that solution in #714 but it was deemed to magical. We could try to re-litigate that one. I think its quite a nice solution myself.

Great, I had forgotten about that PR. IIUC, that only affects the emsdk activate phase and still downloads and extracts the Node.js tarball (or zipball on Windows)?

Yes, in fact it still uses the bundled version of node within emscripten (i.e. it sets NODE_JS in the config file). All it does it avoid adding this version of node to the PATH during emsdk_env. In this sense it is a very minimal solution to this bug (which is about PATH conflicts, and not so much about want emscripten to use the host node).

If so, perhaps a better solution would be to do this check in the emsdk install phase? We should probably only do this if Emscripten is installed (i.e. only for the tags/aliases listed in emscripten-releases-tags.json). This way, users can still install and activate the bundled Node.js provided by emsdk via:

$ ./emsdk install node-14.18.2-64bit
$ ./emsdk active node-14.18.2-64bit

That would be an alternative, and slightly more intrusive/risky change yes. It would solve a slightly different issue which is "I want emcc to use my system version of node".

sbc100 added a commit that referenced this issue Dec 4, 2022
This change means that aliases such as `latest` and `3.10.2` can be
used to install jsut the emscipten-releases packages and not the full
SDK with all the dependencies.

This allows uses who do not want to install dependencies such as node,
python or java to still use this package.

These packages were already available to install on their own via
their full names such as:

```
$ ./emsdk install releases-upstream-48ce0b44015d0182fc8c27aa9fbc0a4474b55982-64bi
```

Now these base packages can also be install via:

```
$ ./emsdk install latest-base
```

Fixes: #1142
sbc100 added a commit that referenced this issue Dec 9, 2022
This change means that aliases such as `latest` and `3.10.2` can be
used to install jsut the emscipten-releases packages and not the full
SDK with all the dependencies.

This allows uses who do not want to install dependencies such as node,
python or java to still use this package.

These packages were already available to install on their own via
their full names such as:

```
$ ./emsdk install releases-upstream-48ce0b44015d0182fc8c27aa9fbc0a4474b55982-64bi
```

Now these base packages can also be install via:

```
$ ./emsdk install latest-base
```

Fixes: #1142
@johnhooks
Copy link

The simplest solution might just be to avoid using emsdk_env.sh completely. Instead you can simply add emsdk/upstream/emscripten to your PATH.

@sbc100 Is there anyway to pin the current solution to the top of this issue? It took me a while to find this amongst the full discussion.

As a JavaScript developer learning Emscripten, this was a very confusing. I am using Emscripten along with JavaScript/TypeScript build tools to bundle an application and I am using a very recent version of Node.js.

@sbc100
Copy link
Collaborator

sbc100 commented Jan 22, 2023

(I don't think there is any way to do that, but I will see if I can edit the description of the add a note).

sbc100 added a commit that referenced this issue Jan 30, 2023
This change means that aliases such as `latest` and `3.10.2` can be
used to install jsut the emscipten-releases packages and not the full
SDK with all the dependencies.

This allows uses who do not want to install dependencies such as node,
python or java to still use this package.

These packages were already available to install on their own via
their full names such as:

```
$ ./emsdk install releases-upstream-48ce0b44015d0182fc8c27aa9fbc0a4474b55982-64bi
```

Now these base packages can also be install via:

```
$ ./emsdk install latest-base
```

Fixes: #1142
sbc100 added a commit that referenced this issue Jan 30, 2023
This change means that aliases such as `latest` and `3.10.2` can be
used to install jsut the emscipten-releases packages and not the full
SDK with all the dependencies.

This allows uses who do not want to install dependencies such as node,
python or java to still use this package.

These packages were already available to install on their own via
their full names such as:

```
$ ./emsdk install releases-upstream-48ce0b44015d0182fc8c27aa9fbc0a4474b55982-64bi
```

Now these base packages can also be install via:

```
$ ./emsdk install latest-base
```

Fixes: #1142
sbc100 added a commit that referenced this issue Jan 30, 2023
This change means that aliases such as `latest` and `3.10.2` can be
used to install jsut the emscipten-releases packages and not the full
SDK with all the dependencies.

This allows uses who do not want to install dependencies such as node,
python or java to still use this package.

These packages were already available to install on their own via
their full names such as:

```
$ ./emsdk install releases-upstream-48ce0b44015d0182fc8c27aa9fbc0a4474b55982-64bi
```

Now these base packages can also be install via:

```
$ ./emsdk install latest-base
```

Fixes: #1142
@sbc100
Copy link
Collaborator

sbc100 commented Mar 15, 2023

This was fixed in #1189

@sbc100 sbc100 closed this as completed Mar 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet