Tracking Rust rewrite (feedbacks are welcome) #196
Comments
|
I really believe this idea is worth pursuing. While your argument is that a Rust rewrite would be less portable than current Python implementation (since you can run Python almost anywhere), I don't necessarily think this will be a problem. If anything, a proper Rust rewrite should be more portable in the sense you could provide fully static binaries with no external dependencies - this would essentially eliminate what I think is the only limitation of PMC as it stands now - the need for a working Python installation. With a fully static binary, you would not need to depend on anything at runtime. The external Python dependency is of course not a problem on Linux (given most distros have python preinstalled etc), but I assume it might be a real problem for the average Windows user. The fact that PMC is implemented with the UNIX philosophy of "do one thing and it do it well" (which is why its so good, tbh), it would make sense for the 'core' of the launcher to be available as a Rust crate/shared lib, and the CLI interface as a thin wrapper of that. This would also allow users to write their own GUI wrappers etc. tl;dr: The lack of dependency hell + ability to do static binary builds alone make this effort worthwhile, imo. |
|
Thank you for this valuable opinion 👍 Your point on portability is right, and I never thought of Windows, but this is really true for non-python-dev users. And yes, the more I work with UNIX systems (Linux), the more I want the launcher to precisely follow UNIX philosophy. And since it is now on AUR, this is more relevant than ever because I really don't like package manager over python, because of the interaction between a stateful (any distro manager) manager and a stateless one (pip), this can lead to really annoying situations if you forget that a package was not installed via pip. |
|
If this rewrite happens, it would be great if CLI args could be entered in any order. I haven't any experience with rust, but I know that there are great CLI arg parsing libraries on crates.io. |
|
Yes that would definitely be the case, for now I'll be going on with the well-known "clap" crate. It is quite heavy but also really powerful. Arguments will be accepted in any order within the sub-command they are defined in, exactly like the current behavior with the python implementation. I've seen some projects interfacing with portablemc using the command line, so I really don't want it to break, or at least be backward-compatible (for example I really want a Also, I think that if the server start feature (#200) is implemented at some point, it will be directly in rust. |
|
I'm only a hobbyist programmer with no real experience, so i don't want assume how easy it is to implement anything (and i don't want to push goals that aren't part of the scope of the project), but it would be absolutely amazing if there was a way to download and manage modrinth modpacks via the CLI (essentially how PrismLauncher/MultiMC does). Prismlauncher does have CLI options for doing this, but it still depends the GUI dependencies for QT6. Anyway, thanks for replying, good luck! |
|
I don't know if this is exactly what you are searching for, but there is a project called ferium you can be interested to check. This was already discussed for portablemc, and this would be a bit out of scope, but because I found ferium I abandoned the mod manager idea for PMC. Mainly because I didn't want to reinvent de wheel but also because it's interesting to have a complementary project (in rust!). |
|
I already found Ferium yesterday, it was quite good, but a TUI PrismLauncher type experience would be ideal. |
|
I'm just thinking again of something like |
|
you can always use prismlauncher instead of using curseforge directly you know? |
|
Oo interesting! You know, I've been using PMC for everything for 3 years now so I'm not really aware of the GUI launcher market, thank you! |
|
This is what i was alluding to in my earlier comment, a prismlauncher type TUI would be awesome! |
|
Side note, thanks for 300 stars this is insane! |
|
Perhaps this could be a dependency? https://github.com/modrinth/theseus/tree/master/theseus/src/launcher |
Sorry, I forgot your message... I don't think that this exactly fit what I need for the launcher, it doesn't seem to handle many corner cases that we found over the years with older versions (class path ordering) so for now I'm just going to rewrite everything, largely inspired by the Python module. I prefer doing this, because I have the feeling that we'll avoid a ton of bugs that we already solved in the history of this launcher. |
|
It would be a huge boost to the whole community if you instead contributed that work to theseus. It would probably be easier for you too, because you'd have a starting point (rust wise). Modrinth launcher users would have a better time with legacy versions of the game, you'd have more maintenance behind the lib you're using. shrug |
|
It would be a huge boost, I agree, but in this specific case I'm afraid that the current architecture I'm going on with would clash with their architecture. For example, they are parsing the version metadata (called "version info" in their code) ahead of time, this isn't compatible with many backward fixes PMC is doing, also they are massively using async, and I'm not a huge fan of async for this case (except for downloads!). These are just examples, but I think you can understand why I'm afraid of trying to convince people to tweak Theseus' code to fit PMC CLI needs. However, I would be really happy to contribute to Theseus, in parallel of PMC if possible, to share my knowledge (PMC code itself can be used as a documentation by the way). |
|
Some progress update... This is actually insane, this download is running in a single thread with an async downloader implementation, I'm pretty sure that there is room for improvement, but this is already insanely fast. This is downloading 866 MB in 24 seconds, for full 1.21, this is not counting the JVM because it's not yet ready, and the 24 MB of the client jar file. |
And regarding this, I went back on some of my positions, mostly on ahead of time parsing, because in the end it's so much more practical. |
|
I introduced a standardized cache (see below) to avoid polluting the minecraft directory, for now it will be used for versions and JVM manifests. Basically, this will be used for manifest files that the game don't require in its standard directories. The reasoning behind this is that I want the cache to be easily cleared, and also I wanted to avoid duplicating caches (this is currently the case, the let mut file = dirs::cache_dir().unwrap_or(env::temp_dir());
file.push("portablemc-cache");
file.push(url_digest);Note that the URL digest is the SHA-1 of the URL. |
|
Standard installation is now complete, now I'll be working on how to launch the game! I've added a much more complex and flexible JVM finding policy, that supports static path (with optional version checking), "System" installs finding (reading PATH, and also the standard Linux You can see that every download left an event, everything that is not an API, that downloads file, is now handled identically. |
mindstorm38
changed the title
|
I would LOVE to see this converted to Rust. I'm currently using PortableMC as a backend for a Minecraft launcher I'm making with Tauri (using Rust to run the PMC Python script). It'd greatly simplify things if I could directly interact with PMC in Rust. Can't wait to see where this goes! |
|
Thank you for the support :) this rewrite is on the right track. |
|
How about using Go for that? |
both go and rust produce statically linked executables, but since rust is more hipsterish, perhaps it will attract more potential contributes. |
|
Honestly I don't know Go and I've a pretty good experience in Rust CLI so it's why I'm going with that language, not really for the potential contribute, your point is right tho. |
|
First time launching the game successfully! I need to do a lot of test with all Mojang versions, and then work on reproducing the CLI. I mean, I already launched the game, but now there is support for Microsoft sessions and a lot of Mojang versions are expected to work, and natives are now properly extracted for older versions, the bin directory is now per-installer instead of per-game-instance, this means that if launching 10 instances of the game, those 10 instances will use the same bin directory which I need to test but it should be working perfectly fine (I'm still afraid of race condition with modern LWJGL auto native extract).
|
|
My PC may burn at any time (shared bin directory seems to work just really fine)... |
|
|
|
I just discovered an issue that Edit: Ok so from my observations the game is effectively downloading the assets, and it's working because I've the betacraft proxies working: Edit2: If I let the launcher download and copy (currently linking, but it will be a copy later), the game only downloads resources that doesn't match the metadata returned by http://s3.amazonaws.com/MinecraftResources/: I think that's I'll keep downloading resources with the launcher, but I'll make a pure copy and not sym/hard link of the virtual directory. I wonder why those 3 files are changed, what betacraft proxy is providing? |
|
Hello, I just wanted to pop in and say that a CLI Minecraft launcher is a dream come true for me! It's awesome that this is being rewritten in Rust, I'll try to contribute to this rewrite if I can. |
|
Beautiful! |
|
1.2MB executable, very nice! |
|
Do you plan on exposing C bindings with something like https://github.com/mozilla/cbindgen? (maybe there are other better solutions, this is just the I know about in Rust) This would allow using the PMC lib from any language that supports FFI. |
|
Yes, it's on the todo list! I've never done this, so I'm very curious how it works. As well as a Python binding. |
By the way, have you compiled it to get this number? If so, that's really cool! On the latest commit, in release mode I get 8.4 MB, which is still very good (CLI is now taking some space I guess), but indeed I'll investigate further how to reduce it before releasing. |
Indeed! With the latest commit on release I now too get ~8.1MB (6.4MB stripped). With LTO enabled, this can be brought down to ~5.9MB (5.2MB stripped). Sadly rustc seems to emit a DWARF format that https://github.com/google/bloaty does not understand, which is a great tool for checking which compile unit takes up how much space in the binary ( |
|
Taking some overkill suggestions from https://github.com/johnthagen/min-sized-rust: brings release size down to 4.5MB! But IMO not worth the hassle :D |
|
For final release I think that codegen-units/strip/lto are quite free optimizations, panic is to discuss ^^
Really interesting! Thank you, I didn't know that |
|
Also, we'll need to think about the license to use, because I'm not comfortable using GPL anymore in the Rust ecosystem, I prefer Apache 2/MIT for that, and it should not be a problem (on Rust code only), Python part will remain GPL as long as it exists. But I didn't think of that before, but it's implied that the documentation and basically the whole repo, not only the Python code, are under GPL, so I'll still need to contact anybody that has written documentation. Or maybe I drop everything because documentation will be reworked anyway. Same question for the C/Python bindings. I really want to respect every contributor opinion and properly change the license... |
|
Improving search command... If you have any idea on what to add here, don't hesitate, I also plan on improving the syntax of the query string! |
|
Just something to think about: if the same query returns required/"recommended" java/lwjgl versions, then maybe that might be helpful to include in some cases? Afaik ARM support requires a certain LWJGL version, so knowing which version you could run would be nice etc. But then again, might not make sense as the returned versions are likely just "recommended" versions, and small deviations are allowed in some cases - like the case where you can use a slightly newer lwjgl native lib etc. But this might not apply for all versions. |
|
It's an excellent idea tbh! However, I don't know how to efficiently implement it, because it would require to download all metadata files, this is possible indeed, but will it be performant? With the new cache system, it should be slow only on the first request. |
|
Damn, I forgot java/lwjgl info is returned only on the per-version API :/ Also even though those files are small (~50kb) it would probably still be a bit too slow, not to mention there exists also the possibility of hitting the rate limit when accessing so many version manifests (there seems to be 700+)... Edit: Maybe better solution is to have a subcommand ( |
|
I like this idea, and I had it for a long time (not for versions, but for other game resources, or a command to prune unused libraries...). In the implementation I'm imagining, it would still need to download the version metadata, at least, and the assets index (only if we want to give details on the number of assets and the kind of assets mapping there is). And yes I didn't think of the rate limit because I never reached it, but accessing 700 files might trigger any existing rate limiter. |
|
Also, I'll be supporting environment variable for some common arguments (game directories, maybe start --login), I think it will be a huge improvement! |
I'm currently quite bored with Python packaging and environment in general. This is just an idea and I know the work is really huge (already tried PoC in the past), so this is a really long-term discussion. Here is a little summary of pros/cons, and the prerequisites we want. Feel free to give feedback.
Prerequisites
Pros
pip.Cons
pipand other package managers can help us to make this transparent to users.The text was updated successfully, but these errors were encountered: