Rewrite launcher for v4

[mindstorm38]

The launcher really needs to be rewritten, the current architecture, single-file oriented, gets more and more complicated over time and doesn't scale with the new forge add-on. I want to simplify the API architecture for beginners.

[mindstorm38]

I found a good architecture for the rewrite, looks promising. It involves more split source files (goodbye to the 2k lines file) and way more static type hinting.

[mindstorm38]

I'm wondering if it would be a good idea to include the mod loader add-ons into the core. Add-ons will still be a thing but with the rewrite it would be possible to add more source files, and it would simplify installation.

If you read this I'm interested in your opinion on this!

[Ristovski]

Personally, I think that the most common modloaders should be included into the core. One of the immediate effects of this would be a much simpler installation method, as you already mentioned.

More exotic and niche modloaders can be left to be implemented as an addon, along with other "specialized" functionality like #20.

[GoodDay360]

If you're going to rewrite the launcher, it would be a great idea if you could make a console interface like the below image.
It kinda looks good and is easy to navigate between options.
And also make the CLI easier to work with. The current CLI is a pain to use and won't be able to get the accurate status of what it doing.

[mindstorm38]

Thank you for the feedback, can you elaborate on the pain to use CLI. I'm really interested in how to improve it!
@GoodDay360

[Ristovski]

While I do agree a more fancier CLI would be nice (perhaps an addon?), I think it is still important to keep something like the current CLI interface for its simplicity.

At the very least, it should still handle running in non-exotic and non-interactive environments.

[GoodDay360]

Thank you for the feedback, can you elaborate on the pain to use CLI. I'm really interested in how to improve it!
@GoodDay360

CLI refers to portablemc API. It was very confusing to understand. Long ago I was working with your CLI it took me 3 whole days to get it to work because while working on it I ran into too many logical errors that didn't tell me what the problem was so I had to figure it out myself.
Overall it works great but the API and API Document kinda not go together expectedly. For better API structure you can follow the art style of this python lib: minecraft-launcher-lib

[mindstorm38]

At the very least, it should still handle running in non-exotic and non-interactive environments.

This is one of my top priorities!

I'm trying to improve the API, but to be honest I'm far from the API of minecraft-launcher-lib, which seems to me a little bit too simple with single function that do all the work (reminds me of OpenGL vs Vulkan). I'm still trying to improve it in order to also provide this kind of functions, but my goal is also to provides high modularity, which is not really the case in current version, imho.

For the GUI, it will be complicated to include this as-is in the core, because one of the thing I want to keep is the zero required dependency. Maybe a minimal interactive interface?

Also, I think that the CLI will remain almost the same, except for the login which will defaults to microsoft login.

[GoodDay360]

At the very least, it should still handle running in non-exotic and non-interactive environments.
This is one of my top priorities!

I'm trying to improve the API, but to be honest I'm far from the API of minecraft-launcher-lib, which seems to me a little bit too simple with single function that do all the work (reminds me of OpenGL vs Vulkan). I'm still trying to improve it in order to also provide this kind of functions, but my goal is also to provides high modularity, which is not really the case in current version, imho.

For the GUI, it will be complicated to include this as-is in the core, because one of the thing I want to keep is the zero required dependency. Maybe a minimal interactive interface?

Also, I think that the CLI will remain almost the same, except for the login which will defaults to microsoft login.

Matter of fact if you're worried about manual installation of the dependency, you can include the dependency in your pypi and it will auto install for the user. So there is no harm in just adding a bit more KB to the disk for better experience. I recommend using this text console UI library: Textual. I haven't researched much about it but I guess it should work on non-exotic and non-interactive environments. If you want it on a more non-interactive don't worry about it there no way someone playing Minecraft without mouse...
https://user-images.githubusercontent.com/554369/197355913-65d3c125-493d-4c05-a590-5311f16c40ff.mov

[mindstorm38]

For the dependency I know that I can install dependency by I generally don't like dependency management in Python (version conflict, downgrades, ...). For this case it will only be one dependency but I'm afraid we'll get used to it and maybe add more in the future.

For the UI itself, it'll require a lot of work in addition to the rewrite. I'm also worried that the launcher would loose its UNIX philosophy about doing one thing well (installing and launcher Minecraft from command line, on most systems as possible, as fast as possible). To be honest I'm not currently convinced by this feature. By the way I'm going to improve this by improving the CLI to allow "machine readable" output, that could be parsed by other programs.

[GoodDay360]

For the dependency I know that I can install dependency by I generally don't like dependency management in Python (version conflict, downgrades, ...). For this case it will only be one dependency but I'm afraid we'll get used to it and maybe add more in the future.

For the UI itself, it'll require a lot of work in addition to the rewrite. I'm also worried that the launcher would loose its UNIX philosophy about doing one thing well (installing and launcher Minecraft from command line, on most systems as possible, as fast as possible). To be honest I'm not currently convinced by this feature. By the way I'm going to improve this by improving the CLI to allow "machine readable" output, that could be parsed by other programs.

Best of luck 🍀. Do what u are satisfied with. Hope the new update will give us a big surprise.

[mindstorm38]

I'll give a lot of updates here, so that we can build a better interface together 🚀

[GoodDay360]

I'll give a lot of updates here, so that we can build a better interface together 🚀

Yup let's just add all. UI, no UI or so what ever let just add all 😅 so that the user can pick what they like.

[mindstorm38]

Because I've time, here's a summary of how the API is going to be...

The new architecture will be focused around tasks executed by an installer, with a shared state and watchers (cool new feature!).

Tasks

The tasks are simple classes implementing an execute(self, state) function, this function takes a state parameter which is a particular type of dictionary, which associate a value to its type. So for example, val = state[MyType], if existing, val will be a instance of the type MyType, and state.insert(MyType("some parameter")) can be used to insert a value in the shared state. I made this because it helps a lot with static type analysis, the type of the value can be statically determined!
Note that I currently don't know how the tasks are going to call back watchers, maybe using a second argument, TBD.

Installer

The installer is basically defined by a sequence of tasks, the internal shared state and watchers. The installer can be simply executed with an execute function that will just execute each of the task in sequence (no parallelism intended).

Watcher

Watchers are simple class that receives events when new tasks are executed and when these tasks trigger events, this will be used for CLI, to avoid the old problem CLI where I was forced to extend classes to add more precise logging (I hate that).

Why?

The problem with the current API is how the installation procedure has to be done in the right order by the frontend using the API. For example in the CLI, because I want to log each step (metadata, libraries, assets, ...) in the terminal, the CLI is required to call each of these functions from the Version instance, in the right order. But imagine if tomorrow I want to change the specification and I introduce a new step (like finalize that I wanted to add for the forge add-on), it would break every frontend implementation instantly, because they would no longer call all required functions, this is big problem. This is where the new API is perfect, because the frontend will no longer be calling each step (called tasks in the new API), this will be the installer's job. And the CLI will just a standalone watcher of this installer.

The version's installer will remain highly customizable, because it will remains possible to insert task in it (it will be the case for forge versions), but modifying it is no longer required to have a proper logging. The API will provide functions like make_vanilla_installer(version: str) -> Installer to be used like this:

installer = make_vanilla_installer("release")
installer.execute()
# Game is installed.

However I've not yet decided how to start the game, the installer's state should contain enough information to know how to run the game. Maybe a step of the installer could add a Start instance to the shared state, that will be possible to start with installer.state[Start].start(), a bit verbose here I must admit. Or Start(installer).start()..

Note

Download will be multi-threaded!

[GoodDay360]

I think making everything bundled in class PMC: as a main parent prob good and for managing stuff do something like this:

isInstalled = PMC.check_loader("vanilla", "1.18.2", "./mc_dir")
if not isInstalled:
 PMC.installer("vanilla", "1.18.2", "./mc_dir/", callback = {
     "setStatus": some_function, # This function is called to set a text
     "setProgress" some_function, # This function is called to set the progress.
     "setMax": some_function, # This function is called to set to max progress.
 })

# Do the same thing like above for checking if Java installed

PMC.args([...])
PMC.auth.Yggdrasil(...)...

# For starting the game
PMC.start(callback->dict)
# At this step I don't even know if it is possible to add call back function for logging

[mindstorm38]

This will be available in the portablemc package, so it can be imported import portablemc as pmc and everything useful for starting up the game will be re-exported to this module. It's also a bad practice imo to use classes as namespaces, better to use the module itself as the namespace. We don't have the same constraint as Java where it's required!

[mindstorm38]

Full Minecraft installation... The standard installer is properly working!

Now I'll be working on the CLI and/or the run part (and I'm still missing JVM install).

[mindstorm38]

I lied! There was a bug, the real time is 68 seconds! (on 4 cores, reaching up to 110 Mbps!)

[mindstorm38]

Now that I doubled the number of threads, it takes 36 seconds, reaching 17 Mo/s with 8 threads. It seems that it scales above the core's count because blocking I/O operations can easily be multi-threaded.

Edit: 16 threads gives 27.4 Mo/s (peaking at 60 Mo/s) in 22 s

[eudach]

It's incredible the job that you are doing

[mindstorm38]

Still working on it, the hard part is the start process but it's looking good for now. The first v4 pre-releases will not support add-ons I think, but forge/fabric/quilt will be integrated.

[mindstorm38]

Version 4.0.0 is almost ready, remaining to-dos:

• LWJGL version fix
• Include bin
• Exclude lib
• Start the game

[mindstorm38]

I forgot mod loaders in the previous message, but I'm also working on them, and it's working well for the moment. It's getting complicated on the API side, but this allows high flexibility. Now I'll have to make simpler functions, with less flexibility but easier to use and with less boilerplate.

The big remaining part is including LWJGL fix into the API (it was previously done only by the CLI).

[mindstorm38]

👀 machine readable output

👀 human readable output is colored by default

[mindstorm38]

A lot of progress today!
Remaining :

• If prompt-toolkit is available, start the beautiful console!
• Support forge add-on by default (the new version, that I was working on before the rewrite, which made me start this rewrite in the first place).

Yes, if I'm correct, this is the only work remaining!

Edit: I was missing that all tests are now broken...

[mindstorm38]

I can't wait to show you how fast and easy forge can be installed

[mindstorm38]

Huge! 46 sec to install, 1 min 10 sec to run the game.

https://youtu.be/xd9QidKgdTM (please share 👍)

[mindstorm38]

Minecraft and Forge are actually so annoying lmao

For Minecraft 1.5.2 to run with forge, the main class has to be put ahead of the global class path. This is so annoying because this is exactly what we tried to fix with #125

Minecraft 1.5.2 also crashes when modern options.txt file is present... because of the language name not being exact.

[mindstorm38]

@onebyte4 Hi! Do you remember which mod loaders were fixed by this PR #125 ? It's true that Mojang's launcher is placing the path at the end, but the older launcher didn't do that like this, and this doesn't seem to be a problem with all forge versions between 1.5.2 and 1.20.1! Maybe I can check the game's version to choose? Same thing that for legacy fixes.

[mindstorm38]

Thanks to betacraft proxies, I can see my skin in beta 1.0 🤯

[mindstorm38]

I'm updating documentation, and the launcher will be ready for the first pre-release.

[mindstorm38]

I'm trying another approach for on the API side, without all this sequence/task/state stuff, a lot closer to what currently is the launcher (v3). This is happening in rewrite-v4-try0 branch, tomorrow I'll try to improve again this architecture by hiding all the intermediate state as local variables in the install method.

Example: Version(ctx, "release").install().run(), "release" can be replaced by any version like 1.19.2. Mod loaders available using proper subclasses FabricVersion(ctx, "release") or ForgeVersion(ctx, "release"). Note that the version and context may become optional in constructor I think, my goal is to have good defaults.

Still on time for pre-release this weekend!

[mindstorm38]

Ready for pre-release! Everything has been updated, documentation, and the new API that I was reworking this week is much better than my previous idea of sequence/tasks, so I decided to keep it.

The new README is available at https://github.com/mindstorm38/portablemc/tree/v4-rc
And the new API documentation at https://github.com/mindstorm38/portablemc/blob/v4-rc/doc/API.md
You can track the pull request at #159

Here is it: https://pypi.org/project/portablemc/4.0.0rc1/
Install with pip install portablemc==4.0.0rc1

I'd be happy if you review this version (and its documentation!), so I can actually release it!

[mindstorm38]

Sadly, I can already spot a few problems :/

• Not fixed: JAR tool not found on startup #156
• Argument --jvm-args is ignored...

[mindstorm38]

Pre-release 2 available, fixing problems described above: https://pypi.org/project/portablemc/4.0.0rc2/

Install with pip install portablemc==4.0.0rc2

Now I'll write the changelog.

[GoodDay360]

Pre-release 2 available, fixing problems described above: https://pypi.org/project/portablemc/4.0.0rc2/

Install with pip install portablemc==4.0.0rc2

Now I'll write the changelog.

The not fun part is writing documents.

[mindstorm38]

I actually like writing changelogs, it's so satisfying to see all the work done and all the contributions!

Here is it by the way: #159

[mindstorm38]

Pre-release 3 available now: https://pypi.org/project/portablemc/4.0.0rc3/
Try with pip install portablemc==4.0.0rc3

I hope this is the last pre-release! We fixed a lot of issues thanks to @rutexd on #133, downloads are now more reliable (on paper).

[Ristovski]

I've been testing 4.0.0rc3, and so far it's been working great! Good job on the rewrite.

[mindstorm38]

Released!
Changelog: https://github.com/mindstorm38/portablemc/releases/tag/v4.0.0
Release page: https://pypi.org/project/portablemc/4.0.0/
Install with pip install portablemc==4.0.0

[rutexd]

Awesome! 😍