Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Player head management plugin #888

Merged

Conversation

SpecialBuilder32
Copy link
Member

@SpecialBuilder32 SpecialBuilder32 commented Jul 26, 2023

This PR is the mostly finished beet tooling that helps manage the custom-player-head-skins we use throughout our datapacks. It replaces the need for the developer to manually transform skin images into the gibberish base64 string that represents the image in an item's nbt, instead storing the skin image file directly in the source code, and using mineskin's api when a skin needs to be sent to Mojang's servers.

This PR also introduces Mecha to the project for the first time, and serves as a prototype for the eventual CustomModelData handler that is planned down the road.

Usage

Skin files placed in the data/namespace/skins directory are accessible via their file names when writing nbt compounds:
{SkullOwner:"$gm4_heart_canisters:heart_canister_tier_1"}

Skin references are prefaced by $, and will assume the namespace to be the project_id (usually the root folder name) if omitted. The following shorthand and regular forms are accepted, and the Name and Signature fields are optional, to be used as needed by modules like lib_machines

  • {SkullOwner:"$skin_name"}
  • {SkullOwner:{Value:"$skin_name"}
  • {SkullOwner:{Value:"$skin_name", Name:"foo", Signature:"bar"}
  • {SkullOwner:{Name:"foo", Properties:{textures:[{Value:"$skin_name", Signature:"bar"}]}}}

Although mecha does not currently parse and process non-function files, nbt fields within loot tables, item modifier and advancements are manually passed into the mecha parser and will be properly filled out.

Skin cache

All data about skins and their in-game texture values is stored in a skin_cache.json file. The hash of each skin image is used to ensure api calls are not duplicated and successive beet build calls produce consistent item data. If a developer is changing skin textures, this file will be changed by beet, and must be included in commits to the github repo.

Since the github build action won't modify the master branch of the repo, it is unable to upload skins to mineskin and cache the result. Developers changing skin images must therefore run at least one beet build or beet dev on affected modules locally after editing images, and commit the resulting cache file. The GitHub action is marked as failed if it finds a skin image that has not been uploaded to mineskin by a developer.

If skin textures are removed from the repo, the skin cache will not be automatically cleaned up, and a message informing the developer of the unused cached texture data emitted to the log. Manually editing the skin cache to remove the unneeded entry is suggested

Mineskin authentication

Since uploading skins to mineskin's api requires an api key, the beet build process will ask for an authentication token in the terminal if any skin file changes are made. Obtaining a token is simple; simply sign up for a mineskin account using a google account login, and create a token. It is not necessary to provide mineskin with your minecraft account details, but you may if you wish your account to be used to upload the skin changes and not queued to one of the many volunteer accounts mineskin has in its possession.

Your authentication token will be cached locally by beet in .beet_cache/mineskin/index.json, and can be manually deleted or removed by using beet cache --clear.

Skin() class

Beet plugins can programmatically create and edit skin files much like other image files in the beet ecosystem. The gm4.plugins.player_heads python module defines a Skin class which attaches to the container skin image files are mounted to automatically.

from gm4.plugins.player_heads import Skin
from PIL import Image
def beet_default(ctx: Context):
    ctx.data[Skin]["gm4_end_fishing:enderpuff"] = Skin(Image.new("RGB", (64, 64), color="lime"))

Mecha Details

So this PR introduces mecha to the project for the first time, and that both opens the floodgates for loads of new powerful beet tools (like bolt), and adds some more complexity to be aware of. Here are some of the details I thought were important to share.

Mecha is a parser, which means that it reads in all function files, processes all the commands, nbt data ect into an abstract syntax tree. This means the build process now can detect most syntax errors without needing to check the output log of minecraft (at least in function files). You'll see nicely formatted diagnostic errors in the terminal for syntax errors.

We can write rules that interact with nodes of this tree very nicely; this PR adds a rule that edits specifically the SkullOwner nbt nodes. As it is now, mecha doesn't do anything with comments, and by default strips them out of the built files. There is a "development only" option to preserve the comments and newlines which I have applied to our project, but this may result in strange spacings if we dive more into bolt modules in the future. There are some plans to add a third formatting option that keeps comments and does bolt spacing better, but that is just a wishlist for now. Just keep in mind this fact if you start doing weird things

At the moment, mecha is only run on module files, not libraries (except CC). This is mostly to is to keep the nice isolated-subproject nature of the build process, and to keep processing to a minimal since libraries don't need mecha atm.

Bloo-dev and others added 30 commits April 26, 2021 11:25
Enhance code and examples (version 1.1)
This is a very very gross and terribly code implementation to extract existing skin data into a cache file. This code was half to learn how to interact with mecha AST rules. Most of this commit's changes *will* be reverted or overwritten in a later commit
A quick-and-dirty utility script to process the texture url data from the previous commit and interactively download and name each skin file. This script will be deleted in a future commit but is included here so it can be retrieved by git at a future date if needed
Creates and registers a custom container on the datapack object for skin textures, allowing them to be auto-mounted to the pack object upon load
- adds PIL (pillow) as dependancy
- serializes skin images immediately on load to prepare for hash comparison
First attempt to match the ast structure of the SkullOwner nbt compound. Ran into issues where the ast stores ordered entries, but we want to match unordered.
Matches the following cases of SkullOwner:
```SkullOwner:$texture
SkullOwner:{Value:"$texture"}
SkullOwner:{Value:"$texture",Name:"foo",Signature:"bar"}
SkullOwner:{Name:"foo",Properties:{textures:[{Value:"$texture",Signature:"bar"}]}}```,
with each additional field functioning as optional if needed
Apparently, the uuid no longer *must* be unique for the skin to work, but for safety we'll keep using a unique uuid since one is already provided by mineskin.
Since we're faking database entries, the lookup key for loottables needs to be unique, which it cant be when one file has two strings needing parsing. Instead we use an enumerated resource location
Also updates mecha to fix a bug with raising diagnostics within subcommands
uses mecha's `yield Diagnostic` magic syntax to both raise errors and return a valid ast, which gets a magenta/black missing-texture
SpecialBuilder32 and others added 16 commits July 23, 2023 02:12
Moves gm4.output to use its exit phase instead, this allows all other plugins to process their exit phases first, doing things like cleaning up extra file containers or saving important data
This reverts commit ac60b4d.

This functionality would be better served in some hypothetical linting checker rather than here; additionally, this should ideally belong to the library files itself, not the player head management tool. For now, the maintainers will need to keep in mind that skin file changes should accompany lib_player_heads registry edits
@SpecialBuilder32 SpecialBuilder32 added the quality-update Improves efficiency or structure without affecting functionality of a module label Jul 26, 2023
@SpecialBuilder32 SpecialBuilder32 requested a review from misode July 26, 2023 05:43
@SpecialBuilder32 SpecialBuilder32 self-assigned this Jul 26, 2023
@SpecialBuilder32
Copy link
Member Author

This PR is functionally complete. The only places that aren't fully converted to the new skin reference system are in some lib_player_heads setup stuff, which is broken to anyway and I've summitted a bug report so we can fix it at a later time.

This PR has some minor elements that setup other indev PRs using mecha in the future

@SpecialBuilder32 SpecialBuilder32 merged commit dd1ddc7 into Gamemode4Dev:master Sep 7, 2023
2 checks passed
github-actions bot pushed a commit that referenced this pull request Sep 7, 2023
* Merge branch 'ver/1.16' of https://github.com/Bloo-dev/lib_player_heads into ver/1.16

* Merge pull request #1 from Luexa/enhance-code-and-examples

Enhance code and examples (version 1.1)

* Recursive json-dict key retrieval

* Dirty existing skin data gathering

This is a very very gross and terribly code implementation to extract existing skin data into a cache file. This code was half to learn how to interact with mecha AST rules. Most of this commit's changes *will* be reverted or overwritten in a later commit

* Dirty skin download to source

A quick-and-dirty utility script to process the texture url data from the previous commit and interactively download and name each skin file. This script will be deleted in a future commit but is included here so it can be retrieved by git at a future date if needed

* Custom container for skin textures

Creates and registers a custom container on the datapack object for skin textures, allowing them to be auto-mounted to the pack object upon load

* Use bind() to process Skin objects

- adds PIL (pillow) as dependancy
- serializes skin images immediately on load to prepare for hash comparison

* Bind the Skin container to a custom mecha rule processor

* Structural pattern matching on AST

First attempt to match the ast structure of the SkullOwner nbt compound. Ran into issues where the ast stores ordered entries, but we want to match unordered.

* Evaluate ast to libnbt dict for robust syntax matching

Matches the following cases of SkullOwner:
```SkullOwner:$texture
SkullOwner:{Value:"$texture"}
SkullOwner:{Value:"$texture",Name:"foo",Signature:"bar"}
SkullOwner:{Name:"foo",Properties:{textures:[{Value:"$texture",Signature:"bar"}]}}```,
with each additional field functioning as optional if needed

* mineskin_upload function for api calls

* trims unneeded data from skin texture url

* Use provided random uuid from mineskin

Apparently, the uuid no longer *must* be unique for the skin to work, but for safety we'll keep using a unique uuid since one is already provided by mineskin.

* Save skin cache out when finished

* Prompt the user for an API token if needed for submitting new skin textures

* cached uuid is list of ints

* Output uuid is array, not string

* Untested ratelimited request resubmit

* Use traceback diagnostics to report invalid textures and unhandled cases

* Log unused cache entries

* Store NbtTransformer in `ctx.inject` so it can be used on loot_tables

* Apply ast rule to snbt within advancements and loot tables

* [WIP] Usable Diagnostics on LootTables

* Cleanup diagnostic printing.

* Fix diagnostics on json files with multiple lines needing subs

Since we're faking database entries, the lookup key for loottables needs to be unique, which it cant be when one file has two strings needing parsing. Instead we use an enumerated resource location

* Spring cleaning

* Update heart-canisters to new shortcuts

* Fix typos

* substitutions in lib_player_heads registry

Also updates mecha to fix a bug with raising diagnostics within subcommands

* Missing skin texture for compilation errors

uses mecha's `yield Diagnostic` magic syntax to both raise errors and return a valid ast, which gets a magenta/black missing-texture

* Pass proper location for lib_player_heads invalid skin names

* Remove unused name field in skin_cache

* Fix skin value subs for other structural cases

* Error if GH Action needs to upload skin

* REVERTABLE: GH Action Test Case

* REVERTABLE: Enale GH Action on test branch

* Allow processing of list-root item modifiers

* REVERTABLE: Fix gh action skin test typo

* Escape newlines in nbt from loot-tables, so mecha parses them right

* Try raising exception to quit gh action

* Try systemexit exception for gh action

* Try sys.exit for gh action

* Comment

* Revert "REVERTABLE: GH Action Test Case"

This reverts commit 11a74ba.

Revert "REVERTABLE: Fix gh action skin test typo"

This reverts commit 3a25f69.

Revert "REVERTABLE: Enale GH Action on test branch"

This reverts commit fdf3ad0.

* Eliminate many #type:ignore with better type annotations

* Support un-prefixed item modifier functions

* small typo

* Apply substitutions to all modules - first pass

* Delete skin_downloader.py

* Record parent_module on new skins

* Cache record of any skins from other modules

* Increment patch on non-native skin reference diff

* Output wraps the build pipeline

Moves gm4.output to use its exit phase instead, this allows all other plugins to process their exit phases first, doing things like cleaning up extra file containers or saving important data

* Configure mecha to keep comments

* Sub-modules properly record nonnative references

* Clarify metallurgy mould naming

* Initial commit

* Metallurgy skins cleanup

* Disable unused thoriub_brass substitution

* Experimental lib_player_heads versioning warnings

* Revert "Experimental lib_player_heads versioning warnings"

This reverts commit ac60b4d.

This functionality would be better served in some hypothetical linting checker rather than here; additionally, this should ideally belong to the library files itself, not the player head management tool. For now, the maintainers will need to keep in mind that skin file changes should accompany lib_player_heads registry edits

* Baryte lump typo

* Update init.mcfunction

* Cleanup unused OLD textures and correct some typos

* Update mecha version to add 1.20 command tree support

* Legacy loot_table

* Update skin_cache.json

---------

Co-authored-by: Bloo-dev <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
quality-update Improves efficiency or structure without affecting functionality of a module
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants