Input System
This page covers at what part of the code the inputs are redirected to enable input playback.
Note
This is for MC 1.12.2 which uses LWJGL 2. This is changed starting with 1.14.4 and LWJGL 3
Consider this diagram. It shows the way an input on your physical keyboard takes, up to the place where it is executed.
For this, we assume that the input is the key W and that we use the default keybindings in Minecraft.
LWJGL notices the keyboard press and writes a "Keyboard Event" into it's internal buffer. One keyboard event consists of 3 values:
- The keycode of the pressed key (17)
- The state of the key:
Truefor key pressed,Falsefor key released (True) - The character associated with the key (w)
For player movement, we only care about the first 2 values, however the third will come up when talking about player chatting or similar features.
This event will stay in the internal LWJGL buffer until Minecraft polls all events, which happens every tick.
This buffer has a limit on how many inputs it can hold and how many you can retrieve at once. Additionally, there are 2 separate buffers for Mouse and Keyboard, however both are polled at the start of each tick.
LWJGL only stores the keyboard events that happened in between ticks. The job of figuring out which keys are currently pressed is done using the keybindings.
It is basically a copy of the physical keyboard and has functionality of assigning actions to certain keys.
In our example, the "moveForward" keybind notices the keypress and sets itself to true.
The player controller checks the keybinds every tick, whether the "action" should be executed. As the "moveForward" keybind was pressed, it executes the logic for moving forward.
Now we have a choice at which point we want to implement our input system. I will go over each and list advantages and disadvantages
The first obvious target would be to virtually press the physical keyboard. This can be achieved for example with a Java Robot.
That has the serious disadvantage that this allows the TASer to target other programs beside Minecraft. This poses lots of security risks when running untrusted Minecraft TASes and is overall a bad idea.
The second target is therefore much safer and requires manipulating the keybindings. Taking control over the keybindings allows us to move the character without pretending to be a keyboard. This was done in RubiksImplosions Minecraft TAS-Mod which allowed you to set these keybindings via a script.
This however has the disadvantage of not containing all of the functionality of Minecraft. Gui screens (like the menu or chat) in particular have hardcoded keys that do not depend on the keybinding system and thus can't be manipulated. Furthermore, modded keybinds can't be addressed properly, which is a minor disadvantage but still worth noting.
This takes us to the target that we have chosen for our Input System. With this we have full control over Mouse and Keyboard, can support Gui screens and modded keybinds. Plus this method only pretends to be a keyboard inside of Minecraft so it will not have access to other programs.
The only downside compared to Keybindings->PlayerController is that the keybindings have to be the same inbetween recording and playing back, which is an issue when playing a TAS that someone else with different keybindings made. This change of keybinds happens all the time with different keyboard layouts, so extra work has to be put in to correct this usability issue, whereas Keybindings->PlayerController has this feature out of the box.
To summarize, this is the figure for TASmod
As you can see, the inputs from LWJGL are redirected to TASmods VirtualInput which internally splits itself up into VirtualKeyboard and VirtualMouse before going back to the Keybindings.
The "virtual" package contains all classes with the logic on how to pass keyevents to the keybindings.
All of the actual Mixin classes which hook into the game code can be found in the package mixin/playbackhooks
Note
I just noticed that a part is missing and can be found in MixinMinecraft... I should really split this to avoid confusion.