[feature] Option to precompile scripts for production

[Ikalou]

I think having the option to precompile typescript files could speedup startup time a lot in some configurations. A lot of low-middle range phones seem to take a lot of time to compile the scripts on startup, sometimes prompting "unresponsive scripts" messages.

Maybe this could be part of the standard build process?

[elisee]

Yeah, would be very nice indeed. Basically we need to change the logic here:

superpowers-game/server/index.ts

Line 15 in 6f532d9

async.each(assetIdsToExport, (assetId, cb) => {

so that it gathers all script assets and put them into the TypeScript compiler (a bit like here: https://github.com/superpowers/superpowers-game/blob/6f532d9a11b96c161329519e2cb815c98fd21616/plugins/default/typescript/data/ScriptAsset.ts) and then output the resulting JS instead of writing the TypeScript files.

[Ikalou]

If I'm not mistaken, there seem to be two different ways to build a project:

• "run" and "debug" use SupCore.system.serverBuild() in server/index.ts to trigger a server-side build (and then browse to the resulting folder).
• "build" uses build() in plugins/default/gameSettings/build/buildGame.ts to trigger a client-side-ish build.

I think the "build" code path definitely needs pre-compiling because that's how I assume most users build their game for release. I'm not so sure about the "run" and "debug" code path, since it's more of a preview/debug kind of thing. What do you think? Am I wrong?

On a semi-related note, I think having a command line option to trigger a server build of a project (using node or npm) would be very useful as well (as opposed to having to start a server to build).

[Ikalou]

I'd like to ask a couple more design questions:

• In addition to user scripts, I would compile plugin code as well. This way, the runtime no longer needs to compile anything on startup. It might even be that there is no longer a need to load each plugin's typescriptAPI.ts(?).
• What about minification (e.g. using uglify-js2)? concatenation (SP does a lot of network requests atm)? Source maps? Error checking/reporting?

[MathieuLoutre]

Is there something that can be done as a quick hack in the meantime? Like manually bundling all the assets in JS and feeding it to the Engine?

[Ikalou]

Yeah, don't actually use this. This is a temporary workaround (and a bad one) I made to make load times bearable under a strict deadline. This should really be integrated into the superpowers build process.

Basically I have a dirty patch that makes getAssetData look into a cache (cache.json) of small assets bundled together (including pre-compiled user scripts). The actual bundling / compiling is done by a node script on our build server. Here: https://gist.github.com/Ikalou/b3d0637435bade661975a306a6a9a4e7.

I have another similar hack for pre compiling all the plugin's typescriptAPI.ts, but it's even more specific to our pipeline so I'm not including it.

Hope that helps somehow.

[MathieuLoutre]

Thanks! I've also got a pretty tight deadline here so this is great. I'll see what I can do but if that reduces load times, it's exactly what I'm after at the moment.

[MathieuLoutre]

@Ikalou I've just tried your patch and it all seems to work great (only a small warning saying The provided value 'application/json' is not a valid enum value of type XMLHttpRequestResponseType.) but that doesn't seem to affect the game.

However what I'm really after is removing the runtime for typescript from the package. Alone it's 3.5mo which on mobile is a killer. Do the plugins need compiling in a similar manner? I must say I haven't looked too deep into the innerworkings of the runtime but I'm happy to if you have pointers for me. If I could find a way to get rid of that I think that would be a clear win for my specific use case. Would really appreciate your expertise on this one!

[Ikalou]

Yes, plugins also have some code that gets compiled on startup. It's basically a two step process (plugins and then user scripts). You can see that here: https://github.com/superpowers/superpowers-game/blob/master/plugins/default/typescript/runtime/script.ts#L73.

I am still loading every plugin's runtime.js and components.js. The only file I managed to skip is typescriptAPI.js because it was only needed for compiling (it holds typescript code and definitions). I'm not knowledgeable enough to know what can effectively be omitted from the typescript plugin's massive runtime.js. Sorry :(

[MathieuLoutre]

@Ikalou I've done some digging. I'm at a point where I'm looking at removing the the compileTypeScript instruction on https://github.com/superpowers/superpowers-game/blob/master/plugins/default/typescript/runtime/script.ts#L74

If that can be bypassed, presumably by not loading/requiring the typescriptAPI.js files of the plugins then we won't have to import the typescript module and it will clean 3mo off the whole file. How did you manage to skip the typescriptAPI.js files? I found that I could remove it from the index.ts file in SupRuntime but that triggers a boatload of errors (probably in script.ts of SupRuntime). So if you've managed to skip the inclusion of the typescriptAPI.js files you may have managed to find how to skip compiling the definitions altogether! (I hope I understood correctly)

[Ikalou]

We can talk about it on superpower's gitter or discord if you want. Until a proper solution is found, we shouldn't be making too much noise on this issue.

[bilou84]

I will give that a shot this weekend, I have a bit of free time.

[Ikalou]

@bilou84 thank you for working on this!

Let me know if you need help with testing. I have a couple of moderately large projects I could try to build.

FYI, in the current state of things, code minification will break things horribly for older browsers that don't implement Function.prototype.name because SP relies on it and threejs's polyfill is broken for minified code. I don't think you guys care too much about legacy browsers, though, but I thought I'd let you know.

[bilou84]

@Ikalou ok so you can try with the precompile-scripts branch of the Superpowers game repository. Also make sure to build latest Superpowers core as well. And you can give it a shot.

On my own testing, it works fine on Chrome and Edge. On average it reduces the size of the build by 5MB and I usually get around 50 files less. I didn't measure the loading time but it also feels faster (less files to load and no compilation to do). Let me know how that work for you ;)

[Occult-Dev]

@bilou84 just so you know, in the current version that sits at precompile-scripts, line-37 in core/systems/game/plugins/default/typescript/data/ScriptAsset.ts reads:
compileTypeScript = serverRequire("../../../../typescriptCompiler/compileTypescript").default;
This, however, results in a 'Cannot Find Module' error as the module is actually called compileTypeScript, not compileTypescript. Changing the casing seems to smooth things over :)

[Occult-Dev]

also, I'm not that great with the architecture of this stuff, so sorry if this is a dumb question, but any idea on why this fails when I start installing other plugins?

It was working fine, up until I downloaded the fMouseInput plugin, then Superpowers stopped being able to compile / build my game [ or even run it ].

Is this because I need to add the fMouseInput plugin to a list of things to compile or something? Or is this method just not compatible with 3rd party plugins?
If it's the former, would you mind walking me through how to do it? Like I said, I'm not to skilled in this area, unfortunately.

[bilou84]

Ooops thanks for the typo, will fix it right now. Stupid windows not complaining about that '^^

As for the 3rd party plugins, this feature breaks compatibility so they will need to be updated accordingly. The change needed is very small so hopefully they can be fixed by authors quickly. I will try to get in touch with them before releasing the new version.

[Occult-Dev]

alright cool, would you mind telling me the small change that needs to be made? I'd like to see if I could handle it on my end before the official version launches