You are close.

You can use xsPrepareMachine in place of fxCloneMachine. xsPrepareMachine is a wrapper on fxCloneMachine to simplify the code a bit. (fxCloneMachine instantiates the VM created at build time as a result of the preload process)

Inside XS, mods are called archives. As you found, an archive needs to be memory mapped to be used. The way that is done is platform dependent.

Once the mod is mapped, you should be able to do exactly what happens in fxScreenLaunch (the simulator):

• Map the be symbol table of the archive against the host using fxMapArchive. This is briefly discussed in "Behind the Scenes" in the Mod documentation.
• Instantiate the machine with the mod using xsPrepareMachine (fxScreenLaunch uses fxPrepareMachine but they are just two paths to the same code. All the microcontroller hosts use xsPrepareMachine.)
• The VM is ready at that a point. Use xsAwaitImport to get the first module loads as usual.

Thanks! While I have not given it much test time, it appears to be working now on Windows, and I see the path to getting it to work on Mac and Linux.

I have a question about the event loop processing - after I start the exported default function (from the host) I simply terminate the app when it returns. Does the Moddable SDK somehow maintain the event loop until all code is done processing (including handling instrumentation sampling as appropriate), or do I need to take some action in my main.c code to ensure the event loop and instrumentation has processing cycles?

Congrats!

I have a question about the event loop processing ...

The Moddable SDK performs no magic. ;)

The host is responsible for the event loop, if you want one at all (command line tools, for example, do not). Different hosts have different ideas about what it means for "until all code is done processing". The REPL, for example, just runs in an endless blocking loop.

On macOS, a basic event loop is pretty easy:

int main(int argc, char* argv[])
{
	xsMachine *the;

	// ...initialize the

	CFRunLoopRun();

	xsDeleteMachine(the);

	return 0;
}

One way to terminate is to bind in a JavaScript function to exit the event loop:

function exit() @ "xs_host_exit";

void xs_host_exit(xsMachine *the)
{
	CFRunLoopStop(CFRunLoopGetCurrent());
}

My goal here is to have the x-cli-XXX platform be able to run the same host and mods as the other platforms. My understanding, perhaps incorrectly, is the microcontroller (and simulators) do run an event loop, and see to have some functionality to automatically maintain the instrumentation status updates (and support for the debugger). It may be that all that loop does it manage the debugger/instrumentation? For example, when the book examples run host, the functions complete when waiting for a mod, and then kick back in again when the mod is uploaded with mcrun. But in my code, when the host is done, the app terminates. Also I see that instrumentation gets set up and run once, but then nothing in my main code would cause it to trigger on any frequency to keep the instrumentation data reported to xsbug updated, and I wasn't sure if that was an indication I needed to do something more here.

Perhaps I'm over thinking this, as the debugger does work, so something in the xs-cli-XXX platform has set that up, and perhaps instrumentation comes along for the ride, and nothing in the modules depends on any other event loop? To be clear, I'm not trying to latch onto any external event loop (like Mac or Windows messages) - I'm only trying to make sure that Moddable itself has what it needs to be successful.

Keep in mind that... x-cli-* builds are different on purpose. They give you the ability to provide whatever host behavior you need. That's a feature. A powerful feature with a great deal of flexibility. It isn't intended to be easy. If you just want the simulator host behavior, use the simulator. ;)

You can think of x-cli-* builds as the most general way to build. That generality means that there's less that the SDK can do for you.

The macOS code above gives the basics of what I understand you want -- to launch from the command line and run until the script signals that it is complete. The same can be done for Windows and Linux.

My understanding, perhaps incorrectly, is the microcontroller (and simulators) do run an event loop...

Yes, they do. On ESP8266, it just uses the Arduino loop function. On ESP32, it uses a FreeRTOS task loop. The host for each platform is different.

...maintain the instrumentation status updates...

Yes, they do this by periodically calling a sampling function. This is done from a native timer and so is host dependent.

... support for the debugger...

This one you get from the SDK, because it has built-in support for debugging on macOS, Windows, and Linux.

I'm not trying to latch onto any external event loop (like Mac or Windows messages)...

But you still need some of that for the modules to work. For example, timers on macOS rely on the native event loop being active. Sockets too.

That all makes sense, and confirms my basic understanding (thanks!). I think the gap for me remains how to establish the event loop from the perspective of XS and the SDK. I took a look at the ESP8266 implementation and loop()... it does the following:

void loop(void)
{
#if mxDebug
	fxReceiveLoop();
#endif

	modTimersExecute();

	if (0 == modMessageService()) {
		int delayMS = modTimersNext();
		if (delayMS)
			modDelayMilliseconds(delayMS);
	}
}

This appears to mostly prime the pump for timer and not much else. Is that all the XS modules require to work from the event loop? I see that the setup() function does the initial importing and execution of the exported function (similar to my main), so upon it's return it's simply up to loop to keep a native timer happy?

But you still need some of that for the modules to work. For example, timers on macOS rely on the native event loop being active. Sockets too.

Do you mean that if my host needs the event loop I need to keep it running, or do you mean XS needs it (such as for sockets)? My gut right now says that all XS (and the Moddable SDK modules) needs is the timer to be fed, and everything else with the event loop is platform specific (which in my case, may be very little)?

Never mind to the prior message - I figured it out.

I now have a main() that starts a thread, which in turn starts the VM, manages the Windows message pump, and terminals the VM when a WM_QUIT message appears (such as ^C). This works great, and works with the win timer implementation as expected.

So now the next question is ... now that I own my own idle loop, I need to also own calling fxSampleInstrumentation on some frequency (as well as add the machine.onBreak to update the instrumentation as well). I've got all the plumbing done (always executing everything in XS in the same thread), but how frequently do you recommend capturing instrumentation? I will likely place the config into the manifest (config), but I still am wondering what a good frequency might be (fast enough for value, but without too much overhead).

That's great news.

All the device targets run instrumentation once per second. It isn't very much overhead, so you could do more, but for general use a second seems to work well.

onBreak is optional. It is nice because it ensures that the memory numbers are accurate at breakpoints by forcing a garbage collection.

Once/second sounds great - I have that working now as well. I understand onBreak is optional, but it's easy to do so why not?!?

Is it possible to detect if the debugger is connected? When I call fxSampleInstrumentation it prints the output to the console of my x-cli-win app, unless xsbug is running, then it correctly gets redirected to the debugger. I'd like to stop that console output when the debugger is not connected. Is there a way to do that (either detecting the debugger, or just stopping console level output for instrumentation)?

Is it possible to detect if the debugger is connected?

#ifdef mxDebug
if (fxIsConnected(the))
   ...;
#endif

@phoddie I've got Windows, Mac and Linux all working, and so far all examples I've tried (that don't need the screen) are working as well (assuming they work at all on the native platform).

However, I'm finding an issue calling this x-cli-XXX as the definition Moddable currently uses for that build target is a user-supplied main entry point (as seen in repl), yet my usage of it now makes a main and has support services surrounding it. This problem becomes obvious if you modify manifest_base.json to support the platform (which is helpful if building the examples) as it then breaks the existing Moddable use (and repl specifically).

I am also finding it's hard to constrain it to just contributed. I've had to update a large number of manifest files for the platform type across the code base. That's generally OK (it is limited to the platform), but there are assumptions about the capabilities of the runtime that may be inconsistent with other uses of x-cli-XXX. And of course specifically manifest_base.json would (eventually) want the target as well, further breaking other use of that target.

Therefore, I propose we add a new platform type of cli-XXX (such as cli-win) that is for command-line based platforms (win, lin, mac) that includes main and the other services. That does further expand the example manifest files, as you would want entries for x-cli-XXX and cli-XXX for the various services, but it is isolated from the other build platforms. This will also make it easier to move from contributed to main code, if that decision gets made in the future.

Taking this approach would maintain the x-cli-XXX as it is today (not breaking repl), allow us to add it to manifest_base.json (so examples can be easily built for the command line platform), and allow the manifest files to be adjusted for cli without impacting anything else.

What's your thoughts on this approach? It will take a bit to instrument, but seems cleaner with less risk of breaking existing code.

I'm not sure about using a separate platform target for here. If I understand well, this is basically a x-cli-*build with a default native main to provide a baseline runtime. There are also supporting changes to allow platform-dependent native modules to build for that platform target.

I'd like to make sure I understand how you intend to use this. This is clearly for projects that don't use a display. When you build those for a device, you'll specify a platform that doesn't include a display;

mcconfig -d -m -p esp32
mcconfig m -p esp

When you build for a computer, you want to run as a headless command line tool rather than launching the usual screen-based simulator. To achieve that, you'll do something like:

mcconfig -d -m -p x-cli-mac
mcconfig -d -m -p cli-win

Is that close to correct?

Yes, that's correct. The the reason I am proposing a new build target is that if I use x-cli-mac (for example), the assumption is the user is responsible to provide their own main code - the repl is a great example of this. But when using this new cli platform, the main code is supplied and you need some way to instruct the build system to use the new main code rather than a user-supplied one. I found that when I updated the manifest_base.json file for x-cli-mac to reference the new CLI file(s) I could now build all of the examples, except repl - which also means it's going to break anyone who is dependent on that existing functionality (and running those examples is really quick and easy, since you don't need to bring up xsbug and a separate simulator window to see the results).

By adding a new platform, these problems are avoided. Additionally, the manifests can be tuned for the unique features of the cli-based platform vs. x-cli (since cli supports event loops, promises, workers and dynamic mods).

My thought was to continue with the current model, where mac is for simulator w/screen, x-cli-mac is for hand-grown main control, and x-mac (used by the build tools). This is cli-mac for nearly the same functionality of mac except command line only (no screen).

At least that was my thinking! The pull is up there now if you want to look at it and see if it's useful to the project or not. I'll be using it for sure either way.

I'll definitely take a look at the PR. I just wanted to understand the big picture first since it is easy to get lost in the details.

I tend to think of build targets as a big thing -- certainly anything that involves cracking open the mcconfig sources is on track to qualify as a big thing. Here the actual difference between the x-cli-* and cli-* platform targets is small (which is great) so it feels out of balance.

I think the PR is proof this issue has been addressed, so I'll close it out.