A klipperscreen story

[g992]

1. Klipperscreen runs on python
2. we have a working framebuffer
3. we have a working python

I don’t know how many free resources we have and I’m not credited to the working touchscreen, perhaps they will require more complex manipulations with the software, but I can disassemble the display, if it does anything, at least in theory, maybe the touchscreen chip will help? since the display is connected simply via a parallel interface

[KaruroChori]

Even just turning it fully off would be a meaningful upgrade over the current situation.

To be honest, I never cared about having a screen on my printers, so I have no idea on how klipper handles them :D.
But knowing how it is driven would surely be helpful.

[xblax]

Yes that also came to my mind already. It's worth a try at a later point. Klipper screen needs X-Window system, libgl, GTK3. That's all available in Buildroot and I believe in theory it could work. Let's see what system resources are left after enabling Webcam and Wifi.

Touch should work, as I think the kernel intefaces for that are standardized. We just need to add libts or libinput in Buildroot to make it usable for userspace applications.

Something useful could be put on the screen, even without Kipper screen, if someone develops it (I probably won't). Maybe it would need a more "embedded" approach with QT5/6 directly rendering to the framebuffer or even something based on embedded touch UI framework like https://lvgl.io/

A very low key approach would be something based on SDL that just renders some stats to the display (no touch). Like IP address, remaining print time etc. But just connecting to Moonraker to get the data takes some effort.

[KaruroChori]

It is! I just cat /dev/random > /dev/fb0 and it shows.
I don't know how to turn it off, but at least I can display stuff.

[xblax]

Yes, I wrote a simple script to convert png into pre-rendered framebuffer images:
https://github.com/xblax/flashforge_adm5_klipper_mod/blob/master/device_files/images/convert.sh

With something like lvgl I think it you also need to directly use libinput or tslib for the touch events. lvgl is not available in Buildroot out of the box, unfortunately.

[KaruroChori]

And the touchscreen is /dev/input/event0

[KaruroChori]

lvgl is not available in Buildroot out of the box, unfortunately.

Statically linking it should not be too hard I hope. It was designed for that purpose, so I don't think there will be too many dependencies.

[KaruroChori]

echo 4 > /sys/class/graphics/fb0/blank turns the screen off, but the back-illumination is still on :/

[jlodew0]

that was my main concern with klipper screen. X-windows.
So that would be pretty expensive on the memory side.
To be honest we cloud even use our smartphone if it's in another room.
There are known memory leaks with klipper screen as well if used and e.g. android phone:
KlipperScreen/KlipperScreen#862

I would focus mainly on the functional side of thing and ease of use for the moment.
And when all those requirements are met, look at optional stuff.
I would prefer to have good main functionality and camer before other luxury.

[xblax]

Yes that's also what I'm thinking. Feature wise on the Linux integration I currently have in mind:

• Camera
• Wifi
• NTP Timesync (that just needs an Init script for Busybox NTP client)

Also I want to move the Mod system initialization directly behind the host system init process. Right now it's S90 in host system init script order, just before Flashforge would start their software. But wee don't need the other host system services long-term and rather have them in the chroot (udev, dbus etc.)

[jlodew0]

wel I got some rtc's lying around. and ntpd is also on the stock 5m firmware.
I honestly bothered a lot about that date in the logs :D

[xblax]

I think the following logic should be good enough:

• read time from file after printer start -> avoid errors with certificate validation
• sync time once after printer start with pool.ntp.org or other service (try until successful)
• update file in time

Basically fake RTC. And it can be a one-shot service that starts then stops automatically. Just needs script for /etc/init.d
Busybox ntp client is included already.

[jlodew0]

problem is timezone info. We need to provide that ourselves. Since we all live somewhere else.
Btw, Belgium here for who want's to know :)

[xblax]

Another think that could be explored is Mobileraker: https://mobileraker.com/

That provides remote control functions and push notifications with a native phone app.

[consp]

As discussed elsewhere:

Running X will be a stretch, but who knows.

The normal firmware runs embedded qt directly on the framebuffer (qt 4.8.2) which is what I would chose for such a machine. I do not know what klipperscreen uses under water (gtk would be my guess), but you definately do not need a full blown X server to do graphics. I know you can do gtk2 on the framebuffer but I don't know if you can do gtk3 directly.

[KaruroChori]

I got the toolchain and the base project all configured to compile stuff and run it on the printer.
The base demo for lvgl is fully working. We can build any interface we want :D.
The ghosting on the image is just because the default demo has some fast scrolling action on display.

[KaruroChori]

Or is LVGL always rending with a fixed framerate?

No, it is my understanding lvgl is not like imgui which is redrawing at a constant fps value. lvgl will try to redraw only when and where needed.
However the demo is a stress-test where the UI is forced to scroll top to bottom in an endless loop, so every frame it must be redrawn. This is clearly a worst case scenario, nowhere close to what is needed for a printer's UI.

I was thinking that you need libts or libinput for touch or is that statically linked as well

lvgl has built-in* drivers to handle most of the basic input devices, so there is no further dependency needed.

As for that project, we do not need to follow the same build process nor the way the program is deployed.
We can just fork it and reuse like 95% of the code as it is in my opinion.
My only "concern" is that they are using an older version of lvgl, so I am even less familiar with it since they fully changed the way drivers are integrated :D.

*8.3 has drivers in a separate repo, from *9.0 they are in the core library

[KaruroChori]

Small update: using a release profile (-O2) and -mfpu=neon, processor usage down to less than <20% for the animated demo, and even less memory used.

I tried crosscompiling guppyscreen, but it is segfaulting hard, it has nss glib dependencies, and its build system refuses to work with floating point. I will give a second go after I clean up the makefiles a bit.

[xblax]

I think the maintainer of Guppyscreen is interested in making that a general purpose project, not just for Creality K1. Maybe he would collaborate with us to make it usable on ADM5. Should be easier as soon as this mod is public and get's some visbility in the 3D printing community.

[consp]

I never saw it before, even after actively searching.

Try incognito windows every now and then. It is a (minor) way of escaping the dreaded bubble you are in due to tracking/cookies/etc. It was one of my first hits but that was probably because I was not actively searching for it.

[consp]

Small update: using a release profile (-O2) and -mfpu=neon, processor usage down to less than <20% for the animated demo, and even less memory used.

Did you also enable vfd? Since neon only does SIMD wich is very applicable to graphics processing but does not enable hardfloat and also does not let you use the MAD instruction (which can only be used if vfd is on afaik) which is also useful.

Try setting -march=armv7-a+simd+vfpv4 for instance. YMMV with respect to compiler used though.

[consp]

I'm going to give X.org a go and see how it will roll (if at all), finally got buildroot running properly so my cores can finally stretch their legs.

@xblax
Is there a reason for disabeling arm neon and only enabeling vfp4?

[xblax]

The current wheel building for klipperscreen looks for packages sdk. Instead it should look in variant-klipperscreen.

That's intended. SDK has the cross toolchain and the host python. The idea behind is that the SDK can also be used to compile additional components or wheels for the mod if somebody would want to develop a custom extension.

The variants are build using the toolchain from the SDK, so we don't have to build the toolchain three times.

I plan to streamline the wheel building a bit more, so that the venv is assembled directly in the build environment. Ideally we would have a buildroot package for each component which also builds the wheel and venv, but that's quite some work.

[ballaswag]

@consp when you get a chance can you share your expertise in getting xserver and klipper screen to work with touch?

I think as of now the variant is configured with evdev and input device at /dev/input/event0. The tslib stuff are not used at all?

On the K1, the display is rotated portrait by default and event0 is showing input when testing with tslib tools inside chroot. But klipper screen is not responding to any touch at all.

My Linux kernel/xserver knowledge is pretty shallow. I see that you built uinput.ko. Is this module needed for xserver to use evdev? If that's the case I would need to figure out how to build this thing for the k1.

If uinput is not needed for evdev with X to work, do you have any other tips on troubleshooting the touch issue with klipper screen. By the way, guppyscreen just works within chroot and it's also using evdev no uinput afaik.

[ballaswag]

For future reference, evdev doesn't need uinput. The uinput module is needed for userspace emulated input devices, e.g. using tslib with X.

Just want to say you guys are awesome again! Sharing here first, a working klipperscreen variant for K1.

trim.B132D34B-0B16-4451-80DF-6A98873D32A0.MOV

[KaruroChori]

Great to see!

[consp]

@consp when you get a chance can you share your expertise in getting xserver and klipper screen to work with touch?

Been off a few days so it looks like you figured it out already.

The X11 tslib driver is s**t and doesn't always work, so to get a calibrated input we used evdev and tslib uinput as the tslib standard event is uncalibrated and we have factory-calibration data to use. This is also something we use for guppy as then you have a reliable way of getting the calibrated input.

Is this module needed for xserver to use evdev?

No, only if you want to use the tslib uinput userspace program.

For future reference, look at the Xorg.conf settings for the touch interface if you want to swap it over vertical/horizontal. You might need to manually create a xorg.conf file. Or use xinput to set it.

What cpu is the K1 running on? It looks a lot smoother than the T113 can do.

[xblax]

I did give the Klipper-Screen preview build a brief test yesterday: https://github.com/xblax/flashforge_adm5_klipper_mod/releases/tag/v00.01-alpha-klipperscreen-test ( thanks for that @consp ).

I think everything that was supposed to work with that build, did indeed work for me out of the box. Memory consumption is quite considerable, obviously. We have to test in combination with WiFi and a 720p camera stream at later point.

G-Code preview also did not work, though I'm not sure if it was supposed to. I did create a ticket for build system improvement, I think that also clarifies a bit how I think the integration can proceed: #32

[consp]

Thank you for testing! I'll have a look at gcode preview, going to print some testcubes I guess, don't remember if it worked or not. I'll also take a look at what's eating all the memory in klipperscreen. Maybe it's something simple.

[KaruroChori]

I think we should stress-test the full combo to make sure there is no problem while generating print moves at full speed run.
The last thing we want is for memory to be depleted mid-print and end up with artifacts on the print. Or the processor not keeping up.
This was surprisingly common with marlin printers, where actions on the UI effectively paused the gcode processing. Klipper is different, but it might still be possible to end up without resources and a similar outcome.

Related to this, we should make sure that klipper runs at higher priority compared to moonraker and klipperscreen, to prevent this sort of freezes in a worst case scenario, and penalizing less important services.

[consp]

Related to this, we should make sure that klipper runs at higher priority compared to moonraker and klipperscreen, to prevent this sort of freezes in a worst case scenario, and penalizing less important services.

That's a good idea. Especially klipper should have a negative niceness. The gui's can even run in the highest niceness if needed since it's not the goal of a 3d printer to have the best gui response (at least for me).

Simply addin -N, --nicelevel to the start-stop-daemon should be enough

[consp]

Merged master and did a cube.

Minor walk of shame: I just completed the first full print, aborted normally since after a few layers, you know how it goes when you see it stick. With everything (master+klipperscreen) combined it seems to run ok, with stock octoslicer settings from FlashForge, which is medium fast I guess. After cooldown no abnormalities or weird measurements.

Unknown G17 command warning on klipperscreen, which kills the ui and you cannot do anything -> needs investigating (issue made). This might be a general klipperscreen issue as it stops the ui from working at all. Anyone experience with this?

CPU usage ~20% max. I'm pretty sure that if you leave it be the CPU will not be the issue. Memory at around 15k free but plenty of buffers to be (ab)used. There is a lot which can be paged out but I do not know what the performance penalty is. Especially the UI can (and should) be paged out to swap. I opened everyting, klipperscreen, fluidd, mainsail and topped out at 20, average about 10-15% cpu. Lowest I got mem wise was 10m free, plenty of buffers and swap available (swap only ~20m).

If we are running into memory issues I guess dropping klipperscreen and going for a guppyscreen solution might be the way forward but for now it should do ok (unless beta testers report issues). I have not looked into the gcode viewer missing yet, then again the screen is tiny ...

Some other tests:
Doing stuff with klipperscreen spikes in top to about 25% during no-print which is fine. Did another print and started messing with the ui which resulted in no issues except for it not really responding. This means the abort also does not work -> reason to not release klipperscreen yet and put it in alpha only. Yes I mashed it franticly.

[consp]

No issues with mdev, so that's great!

Except for evdev no longer working ...

[xblax]

@consp
Can you rebase and split that commit 83fb991, so that i can cherry-pick your general improvements already onto the master branch?

[consp]

Done (I think) in the feature branch. d38a40b, 080eca1 and 7c3ac6e