C compiler

[stevexyz]

Maybe I'm wrong (I just tried to install elks on an original PC IBM XT), but seems to me that no C compiler is included inside elks (I've found the basic language interpreter though), while I think that's one of the basic thing of every Linux system.

If that's the case, I understand that a mammoth GCC might not really fit the project, but maybe the small and powerful tcc might be an option!

[ghaerr]

Hello @stevexyz,

Thanks for your interest in ELKS. A few years back, the project switched from using bcc to ia16-elf-gcc to build the kernel and all the applications. One of the big reasons was the lack of support for 8086 segmented architecture linker and compiler options that are not present in bcc (or tcc). Unfortunately, gcc is way too large to be able to be included in ELKS runtime, so there isn't a way for ELKS to be self-compiled, but this has been viewed as a reasonable tradeoff, all things considered.

I'm a fan of tcc, but the question is, given the limitations of segmented-mode 8086, what purpose would it serve, doing the work to get a self-hosted compiler running, when none available contain the features required to build the current kernel and all of the applications?

Thank you!

[stevexyz]

To me a C compiler is the base of a complete Linux (Unix) system...
Don't know what are the current priority of the project but seems to me this one should be relevant.
And if tcc is not ready or worst suitable maybe there are other options: back in the day there were many compilers, and all run on the basic x86 segmented architecture.
Just willing to add that was amazing to see Linux booting in the XT, this addition is for sure keeping the bar very high! :) For now compliments to all developers!!

[Mellvik]

Hi @stevexyz -
let me add my 2 cents to this discussion.

To me a C compiler is the base of a complete Linux (Unix) system...

Here's the the thing: ELKS is not a complete Linux or Unix system. Like the name says, it's intended for embedded systems. Embedded systems have limited resources and are rarely if ever selfcompiling. Of course our vintage PCs aren't embedded systems, but they have very limited resources. And - as @ghaerr also alluded to - it just doesn't make sense to have that ambition. Possibly fun, but not useful.

Think about it - what we have today is a cross development environment with gcc and tools that today's developers are familiar with and expect. And a decent cross host you can grep the entire source tree for something in a couple of seconds. Try to do something similar on your XT - you'd probably have to come back the next day.

I have Venix running on one of my machines, a 286/12. It's a complete Unix system. It can compile itself if I had full sources, and I've done a lot of development on it. In the mid 80s and recently. It has make, cc, sccs and the most basic (early) PWB tools (basically a V7 system - anno ca. 1980). And there are no cross tools, so everything is local. I can assure you, it's not a development environment you'd want to live in. It's extremely slow - and I keep asking myself how I could possibly stay sane while working in this environment in the 80s. The truth is that expectations were different then. Having your own 'complete' Unix system just qualified the endless waits - and coffee breaks :-) .

And if tcc is not ready or worst suitable maybe there are other options: back in the day there were many compilers, and all run on the basic x86 segmented architecture.

Yes, the compilers you're referring to RUN on the segmented architecture, but they support only parts of it – the small, maybe medium memory model, that's all. They have very limited options and support-tools (like cpp supporting only a few of your favorite #preprocessor commands, no objdump etc etc). Try porting any modern piece of open source code to such a system and you'll start pulling your hair out right away. I spent a lot of time trying to get rz/sz running on Venix a while back - it ended up being a BIG project because of the limitations in the compiler - not to mention in make. And because many of the source files were too big for vi.

Just willing to add that was amazing to see Linux booting in the XT, this addition is for sure keeping the bar very high! :) For now compliments to all developers!!

ELKS has come a long way, the last few years in particular. Your contributions would be very welcome. Even a native C-compiler. It's your time and your choice - and you'll get plenty support from the group regardless of whether the target tool/application is for the few or the many.

--M

[tyama501]

Not for self-compiling but to make small debug program, it is nice to have a small compiler on ELKS. I now uses the basic to peek memory or read ports on the real PC from the background but sometimes wants to do a little more complicated.

[stevexyz]

As already mentioned by @tyama501 it was not to be used to self compiling, even if it would have been a nice thing. And especially for starting, if there is something that is ready to be used, doesn't really matter if it is supporting just a limited memory model, but at least you can compile and run some programs on the system without always access another computer.

For now I've other (unfortunately too many) projects going on and I'll stay on the window looking the ways ELKS will grow up, but in the future if it will still be not developed maybe I'll give it a try!

In the meantime keep up the good work and happy hacking!

[ghaerr]

I thought more deeply about exactly what is entailed when someone says "I'd like a C compiler" to run native on ELKS. As @tyama501 and @stevexyz mentioned, it would be nice to be able to at least just compile some programs from within ELKS.

In order to do that, we'd need the following:

• Compilation of the ELKS C library into .o (object) and .a (archive) formats. The host-based .o and .a formats are incompatible, as they're oriented around the host ia16-elf-gcc toolchain. That means we'd need to compile all the C library on ELKS itself (not cross-compiled, unless the target ELKS compiler also runs on Linux and macOS), and that would have to be done on a VM running ELKS, in order to be kept current.
• We'll need make on the target, as well as any commands used in the C library Makefiles.
• We'll need easy ways to make (large) target disks that contain all the library sources, for compilation. We've pretty much got that, but need scripts to make it happen.
• The C library sources would need to be ported to the new C compiler, as will all .s/.S (assembly language) source files. Although most of the library is portable, any uses of far pointers, asm() directives, or incompatible gcc preprocessor directives need changing. Of course, any of these files could be left out, but that might inhibit desired C library routines.
• All the header files would need to be copied to the target (development) filesystem. I have not checked whether they'd all fit on a floppy, or whether the ELKS C compiler would require an HD. Most of our users now use ELKS on floppy, and we don't yet have an easy way to update hard drive images, but we're working on it.
• The C compiler chosen will have to be modified to output the special ELKS a.out file format, (which is almost but not quite MINIX v1). This will likely take some work, after porting the C compiler, preprocessor, assembler, linker and front-end to ELKS. None of the tools can use any more memory than 64K to compile any programs. The ia16-elf-gcc compiler actually produces Linux ELF-format output, and uses a seperate elf2elks executable to convert the ELF format to a.out format. We could use the same method, but that program has to be written from scratch, not just ported.

After all this, there are all the issues that @Mellvik brings up, which include problems associated with having no objdump, a different ASM language, possibly problems with vi or other editors not being able to edit a file in limited memory, and slow compilation times.

All in all - I have agree with @Mellvik that such a project is not really what people think of "having a C compiler" for ELKS.

On another note, I was thinking about some C interpreters that might be able to provide fast execution of simple C programs, such as the C in 4 functions compiler. It is very cool with a small code size, and allows for calling out of various functions like printf into the host-based C library. However, it is definitely not clear whether such a thing would run on ELKS. For instance, C4 uses 5 allocations of 256K each (hugely over our 64K data limit) by default. I might take a pass at allocating much smaller sizes for the source and produced symbol tables, object files and data, but I imagine this will likely greatly limit what C files can be compiled, and we still won't have the normal ELKS C library (nor multiple file compilation nor any linking).

Thank you!

[tkchia]

Hello @stevexyz, @ghaerr, @tyama501,

I suspect that the Amsterdam Compiler Kit might be a good candidate for an ELKS-hosted C compiler, though I have not really got around to working on such a thing, and it probably needs a fair amount of effort. I believe ACK used to be the standard toolchain for Minix — including Minix/8086 — and besides, it is written to be able to run on small systems.

Thank you!

[stevexyz]

Seems to me that https://github.com/alexfru/SmallerC would be a very good start: seems easy enough and producing already 16 bit x86 code in various models, and with self compilation the ported compiler if it will produce the binary elk file. Maybe the author itself would adapt it if requested and specification of the binary file are given: if it is considered good we can try to ask.

PS: @ghaerr I had a look at c in 4 functions, and while being an amazing exercise of minimization, seems really not easy to port to minimal memory systems for the way it has been designed

[Randrianasulu]

just as comment, I tried to play with old "ACK for Minix" from https://web.archive.org/web/20070910201015/http://www.laurasia.com.au/ack/index.html#download on Minix i86 (not i386) qemu VM.

Well, it ran out of memory :) trying to compile itself under existing 'cc' compiler there.

[Randrianasulu]

Also, Portable C Compiler website seems to be down (and web archive does not have latest copy) so here I found slightly updated (2021) copy of code

https://github.com/matijaskala/pcc
but originally it was at
https://github.com/IanHarvey/pcc

There seems to be some code related by i86 generation by Alan Cox.

Also, someone (Eric J. Korpela) looked at lcc-8086 but not get very far
https://setiathome.berkeley.edu/~korpela/

[ghaerr]

Hello @Randrianasulu and @tkchia,

Thank you @Randrianasulu for the links to PCC, I'll take a look at it. Same for LCC-8086, that work looks extremely old but could be worthwhile. Of course, it would probably be a good idea to consider only using ANSI-capable (vs K&R) compilers, given where most C code is at today.

I tried to play with old "ACK for Minix" from https://web.archive.org/web/20070910201015/http://www.laurasia.com.au/ack/index.html#download on Minix i86 (not i386) qemu VM.

Well, it ran out of memory :) trying to compile itself under existing 'cc' compiler there.

It's probably not needed that the compiler be able to compile itself under ELKS (or MINIX), so that's OK. I am not familiar to what degree ACK has been updated to any ANSI standards, and/or long/long log/float support etc.

In the case of running on ELKS, we now have the issue that some portions of the C library may be using some ia16-elf-gcc-specific features. This would have to be looked into.

@tkchia, you had mentioned you're possibly somewhat familiar with ACK, would that be a version similar to that used for MINIX as described above, or has there been more work done updating it, to your knowledge?

Thank you!

[Randrianasulu]

@ghaerr I found little something supposed to help with backtranslating ANSI C to older dialect:
https://github.com/udo-munk/unproto

Also, may be Xenix (286) a.out variant can be used to get some idea how multiple segments were supported.

https://ibcs-us.sourceforge.io/
look for sources - x.out

[tkchia]

Hello @ghaerr,

@tkchia, you had mentioned you're possibly somewhat familiar with ACK, would that be a version similar to that used for MINIX as described above, or has there been more work done updating it, to your knowledge?

I have not yet done a comparison of the "laurasia" copy of ACK, and David Given's current ACK tree — I hope to do that soon.

At the moment I am more familiar with Mr. Given's source tree (since I have been working on it a bit). Some impressions:

• It is an extremely bog-standard C89 compiler.
  • By that I mean, it has close to no syntactical extensions whatsoever. Inline assembly is simply not a thing. And, there is a #pragma directive, but there are no pragmas (yet).
  • Some machine targets support long long, but not all. In particular the 8086 back-end does not have long long support yet.
  • There are also vestiges of a K&R compiler, but it is most likely dead code.
• The runtime libraries do implement some POSIX functionality in addition to the C89 stuff though. Plus of course we can add our own extension libraries.
• In terms of 8086 support:
  • The compiler front-end can currently produce 8086 PC booter programs (-mpc86) and MS-DOS .com files (-mmsdos86). Unfortunately for some reason it no longer supports MINIX a.out output — I guess MINIX support fell by the wayside while the code was updated (?).
  • There is an out-of-line assembler which mostly works. It can use some improvement (The 386 assembler is terrible davidgiven/ack#271).
  • Currently 8086 floating-point support assumes that there is an 8087. (I believe that for some other target platforms, e.g. CP/M for 8080, it can actually do software floating point.)

Thank you!

[Randrianasulu]

slightly newer ackpack for minix (1.1.2)
https://web.archive.org/web/20060405092432/http://packages.minix3.org/software/ackpack.tar.bz2

weirdly it comes as tar.tar. I only get file by downloading it via browser, not via wget. Same source should still be in Minix3 git, but a bit obscured because it was deleted years ago ...

info from
https://sourceforge.net/p/tack/mailman/message/36608143/

[Randrianasulu]

so, there was another compiler (c86 ?) but license prohibit commercial use.

https://github.com/plusk01/8086-toolchain/tree/master/compiler

[Randrianasulu]

https://web.archive.org/web/20150908032106/http://homepage.ntlworld.com/itimpi/compsrc.htm - so it was named c68, too ...

[Randrianasulu]

ah, it was not complete compiler, just c to asm (nasm in this version) compiler. It needed cc (main driver), ld86 (linker), c preprocessor (it seems for Psion 3 they tried Decus cpp, available in X11R3 distribution - not tried to build it yet). So, some sources are newer in C68 (for QDOS - mk68k/ Sinchlair QL system) but part of older EPOC sources still live at older site:

http://web.archive.org/web/20010414060410/http://www.itimpi.freeserve.co.uk/cpocdown.htm#SOURCE
Only ld86 and cc survived ... and even then, this ld86 outputs by default Psion's IMG format. Still, something!

[stevexyz]

https://web.archive.org/web/20150908032106/http://homepage.ntlworld.com/itimpi/compsrc.htm - so it was named c68, too ...

Seems has already a lot of options, among them the ones for 8086 specific:

...
         INTEL 8086 OPTIONS

              The options  listed in  this section apply when generating
              16-bit code  for use  on Intel processors.  They will only
              be available  if support for the Intel 8086 processor type
              was specified at the time the compiler was built.

              -fpu=yes|no
                        Specify whether  operations  involving  floating
                        point variables should generate in-line calls to
                        the a  hardware floating  point unit, or whether
                        calls  are   made  instead  to  library  support
                        routines. Using  library support routines allows
                        floating point  operations  to  be  carried  out
                        purely in software.

                        Default:  -fpu=yes

              -pointer=16|32
                        Specifies that  the code  should be generated to
                        conform to  the small  memory model  (64K data +
                        64K code segments) which uses 16 bit pointers or
                        the large model which uses 32 bit pointers.

                        Default:  -pointer=16


              If support  for multiple  processors and/or assemblers was
              configured when  the compiler  was  built,  then  you  can
              specify the  target to  be a 8086 processor and a specific
              assembler can be specified using the following options:

              -bas86    Generate 8086  code.  Use  the syntax  for Bruce
                        Evan's 16 bit 8086 assembler for the output.

              -gas86    Generate  8086  code.   Use  the  GNU  assembler
                        syntax for the output.

              -masm86   Generate 8086  code.   Use  the  Microsoft  MASM
                        assembler syntax for the output

              -sysv86   Generate 8086 code.  Use the Unix SVR4 assembler
                        syntax for the output.

...

[Randrianasulu]

so, I have something horribly broken, but it makes .o files!

https://github.com/Randrianasulu/c86

make on linux/termux should give you some binaries.

[tkchia]

@Randrianasulu : it is almost certainly still not GPL-compatible though.

[Randrianasulu]

https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=6f433c5ba2f3dacbf1b3f2859197505dc440b4d9

seems to be very detailed document about c68 by author (I tried to send email to him, but no idea if old email address still works)

[Randrianasulu]

https://qlforum.co.uk/viewtopic.php?t=2112 - may be he has new email, forwarded to it too

[stevexyz]

@Randrianasulu : it is almost certainly still not GPL-compatible though.

just at the beginning of the C68 QL manual it says that it is Public Domain (even with capitals):

INTRODUCTION
The C68 Compilation System provides a Public Domain C compiler for use
under the QDOS operating system.

[tkchia]

Hello @stevexyz,

I mentioned this because @Randrianasulu stated that the source files themselves seem to prohibit commercial use. And I see that system.c states both "All commercial rights reserved" and "The C68 compiler forms part of a complete Public Domain development system for all these [QDOS] environments" (in the same file). So the licensing status of the C68 source code seems to me to be murky at best.

Thank you!

[Randrianasulu]

little aside (feel free to hide) but MAME got Psion 3 emulation inmore working state lately

https://forums.bannister.org/ubbthreads.php?ubb=showflat&Number=121869&page=3
(not very big fan of 2gb gcc process during compilation but I probably can let it happen onse in year or so)

[Randrianasulu]

also faucc (286 & 386 codegen only?)

https://gitlab.cs.fau.de/faumachine/faucc/-/commits/master
(not really changed since 2012 ..?)

EDIT: sadly it does not compile faumachine's new bios :( also, no FP. so, not very useful?

[bocke]

ah, it was not complete compiler, just c to asm (nasm in this version) compiler.

Just a sidenote, but I think NASM does support ELKS a.out format. LCC port might use NASM too.

Anyways, last time I looked into ELKS binary format it had some limits on DATA and CODE segment sizes similar to Minix 1. Is that still the same? What are the limits?

Is it now possible to make something like large memory model executables in MS-DOS with GCC-iA16?

[ghaerr]

@bocke:

I think NASM does support ELKS a.out format. LCC port might use NASM too.

I'm not sure about whether NASM supports a 16-bit MINIX a.out format or not. Does NASM support ELF output? If so, the binary could likely be converted to ELKS a.out format using our own elf2elks tool, which is also used to post-process ia16-elf-gcc ELF output for ELKS.

The ELKS toolchain and kernel currently offer the ability to create and run small (64K code, 64K data) and medium (128K code, 64k data) model programs. Access to a larger data segment is possible through C __far pointers and using the fmemalloc system call to allocate main system memory, but that's all got to be done manually by the programmer. Thus, large model (> 64K data) isn't explicitly supported, but can be done with some effort.

[bocke]

I took a peek at NASM. It doesn't support Minix/ELKS directly. It supports as86 obj files though. They could be then linked with ld86 into elks binary, I think. My memory is a bit murky on this.

Thanx for the info on memory model support ghaerr. Much appreciated.

[ghaerr]

The c86 compiler generates assembly.

Oh, you've got c86 running? That's great! How big is the executable? Can you run size and list the code and data sizes separately? This will give a first estimate of what we're looking at to port. I wonder if I can get it running on macOS.

Here is the "c86 -h" output:

It looks quite complete (and large). The ASM output looks super. Having a C compiler that outputs ASM could help by expanding the choices for assembler and linker.

[rafael2k]

c86 executable for Linux ELF 64-bit (running on my debian) is 215744 bytes. Considering we can wipe out all the 386 code generation, it seems there is hope.

edit: stripped size is 188376 bytes

  text        data      bss      dec         hex     filename
 164409   13312   20712  198433   30721 c86

[ghaerr]

@rafael2k, I just cloned your 8086-toolchain repo. I'm on macOS. Went to compiler/ and typed "make", and bin/c86 was built without modifications, only a few warnings, and runs! Very nice.

I then noticed that assembler/ is missing, I assume renamed nasm/. Typed "make" and I see its now using OWCC for the cross compile. It might be nice to keep an assembler/ directory so that a host version of it can be built. I will have to go back to @hintron's version to play around with the complete system for the time being.

On my system c86 is 180k text and exactly 32k data, interesting if not a coincidence. I'll pull down the other repo and see if I can get the entire system running on macOS to play with and learn more about it.

Since it seems the upstream 8086-toolchain distribution was put together for a CS class, it might be interesting to check out those class files and see what the C programs look like, as well as which linker is being used.

[rafael2k]

It just compiled on ELKS.
Just need to play with memory now!

[rafael2k]

Since it seems the upstream 8086-toolchain distribution was put together for a CS class, it might be interesting to check out those class files and see what the C programs look like, as well as which linker is being used.

It seems the executable is made to be loaded in an emulator for the class.

[rafael2k]

So, habemus C compiler running on ELKS!
: )

[ghaerr]

It seems the executable is made to be loaded in an emulator for the class.

You mean that there is no link phase, the NASM output relocatable modules are loaded by the emulator? I'm wondering what format NASM is outputting for that use. It is possible that the emulator loader code could be used as a linker.

So, habemus C compiler running on ELKS!

That's great! It seems the compiler is very straightforward C. I was able to get it running on macOS by deleting the -m32 flags, but I'm having problem getting it to accept much C input without complaining heavily. I also don't know where its looking for header files during compilation.

I pulled down the upstream repo and got NASM in assembler/ to compile and link by discarding -m32 as well. The emulator/ is in C++ and won't compiler without errors using -m32, and macOS doesn't support 32-bit executables anymore.

[rafael2k]

The the class it is used bin output: https://web.archive.org/web/20210508035736/http://ece425web.groups.et.byu.net/stable/labs/BuildProcess.html

And the is a simple clib for the class (prolly the clib was homework), for DOS I think:
https://github.com/rafael2k/8086-toolchain/tree/dev/website/stable/labs/library

[ghaerr]

Thanks @rafael2k. This is very exciting, having a C compiler running on ELKS. It will be interesting to see what it can do!!!

I notice that you're getting signal handler kernel error messages from C86 and NASM. I'll look into getting that fixed.

[rafael2k]

There is an issue - there is no C preprocessor. In the class course it just mentions "use cpp before using c86".

[ghaerr]

there is no C preprocessor. In the class course it just mentions "use cpp before using c86".

Well, that's kinda huge. It likely means we'll need a CPP/C/ASM compiler wrapper, just like UNIX v6, etc. The good news is that I have one that should work as part of my NCC project.

There are a number of C preprocessors out there, many of which I would think we could get working on ELKS, except now that's another big item on the list. The important part will be selecting one that is conformant.

In the meantime, I think it's still very interesting to continue seeing what the 8086-toolchain C compiler will and won't do with regards to running out of memory, etc. It would be good to allow your tree to compile both ELKS and host versions, perhaps by using seperate Makefiles and usage of #if ELKS in .c files.

My NCC project also has a C preprocessor but it's not standalone. However, I suppose it might be possible to remove the x86_64 code generation and have it run in cc -E form only. It's not yet worth chasing down a CPP until it is determined how well the C compiler runs on ELKS with pre-preprocessed files though, I would think.

[Randrianasulu]

from memory c68 used "Decus cpp" still living in X11R3 archive ... and that compiler was running on x86-ish Psion handheld with segmented model so 64kb segments were very much target constrain ...

But I guess thread grow so large it all scrolled away. nice to see progress anyway..

[rafael2k]

I'm using dev86 cpp. It seems be working ok. I added to the 8086-toolchain repo:
https://github.com/rafael2k/8086-toolchain/tree/dev/cpp
And c86 accepts the input nicely, as it seems.

[ghaerr]

Nice, good news about using dev86 cpp!

I'm running a host version of c86 for now, and giving it preprocessed input using c86 file.c > file.asm is only producing DB/DW output (data only), no code. Any thoughts on that?

[ghaerr]

More information on CC386 v4.2.0:

https://ladsoft.tripod.com

https://ladsoft.tripod.com/cc386_compiler.html

Looks like it was maintained for quite a while into 2017 and then replaced by Orange C. There are apparently Win32 and DOS 16-bit versions.

[rafael2k]

Nice, good news about using dev86 cpp!

I'm running a host version of c86 for now, and giving it preprocessed input using c86 file.c > file.asm is only producing DB/DW output (data only), no code. Any thoughts on that?

No idea. But now I'm running it with "-m16".
: )
Btw, thanks for the suggestions on the repo and the pointer to neatcc, very cool! I'll re-organize that repo as soon as I can complete the workflow, from source to ELKS executable (cpp, c86, nasm, poor's man elks aout writer), akin the Unix v6 stack!

[bocke]

Yes. This I already can do. I want a linker running on ELKS.

What about dev86 linker? Is that of any use? I think I successfully cross-compiled it (or at least one of its earlier versions) using dev86/bcc a decade or so ago and it ran on ELKS without any problems. I think assembler cross-compiled easily, too.

This, of course, doesn't work with ELF. Only with a classic Minix format (I mean older than V3).

[rafael2k]

Hi Bocke. Indeed, I thought about ld86, but the problem is that I don't think it can link any of the nasm output formats. If ld86 is useful, then I think it would be relatively easy to port.

[rafael2k]

Just double checked - nasm has indeed as86 output.

[rafael2k]

So, ld86 indeed links nasm as86 output! I think it is a winner for porting over.

[ghaerr]

I have opened a new issue #2112 for all comments regarding getting @rafael2k's 8086-toolchain (C86, NASM, CPP, LD) running on ELKS. Please post there for details on that specific port, as this issue is getting quite long and should pertain to discussions about general C compilers and their feasibility for running on ELKS.

[rafael2k]

Fuzix C compiler does not generate code for 8086 (but it does for 8080 and 8085), and seems also an interesting option if it gets a 8086 backend.
https://github.com/EtchedPixels/FUZIX/tree/master/Applications/CC

[EtchedPixels]

It's got most of an 8086 backend written but not tested out. Need to finish the assembler first

[rafael2k]

That is really good news @EtchedPixels ! Yay!

[rafael2k]

For the reference, another interesting compiler which can generate 8086:

https://github.com/alexfru/SmallerC

[rafael2k]

Native ELKS C tool chain starting to work!
#2112 (comment)