Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Web assembly backend #69

Open
wants to merge 18 commits into
base: main
Choose a base branch
from
Open

Web assembly backend #69

wants to merge 18 commits into from

Conversation

dandorat
Copy link
Contributor

New WebAssembly Back End

In this branch, following the methods for the development of our compiler, we incrementally developed the WebAssembly back end and the compiler from a subset of Racket to WebAssembly with adding of new files to the repository.

The Wasm compiler works correctly as demonstrated by running our tests for the subset of expressions covered by this compiler, which includes integers, booleans, chars, if expressions, n-ary let expressions, expressions for making heap-allocated data structures, including boxes, pairs, lists, and strings, expressions for doing input and output, including read-byte, peek-byte, and write-byte, unary operations, binary operations, function definitions, and function calls.

We also implemented two runtimes for the WebAssembly back end: The first runtime is written in Node.js and uses the JavaScript API for WebAssembly for interacting with the compiled WebAssembly program via imports to the WebAssembly program and exports from the WebAssembly program. The second runtime is implemented using Wasmtime and WebAssembly code with the use of either the Wasmtime fd_write function or the function print_codepoint that we added to Wasmtime for printing the chars, the strings, and the boolean results, and the functions print_int and error_exit that we added to Wasmtime for printing the integers and printing of "err" followed by exiting the program, respectively.

The Files Added and Changed

The following new files have been added and incrementally committed, and are included in the pull request for this project:

1 – For the version of the Wasm compiler that interacts with a Node.js runtime: compile-wasm.rkt, compile-wat-file.rkt, wtypes.rkt, jsmain.js, makewat, watjs2run, watjs2run.c, watjs2run_README.md, wasm/interp.rkt, wasm/printer.rkt, wtest/compile.rkt, wtest/interp.rkt, wtest/test-runner.rkt, wtest/test-progs.rkt, and wtest/test-programs/get-progs.rkt.

2 – For the version of the Wasm compiler with Wasmtime and WebAssembly runtime: compile-wasm_2.rkt, compile-wat-file_2.rkt, wasm/interp_2.rkt, str1000000lms.rkt, wtest/compile_2.rkt, wtest/test-runner_2.rkt, the-838e-compiler/.wasmtime/bin/wasmtime, the-838e-compiler/.wasmtime/Additions_README.md, and the-838e-compiler/.wasmtime/bin/demo.wat.

3 – To measure the speed of different combinations of our compilers, runtimes, and the methods of printing strings, in the case of running the compiled code obtained from each method for the expression (make-string 1000000 #\λ) in the file str1000000lms.rkt, this file was added. To automate measurement of the times, the following files were added: mkexelms, timestrlms, timestrlms.rkt, timestrlms2screen, and timestrlms2screen.rkt.

4 - For the purpose of a reference for the times, the following files were added for an earlier simpler version of our compiler to x86 without higher-order functions and library functions, and with a make-string function that puts each char in a word (nonpacked representation of a string), and with a simpler C runtime that prints a string accordingly: 3_compile.rkt, 3_compile-file.rkt, 3_parse.rkt, 3_ast.rkt, 3_types.rkt, 3_main.c, 3_char.c, 3_types.h, 3_runtime.h, 3_io.c, and the make file 3_make.

5 - As part of the testing, the folders and the test files in test/test-programs for the languages up to the fraud language were copied unchanged and added to a new wtest/test-programs folder.

In addition, changes were made to .github/workflows/push.yml to install the needed software to run the tests on GitHub.

The Version of the Compiler that Interacts with the Node.js Runtime

The (compile e) function generates the compilation of a Racket program containing the expressions covered by our compiler as data in the form of an s-expression. It has a main program function called $sendResult which is exported and by its signature should return an i32 result (a 32-bit integer), which should be one i32 value remaining on the stack at the end of the function execution after all operations are carried out by the WebAssembly stack machine. The Node.js runtime will receive this result. The functions $writeByte, $readByte, $peekByte, and $error are imported from the Node.js runtime through an import object. These functions are implemented in the Node.js runtime file jsmain.js. This file also contains the implementation of the other Node.js functions needed for printing the different types of the result values.

Converting the S-expression Generated by the Compiler to the WebAssembly Text Format Code, Writing the Code in a .wat File, Making a .wasm File from the .wat File, and Running the Code with a Node.js Program

The function (wasm-string a) in the file wasm/printer.rkt takes the s-expression generated by the compiler and generates the corresponding WebAssembly text format code in readable print form. In this file, it uses the function (s-exp2string a), which uses (define op1 (open-output-string)), (write a op1), and (get-output-string op1), for doing part of this work. The file compile-wat-file.rkt takes a %.rkt file name as an argument and compiles the Racket code in the file to WebAssembly text format code by using the (compile e) function in compile-wasm.rkt and the (wasm-string a) function. For compiling a Racket program covered by our compiler, the Linux command: racket -t compile-wat-file.rkt -m %.rkt > %.wat compiles the Racket code in %.rkt file to WebAssembly text format code and writes the code to a %.wat file. Then, we make the %.wasm file from the %.wat file using the wat2wasm tool.

We have made the executable watjs2run from watjs2run.c by gcc. This executable takes a %.wrun file name as an argument and creates a C file which contains the command system("node [working directory of watjs2run]/jsmain.js %.wasm");, and then makes %.wrun file from this file with gcc. Thus, %.wrun is an executable that runs the above Linux command, which runs our Node.js runtime file jsmain.js with the %.wasm file name as an argument. The jsmain.js program then does the interactions mentioned above with the Wasm code to provide the imports to the Wasm program and to get the result exported from the Wasm program and print the result (for details please see the watjs2run_README.md file).

The make file makewat automates the execution of the above commands. Given a %.rkt file, the command make -f makewat %.wrun will compile the Racket code to WebAssembly text format code writing it to a %.wat file, makes the %.wasm file from %.wat file using wat2wasm, and then makes the %.wrun file using watjs2run. Then, using the command ./%.wrun from the Linux command line, we can run the WebAssembly code with the Node.js runtime.

Running the tests

The file wtest/test-runner.rkt in the new folder wtest contains all the tests in the file test/test-runner.rkt for the expressions covered by the Wasm compiler in the (test-runner run) and (test-runner-io run) functions plus some additional tests. The file wtest/compile.rkt runs the test-runner function with the argument run being (λ (e) (wasm-interp (compile (parse e))))) and the test-runner-io function with the argument run being (λ (e s) (match (wasm-interp/io (compile (parse e)) s) ['err 'err] [(cons r o) (cons r o)]))).

The (wasm-interp a) and (wasm-interp/io a input) functions are implemented in the file wasm/interp.rkt following the methods in a previous compiler of our course with the addition of the functions (result-str->value str), (string->result str), (cons->result s) and reading of two lines, written to stdout by jsmain.js (one for the result value of the Wasm code and the other for the output that the Wasm code writes to stdout), in the case when wasm-interp/io function is used with an input string as the second argument. This case is communicated to jsmain.js via a second command line argument “ io”.

The file wtest/interp.rkt is copied unchanged from test/interp.rkt and runs the tests in wtest/test-runner.rkt using the interp function in interp.rkt and the interp-io function in interp-io.rkt files.

The file wtest/test-progs.rkt is similar to test/test-progs.rkt and runs the function (test-prog p.rkt) from wtest/test-programs/get-progs.rkt for each of the Racket programs in the list obtained by the function (get-progs lang) from the same file with the lang argument being “fraud”. The (get-progs lang) and (lang-before lang ols) functions in this file are the same as those in test/test-programs/get-progs.rkt. This list, therefore, will include all the test .rkt files in the folders for the languages up to the fraud language folder in the wtest/test-programs folder. These files are the same as those in the test/test-programs folder.

The (test-prog p.rkt) function has been modified from that in test/test-programs/get-progs.rkt in that it uses the .wrun, .wat and .wasm files and a modified (make p.wrun) function instead of the .run file and the (make p.run) function. The (make p) function has been modified to run the command make -C .. -s -f makewat [full path of p.wrun] instead of the command make -C .. -s [full path of p.run].

All these tests and the added tests work correctly indicating the correctness of the Wasm compiler, to the extent of our compiler to x86, for the subset of Racket expressions covered by the Wasm compiler and tested in these tests.

The Version of the Compiler that Uses WebAssembly Code and Wasmtime as Runtime

The compile-wasm_2.rkt file contains the version of the compiler that uses the Wasmtime and WebAssembly code as the runtime to print the result and execute external functions such as $error_exit. This is a partial implementation of this runtime as printing of chars, strings, integers, and booleans, and calling an error function with printing of "err" and exiting the program have been implemented for the purpose of delineating the method to write the runtime in WebAssembly, but printing of boxes, pairs, and lists, and operations of read-byte, peek-byte, and write-byte have not been implemented.

The file compile-wat-file_2.rkt calls the (compile e) function in compile-wasm_2.rkt and the (wasm-string a) function in wasm/printer.rkt to generate the compiled WebAssembly text format code similar to what is done from the compile-wat-file.rkt file.

A make rule %.wasm2 with dependency %.rkt has also been added to the make file makewat to automate the generation of a target %_2.wasm file of the compiled WebAssembly code by running the following commands:

racket -t compile-wat-file_2.rkt -m $< > $(patsubst %.wasm2, %_2.wat, $@)
wat2wasm $(patsubst %.wasm2, %_2.wat, $@).

Thus, running the Linux command make -f makewat %.wasm2 will result in the creation of a WebAssembly file %_2.wasm in binary format from the compilation of the Racket code in %.rkt using the (compile e) function in compile-wat-file_2.rkt that enables this file to interact with Wasmtime. Then, the Linux command wasmtime %_2.wasm will run the code in this file in the Wasmtime runtime resulting in the printing of the result.

The Wasmtime fd_write function takes as arguments a file descriptor (usually 1 for stdout), the pointer to iov array (0 for memory address 0), iov_len (the number of the strings printed), and the address to store the number of bytes written. In the linear memory of the WebAssembly module, in addresses 0 to 3, as part of the iov array, the address of the start of the data section is stored as an i32 value type in 4 bytes. This is the address in memory where we start the writing of the UTF-8 encodings of the characters of the string and is the integer 8. In the 4 bytes of the addresses 4 to 7, the length of the string in the number of bytes is written. As such, here we write the number of the bytes that we write to the memory for the UTF-8 encodings of the chars of the string. We allow the data section to go up to the address of 224 – 1 = 16777215 (which can hold about 223 chars whose UTF-8 encoding is 2 bytes [e.g., λ] before overflowing into the heap section). Then, the heap section starts from the address 224 and can go up to the address 225, and therefore, this will allow strings of up to the length (225 - 224) / 4 = 222 = 4194304 to be stored on the heap (given that we do not use the packed string representation in the Wasm compiler).

We have two methods for writing the Unicode characters of the strings to stdout in our compiler. The first method, which we have implemented in the function $process_string with calls to the function $write_utf_to_memory, is to write the UTF-8 encodings of the chars of the string to the data section in memory, keeping track of the number of bytes written, and then call the Wasmtime fd_write function with this data in memory. The fd_write function then uses the Rust write_vectored function to write the chars of the string (using UTF-8 encoding) from memory to stdout.

The second method, (which we have implemented in the function $process_string_with_print_codepoint), is to use the print_codepoint function, which is a function that we have written in Rust and added to Wasmtime following the method described by Radu Matei with modifications needed for the new version of Wasmtime (for details of the method to add this function please see the file .wasmtime/Additions_README.md). Similarly, we have written the print_int and error_exit functions in Rust and added them to Wasmtime for printing of integers, and printing of “'err” and exiting the program when there is an error, respectively. We also have implemented the printing of chars and booleans using print_codepoint.

To toggle between the two methods for printing the strings, we comment out one of the calls to $process_string_with_print_codepoint or $process_string in the $process_result function.

The executable wasmtime in the the-838e-compiler/.wasmtime/bin/ folder is a modified build of wasmtime after making the above changes in a git clone of the Wasmtime repository and doing a new build with cargo build --release. The push.yml file is modified to install g++, cmake, wabt, and nodejs, and add the paths of wabt/bin and .wasmtime/bin to the $PATH environment variable, so that the tests can be run on GitHub.

For testing the compile-wasm_2.rkt code with the modified wasmtime and the added WebAssembly code as the runtime, the following files were added: wasm/interp_2.rkt, wtest/compile_2.rkt, and wtest/test-runner_2.rkt. The file wtest/test-runner_2.rkt includes the subset of the test expressions from wtest/test-runner.rkt covered by this system, and the file wtest/compile_2.rkt uses the (compile e) function from compile-wasm_2.rkt for running these tests. The file wtest/interp_2.rkt includes a modified (wasm-interp a) function adapted to running the tests with wasmtime. All these tests work correctly indicating the correctness of the code in compile-wasm_2.rkt and the modified wasmtime to the extent tested in these tests.

Comparing the Speed of Different Combinations of Compilers, Runtimes, and the Methods of Printing Strings for Running the Executable for the Expression (make-string 1000000 #\λ)

We tested the speed of the different combinations of our compilers, runtimes, and the methods of printing strings in the case of running the compiled code obtained from each method for the expression (make-string 1000000 #\λ) in the file str1000000lms.rkt, which results in the forming of a string of 1000000 λ’s on the heap, traversing this string on the heap, sending the characters of the string to stdout in UTF-8 encoding, and displaying the 1000000 λ’s on the screen. The 5 combinations are listed below. The average times over 5 measurements for each combination were calculated and are shown in Table 1 below.

We automated the measurement of these times with the make file mkexeslms, the shell script timestrlms2screen, and the Racket file timestrlms2screen.rkt. The shell script timestrlms2screen takes a first integer argument that specifies the choice of one of the combinations 1 to 5 below. For choice 3 versus 4, we need to set the calling of the appropriate function ($process_string for choice 3 and $process_string_with_print_codepoint for choice 4) in the function $process_result in the file compile-wasm_2.rkt before running the script. The shell script first creates all the executables for the combinations 1, 2, (3 or 4), and 5 with executing the command make -f mkexeslms all. Then, it times one of the combinations 1, 2, (3 or 4), or 5 based on its first argument. It can take a second argument also. If this argument is the word “clean”, then it also removes the created files with the command make -f mkexeslms clean. The file timestrlms2screen.rkt executes timestrlms2screen with the argument 1 by a system command.

  1. Compiler to x86 in compile.rkt with C runtime: Each run of this test was done with running the executable str1000000lms.run made from linking the compiled x86 code of the program and the libarary functions (generated using our compiler to x86 in compile.rkt) with x86 code of other libraries and our C runtime. For this test, we increased the heap_size in runtime.h to 333400 to have enough memory to put this string on the heap. Because we use a packed string representation with 3 characters in each word in our compiler to x86, 333334 words is enough. Less than 333310 resulted in segmentation fault as expected from accessing parts of the memory not allocated for the program. If the string representation was nonpacked with each char in one word, then we would have needed more than 1000000 words for the heap size.

  2. Wasm compiler in compile-wasm.rkt with Node.js runtime: Each run of this test was with running the executable str1000000lms.wrun which runs the WebAssembly binary format code in the %.wasm file with jsmain.js, our Node.js runtime program. The Wasm binary format code in %.wasm has been obtained by the WABT wat2wasm tool from the %.wat WebAssembly text code generated using our Wasm compiler in compile-wasm.rkt.

  3. Wasm compiler in compile-wasm_2.rkt with WebAssembly and wasmtime runtime with using the fd_write function for printing the string: Each run of this test was done with the command wasmtime str1000000lms_2.wasm which uses the wasmtime runtime to run the %_2.wasm file, with %_2.wasm file created from the WebAssembly code compiled with the version of our Wasm compiler in compile-wasm_2.rkt that in the WebAssembly runtime code writes the UTF-8 encoding of the characters of the string to memory and then uses the Wasmtime fd_write function for sending the encodings to stdout.

  4. Wasm compiler in compile-wasm_2.rkt with WebAssembly and wasmtime runtime with using the print_codepoint function for printing the string: Each run of this test was done with a process similar to the process in number 3 above, except that the version of our Wasm compiler in compile-wasm_2.rkt was used that in the WebAssembly runtime code prints the strings with the function print_codepoint.

  5. A simpler version of our compiler to x86 in the file 3_compile.rkt without higher-order functions and library functions, and with a make-string function that puts each char in a word (nonpacked representation of a string), and with a simpler C runtime that prints a string accordingly: Each run of this test was done with running the executable str1000000lms.3_run made from linking the compiled x86 code of the program (generated using this version of compiler to x86) with this simple version of C runtime. The executable was made using the make file 3_make with the command make -f 3_make str1000000lms.3_run.

image

We also measured the above timings with redirecting the output of 1000000 λ’s to the disc to be written to a temp file instead of the screen. We automated the measurement of these times with the shell script timestrlms and the Racket file timestrlms.rkt. The shell script timestrlms takes a first integer argument that specifies the number of times the loop for timing the combinations runs. It first creates all the executables for the combinations 1, 2, (3 or 4), and 5 above with executing the command make -f mkexeslms all. Then, it times the combinations with the output redirected to the temp file. It can take a second argument also. If this argument is the word “clean”, then it also removes the created files with the command make -f mkexeslms clean. The file timestrlms.rkt executes the shell script timestrlms with the argument 1 by a system command.

With executing ./timestrlms 5, the average times for 5 measurements for each of the above 5 combinations of compilers, runtimes, and printing methods with the output redirected to a temp file were calculated and are shown in Table 1 above.

Other Changes

We also added the files compile-intmd-utils_2.rkt, compile-exprs_2.rkt, compile_2.rkt, and test/compile_2.rkt, and added the instruction (ICall x) to a86/ast.rkt. The (ICall x) instruction is for abstracting out padding the stack before, and unpadding the stack after a call to an external C function to ensure alignment of the stack frame to 16-byte boundaries before the call (to avoid faults due to the software conventions for 128-bit SIMD technologies in Intel® 64 and IA-32 architectures). The function (intmd-to-a86 c) in compile-intmd-utils_2.rkt desugars the (ICall x) instruction to (seq (pad-stack c) (Call x) (unpad-stack c)). In addition, we replaced the instructions of the form (Jx (error-label c)), where Jx is one of the a86 instructions Je, Jl, Jg, Jmp, Jne, Jle, or Jge, with (Jx 'err). The desugaring of (Jx 'err) to (Jx (error-label c)) is also implemented in the function (intmd-to-a86 c). We did the replacements of the above intermediate instructions in a copy of the file compile.rkt and divided this file into two smaller files compile_2.rkt and compile-exprs_2.rkt, which makes the organization of the files better. Desugaring of these instructions is done in compile-exprs_2.rkt by making calls to (intmd-to-a86 c). The file test/compile_2.rkt runs the tests in test/test-runner.rkt for this implementation. The tests work correctly with this implementation.

Discussion

We wrote a Wasm compiler and a Node.js runtime with the features discussed above that work together correctly to compile the programs and print the result of the execution of the programs. In addition, we implemented the interaction of the Wasm compiler with a second runtime that includes functions written in WebAssembly to process the result and write the data to the linear memory of the Wasm module and then interact with Wasmtime through the fd_write function of Wasmtime, and also through functions that we have implemented in Rust and added to Wasmtime, to print the result or to print “'err” and exit the program when there is an error.

In terms of performance, for making a string of 1000000 λ’s on the heap, traversing the string on the heap, sending the characters of the string to stdout, and displaying the characters on the screen, our Wasm compiler with the WebAssembly and Wasmtime runtime, with the use of the fd_write function, performed better than our compiler to x86 with the C runtime, with an average real time of 0.252s and an average user time of 0.013s (column 3 in Table 1), compared with an average real time of 0.584s and an average user time of 0.059s for the compiler to x86 with the C runtime (column 1 in Table 1), over 5 measurements for each time. Although we did not formally calculate the standard deviations and the p-values for these differences, the variations observed for each time over the 5 measurements were small compared with the difference in the case of user time and practically negligible in the case of real time.

This difference was also seen in the case the output was redirected to the disc to be written to a temp file, with the average real time and user time for the combination number 3 being 0.030s and 0.012s, as compared with 0.080s and 0.060s for the combination number 1.

We also measured these times for the combination number 5, with a simpler version of our compiler to x86 without higher-order functions and library functions, and with a make-string function that puts each char in a word (nonpacked representation of a string), and with a simpler C runtime that prints a string accordingly, to see if any part of this difference was caused by these factors. The times are very close to each other for the combinations number 5 and 1, indicating that these factors have negligible effect. This includes the time that it takes for the additional linking of some of the libraries and loading a larger final executable in the memory, which is expected to be negligible compared with the time that it takes for I/O. Our make-string function for the packed string representation, and our traversal of the string on the heap for obtaining the characters are also efficient being of complexity O(n) similar to the implementation in the nonpacked representation case, and therefore this factor is also negligible.

The main factor that probably accounts for this difference is that in the C runtime for each character of the string, the function printf is called, whereas in the WebAssembly and Wasmtime runtime with using the fd_write function, the UTF-8 encodings of the characters of the string are all written to memory before the function fd_write is called and then fd_write uses the Rust write_vectored function to write from memory to either the screen or the file on the disk. This process is more efficient likely because of a lack of need for repeated scheduling for I/O and because of DMA (direct-memory-access) with a DMA command block with the source pointer, the destination pointer, and the count of the number of bytes to be transferred [Abraham Silberschatz, Peter Baer Galvin, Greg Gagne. Operating System Concepts, 10th Edition. Chapter 12 I/O systems, 12.2.5. Direct Memory Access]. The CPU writes the address of the command block to the DMA controller and then goes on with other work [Same Ref.]. The DMA controller operates the memory bus directly, performing the transfer without the help of the CPU [Same Ref]. This obviates the need for programmed I/O (PIO), using the CPU to watch the status bits and to feed data into a controller register one byte at a time [Same Ref.]. Calling printf for each character likely results in the involvement of the CPU for each character with printing each character becoming a scheduled I/O that the operating system scheduler needs to schedule. This results in time delays causing the large differences in real time observed between the combination number 1 and the combination number 3 both with output to the screen and to the disc for writing to the temp file.

Using the print_codepoint function [which uses Rust print! function, std::char::from_u32(i32 argument cast as u32), and a match with Some(x)] added to the wasmtime executable is also not efficient as compared with using the Wasmtime’s own fd_write function, as demonstrated by the large differences between the times in column 4 and column 3, likely in part because of the same problems with I/O described above caused by calling the print_codepoint function for each character. The times indicate this is worse than the case of the combination number 1, maybe because the x86 code with C runtime is more efficient than the WebAssembly with Wasmtime runtime. Other possible factors may be that some optimizations associated with calls to printf and buffering of the characters in C are performed and that some inefficiency is introduced in the writing of a print function and making calls to it as opposed to using one implemented at lower level.

Using the Node.js runtime is less efficient as the times for the combination number 2 indicate. Part of this is likely because of calling the console.log() function for each character of the string causing the same problems with I/O as described above. In addition, part of this can be because JavaScript is an interpreted language and even with the just-in-time compilations, optimizations, and reoptimizations performed by the V8 JavaScript engine, it can still not be as efficient as C and the code compiled to an executable. The distinctly long user time with both output to the screen and to the disc for writing to the temp file with combination number 2, as compared with the other 4 combinations, may be because of this, as the program is likely busier running code in the user space in the case of the Node.js runtime.

Even though using the fd_write function with the WebAssembly and Wasmtime runtime is more efficient than using the print_codepoint function, especially for long strings, one advantage of print_codepoint is that it will not overflow the data section of the memory. We set the size of the data section in the linear memory of our Wasm module to be 16777216 = 224 bytes from addresses 0 to 16777215. We write the characters to this section in memory in UTF-8 encoding, because the fd_write function uses this encoding to write the characters to stdout. Then, given that the UTF-8 encoding of λ is 2 bytes, we cannot use fd_write to print a string of longer than about 8388608 λ’s, because this will overflow the data section into the heap section.

But because we do not allow the heap to grow beyond the address 225, and given that we do not use a packed string representation in our Wasm compiler, with the word-size of 4 bytes, this will allow strings of up to the length about (225 – 224) / 4 = 222 = 4194304 characters to be placed on the heap, and therefore this is the limit for the length of the longest string of λ’s that we can print. If we set the stack section to be smaller and allow the heap to grow to close to the top address of 226 – 4 of our memory, then we can print a string of up to length about (226 – 224) / 4 = 3 . 222 = 12582912 characters. Trying to put a longer string on the heap, even if we allow overflowing into the stack section, will hit the top address of our memory and result in WebAssembly generating a trap with the message “wasm trap: out of bounds memory access”.

In the more general case that the size of the UTF-8 encoding of a character can be up to 4 bytes, a string of longer than about 224 / 4 = 4194304 characters will overflow the data section, and therefore this is the upper limit that we should allow for the length of a string. If we increase the size of the data section, then the heap section will become smaller, and this will decrease the length of the string that we can put on the heap to less than 4194304 characters. So, the size of the data section of 224 obtained from the equation (225 - x) / 4 = x / 4 is the size that maximizes the length of the string that we can put on the heap and print with this method. Writing the code for a method to call the fd_write function repeatedly may allow us to decrease the size of the data section and to increase the maximum length of the string that we can put on the heap and print, to the limit imposed by the heap.

Therefore, with the use of the fd_write function, overflowing of the data section can occur, and we need to set a limit for the length of the strings. We also need to carefully balance the size of our data section and heap section to allow the maximum length of strings to be printed. If we use a packed string representation, we can make the heap size smaller and print longer strings with the fd_write function.

With using the print_codepoint function to print the string, we cannot overflow the data section of the memory, because we do not write the characters of the string to memory, but still the string have been placed on the heap, and with our memory size of 226, with the nonpacked string representation, the maximum length of a string that can be placed in this memory is 226 / 4 = 224 = 16777216, even if we set the sizes of data and stack sections to 0. With a more packed string representation, this length can be increased. This is an advantage of about 222 = 4194304 characters over the case when we use the fd_write function. Given that using the print_codepoint function is slower than using the fd_write function, this is not a major advantage, and using the fd_write function with a larger data section still is better. As mentioned above, one feature that can be added is to implement making repeated calls to the fd_write function for longer strings, and by this way, the problem of fd_write function overflowing the data section can be solved.

The processing of the rest of the value types, such as boxes, pairs, and lists, for writing the UTF-8 encoding of the appropriate characters to memory to be printed by the fd_write function can also be implemented in WebAssembly to complete the runtime of WebAssembly and Wasmtime. Printing of the integers can also be done in this way. The error_exit function added to Wasmtime can still be used for errors, or a more sophisticated way can be found or implemented in Wasmtime for this.

Implementation of the interaction with Wasmtime for the read-byte, peek-byte, and write-byte functions can also be done by adding functions similar to error_exit to Wasmtime. Alternatively, this can be done by more sophisticated uses of the Wasmtime functions or implementation in Wasmtime.

dandorat added 18 commits April 20, 2021 19:31
…ionals, zero?, if, boolean and char types, integer?, integer-length, char?, char->integer, integer->char, eof-object?, write-byte, read-byte, peek-byte, compile-begin, eq-imm, assert-type, assert-integer, assert-char, assert-byte, assert-codepoint to WebAssembly. Added functionalities to wasm/interp.rkt, wasm/printer.rkt, and jsmain.js. Added jsmain-io.js. Updated wtest/test-runner.rkt, wtest/test-programs, wtest/test-progs.rkt, and wtest/test-programs/get-progs.rkt.
…s, and strings, unbox, car, cdr, empty?, string?, string-length, eq?, string-ref, and make-string to WebAssembly. Added tests. Made changes to wasm/interp.rkt, wasm/printer.rkt, and jsmain.js. Removed jsmain-io.js and incorporated its functionalities in jsmain.js.
…r an alternative faster runtime with printing of chars and strings with the wasmtime fd_write or with a print_codepoint function written in rust added to wasmtime, printing of integers with a print_int function added to wasmtime, printing of 'err and exiting the program with an error-exit function added to wasmtime for errors, and printing of booleans. This is a partial implementation of this runtime and printing of boxes, pairs and lists, and addition of write-byte, read-byte, and peek-byte functions to wasmtime have not been implemented. Added the code in new files compile-wasm_2.rkt, compile-file-wat_2.rkt, and wasm/interp_2.rkt. Updated wasm/printer.rkt and makewat. Added tests for testing this runtime in a new file wtest/test-runner_2.rkt with the call to the test-runner function in wtest/compile_2.rkt. Tested the speed of different runtimes with different compiled code and executables with running the compilation of the file str1000000lms.rkt which makes a string of 1000000 λs, sending the output of 1000000 λs to stdout. With running the executable str1000000lms.run file made from linking the compiled x86 code of the program and the libarary functions with x86 code of other libraries and our C runtime, the average times were real 0.584s, user 0.059s, and sys 0.053s. With running the executable str1000000lms.wrun which will make a %.wasm file from the compiled %.wat WebAssembly code and then runs it with a node.js Javascript runtime program, the average times were real 0.998s, user 0.622, and sys 0.169s. With the command wasmtime str1000000lms_2.wasm which uses wasmtime runtime to run the %_2.wasm file, with %_2.wasm file created from the WebAssembly code compiled with the version of compile-wasm_2.rkt that in the WebAssembly runtime code writes the UTF-8 encoding of the characters of the string to memory and then uses wasmtime fd_write function for sending the encodings to stdout, the average times were real 0.252, user 0.013, and sys 0.031. The wasmtime fd_write function uses the rust write_vectored function. With the command wasmtime str1000000lms_2.wasm which uses wasmtime runtime to run the %_2.wasm file, with %_2.wasm file created from the WebAssembly code compiled with the version of compile-wasm_2.rkt that in the WebAssembly runtime code writes the Unicode characters to stdout using the function print_codepoint written in rust added to wasmtime, the average times were real 1.0894, user 0.234, and sys 0.053. Increased the heap_size in runtime.h to 3000000 to have enough memory to put this string on the heap when running str1000000lms.run.
…rnal functions with pad-stack and unpad-stack, and jumps to (error-label c), with intermediate instructions. Dividing compile.rkt into two files.
… watjs2run_README.md, makewat. For the purpose of a reference for times, added the 3_* files for an early simpler version of our compiler to x86 without higher-order functions and library functions, and added to this a make-string function that puts each char in a word (nonpacked representation of a string) and to its C runtime the code for printing of a string accordingly. To automate measurement of the times, added the files mkexelms timestrlms timestrlms.rkt timestrlms2screen timestrlms2screen.rkt.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant