Thoughts on a side-car GPU path tracing course

[armansito]

Hi there!

I've built a number path tracers on the side over the years. My first exposure to the Kajiya paper was in a graduate level course that I took as an undergrad 12 years ago. The course was fascinating to me but the way it presented the topics was complicated and math heavy, which led to several students to drop out before they even started hacking away at their first path tracer.

I find that many teaching materials make things too complicated than they need to be, especially for someone who is simply looking for an introduction into the field without having to read through 100 pages of PBRT when it takes only ~100 lines of code to put together a basic renderer. I think Ray Tracing in One Weekend has accomplished that, so I would like to commend the authors.

Someone who is getting started with GPU programming and wants to take a shot at implementing a real-time(ish) path-tracer will likely face a similar lack of simple introductory materials. There is certainly a ton of resources out there but picking out the useful and practical ones for a newcomer is not straightforward. GPU programming is not easy (especially when it comes to making effective use of the hardware).

Your book has inspired me to write my own book. I was originally planning this as a blog series but after attending your Birds of a Feather discussion at SIGGRAPH '23 and chatting with @hollasch and @trevordblack, I am now encouraged to think that this may be a good addition to the RayTracing project as a side-car book/chapter.

I've put some thought into how this might be structured but I would love to hear your feedback. I'm more than happy to do 100% of the work on it. The primary thing that I'm asking of the authors and the community here is feedback and suggestions on how to make sure that it accomplishes its teaching goals without getting too lost in the less-than-rewarding details of building GPU infrastructure.

Goals

The first milestone is for the reader to first implement a simple, interactive "reference path-tracer" (if this were to be structured like the RTIOW books, then this would be Book 1) . I put together a WebGPU demo for what this may look like at https://armansito.github.io/path-tracer (I've tested it on Chrome 115 on macOS and Windows if you'd like to try it out; it demonstrates the 2 test scenes from Book 1 and one additional scene that I added).

Thoughts on the structure of the book

Prerequisites: The reader has gone through Book 1 of RTIOW. The reader has some familiarity with a modern graphics API and at least one shading language. I think this book should NOT be a tutorial on writing shaders or a particular GPU framework (except perhaps as an Appendix?). I think the reader should be able implement this using any graphics API of their choice but perhaps the course provides some amount support code for people who don't have a ton of prior experience and just want to get started. I've written more of my thoughts on this below.

These are the teaching goals of the first book:

1. A compute shader setup that renders 1 sample-per-pixel per frame and accumulates the radiance values in a texture over time.
2. Implementing a simple PRNG on the GPU.
3. Building the intersection and material evaluation loop as a mega-kernel.
4. Representing Shape and Material object instances first as GPU-only constants in the shader program. Moving the Shape and Material objects into shared buffers in a later stage.
5. Early path termination using Russian roulette probability.
6. Discussion of occupancy and thread divergence as scene complexity increases.

I think all of the above can be taught in the exact same progression as RTIOW. Beyond this milestone, I imagine a second chapter would go into BVH construction and packed representation in memory, GPU-side BVH traversal, as well as construction, emissive materials and importance sampling, etc. From there the series could take several directions, each of which could be their own small book/chapter that builds upon the foundation: such as incrementally transitioning to a wavefront architecture, dynamic denoising, etc. Generally I would like to avoid getting into the specifics of the various modern ray tracing APIs and focus on building a compute shader pipeline on your own, though who knows? Maybe that deserves its own separate chapter(s).

Thoughts on support code

One explicit goal here is that the reader can implement this using the graphics API of their choice. However, I would like to provide support code/template in one language/framework that the reader can use as a reference if they wish. My choice for this is Rust+wgpu+winit. I know that providing code that is tied to one API and windowing environment might be slightly against the philosophy of RTIOW (which has taken the approach of starting with an empty file and zero code dependencies which makes it so accessible and flexible) but I think this is somewhat unavoidable when interacting with a GPU. The Rust+wgpu+winit combination has the advantage that it makes the code easily portable across a range of OS's and APIs (Metal, Vulkan, D3D12, and WebGPU are all easily supported).

The shader snippets could come in both WGSL and GLSL flavors (for example, both version of a given snippet could be embedded together in a frame with toggles for the two). GLSL is familiar to many but WGSL unlocks WebGPU and portability via wgpu (that said GLSL can be compiled down to WGSL).

I think the "Book 1" milestone can be implemented completely in a shader and someone should be able to build it in shadertoy if they want to, though managing some of the API-side logic (e.g. resource management and the pipeline setup) becomes somewhat important for later chapters.

Let me know what you think!

[hollasch]

I think this sounds great — go for it! One slight hesitation is that Rust is a bit less known to the broad audience, but I assume you'll anticipate and address questions for newcomers to the language. You'll obviously have to do this anyway for the other components of the setup.

How would you like to host this project? We can certainly create a new repository in the raytracing project and assign you administrative rights. You could even make this private if you'd prefer while you ramp up. On the flip side, I would certainly understand if you wanted to hold this project a bit closer to your chest and host it on your own personal GitHub page. If you go this route, we can provide whatever links and references you think would help.

I've sent you a Slack invite for our collaborators workspace, where we can discuss this further.

[armansito]

After a discussion on the Slack, @hollasch has kindly set up the gpu-tracing repository. I will start putting the basics of the repo together shortly. After that I'll start sketching out the book's outline and putting together the beginnings of the code template.

[rchiaramo]

Hey this sounds like a cool project. I've done the implementation in Rust using winit-wgpu-imgui, and I'm now attempting to rewrite my code to move from a megakernel approach to a wavefront approach. Are you considering wavefront at all? I plan to do it without any hardware acceleration (wgpu doesn't support it at this point anyway to the best of my knowledge).

I'd be psyched to participate if you would like some more hands on the project. I've implemented a BVH using SAH, and an interactive camera, as well as very simple GUI interaction to set several render parameters.

[armansito]

@rchiaramo Hi there! I apologize for the late reply. Yes, I would love for you to participate! The GPU book is hosted at github.com/RayTracing/gpu-tracing and you can view the current working draft at https://raytracing.github.io/gpu-tracing. I suggest starting a Discussion or Issue there with your ideas. You can also help by going over the current working draft and sending edits and fixes, as well as your overall feedback. There is still more to do (I would say the book is about 35% done). I had to take a break from working on the book recently due to personal reasons but I plan to pick up the pace again soon.

I'll follow up with my planned outline for the remainder of the book in that repo. I think wavefront is an excellent topic, though I intentionally didn't consider it for the first installment. My overall idea is for the first tutorial to result in a megakernel that implements most aspects of the RTIOW books 1-3 in a GPU renderer without overly optimizing it. I would like to then have smaller, more focused follow-up books/tutorials that build on that foundation. Wavefront is an excellent topic for one of those follow ups and I can imagine many others (for example: integrating imgui/egui, GPU BVH construction, GPU algorithms like sorting and stream compaction, hardware accelerated RT, sample reuse ala ReSTIR, even perhaps Spherical Harmonic lighting, ambient occlusion, etc). The follow-up books/tutorials can be independent of each other; the general idea is that the first book seeds the initial playground that can branch off to explore different topics.

Let me know what you think and please participate in the GPU book repo!

[Juanxpeke]

Hi, I am finishing Book 1 of RTIOW and I have always wondered how to make ray tracing work in real time, so I think your idea would be awesome for programmers.

Sorry if this is not useful to the topic, but you said: "There is certainly a ton of resources out there but picking out the useful and practical ones for a newcomer is not straightforward. GPU programming is not easy (especially when it comes to making effective use of the hardware)". I have some background in GPU programming using CUDA and OpenGL compute shaders, but I think I lack a lot of low-level GPU knowledge and it is hard to find documentation about it. Would you mind suggesting some sources for learning?