OnnxStack is a library that provides higher-level services and integrations for .NET applications, enhancing the functionality and versatility of the ONNX Runtime and Microsoft ML projects, offering a comprehensive solution that empowers developers to build, deploy, and execute machine learning models seamlessly in .NET environments, unlocking a world of possibilities for intelligent applications.
Inference Stable Diffusion with C# and ONNX Runtime
Stable Diffusion models take a text prompt and create an image that represents the text.
Example:
High-fashion photography in an abandoned industrial warehouse, with dramatic lighting and edgy outfits, detailed clothing, intricate clothing, seductive pose, action pose, motion, beautiful digital artwork, atmospheric, warm sunlight, photography, neo noir, bokeh, beautiful dramatic lighting, shallow depth of field, photorealism, volumetric lighting, Ultra HD, raytracing, studio quality, octane render
A negative prompt can be provided to guide the inference to exclude in calculations
Example:
painting, drawing, sketches, monochrome, grayscale, illustration, anime, cartoon, graphic, text, crayon, graphite, abstract, easynegative, low quality, normal quality, worst quality, lowres, close up, cropped, out of frame, jpeg artifacts, duplicate, morbid, mutilated, mutated hands, poorly drawn hands, poorly drawn face, mutation, deformed, blurry, glitch, deformed, mutated, cross-eyed, ugly, dehydrated, bad anatomy, bad proportions, gross proportions, cloned face, disfigured, malformed limbs, missing arms, missing legs fused fingers, too many fingers,extra fingers, extra limbs,, extra arms, extra legs,disfigured,
Many different scheduler algorithms can be used for this computation, each having its pro- and cons.
So far OnnxStack.StableDiffusion as included LMS Discrete, Euler Ancestral, DDPM, DDIM, and KDPM2 options with more in the works.
Example:
| LMS Scheduler | Euler Ancestral Scheduler | DDPM Scheduler |
|---|---|---|
 |
 |
 |
Seed: 624461087 GuidanceScale: 8 NumInferenceSteps: 22
Text To Image Stable Diffusion is a powerful machine learning technique that allows you to generate high-quality images from textual descriptions. It combines the capabilities of text understanding and image synthesis to convert natural language descriptions into visually coherent and meaningful images
| Input Text | Output Image | Diffusion Steps |
|---|---|---|
 |
 |
 |
Image To Image Stable Diffusion is an advanced image processing and generation method that excels in transforming one image into another while preserving the visual quality and structure of the original content. Using stable diffusion, this technique can perform a wide range of image-to-image tasks, such as style transfer, super-resolution, colorization, and more
| Input Image | Output Image | Diffusion Steps |
|---|---|---|
 |
 |
 |
Prompt: Dog wearing storm trooper helmet, head shot
Image inpainting is an image modification/restoration technique that intelligently fills in missing or damaged portions of an image while maintaining visual consistency. It's used for tasks like photo restoration and object removal, creating seamless and convincing results.
In the below example we use a simple mask image + prompt to add a rider to the horse The black part of the mask will be used buy the process to generate new content, in this case the rider
| Input Image | Mask Image | Masked Image | Result |
|---|---|---|---|
 |
 |
 |
 |
Prompt: Rider on horse
More information and Examples can be found in the OnnxStack.StableDiffusion project README
Image recognition with ResNet50v2 and ONNX Runtime
Harness the accuracy of the ResNet50v2 deep learning model for image recognition, seamlessly integrated with ONNX for efficient deployment. This combination empowers your applications to classify images with precision, making it ideal for tasks like object detection, content filtering, and image tagging across various platforms and hardware accelerators. Achieve high-quality image recognition effortlessly with ResNet50v2 and ONNX integration.
work in progress
Object detection with Faster RCNN Deep Learning with C# and ONNX Runtime
Enable robust object detection in your applications using RCNN (Region-based Convolutional Neural Network) integrated with ONNX. This powerful combination allows you to accurately locate and classify objects within images. Whether for surveillance, autonomous vehicles, or content analysis, RCNN and ONNX integration offers efficient and precise object detection across various platforms and hardware, ensuring your solutions excel in recognizing and localizing objects in images.
work in progress
You will need an ONNX compatible model to use, Hugging Face is a great place to download the Stable Diffusion models
Download the ONNX Stable Diffusion models from Hugging Face.
Once you have selected a model version repo, click Files and Versions, then select the ONNX branch. If there isn't an ONNX model branch available, use the main branch and convert it to ONNX. See the ONNX conversion tutorial for PyTorch for more information.
Clone the model repo:
git lfs install
git clone https://huggingface.co/runwayml/stable-diffusion-v1-5 -b onnx
The following are othe stable diffusion models we've converted to ONNX and tested with OnnxStack.
- OpenJourney V4
- DreamLike PhotoReal 2.0
- CyberPunk Anime Diffusion
- InkPunk Diffusion
- Mo-Di Diffusion
- epiCRealism
- Comic Diffusion
- Redshift Diffusion
You can choose between Cpu and DirectML(GPU) for inference,
Other Microsoft.ML.OnnxRuntime.* executors like Cuda may work but are untested
Cpu > 12GB RAM
DirectML > 10GB VRAM
We welcome contributions to OnnxStack! If you have any ideas, bug reports, or improvements, feel free to open an issue or submit a pull request.