Skip to content

Latest commit

 

History

History
146 lines (103 loc) · 9.86 KB

README.md

File metadata and controls

146 lines (103 loc) · 9.86 KB
Raw

DeOldify.NET

C# implementation of Jason Antic's DeOldify(https://github.com/jantic/DeOldify) Only for photos for now!

Paper "DeOldify.NET: cross-platform application for coloring black and white photos" was accepted to poster session of Neuroinformatics - 2022 conference. The paper describes technical details of managed C# implementation of the original DeOldify and contains some comparisons with different other image colorization products.

How to run

On Windows 7, 8, 8.1, 10, 11

  • Make sure that .NET Framework 4.5+ (4.6+ for SIMD-accelerated version) or higher is installed on your computer.

  • You can use any bit depth (x32 or x64), but on a 32-bit system you will not be able to process large images due to the limited amount of memory.

  • SIMD and Stable model are supported only in 64-bit mode. On a 32-bit machine, you should use the regular artistic version.

  • At least 3 GB 1.5 GB with new convolution algorithm of free RAM is required to run Artistic model. About 3 GB is required for Stable model.

  • Select a version of DeOldify.NET the one you want to build. Versions with and without simd are available, with float32 typed weights (higher accuracy) and float16 typed weights (lower accuracy and smaller file size), with a stable model and an artistic model, as in the original DeOldify. Further actions depend on your choice.

  • Download and unpack the repository, then download model from the releases (https://github.com/ColorfulSoft/DeOldify.NET/releases/download/Weights) and place it in Implementation\src\Resources.

Model Details File
float32 Artistic Artistic model with single-precision floating point weights. More accurate than compressed float16 model. https://github.com/ColorfulSoft/DeOldify.NET/releases/download/Weights/Artistic.model
float16 Artistic Artistic model with half-precision floating point weights. Less accurate than original float32 model, but requires 2 times less disk space. https://github.com/ColorfulSoft/DeOldify.NET/releases/download/Weights/Artistic.hmodel
float32 Stable Stable model with single-precision floating point weights. More accurate than compressed float16 model. https://github.com/ColorfulSoft/DeOldify.NET/releases/download/Weights/Stable.model
float16 Stable Stable model with single-precision floating point weights. Less accurate than original float32 model, but requires 2 times less disk space. https://github.com/ColorfulSoft/DeOldify.NET/releases/download/Weights/Stable.hmodel
  • Select compilation script and run it.
Build Details Script
Artistic Basic version of Artistic colorizer with float16 weights Compile.artistic.bat
Artistic.w32 Artistic colorizer with float32 weights Compile.artistic.float.bat
Artistic.simd Artistic colorizer with SIMD acceleration and float16 weights Compile.artistic.simd.bat
Artistic.simd.w32 Artistic colorizer with SIMD acceleration and float32 weights Compile.artistic.simd.float.bat
Stable Basic version of Stable colorizer with float16 weights Compile.stable.bat
Stable.w32 Stable colorizer with float32 weights Compile.stable.float.bat
Stable.simd Stable colorizer with SIMD acceleration and float16 weights Compile.stable.simd.bat
Stable.simd.w32 Stable colorizer with SIMD acceleration and float32 weights Compile.stable.simd.float.bat

  • The executable file will appear in the Implementation\Release folder. The application is ready to work!

  • Use!

Windows GUI

On Linux (Tested on Mint)

  • We recommend that the first step is to update everything. It may take time, but it's worth it:
sudo apt-get update
sudo apt-get upgrade
  • Install Mono:
sudo apt-get install mono-complete
  • Get sources; select and download model
Model Details File
float32 Artistic Artistic model with single-precision floating point weights. More accurate than compressed float16 model. https://github.com/ColorfulSoft/DeOldify.NET/releases/download/Weights/Artistic.model
float16 Artistic Artistic model with half-precision floating point weights. Less accurate than original float32 model, but requires 2 times less disk space. https://github.com/ColorfulSoft/DeOldify.NET/releases/download/Weights/Artistic.hmodel
float32 Stable Stable model with single-precision floating point weights. More accurate than compressed float16 model. https://github.com/ColorfulSoft/DeOldify.NET/releases/download/Weights/Stable.model
float16 Stable Stable model with single-precision floating point weights. Less accurate than original float32 model, but requires 2 times less disk space. https://github.com/ColorfulSoft/DeOldify.NET/releases/download/Weights/Stable.hmodel
Using git and terminal 
git clone https://github.com/ColorfulSoft/DeOldify.NET.git
cd DeOldify.NET
wget <model url> -O Implementation/src/Resources/<model name>
Using GUI
  • Select and run compilation script
Build Details Script
Artistic Basic version of Artistic colorizer with float16 weights Compile.artistic.sh
Artistic.w32 Artistic colorizer with float32 weights Compile.artistic.float.sh
Artistic.simd Artistic colorizer with SIMD acceleration and float16 weights Compile.artistic.simd.sh
Artistic.simd.w32 Artistic colorizer with SIMD acceleration and float32 weights Compile.artistic.simd.float.sh
Stable Basic version of Stable colorizer with float16 weights Compile.stable.sh
Stable.w32 Stable colorizer with float32 weights Compile.stable.float.sh
Stable.simd Stable colorizer with SIMD acceleration and float16 weights Compile.stable.simd.sh
Stable.simd.w32 Stable colorizer with SIMD acceleration and float32 weights Compile.stable.simd.float.sh

  • The executable file will appear in the Implementation/Release folder. The application is ready to work!

  • Run application using mono <build name>.exe command in terminal or double click as in Windows.

  • Use!

Linux GUI

Please note, that DeOldify.NET using Mono is a bit slower, than using .NET Framework

Examples

Example1

Example2

Original Artistic Stable
Original Artistic Stable

New algorithms

DeOldify.NET has become a platform for testing the latest highly optimized algorithms, which will then be applied in the System.AI project. In this section you can find some information about the results of the experiments.

Patch2Vec Conv2d

The meaning of most fast convolution algorithms, such as im2col or im2row, involves bringing the convolution to matrix multiplication, which allows optimizing memory access operations by using the processor cache. However, such methods either require a buffer for srcC * kernelY * kernelH * dstH * dstW elements, which is extremely irrational. The proposed patch2vec method unwraps each patch of the input image on the fly, and then applies all convolution filters to it. This implementation is not inferior in efficiency to classical algorithms like im2col, and in practice even surpasses them. The buffer for this algorithm will have the size of srcC * kernelY * kernelX, which is much smaller than in the case of similar methods. Moreover, patch2vec does not impose restrictions on the convolution parameters, unlike, for example, the Shmuel Vinograd method. The proposed algorithm is difficult to fit into classical machine learning frameworks due to the fact that they are focused on using GEMM as the core. Pure C#-based implementations make it easy to do this.

For more detailed information, please see official Patch2Vec repository: https://github.com/GlebSBrykin/Patch2Vec

In DeOldify.NET two versions of the patch2vec conv2d algorithm are implemented - with and without SIMD support. You can choose, which version to use by executing the corresponding compilation command file. Vectorization is implemented through the Vector4 structure of the System.Numerics namespace. Vectorization is only available for x86-64 processors at version .NET Framework 4.6 and higher, or when using Mono newer than 2008.

Method Time (ms)
im2col 123902
patch2vec 114970
patch2vec + simd 33270

All tests was done in Windows 7 x64 laptop with Intel(R) Core(TM) i5-6300HQ CPU and 32 GB of RAM

Updates

  • [10.09.2022] - Version 2.1 has been released. The ability to drag and drop images into the application has been added, error handling has been improved, and the process of saving the result has been simplified.
  • [20.07.2022] - The Windows and Linux versions are combined.
  • [11.07.2022] - DeOldify.NET now supports Stable model from original DeOldify. Added the ability to build DeOldify.NET with uncompressed original weights to obtain the maximum quality of coloring.
  • [27.04.2022] - DeOldify.NET has become a testing ground for the latest optimized algorithms. The Conv2d layer has been optimized and now requires significantly less memory. Support for SIMD vectorization will allow you to get about a fourfold increase in performance.
  • [29.10.2021] - Big refactoring and code clean up
  • [29.10.2021] - Linux support
  • [16.09.2021] - Fixed a memory leak issue