The C++ code in this repository is a hopefully accurate port of the python code in Michael Nielsen's book Neural Networks and Deep Learning. I recommend you to read Michael Nielson's book and if you wish to use C++ rather than python you can use the code in this repository to supplement your understanding. I would also like to thank Grant Sanderson, 3Blue1Brown for his entertaining and educational videos on Machine learning and for introducing me to Michael's book. If you're new to Machine learning I can also recommend as an excellent introduction Machine Learning by Stanford University on Coursera. Finally, thanks to the people who created the MNIST dataset for making their dataset available to us.
I've aimed to produce code as simple and concise as possible using only the features in the standard C++17, but as the standard library does not contain a linear algebra library I've used the uBLAS. I've tried to showcase some of the new features of Modern C++ here, STL algorithms and lambda functions. With just a couple of 100 lines of code we can produce a program which can recognize handwritten digits and implements the standard learning algorithm for neural networks, known as stochastic gradient descent.
In chapter 1 A C++ version of network.py is constructed in Python style as all the code is in one cpp file. The purpose of this version is to create code similar to the python code, rather than the best C++ possible, to allow you to read Using neural nets to recognize handwritten digits and follow the online book with C++.
Michael explains how the backpropagation algorithm works but adds no more Python code to his book. Giving me time to read up on some C++17 features I'd like to try out.
This time I'm aiming to produce C++ worthy of being called a library, but still maintaining code which is recognisable to the python network2.py from chapter 3 of Michael's book. Of course the main body of the code goes into a header file and I show how to construct some policy classes and how to use them in Network2.cpp.
The code written so far all runs in a single thread. Of course thanks to the C++ compilers vectorization skills it is able to do its job in a reasonable amount of time. In C++ 17 the Extensions for parallelism TR lets us change the execution plan of algorithms. I introduce Structured bindings into the codebase which increase conciseness and readability. Finally, I use some ideas outlined in Michael's book and add different activation functions to the library.
I hope you enjoyed this excursion into both C++ and Machine learning. I started writing this code as I was unable to find any easily approachable code for experimenting with Machine learning in C++. Once again thanks to Michael Nielsen for writing such an accessible and readable book.
Gareth Richards 14/11/2017