Move all non-compute MIOpen code from C++ to embedded Python

[dmikushin]

Besides its main purpose, MIOpen involves a lot of general-purpose system programming, such as working with files, env variables and strings. All this logic in C++ is hard to read and often has portability issues. Moreover, many PRs here these days are focused on even more system code development for the Windows port.

I understand some may argue that having the entire MIOpen source in C++ is a reasonable unification. But in fact developing and reviewing a growing volume of non-compute logic handled with C++ only makes our life harder. C++17, boost and platform APIs are not the right tools for housekeeping, e.g. filesystem. They might work very well, but are marginal and consume more engineering resources than they deserve.

Having them in any language other than C++ will not have any sensible performance penalty.

Why not to offload all compute-unrelated code portions from C++ to Python?

Example:

// Embedded Python interpreter in C++ source.
#include <pybind11/embed.h>

namespace py = pybind11;
using namespace py::literals;
...
py::scoped_interpreter guard;
...

// Execute some general-purpose logic within an
// embedded Python interpreter, providing it with the
// input data from the C++ context.
// Here, it can be any database or system-wide logic,
// working with files and env variables
// in a CROSS-PLATFORM way without any extra effort.
auto locals = py::dict("filename"_a=filename);
py::exec(R"(
        # Note: the interpreter also has all power of Python modules.
	import numpy as np
	is_numpy_matrix = True
	try:
		matrix = np.load(filename, allow_pickle=True)
	except:
		is_numpy_matrix = False
	
	if is_numpy_matrix:
		if matrix.ndim > 2:
			is_numpy_matrix = False
		else:
			if matrix.ndim == 1:
				nrows = 1
				ncols = matrix.shape[0]
			else:
				nrows, ncols = matrix.shape
)", py::globals(), locals);

// Return the script results back to C++ program.
auto is_numpy_matrix = locals["is_numpy_matrix"].cast<bool>();
if (!is_numpy_matrix) return -1;

if (nrows) *nrows = locals["nrows"].cast<int>();
if (ncols) *ncols = locals["ncols"].cast<int>();

Resume: do whatever system dirty coding in Python for all platforms, and then return to the clean C++ world for only thing it is really important in: compute. Isn't it an efficient compromise?

This is easier and much more accessible to the community, as the Python scripting can be reviewed by x100 more people than those who know how to use Boost.

[CAHEK7]

It's almost perfect unless mixing and interoperating python and C++ are come into place. And it's x10000 less people who know how to do it properly.
They must know:
1 python
1.5 all the python libraries and compatibility with embedding
2 C++
3 interop
4 how to constantly switch between them

[CAHEK7]

BTW, when you are using std library or the most of boost library it produces fairly good CROSS-PLATFORM code and most of the windows related changes were not related to the libraries at all. So, it looks like we want to solve the problem which does not even exist.

[dmikushin]

I understand Boost is something many of us wish to enjoy, but C++ it is not robust enough for dealing with system management and tooling. Look at clang or gcc frontend or any classic code that does process management, piping, command line construction etc from C/C++ -- it's always an unreadable dead beef altogether.

I wouldn't say this myself few years ago, and would fully share your point of view. But pybind11's approach to embedding is totally zero-effort. I've recently learned that it is not only good for making bindings, but its embedded interpreter can save thousands of hours spent on re-implementing obvious things in C++, just because we like it.

Opinions are made by experience. Mine is based on using both C++ and pybind11/Python. It's difficult to assess any idea without trying it for anything real. Or in other words it's easy to reject something that I barely experienced.

[pfultz2]

Providing a python API(which is quite easy to do with pybind) could help replace some of the driver usages as developers could have more interactive environment.

However, rewriting core parts of miopen in python has many problems:

1. miopen would require a python runtime which could be a big dependency for some usecases of miopen(like for OS's gpu driver, etc).
2. It would prevent us from supporting a manylinux version since distros do not have the same python version or same ABI across other distros.
3. Lack of type safety would introduce more bugs. More test will be needed to compensate for this and we already need more testing for the current code, so this will move the testing in the wrong direction.

[dmikushin]

Hi @pfultz2 ,

2. Nono, we definitely leave GPU stuff to C++ for sure. Therefore, dependency on drivers is out of consideration. But I think we can safely rely on core language features such as os, sys, re and on more or less standardized things, such as numpy, scipy. Imagine for example that any MIOpen tensor/tensorOp can be interoperable with NumPy - this is a golden mine for testing and prototyping. Of course NOT for performance, but utility, testing and community code can blossom. To this extent, it should not require dealing with manylinux.

3. There is no fundamental flaw in Python, but of course linting of embedded code must be taken seriously, and the available linters are not bad at all.

Aside of Python, the idea of embedding a lightweight language into a big software is not something new or crazy that only I propose. Lua or lisp have been quite successful in this role too. But looking at the PRs and test suites that we need to support, I think Python embedded interpreter is the best choice.

[CAHEK7]

I guess the best way to make it interesting is to provide few solid examples where it can be useful, simplify everything, reduce amount of boilerplate code and keep all the interactions between C and Python clear.
Few snippets or even files is a good start. (probably in a PR)

Current snippet does not make much sense and the most interesting part of interpreting numpy array is missing.

[dmikushin]

I may miss something, but how does numpy array help with comgr?

I show an intro example that I have at hand. I believe comgr only exists because it has to be compatible with C++ and be made as a solid native library. If it is scripted in an embedded interpreter, it could be basically a function that awaits a process invocation on a couple of temporary files.

[DrizztDoUrden]

Please, correct me if I am wrong, but the entire point of using comgr is not having temporary files at all. It is not "just" a "compatible with C++" "solid native library", but a very specific tool with a very specific purpose that would be extremely difficult to reimplement in python. I am not very well versed in python, but I don't think it is well designed to invoke LLVM internals out of the box.

[dmikushin]

Please, correct me if I am wrong

@DrizztDoUrden , MIOpen uses comgr for runtime compilation only. In this scenario, its only theoretical benefit is to get a binary in memory without touching the filesystem by creating temporary files (which is known to be untrue, because comgr still creates temp files for something else). Anyway, I meant to say in this particular scenario the use of comgr in MIOpen could be replaced by an embedded script quite easily.

[atamazov]

COMgr and HIPRTC are used because offline compilers are not included into normal ROCm packages. Our users do not want to install -dev packages to be able to use MIOpen.

[CAHEK7]

Am I correct that invocation of llvm internals and just-in-time compilation is much easier to implement on python than simply invoke a few functions through special llvm compilation interface?
COMGR creates files just to make debugger happy. For sure the files with embedded debugging information can be easily created using python, we just need to import llvm bindings module, stop, what?

I still don't see any will to provide any examples (I mean code snippets, explanations and so on). Just saying that you know a cool technology doesn't mean anything. Please share your knowledge and experience and show how we can use this cool technology in this particular project or in any other project. You've already spent a time and effort, did some research and instead of blindly throwing the idea to the crowd you can make some good examples to show pros, cons, pitfall of that cool technology.

1. How do we deal with mixing different syntax in one file (formatting, syntax highlighting)?
2. How do we deal with stabilizing interfaces between different languages, especially when python heavily relies on linters and linters do not know about C++?
3. How do we deal with semantic synchronization if Python source will be in another file - since there is almost no interfaces and no types on the boundary.
4. How do we deal with debugging of mixed code? How do we deal with python dependency and with all the potential python modules in the system?
5. How do we deal with memory management and pointers validity (because // The data is resident thorough the lifetime of embedded interpreter. sounds scary - we can get either indirect memory leaks or sudden pointers invalidation).
6. How do we deal with type compatibility for the python list? For the numpy array? (what is the default calculation type in python for integers? for floats? can it trigger sudden type conversion and we can write twice bigger data chunk to disk).
7. What is memory consumption on data transfer between two languages (especially if implicit type conversion happened).