Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add float32 array EclSum.to_numpy #820

Closed
wants to merge 2 commits into from
Closed

Add float32 array EclSum.to_numpy #820

wants to merge 2 commits into from

Conversation

pinkwah
Copy link
Collaborator

@pinkwah pinkwah commented Jul 12, 2021

This PR is a massive experiment wherein I add a new Python native extension ecl._native written in C++ using pybind11.

I'd like us to use something along these lines for future C++ Pythonic code. My goal also is to have a unified API for Python and C++. That is, my hopes are:

  1. C++ public API and Python API should expose the same classes and class functions, where it makes sense. (ie, no numpy code in C++, no pointers in Python)
  2. The C code is either generated or doesn't even exist.

It should be possible to create pybind11 converters between libecl types such as stringlist_type, std::vector<std::string> and List[str], so that it's easier to avoid using cwrap's Prototypes in favour of simply specifying the function in pybind.

Eg, instead of

    # Python
    _alloc_rectangular = EclPrototype(
        "ecl_grid_obj ecl_grid_alloc_rectangular(int, int, int, double, double, double, int*)",
        bind=False,
    )

we can do this in C++ directly:

    py::class_<Grid>(m, "Grid")
        .def("_alloc_rectangular", &ecl_grid_alloc_rectangular);

The difficulty here is that pybind's C++ classes specifically don't expose Python functionality like self. That is, they're mostly C++ class wrappers. This is fine if we can convert all the cwrap functionality out of the Python classes.

In order to accomplish these things I've bumped the C++ standards version to 17 (up from 11) and CMake to 3.12 (up from 2.8). Will investigate whether we can lower the C++ standard to 14, although I don't particularly want this.

TODO:

  • ecl_sum_init_double_vector
  • ecl_sum_init_double_vector_interp
  • ecl_sum_init_double_frame: Pandas DataFrame version
  • ecl_sum_init_double_frame_interp: Pandas DataFrame version
  • Investigate whether C++ 14 is feasible (or whether we want to keep C++ 17)

Resolves: #797

@pinkwah pinkwah self-assigned this Jul 13, 2021
smspec_node instances. The actual smspec_node instances are
owned by the smspec_nodes vector;
*/
node_map field_var_index;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is now moving to the header file, perhaps its a good time to introduce doxygen?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good idea 👍

#include <ert/ecl/ecl_smspec.hpp>
#include <ert/ecl/ecl_sum_tstep.hpp>
#include <ert/ecl/ecl_file.hpp>
typedef struct ecl_file_struct ecl_file_type;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe change the naming convention to

namespace ecl {
    typedef struct fortio_struct fortio;
}

But keep the fortio_type for c code? Possibly use fortio_struct for c code?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would probably do something like:

#ifdef __cplusplus
namespace ecl { class file; }
using ecl_file_type = ecl::file;
#else
typedef struct ecl_file_struct ecl_file_type;
#endif

Ie, change the API to use C++ classes with namespaces, while aliasing them to the old names in C++ only. Should not give us any ABI breakage.

This commit introduces pybind11 in order to accomplish the goal of
dumping float32 arrays. This could've been done by adding yet another C
function accessible from Python via cwrap. However, I made the decision
to have the Python API not depend on the C API directly, instead adding
Python bindings via pybind11. This allows us to extend the Python API
without exposing implementation details in C and avoids having the
Python API do manual memory management to deal with C.
@pinkwah pinkwah closed this Feb 15, 2022
@pinkwah pinkwah deleted the pybind11 branch February 15, 2022 08:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Option to load UNSMRY data as 32 bit floats
2 participants