Intel oneAPI DPC++/C++ Compiler support

[jakeheke75]

Hello Antony,

This is a draft PR to integrate the build process with the Intel oneAPI DPC++/C++ Compiler #58 #218.
Initially in my mind I had the following integration process:

• integrate the build process in Windows
• integrate the build process in Linux
• integrate the Github workflow for both of them

No joy. I started with the first step and immediately had to fight with compilation issues. Below the steps done (starting from a clean repository situation):

• Install the Intel oneAPI Base Toolkit with Visual Studio support
• Manual update of settings.yml file in order to add versions of the intel-cc compiler

    intel-cc:
        version: ["2021.1", "2021.2", "2021.3", "2021.4", "2022.0", "2022.1", "2022.2", "2022.3", "2023.0", "2023.1", "2023.2" ]

• Launch the Intel oneAPI command prompt for Intel 64 for Visual Studio and execute the following commands:
• python -m venv .venv
• .venv\Scripts\activate.bat
• pip install -r ./requirements.txt
• conan profile detect --force
• conan config install ./.conan
• conan install ./ -pr:h .conan2/profiles/intel/2023.2/x64-libstdc++-release -pr:b .conan2/profiles/intel/2023.2/x64-libstdc++-release --build missing -c tools.cmake.cmaketoolchain:generator="Ninja Multi-Config"

Immediately getting error message about package ms-gsl/4.0.0:

ERROR: ms-gsl/4.0.0: Error in validate() method, line 70
        self.output.warn(f"{self.ref} requires C++{self._minimum_cpp_standard}. "
        AttributeError: 'ConanOutput' object has no attribute 'warn'

the Intel oneAPI compiler is not available for the package, here is a code snippet of the Conan recipe where it is triggering the error:

    def _compilers_minimum_version(self):
        return {
            "gcc": "5",
            "clang": "3.4",
            "apple-clang": "3.4",
        }

Only for testing purposes I by-passed this error by excluding the ms-gls package from the requirements in conanfile.py.
Then we start to get errors in the compile process because it seems that the required dependencies are not supporting the compiler.
This is what happens for package glbinding/3.1.0 (6 of 30 to be installed) where we get the compilation error "Unsupported compiler":

In file included from C:\Morpheus\.conan2\p\b\glbinbf829c08ada13\b\src\source\glbinding\source\AbstractState.cpp:2:
In file included from C:\Morpheus\.conan2\p\b\glbinbf829c08ada13\b\src\source\glbinding\include\glbinding/AbstractState.h:8:
In file included from C:\Morpheus\.conan2\p\b\glbinbf829c08ada13\b\src\source\glbinding\include\glbinding/CallbackMask.h:6:
C:\Morpheus\.conan2\p\b\glbinbf829c08ada13\b\build\source\glbinding\include\glbinding/glbinding_features.h(361,6): error: Unsupported compiler
#    error Unsupported compiler

If we investigate the source file glbinding_features.h it seems a compiler macro issue; in fact in my opinion the Intel oneAPI DPC++/C++ Compiler should be supported because the compiler macro signature __INTEL_LLVM_COMPILER is there and should trigger the instruction define GLBINDING_COMPILER_IS_IntelLLVM 1:

#elif defined(__INTEL_COMPILER) || defined(__ICC)
# undef GLBINDING_COMPILER_IS_Intel
# define GLBINDING_COMPILER_IS_Intel 1

#elif (defined(__clang__) && defined(__INTEL_CLANG_COMPILER)) || defined(__INTEL_LLVM_COMPILER)
# undef GLBINDING_COMPILER_IS_IntelLLVM
# define GLBINDING_COMPILER_IS_IntelLLVM 1

who knows where is the catch here... The execution then jumps at row 361 and the error message is triggered:

#  else
#    error Unsupported compiler
#  endif

So, not sure what we have to do here. I have the feeling that most of the Conan packages are mainly supporting the "big three" compilers and that the support for the Intel compiler is currently more a matter of luck than of a real intention 😄
Which is your opinion? How do we have to approach this?

Thanks.

Cheers
Gianmario

[Twon]

Hey Gianmario,

Thanks for this PR, this looks like a great start to adding Intel compiler support! I've had a little play around with this locally and was expecting there to be numerous issues in downstream repositories. It may take some time and effort but ideally, we'd look to address these issues by raising pull requests addressing the issues and be good citizens of the open-source world :)

Hopefully, we could spearhead Morpheus into a leading example of using the Intel Compiler over time. I've previously used this compiler (an older version) a lot for generating high-performance computing libraries using vector instruction sets, and have plans for using this approach for things like physics, collision detection, matrix computation when we start getting into some of the interesting work with this project.

For now, let's try to keep track of the issues we discover and need to address as we go.

Dependency issues for Intel compiler support:

• ms-gsl: MS Guideline Support Library
• glbinding: C++ binding for the OpenGL API

Also, the next issue we'd need to address for this is the issue of CI support for github actions. When I installed the Intel Compiler in my Ubuntu WSL on Windows it took about 45 minutes to install so is not practical to run on a CI every time. However, I have some ideas here around building containerises build environments which we can use for github action so that we could provide containers with all of the required build tools. In fact, I've gone as far as creating a project for this, aimed at being a Python tool which uses Jinja2 templates to generate dockerfile which then build into containers: https://github.com/Twon/Bebe. However, I've not made progress on this yet, but just a heads up on intended directions for how to solve this. Of course, let me know if you have any other ideas.

Does that all sound reasonable?

Btw, sorry for the delay coming back to you, last week in Hawaii at the ISO C++ meeting was intense, and I'm still recovering from jet lag this week and working towards a deadline of November 21st to take our paper to electronic polling to be sent to LWG. Hopefully, be back on track next week!

Cheers,
Antony

[jakeheke75]

Hey Gianmario,

Thanks for this PR, this looks like a great start to adding Intel compiler support! I've had a little play around with this locally and was expecting there to be numerous issues in downstream repositories. It may take some time and effort but ideally, we'd look to address these issues by raising pull requests addressing the issues and be good citizens of the open-source world :)

Totally agree. I will start to do it then. I have some newbie questions about how to proceed 😄

Hopefully, we could spearhead Morpheus into a leading example of using the Intel Compiler over time. I've previously used this compiler (an older version) a lot for generating high-performance computing libraries using vector instruction sets, and have plans for using this approach for things like physics, collision detection, matrix computation when we start getting into some of the interesting work with this project.

I am not an expert at all but it seems a very good tool. Since it is probably specifically tailored for Intel CPUs I believe there is also a potential to use it extensively for low latency applications. It doesn't seem a very popular compiler tho! Maybe it's just a relatively new thing and human nature is to naturally for consolidated habits. Anyway, sounds like a very interesting roadmap and a big rabbit hole to go down. It will be a lot of fun.

For now, let's try to keep track of the issues we discover and need to address as we go.

Dependency issues for Intel compiler support:

* [ ]  ms-gsl: MS Guideline Support Library

Let's talk about how practically approach those two dependencies. About ms-gsl I believe that I can sort it out if I debug the Conan recipe. Looks like it is possible to do it.
Once that I am able to debug it I will try to fix it locally in my .conan2 directory. If it works I will then submit a PR to Conancenter recipe. Do you agree with this approach or you would do it differently?

* [ ]  glbinding: C++ binding for the OpenGL API

About this dependency how would you approach the fix? Because I believe it is not possible to debug the build process. I think that I have to fork the repository locally and try to build it with Intel compiler until it works. Or maybe it is quicker to manually modify the header file in local .conan2 build directory and see if it works? Which is in your opinion the most effective approach?

Also, the next issue we'd need to address for this is the issue of CI support for github actions. When I installed the Intel Compiler in my Ubuntu WSL on Windows it took about 45 minutes to install so is not practical to run on a CI every time. However, I have some ideas here around building containerises build environments which we can use for github action so that we could provide containers with all of the required build tools. In fact, I've gone as far as creating a project for this, aimed at being a Python tool which uses Jinja2 templates to generate dockerfile which then build into containers: https://github.com/Twon/Bebe. However, I've not made progress on this yet, but just a heads up on intended directions for how to solve this. Of course, let me know if you have any other ideas.

I totally agree that 45 minutes is not an acceptable time for a CI job. Once we resolved all the dependencies and we are able to build it successfully I will try to find if there is some custom tool/action which can help to overcome this issue.
Nice idea about containerises build environments! I will have a look at your project also.

Btw, sorry for the delay coming back to you, last week in Hawaii at the ISO C++ meeting was intense, and I'm still recovering from jet lag this week and working towards a deadline of November 21st to take our paper to electronic polling to be sent to LWG. Hopefully, be back on track next week!

No worries, thank you for all the suggestions.

[jakeheke75]

Update:
ms-gsl is now working.
for glbinding I submitted a draft PR, still no feedback so far.

[Twon]

Update: ms-gsl is now working. for glbinding I submitted a draft PR, still no feedback so far.

Thanks for the update. This project was also affected when WriteCompilerDetectionHeader was deprecated, shame was a useful utility. The ticket to deprecate it mentioned Hedley as a simple drop-in replacement. But to some extent will depend on what the glbinding author's preferred approach is.

[Twon]

Looks like Conan might require extending in places to support the Intel compiler:
https://github.com/conan-io/conan/blob/22ca7546d2a8cf4c1ca5d8e7b5f379d2fb2596ad/conan/tools/build/cppstd.py#L111
https://github.com/conan-io/conan/blob/96e1f8139064684706ed05fae4079dfe69c0ba25/conan/internal/api/detect_api.py#L205

[jakeheke75]

Looks like Conan might require extending in places to support the Intel compiler: https://github.com/conan-io/conan/blob/22ca7546d2a8cf4c1ca5d8e7b5f379d2fb2596ad/conan/tools/build/cppstd.py#L111 https://github.com/conan-io/conan/blob/96e1f8139064684706ed05fae4079dfe69c0ba25/conan/internal/api/detect_api.py#L205

Right, I tried to make a proposal, let's see what they think.

[jakeheke75]

Update: I opened a couple of issues in Conan repository

• [feature] Improve detect API to support Intel oneAPI compiler #15344
  Pull request Intel oneapi detection #15358
• [feature] Improve build tools to support Intel oneAPI compiler #15345