[not ready] Switch to C++14 standard

[daniel-j-h]

See if Travis gives okay for Project-OSRM/osrm-backend#1975

[daniel-j-h]

Looks like some builds are failing https://travis-ci.org/Project-OSRM/node-osrm/builds/111112017 but I can not see why from staring at the npm log that I get on Travis.

[daniel-j-h]

Now that the lto binutils wrappers work, ccache does not pick them up. Nice.

Pushing off a new build with properly using update-alternatives. we're using sudo: false ..

[daniel-j-h]

This got out of sync. with osrm-backend for some reason. Re-triggering to see if it helps.

[daniel-j-h]

Travis still not pulling in the osrm-backend changes, especially not this line: https://github.com/Project-OSRM/osrm-backend/pull/1975/files#diff-af3b638bc2a3e6c650974192a53c7291R32.

[TheMarex]

You need to also update OSRM_RELEASE in .travis.yml. This is only used for local builds!

[TheMarex]

@daniel-j-h seems like we are hitting internal compiler errors? oO

https://travis-ci.org/Project-OSRM/node-osrm/jobs/113076067#L789

[daniel-j-h]

Yep. No idea what to think of it. What strikes me is that we're using GCC 5.2 when 5.3 is the stable release.

But GCC 5.3 is only packaged for 14.04 in the ubuntu-toolchain-r ppa.

[daniel-j-h]

Still blocked by ^ @springmeyer is it possible to package not only direct dependencies but also compiler and stdlib (e.g. gcc 5.3 or even gcc 6 already as it is in it's last weeks before release) with mason and then use it on Travis?

[springmeyer]

@daniel-j-h yes, it should be possible but would be a big lift. I think we should get to the bottom of why the existing g++-5 build is being killed. My hunch is that it is running out of memory and the OOM is killing the compiler. We default to 3 parallel jobs which may be too much. So I've pushed a testing branch to see if running with 1 JOB can dodge the issue.

[springmeyer]

In testing I found that:

• The killed problem was resolved by building with JOBS=1. So, this does not require upgrading g++-5 from 5.2->5.3
• Next problem was linking because the gcc 5.2 ar/ranlib are called gcc-ar-5 so the hardcoding of gcc-ar breaks (but is fixable by removing it like Project-OSRM/osrm-backend@c++14...c++14-unhardcode-linking)
• Next problem is that -flto is broken in g++-5 (not a big surprise, it seems it always broken, or constantly regresses in my experience) leading to symbol errors due to Storage::Storage() and Storage::Run() being undefined in libosrm_store.a
• Locally on linux I tested that removing flto from the osrm-backend build fixes things
• But another problem exists: g++-5 binaries are incompatible with the clang++3.5 built binaries pulled from mason for the boost deps. This leads to obscure symbol errors when linking boost_filesystem. I'm currently working on creating a mason package for boost that is compiled with g++-5 to workaround this.

Note: all of the above issues are with osrm-backend with g++5.2 + mason deps. Nothing yet seems to be unique to c++14. Not yet tried to get node-osrm running at all.

[springmeyer]

Noting that this linking error is the only thing now breaking with the osrm-backend compile with g++ 5.2 and -flto. It is fixable by not using -flto, or by upgrading to g++ 5.3:

Linking CXX executable osrm-datastore
/home/travis/build/Project-OSRM/node-osrm/mason_packages/linux-x86_64/cmake/3.2.2/bin/cmake -E cmake_link_script CMakeFiles/osrm-datastore.dir/link.txt --verbose=1
/usr/bin/g++-5    -flto=32 -Wall -Wextra -pedantic -Wuninitialized -Wunreachable-code -Wstrict-overflow=1 -D_FORTIFY_SOURCE=2 -fdiagnostics-color=auto -fPIC -ffunction-sections -fdata-sections -std=c++14  -fopenmp -O3 -DNDEBUG   -Wl,-z,origin -Wl,-rpath=\$ORIGIN  -Wl,--gc-sections -Wl,-O1 -Wl,--hash-style=gnu -Wl,--sort-common CMakeFiles/osrm-datastore.dir/src/tools/store.cpp.o CMakeFiles/UTIL.dir/src/util/assert.cpp.o CMakeFiles/UTIL.dir/src/util/coordinate.cpp.o CMakeFiles/UTIL.dir/src/util/coordinate_calculation.cpp.o CMakeFiles/UTIL.dir/src/util/exception.cpp.o CMakeFiles/UTIL.dir/src/util/fingerprint.cpp.o CMakeFiles/UTIL.dir/src/util/hilbert_value.cpp.o CMakeFiles/UTIL.dir/src/util/simple_logger.cpp.o  -o osrm-datastore -rdynamic libosrm_store.a -Wl,-Bstatic -lboost_date_time -lboost_filesystem -lboost_iostreams -lboost_program_options -lboost_regex -lboost_system -lboost_thread -lboost_unit_test_framework -Wl,-Bdynamic -lpthread -ltbb -ltbbmalloc -lrt 
/tmp/cc5xQGlT.ltrans10.ltrans.o: In function `main':
<artificial>:(.text.startup.main+0x80): undefined reference to `osrm::storage::Storage::Storage(std::unordered_map<std::string, boost::filesystem::path, std::hash<std::string>, std::equal_to<std::string>, std::allocator<std::pair<std::string const, boost::filesystem::path> > > const&)'
<artificial>:(.text.startup.main+0x88): undefined reference to `osrm::storage::Storage::Run()'
collect2: error: ld returned 1 exit status

[daniel-j-h]

Next problem was linking because the gcc 5.2 ar/ranlib are called gcc-ar-5 so the hardcoding of gcc-ar breaks (but is fixable by removing it like Project-OSRM/osrm-backend@c++14...c++14-unhardcode-linking)

Hm yes that's why I had to export them here: 34ea54a if we remove it from CMake users on gcc 4.9+ also have to export nm/ar/ranlib with the gcc-lto-wrappers.

Next problem is that -flto is broken in g++-5 (not a big surprise, it seems it always broken, or constantly regresses in my experience) leading to symbol errors due to Storage::Storage() and Storage::Run() being undefined in libosrm_store.a

That's a surprise to me, since at least @TheMarex builds it all the time locally with gcc 5.2+ (?) and my local Nix setup also used gcc 5.3. /edit: just read your second comment: ha, so gcc 5.2 is broken in that regard.

Upgrding to gcc 5.3 is not possible as the ppa only packages it for Trusty (14.04) but node-osrm's Travis setup does not run on Trusty.

Locally on linux I tested that removing flto from the osrm-backend build fixes things

But then we don't get lto for our builds :) Good to know it's not C++14 causing these issues, though.

[springmeyer]

per chat with @daniel-j-h - this linking error only impacts osrm-datastore so we should be good to go using gcc 5.2 if we disable -flto for just osrm-datastore.

But... before giving up I'll note a few more things from investigations.

I've found that clang-3.7 also hits the same problem:

/usr/bin/clang++-3.7   -flto  -flto -Wall -Wextra -pedantic -Wuninitialized -Wunreachable-code -Wstrict-overflow=2 -D_FORTIFY_SOURCE=2 -fPIC -ffunction-sections -fdata-sections -std=c++14  -O3 -DNDEBUG   -Wl,-z,origin -Wl,-rpath=\$ORIGIN  -Wl,--gc-sections -Wl,-O1 -Wl,--hash-style=gnu -Wl,--sort-common CMakeFiles/osrm-datastore.dir/src/tools/store.cpp.o CMakeFiles/UTIL.dir/src/util/exception.cpp.o CMakeFiles/UTIL.dir/src/util/simple_logger.cpp.o CMakeFiles/UTIL.dir/src/util/coordinate_calculation.cpp.o CMakeFiles/UTIL.dir/src/util/coordinate.cpp.o CMakeFiles/UTIL.dir/src/util/assert.cpp.o CMakeFiles/UTIL.dir/src/util/hilbert_value.cpp.o CMakeFiles/UTIL.dir/src/util/fingerprint.cpp.o  -o osrm-datastore -rdynamic libosrm_store.a -Wl,-Bstatic -lboost_date_time -lboost_filesystem -lboost_iostreams -lboost_program_options -lboost_regex -lboost_system -lboost_thread -lboost_unit_test_framework -Wl,-Bdynamic -lpthread -ltbb -ltbbmalloc -lrt 
/usr/bin/ld: error: libosrm_store.a: no archive symbol table (run ranlib)
^[[B/tmp/lto-llvm-1ebb8c.o:ld-temp.o:function main: error: undefined reference to 'osrm::storage::Storage::Storage(std::unordered_map<std::string, boost::filesystem::path, std::hash<std::string>, std::equal_to<std::string>, std::allocator<std::pair<std::string const, boost::filesystem::path> > > const&)'
/tmp/lto-llvm-1ebb8c.o:ld-temp.o:function main: error: undefined reference to 'osrm::storage::Storage::Run()'
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[3]: *** [osrm-datastore] Error 1

However, this is with clang++-3.7 and the normal AR, RANLIB, NM settings. I've found that this can be fixed by setting RANLIB=/usr/bin/llvm-ranlib-3.7. So, that makes me think that the bug is not with g++ 5.2 but rather gcc-ranlib-5 or the linker.

So, next I tried a full build with clang++-3.7 + the llvm tools for AR and RANLIB. This bailed with:

/usr/bin/ld: error: /usr/bin/../lib/gcc/x86_64-linux-gnu/5.2.1/libstdc++.so: symbol 'std::__once_callable' used as both __thread and non-__thread
/usr/bin/ld: CMakeFiles/osrm-routed.dir/src/tools/routed.cpp.o: previous definition here
/usr/bin/ld: error: /usr/bin/../lib/gcc/x86_64-linux-gnu/5.2.1/libstdc++.so: symbol 'std::__once_call' used as both __thread and non-__thread
/usr/bin/ld: CMakeFiles/osrm-routed.dir/src/tools/routed.cpp.o: previous definition here
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Which looks like a gold linker bug (https://sourceware.org/bugzilla/show_bug.cgi?id=14342).

Noting for future reference that I'm hitting this with GNU gold on ubuntu precise (apt-get install binutils-gold):

$ /usr/bin/ld -v
GNU gold (GNU Binutils for Ubuntu 2.22) 1.11

And the normal linker on ubuntu precise is:

$ /usr/bin/ld.bfd -v
GNU ld (GNU Binutils for Ubuntu) 2.22

Next action: try upgrading ld-gold to see if clang-3.7 + ld-gold + -flto gives us working builds.

[springmeyer]

Next action: try upgrading ld-gold to see if clang-3.7 + ld-gold + -flto gives us working builds.

\o/ Success. Combining clang-3.7 (via apt install) with an upgraded binutils (via mason) allows all of osrm-backend develop to compile cleanly with -flto -std=c++14 on ubuntu precise/sudo:false machines: https://travis-ci.org/Project-OSRM/node-osrm/jobs/114673231.

[daniel-j-h]

Wow, great results @springmeyer! Interesting how binutils comes into play here! Nice 🎉

[springmeyer]

My next actions here:

• Test these clang++-3.7 binaries to ensure:
  • They are portable to vanilla trusty/precise systems
  • Their performance is the same or better to the previous g++-4.8 binaries
• Also test with clang++3.8 (waiting on Add llvm 3.8 repo to make available clang-3.8 travis-ci/apt-source-safelist#255) not going to wait
• ~~Figure out if there is a way to upgrade binutils to be used by the gcc lto-wrapper?~~~ Let's just use clang++ instead of gcc so as not to worry about this.

Assuming the above steps go well, then:

• Provide pull to osrm-backend CMakeLists.txt + travis to test LTO/C++14 on both gcc and clang with sudo:false machines

[TheMarex]

Please close and reopen on master since develop is going away.

[springmeyer]

Please close and reopen on master since develop is going away.

Now at #227