Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/sync with latest upstream(also fix mac arm fail) #2

Open
wants to merge 13 commits into
base: master
Choose a base branch
from

Conversation

yokofly
Copy link
Collaborator

@yokofly yokofly commented Oct 9, 2023

This pr contains the latest commit: daf4c6b
I have run ci with the specific commit: a781bd8...daf4c6b

ci status: https://github.com/timeplus-io/proton/actions/runs/6451163534

gengjiawen and others added 9 commits December 8, 2022 17:34
Add aarch64 to the list of arm64 devices.

Fixes an issue on linux where arm64 is not defined.

Previously, arm64 was only defined as arm64, which only matches modern
macOS on apple silicon.

As it turns out, there other arm64 devices besides apple silicon, and
being able to compile against them is actually kind of nice. This PR
adds detection for aarch64, which is also considered arm64 and allows
V8 to compile and run.

Fixes bnoordhuis#70.
    1. Additional Compile Option: Introduced extra compile options.

    2. Explicit Path Usage: Modified add_custom_command to be invoked with an explicit path.

    3. Type Conversion Correction: Addressed a conversion between int64_t and double.

    4. Obsolete File Removal: Removed an unnecessary, manually generated file that was previously used.

Historically, I had disabled the file generation process because it was time-consuming. However, to prioritize safety and flexibility, I’ve decided to re-enable it. Consequently, each time V8 is built, a new file will be generated and utilized. Previously, I manually added the file, circumventing the need for generation during each build.
@yokofly
Copy link
Collaborator Author

yokofly commented Oct 9, 2023

cmake --build .
[0/2] Re-checking globbed directories...
[1177/5155] Building CXX object contrib/v8-cmake/CMakeFiles/v8_base_without_compiler.dir/v8/src/trap-handler/handler-inside-posix.cc.o
FAILED: contrib/v8-cmake/CMakeFiles/v8_base_without_compiler.dir/v8/src/trap-handler/handler-inside-posix.cc.o 
/opt/homebrew/bin/ccache /opt/homebrew/opt/llvm/bin/clang++ -DHAS_RESERVED_IDENTIFIER -DV8_CONCURRENT_MARKING -DV8_HAVE_TARGET_OS -DV8_TARGET_ARCH_ARM64 -DV8_TARGET_OS_MACOS -D_LIBCPP_ENABLE_THREAD_SAFETY_ANNOTATIONS -I/Users/ec2-user/proton/contrib/v8-cmake/generated -I/Users/ec2-user/proton/contrib/v8-cmake/v8/include -I/Users/ec2-user/proton/contrib/v8-cmake/v8 -I/Users/ec2-user/proton/contrib/v8-cmake/v8/third_party/zlib -I/Users/ec2-user/proton/build/contrib/v8-cmake -isystem /Users/ec2-user/proton/contrib/llvm-project/libcxx/include -isystem /Users/ec2-user/proton/contrib/llvm-project/libcxxabi/include -fdiagnostics-color=always -Xclang -fuse-ctor-homing -Wno-enum-constexpr-conversion -fsized-deallocation  -gdwarf-aranges -pipe -march=armv8-a+crc -fasynchronous-unwind-tables -ffile-prefix-map=/Users/ec2-user/proton=. -falign-functions=32   -Wall -Wno-unused-command-line-argument  -stdlib=libc++ -fdiagnostics-absolute-paths -fstrict-vtable-pointers -w -O2 -g -DNDEBUG -O3 -g -gdwarf-4  -std=gnu++2b -arch arm64 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX13.3.sdk -mmacosx-version-min=10.15   -D OS_DARWIN -DV8_ENABLE_WEBASSEMBLY -DV8_ADVANCED_BIGINT_ALGORITHMS -fno-exceptions -Wno-undef -Wno-unused-parameter -Wno-covered-switch-default -Wno-poison-system-directories -Wno-gnu-statement-expression -Wno-comma -Wno-format-nonliteral -Wno-format-pedantic -Wno-missing-noreturn -Wno-conditional-uninitialized -Wno-extra-semi-stmt -Wno-non-virtual-dtor -Wno-shadow-field-in-constructor -Wno-documentation -Wno-reserved-identifier -Wno-newline-eof -Wno-undefined-reinterpret-cast -Wno-error -Wno-implicit-fallthrough -Wno-unused-function -Wno-sign-compare -Wno-deprecated-declarations -Wno-deprecated-pragma -Wno-redundant-parens -Wno-deprecated-anon-enum-enum-conversion -Wno-zero-as-null-pointer-constant -Wno-suggest-override -Wno-dtor-name -Wno-suggest-destructor-override -Wno-inconsistent-missing-destructor-override -Wno-frame-larger-than -Wno-unused-macros -Wno-deprecated-volatile -Wno-cast-function-type -Wno-deprecated-copy-with-dtor -Wno-deprecated-this-capture -Wno-unused-member-function -Wno-shadow -Wno-gnu-anonymous-struct -Wno-shadow-field -Wno-cast-qual -Wno-used-but-marked-unused -Wno-gnu-zero-variadic-macro-arguments -Wno-ambiguous-reversed-operator -Wno-deprecated-copy-with-user-provided-dtor -Wno-shadow-uncaptured-local -Wno-duplicate-enum -Wno-old-style-cast -Werror -nostdinc++ -MD -MT contrib/v8-cmake/CMakeFiles/v8_base_without_compiler.dir/v8/src/trap-handler/handler-inside-posix.cc.o -MF contrib/v8-cmake/CMakeFiles/v8_base_without_compiler.dir/v8/src/trap-handler/handler-inside-posix.cc.o.d -o contrib/v8-cmake/CMakeFiles/v8_base_without_compiler.dir/v8/src/trap-handler/handler-inside-posix.cc.o -c /Users/ec2-user/proton/contrib/v8-cmake/v8/src/trap-handler/handler-inside-posix.cc
/Users/ec2-user/proton/contrib/v8-cmake/v8/src/trap-handler/handler-inside-posix.cc:128:40: error: member reference type 'struct __darwin_mcontext64 *' is a pointer; did you mean to use '->'?
    auto* context_ip = &uc->uc_mcontext.pc;
                        ~~~~~~~~~~~~~~~^
                                       ->
/Users/ec2-user/proton/contrib/v8-cmake/v8/src/trap-handler/handler-inside-posix.cc:128:41: error: no member named 'pc' in '__darwin_mcontext64'
    auto* context_ip = &uc->uc_mcontext.pc;
                        ~~~~~~~~~~~~~~~ ^
2 errors generated.
[1186/5155] Building CXX object base/poco/Data/CMakeFiles/_poco_data.dir/src/StatementImpl.cpp.o
ninja: build stopped: subcommand failed.

* Integrated v8 into our core engine (proton), necessitating cross-compilation support for Proton.

* During the v8 build process, intermediate binaries must run on the host. To simplify this, we utilize pre-built binaries, streamlining the file generation process for both host and target with their distinct requirements. In the future, we can consider replacing it with ClickHouse/ClickHouse@b766552
```
FAILED: contrib/v8-cmake/CMakeFiles/v8_base_without_compiler.dir/v8/third_party/zlib/zutil.c.o
/opt/homebrew/bin/ccache /opt/homebrew/opt/llvm/bin/clang -DHAS_RESERVED_IDENTIFIER -DSTD_EXCEPTION_HAS_STACK_TRACE=1 -DV8_CONCURRENT_MARKING -DV8_HAVE_TARGET_OS -DV8_TARGET_ARCH_ARM64 -DV8_TARGET_OS_MACOS -D_LIBCPP_ENABLE_THREAD_SAFETY_ANNOTATIONS -I/Volumes/41plus/proton-enterprise/contrib/v8-cmake/v8/include -I/Volumes/41plus/proton-enterprise/build_clang18/contrib/v8-cmake/generated -I/Volumes/41plus/proton-enterprise/contrib/v8-cmake/v8 -I/Volumes/41plus/proton-enterprise/contrib/v8-cmake/v8/third_party/zlib -I/Volumes/41plus/proton-enterprise/build_clang18/contrib/v8-cmake -isystem /Volumes/41plus/proton-enterprise/contrib/llvm-project/libcxxabi/include -isystem /Volumes/41plus/proton-enterprise/contrib/libunwind/include -fdiagnostics-color=always -Wno-enum-constexpr-conversion  -pipe -march=armv8-a+crc -fasynchronous-unwind-tables -ffile-prefix-map=/Volumes/41plus/proton-enterprise=. -falign-functions=32  -Wall -Wno-unused-command-line-argument  -fdiagnostics-absolute-paths -w -O3 -DNDEBUG -std=gnu11 -arch arm64 -isysroot "/Applications/Xcode-beta 2.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.2.sdk" -mmacosx-version-min=10.15   -D OS_DARWIN -DV8_ENABLE_WEBASSEMBLY -DV8_ADVANCED_BIGINT_ALGORITHMS -DV8_ENABLE_WASM_SIMD256_REVEC -fno-exceptions -Wno-invalid-offsetof -Wno-deprecated-non-prototype -Werror -MD -MT contrib/v8-cmake/CMakeFiles/v8_base_without_compiler.dir/v8/third_party/zlib/zutil.c.o -MF contrib/v8-cmake/CMakeFiles/v8_base_without_compiler.dir/v8/third_party/zlib/zutil.c.o.d -o contrib/v8-cmake/CMakeFiles/v8_base_without_compiler.dir/v8/third_party/zlib/zutil.c.o -c /Volumes/41plus/proton-enterprise/contrib/v8-cmake/v8/third_party/zlib/zutil.c
In file included from /Volumes/41plus/proton-enterprise/contrib/v8-cmake/v8/third_party/zlib/zutil.c:10:
In file included from /Volumes/41plus/proton-enterprise/contrib/v8-cmake/v8/third_party/zlib/gzguts.h:21:
/Applications/Xcode-beta 2.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/stdio.h:220:7: error: expected identifier or '('
  220 | FILE    *fdopen(int, const char *) __DARWIN_ALIAS_STARTING(__MAC_10_6, __IPHONE_2_0, __DARWIN_ALIAS(fdopen));
      |          ^
/Volumes/41plus/proton-enterprise/contrib/v8-cmake/v8/third_party/zlib/zutil.h:162:33: note: expanded from macro 'fdopen'
  162 | #        define fdopen(fd,mode) NULL /* No fdopen() */
      |                                 ^
/opt/homebrew/Cellar/llvm/18.1.6/lib/clang/18/include/__stddef_null.h:26:16: note: expanded from macro 'NULL'
   26 | #define NULL ((void*)0)
      |                ^
In file included from /Volumes/41plus/proton-enterprise/contrib/v8-cmake/v8/third_party/zlib/zutil.c:10:
In file included from /Volumes/41plus/proton-enterprise/contrib/v8-cmake/v8/third_party/zlib/gzguts.h:21:
/Applications/Xcode-beta 2.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/stdio.h:220:7: error: expected ')'
/Volumes/41plus/proton-enterprise/contrib/v8-cmake/v8/third_party/zlib/zutil.h:162:33: note: expanded from macro 'fdopen'
  162 | #        define fdopen(fd,mode) NULL /* No fdopen() */
      |                                 ^
/opt/homebrew/Cellar/llvm/18.1.6/lib/clang/18/include/__stddef_null.h:26:16: note: expanded from macro 'NULL'
   26 | #define NULL ((void*)0)
      |                ^
/Applications/Xcode-beta 2.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/stdio.h:220:7: note: to match this '('
/Volumes/41plus/proton-enterprise/contrib/v8-cmake/v8/third_party/zlib/zutil.h:162:33: note: expanded from macro 'fdopen'
  162 | #        define fdopen(fd,mode) NULL /* No fdopen() */
      |                                 ^
/opt/homebrew/Cellar/llvm/18.1.6/lib/clang/18/include/__stddef_null.h:26:15: note: expanded from macro 'NULL'
   26 | #define NULL ((void*)0)
      |               ^
In file included from /Volumes/41plus/proton-enterprise/contrib/v8-cmake/v8/third_party/zlib/zutil.c:10:
In file included from /Volumes/41plus/proton-enterprise/contrib/v8-cmake/v8/third_party/zlib/gzguts.h:21:
/Applications/Xcode-beta 2.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/stdio.h:220:7: error: expected ')'
  220 | FILE    *fdopen(int, const char *) __DARWIN_ALIAS_STARTING(__MAC_10_6, __IPHONE_2_0, __DARWIN_ALIAS(fdopen));
      |          ^
/Volumes/41plus/proton-enterprise/contrib/v8-cmake/v8/third_party/zlib/zutil.h:162:33: note: expanded from macro 'fdopen'
  162 | #        define fdopen(fd,mode) NULL /* No fdopen() */
      |                                 ^
/opt/homebrew/Cellar/llvm/18.1.6/lib/clang/18/include/__stddef_null.h:26:22: note: expanded from macro 'NULL'
   26 | #define NULL ((void*)0)
      |                      ^
/Applications/Xcode-beta 2.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/stdio.h:220:7: note: to match this '('
/Volumes/41plus/proton-enterprise/contrib/v8-cmake/v8/third_party/zlib/zutil.h:162:33: note: expanded from macro 'fdopen'
  162 | #        define fdopen(fd,mode) NULL /* No fdopen() */
      |                                 ^
/opt/homebrew/Cellar/llvm/18.1.6/lib/clang/18/include/__stddef_null.h:26:14: note: expanded from macro 'NULL'
   26 | #define NULL ((void*)0)
      |              ^
3 errors generated.
[4352/8377] Building CXX object contrib/v8-cmake/CMakeFiles/v8_base_without_compiler.dir/v8/src/wasm/baseline/liftoff-compiler.cc.o
ninja: build stopped: subcommand failed.
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants