Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support local OSX SDK #10

Open
sluongng opened this issue Mar 10, 2023 · 12 comments · May be fixed by #184
Open

Support local OSX SDK #10

sluongng opened this issue Mar 10, 2023 · 12 comments · May be fixed by #184
Labels
enhancement New feature or request

Comments

@sluongng
Copy link

Toolchain setup

http_archive(
    name = "bazel-zig-cc",
    sha256 = "e9f82bfb74b3df5ca0e67f4d4989e7f1f7ce3386c295fd7fda881ab91f83e509",
    strip_prefix = "bazel-zig-cc-v1.0.1",
    urls = ["https://github.com/uber/bazel-zig-cc/archive/v1.0.1.tar.gz"],
)

load("@bazel-zig-cc//toolchain:defs.bzl", zig_toolchains = "toolchains")

zig_toolchains()

register_toolchains(
    "@zig_sdk//toolchain:linux_amd64_gnu.2.34",
    "@zig_sdk//toolchain:linux_arm64_gnu.2.34",
    "@zig_sdk//toolchain:darwin_amd64",
    "@zig_sdk//toolchain:darwin_arm64",
)

Error outputs

> bazel build @com_github_mattn_go_ieproxy//...
ERROR: /private/var/tmp/_bazel_sluongng/06e573a93bc2d6a9cad4ad41f00b4310/external/com_github_mattn_go_ieproxy/BUILD.bazel:57:8: GoCompilePkg external/com_github_mattn_go_ieproxy/go-ieproxy_test.internal.a failed: (Exit 1): builder failed: error executing command (from target @com_github_mattn_go_ieproxy//:go-ieproxy_test) bazel-out/darwin_arm64-opt-exec-2B5CBBC6/bin/external/go_sdk_darwin_arm64/builder_reset/builder compilepkg -sdk external/go_sdk_darwin_arm64 -installsuffix darwin_arm64 -src ... (remaining 73 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
buildbuddy/external/com_github_mattn_go_ieproxy/ieproxy_darwin.go:7:10: fatal error: 'CFNetwork/CFProxySupport.h' file not found
#include <CFNetwork/CFProxySupport.h>
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
compilepkg: error running subcommand external/go_sdk_darwin_arm64/pkg/tool/darwin_arm64/cgo: exit status 2

Gazelle generated build files

> cat /private/var/tmp/_bazel_sluongng/06e573a93bc2d6a9cad4ad41f00b4310/external/com_github_mattn_go_ieproxy/BUILD.bazel
load("@io_bazel_rules_go//go:def.bzl", "go_library", "go_test")

go_library(
    name = "go-ieproxy",
    srcs = [
        "ieproxy.go",
        "ieproxy_darwin.go",
        "ieproxy_unix.go",
        "ieproxy_windows.go",
        "kernel32_data_windows.go",
        "pac_darwin.go",
        "pac_unix.go",
        "pac_windows.go",
        "proxy_middleman.go",
        "proxy_middleman_darwin.go",
        "proxy_middleman_unix.go",
        "proxy_middleman_windows.go",
        "utils.go",
        "winhttp_data_windows.go",
    ],
    cgo = True,
    clinkopts = select({
        "@io_bazel_rules_go//go/platform:darwin": [
            "-framework CFNetwork",
            "-framework CoreFoundation",
        ],
        "@io_bazel_rules_go//go/platform:ios": [
            "-framework CFNetwork",
            "-framework CoreFoundation",
        ],
        "//conditions:default": [],
    }),
    importpath = "github.com/mattn/go-ieproxy",
    visibility = ["//visibility:public"],
    deps = select({
        "@io_bazel_rules_go//go/platform:darwin": [
            "@org_golang_x_net//http/httpproxy",
        ],
        "@io_bazel_rules_go//go/platform:ios": [
            "@org_golang_x_net//http/httpproxy",
        ],
        "@io_bazel_rules_go//go/platform:windows": [
            "@org_golang_x_net//http/httpproxy",
            "@org_golang_x_sys//windows",
            "@org_golang_x_sys//windows/registry",
        ],
        "//conditions:default": [],
    }),
)

alias(
    name = "go_default_library",
    actual = ":go-ieproxy",
    visibility = ["//visibility:public"],
)

go_test(
    name = "go-ieproxy_test",
    srcs = [
        "darwin_test.go",
        "example_test.go",
        "utils_test.go",
        "windows_test.go",
    ],
    embed = [":go-ieproxy"],
)

Bazel version is 6.0.0 on M1 Mac.

Using Grail's toolchain, the sysroot is set to OSX SDK dynamically with https://github.com/grailbio/bazel-toolchain/blob/069ee4e20ec605a6c76c1798658e17175b2eb35e/toolchain/internal/sysroot.bzl#L24-L39

Ultimately this should affect cxx_builtin_include_directories and builtin_sysroot attribute when call cc_common. create_cc_toolchain_config_info()

@motiejus
Copy link
Collaborator

motiejus commented Mar 10, 2023

W are not planning to add support non-hermetic osx compilation with bazel-zig-cc. If we go with osx support, it needs to be hermetic. Zig supports linking MachO, but it needs the libc definitions (tbd files). That is the biggest problem, which are gray area (feel free to search for tbd in ziglang issues).

We are open, though, to support tbd files via a user-provided http_archive.

@motiejus motiejus closed this as not planned Won't fix, can't repro, duplicate, stale Mar 10, 2023
@shs96c
Copy link

shs96c commented Mar 10, 2023

If there were some way of providing the sysroot it would be possible to set this up.

@motiejus
Copy link
Collaborator

If there were some way of providing the sysroot it would be possible to set this up.

I have no idea. It's probably not too difficult, but I never tried. We did not get to the point to try it at Uber too.

If someone in the community needs it and implements support for this sysroot, we will review/accept the PR.

@sluongng
Copy link
Author

My research led me to rules_apple where cc_library could be linked like this https://github.com/bazelbuild/rules_apple/blob/master/test/starlark_tests/resources/kext_resources/BUILD

I have not tested whether this copt could be added to rules_go just yet, but if it could then that would solve my problem.

@motiejus how do you solve building application with transitive deps depending on MacOS libraries like this? Do you simply block these deps from being used at Uber entirely?

@motiejus
Copy link
Collaborator

@motiejus how do you solve building application with transitive deps depending on MacOS libraries like this? Do you simply block these deps from being used at Uber entirely?

I have only (and by extension Uber) used zig c++ for building Linux targets; I have not built anything non-trivial for OSX with Darwin except the launcher, which does not need any more frameworks except CoreFoundation (thus trivial).

As for how to compile more OSX applications, here are the rough steps:

  1. Background reading: ability to link against darwin frameworks (such as CoreFoundation) when cross compiling ziglang/zig#1349 (comment)
  2. Implement ability to specify a Darwin sysroot (via http_archive?) and attach it to a particular Darwin toolchain.
  3. Configure the Darwin toolchain to specify the right path to the sysroot, when invoked.

You are welcome to wait (Uber may get to it, but I can't promise anything; someone might as well do it too), implement this yourself or ask/pay someone in the community to implement it for you. Like I said before, if we get a patch that adds a .tbd sysroot support for Darwin targets, we will happily merge and maintain it.

@fparga
Copy link

fparga commented Jul 27, 2023

[...] except the launcher, which does not need any more frameworks except CoreFoundation (thus trivial).

Is there something to do to have access to CoreFoundation? We recently updated our dependency on protobuf that since v22.0 depends on abseil that itself internally depends on CoreFoundation:

https://github.com/abseil/abseil-cpp/blob/4ff6968df3833b11f9ab063675126b1562b8b73b/absl/time/internal/cctz/BUILD.bazel#L56-L60

Since then, building any go_library that depends on google libraries that ship protos on a mac fails with that error:

/private/var/tmp/_bazel_user/b36a059997bad77ac57dde217c426f28/external/com_google_absl/absl/time/internal/cctz/BUILD.bazel:32:11: Compiling absl/time/internal/cctz/src/time_zone_lookup.cc [for tool] failed: (Exit 1): sandbox-exec failed: error executing command
  (cd /private/var/tmp/_bazel_user/b36a059997bad77ac57dde217c426f28/sandbox/darwin-sandbox/503/execroot/project && \
  exec env - \
    PATH=/bin:/usr/bin:/usr/local/bin \
    PWD=/proc/self/cwd \
    TMPDIR=/var/folders/jh/bh0m7pbj2nqc2b8f5v98sl900000gp/T/ \
  /usr/bin/sandbox-exec -f /private/var/tmp/_bazel_user/b36a059997bad77ac57dde217c426f28/sandbox/darwin-sandbox/503/sandbox.sb /var/tmp/_bazel_user/install/202e556d99f024ca401d77f7be1827ad/process-wrapper '--timeout=0' '--kill_delay=15' '--stats=/private/var/tmp/_bazel_user/b36a059997bad77ac57dde217c426f28/sandbox/darwin-sandbox/503/stats.out' external/zig_sdk/tools/aarch64-macos-none/c++ -MD -MF bazel-out/darwin_arm64-opt-exec-2B5CBBC6/bin/external/com_google_absl/absl/time/internal/cctz/_objs/time_zone/time_zone_lookup.d '-frandom-seed=bazel-out/darwin_arm64-opt-exec-2B5CBBC6/bin/external/com_google_absl/absl/time/internal/cctz/_objs/time_zone/time_zone_lookup.o' '-DBAZEL_CURRENT_REPOSITORY="com_google_absl"' -iquote external/com_google_absl -iquote bazel-out/darwin_arm64-opt-exec-2B5CBBC6/bin/external/com_google_absl -no-canonical-prefixes -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' '-mcpu=apple_m1' -O2 -DNDEBUG -g0 -Werror -w -g0 -c external/com_google_absl/absl/time/internal/cctz/src/time_zone_lookup.cc -o bazel-out/darwin_arm64-opt-exec-2B5CBBC6/bin/external/com_google_absl/absl/time/internal/cctz/_objs/time_zone/time_zone_lookup.o)
external/com_google_absl/absl/time/internal/cctz/src/time_zone_lookup.cc:26:10: fatal error: 'CoreFoundation/CFTimeZone.h' file not found
#include <CoreFoundation/CFTimeZone.h>
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.

Our zig_toolchain is configured following the Use case: always compile with zig cc instructions, so with a list of unconditionally added toolchains, and with BAZEL_DO_NOT_DETECT_CPP_TOOLCHAIN=1.

Is it possible to have such a setup working? Or is zig-cc not an option anymore?

I'm posting this question here, because it's related to the original issue, but please let me know if I should create a new one.

@motiejus
Copy link
Collaborator

Is it possible to have such a setup working? Or is zig-cc not an option anymore?

I am honestly surprised you were able to compile a real binary to osx using hermetic_cc_toolchain. :)

I'm posting this question here, because it's related to the original issue, but please let me know if I should create a new one.

I'm afraid it won't be actionable, since osx target is not supported in hermetic_cc_toolchain until someone does the work and implements it.

@fparga
Copy link

fparga commented Jul 27, 2023

I am honestly surprised you were able to compile a real binary to osx using hermetic_cc_toolchain. :)

Yes it worked perfectly for several month, up until that dependency on abseil got added. I guess I got lucky and my use case was quite trivial, haha.

I'm afraid it won't be actionable, since osx target is not supported in hermetic_cc_toolchain until someone does the work and implements it.

Ok that was my understanding, I was just curious if somehow it was different if the only dependency on OSX SDK was CoreFoundation, but I guess not.

Thank you @motiejus, that's some incredible work you've done there!

@ghost
Copy link

ghost commented Oct 24, 2023

This comes up often. Reopening for visibility. Also see my comment in #104 (comment)

@ghost ghost reopened this Oct 24, 2023
@ghost ghost pinned this issue Oct 24, 2023
ghost pushed a commit that referenced this issue Dec 13, 2023
- `.bazelrc`: `sandbox_add_mount_pair=/tmp` is now required everywhere.
- `.bazelrc` and `README`: remove deprecated/now-default flags.
- `README`: add links to #10 and #83.
- `README`: mention bzlmod.
- `examples/*/.bazelrc`: make them the same for a better recommendation.

The top-level project itself is not bzlmod yet, because that's
book-keeping (it can already be used with bzlmod, so this is purely
book-keeping). We can tackle this out of band with dependency updates.
ghost pushed a commit that referenced this issue Dec 13, 2023
- `.bazelrc`: `sandbox_add_mount_pair=/tmp` is now required everywhere.
- `.bazelrc` and `README`: remove deprecated/now-default flags.
- `README`: add links to #10 and #83.
- `README`: mention bzlmod.
- `examples/*/.bazelrc`: make them the same for a better recommendation.

The top-level project itself is not bzlmod yet, because that's
book-keeping (it can already be used with bzlmod, so this is purely
book-keeping). We can tackle this out of band with dependency updates.

Fixes #134
@motiejus motiejus added the enhancement New feature or request label Dec 13, 2023
ghost pushed a commit that referenced this issue Dec 14, 2023
- bump bazel to 6.4.0
- bump rules_go, gazelle, platforms
- enable bzlmod
- remove WORKSPACE
- remove `gofmt`, github.com//issues/136
- remove Go MacOS x86_64 tests, github.com//issues/10
ghost pushed a commit that referenced this issue Dec 14, 2023
- bump bazel to 6.4.0
- bump rules_go, gazelle, platforms
- enable bzlmod
- remove WORKSPACE
- remove `gofmt`, #136
- remove Go MacOS x86_64 tests, #10
ghost pushed a commit that referenced this issue Dec 14, 2023
- bump bazel to 6.4.0
- bump rules_go, gazelle, platforms
- enable bzlmod
- clear WORKSPACE (cannot remove yet due to examples)
- remove `gofmt`, #136
- remove Go MacOS x86_64 tests, #10
ghost pushed a commit that referenced this issue Dec 14, 2023
- bump bazel to 6.4.0
- bump rules_go, gazelle, platforms
- enable bzlmod
- clear WORKSPACE (cannot remove yet due to examples)
- remove `gofmt`, #136
- remove Go MacOS x86_64 tests, #10
ghost pushed a commit that referenced this issue Dec 14, 2023
- bump bazel to 6.4.0
- bump rules_go, gazelle, platforms
- enable bzlmod
- clear WORKSPACE (cannot remove yet due to examples)
- remove `gofmt`, #136
- remove Go MacOS x86_64 tests, #10
ghost pushed a commit that referenced this issue Dec 14, 2023
- bump bazel to 6.4.0
- bump rules_go, gazelle, platforms
- enable bzlmod
- clear WORKSPACE (cannot remove yet due to examples)
- remove `gofmt`, #136
- remove Go MacOS x86_64 tests, #10
ghost pushed a commit that referenced this issue Dec 14, 2023
- bump bazel to 6.4.0
- bump rules_go, gazelle, platforms
- enable bzlmod
- clear WORKSPACE (cannot remove yet due to examples)
- remove `gofmt`, #136
- remove Go MacOS x86_64 tests, #10
ghost pushed a commit that referenced this issue Dec 14, 2023
- `.bazelrc`: `sandbox_add_mount_pair=/tmp/zig-cache` (if you can) or
  `/tmp` (if not) is now required everywhere.
- `.bazelrc` and `README`: remove deprecated/now-default flags.
- `README`: add links to #10 and #83.
- `README`: mention bzlmod.
- `examples/*/.bazelrc`: make them the same for a better recommendation.

Fixes #134
ghost pushed a commit that referenced this issue Dec 14, 2023
- bump bazel to 6.4.0
- bump rules_go, gazelle, platforms
- enable bzlmod
- clear WORKSPACE (cannot remove yet due to examples)
- remove `gofmt`, #136
- remove Go MacOS x86_64 tests, #10
ghost pushed a commit that referenced this issue Dec 14, 2023
- `.bazelrc`: `sandbox_add_mount_pair=/tmp/zig-cache` (if you can) or
  `/tmp` (if not) is now required everywhere.
- `.bazelrc` and `README`: remove deprecated/now-default flags.
- `README`: add links to #10 and #83.
- `README`: mention bzlmod.
- `examples/*/.bazelrc`: make them the same for a better recommendation.

Fixes #134
ghost pushed a commit that referenced this issue Dec 14, 2023
- `.bazelrc`: `sandbox_add_mount_pair=/tmp/zig-cache` (if you can) or
  `/tmp` (if not) is now required everywhere.
- `.bazelrc` and `README`: remove deprecated/now-default flags.
- `README`: add links to #10 and #83.
- `README`: mention bzlmod.
- `examples/*/.bazelrc`: make them the same for a better recommendation.

Fixes #134.
ghost pushed a commit that referenced this issue Dec 15, 2023
- bump bazel to 6.4.0
- bump rules_go, gazelle, platforms
- enable bzlmod
- clear WORKSPACE (cannot remove yet due to examples)
- remove `gofmt`, #136
- remove Go MacOS x86_64 tests, #10
ghost pushed a commit that referenced this issue Dec 15, 2023
- bump bazel to 6.4.0
- bump rules_go, gazelle, platforms
- enable bzlmod
- clear WORKSPACE (cannot remove yet due to examples)
- remove `gofmt`, #136
- remove Go MacOS x86_64 tests, #10
ghost pushed a commit that referenced this issue Dec 15, 2023
- `.bazelrc`: `sandbox_add_mount_pair=/tmp/zig-cache` (if you can) or
  `/tmp` (if not) is now required everywhere.
- `.bazelrc` and `README`: remove deprecated/now-default flags.
- `README`: add links to #10 and #83.
- `README`: mention bzlmod.
- `examples/*/.bazelrc`: make them the same for a better recommendation.

Fixes #134.
ghost pushed a commit that referenced this issue Dec 15, 2023
- bump bazel to 6.4.0
- bump rules_go, gazelle, platforms
- enable bzlmod
- clear WORKSPACE (cannot remove yet due to examples)
- remove `gofmt`, #136
- remove Go MacOS x86_64 tests, #10
ghost pushed a commit that referenced this issue Dec 15, 2023
- `.bazelrc`: `sandbox_add_mount_pair=/tmp/zig-cache` (if you can) or
  `/tmp` (if not) is now required everywhere.
- `.bazelrc` and `README`: remove deprecated/now-default flags.
- `README`: add links to #10 and #83.
- `README`: mention bzlmod.
- `examples/*/.bazelrc`: make them the same for a better recommendation.

Fixes #134.
ghost pushed a commit that referenced this issue Dec 19, 2023
- bump bazel to 6.4.0
- bump rules_go, gazelle, platforms
- enable bzlmod
- clear WORKSPACE (cannot remove yet due to examples)
- remove `gofmt`, #136
- remove Go MacOS x86_64 tests, #10
ghost pushed a commit that referenced this issue Dec 19, 2023
- bump bazel to 6.4.0
- bump rules_go, gazelle, platforms
- enable bzlmod
- clear WORKSPACE (cannot remove yet due to examples)
- remove `gofmt`, #136
- remove Go MacOS x86_64 tests, #10
ghost pushed a commit that referenced this issue Dec 19, 2023
- `.bazelrc`: `sandbox_add_mount_pair=/tmp/zig-cache` (if you can) or
  `/tmp` (if not) is now required everywhere.
- `.bazelrc` and `README`: remove deprecated/now-default flags.
- `README`: add links to #10 and #83.
- `README`: mention bzlmod.
- `examples/*/.bazelrc`: make them the same for a better recommendation.

Fixes #134.
ghost pushed a commit that referenced this issue Dec 19, 2023
- bump bazel to 6.4.0
- bump rules_go, gazelle, platforms
- enable bzlmod
- clear WORKSPACE (cannot remove yet due to examples)
- remove `gofmt`, #136
- remove Go MacOS x86_64 tests, #10
ghost pushed a commit that referenced this issue Dec 19, 2023
- bump bazel to 6.4.0
- bump rules_go, gazelle, platforms
- enable bzlmod
- clear WORKSPACE (cannot remove yet due to examples)
- remove `gofmt`, #136
- remove Go MacOS x86_64 tests, #10
ghost pushed a commit that referenced this issue Dec 19, 2023
- `.bazelrc`: `sandbox_add_mount_pair=/tmp/zig-cache` (if you can) or
  `/tmp` (if not) is now required everywhere.
- `.bazelrc` and `README`: remove deprecated/now-default flags.
- `README`: add links to #10 and #83.
- `README`: mention bzlmod.
- `examples/*/.bazelrc`: make them the same for a better recommendation.

Fixes #134.
motiejus pushed a commit that referenced this issue Dec 19, 2023
- bump bazel to 6.4.0
- bump rules_go, gazelle, platforms
- enable bzlmod
- clear WORKSPACE (cannot remove yet due to examples)
- remove `gofmt`, #136
- remove Go MacOS x86_64 tests, #10
motiejus pushed a commit that referenced this issue Dec 19, 2023
- `.bazelrc`: `sandbox_add_mount_pair=/tmp/zig-cache` (if you can) or
  `/tmp` (if not) is now required everywhere.
- `.bazelrc` and `README`: remove deprecated/now-default flags.
- `README`: add links to #10 and #83.
- `README`: mention bzlmod.
- `examples/*/.bazelrc`: make them the same for a better recommendation.

Fixes #134.
@steeve
Copy link

steeve commented Apr 9, 2024

I have some updates on that front: ziglang/zig#10299 (comment)

Also, here is a repo rule to download and unpack it:

load(
    "@bazel_tools//tools/build_defs/repo:utils.bzl",
    "get_auth",
    "patch",
    "workspace_and_buildfile",
)

def _http_pkg_archive_impl(rctx):
    if rctx.attr.build_file and rctx.attr.build_file_content:
        fail("Only one of build_file and build_file_content can be provided.")
    rctx.download(
        url = rctx.attr.urls,
        output = ".downloaded.pkg",
        sha256 = rctx.attr.sha256,
        canonical_id = " ".join(rctx.attr.urls),
        auth = get_auth(rctx, rctx.attr.urls),
    )

    res = rctx.execute(["/usr/sbin/pkgutil", "--expand-full", ".downloaded.pkg", "tmp"])
    if res.return_code != 0:
        fail("Failed to extract package: {}".format(res.stdout))
    rctx.delete(".downloaded.pkg")

    strip_prefix = "tmp/Payload/"
    if rctx.attr.strip_prefix:
        strip_prefix += rctx.attr.strip_prefix + "/"
    extracted = rctx.path(strip_prefix)
    if not extracted.is_dir or not extracted.exists:
        fail("Extracted package does not contain expected directory: {}".format(strip_prefix))
    entries = extracted.readdir(watch = "no")
    for entry in entries:
        rctx.execute(["mv", str(entry), entry.basename])
    rctx.delete(extracted)
    workspace_and_buildfile(rctx)
    patch(rctx)

http_pkg_archive = repository_rule(
    _http_pkg_archive_impl,
    attrs = {
        "urls": attr.string_list(mandatory = True),
        "sha256": attr.string(mandatory = True),
        "strip_prefix": attr.string(),
        "build_file": attr.label(allow_single_file = True),
        "build_file_content": attr.string(),
        "workspace_file": attr.label(allow_single_file = True),
        "workspace_file_content": attr.string(),
    },
)

@thesayyn
Copy link

I have looked into this briefly; looks like those .pkg files are just a xar format which can be extracted by

Download using this ziglang/zig#10299 (comment)
Extract the .pkg archive using bsdtar toolchain
Extract the Payload using https://github.com/NiklasRosenstein/pbzx which needs be compiled from source
and pipe into bsdtar again using the cpio format to lay out files on disk.

@thesayyn
Copy link

However, i believe this sysroot repository rule can be built separately and should live in https://github.com/bazelbuild/rules_apple

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants