`tvOS`/`watchOS` compilation errors #118

05nelsonm · 2023-09-30T17:02:45Z

Trying to compile for watchOS and tvOS but keep running into the following errors related to usage of syscall

error: 'syscall' is unavailable: not available on tvOS
    if (syscall(SYS_getentropy, buf, n) == 0)
        ^

note: 'syscall' has been explicitly marked unavailable here
int        syscall(int, ...);
           ^

Everything's working great though for iOS 👍

Seeing in the amalgamations that there is conditional arguments for macOS and iOS only. Would it be possible to also include tvOS and watchOS?

The text was updated successfully, but these errors were encountered:

utelle · 2023-09-30T22:20:02Z

Trying to compile for watchOS and tvOS but keep running into the following errors related to usage of syscall
error: 'syscall' is unavailable: not available on tvOS
    if (syscall(SYS_getentropy, buf, n) == 0)
        ^
note: 'syscall' has been explicitly marked unavailable here
int        syscall(int, ...);
           ^
Everything's working great though for iOS 👍

Seeing in the amalgamations that there is conditional arguments for macOS and iOS only. Would it be possible to also include tvOS and watchOS?

In principle, it should be possible to include tvOS and watchOS. However, I have currently no means to test this myself.

The error you're receiving results from compiling the code in source file chacha20poly1305.c:

static size_t entropy(void* buf, size_t n)
{
#if defined(__APPLE__) && defined(__MAC_10_12) && __MAC_OS_X_VERSION_MAX_ALLOWED >= __MAC_10_12
  if (getentropy(buf, n) == 0)
    return n;
#elif defined(__linux__) && defined(SYS_getrandom)
  if (syscall(SYS_getrandom, buf, n, 0) == n)
    return n;
#elif defined(SYS_getentropy)
  if (syscall(SYS_getentropy, buf, n) == 0)
    return n;
#endif
  return read_urandom(buf, n);
}

Interestingly, the symbol SYS_getentropy seems to be defined for tvOS and watchOS, although the function syscall is not available, and therefore you get a compile time error.

As far as I know the function getentropy is available for tvOS and watchOS. However, at the moment I don't know how to check for tvOS or watchOS via the C preprocessor, so that function getentropy will be used.

utelle · 2023-10-01T09:00:07Z

Could you please test to change the code for function entropy as follows:

Add include for Apple target conditionals (3 lines)
Add check for tvOS and watchOS

#if defined(__APPLE__)
#include "TargetConditionals.h"
#endif

static size_t entropy(void* buf, size_t n)
{
#if defined(__APPLE__) && ((defined(__MAC_10_12) && __MAC_OS_X_VERSION_MAX_ALLOWED >= __MAC_10_12) || TARGET_OS_TV || TARGET_OS_WATCH)
  if (getentropy(buf, n) == 0)
    return n;
#elif defined(__linux__) && defined(SYS_getrandom)
  if (syscall(SYS_getrandom, buf, n, 0) == n)
    return n;
#elif defined(SYS_getentropy)
  if (syscall(SYS_getentropy, buf, n) == 0)
    return n;
#endif
  return read_urandom(buf, n);
}

If that works for you please let me know.

05nelsonm · 2023-10-01T10:45:02Z

Had a few compilation errors

tvOS and watchOS

implicit declaration of function 'getentropy' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
  if (getentropy(buf, n) == 0
      ^

After adding the following compiler flags to suppress errors, everything built successfully for tvOS and watchOS:

-Wno-implicit-function-declaration

The following Kotlin Multiplatform targets were compiled successfully:

iosArm64
iosSimulatorArm64
iosX64
tvosArm64
tvosSimulatorArm64
tvosX64
watchosArm32
watchosArm64
watchosX64

My tests that exercised SQLite3MultipleCiphers for all ciphers, rekeying, etc. were run only for only the *X64 targets as I am on x86_64 machine, but I assume that the other architectures work just fine considering the added logic to entropy has nothing to do with architecture.

Have to figure out configuration yet to enable watchosSimulatorArm64, as it throws the following error during compilation:

overriding the module target triple with arm64-apple-watch7.0-simulator [-Woverride-module]

Adding compiler flag -Wno-override-module doesn't do anything, so.

utelle · 2023-10-01T11:26:15Z

Had a few compilation errors

* `tvOS` and `watchOS`

      implicit declaration of function 'getentropy' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
        if (getentropy(buf, n) == 0
            ^

After adding the following compiler flags to suppress errors, everything built successfully for tvOS and watchOS:

* `-Wno-implicit-function-declaration`

As an alternative you may add an explicit function declaration:

#if TARGET_OS_TV || TARGET_OS_WATCH
extern int getentropy(void* buffer, size_t buflen);
#endif
  if (getentropy(buf, n) == 0)

I will add these modifications in my SQLite3MultipleCiphers repository.

My tests that exercised SQLite3MultipleCiphers for all ciphers, rekeying, etc. were run only for the *X64 targets as I am on x86_64 machine, but I assume that the other architectures work just fine considering the added logic to entropy has nothing to do with architecture.

Right, the other architectures should not be affected.

Have to figure out configuration yet to enable watchosSimulatorArm64, as it throws the following error during compilation:
overriding the module target triple with arm64-apple-watch7.0-simulator [-Woverride-module]
Adding compiler flag -Wno-override-module doesn't do anything, so.

Which target triple is actually defined in your build script? Maybe just specifying the target triple mentioned in the error message in your build script helps, that is:

target triple = "arm64-apple-watch7.0-simulator"

05nelsonm · 2023-10-01T13:44:09Z

As an alternative you may add an explicit function declaration:

#if TARGET_OS_TV || TARGET_OS_WATCH
extern int getentropy(void* buffer, size_t buflen);
#endif
  if (getentropy(buf, n) == 0)

That worked in that the error suppression is no-longer necessary. 👍

I will add these modifications in my SQLite3MultipleCiphers repository.

🥳

Which target triple is actually defined in your build script? Maybe just specifying the target triple mentioned in the error message in your build script helps, that is:
target triple = "arm64-apple-watch7.0-simulator"

Yeah it's a little problematic as I'm using an older version of kotlin's llvm toolchain. It needs updating (see touchlab/cklib#11) in order to resolve this specific issue so I can add the watchosSimulatorArm64 target.

Check for _tvOS_ or _watchOS_ to call `getentropy` (instead of using `syscall`)

utelle · 2023-10-01T18:02:22Z

As an alternative you may add an explicit function declaration:
#if TARGET_OS_TV || TARGET_OS_WATCH
extern int getentropy(void* buffer, size_t buflen);
#endif
  if (getentropy(buf, n) == 0)
That worked in that the error suppression is no-longer necessary. 👍

I will add these modifications in my SQLite3MultipleCiphers repository.

In commit dd508dc I added the precompiler checks for tvOS and watchOS.

I also added the explicit function declaration for getentropy. However, maybe we could avoid that by including

#include <sys/random.h>

Could you please check whether this include is available for tvOS and watchOS? That is, check for tvOS and watchOS like this:

#if defined(__APPLE__) && ((defined(__MAC_10_12) && !defined(__IPHONE_OS_VERSION_MIN_REQUIRED)) || TARGET_OS_TV || TARGET_OS_WATCH)
#include <sys/random.h>
#endif

You have to move the include of the target conditionals in front of the above code:

#if defined(__APPLE__)
#include "TargetConditionals.h"
#endif

#if defined(__APPLE__) && ((defined(__MAC_10_12) && !defined(__IPHONE_OS_VERSION_MIN_REQUIRED)) || TARGET_OS_TV || TARGET_OS_WATCH)
#include <sys/random.h>
#endif

05nelsonm · 2023-10-01T19:54:03Z

TL;DR >> sys/random.h is available for iOS and tvOS, but not watchOS.

Was not available for watchosArm32, watchosArm64, watchosX64, and presumably watchosSimulatorArm64

fatal error: 'sys/random.h' file not found

I disabled those targets (all watchOS) and recompiled iOS and tvOS targets; everything worked (build, tests).

utelle · 2023-10-01T20:31:27Z

TL;DR >> sys/random.h is available for iOS and tvOS, but not watchOS.

According to Apple's documentation the function getentropy is available in watchOS. So, in which header file is it defined? Of course, we can define it explicitly (as is currently done), but it would be cleaner to include the correct header file, if it exists, that is.

Was not available for watchosArm32, watchosArm64, watchosX64, and presumably watchosSimulatorArm64
fatal error: 'sys/random.h' file not found

Well, I found this post that indicates that the function is not considered to be part of the public Apple API (contrary to the claim in Apple's documentation). However, it would not be the first flaw in Apple's documentation.

Maybe the function SecRandomCopyBytes could be used instead for tvOS and watchOS?

I disabled those targets (all watchOS) and recompiled iOS and tvOS targets; everything worked (build, tests).

I'm unsure about which approach should be used to support watchOS.

05nelsonm · 2023-10-01T20:57:02Z

Imo, just use SecRandomCopyBytes for all Apple targets.

https://github.com/KotlinCrypto/secure-random/blob/591ba1dbea955f1a4673cbf3189d8d11ad8fe8bf/secure-random/src/darwinMain/kotlin/org/kotlincrypto/internal/SecRandomDelegate.kt#L40

utelle · 2023-10-01T22:02:13Z

Imo, just use SecRandomCopyBytes for all Apple targets.

Ok. I'd be fine with that approach. However, I don't know whether this really works for all Apple OS variants or whether checking for certain targets and/or versions is still required. Maybe you could try the following code and report the results:

#if defined(__APPLE__)
#include "TargetConditionals.h"
#endif

#if defined(__APPLE__)
#include <Security/SecRandom.h>
#endif

static size_t entropy(void* buf, size_t n)
{
#if defined(__APPLE__)
  if (SecRandomCopyBytes(kSecRandomDefault, n, (uint8_t*) buf) == 0)
    return n;
#elif defined(__linux__) && defined(SYS_getrandom)
  if (syscall(SYS_getrandom, buf, n, 0) == n)
    return n;
#elif defined(SYS_getentropy)
  if (syscall(SYS_getentropy, buf, n) == 0)
    return n;
#endif
  return read_urandom(buf, n);
}

05nelsonm · 2023-10-01T22:18:25Z

I don't know whether this really works for all Apple OS variants or whether checking for certain targets and/or versions is still required

Oh it's available 😄

You could (if you so desired to support macOS 10.6 and below) fallback to using getentropy.

Rust for macOS just straight up defaults to getentropy; see HERE. All other apple products they use SecRandomCopyBytes.

Maybe you could try the following code and report the results:

Worked like a charm. Didn't even need the #include "TargetConditionals.h".

05nelsonm · 2023-10-01T22:36:49Z

Just realized that I am able to enable the watchosSimulatorArm64 target; my compiler issues went away entirely.

No clue what that was about, but it seems to have been something with the inclusion of TargetConditionals.h?

Just so we are on the same page, this is what my sqlite3mc_amalgamation.c was modified to:

#if defined(__APPLE__)
#include <Security/SecRandom.h>
#endif

static size_t entropy(void* buf, size_t n)
{
#if defined(__APPLE__)
  if (SecRandomCopyBytes(kSecRandomDefault, n, (uint8_t*) buf) == 0)
    return n;
#elif defined(__linux__) && defined(SYS_getrandom)
  if (syscall(SYS_getrandom, buf, n, 0) == n)
    return n;
#elif defined(SYS_getentropy)
  if (syscall(SYS_getentropy, buf, n) == 0)
    return n;
#endif
  return read_urandom(buf, n);
}

05nelsonm · 2023-10-01T23:25:39Z

Another aside about the linux implementation, unless this is being compiled against glibc 2.25 +, it will never use getrandom because SYS_getrandom isn't defined at compile time for the shared lib, right?

https://man7.org/linux/man-pages/man2/getrandom.2.html#HISTORY

Might be worth it to have an actual implementation to check for it where by SYS_getrandom is alternatively defined, and a simple non-blocking check for it is made once to see if it is present on the machine.

https://github.com/KotlinCrypto/secure-random/blob/master/secure-random/src/linuxAndroidMain/kotlin/org/kotlincrypto/internal/GetRandom.kt

utelle · 2023-10-02T12:45:25Z

I don't know whether this really works for all Apple OS variants or whether checking for certain targets and/or versions is still required

Oh it's available 😄

Great. I adjusted the code accordingly in commit 21307d5.

You could (if you so desired to support macOS 10.6 and below) fallback to using getentropy.

For the time being I think the new implementation is good enough. If there are developers in need of support for older versions they can ask for it on the issue tracker.

Maybe you could try the following code and report the results:

Worked like a charm. Didn't even need the #include "TargetConditionals.h".

For now, I removed the include.

utelle · 2023-10-02T12:48:56Z

Just realized that I am able to enable the watchosSimulatorArm64 target; my compiler issues went away entirely.

Good news.

No clue what that was about, but it seems to have been something with the inclusion of TargetConditionals.h?

I have no idea either. However, it would be really strange, if including TargetConditionals.h incurs such strange issues.

Just so we are on the same page, this is what my sqlite3mc_amalgamation.c was modified to:

#if defined(__APPLE__)
#include <Security/SecRandom.h>
#endif

static size_t entropy(void* buf, size_t n)
{
#if defined(__APPLE__)
  if (SecRandomCopyBytes(kSecRandomDefault, n, (uint8_t*) buf) == 0)
    return n;
#elif defined(__linux__) && defined(SYS_getrandom)
  if (syscall(SYS_getrandom, buf, n, 0) == n)
    return n;
#elif defined(SYS_getentropy)
  if (syscall(SYS_getentropy, buf, n) == 0)
    return n;
#endif
  return read_urandom(buf, n);
}

Yes, that's how the code in the SQLite3MultipleCiphers git repository now looks like, too.

utelle · 2023-10-02T13:42:15Z

Unfortunately, the CI run fails for macOS, see GitHub CI macOS. Probably it is necessary to specify a link library, but which one?

05nelsonm · 2023-10-02T13:55:12Z

Unfortunately, the CI run fails for macOS, see GitHub CI macOS. Probably it is necessary to specify a link library, but which one?

Need to add link flag -framework Security for apple

utelle · 2023-10-02T14:23:34Z

Unfortunately, the CI run fails for macOS, see GitHub CI macOS. Probably it is necessary to specify a link library, but which one?

Need to add link flag -framework Security for apple

Thanks. I adjusted the autoconf build files and now it works.

05nelsonm · 2023-10-02T15:55:21Z

Note that, that'll work for sure for macOS.

For iOS, tvOS and watchOS, I'm not sure if you need to also include -isysroot /path/to/sdk

e.g.

# xcrun is the CLI tool for Xcode
SDK_IOS="$(xcrun --sdk iphoneos --show-sdk-path)"
SDK_IOS_SIM="$(xcrun --sdk iphonesimulator --show-sdk-path)"
SDK_TVOS="$(xcrun --sdk appletvos --show-sdk-path)"
SDK_TVOS_SIM="$(xcrun --sdk appletvsimulator --show-sdk-path)"
SDK_WATCHOS="$(xcrun --sdk watchos --show-sdk-path)"
SDK_WATCHOS_SIM="$(xcrun --sdk watchsimulator --show-sdk-path)"

# e.g. if compiling for iOS on a macOS machine
LDFLAGS+="-isysroot $SDK_IOS"

Not sure if you wanted to take it that far, but I think that's what's necessary in order for link flag -framework Security to find the correct platform's framework. Again, not super familiar with all this as I use Kotlin native compiler and it takes care of a lot of that automatically, so.

utelle · 2023-10-02T16:30:32Z

Note that, that'll work for sure for macOS.

For the current GitHub CI workflow this is good enough.

For iOS, tvOS and watchOS, I'm not sure if you need to also include -isysroot /path/to/sdk

I have to admit that I'm not very familiar with all those Apple OS variants. I would certainly like to add CI runs to the GitHub workflow for these target systems. However, I have no experience in setting up macOS, iOS etc specific workflows. I would need some assistance by someone experienced.

e.g.
# xcrun is the CLI tool for Xcode
SDK_IOS="$(xcrun --sdk iphoneos --show-sdk-path)"
SDK_IOS_SIM="$(xcrun --sdk iphonesimulator --show-sdk-path)"
SDK_TVOS="$(xcrun --sdk appletvos --show-sdk-path)"
SDK_TVOS_SIM="$(xcrun --sdk appletvsimulator --show-sdk-path)"
SDK_WATCHOS="$(xcrun --sdk watchos --show-sdk-path)"
SDK_WATCHOS_SIM="$(xcrun --sdk watchsimulator --show-sdk-path)"

# e.g. if compiling for iOS on a macOS machine
LDFLAGS+="-isysroot $SDK_IOS"
Not sure if you wanted to take it that far, but I think that's what's necessary in order for link flag -framework Security to find the correct platform's framework. Again, not super familiar with all this as I use Kotlin native compiler and it takes care of a lot of that automatically, so.

Of course, it would be nice to be able to support Apple's OS variants right out of the box, but as said above I have only limited experience with Apple systems.

utelle · 2023-10-02T16:37:44Z

Another aside about the linux implementation, unless this is being compiled against glibc 2.25 +, it will never use getrandom because SYS_getrandom isn't defined at compile time for the shared lib, right?

Actually, I did not really check which source path gets compiled under Linux. The implementation of the function entropy is based on the code from sqleet. The code works as is on all Linux platforms I personally use, and I didn't get feedback from other developers about any problems. So I feel no pressure to change anything.

Might be worth it to have an actual implementation to check for it where by SYS_getrandom is alternatively defined, and a simple non-blocking check for it is made once to see if it is present on the machine.

I'm of course open to improve the code if necessary.

05nelsonm · 2023-10-02T19:04:42Z

I have to admit that I'm not very familiar with all those Apple OS variants. I would certainly like to add CI runs to the GitHub workflow for these target systems. However, I have no experience in setting up macOS, iOS etc specific workflows. I would need some assistance by someone experienced.

Of course, it would be nice to be able to support Apple's OS variants right out of the box, but as said above I have only limited experience with Apple systems.

Yeah I've had quite a few hiccups compiling it with Kotlin (not your code, just apple targets in general). Learned a lot which may be useful here if I can find the time to contribute in that area.

Actually, I did not really check which source path gets compiled under Linux. The implementation of the function entropy is based on the code from sqleet. The code works as is on all Linux platforms I personally use, and I didn't get feedback from other developers about any problems. So I feel no pressure to change anything.

Ah ok. Yeah I'm definitely going to have a look at and how that is getting used. Will let ya know.

I'm of course open to improve the code if necessary.

👍

utelle · 2023-10-02T20:04:07Z

[...] Of course, it would be nice to be able to support Apple's OS variants right out of the box, but as said above I have only limited experience with Apple systems.

Yeah I've had quite a few hiccups compiling it with Kotlin (not your code, just apple targets in general). Learned a lot which may be useful here if I can find the time to contribute in that area.

That would be great, because from time to time developers are asking for Apple support.

Actually, I did not really check which source path gets compiled under Linux. The implementation of the function entropy is based on the code from sqleet. The code works as is on all Linux platforms I personally use, and I didn't get feedback from other developers about any problems. So I feel no pressure to change anything.

Ah ok. Yeah I'm definitely going to have a look at and how that is getting used. Will let ya know.

TIA!

- Fixed issue #118 - tvOS/watchOS compilation errors. On Apple platforms the function SecRandomCopyBytes() will now be used instead of getentropy(). - Fixed issue #119 - "PRAGMA mmap_size" conflicts with encrypted databases, a check has been added to allow this pragma for unencrypted databases. - Added "PRAGMA memory_security" to allow to clear memory before it is freed. This feature can have a considerable impact on performance and is therefore disabled by default.

utelle · 2023-10-03T20:40:44Z

Now, that version 1.7.0 is released, I think this issue can be closed as completed. Feel free to reopen it, if necessary.

05nelsonm mentioned this issue Oct 1, 2023

Enable tvOS/watchOS targets toxicity-io/sqlite-mc#39

Closed

utelle added a commit that referenced this issue Oct 1, 2023

Fix issue #118

dd508dc

Check for _tvOS_ or _watchOS_ to call `getentropy` (instead of using `syscall`)

05nelsonm mentioned this issue Oct 2, 2023

Use SecRandomCopyBytes toxicity-io/sqlite-mc#43

Merged

utelle closed this as completed Oct 3, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`tvOS`/`watchOS` compilation errors #118

`tvOS`/`watchOS` compilation errors #118

05nelsonm commented Sep 30, 2023

utelle commented Sep 30, 2023

utelle commented Oct 1, 2023

05nelsonm commented Oct 1, 2023

utelle commented Oct 1, 2023

05nelsonm commented Oct 1, 2023 •

edited

Loading

utelle commented Oct 1, 2023

05nelsonm commented Oct 1, 2023

utelle commented Oct 1, 2023

05nelsonm commented Oct 1, 2023

utelle commented Oct 1, 2023

05nelsonm commented Oct 1, 2023

05nelsonm commented Oct 1, 2023 •

edited

Loading

05nelsonm commented Oct 1, 2023 •

edited

Loading

utelle commented Oct 2, 2023

utelle commented Oct 2, 2023

utelle commented Oct 2, 2023

05nelsonm commented Oct 2, 2023 •

edited

Loading

utelle commented Oct 2, 2023

05nelsonm commented Oct 2, 2023

utelle commented Oct 2, 2023

utelle commented Oct 2, 2023

05nelsonm commented Oct 2, 2023 •

edited

Loading

utelle commented Oct 2, 2023

utelle commented Oct 3, 2023

tvOS/watchOS compilation errors #118

tvOS/watchOS compilation errors #118

Comments

05nelsonm commented Sep 30, 2023

utelle commented Sep 30, 2023

utelle commented Oct 1, 2023

05nelsonm commented Oct 1, 2023

utelle commented Oct 1, 2023

05nelsonm commented Oct 1, 2023 • edited Loading

utelle commented Oct 1, 2023

05nelsonm commented Oct 1, 2023

utelle commented Oct 1, 2023

05nelsonm commented Oct 1, 2023

utelle commented Oct 1, 2023

05nelsonm commented Oct 1, 2023

05nelsonm commented Oct 1, 2023 • edited Loading

05nelsonm commented Oct 1, 2023 • edited Loading

utelle commented Oct 2, 2023

utelle commented Oct 2, 2023

utelle commented Oct 2, 2023

05nelsonm commented Oct 2, 2023 • edited Loading

utelle commented Oct 2, 2023

05nelsonm commented Oct 2, 2023

utelle commented Oct 2, 2023

utelle commented Oct 2, 2023

05nelsonm commented Oct 2, 2023 • edited Loading

utelle commented Oct 2, 2023

utelle commented Oct 3, 2023

`tvOS`/`watchOS` compilation errors #118

`tvOS`/`watchOS` compilation errors #118

05nelsonm commented Oct 1, 2023 •

edited

Loading

05nelsonm commented Oct 1, 2023 •

edited

Loading

05nelsonm commented Oct 1, 2023 •

edited

Loading

05nelsonm commented Oct 2, 2023 •

edited

Loading

05nelsonm commented Oct 2, 2023 •

edited

Loading