Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI build for Android #3

Merged
merged 27 commits into from
Jun 30, 2024
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
b0580fc
Squash commits - Build for Android
ViliusSutkus89 Jun 15, 2024
a8405c0
[CI] Use host profile when exporting binary pkg
ViliusSutkus89 Jun 16, 2024
220e557
Remove conan graph, it generated way too much files
ViliusSutkus89 Jun 16, 2024
b3088ef
Login to artifactory only if there will be an actual upload
ViliusSutkus89 Jun 16, 2024
b2cba3f
Update build matrix
ViliusSutkus89 Jun 16, 2024
64f73e7
Exclude odrcore/1.0.0 from build matrix when building for Android
ViliusSutkus89 Jun 16, 2024
7d1a1bd
Use variables in conan profiles to set ANDROID_HOME
ViliusSutkus89 Jun 17, 2024
300187e
Use package/version and host profile as CI build job name
ViliusSutkus89 Jun 18, 2024
2ce3cb9
Rename package_version to package_reference
ViliusSutkus89 Jun 20, 2024
5d0d644
Add Android-21 and -23 conan profiles
ViliusSutkus89 Jun 20, 2024
7020959
List (and build in CI) newer versions first
ViliusSutkus89 Jun 20, 2024
92d6978
[2/2] Rename package_version to package_reference
ViliusSutkus89 Jun 20, 2024
5dc5afc
Add deep test workflow
ViliusSutkus89 Jun 20, 2024
6175876
Disable package exporter call in deep-test
ViliusSutkus89 Jun 20, 2024
71d8669
Update deep test filter
ViliusSutkus89 Jun 20, 2024
e0495be
Export all conan recipes to local cache
ViliusSutkus89 Jun 24, 2024
96075e5
Trigger build pipeline on generic push (this should also cover on PR)
ViliusSutkus89 Jun 24, 2024
74e2ad7
[2/2] Trigger build pipeline on generic push (this should also cover …
ViliusSutkus89 Jun 24, 2024
3c5a3a3
Disable ccache in CI. Build will not fit anyways
ViliusSutkus89 Jun 24, 2024
1f949b9
Merge deep-test into build pipeline
ViliusSutkus89 Jun 24, 2024
2da8205
Use ubuntu build profile instead of relying on default build profile
ViliusSutkus89 Jun 24, 2024
9db47be
Cache pip dependencies
ViliusSutkus89 Jun 24, 2024
befbf53
Run build pipeline on workflow_dispatch too
ViliusSutkus89 Jun 24, 2024
cb0cf52
Remove pip cache, turns out it's only for regular python projects wit…
ViliusSutkus89 Jun 24, 2024
a39d79b
Revert "Disable ccache in CI. Build will not fit anyways". I may have…
ViliusSutkus89 Jun 24, 2024
85ac402
Show ccache stats
ViliusSutkus89 Jun 24, 2024
37bb728
Update conan_export_all_packages to properly handle paths when starte…
ViliusSutkus89 Jun 24, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/config/ubuntu-22.04-clang-15/conan/profiles/host
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
include(host)

[settings]
arch=armv7
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
include(host)

[settings]
arch=armv8
13 changes: 13 additions & 0 deletions .github/config/ubuntu-22.04-ndk-26.3.11579264/conan/profiles/host
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
include(default)

[settings]
os=Android
os.api_level=23
arch=armv8
compiler=clang
compiler.version=17
compiler.libcxx=c++_static
compiler.cppstd=20

[conf]
tools.android:ndk_path=$ANDROID_HOME/ndk/26.3.11579264
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
include(host)

[settings]
arch=x86
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
include(host)

[settings]
arch=x86_64
31 changes: 26 additions & 5 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,11 @@ jobs:
matrix:
package: ${{ fromJson(needs.find-all-packages.outputs.packages) }}
config:
- { os: ubuntu-22.04, compiler: clang-15 }
- { os: ubuntu-22.04, compiler: clang-15, host-profile: host }
- { os: ubuntu-22.04, compiler: ndk-26.3.11579264, host-profile: armv8 }
- { os: ubuntu-22.04, compiler: ndk-26.3.11579264, host-profile: x86_64 }
- { os: ubuntu-22.04, compiler: ndk-26.3.11579264, host-profile: armv7 }
- { os: ubuntu-22.04, compiler: ndk-26.3.11579264, host-profile: x86 }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure yet if Android makes sense here but I would say at maximum one? Because otherwise we launch a very big amount of jobs.

I think compiler should stick with the clang rather than ndk-26.3.11579264. I guess we could have build OS and host OS to make this a bit cleaner?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well it depends on how we want to use this odr-index repo.

Small matrix here means faster builds here and our artifactory is more like a recipe index, but this means slow builds for downstream users, because most of the time, downstream users would have to build these conan packages from source.

Big matrix here means slower builds here, but fast builds for downstream users, because downstream users would be able to find binary packages that were built here. Mind you, downstream users (odr.droid) need binaries from all 4 Android ABIs.

How is artifactory charging us for the repository? Does the amount of binaries that we upload have an actual cost? Or is it in gigabytes and we don't actually have to care? Big matrix here does not have that much of an impact on GitHub side, the repo is open source, so we don't have to worry about GitHub charges, only the run time, but if all the dependencies are already available as binaries, each of them wouldn't take that long, it's not in the hours as it was when I was building pdf2htmlEX dependencies from source each time :D

As for the compiler, clang and ndk-26.3.11579264, I don't really care about the actual naming, what I need is a way to specify build and host profiles. And I also need some kind of a variable to specify NDK version, because NDK needs to be installed manually

steps:
- name: checkout
uses: actions/checkout@v4
Expand All @@ -64,6 +68,13 @@ jobs:
brew install ccache
ccache -V

- name: install ndk
if: startsWith(matrix.config.compiler, 'ndk-')
run: echo "y" | sudo ${ANDROID_HOME}/cmdline-tools/latest/bin/sdkmanager --install `echo "${{ matrix.config.compiler }}" | tr - ";"`
- name: expand ANDROID_HOME env var in conan profile
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couldn't find a way to put $ANDROID_HOME in conan profile. Don't want to hardcode the path because it would be unusable on my development machine

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we had a similar problem in the android repo. Hardcoding would be last resort but I don't think it is terrible because it is an github specific profile. Locally we could emulate the github ci with https://github.com/nektos/act

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I should file an enhancement request to conan, because I don't believe we are the only ones who want to put $ANDROID_HOME env var in conan profile

if: startsWith(matrix.config.compiler, 'ndk-')
run: perl -pi -e 's/\$ANDROID_HOME/$ENV{ANDROID_HOME}/g' .github/config/${{ matrix.config.os }}-${{ matrix.config.compiler }}/conan/profiles/host

- name: setup python 3.12
uses: actions/setup-python@v5
with:
Expand All @@ -89,20 +100,30 @@ jobs:
${{ matrix.config.os }}-${{ matrix.config.compiler }}-

- name: conan install
run: conan install ${{ matrix.package.conanfile }} --version ${{ matrix.package.version }} --build missing
run: conan install ${{ matrix.package.conanfile }} --version ${{ matrix.package.version }} --build missing --profile:host ${{ matrix.config.host-profile }}

- name: conan graph
if: always()
run: conan graph info ${{ matrix.package.conanfile }} --version ${{ matrix.package.version }} --format=html --profile:host ${{ matrix.config.host-profile }} > graph.html
- uses: actions/upload-artifact@v4
if: always()
with:
name: graph.html.${{ matrix.package.package }}-${{ matrix.package.version }}-${{ matrix.config.os }}-${{ matrix.config.compiler }}-${{ matrix.config.host-profile }}
path: graph.html

- name: conan source
run: conan source ${{ matrix.package.conanfile }} --version ${{ matrix.package.version }}
- name: conan build
run: conan build ${{ matrix.package.conanfile }} --version ${{ matrix.package.version }}
run: conan build ${{ matrix.package.conanfile }} --version ${{ matrix.package.version }} --profile:host ${{ matrix.config.host-profile }}
- name: conan export
run: conan export ${{ matrix.package.conanfile }} --version ${{ matrix.package.version }}
- name: conan export-pkg
run: conan export-pkg ${{ matrix.package.conanfile }} --version ${{ matrix.package.version }}
run: conan export-pkg ${{ matrix.package.conanfile }} --version ${{ matrix.package.version }} --profile:host ${{ matrix.config.host-profile }}

- name: conan test
run: |
test_conanfile=$(dirname ${{ matrix.package.conanfile }})/test_package/conanfile.py
conan test $test_conanfile ${{ matrix.package.package_version }} --build missing
conan test $test_conanfile ${{ matrix.package.package_version }} --build missing --profile:host ${{ matrix.config.host-profile }}

- name: conan upload
if: github.ref == 'refs/heads/main' || github.event_name == 'workflow_dispatch'
Expand Down
Loading