Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

github actions for building and testing - no commit history #8

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions .github/workflows/cmake.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# Build the library without tests
name: Build

on:
push:
branches: [ "master" ]
pull_request:
branches: [ "master" ]

env:
# Customize the CMake build type here (Release, Debug, RelWithDebInfo, etc.)
BUILD_TYPE: Release

jobs:
build:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v3

- name: Configure CMake
run: |
cmake -B ${{github.workspace}}/build \
-DCMAKE_BUILD_TYPE=${{env.BUILD_TYPE}} -DNF9_BUILD_TESTS=OFF

- name: Build
# Build your program with the given configuration
run: |
cmake --build ${{github.workspace}}/build \
--config ${{env.BUILD_TYPE}}
46 changes: 46 additions & 0 deletions .github/workflows/tests-own.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# Build tests using external libraries referencd by the project
name: TestsOwnLibs

on:
push:
branches: [ "master" ]
pull_request:
branches: [ "master" ]

env:
# Customize the CMake build type here (Release, Debug, RelWithDebInfo, etc.)
BUILD_TYPE: Debug

jobs:
build:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v3

- name: Install libpcap
run: sudo apt-get install -y libpcap-dev

- name: Initialize submodules
run: git submodule init && git submodule update

- name: Build libtins library
run: |
cd ${{github.workspace}}/external/libtins && mkdir build && \
cd build && cmake .. -DLIBTINS_ENABLE_CXX11=1 && cmake --build .
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Marking submodule to fixed commit would be nice IMHO. It might be a little more painful to switch any external library to newer version but won't cause pipeline failures when something go wrong in dependencies.

Anyway I don't get how own version of googletest is installed - git submodule update handles it? It's a bit implicit for me tbh.


- name: Configure CMake
run: |
cmake -B ${{github.workspace}}/build \
-DCMAKE_BUILD_TYPE=${{env.BUILD_TYPE}} -DNF9_BUILD_TESTS=ON \
-DNF9_USE_OWN_LIBTINS=ON -DNF9_USE_OWN_GTEST=ON

- name: Build Tests
run: |
cmake --build ${{github.workspace}}/build \
--config ${{env.BUILD_TYPE}}

- name: Test
working-directory: ${{github.workspace}}/build
run: ./test/netflowtests
# ctest -C ${{env.BUILD_TYPE}}
44 changes: 44 additions & 0 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# Build tests using the libraries installed in system
name: TestsSysLibs

on:
push:
branches: [ "master" ]
pull_request:
branches: [ "master" ]

env:
# Customize the CMake build type here (Release, Debug, RelWithDebInfo, etc.)
BUILD_TYPE: Debug

jobs:
build:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v3

- name: Install libpcap
run: sudo apt-get install -y libpcap-dev

- name: Install googletest
run: sudo apt-get install -y libgtest-dev

- name: Install libtins
run: sudo apt-get install -y libtins-dev
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We probably may install all dependencies in one step:
sudo apt-get install -y libpcap-dev libgtest-dev libtins-dev


- name: Configure CMake
run: |
cmake -B ${{github.workspace}}/build \
-DCMAKE_BUILD_TYPE=${{env.BUILD_TYPE}} -DNF9_BUILD_TESTS=ON

- name: Build Tests
# Build your program with the given configuration
run: |
cmake --build ${{github.workspace}}/build \
--config ${{env.BUILD_TYPE}}

- name: Test
working-directory: ${{github.workspace}}/build
run: ./test/netflowtests
# ctest -C ${{env.BUILD_TYPE}}
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,4 @@
/clang_build/
__pycache__
ENV
.vscode/
3 changes: 3 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[submodule "external/libtins"]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it really better than using a system library?
Also, is a submodule better than, e.g. a subtree? Personally I always found git-submodules difficult to use.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

System library is good, provided it is there. It happens to be the case on Ubuntu - github runner, so we can switch back to it there. On RedHat/CentOS/Fedora derivatives it may not be the case. I tried it on AWS Linux 2 (ID_LIKE="centos rhel fedora") and Oracle Linux (fedora) and needed to compile libtins myself, hence referencing it as a submodule. You may need to get used to using submodules, but they are based on healthy principles. You keep just a reference to the dependant repository, so there is no code duplication. Actually, libtins library itself is using googletest as a submodule https://github.com/mfontanini/libtins/blob/master/.gitmodules .

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should still work in airgapped environment or via an internal cache (source of truth for this repository up to this PR was internal company Bitbucket. There's sometimes limited access to gh too. It's nice idea to have it build easily on RH-like distros, we'd just need to make certain it still works in current envs - and/or review policies.

Also, probably this should reference a specific tag/commit which is known to work?

path = external/libtins
url = https://github.com/mfontanini/libtins.git
12 changes: 11 additions & 1 deletion CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ option(NF9_FUZZ "Enable fuzzing with LLVM fuzzer" OFF)
option(NF9_BUILD_BENCHMARK
"If set, build benchmarks (requires google benchmark library)" OFF)
option(NF9_BUILD_EXAMPLES "If set, build examples" OFF)
option(NF9_USE_OWN_LIBTINS "If set, use the libtins in the submodule" OFF)
option(NF9_USE_OWN_GTEST "If set, use googtest library from github" OFF)

include(CheckCXXCompilerFlag)
include(CheckIncludeFileCXX)
Expand All @@ -29,7 +31,15 @@ else ()
add_library(netflow9 STATIC ${CXX_SRC})
endif ()

target_include_directories(netflow9 SYSTEM PUBLIC include)
if (NOT NF9_USE_OWN_LIBTINS)
target_include_directories(netflow9 SYSTEM PUBLIC include)
else ()
get_property(SRC_DIR DIRECTORY PROPERTY SOURCE_DIR)
target_include_directories(netflow9 SYSTEM PUBLIC include
${SRC_DIR}/external/libtins/include/
)
endif()

target_compile_features(netflow9 PRIVATE cxx_std_17)

if (MSVC)
Expand Down
31 changes: 30 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,18 @@ information about the traffic.

libnetflow9 is written in C++17, and has a compatible C API.

# Building #
## Badges ##

### Building ###

![build workflow](https://github.com/doodeck/libnetflow9/actions/workflows/cmake.yml/badge.svg)

### Testing ###

![test workflow sys](https://github.com/doodeck/libnetflow9/actions/workflows/tests.yml/badge.svg)

![test workflow own](https://github.com/doodeck/libnetflow9/actions/workflows/tests-own.yml/badge.svg)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't those links link to the exatel/libnetflow9 repo?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent point! Ultimately - they should. As long as the changes exist only in a pull-request we will probably see an empty status. This is a chicken - egg problem, I guess, I should change it right now.



## Dependencies ##

Expand All @@ -33,6 +44,24 @@ cmake ..
make -j4
```

## Building on MacOS M1

```
mkdir build
cd build
cmake .. -DCMAKE_C_COMPILER=/opt/homebrew/bin/gcc-12 -DCMAKE_CXX_COMPILER=/opt/homebrew/bin/g++-12
cmake --build .
```

Otherwise it defaults to Clang toolset, which as of version:

`Apple clang version 14.0.0 (clang-1400.0.29.202)`

is incapable of compiling the library. More details in the build log

https://github.com/doodeck/libnetflow9/actions/runs/4681045806/jobs/8293144058


## Building and running tests ##

```console
Expand Down
28 changes: 28 additions & 0 deletions external/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# external libraries sources directory

The external libraries are not used by default, instead it's expected they
are installed system-wide. System libraries have advantages, e.g. you can
rely on them in an airgapped environment. On the other hand, they are missing
from some distributions.

In order to activate the external libraries set to ON the correspoding cmake
option[-s]:

* `NF9_USE_OWN_LIBTINS`
* `NF9_USE_OWN_GTEST`

## libtins
https://github.com/mfontanini/libtins.git

Incorporated as a submodule as described here:

https://git-scm.com/book/en/v2/Git-Tools-Submodules

## googtest
pulled directly from github release as recommended here:

https://github.com/google/googletest/blob/main/googletest/README.md

Overthere see the description below the last bullet point "Use CMake
to download GoogleTest as part of the build's configure step. This approach
doesn't have the limitations of the other methods."
1 change: 1 addition & 0 deletions external/libtins
Submodule libtins added at fa87e1
1 change: 1 addition & 0 deletions src/types.h
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
#include <netinet/in.h>
#include <iostream>
#include <mutex>
#include <memory> // unique_ptr

#include "config.h"

Expand Down
33 changes: 29 additions & 4 deletions test/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -1,9 +1,35 @@
cmake_minimum_required(VERSION 3.7)

enable_testing()
find_package(GTest REQUIRED)
if (NOT NF9_USE_OWN_GTEST)
find_package(GTest REQUIRED)
list (APPEND GTEST_LIBRARIES
GTest::GTest
GTest::Main
)
else()
include(FetchContent)
FetchContent_Declare(
googletest
# Specify the commit you depend on and update it regularly.
URL https://github.com/google/googletest/archive/refs/tags/v1.13.0.zip
)
# For Windows: Prevent overriding the parent project's
# compiler/linker settings
set(gtest_force_shared_crt ON CACHE BOOL "" FORCE)
FetchContent_MakeAvailable(googletest)
list (APPEND GTEST_LIBRARIES
gtest_main
)
endif()

find_library(LIBTINS_LIBRARIES tins)
if (NOT NF9_USE_OWN_LIBTINS)
find_library(LIBTINS_LIBRARIES tins)
else()
get_property(SRC_DIR DIRECTORY PROPERTY SOURCE_DIR)
find_library(LIBTINS_LIBRARIES tins
${SRC_DIR}/../external/libtins/build/lib/)
endif()

file(GLOB src *.cpp)
add_executable(netflowtests ${src})
Expand All @@ -17,8 +43,7 @@ target_include_directories(netflowtests PRIVATE
)
target_link_libraries(netflowtests
netflow9
GTest::GTest
GTest::Main
"${GTEST_LIBRARIES}"
"${LIBTINS_LIBRARIES}"
)

Expand Down
8 changes: 8 additions & 0 deletions test/memory-stress-test/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,18 @@ add_executable(netflow-memory-stress test.cpp)
add_test(netflow-memory-stress netflow-memory-stress)

target_compile_features(netflow-memory-stress PRIVATE cxx_std_17)

if (NF9_USE_OWN_GTEST)
list (APPEND GTEST_INCLUDES
"${googletest_SOURCE_DIR}/googletest/include"
)
endif ()

target_include_directories(netflow-memory-stress PRIVATE
"../../src"
"../"
"${CMAKE_CURRENT_BINARY_DIR}/../.."
"${GTEST_INCLUDES}"
)
target_link_libraries(netflow-memory-stress
netflow9
Expand Down