Skip to content

Commit

Permalink
Btc master updates 01/08/2022 (#51)
Browse files Browse the repository at this point in the history
* rpc, refactor: Add `getblock_prevout`

This change eliminates memory usage spike when compiling with Visual
Studio 2022 (at least in Cirrus CI environment).

Easy to review using
`git diff --color-moved-ws=allow-indentation-change --color-moved=dimmed-zebra`

* rpc, refactor: Add `decodepsbt_inputs`

This change eliminates memory usage spike when compiling with Visual
Studio 2022 (at least in Cirrus CI environment).

Easy to review using
`git diff --color-moved-ws=allow-indentation-change --color-moved=dimmed-zebra`

* rpc, refactor: Add `decodepsbt_outputs`

This change eliminates memory usage spike when compiling with Visual
Studio 2022 (at least in Cirrus CI environment).

Easy to review using
`git diff --color-moved-ws=allow-indentation-change --color-moved=dimmed-zebra`

* build: Increase MS Visual Studio minimum version

Visual Studio 2022 with `/std:c++20` supports designated initializers.

* refactor: Drop no longer needed `util/designator.h`

* Remove my key from trusted-keys

* refactor: add most of src/util to iwyu

These files change infrequently, and not much header shuffling is required.

We don't add everything in src/util/ yet, because IWYU makes some
dubious suggestions, which I'm going to follow up with upstream.

* Introduce generic 'Result' class

Useful to encapsulate the function result object (in case of having it) or, in case of failure, the failure reason.

This let us clean lot of boilerplate code, as now instead of returning a boolean and having to add a ref arg for the
return object and another ref for the error string. We can simply return a 'BResult<Obj>'.

Example of what we currently have:
```
bool doSomething(arg1, arg2, arg3, arg4, &result, &error_string) {
    do something...
    if (error) {
        error_string = "something bad happened";
        return false;
    }

    result = goodResult;
    return true;
}
```

Example of what we will get with this commit:
```
BResult<Obj> doSomething(arg1, arg2, arg3, arg4) {
    do something...
    if (error) return {"something happened"};

    // good
    return {goodResult};
}
```

This allows a similar boilerplate cleanup on the function callers side as well. They don't have to add the extra
pre-function-call error string and result object declarations to pass the references to the function.

* wallet: refactor, include 'FeeCalculation' inside 'CreatedTransactionResult'

* send: refactor CreateTransaction flow to return a BResult<CTransactionRef>

* wallet: refactor GetNewDestination, use BResult

* test: refactor: pass absolute fee in `create_lots_of_big_transactions` helper

* doc: update the URLs to thread functions in developer-notes

ThreadMapPort() does not appear on doxygen.bitcoincore.org
because it is inside `#ifdef`.

* test: speedup wallet_coinbase_category.py

No need to create a chain for it (nor use the cache).

* wallet: Precompute Txdata after setting PSBT inputs' UTXOs

If we are given a PSBT that is missing one or more input UTXOs, our
PrecomputedTransactionData will be incorrect and missing information
that it should otherwise have, and therefore we may not produce a
signature when we should. To avoid this problem, we can do the
precomputation after we have set the UTXOs the wallet is able to set for
the PSBT.

Also adds a test for this behavior.

* Address comments remaining from bitcoin#25353

* move-only: Version handshake to libtest_util

* move-only: InitializeNode to handshake helper

* [test] persist prioritisation of transactions not in mempool

* wallet: change `ScanForWalletTransactions` to use `Ticks()`

* scripted-diff: [test] Rename BIP125_SEQUENCE_NUMBER to MAX_BIP125_RBF_SEQUENCE

-BEGIN VERIFY SCRIPT-
 sed -i 's:BIP125_SEQUENCE_NUMBER:MAX_BIP125_RBF_SEQUENCE:g' $(git grep -l BIP125_SEQUENCE_NUMBER ./test)
-END VERIFY SCRIPT-

* test: Remove duplicate MAX_BIP125_RBF_SEQUENCE constant

* Prepare BResult for non-copyable types

* refactor: Return BResult from restoreWallet

* Remove atomic for m_last_getheaders_timestamp

This variable is only used in a single thread, so no atomic or mutex is
necessary to guard it.

* test: remove unnecessary parens

* test/mempool_persist: Test manual savemempool when -persistmempool=0

* depends: update urls for dmg tools

These repos have migrated from https://github.com/al45tair/ to
https://github.com/dmgbuild/, so update our URLs to point to the new
location. Note that GitHub is also already performing the redirect
automatically.

* Expose underlying clock in CThreadInterrupt

Overloading sleep_for is not needed, as

* seconds and minutes can be converted to milliseconds by the compiler,
  not needing a duration_cast
* std::condition_variable::wait_for will convert milliseconds to the
  duration type of the underlying clock

So simply expose the clock.

* refactor: Make FEELER_SLEEP_WINDOW type safe (std::chrono)

* refactor: Default options in walletcreatefundedpsbt to VOBJ instead of VNULL

This should not change behavior and makes the code consistent with other
places.

* univalue: Throw exception on invalid pushes over silent ignore

* rpc: Select int-UniValue constructor for enum value in upgradewallet RPC

UniValue does not have a constructor for enum values, however the
compiler will decay the enum into an int and select that constructor.
Avoid this compiler magic and clarify the code by explicitly selecting
the int-constructor.

This is needed for the next commit.

* miniscript: don't check for top level validity at parsing time

Letting the caller perform the checks allows for finer-grained error
reporting.

* miniscript: add a helper to find the first insane sub with no child

This is helpful for finer grained descriptor parsing error: when there
are multiple errors to report in a Miniscript descriptor start with the
"smallest" fragments: the ones closer to be a leaf.

Co-Authored-By: Pieter Wuille <[email protected]>

* qa: better error reporting on descriptor parsing error

A nit, but was helpful when writing unit tests for Miniscript parsing

* Miniscript support in output descriptors

Miniscript descriptors are defined under P2WSH context (either `wsh()`
or `sh(wsh())`).
Only sane Miniscripts are accepted, as insane ones (although valid by
type) can have surprising behaviour with regard to malleability
guarantees and resources limitations.
As Miniscript descriptors are longer and more complex than "legacy"
descriptors, care was taken in error reporting to help a user determine
for what reason a provided Miniscript is insane.

Co-authored-by: Pieter Wuille <[email protected]>

* qa: functional test Miniscript watchonly support

* univalue: Avoid narrowing and verbose int constructors

As UniValue provides several constructors for integral types, the
compiler is unable to select one if the passed type does not exactly
match. This is unintuitive for developers and forces them to write
verbose and brittle code.

For example, there are many places where an unsigned int is cast to a
signed int. While the cast is safe in practice, it is still needlessly
verbose and confusing as the value can never be negative. In fact it
might even be unsafe if the unsigned value is large enough to map to a
negative signed one.

* Move ChainstateManagerOpts into kernel:: namespace

It should have been there in the first place.

* Use designated initializers for ChainstateManager::Options

This wasn't available at the time when ChainstateManager::Options was
introduced but is helpful to be explicit and ensure correctness.

* [net processing] Add m_our_services and m_their_services to Peer

Track services offered by us and the peer in the Peer object.

* [tests] Connect peer in outbound_slow_chain_eviction by sending p2p messages

Prior to this commit, the peer was connected, and then the services and
connectivity fields in the CNode object were manually set. Instead, send
p2p `version` and `verack` messages, and have net_processing's internal
logic set the state of the node.

This ensures that the node's internal state is consistent with how it
would be set in the live code.

Prior to this commit, `dummyNode1.nServices` was set to `NODE_NONE`
which was not a problem since `CNode::fClient` and
`CNode::m_limited_node` are default initialised to false. Now that we
are doing the actual version handshake, the values of `fClient` and
`m_limited_node` are set during the handshake and cause the test to fail
if we do not set `dummyNode1.nServices` to a reasonable value
(NODE_NETWORK | NODE_WITNESS).

* [net processing] Remove fClient and m_limited_node

fClient is replaced by CanServeBlocks(), and m_limited_node is replaced
by IsLimitedPeer().

* [net processing] Replace fHaveWitness with CanServeWitnesses()

* [net processing] Remove CNode::nServices

Use Peer::m_their_services instead

* [net] Return CService from GetLocalAddrForPeer and GetLocalAddress

* [net processing] Remove CNode::nLocalServices

* fix gettxout help text

* Release notes for Miniscript support in P2WSH descriptors

* DumpMempool: Use std::chrono instead of weird int64_t arthmetics

This makes it so that DumpMempool doesn't depend on MICRO anymore

* DumpMempool: Pass in dump_path, stop using gArgs

Also introduce node::{ShouldPersistMempool,MempoolPath} helper functions
in node/mempool_persist_args.{h,cpp} which are used by non-kernel
DumpMempool callers to determine whether or not to automatically dump
the mempool and where to dump it to.

* scripted-diff: Rename m_is_loaded -> m_load_tried

m_is_loaded/IsLoaded() doesn't actually indicate whether or not the
mempool was successfully, loaded, but rather if a load has been
attempted and did not result in a catastrophic ShutdownRequested.

-BEGIN VERIFY SCRIPT-
find_regex="\bm_is_loaded\b" \
    && git grep -l -E "$find_regex" \
        | xargs sed -i -E "s@$find_regex@m_load_tried@g"

find_regex="\bIsLoaded\b" \
    && git grep -l -E "$find_regex" \
        | xargs sed -i -E "s@$find_regex@GetLoadTried@g"

find_regex="\bSetIsLoaded\b" \
    && git grep -l -E "$find_regex" \
        | xargs sed -i -E "s@$find_regex@SetLoadTried@g"
-END VERIFY SCRIPT-

* mempool: Improve comments for [GS]etLoadTried

Also change the param name for SetLoadTried to load_tried.

* mempool: Use NodeClock+friends for LoadMempool

* Disallow encryption of watchonly wallets

Watchonly wallets do not have any private keys to encrypt. It does not
make sense to encrypt such wallets, so disable the option to encrypt
them.

This avoids an assertion that can be hit when encrypting watchonly descriptor
wallets.

* Move FopenFn to fsbridge namespace

[META] In a future commit in this patchset, it will be used by more than
       just validation, and it needs to align with fopen anyway.

* test/fuzz: Invoke LoadMempool via CChainState

Not only does this increase coverage, it is also more correct in that
when ::LoadMempool is called with a mempool and chainstate, it calls
AcceptToMemoryPool with just the chainstate.

AcceptToMemoryPool will then act on the chainstate's mempool via
CChainState::GetMempool, which may be different from the mempool
originally passed to ::LoadMempool. (In this fuzz test's case, it
definitely is different)

Also, move DummyChainstate to its own file since it's now used by the
validation_load_mempool fuzz test to replace CChainState's m_mempool.

* LoadMempool: Pass in load_path, stop using gArgs

Also:
1. Have CChainState::LoadMempool and ::ThreadImport take in paths and
   pass it through untouched to LoadMempool.
2. Make LoadMempool exit early if the load_path is empty.
3. Adjust the call to ::ThreadImport in ::AppInitMain to correctly pass
   in an empty path if mempool persistence is disabled.

* Move DEFAULT_PERSIST_MEMPOOL out of libbitcoinkernel

It is no longer used by anything inside libbitcoinkernel, move it to
node/mempool_persist_args.h where it belongs.

* Move {Load,Dump}Mempool to kernel namespace

Also:
1. Add the newly introduced kernel/mempool_persist.cpp to IWYU CI script
2. Add chrono mapping for iwyu

* build: pass -fno-lto when building expat

Otherwise it's autoconf endianess check will fail to determine what the
endianess is..

* build: Use Link Time Optimization for Qt code on Linux

See: https://www.qt.io/blog/2019/01/02/qt-applications-lto

* fuzz: Fix assert bug in txorphan target

* doc: remove references to downstream

Having references to downstream no-longer make sense now that we've
unsubtree'd.

* refactor: integrate no_nul into univalue unitester

* refactor: remove BOOST_*_TEST_ macros

* move-only: Move UniValue::getInt definition to keep class with definitions only

Can be reviewed with the git options

--color-moved=dimmed-zebra --color-moved-ws=ignore-all-space

* univalue: Return more detailed type check error messages

* Add symlinks for hardcoded Makefiles in out of tree builds

* build: Check for std::atomic::exchange rather than std::atomic_exchange

Our usage of std::atomic is with it's own exchange function, not
std::atomic_exchange. So we should be looking specifically for that
function.

Additionally, -pthread and -lpthread have an effect on whether -latomic
will be needed, so the atomics check needs to use these flags as well.
This will make the flags in use better match what is actually used when
linking.

This removes the need for -latomic for riscv builds, which resolves a
guix cross architecture reproducibility issue.

* build: Fix autoconf variable names for tools found by `AC_PATH_TOOL`

See the `AC_PATH_TOOL` macro implementation.

* depends: default to using GCC tool wrappers (with GCC)

This improves support for LTO by using gcc wrappers for ar, nm, ranlib,
that correctly setup plugin arguments for LTO.

Other HOSTS are using clang.

* validation: remove unused using directives

The following were unused from the node namespace:
- BLOCKFILE_CHUNK_SIZE
- nPruneTarget
- OpenBlockFile
- UNDOFILE_CHUNK_SIZE

* refactor: remove unused using directives

* tidy: use misc-unused-using-decls

https://clang.llvm.org/extra/clang-tidy/checks/misc/unused-using-decls.html

* refactor: Make mapBlocksUnknownParent local, and rename it

Co-authored-by: Larry Ruane <[email protected]>

* interfaces, refactor: Add more block information to block connected notifications

Add new interfaces::BlockInfo struct to be able to pass extra block
information (file and undo information) to indexes which they are
updated to use high level interfaces::Chain notifications.

This commit does not change behavior in any way.

* indexes, refactor: Pass Chain interface instead of CChainState class to indexes

Passing abstract Chain interface will let indexes run in separate
processes.

This commit does not change behavior in any way.

* indexes, refactor: Remove CBlockIndex* uses in coinstatsindex LookUpOne function

This commit does not change behavior in any way.

* indexes, refactor: Remove CBlockIndex* uses in index Init methods

Replace overriden index Init() methods that use the best block
CBlockIndex* pointer with pure CustomInit() callbacks that are passed
the block hash and height.

This gets rid of more CBlockIndex* pointer uses so indexes can work
outside the bitcoin-node process. It also simplifies the initialization
call sequence so index implementations are not responsible for
initializing the base class.

There is a slight change in behavior here since now the best block
pointer is loaded and checked before the custom index init functions are
called instead of while they are called.

* indexes, refactor: Remove CBlockIndex* uses in index WriteBlock methods

Replace WriteBlock method with CustomAppend and pass BlockInfo struct
instead of CBlockIndex* pointer

This commit does not change behavior in any way.

* indexes, refactor: Remove CBlockIndex* uses in index Rewind methods

Replace Rewind method with CustomRewind and pass block hashes and
heights instead of CBlockIndex* pointers

This commit does not change behavior in any way.

* indexes, refactor: Remove CChainState use in index CommitInternal method

Replace CommitInternal method with CustomCommit and use interfaces::Chain
instead of CChainState to generate block locator.

This commit does not change behavior in any way, except in the
(m_best_block_index == nullptr) case, which was added recently in
bitcoin#24117 as part of an ongoing attempt to
prevent index corruption if bitcoind is interrupted during startup. New
behavior in that case should be slightly better than the old behavior (skipping
the entire custom+base commit now vs only skipping the base commit previously)
and this might avoid more cases of corruption.

* refactor: Use chainman() helper consistently in ChainImpl

* guix: Drop repetition of option's default value

* Fix `-Wparentheses` gcc warning

* depends: modify FastFixedDtoa optimisation flags

This fixes a non-determinism issue in the asm produced for
this function when cross-compiling on x86_64 and aarch64 for
the arm-linux-gnueabihf HOST.

Related to bitcoin#21194.

* Add missing includes to node/chainstate

This is needed for the next commit

* Add missing includes

They are needed, otherwise the next commit will not compile

* Remove unused includes from dbwrapper.h

* Remove unused includes in rpc/fees.cpp

IWYU confirms that they are unused

* refactor: store by OutputType in CoinsResult

Store COutputs by OutputType in CoinsResult.

The struct stores vectors of `COutput`s by `OutputType`
for more convenient access

* refactor: use CoinsResult struct in SelectCoins

Pass the whole CoinsResult struct to SelectCoins instead of only a
vector. This means we now have to remove preselected coins from each
OutputType vector and shuffle each vector individually.

Pass the whole CoinsResult struct to AttemptSelection. This involves
moving the logic in AttemptSelection to a newly named function,
ChooseSelectionResult. This will allow us to run ChooseSelectionResult
over each OutputType in a later commit. This ensures the backoffs work
properly.

Update unit and bench tests to use CoinResult.

* scripted-diff: rename `FromBinary` helper to `from_binary` (signet miner)

-BEGIN VERIFY SCRIPT-
sed -i s/FromBinary/from_binary/g ./contrib/signet/miner
-END VERIFY SCRIPT-

* refactor: move `from_binary` helper from signet miner to test framework

Can be easily reviewed with `--color-moved=dimmed-zebra`.

* refactor: move PSBT(Map) helpers from signet miner to test framework

Can be easily reviewed with `--color-moved=dimmed-zebra`.

* test: add constants for PSBT key types (BIP 174)

Also take use of the constants in the signet miner to get rid of
magic numbers and increase readability and maintainability.

* refactor: move helper `random_bytes` to util library

Can be easily reviewed with `--color-moved=dimmed-zebra`.

* test: add test for decoding PSBT with per-input preimage types

* wallet: run coin selection by `OutputType`

Run coin selection on each OutputType separately, choosing the best
solution according to the waste metric.

This is to avoid mixing UTXOs that are of different OutputTypes,
which can hurt privacy.

If no single OutputType can fund the transaction, then coin selection
considers the entire wallet, potentially mixing (current behavior).

This is done inside AttemptSelection so that all OutputTypes are
considered at each back-off in coin selection.

* test: functional test for new coin selection logic

Create a wallet with mixed OutputTypes and send a volley of payments,
ensuring that there are no mixed OutputTypes in the txs. Finally,
verify that OutputTypes are mixed only when necessary.

* test: add unit test for AvailableCoins

test that UTXOs are bucketed correctly after
running AvailableCoins

* refactor: Reduce number of LoadChainstate parameters

* refactor: Reduce number of LoadChainstate return values

* refactor: Reduce number of SanityChecks return values

* ci: Enable IWYU in src/kernel directory

Suggested bitcoin#25308 (comment)

* refactor: move compat.h into compat/

* compat: document FD_SETSIZE redefinition for WIN32

* compat: remove unused WSA* definitions

* compat: extract and document MAX_PATH

* compat: document S_I* defines when building for Windows

* compat: document sockopt_arg_type definition

* compat: document error-code mapping

See:
https://docs.microsoft.com/en-us/windows/win32/winsock/windows-sockets-error-codes-2

* compat: document redefining ssize_t when using MSVC

See:
https://docs.microsoft.com/en-us/windows/win32/winprog/windows-data-types#ssize_t
https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/sys_types.h.html

* Add HashWriter without ser-type and ser-version

The moved parts can be reviewed with "--color-moved=dimmed-zebra".

* Use HashWriter where possible

* ci: better pin to dwarf4 in valgrind job

Use `-gdwarf` and also set CFLAGS. I was seeing Valgrind issues otherwise.

* contrib: remove unneeded valgrind suppressions

* doc: BaseIndex sync behavior with empty datadir

Make a note about a potentially confusing behavior with `BaseIndex::m_synced`;
if the user starts bitcoind with an empty datadir and an index enabled,
BaseIndex will consider itself synced (as a degenerate case). This affects
how indices are built during IBD (relying solely on BlockConnected signals vs.
using ThreadSync()).

* refactor: Fix iwyu on node/chainstate

* CBlockIndex: ensure phashBlock is not nullptr before dereferencing

and remove a now-redundant assert preceding a GetBlockHash() caller.

This protects against UB here, and in case of failure (which would
indicate a consensus bug), the debug log will print

bitcoind: chain.h:265: uint256 CBlockIndex::GetBlockHash() const: Assertion `phashBlock != nullptr' failed.
Aborted

instead of

Segmentation fault

* CDiskBlockIndex: remove unused ToString() class member

and mark its inherited CBlockIndex#ToString public interface member
as deleted, to disallow calling it in the derived CDiskBlockIndex class.

* CDiskBlockIndex: rename GetBlockHash() to ConstructBlockHash()

and mark the inherited CBlockIndex#GetBlockHash public interface member
as deleted, to disallow calling it in the derived CDiskBlockIndex class.

Here is a failing test on master demonstrating the inconsistent behavior of the
current design: calling the same inherited public interface functions on the
same CDiskBlockIndex object should yield identical behavior.

```diff
diff --git a/src/test/validation_chainstatemanager_tests.cpp b/src/test/validation_chainstatemanager_tests.cpp
index 6dc522b421..dac3840f32 100644
--- a/src/test/validation_chainstatemanager_tests.cpp
+++ b/src/test/validation_chainstatemanager_tests.cpp
@@ -240,6 +240,15 @@ BOOST_FIXTURE_TEST_CASE(chainstatemanager_activate_snapshot, TestChain100Setup)

     const CBlockIndex* tip = chainman.ActiveTip();

     BOOST_CHECK_EQUAL(tip->nChainTx, au_data.nChainTx);

+    // CDiskBlockIndex "is a" CBlockIndex, as it publicly inherits from it.
+    // Test that calling the same inherited interface functions on the same
+    // object yields identical behavior.
+    CDiskBlockIndex index{tip};
+    CBlockIndex *pB = &index;
+    CDiskBlockIndex *pD = &index;
+    BOOST_CHECK_EQUAL(pB->GetBlockHash(), pD->GetBlockHash());
+    BOOST_CHECK_EQUAL(pB->ToString(), pD->ToString());
+
```

The GetBlockHash() test assertion only passes on master because the different
methods invoked by the current design happen to return the same result.  If one
of the two is changed, it fails like the ToString() assertion does.

Redefining inherited non-virtual functions is well-documented as incorrect
design to avoid inconsistent behavior (see Scott Meyers, "Effective C++", Item
36).  Class usage is confusing when the behavior depends on the pointer
definition instead of the object definition (static binding happening where
dynamic binding was expected).  This can lead to unsuspected or hard-to-track
bugs.

Outside of critical hot spots, correctness usually comes before optimisation,
but the current design dates back to main.cpp and it may possibly have been
chosen to avoid the overhead of dynamic dispatch.  This solution does the same:
the class sizes are unchanged and no vptr or vtbl is added.

There are better designs for doing this that use composition instead of
inheritance or that separate the public interface from the private
implementations.  One example of the latter would be a non-virtual public
interface that calls private virtual implementation methods, i.e. the Template
pattern via the Non-Virtual Interface (NVI) idiom.

* refactor: move CBlockIndex#ToString() from header to implementation

which allows dropping tinyformat.h from the header file.

* gui: Fix translator comment for Restore Wallet QInputDialog

This also changes the window title name
from `Restore Name` to `Restore Wallet`.

* test: support passing PSBTMaps directly to PSBT ctor

This will allow to create simple PSBTs as short one-liners, without the
need to have three individual assignments (globals, inputs, outputs).

* test: check that combining PSBTs with different txs fails

* fuzz: Remove no-op SetMempoolConstraints

* Bugfix: RPC/blockchain: Correct type of "value" in getblock docs; add missing "desc"

* RPC: Document "asm" and "hex" fields for scripts

* test: remove unused if statements

* refactor: Make CTransaction constructor explicit

It involves calculating two hashes, so the performance impact should be
made explicit.

Also, add the module to iwyu.

* fuzz: refactor: Replace NullUniValue with UniValue{}

This is needed for the scripted-diff to compile in the next commit

* scripted-diff: Replace NullUniValue with UniValue::VNULL

This is required for removing the UniValue copy constructor.

-BEGIN VERIFY SCRIPT-
 sed -i 's/return NullUniValue/return UniValue::VNULL/g' $(git grep -l NullUniValue ':(exclude)src/univalue')
-END VERIFY SCRIPT-

* psbt: Fix unsigned integer overflow

* fix comment spellings from the codespell lint

test/lint/all-lint.py includes the codespell lint

* depends: always use correct ar for win qt

If we don't set this explicitly, then qt will still use it's default
windows ar, when building with LTO (when we want it to use gcc-ar).

So set `QMAKE_LIB` which is used for win32, and defaults to `ar -rc`.
This way we always get the correct ar.

Issue can be seen building in Guix with LTO. i.e:
```bash
x86_64-w64-mingw32-ar: .obj/release/hb-blob.o: plugin needed to handle lto object
```

* refactor: Remove not needed std::max

* scripted-diff: Rename addrman time symbols

-BEGIN VERIFY SCRIPT-
 ren() { sed -i "s:\<$1\>:$2:g" $(git grep -l "\<$1\>" ./src ./test); }

 ren nLastTry          m_last_try
 ren nLastSuccess      m_last_success
 ren nLastGood         m_last_good
 ren nLastCountAttempt m_last_count_attempt
 ren nSinceLastTry     since_last_try
 ren nTimePenalty      time_penalty
 ren nUpdateInterval   update_interval
 ren fCurrentlyOnline  currently_online
-END VERIFY SCRIPT-

* util: Add HoursDouble

* Add ChronoFormatter to serialize

* Add type-safe AdjustedTime() getter to timedata

Also, fix includes.

The getter will be used in a future commit.

* refactor: Use type-safe std::chrono for addrman time

* refactor: remove unnecessary string initializations

* tidy: enable readability-redundant-string-init

See:
https://releases.llvm.org/14.0.0/tools/clang/tools/extra/docs/clang-tidy/checks/readability-redundant-string-init.html

* depends: expat 2.4.8

* depends: re-enable using -flto when building expat

* script: actually trigger the optimization in BuildScript

The counter is an optimization over calling `ret.empty()`. It was
suggested that the compiler would realize `cnt` is only `0` on the first
iteration, and not actually emit the check and conditional.

This optimization was actually not triggered at all, since we
incremented `cnt` at the beginning of the first iteration. Fix it by
incrementing at the end instead.

This was reported by Github user "Janus".

* test: Drop unused boost workaround

* refactor: log `nEvicted` message in `LimitOrphans` then return void

`LimitOrphans()` can log expired tx and it should log evicted tx as well
instead of returning the number for caller to print the message.
Since `LimitOrphans()` now return void, the redundant assertion check in
fuzz test is also removed.

* [unit tests] individual RBF Rules in isolation

Test each component of the RBF policy in isolation. Unlike the RBF
functional tests, these do not rely on things like RPC results, mempool
submission, etc.

* guix: enable SSP for RISC-V glibc (2.27)

Pass `--enable-stack-protector=all` when building the glibc used for the
RISC-V toolchain, to enable stack smashing protection on all functions,
in the glibc code.

* guix: pass enable-bind-now to glibc

Both glibcs we build support `--enable-bind-now`:
Disable lazy binding for installed shared objects and programs.
This provides additional security hardening because it enables full RELRO
and a read-only global offset table (GOT), at the cost of slightly
increased program load times.

See:
https://www.gnu.org/software/libc/manual/html_node/Configuring-and-compiling.html

* guix: enable hardening options in GCC Build

Pass `--enable-default-pie` and `--enable-default-ssp` when configuring
our GCCs. This achieves the following:

--enable-default-pie
	Turn on -fPIE and -pie by default.

--enable-default-ssp
	Turn on -fstack-protector-strong by default.

Note that this isn't a replacement for passing hardneing flags
ourselves, but introduces some redundency, and there isn't really a
reason to not build a more "hardenings enabled" toolchain by default.

See also:
https://gcc.gnu.org/install/configure.html

* libxcb: use a patch instead of sed

To remove the unneeded pthread-stubs requirements.

* tidy: run clang-tidy in quiet mode

Co-authored-by: fanquake <[email protected]>
Co-authored-by: MacroFake <[email protected]>
Co-authored-by: Hennadii Stepanov <[email protected]>
Co-authored-by: Pieter Wuille <[email protected]>
Co-authored-by: Andrew Chow <[email protected]>
Co-authored-by: furszy <[email protected]>
Co-authored-by: Sebastian Falbesoner <[email protected]>
Co-authored-by: Vasil Dimov <[email protected]>
Co-authored-by: Antoine Riard <[email protected]>
Co-authored-by: glozow <[email protected]>
Co-authored-by: w0xlt <[email protected]>
Co-authored-by: Suhas Daftuar <[email protected]>
Co-authored-by: Carl Dong <[email protected]>
Co-authored-by: Antoine Poinsot <[email protected]>
Co-authored-by: Pieter Wuille <[email protected]>
Co-authored-by: John Newbery <[email protected]>
Co-authored-by: dergoegge <[email protected]>
Co-authored-by: Marnix <[email protected]>
Co-authored-by: chinggg <[email protected]>
Co-authored-by: Pablo Greco <[email protected]>
Co-authored-by: eugene <[email protected]>
Co-authored-by: Larry Ruane <[email protected]>
Co-authored-by: Ryan Ofsky <[email protected]>
Co-authored-by: josibake <[email protected]>
Co-authored-by: Russell Yanofsky <[email protected]>
Co-authored-by: James O'Beirne <[email protected]>
Co-authored-by: Jon Atack <[email protected]>
Co-authored-by: Luke Dashjr <[email protected]>
Co-authored-by: Aurèle Oulès <[email protected]>
Co-authored-by: Greg Weber <[email protected]>
  • Loading branch information
1 parent 2597bb9 commit 30f4f83
Show file tree
Hide file tree
Showing 322 changed files with 6,667 additions and 3,815 deletions.
5 changes: 4 additions & 1 deletion build-aux/m4/l_atomic.m4
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ m4_define([_CHECK_ATOMIC_testbody], [[
int main() {
std::atomic<bool> lock{true};
std::atomic_exchange(&lock, false);
lock.exchange(false);
std::atomic<std::chrono::seconds> t{0s};
t.store(2s);
Expand All @@ -34,6 +34,8 @@ m4_define([_CHECK_ATOMIC_testbody], [[
AC_DEFUN([CHECK_ATOMIC], [
AC_LANG_PUSH(C++)
TEMP_CXXFLAGS="$CXXFLAGS"
CXXFLAGS="$CXXFLAGS $PTHREAD_CFLAGS"
AC_MSG_CHECKING([whether std::atomic can be used without link library])
Expand All @@ -51,5 +53,6 @@ AC_DEFUN([CHECK_ATOMIC], [
])
])
CXXFLAGS="$TEMP_CXXFLAGS"
AC_LANG_POP
])
4 changes: 3 additions & 1 deletion build_msvc/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,9 @@ Building Navcoin Core with Visual Studio

Introduction
---------------------
Solution and project files to build Navcoin Core with `msbuild` or Visual Studio can be found in the `build_msvc` directory. The build has been tested with Visual Studio 2022 and Visual Studio 2019 (building with earlier versions of Visual Studio should not be expected to work).
Visual Studio 2022 is minimum required to build Navcoin Core.

Solution and project files to build with `msbuild` or Visual Studio can be found in the `build_msvc` directory.

To build Navcoin Core from the command-line, it is sufficient to only install the Visual Studio Build Tools component.

Expand Down
4 changes: 2 additions & 2 deletions build_msvc/bitcoin_config.h.in
Original file line number Diff line number Diff line change
Expand Up @@ -50,8 +50,8 @@
/* Define this symbol if the consensus lib has been built */
#define HAVE_CONSENSUS_LIB 1

/* define if the compiler supports basic C++17 syntax */
#define HAVE_CXX17 1
/* define if the compiler supports basic C++20 syntax */
#define HAVE_CXX20 1

/* Define to 1 if you have the declaration of `be16toh', and to 0 if you
don't. */
Expand Down
4 changes: 2 additions & 2 deletions build_msvc/common.init.vcxproj.in
Original file line number Diff line number Diff line change
Expand Up @@ -87,10 +87,10 @@
<ClCompile>
<WarningLevel>Level3</WarningLevel>
<PrecompiledHeader>NotUsing</PrecompiledHeader>
<AdditionalOptions>/utf-8 /Zc:__cplusplus /std:c++17 %(AdditionalOptions)</AdditionalOptions>
<AdditionalOptions>/utf-8 /Zc:__cplusplus /std:c++20 %(AdditionalOptions)</AdditionalOptions>
<DisableSpecificWarnings>4018;4244;4267;4334;4715;4805;4834</DisableSpecificWarnings>
<TreatWarningAsError>true</TreatWarningAsError>
<PreprocessorDefinitions>DISABLE_DESIGNATED_INITIALIZER_ERRORS;_SILENCE_CXX17_CODECVT_HEADER_DEPRECATION_WARNING;_SILENCE_CXX17_OLD_ALLOCATOR_MEMBERS_DEPRECATION_WARNING;ZMQ_STATIC;NOMINMAX;WIN32;HAVE_CONFIG_H;_CRT_SECURE_NO_WARNINGS;_SCL_SECURE_NO_WARNINGS;_CONSOLE;_WIN32_WINNT=0x0601;_WIN32_IE=0x0501;WIN32_LEAN_AND_MEAN;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<PreprocessorDefinitions>_SILENCE_CXX20_U8PATH_DEPRECATION_WARNING;_SILENCE_CXX17_CODECVT_HEADER_DEPRECATION_WARNING;_SILENCE_CXX17_OLD_ALLOCATOR_MEMBERS_DEPRECATION_WARNING;ZMQ_STATIC;NOMINMAX;WIN32;HAVE_CONFIG_H;_CRT_SECURE_NO_WARNINGS;_SCL_SECURE_NO_WARNINGS;_CONSOLE;_WIN32_WINNT=0x0601;_WIN32_IE=0x0501;WIN32_LEAN_AND_MEAN;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<AdditionalIncludeDirectories>..\..\src;..\..\src\minisketch\include;..\..\src\univalue\include;..\..\src\secp256k1\include;..\..\src\leveldb\include;..\..\src\leveldb\helpers\memenv;%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
</ClCompile>
<Link>
Expand Down
2 changes: 1 addition & 1 deletion build_msvc/msvc-autogen.py
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ def set_properties(vcxproj_filename, placeholder, content):
def main():
parser = argparse.ArgumentParser(description='Bitcoin-core msbuild configuration initialiser.')
parser.add_argument('-toolset', nargs='?', default=DEFAULT_PLATFORM_TOOLSET,
help='Optionally sets the msbuild platform toolset, e.g. v142 for Visual Studio 2019, or v143 for Visual Studio 2022.'
help='Optionally sets the msbuild platform toolset, e.g. v143 for Visual Studio 2022.'
' default is %s.'%DEFAULT_PLATFORM_TOOLSET)
args = parser.parse_args()
set_properties(os.path.join(SOURCE_DIR, '../build_msvc/common.init.vcxproj'), '@TOOLSET@', args.toolset)
Expand Down
2 changes: 1 addition & 1 deletion ci/test/00_setup_env_native_fuzz_with_valgrind.sh
Original file line number Diff line number Diff line change
Expand Up @@ -16,5 +16,5 @@ export RUN_FUZZ_TESTS=true
export FUZZ_TESTS_CONFIG="--valgrind"
export GOAL="install"
# Temporarily pin dwarf 4, until valgrind can understand clang's dwarf 5
export BITCOIN_CONFIG="--enable-fuzz --with-sanitizers=fuzzer CC=clang CXX=clang++ CXXFLAGS='-fdebug-default-version=4'"
export BITCOIN_CONFIG="--enable-fuzz --with-sanitizers=fuzzer CC=clang CXX=clang++ CFLAGS='-gdwarf-4' CXXFLAGS='-gdwarf-4'"
export CCACHE_SIZE=200M
2 changes: 1 addition & 1 deletion ci/test/00_setup_env_native_valgrind.sh
Original file line number Diff line number Diff line change
Expand Up @@ -14,4 +14,4 @@ export NO_DEPENDS=1
export TEST_RUNNER_EXTRA="--nosandbox --exclude feature_init,rpc_bind,feature_bind_extra" # Excluded for now, see https://github.com/bitcoin/bitcoin/issues/17765#issuecomment-602068547
export GOAL="install"
# Temporarily pin dwarf 4, until valgrind can understand clang's dwarf 5
export BITCOIN_CONFIG="--enable-zmq --with-incompatible-bdb --with-gui=no CC=clang CXX=clang++ CXXFLAGS='-fdebug-default-version=4'" # TODO enable GUI
export BITCOIN_CONFIG="--enable-zmq --with-incompatible-bdb --with-gui=no CC=clang CXX=clang++ CFLAGS='-gdwarf-4' CXXFLAGS='-gdwarf-4'" # TODO enable GUI
7 changes: 5 additions & 2 deletions ci/test/06_script_a.sh
Original file line number Diff line number Diff line change
Expand Up @@ -11,16 +11,19 @@ if [ -z "$NO_WERROR" ]; then
BITCOIN_CONFIG_ALL="${BITCOIN_CONFIG_ALL} --enable-werror"
fi

CI_EXEC "ccache --zero-stats --max-size=$CCACHE_SIZE"
PRINT_CCACHE_STATISTICS="ccache --version | head -n 1 && ccache --show-stats"

if [ -n "$ANDROID_TOOLS_URL" ]; then
CI_EXEC make distclean || true
CI_EXEC ./autogen.sh
CI_EXEC ./configure "$BITCOIN_CONFIG_ALL" "$BITCOIN_CONFIG" || ( (CI_EXEC cat config.log) && false)
CI_EXEC "make $MAKEJOBS && cd src/qt && ANDROID_HOME=${ANDROID_HOME} ANDROID_NDK_HOME=${ANDROID_NDK_HOME} make apk"
CI_EXEC "${PRINT_CCACHE_STATISTICS}"
exit 0
fi

BITCOIN_CONFIG_ALL="${BITCOIN_CONFIG_ALL} --enable-external-signer --bindir=$BASE_OUTDIR/bin --libdir=$BASE_OUTDIR/lib"
CI_EXEC "ccache --zero-stats --max-size=$CCACHE_SIZE"

if [ -n "$CONFIG_SHELL" ]; then
CI_EXEC "$CONFIG_SHELL" -c "./autogen.sh"
Expand Down Expand Up @@ -57,6 +60,6 @@ fi

CI_EXEC "${MAYBE_BEAR}" "${MAYBE_TOKEN}" make "$MAKEJOBS" "$GOAL" || ( echo "Build failure. Verbose build follows." && CI_EXEC make "$GOAL" V=1 ; false )

CI_EXEC "ccache --version | head -n 1 && ccache --show-stats"
CI_EXEC "${PRINT_CCACHE_STATISTICS}"
CI_EXEC du -sh "${DEPENDS_DIR}"/*/
CI_EXEC du -sh "${PREVIOUS_RELEASES_DIR}"
23 changes: 21 additions & 2 deletions ci/test/06_script_b.sh
Original file line number Diff line number Diff line change
Expand Up @@ -35,17 +35,36 @@ if [ "$RUN_FUNCTIONAL_TESTS" = "true" ]; then
fi

if [ "${RUN_TIDY}" = "true" ]; then
set -eo pipefail
export P_CI_DIR="${BASE_BUILD_DIR}/navcoin-$HOST/src/"
CI_EXEC run-clang-tidy "${MAKEJOBS}"
export P_CI_DIR="${BASE_BUILD_DIR}/navcoin-$HOST/"
( CI_EXEC run-clang-tidy -quiet "${MAKEJOBS}" ) | grep -C5 "error"
export P_CI_DIR="${BASE_BUILD_DIR}/bitcoin-$HOST/"
CI_EXEC "python3 ${DIR_IWYU}/include-what-you-use/iwyu_tool.py"\
" src/compat"\
" src/dbwrapper.cpp"\
" src/init"\
" src/kernel"\
" src/node/chainstate.cpp"\
" src/policy/feerate.cpp"\
" src/policy/packages.cpp"\
" src/policy/settings.cpp"\
" src/primitives/transaction.cpp"\
" src/rpc/fees.cpp"\
" src/rpc/signmessage.cpp"\
" src/test/fuzz/txorphan.cpp"\
" src/threadinterrupt.cpp"\
" src/util/bip32.cpp"\
" src/util/bytevectorhash.cpp"\
" src/util/error.cpp"\
" src/util/getuniquepath.cpp"\
" src/util/hasher.cpp"\
" src/util/message.cpp"\
" src/util/moneystr.cpp"\
" src/util/serfloat.cpp"\
" src/util/spanparsing.cpp"\
" src/util/strencodings.cpp"\
" src/util/syserror.cpp"\
" src/util/url.cpp"\
" -p . ${MAKEJOBS} -- -Xiwyu --cxx17ns -Xiwyu --mapping_file=${BASE_BUILD_DIR}/navcoin-$HOST/contrib/devtools/iwyu/bitcoin.core.imp"
fi

Expand Down
2 changes: 1 addition & 1 deletion ci/test/wrap-qemu.sh
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

export LC_ALL=C.UTF-8

for b_name in {"${BASE_OUTDIR}/bin"/*,src/secp256k1/*tests,src/minisketch/test{,-verify},src/univalue/{no_nul,test_json,unitester,object}}; do
for b_name in {"${BASE_OUTDIR}/bin"/*,src/secp256k1/*tests,src/minisketch/test{,-verify},src/univalue/{test_json,unitester,object}}; do
# shellcheck disable=SC2044
for b in $(find "${BASE_ROOT_DIR}" -executable -type f -name "$(basename "$b_name")"); do
echo "Wrap $b ..."
Expand Down
2 changes: 1 addition & 1 deletion ci/test/wrap-wine.sh
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

export LC_ALL=C.UTF-8

for b_name in {"${BASE_OUTDIR}/bin"/*,src/secp256k1/*tests,src/minisketch/test{,-verify},src/univalue/{no_nul,test_json,unitester,object}}.exe; do
for b_name in {"${BASE_OUTDIR}/bin"/*,src/secp256k1/*tests,src/minisketch/test{,-verify},src/univalue/{test_json,unitester,object}}.exe; do
# shellcheck disable=SC2044
for b in $(find "${BASE_ROOT_DIR}" -executable -type f -name "$(basename "$b_name")"); do
if (file "$b" | grep "Windows"); then
Expand Down
10 changes: 7 additions & 3 deletions configure.ac
Original file line number Diff line number Diff line change
Expand Up @@ -87,9 +87,6 @@ else
AX_CXX_COMPILE_STDCXX([20], [noext], [mandatory])
fi

dnl Check if -latomic is required for <std::atomic>
CHECK_ATOMIC

dnl check if additional link flags are required for std::filesystem
CHECK_FILESYSTEM

Expand Down Expand Up @@ -887,6 +884,9 @@ AC_C_BIGENDIAN
dnl Check for pthread compile/link requirements
AX_PTHREAD

dnl Check if -latomic is required for <std::atomic>
CHECK_ATOMIC

dnl The following macro will add the necessary defines to bitcoin-config.h, but
dnl they also need to be passed down to any subprojects. Pull the results out of
dnl the cache and add them to CPPFLAGS.
Expand Down Expand Up @@ -1978,6 +1978,9 @@ AC_CONFIG_LINKS([test/functional/test_runner.py:test/functional/test_runner.py])
AC_CONFIG_LINKS([test/fuzz/test_runner.py:test/fuzz/test_runner.py])
AC_CONFIG_LINKS([test/util/test_runner.py:test/util/test_runner.py])
AC_CONFIG_LINKS([test/util/rpcauth-test.py:test/util/rpcauth-test.py])
AC_CONFIG_LINKS([src/qt/Makefile:src/qt/Makefile])
AC_CONFIG_LINKS([src/qt/test/Makefile:src/qt/test/Makefile])
AC_CONFIG_LINKS([src/test/Makefile:src/test/Makefile])

dnl boost's m4 checks do something really nasty: they export these vars. As a
dnl result, they leak into secp256k1's configure and crazy things happen.
Expand Down Expand Up @@ -2049,5 +2052,6 @@ echo " CPPFLAGS = $DEBUG_CPPFLAGS $HARDENED_CPPFLAGS $CORE_CPPFLAGS $CPP
echo " CXX = $CXX"
echo " CXXFLAGS = $LTO_CXXFLAGS $DEBUG_CXXFLAGS $HARDENED_CXXFLAGS $WARN_CXXFLAGS $NOWARN_CXXFLAGS $ERROR_CXXFLAGS $GPROF_CXXFLAGS $CORE_CXXFLAGS $CXXFLAGS"
echo " LDFLAGS = $LTO_LDFLAGS $PTHREAD_LIBS $HARDENED_LDFLAGS $GPROF_LDFLAGS $CORE_LDFLAGS $LDFLAGS"
echo " AR = $AR"
echo " ARFLAGS = $ARFLAGS"
echo
1 change: 1 addition & 0 deletions contrib/builder-keys/keys.txt
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ BF6273FAEF7CC0BA1F562E50989F6B3048A116B5 Dev Random (devrandom)
D35176BE9264832E4ACA8986BF0792FBE95DC863 fivepiece (fivepiece)
6F993B250557E7B016ADE5713BDCDA2D87A881D9 Fuzzbawls (Fuzzbawls)
01CDF4627A3B88AAE4A571C87588242FBE38D3A8 Gavin Andresen (gavinandresen)
6B002C6EA3F91B1B0DF0C9BC8F617F1200A6D25C Gloria Zhao (glozow)
D1DBF2C4B96F2DEBF4C16654410108112E7EA81F Hennadii Stepanov (hebasto)
A2FD494D0021AA9B4FA58F759102B7AE654A4A5A Ilyas Ridhuan (IlyasRidhuan)
2688F5A9A4BE0F295E921E8A25F27A38A47AD566 James O'Beirne (jamesob)
Expand Down
1 change: 1 addition & 0 deletions contrib/devtools/iwyu/bitcoin.core.imp
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,5 @@
{ include: [ "<bits/termios-c_lflag.h>", private, "<termios.h>", public ] },
{ include: [ "<bits/termios-struct.h>", private, "<termios.h>", public ] },
{ include: [ "<bits/termios-tcflow.h>", private, "<termios.h>", public ] },
{ include: [ "<bits/chrono.h>", private, "<chrono>", public ] },
]
1 change: 1 addition & 0 deletions contrib/guix/libexec/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -223,6 +223,7 @@ CONFIGFLAGS="--enable-reduce-exports --disable-bench --disable-gui-tests --disab

# CFLAGS
HOST_CFLAGS="-O2 -g"
HOST_CFLAGS+=$(find /gnu/store -maxdepth 1 -mindepth 1 -type d -exec echo -n " -ffile-prefix-map={}=/usr" \;)
case "$HOST" in
*linux*) HOST_CFLAGS+=" -ffile-prefix-map=${PWD}=." ;;
*mingw*) HOST_CFLAGS+=" -fno-ident" ;;
Expand Down
37 changes: 27 additions & 10 deletions contrib/guix/manifest.scm
Original file line number Diff line number Diff line change
Expand Up @@ -132,12 +132,19 @@ chain for " target " development."))
(define base-gcc gcc-10)
(define base-linux-kernel-headers linux-libre-headers-5.15)

;; https://gcc.gnu.org/install/configure.html
(define (hardened-gcc gcc)
(package-with-extra-configure-variable (
package-with-extra-configure-variable gcc
"--enable-default-ssp" "yes")
"--enable-default-pie" "yes"))

(define* (make-bitcoin-cross-toolchain target
#:key
(base-gcc-for-libc base-gcc)
(base-kernel-headers base-linux-kernel-headers)
(base-libc (make-glibc-without-werror glibc-2.24))
(base-gcc (make-gcc-rpath-link base-gcc)))
(base-libc (make-glibc-with-bind-now (make-glibc-without-werror glibc-2.24)))
(base-gcc (make-gcc-rpath-link (hardened-gcc base-gcc))))
"Convenience wrapper around MAKE-CROSS-TOOLCHAIN with default values
desirable for building Bitcoin Core release binaries."
(make-cross-toolchain target
Expand All @@ -147,7 +154,10 @@ desirable for building Bitcoin Core release binaries."
base-gcc))

(define (make-gcc-with-pthreads gcc)
(package-with-extra-configure-variable gcc "--enable-threads" "posix"))
(package-with-extra-configure-variable
(package-with-extra-patches gcc
(search-our-patches "gcc-10-remap-guix-store.patch"))
"--enable-threads" "posix"))

(define (make-mingw-w64-cross-gcc cross-gcc)
(package-with-extra-patches cross-gcc
Expand Down Expand Up @@ -253,7 +263,7 @@ thus should be able to compile on most platforms where these exist.")
(license license:gpl3+))) ; license is with openssl exception

(define-public python-elfesteem
(let ((commit "87bbd79ab7e361004c98cc8601d4e5f029fd8bd5"))
(let ((commit "2eb1e5384ff7a220fd1afacd4a0170acff54fe56"))
(package
(name "python-elfesteem")
(version (git-version "0.1" "1" commit))
Expand All @@ -266,8 +276,7 @@ thus should be able to compile on most platforms where these exist.")
(file-name (git-file-name name commit))
(sha256
(base32
"1nyvjisvyxyxnd0023xjf5846xd03lwawp5pfzr8vrky7wwm5maz"))
(patches (search-our-patches "elfsteem-value-error-python-39.patch"))))
"07x6p8clh11z8s1n2kdxrqwqm2almgc5qpkcr9ckb6y5ivjdr5r6"))))
(build-system python-build-system)
;; There are no tests, but attempting to run python setup.py test leads to
;; PYTHONPATH problems, just disable the test
Expand Down Expand Up @@ -518,6 +527,12 @@ inspecting signatures in Mach-O binaries.")
(define (make-glibc-without-werror glibc)
(package-with-extra-configure-variable glibc "enable_werror" "no"))

(define (make-glibc-with-stack-protector glibc)
(package-with-extra-configure-variable glibc "--enable-stack-protector" "all"))

(define (make-glibc-with-bind-now glibc)
(package-with-extra-configure-variable glibc "--enable-bind-now" "yes"))

(define-public glibc-2.24
(package
(inherit glibc-2.31)
Expand All @@ -535,7 +550,8 @@ inspecting signatures in Mach-O binaries.")
"glibc-versioned-locpath.patch"
"glibc-2.24-elfm-loadaddr-dynamic-rewrite.patch"
"glibc-2.24-no-build-time-cxx-header-run.patch"
"glibc-2.24-fcommon.patch"))))))
"glibc-2.24-fcommon.patch"
"glibc-2.24-guix-prefix.patch"))))))

(define-public glibc-2.27/bitcoin-patched
(package
Expand All @@ -552,7 +568,8 @@ inspecting signatures in Mach-O binaries.")
"1b2n1gxv9f4fd5yy68qjbnarhf8mf4vmlxk10i3328c1w5pmp0ca"))
(patches (search-our-patches "glibc-ldd-x86_64.patch"
"glibc-2.27-riscv64-Use-__has_include-to-include-asm-syscalls.h.patch"
"glibc-2.27-dont-redefine-nss-database.patch"))))))
"glibc-2.27-dont-redefine-nss-database.patch"
"glibc-2.27-guix-prefix.patch"))))))

(packages->manifest
(append
Expand Down Expand Up @@ -603,8 +620,8 @@ inspecting signatures in Mach-O binaries.")
((string-contains target "-linux-")
(list (cond ((string-contains target "riscv64-")
(make-bitcoin-cross-toolchain target
#:base-libc (make-glibc-without-werror glibc-2.27/bitcoin-patched)
#:base-kernel-headers base-linux-kernel-headers))
#:base-libc (make-glibc-with-stack-protector
(make-glibc-with-bind-now (make-glibc-without-werror glibc-2.27/bitcoin-patched)))))
(else
(make-bitcoin-cross-toolchain target)))))
((string-contains target "darwin")
Expand Down
13 changes: 0 additions & 13 deletions contrib/guix/patches/elfsteem-value-error-python-39.patch

This file was deleted.

25 changes: 25 additions & 0 deletions contrib/guix/patches/gcc-10-remap-guix-store.patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
From aad25427e74f387412e8bc9a9d7bbc6c496c792f Mon Sep 17 00:00:00 2001
From: Andrew Chow <[email protected]>
Date: Wed, 6 Jul 2022 16:49:41 -0400
Subject: [PATCH] guix: remap guix store paths to /usr

---
libgcc/Makefile.in | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libgcc/Makefile.in b/libgcc/Makefile.in
index 851e7657d07..476c2becd1c 100644
--- a/libgcc/Makefile.in
+++ b/libgcc/Makefile.in
@@ -854,7 +854,7 @@ endif
# libgcc_eh.a, only LIB2ADDEH matters. If we do, only LIB2ADDEHSTATIC and
# LIB2ADDEHSHARED matter. (Usually all three are identical.)

-c_flags := -fexceptions
+c_flags := -fexceptions $(shell find /gnu/store -maxdepth 1 -mindepth 1 -type d -exec echo -n " -ffile-prefix-map={}=/usr" \;)

ifeq ($(enable_shared),yes)

--
2.37.0

25 changes: 25 additions & 0 deletions contrib/guix/patches/glibc-2.24-guix-prefix.patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
Without ffile-prefix-map, the debug symbols will contain paths for the
guix store which will include the hashes of each package. However, the
hash for the same package will differ when on different architectures.
In order to be reproducible regardless of the architecture used to build
the package, map all guix store prefixes to something fixed, e.g. /usr.

We might be able to drop this in favour of using --with-nonshared-cflags
when we being using newer versions of glibc.

--- a/Makeconfig
+++ b/Makeconfig
@@ -950,6 +950,10 @@ object-suffixes-for-libc += .oS
# shared objects. We don't want to use CFLAGS-os because users may, for
# example, make that processor-specific.
CFLAGS-.oS = $(CFLAGS-.o) $(PIC-ccflag)
+
+# Map Guix store paths to /usr
+CFLAGS-.oS += `find /gnu/store -maxdepth 1 -mindepth 1 -type d -exec echo -n " -fdebug-prefix-map={}=/usr" \;`
+
CPPFLAGS-.oS = $(CPPFLAGS-.o) -DPIC -DLIBC_NONSHARED=1
libtype.oS = lib%_nonshared.a
endif
--
2.35.1

Loading

0 comments on commit 30f4f83

Please sign in to comment.