Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: poseidon hash #388

Merged
merged 146 commits into from
Dec 10, 2024
Merged
Show file tree
Hide file tree
Changes from 140 commits
Commits
Show all changes
146 commits
Select commit Hold shift + click to select a range
f7b5940
++
qalisander Oct 3, 2024
0b0cf3d
add original poseidon2 hash and vesta prime field
qalisander Oct 6, 2024
f305557
++
qalisander Oct 10, 2024
3369577
++
qalisander Oct 16, 2024
e41a63e
++
qalisander Oct 17, 2024
57d880c
++
qalisander Oct 18, 2024
76250c4
++
qalisander Oct 18, 2024
e897579
++
qalisander Oct 18, 2024
423503e
++
qalisander Oct 19, 2024
2563a53
++
qalisander Oct 19, 2024
264afa5
++
qalisander Oct 19, 2024
bd7e14c
compiles: port fields traits
qalisander Oct 19, 2024
5557280
compiles: port fp and biginteger
qalisander Oct 26, 2024
c10c78a
compiles: poseidon uses inhouse fields
qalisander Oct 26, 2024
1c76850
compiles: remove ark-ff-macros from deps
qalisander Oct 26, 2024
f62c298
++
qalisander Oct 26, 2024
a29a630
++
qalisander Oct 27, 2024
e8b722a
++
qalisander Oct 27, 2024
d80db6e
compiles: remove fft field
qalisander Oct 27, 2024
b8b7286
compiles: remove sqrt
qalisander Oct 27, 2024
f6ddc87
++
qalisander Oct 27, 2024
2d84081
++
qalisander Oct 27, 2024
e53769e
compiles: remove montgomery_backend abstraction
qalisander Oct 27, 2024
2500b1a
++
qalisander Oct 27, 2024
375d042
++
qalisander Oct 27, 2024
fc77caf
compiles: remove BasePrimeField associated type
qalisander Oct 27, 2024
d461172
++
qalisander Oct 27, 2024
745edcf
compiles: remove num_bigint from fields
qalisander Oct 27, 2024
2de8819
compiles
qalisander Oct 27, 2024
b489a8e
++
qalisander Oct 27, 2024
ed3835b
++
qalisander Oct 27, 2024
cfc6d91
++
qalisander Oct 27, 2024
181339c
++
qalisander Oct 27, 2024
617f141
++
qalisander Oct 27, 2024
b0b1b1d
++
qalisander Oct 27, 2024
343064a
++
qalisander Oct 27, 2024
e77c5d0
++
qalisander Oct 27, 2024
b5b4a3a
compiles: test not pass
qalisander Oct 27, 2024
74b86f7
++
qalisander Oct 27, 2024
886b955
++
qalisander Oct 27, 2024
1662190
compiles: test fails
qalisander Oct 28, 2024
090024b
compiles: test pass
qalisander Oct 28, 2024
c19dbf7
++
qalisander Oct 28, 2024
f86e6cc
++
qalisander Oct 28, 2024
8d43485
++
qalisander Oct 28, 2024
ffad719
++
qalisander Oct 28, 2024
b64460e
++
qalisander Oct 28, 2024
7275e0e
++
qalisander Oct 28, 2024
bb1eac4
compiles: remove ark-std
qalisander Oct 28, 2024
d71142d
++
qalisander Oct 28, 2024
5ace24d
++
qalisander Oct 28, 2024
d08b77c
store Residue in Fp
qalisander Oct 28, 2024
bdc066c
++
qalisander Oct 28, 2024
cd180c0
++
qalisander Oct 28, 2024
5dc350a
++
qalisander Oct 28, 2024
de78208
++
qalisander Oct 28, 2024
f5e64af
++
qalisander Oct 28, 2024
cb47ec0
++
qalisander Oct 29, 2024
c8f51ee
++
qalisander Oct 29, 2024
91d5b80
++
qalisander Oct 29, 2024
0771112
++
qalisander Oct 29, 2024
d822dce
++
qalisander Oct 29, 2024
1d47d65
parse from str number
qalisander Oct 29, 2024
5a85494
compiles: refactor
qalisander Oct 29, 2024
4a85e8e
introduce fp and uint macros
qalisander Oct 29, 2024
7110fec
++
qalisander Oct 29, 2024
13d1ee7
++
qalisander Oct 29, 2024
4826fb5
++
qalisander Oct 29, 2024
702052c
++
qalisander Oct 29, 2024
0fa044f
++
qalisander Oct 29, 2024
474711c
++
qalisander Oct 29, 2024
3d6f402
++
qalisander Oct 29, 2024
c082f78
add field docs
qalisander Oct 29, 2024
5ddd2bc
++
qalisander Oct 29, 2024
fbbf8f0
++
qalisander Oct 30, 2024
40f08a5
Merge remote-tracking branch 'origin/main' into finite-fields-2
qalisander Oct 30, 2024
3361106
++
qalisander Oct 30, 2024
5c3961c
fix compilation for wasm
qalisander Oct 30, 2024
444754a
Merge remote-tracking branch 'origin/main' into finite-fields-2
qalisander Oct 30, 2024
9f39a09
++
qalisander Oct 30, 2024
80ce59a
++
qalisander Oct 30, 2024
5c5251a
++
qalisander Oct 30, 2024
ddce394
++
qalisander Oct 30, 2024
a3b89ad
++
qalisander Oct 30, 2024
ed1ce9a
++
qalisander Oct 30, 2024
43b0483
++
qalisander Oct 30, 2024
fb0932f
++
qalisander Oct 30, 2024
dea904f
++
qalisander Oct 30, 2024
9c9e8fa
++
qalisander Oct 30, 2024
b80fb7a
++
qalisander Oct 30, 2024
bdce479
++
qalisander Oct 30, 2024
1b543c4
++
qalisander Oct 30, 2024
66c85cd
++
qalisander Oct 30, 2024
6cc65fc
++
qalisander Oct 31, 2024
e98a830
++
qalisander Oct 31, 2024
c964fb2
refactor
qalisander Oct 31, 2024
2998534
++
qalisander Nov 1, 2024
de698b0
++
qalisander Nov 1, 2024
387527e
++
qalisander Nov 1, 2024
f0c8e94
++
qalisander Nov 1, 2024
30430e6
refactor
qalisander Oct 31, 2024
24c1e6e
++
qalisander Nov 1, 2024
7e213d2
++
qalisander Nov 1, 2024
1df589e
++
qalisander Nov 1, 2024
79caad0
++
qalisander Nov 1, 2024
14d7546
++
qalisander Nov 1, 2024
2043e83
compiles: other fields for poseidon hash
qalisander Nov 2, 2024
6fa8405
move tests to instances
qalisander Nov 2, 2024
f8886b8
++
qalisander Nov 3, 2024
bdc94ad
++
qalisander Nov 3, 2024
3e7b37a
remove _mat_internal
qalisander Nov 3, 2024
f02c161
use generic params for vesta
qalisander Nov 3, 2024
9f580cc
from_str_hex
qalisander Nov 3, 2024
1147c88
use generic params for other fields
qalisander Nov 3, 2024
99cf440
Merge remote-tracking branch 'origin/main' into poseidon
qalisander Nov 7, 2024
080c60f
Merge remote-tracking branch 'origin/main' into poseidon
qalisander Nov 10, 2024
088b5c1
Merge branch 'main' into poseidon
bidzyyys Nov 20, 2024
639c720
++
qalisander Nov 20, 2024
f5d870a
remove computed constants
qalisander Nov 20, 2024
c48488d
add capacity param
qalisander Nov 22, 2024
850023d
turn poseidon into sponge hash function
qalisander Nov 22, 2024
9e8cdb5
++
qalisander Nov 24, 2024
2540dbb
++
qalisander Nov 25, 2024
745e3e4
add 64 bit field params + docs
qalisander Nov 25, 2024
dfeb6cc
++
qalisander Nov 27, 2024
231e739
++
qalisander Nov 27, 2024
a000bc5
Merge remote-tracking branch 'origin/main' into poseidon
qalisander Nov 27, 2024
b866627
++
qalisander Nov 27, 2024
33d13e1
++
qalisander Nov 27, 2024
0593bc4
++
qalisander Nov 27, 2024
f73cbae
++
qalisander Nov 27, 2024
c762be8
add convert_from_str_hex fuzz test
qalisander Nov 28, 2024
fcb1db5
++
qalisander Nov 28, 2024
72b01ec
++
qalisander Nov 29, 2024
8b2da4b
++
qalisander Dec 9, 2024
0213da1
++
qalisander Dec 9, 2024
5a4b537
introduce external_round & internal_round fns
qalisander Dec 9, 2024
93728e5
update CHANGELOG.md
qalisander Dec 9, 2024
3444407
Merge branch 'main' into poseidon
qalisander Dec 9, 2024
b29f99c
++
qalisander Dec 10, 2024
e31cd8c
add docks to poseidon instances
qalisander Dec 10, 2024
a125af0
add docks to poseidon instances
qalisander Dec 10, 2024
4c8f3e8
add docks to poseidon instances
qalisander Dec 10, 2024
7f31aed
Merge branch 'main' into poseidon
bidzyyys Dec 10, 2024
4d7070e
update CHANGELOG.md
qalisander Dec 10, 2024
215ddf4
Merge branch 'main' into poseidon
bidzyyys Dec 10, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Finite Fields arithmetics. #376
- `Ownable2Step` contract. #352
- `IOwnable` trait. #352
- `Poseidon2` sponge hash function. #388
qalisander marked this conversation as resolved.
Show resolved Hide resolved

### Changed(breaking)

Expand All @@ -40,7 +41,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Changed

- Mini alloc is now used by default via the stylus-sdk. This avoids conflicts with duplicate `#[global_allocator]` definitions. #373
- Mini alloc is now used by default via the stylus-sdk. This avoids conflicts with duplicate `#[global_allocator]`
qalisander marked this conversation as resolved.
Show resolved Hide resolved
definitions. #373
- Removed the panic handler from the library, making it easier for `std` and `no_std` projects to use the library. #373

## [0.1.0] - 2024-10-17
Expand Down
7 changes: 7 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -136,3 +136,10 @@ incremental = false

[profile.dev]
panic = "abort"

[workspace.metadata.typos]
default = { extend-ignore-identifiers-re = [
# ignore hex data samples.
"[0-9a-fA-F][0-9a-fA-F]",
] }
files = { extend-exclude = [] }
0xNeshi marked this conversation as resolved.
Show resolved Hide resolved
94 changes: 80 additions & 14 deletions lib/crypto/src/bigint.rs
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ use core::{
#[allow(clippy::module_name_repetitions)]
pub use crypto_bigint;
use crypto_bigint::{Integer, Limb, Uint, Word, Zero};
use num_traits::ConstZero;
use zeroize::Zeroize;

use crate::bits::BitIteratorBE;
Expand Down Expand Up @@ -150,13 +151,17 @@ impl<const N: usize> BitIteratorBE for Uint<N> {

/// Parse a number from a string in a given radix.
///
/// This implementation can be slow on big numbers and possibly fail constant
/// compilation by timeout.
///
/// I.e., convert string encoded integer `s` to base-`radix` number.
#[must_use]
pub const fn from_str_radix<const LIMBS: usize>(
s: &str,
radix: u32,
) -> Uint<LIMBS> {
let bytes = s.as_bytes();
assert!(!bytes.is_empty(), "empty string");

// The lowest order number is at the end of the string.
// Begin parsing from the last index of the string.
Expand All @@ -167,14 +172,7 @@ pub const fn from_str_radix<const LIMBS: usize>(
let uint_radix = Uint::from_u32(radix);

loop {
// Try to parse a digit from utf-8 byte
let ch = parse_utf8_byte(bytes[index]);
let digit = match ch.to_digit(radix) {
None => {
panic!("invalid digit");
}
Some(digit) => Uint::from_u32(digit),
};
let digit = Uint::from_u32(parse_digit(bytes[index], radix));

// Add a digit multiplied by order.
uint = add(&uint, &mul(&digit, &order));
Expand All @@ -192,9 +190,53 @@ pub const fn from_str_radix<const LIMBS: usize>(
}
}

/// Parse a number from a hex string.
///
/// This implementation performs faster than [`from_str_radix`], since it
/// assumes the radix is already `16`.
///
/// If the string number is shorter, then [`Uint`] can store.
/// Returns a [`Uint`] with leading zeroes.
#[must_use]
pub const fn from_str_hex<const LIMBS: usize>(s: &str) -> Uint<LIMBS> {
bidzyyys marked this conversation as resolved.
Show resolved Hide resolved
let bytes = s.as_bytes();
assert!(!bytes.is_empty(), "empty string");

// The lowest order number is at the end of the string.
// Begin parsing from the last index of the string.
let mut index = bytes.len() - 1;

// The lowest order limb is at the beginning of the `num` array.
// Begin indexing from `0`.
let mut num = [Word::ZERO; LIMBS];
let mut num_index = 0;

let digit_radix = 16;
let digit_size = 4; // Size of a hex digit in bits (2^4 = 16).
let digits_in_limb = Limb::BITS / digit_size;

loop {
let digit = parse_digit(bytes[index], digit_radix) as Word;

// Since a base-16 digit can be represented with the same bits, we can
// copy these bits.
let digit_mask = digit << ((num_index % digits_in_limb) * digit_size);
num[num_index / digits_in_limb] |= digit_mask;

// If we reached the beginning of the string, return the number.
if index == 0 {
return Uint::from_words(num);
}

// Move to the next digit.
index -= 1;
num_index += 1;
}
}

/// Multiply two numbers and panic on overflow.
#[must_use]
pub const fn mul<const LIMBS: usize>(
const fn mul<const LIMBS: usize>(
a: &Uint<LIMBS>,
b: &Uint<LIMBS>,
) -> Uint<LIMBS> {
Expand All @@ -205,7 +247,7 @@ pub const fn mul<const LIMBS: usize>(

/// Add two numbers and panic on overflow.
#[must_use]
pub const fn add<const LIMBS: usize>(
const fn add<const LIMBS: usize>(
a: &Uint<LIMBS>,
b: &Uint<LIMBS>,
) -> Uint<LIMBS> {
Expand All @@ -214,6 +256,17 @@ pub const fn add<const LIMBS: usize>(
low
}

// Try to parse a digit from utf-8 byte.
const fn parse_digit(utf8_digit: u8, digit_radix: u32) -> u32 {
let ch = parse_utf8_byte(utf8_digit);
match ch.to_digit(digit_radix) {
None => {
panic!("invalid digit");
}
Some(digit) => digit,
}
}

/// Parse a single UTF-8 byte.
pub(crate) const fn parse_utf8_byte(byte: u8) -> char {
match byte {
Expand All @@ -234,17 +287,19 @@ macro_rules! from_num {
#[macro_export]
macro_rules! from_hex {
($num:literal) => {
$crate::bigint::crypto_bigint::Uint::from_be_hex($num)
$crate::bigint::from_str_hex($num)
};
}

#[cfg(test)]
mod test {
use proptest::proptest;

use super::*;

qalisander marked this conversation as resolved.
Show resolved Hide resolved
#[test]
fn convert_from_str_radix() {
let uint_from_base10 = from_str_radix::<4>(
let uint_from_base10: Uint<4> = from_str_radix(
"28948022309329048855892746252171976963363056481941647379679742748393362948097",
10
);
Expand All @@ -257,14 +312,25 @@ mod test {
]);
assert_eq!(uint_from_base10, expected);

let uint_from_base10 = from_str_radix::<1>("18446744069414584321", 10);
let uint_from_binary = from_str_radix::<1>(
let uint_from_base10: Uint<1> =
from_str_radix("18446744069414584321", 10);
let uint_from_binary: Uint<1> = from_str_radix(
"1111111111111111111111111111111100000000000000000000000000000001",
2,
);
assert_eq!(uint_from_base10, uint_from_binary);
}

#[test]
fn convert_from_str_hex() {
// Test different implementations of hex parsing on random hex inputs.
proptest!(|(s in "[0-9a-fA-F]{1,64}")| {
let uint_from_hex: Uint<4> = from_str_hex(&s);
let expected: Uint<4> = from_str_radix(&s, 16);
assert_eq!(uint_from_hex, expected);
});
}

#[test]
fn uint_bit_iterator_be() {
let words: [Word; 4] = [0b1100, 0, 0, 0];
Expand Down
1 change: 0 additions & 1 deletion lib/crypto/src/bits.rs
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,6 @@ impl_bit_iter_be!(usize);

#[cfg(test)]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again curios, why are these not marked as other tests with:

#[cfg(all(test, feature = "std"))]

though merkle and hash have it?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm also curious how "std" feature flag actual to all our lib.. Since it usually runs altogether with test cases..

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just tried updating all lib/crypto-related tests to have the #[cfg(all(test, feature = "std"))] annotation, and it completely solved all the compilation issues, many of which you addressed in this PR (see previous comments).

mod tests {

use super::*;

#[test]
Expand Down
4 changes: 1 addition & 3 deletions lib/crypto/src/field/fp.rs
Original file line number Diff line number Diff line change
Expand Up @@ -859,9 +859,7 @@ macro_rules! fp_from_num {
#[macro_export]
macro_rules! fp_from_hex {
($num:literal) => {{
$crate::field::fp::Fp::new(
$crate::bigint::crypto_bigint::Uint::from_be_hex($num),
)
$crate::field::fp::Fp::new($crate::bigint::from_str_hex($num))
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just curios, why was Uint::from_be_hex not appropriate to use here?

Copy link
Member Author

@qalisander qalisander Nov 28, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Uint::from_be_hex requires specific length of hex string. Like 2 byte "1234" number wont be converted to U64 with leading zeros. Only 8 byte hex number does so. Of course we could format all our const values to proper length, but I've just decided to add our const hex conversion and copy constants from referenced implementation here in whitepaper

}};
}

0xNeshi marked this conversation as resolved.
Show resolved Hide resolved
Expand Down
1 change: 1 addition & 0 deletions lib/crypto/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -30,5 +30,6 @@ pub mod field;
pub mod hash;
pub mod keccak;
pub mod merkle;
pub mod poseidon2;

pub use keccak::KeccakBuilder;
Loading
Loading