Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wasm support #351

Merged
merged 11 commits into from
Jun 7, 2022
Merged

Wasm support #351

merged 11 commits into from
Jun 7, 2022

Conversation

tjsharp1
Copy link
Contributor

Opening as a draft for now to get some eyes on it. There were some build issues on my machine that looked like they may be config related, so I wanted to see if it passes CI.

It looks like this is about all I need in order to make things work in WASM. As long as the codegen output use statements that are from fuels_core, I think it will compile fine.

I've added a separate directory for WASM tests, since I have to override the default build target. I added the .cargo/config.toml to set it to wasm32-unknown-unkown.

I believe this will address #317 and #318

};

println!("It works! {}", a_struct.id);
}
Copy link
Member

@Voxelot Voxelot Jun 3, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When running cargo test --target wasm32-unknown-unknown with the following code:

Suggested change
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test() {
the_fn()
}
}

I encountered a strange error:

/home/voxelot/code/fuel/fuels-rs/target/wasm32-unknown-unknown/debug/deps/wasm_tests-b1e4d09cfabc0966.wasm: 1: Syntax error: word unexpected (expecting ")")
error: test failed, to rerun pass '--lib'

this was on ubuntu 20.04. My m1 mac wasn't even able to compile this, so that should be factored into the devux of writing wasm indexer modules since it may require a lot of llvm toolchain config and setup to get this working in most dev environments.

Copy link
Member

@Voxelot Voxelot Jun 3, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given fuels-core still largely relies on std, I'm skeptical about whether any of the fs methods (e.g. loading the json abi file) will actually be implemented or just panic at runtime. Guess it depends on the runtime we use the wasm in. However, that should be accounted for in a macro billed as wasm_abigen since we don't know where people may attempt to run this publicly exported macro.

It seems like the safer option is to perform this abigen as a build step using build-deps in a std env, rather than a normal dep in a wasm environment.

cc @vlopes11 since he also has experience with no_std and might have something to add.

Copy link
Member

@Voxelot Voxelot Jun 3, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternately, would using the typescript sdk abigen w/ assemblyscript be a more dev friendly experience for writing wasm indexer modules? @digorithm

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That first error was because cargo test was trying to run WASM as if it were a native executable... so not gonna work. I put in a WASM test harness to run that test.
I think for mac, the only thing needed is a brew install llvm, the one shipped by default does not have a WASM target support.

For loading the json abi file, that only happens in the proc macro. Proc macros are actually compiled prior to the crate that's being built, and only needs to be built for the host architecture, not to WASM. The thing that is important here is that the token stream that is output by the proc macro needs to be supported by WASM. All I need are rust type definitions and a way to deserialize a binary blob into those types, and that's the only thing that the ABI gen should output for WASM.

Copy link
Member

@Voxelot Voxelot Jun 4, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah that makes sense, thanks for completing the test harness example. It works for me on ubuntu, however it appears that it doesn't actually run in CI. Maybe we need a separate job to run cargo test with the appropriate target instead of using a workspace level cargo test? https://github.com/FuelLabs/fuels-rs/runs/6729727596?check_suite_focus=true#step:8:358

As far as my mac issue, I used brew llvm and had the wasm target working, but it got stuck on secp256k:

LLVM ERROR: malformed uleb128, extends past end

As far as the proc macro/no-std issue, that is reassuring. It's surprising to me that cargo loads proc-macros from normal deps instead of build deps, considering they're only run at build time. I wonder if that's because types from the macro deps that get used in the resultant token stream need to be transitively imported for hygiene purposes?

Copy link

@ghost ghost Jun 6, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought the .cargo/config.toml would've handled that? I guess not though. I'll put in a separate command for it.

For mac are you env vars pointed at the right binary? I had to do this:

export PATH="/opt/homebrew/opt/llvm/bin:$PATH"
export CC=/opt/homebrew/opt/llvm/bin/clang
export AR=/opt/homebrew/opt/llvm/bin/llvm-ar

Though that issue looks like it's building secp256k to wasm? I don't think we'll ever have a need for making signatures in wasm, so maybe it should be an optional dep of fuels-core?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah nice, these exports fixed the mac issue for me!

Having secp256k be optional in fuels-core would make sense since fuels-rs only uses secp256k for signing. If someone was writing fancy indexer logic involving some cryptography, they could source their own no-std libs for that. Looks like we'd just need to disable default features for fuel-tx when importing it within fuels-core.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool, I put in a CI step to check WASM. I looked at making fuel-tx optional, but fuels-core would not compile with --no-default-features. Mainly because of the Receipt type, which is something we do need in the indexers, so maybe we should keep it as-is for now?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm yeah maybe file an issue to investigate this later.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, filed it here: #371

Copy link
Member

@Voxelot Voxelot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, thanks for getting the CI and tests setup

Copy link
Member

@digorithm digorithm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is really cool (especially the CI work to support wasm stuff), thanks for putting this one up!

@tjsharp1 tjsharp1 merged commit ed239e0 into master Jun 7, 2022
@tjsharp1 tjsharp1 deleted the wasm-support branch June 7, 2022 22:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

4 participants