Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Sanction List #30

Merged
merged 25 commits into from
Feb 13, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
03cc81f
feat: add thread-friendly refresh_list
ppoliani Jan 28, 2024
da4ee6e
feat: implement address verifier what keeps a list of sanctioned addr…
ppoliani Jan 28, 2024
1f15224
chore: format the code
ppoliani Jan 28, 2024
0118e00
feat: extract publish_date
ppoliani Jan 29, 2024
e81e101
chore: add additional logs
ppoliani Jan 29, 2024
4aa65fa
chore: move tick to the top of the loop
ppoliani Jan 29, 2024
e3e680a
chore: remoev regex crate
ppoliani Jan 29, 2024
61be243
feat: implement is_sanctioned
ppoliani Jan 29, 2024
867395e
feat: move the sync logic to a separate fn
ppoliani Jan 29, 2024
7ac17d1
Merge branch 'main' into feat/sanctioned_list
ppoliani Jan 30, 2024
9656352
feat: return JoinHandle from the addresse verifier start fn
ppoliani Jan 31, 2024
bda4e3d
feat: sync sanction list before starting the node
ppoliani Jan 31, 2024
5433684
feat: start address verifier in the node
ppoliani Jan 31, 2024
6bea209
feat: update last_udpate ts in the sync fn
ppoliani Jan 31, 2024
d6abe15
feat: remove async from start fn
ppoliani Jan 31, 2024
b1f4845
refactor: rename to compliance
ppoliani Jan 31, 2024
fcd558e
chore: do not await on the compliance loop
ppoliani Jan 31, 2024
76dfc14
chore: start compliance in the orchestrator
ppoliani Jan 31, 2024
4f112ac
feat: use concurrency primitives for the compliance struct
ppoliani Jan 31, 2024
33a3cd6
feat: wrap compliance instance in Arc
ppoliani Jan 31, 2024
6a1d56d
feat: check compliance of the zkapp tx
ppoliani Jan 31, 2024
9e50f5a
chore: keep sync_internal private
ppoliani Jan 31, 2024
d85cc9e
fix: linting issues
ppoliani Feb 1, 2024
054ed3a
chore: check that it's not sanctioned
ppoliani Feb 5, 2024
2e7f604
chore: update error message
ppoliani Feb 5, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -13,3 +13,4 @@ Cargo.lock
# MSVC Windows builds of rustc generate these, which store debugging information
*.pdb
.vscode
local_keys
3 changes: 3 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,9 @@ tokio = { version = "1.34", features = [
"macros",
] }
tokio-stream = "0.1.14"
xml = "0.8.10"
fancy-regex = "0.13.0"
chrono = "0.4.33"

[patch.crates-io]
# see docs/serialization.md
Expand Down
21 changes: 20 additions & 1 deletion src/bob_request.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
use std::{collections::HashMap, path::Path, str::FromStr, vec};
use std::{collections::HashMap, path::Path, str::FromStr, sync::Arc, vec};

use anyhow::{bail, ensure, Context, Result};
use bitcoin::{
Expand All @@ -11,10 +11,12 @@ use serde::{Deserialize, Serialize};

use crate::{
circom_field_from_bytes, circom_field_to_bytes,
compliance::Compliance,
constants::{
FEE_ZKBITCOIN_SAT, MINIMUM_CONFIRMATIONS, STATEFUL_ZKAPP_PUBLIC_INPUT_LEN,
ZKBITCOIN_FEE_PUBKEY, ZKBITCOIN_PUBKEY,
},
get_network,
json_rpc_stuff::{
createrawtransaction, fund_raw_transaction, get_transaction, json_rpc_request,
TransactionOrHex,
Expand Down Expand Up @@ -445,6 +447,23 @@ impl BobRequest {
Ok(())
}

/// Check that the zkapp input transactions are compliant
pub async fn check_compliance(&self, compliance: Arc<Compliance>) -> Result<()> {
for zkapp_txin in &self.zkapp_tx.input {
let addr = Address::from_script(
&zkapp_txin.script_sig.clone().into_boxed_script(),
get_network(),
)?;

ensure!(
!compliance.is_sanctioned(&addr).await,
"ZkApp input transaction is sanctioned"
);
ppoliani marked this conversation as resolved.
Show resolved Hide resolved
}

Ok(())
}

/// Validates a request received from Bob.
pub async fn validate_request(&self) -> Result<SmartContract> {
// extract smart contract from tx
Expand Down
24 changes: 21 additions & 3 deletions src/committee/orchestrator.rs
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ use tokio::time::sleep;
use crate::{
bob_request::{BobRequest, BobResponse},
committee::node::Round1Response,
compliance::Compliance,
constants::{KEEPALIVE_MAX_RETRIES, KEEPALIVE_WAIT_SECONDS, ZKBITCOIN_PUBKEY},
frost,
json_rpc_stuff::{json_rpc_request, RpcCtx},
Expand Down Expand Up @@ -268,24 +269,30 @@ pub struct Orchestrator {
pub pubkey_package: frost_secp256k1_tr::keys::PublicKeyPackage,
pub committee_cfg: CommitteeConfig,
pub member_status: Arc<RwLock<MemberStatusState>>,
compliance: Arc<Compliance>,
}

impl Orchestrator {
pub fn new(
pubkey_package: frost_secp256k1_tr::keys::PublicKeyPackage,
committee_cfg: CommitteeConfig,
member_status: Arc<RwLock<MemberStatusState>>,
compliance: Arc<Compliance>,
) -> Self {
Self {
pubkey_package,
committee_cfg,
member_status,
compliance,
}
}

/// Handles bob request from A to Z.
pub async fn handle_request(&self, bob_request: &BobRequest) -> Result<BobResponse> {
// Validate transaction before forwarding it, and get smart contract
bob_request
.check_compliance(Arc::clone(&self.compliance))
.await?;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wondering: should individual nodes perform that check as well? What would be the rational for not performing that check at the node level (and would node be liable if something bad happens there...)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could, but we've got orchestrator as a centralized entity that coordinates the entire process.

let smart_contract = bob_request.validate_request().await?;

// TODO: we might want to check that the zkapp/UTXO is unspent here, but this requires us to have access to a bitcoin node, so for now we don't do it :o)
Expand Down Expand Up @@ -567,15 +574,26 @@ pub async fn run_server(
let address = address.unwrap_or("127.0.0.1:6666");
info!("- starting orchestrator at address http://{address}");

let mut compliance = Compliance::new();
// Orchestrator should sync the Sanction ist before doing anything else
ppoliani marked this conversation as resolved.
Show resolved Hide resolved
compliance.sync().await.expect("sync sanction list");

// wrap in an Arc after the first sync so it can be used in multiple request contexts
let compliance: Arc<Compliance> = Arc::new(compliance);

let member_status_state = Arc::new(RwLock::new(MemberStatusState::new(&committee_cfg).await));
let mss_thread_copy = member_status_state.clone();
tokio::spawn(async move { MemberStatusState::keepalive_thread(mss_thread_copy).await });

let ctx = Orchestrator {
let ctx = Orchestrator::new(
pubkey_package,
committee_cfg,
member_status: member_status_state,
};
member_status_state,
Arc::clone(&compliance),
);

// Sync sanction list in a parallel thread
compliance.start();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't you have to use the joinhandle returned?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not if you don't want to wait for it to complete


let server = Server::builder()
.build(address.parse::<SocketAddr>()?)
Expand Down
171 changes: 171 additions & 0 deletions src/compliance.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,171 @@
use anyhow::{Context, Result};
use bitcoin::Address;
use chrono::prelude::*;
use fancy_regex::Regex;
use futures::StreamExt;
use log::{error, info};
use std::{
collections::HashMap,
sync::Arc,
time::{Duration, Instant},
};
use tokio::{spawn, sync::RwLock, task::JoinHandle, time::interval};
use xml::reader::{EventReader, XmlEvent};

pub struct Compliance {
sanctioned_addresses: Arc<RwLock<HashMap<String, bool>>>,
last_update: Arc<RwLock<i64>>,
}

impl Default for Compliance {
fn default() -> Self {
Self::new()
}
}
ppoliani marked this conversation as resolved.
Show resolved Hide resolved

impl Compliance {
const BTC_ID: &'static str = "344";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you document where you got that value from? (if it's possible)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh I couldn't find any documentation. This is a gov document, didn't expect anything better to be honest. I had to look into the XML file and find that number.

const OFAC_URL: &'static str =
"https://www.treasury.gov/ofac/downloads/sanctions/1.0/sdn_advanced.xml";

pub fn new() -> Self {
Self {
sanctioned_addresses: Arc::new(RwLock::new(HashMap::new())),
last_update: Arc::new(RwLock::new(0)),
}
}

fn extract_from_xml(str_value: &str, tag: &str) -> Result<u32> {
let re = Regex::new(&format!(r"(?<={}>)\s*(\w+)(?=<\/{})", tag, tag)).unwrap();
let value = re.find(str_value)?.context("no regex result")?.as_str();

Ok(value.parse()?)
}

/// read the first few bytes from the remote XML file and extract the last update date.
/// If there is no fresh data we can skip the parsing of XML which is slow.
async fn publish_date() -> Result<i64> {
let res = reqwest::get(Self::OFAC_URL).await?;

let head = res
.bytes_stream()
.take(1)
.collect::<Vec<reqwest::Result<_>>>()
.await
.into_iter()
.collect::<reqwest::Result<Vec<_>>>()?;

let str_value = String::from_utf8(head[0].to_vec())?;
let year = Self::extract_from_xml(&str_value, "Year")?;
let day = Self::extract_from_xml(&str_value, "Day")?;
let month = Self::extract_from_xml(&str_value, "Month")?;
let date = Utc
.with_ymd_and_hms(year as i32, month, day, 0, 0, 0)
.single()
.context("date parse error")?
.timestamp();

Ok(date)
}

pub async fn sync(&mut self) -> Result<()> {
Self::sync_internal(
Arc::clone(&self.sanctioned_addresses),
Arc::clone(&self.last_update),
)
.await?;

Ok(())
}

/// Runs the Sanction list syncronization. Downloads the remote XML file and extracts the sanctioned addresses
async fn sync_internal(
sanctioned_addresses: Arc<RwLock<HashMap<String, bool>>>,
last_update: Arc<RwLock<i64>>,
) -> Result<()> {
let publish_date = Self::publish_date().await?;

if *last_update.read().await >= publish_date {
info!("Sanction list is up-to-date");
return Ok(());
}

let mut last_update = last_update.write().await;
*last_update = publish_date;
ppoliani marked this conversation as resolved.
Show resolved Hide resolved

info!("Syncing sanction list...");
let start = Instant::now();
let res = reqwest::get(Self::OFAC_URL).await?;

let xml = res.text().await?;
let parser: EventReader<&[u8]> = EventReader::new(xml.as_bytes());
let mut inside_feature_elem = false;
let mut inside_final_elem = false;

let mut sanctioned_addresses = sanctioned_addresses.write().await;
for e in parser {
match e {
Ok(XmlEvent::StartElement {
name, attributes, ..
}) => {
if name.local_name == "Feature" {
if attributes.iter().any(|a| {
a.name.local_name == "FeatureTypeID" && a.value == Self::BTC_ID
}) {
inside_feature_elem = true;
}
} else if name.local_name == "VersionDetail" && inside_feature_elem {
inside_final_elem = true;
}
}
Ok(XmlEvent::Characters(value)) => {
if inside_final_elem {
sanctioned_addresses.insert(value, true);
}
}
Ok(XmlEvent::EndElement { name, .. }) => {
if name.local_name == "VersionDetail" && inside_feature_elem {
inside_feature_elem = false;
inside_final_elem = false;
}
}
Err(e) => {
error!("Error parsing xml: {e}");
break;
}
_ => {}
}
}

let duration = start.elapsed();
info!("Sanction list synced in {:?}", duration);

Ok(())
}

/// Periodically fetces the latest list from OFAC_URL and updates the local list
pub fn start(&self) -> JoinHandle<()> {
let sanctioned_addresses = Arc::clone(&self.sanctioned_addresses);
let last_update = Arc::clone(&self.last_update);

spawn(async move {
let mut interval = interval(Duration::from_secs(600));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should hardcode that value somewhere

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes I will do that


loop {
interval.tick().await;
if let Err(error) =
Self::sync_internal(Arc::clone(&sanctioned_addresses), Arc::clone(&last_update))
.await
{
error!("Sanction list sync error: {}", error);
};
}
})
}

/// Returns true if the given address is in the sanction list
pub async fn is_sanctioned(&self, address: &Address) -> bool {
let sanctioned_addresses = self.sanctioned_addresses.read().await;
sanctioned_addresses.contains_key(&address.to_string())
}
}
1 change: 1 addition & 0 deletions src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ use secp256k1::hashes::Hash;

pub mod capped_hashmap;
pub mod committee;
pub mod compliance;
pub mod constants;
pub mod frost;
pub mod json_rpc_stuff;
Expand Down
Loading