Skip to content

Commit

Permalink
Merge branch 'v4' into v3
Browse files Browse the repository at this point in the history
  • Loading branch information
Nuhvi committed Dec 12, 2024
2 parents e0edf75 + 8069e1e commit c3510a8
Show file tree
Hide file tree
Showing 39 changed files with 1,665 additions and 397 deletions.
5 changes: 2 additions & 3 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[workspace]
members = [
"pkarr", "server",
"pkarr", "relay",
]

# See: https://github.com/rust-lang/rust/issues/90148#issuecomment-949194352
Expand Down
10 changes: 5 additions & 5 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ COPY . .
RUN cargo build --release --target x86_64-unknown-linux-musl

# Strip the binary to reduce size
RUN strip target/x86_64-unknown-linux-musl/release/pkarr-server
RUN strip target/x86_64-unknown-linux-musl/release/pkarr-relay

# ========================
# Runtime Stage
Expand All @@ -44,13 +44,13 @@ FROM alpine:3.20
RUN apk add --no-cache ca-certificates

# Copy the compiled binary from the builder stage
COPY --from=builder /usr/src/app/target/x86_64-unknown-linux-musl/release/pkarr-server /usr/local/bin/pkarr-server
COPY --from=builder /usr/src/app/target/x86_64-unknown-linux-musl/release/pkarr-relay /usr/local/bin/pkarr-relay

# Set the working directory
WORKDIR /usr/local/bin

# Expose the port the pkarr server listens on (should match that of config.toml)
# Expose the port the pkarr relay listens on (should match that of config.toml)
EXPOSE 6881

# Set the default command to run the homeserver binary
CMD ["pkarr-server", "--config=./config.toml"]
# Set the default command to run the relay binary
CMD ["pkarr-relay", "--config=./config.toml"]
34 changes: 17 additions & 17 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@ Where we are going, this [https://o4dksfbqk85ogzdb5osziw6befigbuxmuxkuxq8434q89u
## TLDR
- To publish resource records for your key, sign a small encoded DNS packet (<= 1000 bytes) and publish it on the DHT (through a relay if necessary).
- To resolve some key's resources, applications query the DHT directly, or through a [relay](./design/relays.md), and verify the signature themselves.
- Clients and Pkarr servers cache records extensively and minimize DHT traffic as much as possible for improved scalability.
- The DHT drops records after a few hours, so users, their friends, or service providers should periodically republish their records to the DHT. Also Pkarr servers could republish records recently requested, to keep popular records alive too.
- Clients and Relays cache records extensively and minimize DHT traffic as much as possible for improved scalability.
- The DHT drops records after a few hours, so users, their friends, or service providers should periodically republish their records to the DHT. Also Pkarr relays could republish records recently requested, to keep popular records alive too.
- Optional: Existing applications unaware of Pkarr can still function if the user added a Pkarr-aware DNS servers to their operating system DNS servers.

## DEMO
Expand All @@ -30,14 +30,14 @@ Or if you prefer Rust [Examples](./pkarr/examples/README.md)
```mermaid
sequenceDiagram
participant Client
participant Server
participant Relay
participant DHT
participant Republisher
Client->>Server: Publish
note over Server: Optional Pkarr Server
Server->>DHT: Put
Note over Server,DHT: Store signed DNS packet
Client->>Relay: Publish
note over Relay: Optional Pkarr Relay
Relay->>DHT: Put
Note over Relay,DHT: Store signed DNS packet
Client->>Republisher: Republish request
note over Client, Republisher: Notify Hosting provider mentioned in RRs
Expand All @@ -46,31 +46,31 @@ sequenceDiagram
Republisher->>DHT: Republish
end
Client->>Server: Resolve
Server->>DHT: Get
DHT->>Server: Response
Server->>Client: Response
Client->>Relay: Resolve
Relay->>DHT: Get
DHT->>Relay: Response
Relay->>Client: Response
```

### Clients
#### Pkarr enabled applications.

Native applications, can directly query and verify signed records from the DHT if they are not behind NAT. Otherwise, they will need to use a Pkarr server as a relay.
Native applications, can directly query and verify signed records from the DHT if they are not behind NAT. Otherwise, they will need to use a Pkarr Relay.

Browser web apps should try calling local Pkarr server at the default port `6881`, if not accessible, they have to query a remote server instead. Eitherway, these apps should allow users to configure servers of their choice.
Browser web apps should try calling local Pkarr relay at the default port `6881`, if not accessible, they have to query a remote relay in parallel to fallback on. Eitherway, these apps should allow users to configure relays of their choice.

Clients with private keys are also capable of submitting signed records either to the DHT directly, or through Pkarr relay server, to update user's records when needed.
Clients with private keys are also capable of submitting signed records either to the DHT directly, or through Pkarr relay, to update user's records when needed.

#### Existing applications
To support existing applications totally oblivious of Pkarr, users will have to (manually or programatically) edit their OS DNS servers to add one or more DNS servers that recognize Pkarr and query the DHT to resolve packets from there. However, the best outcome would be adoption from existing widely used resolvers like `1.1.1.1` and `8.8.8.8`.

### Servers
### Relays

Pkarr relays are optional but they:
1. Act as [relays](https://pkarr.org/relays) to enable web applications to query the DHT.
2. Act as [resolvers](https://pkarr.org/resolvers) to provide lower latency, more reliability and scalability.

Relays are very light and cheap to operate, that they can easily run altruistically, but private, and paid servers are possible too.
Relays are very light and cheap to operate, that they can easily run altruistically, but private, and paid relays are possible too.

### Republishers

Expand All @@ -96,7 +96,7 @@ To ensure a good chance of scalability and resilience, a few expectations need t
- Popular records may or may not be refreshed by the DNS servers as they get queries for them.
2. This is **not a realtime communication** medium
- Records are heavily cached like in any DNS system.
- You are expected to update your records rarely, so you should expect servers to enforce harsh rate-limiting and maybe demand proof of work.
- You are expected to update your records rarely, so you should expect relays to enforce harsh rate-limiting.
- Records are going to be cached heavily to reduce traffic on the DHT, so updates might take some time to propagate, even if you set TTL to 1 second.
- In case of a chache miss, traversing the DHT might take few seconds.

Expand Down
10 changes: 5 additions & 5 deletions flake.nix
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@
"Cargo.toml"
"Cargo.lock"
"pkarr"
"server"
"relay"
];

buildSrc = flakeboxLib.filterSubPaths {
Expand Down Expand Up @@ -56,16 +56,16 @@
workspaceBuild = craneLib.buildWorkspace {
cargoArtifacts = workspaceDeps;
};
"pkarr-server" = craneLib.buildPackageGroup {
packages = [ "pkarr-server" ];
mainProgram = "pkarr-server";
"pkarr-relay" = craneLib.buildPackageGroup {
packages = [ "pkarr-relay" ];
mainProgram = "pkarr-relay";
};
}
);
in
{
packages = {
pkarr-server = multiBuild.pkarr-server;
pkarr-relay = multiBuild.pkarr-relay;
};

legacyPackages = multiBuild;
Expand Down
15 changes: 14 additions & 1 deletion CHANGELOG.md → pkarr/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,24 @@
# Changelog

All notable changes to pkarr client and server will be documented in this file.
All notable changes to pkarr will be documented in this file.

## [Unreleased]

### Added

- Add feature `endpoints` to resolve `HTTPS` and `SVCB` endpoints over Pkarr
- Add feature `reqwest-resolve` to create a custom `reqwest::dns::Resolve` implementation from `Client` and `relay::client::Client`
- Add feature `tls` to create `rustls::ClientConfig` from `Client` and `relay::client::Client` and create `rustls::ServerCongif` from `KeyPair`.
- Add feature `reqwest-builder` to create a `reqwest::ClientBuilder` from `Client` and `relay::client::Client` using custom dns resolver and preconfigured rustls client config.

### Changed

- Replace `Settings` with `Config` with public fields.

## [3.0.0](https://github.com/pubky/mainline/compare/v2.2.0...v3.0.0) - 2024-12-05

### Added

- Add `SignedPacket::builder()` and convenient methods to create `A`, `AAAA`, `CNAME`, `TXT`, `SVCB`, and `HTTPS` records.
- Add `SignedPacket::all_resource_records()` to access all resource records without accessing the dns packet.
- Use `pubky_timestamp::Timestamp`
Expand Down
14 changes: 11 additions & 3 deletions pkarr/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ webpki-roots = { version = "0.26.7", optional = true }
pubky-timestamp = { version = "0.2.0", default-features = false }

# feat: dht dependencies
mainline = { version = "4.1.0", optional = true }
mainline = { git = "https://github.com/pubky/mainline", branch="v5", optional = true }

# feat: relay dependencies
reqwest = { version = "0.12.9", default-features = false, features = ["rustls-tls"], optional = true }
Expand Down Expand Up @@ -92,11 +92,19 @@ serde = ["dep:serde", "pubky-timestamp/serde", "pubky-timestamp/httpdate"]
# Extra
## Use [crate::extra::lmdb-cache::LmdbCache]
lmdb-cache = ["dep:heed", "dep:byteorder", "dep:page_size"]
## Use [extra::endpoints::EndpointsResolver] trait implementation for [Client] and [client::relay::Client]
endpoints = ["dep:futures-lite", "dep:genawaiter"]
## Use [reqwest::dns::Resolve] trait implementation for [Client] and [client::relay::Client]
reqwest-resolve = ["dep:reqwest", "endpoints"]
## Use [rustls::ClientConfig] from [Client] and [client::relay::Client] for e2ee transport to Pkarr endpoints
tls = ["rustls", "ed25519-dalek/pkcs8", "dep:webpki"]
## Create a [reqwest::ClientBuilder] from [Client] or [client::relay::Client]
reqwest-builder = ["tls", "reqwest-resolve"]

## Use all features
full = ["dht", "relay", "serde","lmdb-cache"]
full = ["dht", "relay", "serde", "endpoints", "lmdb-cache", "reqwest-resolve", "tls", "reqwest-builder"]

default = ["dht"]
default = ["full"]

[package.metadata.docs.rs]
all-features = true
Expand Down
16 changes: 16 additions & 0 deletions pkarr/examples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,3 +23,19 @@ or to use a Relay client:
```sh
cargo run --features relay --example resolve <zbase32 public key from Publish step>
```

## HTTP

Run an HTTP server listening on a Pkarr key

```sh
cargo run --features endpoints --example http-serve <ip address> <port number>
```

An HTTPs url will be printend with the Pkarr key as the TLD, paste in another terminal window:

```sh
cargo run --features reqwest-resolve --example http-get <url>
```

And you should see a `Hello, World!` response.
42 changes: 42 additions & 0 deletions pkarr/examples/http-get.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
//! Make an HTTP request over to a Pkarr address using Reqwest
use reqwest::Method;
use tracing::Level;
use tracing_subscriber;

use clap::Parser;

use pkarr::{Client, PublicKey};

#[derive(Parser)]
#[command(author, version, about, long_about = None)]
struct Cli {
/// Url to GET from
url: String,
}

#[tokio::main]
async fn main() -> anyhow::Result<()> {
tracing_subscriber::fmt().with_max_level(Level::INFO).init();

let cli = Cli::parse();
let url = cli.url;

let reqwest = if PublicKey::try_from(url.as_str()).is_err() {
// If it is not a Pkarr domain, use normal Reqwest
reqwest::Client::new()
} else {
let client = Client::builder().build()?;

reqwest::ClientBuilder::from(client).build()?
};

println!("GET {url}..");
let response = reqwest.request(Method::GET, &url).send().await?;

let body = response.text().await?;

println!("{body}");

Ok(())
}
76 changes: 76 additions & 0 deletions pkarr/examples/http-serve.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
//! Run an HTTP server listening on a Pkarr domain
//!
//! This server will _not_ be accessible from other networks
//! unless the provided IP is public and the port number is forwarded.
use tracing::Level;
use tracing_subscriber;

use axum::{routing::get, Router};
use axum_server::tls_rustls::RustlsConfig;

use std::net::{SocketAddr, ToSocketAddrs};
use std::sync::Arc;

use clap::Parser;

use pkarr::{dns::rdata::SVCB, Client, Keypair, SignedPacket};

#[derive(Parser)]
#[command(author, version, about, long_about = None)]
struct Cli {
/// IP address to listen on
ip: String,
/// Port number to listen no
port: u16,
}

#[tokio::main]
async fn main() -> anyhow::Result<()> {
tracing_subscriber::fmt().with_max_level(Level::INFO).init();

let cli = Cli::parse();

let addr = format!("{}:{}", cli.ip, cli.port)
.to_socket_addrs()?
.next()
.ok_or(anyhow::anyhow!(
"Could not convert IP and port to socket addresses"
))?;

let keypair = Keypair::random();

let client = Client::builder().build()?;

// Run a server on Pkarr
println!("Server listening on {addr}");

// You should republish this every time the socket address change
// and once an hour otherwise.
publish_server_pkarr(&client, &keypair, &addr).await;

println!("Server running on https://{}", keypair.public_key());

let server = axum_server::bind_rustls(
addr,
RustlsConfig::from_config(Arc::new(keypair.to_rpk_rustls_server_config())),
);

let app = Router::new().route("/", get(|| async { "Hello, world!" }));
server.serve(app.into_make_service()).await?;

Ok(())
}

async fn publish_server_pkarr(client: &Client, keypair: &Keypair, socket_addr: &SocketAddr) {
let mut svcb = SVCB::new(0, ".".try_into().expect("infallible"));
svcb.set_port(socket_addr.port());

let signed_packet = SignedPacket::builder()
.https(".".try_into().unwrap(), svcb, 60 * 60)
.address(".".try_into().unwrap(), socket_addr.ip(), 60 * 60)
.sign(&keypair)
.unwrap();

client.publish(&signed_packet).await.unwrap();
}
5 changes: 0 additions & 5 deletions pkarr/src/base/mod.rs

This file was deleted.

File renamed without changes.
Loading

0 comments on commit c3510a8

Please sign in to comment.