Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Webtransport #183

Merged
merged 101 commits into from
Jun 13, 2023
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
101 commits
Select commit Hold shift + click to select a range
84701a7
add cert generator
darioalessandro Mar 29, 2023
dcc248d
save
darioalessandro Mar 29, 2023
21f564d
adding SETTINGS_ENABLE_WEBTRANSPORT
darioalessandro Mar 29, 2023
22aaaa8
revert dup setting
darioalessandro Mar 30, 2023
6ebd3e1
chore: extract connection config to struct
ten3roberts Mar 30, 2023
203d59d
chore: add some helper comments
ten3roberts Mar 30, 2023
54e8946
feat: bare WebTransportSession
ten3roberts Mar 30, 2023
e730761
chore: update .gitignore
ten3roberts Mar 30, 2023
a1c35fd
feat: creating a connection using custom config
ten3roberts Mar 30, 2023
d4812c3
wip: server webtransport settings negotiation
ten3roberts Mar 30, 2023
943d334
fix: cert paths
ten3roberts Mar 30, 2023
47c3120
chore: trust self-signed cert script
ten3roberts Mar 31, 2023
f3f2a80
chore: configure server for webtransport
ten3roberts Mar 31, 2023
38ad758
feat: handle extended settings frame
ten3roberts Mar 31, 2023
52eb68d
feat: support CONNECT and :protocol
ten3roberts Mar 31, 2023
7992a2b
revert changes to original demo
darioalessandro Apr 1, 2023
6337f25
print client header fields for debugging
darioalessandro Apr 1, 2023
c482960
delete mkcert script because it does not work and it is confusing
darioalessandro Apr 1, 2023
d48a11a
got connect to work, now we need to connect datagrams. unidirectional…
darioalessandro Apr 1, 2023
d0bb808
enable chrome debugging
darioalessandro Apr 1, 2023
3accb2d
checkin chrome extension to debug connect request
darioalessandro Apr 1, 2023
f89acdd
add datagrams to h3 connection, also, never drop the stream used to a…
darioalessandro Apr 2, 2023
7d3f1a2
got datagrams echo mode to work
darioalessandro Apr 2, 2023
495e1cd
got datagrams echo mode to work
darioalessandro Apr 2, 2023
3ae8832
feat: use upgrade style api for wt session
ten3roberts Apr 3, 2023
57dcb92
fix: future requiring double await
ten3roberts Apr 3, 2023
8a6dd1d
feat: datagram api
ten3roberts Apr 3, 2023
add97ff
remove Google Chrome extension
darioalessandro Apr 4, 2023
6d503d6
revert examples/server changes
darioalessandro Apr 4, 2023
9c69ca1
remove unrelated changes on examples/client
darioalessandro Apr 4, 2023
254a013
got halfway done implementing unidirectional streams
darioalessandro Apr 4, 2023
7628e3e
feat: proper datagram encoding/decoding
ten3roberts Apr 4, 2023
ff4d51a
chore: move WebTransportSession
ten3roberts Apr 4, 2023
a009765
fix: don't discard incoming recv streams
ten3roberts Apr 5, 2023
126fa50
chore: save session_id
ten3roberts Apr 5, 2023
5091639
feat: resolve session id for uni streams
ten3roberts Apr 6, 2023
0fde7d4
wip: partial read in uni stream
ten3roberts Apr 6, 2023
cc7dea2
feat: bidirectional streams albeit quite hacky
ten3roberts Apr 11, 2023
7f2bc2d
feat: make SendStream allow writing arbitrary data like io::Write
ten3roberts Apr 12, 2023
16eead4
fix: remove generic for SendStream
ten3roberts Apr 12, 2023
e07286c
remove print statements from poll accept request and add keep alive i…
darioalessandro Apr 13, 2023
5f0f3da
Merge branch 'master' into add-webtransport
ten3roberts Apr 13, 2023
92ca2a7
feat: make Connection agnostic to underlying send buffer implementation
ten3roberts Apr 13, 2023
0c42603
feat: generalize peeked reading of streams
ten3roberts Apr 13, 2023
51d66c3
feat: allow requests to pass through WebTransportSession
ten3roberts Apr 13, 2023
9af9ad5
feat: implement AsyncWrite for wt SendStream
ten3roberts Apr 13, 2023
687e3eb
fix: (some) msrv
ten3roberts Apr 14, 2023
869add4
fix: doc tests
ten3roberts Apr 14, 2023
669bee6
wip: open bi
ten3roberts Apr 14, 2023
0e178e2
read unidirectional stream until the end
darioalessandro Apr 18, 2023
d8053b4
got system to echo datagrams and bidi_streams, we can receive uni_str…
darioalessandro Apr 19, 2023
1d8339d
add missing import
darioalessandro Apr 19, 2023
4441c68
send frame type and session id
darioalessandro Apr 19, 2023
fa150c8
almost?
darioalessandro Apr 19, 2023
d5ef8d8
fix: unidirectional stream header encoding
ten3roberts Apr 19, 2023
8597893
chore: cleanup logs
ten3roberts Apr 19, 2023
cde007d
added BIDI stream type, removed unused constants
darioalessandro Apr 20, 2023
6c138d6
Merge branch 'add-webtransport' of github.com:security-union/h3 into …
darioalessandro Apr 20, 2023
644f991
got server initiated uni-streams to work
darioalessandro Apr 20, 2023
9ec8012
wip: multi session support
ten3roberts Apr 20, 2023
0d49d38
wip: new server and session api
ten3roberts Apr 20, 2023
6a355a9
Revert "wip: new server and session api"
darioalessandro Apr 21, 2023
08e39eb
Revert "wip: multi session support"
darioalessandro Apr 21, 2023
374a137
took a stab at breaking webtransport into it's own crate
darioalessandro Apr 21, 2023
e385d6d
add allow_access_to_core feature so that h3 users need to go out of t…
darioalessandro Apr 21, 2023
0540c1d
chore: concurrent handling message in webtransport example
ten3roberts Apr 24, 2023
0c3a07c
chore: use pin-project
ten3roberts Apr 25, 2023
32b1560
chore: move streams to h3-webtransport
ten3roberts Apr 25, 2023
7b6a260
chore: fix warnings
ten3roberts Apr 25, 2023
57576b9
chore: cleanup
ten3roberts Apr 25, 2023
cf2044b
fix: don't always buffer webtransport streams
ten3roberts Apr 25, 2023
64c1da7
fix: don't log header fields
ten3roberts Apr 26, 2023
f4e83fb
feat: webtransport combined bidi streams
ten3roberts Apr 26, 2023
7141521
remove DUMMY setting
darioalessandro Apr 26, 2023
bf13d83
Merge branch 'add-webtransport' of github.com:security-union/h3 into …
darioalessandro Apr 26, 2023
24e6e00
fix: typo
ten3roberts May 3, 2023
63388af
fix: relax OpenStream bounds
ten3roberts May 3, 2023
9ff3c2e
chore: revert quic traits
ten3roberts May 4, 2023
14fba6b
feat: separate trait for unframed stream data
ten3roberts May 4, 2023
3708407
fix: import
ten3roberts May 11, 2023
757dedc
feat: ext module
ten3roberts May 11, 2023
fa7785c
chore: revert 'static + Error
ten3roberts May 11, 2023
74935b5
chore: fix public error internals
ten3roberts May 11, 2023
3dd7732
fix: session id exposed field
ten3roberts May 11, 2023
d781040
feat: split datagram support into traits
ten3roberts May 11, 2023
0bc780f
fix: make BufList private be exposing relevant method in BufRecvStream
ten3roberts May 11, 2023
3cb7955
fix: duplicate method to construct server connection
ten3roberts May 16, 2023
4e44e04
chore: rename `conn`
ten3roberts May 16, 2023
fc67a31
chore: remove inner connection Mutex on `C`
ten3roberts May 16, 2023
cf8fd16
Merge branch 'master' into add-webtransport
ten3roberts May 16, 2023
185e55d
feat: `i-implement-a-third-party-backend-and-opt-into-breaking-change…
ten3roberts May 16, 2023
6c8de63
fix: unnused import
ten3roberts May 16, 2023
6449f79
fix: shutdown todo
ten3roberts May 17, 2023
2b536b7
feat(webtransport): tokio AsyncRead/AsyncWrite
ten3roberts May 17, 2023
64fe30d
fix: panic in shutdown during request
ten3roberts May 25, 2023
48d5f1f
chore: split config into separate struct
ten3roberts May 29, 2023
483eac9
fix: msrv and move datagram to exposed module
ten3roberts May 29, 2023
d68f1c0
fix: doctests
ten3roberts May 29, 2023
db5c723
fix: remove spec comment due to failing to find draft rfc
ten3roberts Jun 1, 2023
a45bfd4
fix: missing feature flag due to feature unification
ten3roberts Jun 13, 2023
fa956e0
chore: address visibility concerns
ten3roberts Jun 13, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion examples/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ rustls-native-certs = "0.6"
structopt = "0.3"
tokio = { version = "1.6", features = ["full"] }
tracing = "0.1.10"
tracing-subscriber = { version = "0.3", default-features = false, features = ["fmt", "ansi", "env-filter", "time", "tracing-log"] }
tracing-subscriber = { version = "0.3", default-features = true, features = ["fmt", "ansi", "env-filter", "time", "tracing-log"] }
darioalessandro marked this conversation as resolved.
Show resolved Hide resolved

[[example]]
name = "client"
Expand Down
7 changes: 7 additions & 0 deletions examples/generate_certs.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@

# use mkcert to generate certs for localhost
mkcert localhost
darioalessandro marked this conversation as resolved.
Show resolved Hide resolved

# convert certs from pem to der
openssl x509 -outform der -in localhost.pem -out localhost.crt
openssl rsa -outform der -in localhost-key.pem -out localhost.key
7 changes: 3 additions & 4 deletions examples/server.rs
Original file line number Diff line number Diff line change
Expand Up @@ -54,15 +54,13 @@ pub struct Certs {
pub key: PathBuf,
}

static ALPN: &[u8] = b"h3";

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
tracing_subscriber::fmt()
.with_env_filter(tracing_subscriber::EnvFilter::from_default_env())
.with_span_events(tracing_subscriber::fmt::format::FmtSpan::FULL)
.with_writer(std::io::stderr)
.with_max_level(tracing::Level::INFO)
.with_max_level(tracing::Level::TRACE)
darioalessandro marked this conversation as resolved.
Show resolved Hide resolved
.init();

// process cli arguments
Expand Down Expand Up @@ -97,7 +95,8 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
.with_single_cert(vec![cert], key)?;

tls_config.max_early_data_size = u32::MAX;
tls_config.alpn_protocols = vec![ALPN.into()];
let ALPN: Vec<Vec<u8>> = vec!(b"h3".to_vec(), b"h3-32".to_vec(), b"h3-31".to_vec(), b"h3-30".to_vec(), b"h3-29".to_vec());
tls_config.alpn_protocols = ALPN;

let server_config = quinn::ServerConfig::with_crypto(Arc::new(tls_config));
let (endpoint, mut incoming) = quinn::Endpoint::server(server_config, opt.listen)?;
Expand Down
6 changes: 6 additions & 0 deletions h3/src/connection.rs
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,12 @@ where
settings
.insert(SettingId::MAX_HEADER_LIST_SIZE, max_field_section_size)
.map_err(|e| Code::H3_INTERNAL_ERROR.with_cause(e))?;
settings.insert(SettingId::ENABLE_CONNECT_PROTOCOL, 1)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

attempt to send the webtransport headers on the server side

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would like to know the release time of the webtransport server. I am really looking forward to your project

Copy link
Contributor Author

@darioalessandro darioalessandro Apr 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @redoriental thanks for reaching out!!

I encourage you to try out the h3-webtransport crate by installing it from this branch by adding this to your cargo dependencies:

futures = "0.3"
h3-webtransport = { git= "https://github.com/security-union/h3.git", rev = "f4e83fb9be28ef3ea99e5feed725550f0169e5c2" } 

that way you wont have to wait for this PR to be merged.

We are currently working on separating h3 and h3-webtransport, and based on the ongoing discussions, it appears that this process may take some time.

The nature of open-source development can result in a slower pace, as contributors often have limited time to dedicate, and individuals from diverse backgrounds, opinions and cultures must come to a consensus on various objectives. However, rest assured that we will ultimately deliver a well-organized and efficient webtransport server!

.map_err(|e| Code::H3_INTERNAL_ERROR.with_cause(e))?;
settings.insert(SettingId::ENABLE_WEBTRANSPORT, 1)
.map_err(|e| Code::H3_INTERNAL_ERROR.with_cause(e))?;
settings.insert(SettingId::H3_DATAGRAM, 1)
.map_err(|e| Code::H3_INTERNAL_ERROR.with_cause(e))?;

if grease {
// Grease Settings (https://www.rfc-editor.org/rfc/rfc9114.html#name-defined-settings-parameters)
Expand Down
26 changes: 26 additions & 0 deletions h3/src/proto/frame.rs
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ pub enum Frame<B> {
PushPromise(PushPromise),
Goaway(VarInt),
MaxPushId(PushId),
WebTransportStream(WebTransportStream),
Grease,
}

Expand Down Expand Up @@ -80,13 +81,15 @@ impl Frame<PayloadLen> {
}

let mut payload = buf.take(len as usize);
trace!("frame ty: {:?}", ty);
let frame = match ty {
FrameType::HEADERS => Ok(Frame::Headers(payload.copy_to_bytes(len as usize))),
FrameType::SETTINGS => Ok(Frame::Settings(Settings::decode(&mut payload)?)),
FrameType::CANCEL_PUSH => Ok(Frame::CancelPush(payload.get_var()?.try_into()?)),
FrameType::PUSH_PROMISE => Ok(Frame::PushPromise(PushPromise::decode(&mut payload)?)),
FrameType::GOAWAY => Ok(Frame::Goaway(VarInt::decode(&mut payload)?)),
FrameType::MAX_PUSH_ID => Ok(Frame::MaxPushId(payload.get_var()?.try_into()?)),
FrameType::WEBTRANSPORT_STREAM =>Ok(Frame::WebTransportStream(WebTransportStream { buffer: payload.copy_to_bytes(len as usize)})),
FrameType::H2_PRIORITY
| FrameType::H2_PING
| FrameType::H2_WINDOW_UPDATE
Expand Down Expand Up @@ -132,6 +135,7 @@ where
buf.write_var(6);
buf.put_slice(b"grease");
}
Frame::WebTransportStream(_) => todo!(),
}
}
}
Expand Down Expand Up @@ -189,6 +193,7 @@ impl fmt::Debug for Frame<PayloadLen> {
Frame::Goaway(id) => write!(f, "GoAway({})", id),
Frame::MaxPushId(id) => write!(f, "MaxPushId({})", id),
Frame::Grease => write!(f, "Grease()"),
Frame::WebTransportStream(_) => todo!(),
}
}
}
Expand All @@ -207,6 +212,7 @@ where
Frame::Goaway(id) => write!(f, "GoAway({})", id),
Frame::MaxPushId(id) => write!(f, "MaxPushId({})", id),
Frame::Grease => write!(f, "Grease()"),
Frame::WebTransportStream(_) => write!(f, "WebTransportStream()"),
}
}
}
Expand All @@ -226,6 +232,9 @@ impl<T, U> PartialEq<Frame<T>> for Frame<U> {
Frame::Goaway(x) => matches!(other, Frame::Goaway(y) if x == y),
Frame::MaxPushId(x) => matches!(other, Frame::MaxPushId(y) if x == y),
Frame::Grease => matches!(other, Frame::Grease),
Frame::WebTransportStream(x) => {
matches!(other, Frame::WebTransportStream(y) if x == y)
}
}
}
}
Expand Down Expand Up @@ -257,6 +266,7 @@ frame_types! {
H2_WINDOW_UPDATE = 0x8,
H2_CONTINUATION = 0x9,
MAX_PUSH_ID = 0xD,
WEBTRANSPORT_STREAM = 0x41,
}

impl FrameType {
Expand Down Expand Up @@ -297,6 +307,11 @@ pub struct PushPromise {
encoded: Bytes,
}

#[derive(Debug, PartialEq)]
pub struct WebTransportStream
{
buffer: Bytes,
}
impl FrameHeader for PushPromise {
const TYPE: FrameType = FrameType::PUSH_PROMISE;

Expand Down Expand Up @@ -389,6 +404,17 @@ setting_identifiers! {
QPACK_MAX_TABLE_CAPACITY = 0x1,
QPACK_MAX_BLOCKED_STREAMS = 0x7,
MAX_HEADER_LIST_SIZE = 0x6,
QPACK_BLOCKED_STREAMS = 0x7,
// https://datatracker.ietf.org/doc/html/rfc9220#section-5
ENABLE_CONNECT_PROTOCOL = 0x8,
// https://datatracker.ietf.org/doc/html/draft-ietf-masque-h3-datagram-05#section-9.1
H3_DATAGRAM = 0xFFD277,
// https://datatracker.ietf.org/doc/html/draft-ietf-webtrans-http2-02#section-10.1
ENABLE_WEBTRANSPORT = 0x2B603742,
H3_SETTING_ENABLE_DATAGRAM_CHROME_SPECIFIC= 0xFFD277,
// Dummy setting to check it is correctly ignored by the peer.
// https://datatracker.ietf.org/doc/html/rfc9114#section-7.2.4.1
DUMMY = 0x21,
darioalessandro marked this conversation as resolved.
Show resolved Hide resolved
}

const SETTINGS_LEN: usize = 4;
Expand Down
1 change: 1 addition & 0 deletions h3/src/proto/stream.rs
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ stream_types! {
PUSH = 0x01,
ENCODER = 0x02,
DECODER = 0x03,
WEBTRANSPORT = 0x54,
}

impl StreamType {
Expand Down
7 changes: 5 additions & 2 deletions h3/src/server.rs
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ use crate::{
quic::{self, RecvStream as _, SendStream as _},
stream,
};
use tracing::{error, trace, warn};
use tracing::{error, trace, warn, info};

/// Create a builder of HTTP/3 server connections
///
Expand Down Expand Up @@ -353,8 +353,11 @@ where
&mut self,
cx: &mut Context<'_>,
) -> Poll<Result<Option<C::BidiStream>, Error>> {
info!("poll_accept_request");
let _ = self.poll_control(cx)?;
info!("poll_accept_request: poll_control done");
let _ = self.poll_requests_completion(cx);
info!("poll_accept_request: poll_requests_completion done");
loop {
match self.inner.poll_accept_request(cx) {
Poll::Ready(Err(x)) => break Poll::Ready(Err(x)),
Expand Down Expand Up @@ -400,7 +403,7 @@ where
fn poll_control(&mut self, cx: &mut Context<'_>) -> Poll<Result<(), Error>> {
while let Poll::Ready(frame) = self.inner.poll_control(cx)? {
match frame {
Frame::Settings(_) => trace!("Got settings"),
Frame::Settings(w) => trace!("Got settings > {:?}", w),
darioalessandro marked this conversation as resolved.
Show resolved Hide resolved
Frame::Goaway(id) => self.inner.process_goaway(&mut self.recv_closing, id)?,
f @ Frame::MaxPushId(_) | f @ Frame::CancelPush(_) => {
warn!("Control frame ignored {:?}", f);
Expand Down