100-continue response with BodyStream impossible

[fungs]

Version
Hyper 1.5 server (HTTP1 mode)

Description
Hey there, Hyper team! I've run into a bit of a pickle with Hyper 1.5 and the handling of 100-continue responses. Let me break it down:

1. The newer streaming API (BodyStream) doesn't trigger an automatic 100-continue informational response when a data frame is polled. This is different from the older Body API, which sends the 100-continue when the body is first polled.

2. Trying to send a 100-continue response manually in the application using return Ok(Response::new(Body::empty())) results in a runtime error: UnsupportedStatusCode. This leaves us in a bit of a bind.

3. I've seen suggestions to inject the response at the TCP layer, but that's pretty awkward, especially when using TLS encryption. It feels like we're working against the library rather than with it.

I'm curious why the logic that works for Body and the older API (as described in PR #2119, which added automatic 100-continue support for Body) doesn't work with BodyStream. That PR essentially made Hyper automatically send a 100-continue response when the body is first polled if the request included an Expect: 100-continue header.

Would it be possible to implement support for the 100-continue response using the body streaming interface in the same way as it's done for Body? It seems like this would provide a consistent experience across both APIs.

I noticed PR #3815, which adds support for sending informational responses. Will this solve the problem? It looks like it adds a send_informational method to the response, but I'm not sure if this will integrate with the automatic 100-continue behavior.

Thanks for your time, and I'm looking forward to hearing your thoughts on this!

Related

• Change 100-continue behavior to send when Body has been polled #838
• hyper does not comply with Expect: 100-continue statement #369
• Expect: 100-continue not implemented for HTTP/2 #2743
• fix(http1): only send 100 Continue if request body is polled #2119
• prototype support for 1xx informational responses #3815

[sfackler]

Doesn't this test imply this handling still exists?

hyper/tests/server.rs

Lines 876 to 907 in e981a91

	#[test]
	fn expect_continue_sends_100() {
	let server = serve();
	let mut req = connect(server.addr());
	server.reply();
	req.write_all(
	b"\
	POST /foo HTTP/1.1\r\n\
	Host: example.domain\r\n\
	Expect: 100-continue\r\n\
	Content-Length: 5\r\n\
	Connection: Close\r\n\
	\r\n\
	",
	)
	.expect("write 1");
	let msg = b"HTTP/1.1 100 Continue\r\n\r\n";
	let mut buf = vec![0; msg.len()];
	req.read_exact(&mut buf).expect("read 1");
	assert_eq!(buf, msg);
	let msg = b"hello";
	req.write_all(msg).expect("write 2");
	let mut body = String::new();
	req.read_to_string(&mut body).expect("read 2");
	let body = server.body();
	assert_eq!(body, msg);
	}

[fungs]

Doesn't this test imply this handling still exists?

hyper/tests/server.rs

Lines 876 to 907 in e981a91

	#[test]
	fn expect_continue_sends_100() {
	let server = serve();
	let mut req = connect(server.addr());
	server.reply();
	req.write_all(
	b"\
	POST /foo HTTP/1.1\r\n\
	Host: example.domain\r\n\
	Expect: 100-continue\r\n\
	Content-Length: 5\r\n\
	Connection: Close\r\n\
	\r\n\
	",
	)
	.expect("write 1");
	let msg = b"HTTP/1.1 100 Continue\r\n\r\n";
	let mut buf = vec![0; msg.len()];
	req.read_exact(&mut buf).expect("read 1");
	assert_eq!(buf, msg);
	let msg = b"hello";
	req.write_all(msg).expect("write 2");
	let mut body = String::new();
	req.read_to_string(&mut body).expect("read 2");
	let body = server.body();
	assert_eq!(body, msg);
	}

I believe this handling is only triggered when it is using the function poll() at the lower level, but the BodyStream interface seems to use the function poll_frame() instead.

[sfackler]

Which poll function? Are you sure that poll_frame doesn't call down into that poll function? Could you put together a self contained example that shows the issue?

[fungs]

Sorry for the fuzzy request/report. I'm not super familiar with hyper's codebase and not a professional Rust programmer.

I'm currently trying to debug, why a stream wrapper (http_body_util::BodyStream::new()) that according to the source code (https://docs.rs/http-body-util/latest/src/http_body_util/stream.rs.html#69-71) calls poll_frame() does not trigger the 100-continue response in the connection's poll_read_body()

I will come back with my findings, which could well be in the application code itself.

[mladedav]

I think hyper sends 100 only when the Expect: 100-continue header is present. Otherwise, it just doesn't.

[fungs]

I played around with the code for a while, but I think I couldn't figure out why hyper was not responding with a 100 continue. I used curl with the corresponding option --expect100-timeout and observed the raw communication, which showed that no such response was sent by hyper. Then, I had to jump over to other tasks. So now, I'm trying to recall the details...

[mladedav]

Note that the option you used only sets a timeout but does not send the actual header. You might want to include -H 'Expect: 100-continue'.

Also it seems that hyper sends 100 only if the request body is actually polled.

[fungs]

I think curl adds the header by default, with bodies larger than 1MB. Anyway, I think I also explicitly tested with adding the header.