Concurrent requests do not work with HTTP/2

[gaborcsardi]

web <- webfakes::local_app_process(
  webfakes::httpbin_app(),
  opts = webfakes::server_opts(num_threads = 3)
)

library(testthat)
test_that("", {
  url <- web$url("/delay/1")
  p <- curl::new_pool()
  handles <- replicate(3, curl::new_handle(url = url, http_version = 3))
  resps <- list()
  for (handle in handles) {
    curl::multi_add(
      handle,
      done = function(x) message("one is done"),
      fail = stop,
      pool = p
    )
  }
  st <- system.time(curl::multi_run(timeout = 5, pool = p))
  print(st)
  expect_true(st[["elapsed"]] < 3.0)
})

[gaborcsardi]

It seems hard to fix this. civetweb does not support HTTP/2 currently, and it is unlikely that it would support it in the future.

Why the behavior is different when client tries HTTP/2 first, I am not sure. Maybe this is something worth reporting to civetweb.

[jeroen]

Maybe this is happened because the latest release of curl enabled CURLOPT_PIPEWAIT by default, which does not work well if keepalive is disabled: curl/curl#15838

[gaborcsardi]

For the record, according to https://curl.se/libcurl/c/CURLOPT_PIPEWAIT.html this is (was) happening, because libcurl does not start another request to the same server until it receives a reply about whether multiplexing is supported or not:

The waiting time is as long as it takes for the connection to get up and for libcurl to get the necessary response back that informs it about its protocol and support level.

I am not sure what exactly this means for HTTP and HTTPS, but if I send the headers quickly, that seems to be enough for HTTP.

EDIT: or maybe not. Nevertheless it is still working with keepalive on, and now I have a test case for it.