Tracking HTTP/2 Prioritization Issues

Introduction

HTTP/2 uses multiplexing to support multiple concurrent requests over the same TCP connection.

HTTP/2 also has a prioritization scheme (via weights & dependencies) that enables browsers to communicate the relative priority of each request.

Unfortunately not all servers are equal – some don't appear to implement prioritization and so serve responses on a 'first come, first served' basis, and others that implement prioritization seem to have a few bugs too – and these issues can create a suboptimal experience for the visitor. This can be a result of the server implementation itself or (more likely) from how it is configured in production.

Browsers implement their priority trees in quite different ways too – if you're interested in digging deeper @DaanDeMeyer's h2vis and WebPageTest can visualize these for Chrome and Firefox.

Now WebPageTest visualises HEADERS and DATA frames for responses we can get a much clearer picture of how prioritization is implemented across different servers and CDNs.

This repo aims to track and highlight prioritisation issues – if they get fixed then the web become better for all of us. As long as prioritisation is broken there will be significant performance issues with HTTP/2 which isn't good for anybody.

Getting Started

@PatMeenan's http2priorities test page is the easiest way to get started – find an appropriate image on your server / host / CDN of choice and then test it using Chrome with WebPageTest using the "3G Fast" network profile (do not use 3G, as of Chrome 74 Chrome restricts the number of in-flight requests on low-speed connections). It is recommended that you do a full 9 runs to make sure it consistently passes and is not intermittent.

Pay close attention to requests 33-34, these are high-priority visible images that are loaded after the low-priority images start to load. They have a higher priority so their frames should be re-prioritized ahead of the earlier images.

CloudFlare - reprioritizes the requests

CloudFront - requests are only partially reprioritized

Netlify - requests are not reprioritized

If requests 33-34 don't appear to be prioritised correctly, please raise an issue with the relevant project or vendor.

Visually the difference can be quite dramatic in the filmstrip view:

Current Status

If you create an issue in this repo with a link to your test (whether good or bad) we'll add it to the table below.

It is important to note that what matters most is whatever the edge is that terminates the HTTP/2 connection from the browser. For example, if you have a CDN in front of a hosting provider (or load balancer) then the CDN will be the thing being tested (and that has the most impact). That also means that you can potentially solve broken prioritisation by putting your site behind a CDN that passes.

CDNs / Cloud Hosting Services

* Intermittent Failure

Load Balancers

This is for cloud or on-premises load balancers (Amazon ALB, Citrix Netscaler, Foundry F5, etc).

Servers

Most servers technically support HTTP/2 prioritisation but are effectively broken when deployed because of buffering in the networking path (within the server, in the TCP stack or in the network itself). Read more about it here.

TODO: Add notes about configuration settings to get prioritisation working for various operating systems and servers.

Finding Test Images

The HTTP Archive BigQuery dataset is particularly useful for finding images for a given CDN or host for testing.

Here is a sample query (warning, each query run can consume close to 1TB of BigQuery quota):

select
  RTRIM(LTRIM(JSON_EXTRACT(payload, '$._full_url'),"\""),"\"") as url,
  INTEGER(JSON_EXTRACT(payload, '$._bytesIn')) as size
FROM
  [httparchive:requests.2018_12_01_desktop]
WHERE
  RTRIM(LTRIM(JSON_EXTRACT(payload, '$._protocol'),"\""),"\"") = "HTTP/2" AND
  RTRIM(LTRIM(JSON_EXTRACT(payload, '$._contentType'),"\""),"\"") LIKE "image/%" AND
  INTEGER(JSON_EXTRACT(payload, '$._bytesIn')) > 90000 AND
  INTEGER(JSON_EXTRACT(payload, '$._bytesIn')) < 110000 AND
  RTRIM(LTRIM(JSON_EXTRACT(payload, '$._cdn_provider'),"\""),"\"") = "Cloudflare"
LIMIT 20