Airbnb Improves Web Performance with HTTP Streaming

InfoQ Homepage News Airbnb Improves Web Performance with HTTP Streaming

Jun 20, 2023 2 min read

Rafal Gancarz

Airbnb introduced HTTP streaming to improve the page-loading performance of their website. They reduced the First Contentful Paint (FCP) metric by around 100 milliseconds on every page tested, including the homepage. They also minimized the impact of slow backend queries on the loading times.

Airbnb has been looking for possible improvements in presenting content to their website users as quickly as possible and identified that sending the page body only after it has been fully rendered is not offering the best user experience, particularly if the body content depends on backend queries. Additionally, web pages usually require many additional resources, such as CSS files and external Javascript files, to be downloaded by the browser to display the content to the user correctly. These dependencies often result in cascades of resource requests, best illustrated in the network sequence view, such as Chrome’s Waterfall.

Source: https://medium.com/airbnb-engineering/improving-performance-with-http-streaming-ba9e72c66408

A well-known practice to allow the browser to download external resources earlier is placing all tags referencing them inside the <head> tag near the beginning of the HTML document. The browser downloads external resources when it reads the <head> tag. Typically, that only happens once the entire HTML document is delivered, which can take some time if the content is based on slow backend queries.

Early flush is one technique that utilizes HTTP streaming to make the browser aware of external resources even earlier. It requires splitting the HTML document into two portions and sending them separately using chunked transfer encoding. The browser can start downloading external resources as soon as the initial chunk, containing only the beginning of the HTML document, is received and parsed.

Even though the early flush technique isn’t new, it hasn’t been widely used as it requires rendering and sending incomplete portions of HTML without closing tags. Airbnb employs an Express-based NodeJS server to render web pages using React and had to rework the single React component previously used to render the entire HTML document into three separate ones.

Using the early flush helps optimize network waterfall for CSS and Javascript resources but doesn’t deal with delays in rendering the page body. With modern web app frameworks, it’s possible to render the content on the client or the server side (Server-Side Render) and fetch the data separately, but this requires another network request.

Airbnb took their streaming approach further by introducing the third chunk, which they called the deferred data chunk, containing the data required by the page. They used MutationObserver to detect when the deferred data is loaded and injected the data into the application’s network data store, essentially replacing the additional network request.

Server-side Rendering (SSR) and client-side data fetch executed in parallel

Source: https://medium.com/airbnb-engineering/improving-performance-with-http-streaming-ba9e72c66408

The team had to resolve some issues to enable HTTP streaming in their technology stack. They turned off response buffering in NGINX and Nagle’s algorithm in their haproxy load balancer to allow chunked responses to reach the browser unaltered.

Victor Lin, a software engineer at Airbnb, summarizes his team’s experiences and a growing ecosystem supporting HTTP streaming:

While there were challenges along the way, we found that adapting our existing React application to support streaming was very feasible and robust, despite not being designed for it originally. We’re also excited to see the broader frontend ecosystem trend in the direction of prioritizing streaming, from @defer and @stream in GraphQL to streaming SSR in Next.js.

Get actionable advice for your engineering challenges. Adopt the right emerging trends at QCon San Francisco (Oct 2-6, 2023)!

Writing for InfoQ has opened many doors and increased career opportunities for me. I was able to deeply engage with experts and thought leaders to learn more about the topics I covered. And I can also disseminate my learnings to the wider tech community and understand how the technologies are used in the real world.

I discovered InfoQ’s contributor program earlier this year and have enjoyed it since then! In addition to providing me with a platform to share learning with a global community of software developers, InfoQ’s peer-to-peer review system has significantly improved my writing. If you’re searching for a place to share your software expertise, start contributing to InfoQ.

I started writing news for the InfoQ .NET queue as a way of keeping up to date with technology, but I got so much more out of it. I met knowledgeable people, got global visibility, and improved my writing skills.

Becoming an editor for InfoQ was one of the best decisions of my career. It has challenged me and helped me grow in so many ways. We'd love to have more people join our team.

InfoQ seeks a full-time Editor-in-Chief to join C4Media's international, always remote team. Join us to cover the most innovative technologies of our time, collaborate with the world's brightest software practitioners, and help more than 1.6 million dev teams adopt new technologies and practices that push the boundaries of what software and teams can deliver!

A round-up of last week’s content on InfoQ sent out every Tuesday. Join a community of over 250,000 senior developers. View an example

We protect your privacy.

You need to Register an InfoQ account or Login or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Join a community of experts. Rafal Gancarzhas opened many doors and increased career opportunitiesVivian HuInfoQ’s peer-to-peer review system has significantly improved my writingOghenevwede Emeni got global visibility, and improved my writing skillsEdin Kapićbest decisions of my careerhelped me grow in so many waysjoin our teamThomas Bettsfull-time Editor-in-ChiefThe InfoQGet the most out of the InfoQ experience.