Implement ChatGPT-like Streaming Output Using SSE (Server-Sent Events)

Today, I'd like to share an awesome technology with you-Server-Sent Events (SSE) streaming. Just as the title suggests, with it, we no longer need to obtain the results returned by the server through polling, and the performance of the front-end page can be greatly enhanced. For business scenarios that require polling, SSE is undoubtedly a better technical option. Next, I will introduce SSE streaming from multiple aspects, including its concept, comparison with WebSocket, and application scenarios. If you're interested, let's dive in and learn more about it together!

What is SSE Streaming?

SSE, short for Server-Sent Events, is a communication technology based on the HTTP protocol. It is part of the HTML5 standard and enables the server to actively send updated data to the client (usually the Web browser we commonly use). Designed to establish a one-way connection from the server to the client, SSE allows the server to push data to the client in real-time. This way of transmitting data from the server to the client in real-time is what we call streaming.

Have you ever noticed that when chatting with ChatGPT, its responses don't appear all at once but come in segments? By carefully studying ChatGPT's network transmission mode, we can find that it uses streaming technology.

The Advantages of SSE Streaming

Before the emergence of SSE, when we wanted to get data returned by the server, we often used long-polling. Long-polling is also implemented through HTTP requests. The whole process is as follows: The client initiates a request first and then keeps the connection waiting for the server to process. Before the server responds, if the client sends other requests, these requests will be lost. Only after the server finishes processing the request and sends the response, and the client receives it, will the client initiate another request.

From this process, we can see that long-polling has a significant drawback: before the server responds to the request, all subsequent requests sent by the client will not be processed, and the server's response depends on the client's prior request. Usually, when communicating between the front-end and the back-end, we often use ajax or axios to obtain results asynchronously, and this process is actually long-polling.

Compared with long-polling, SSE streaming, which is also based on the HTTP protocol, has obvious advantages. It can send multiple responses to the client without the need for client intervention until the client actively closes the connection. For scenarios where the server needs to push content to the client in real-time, SSE is extremely convenient!

SSE Technical Principles

Parameter Settings

As mentioned before, SSE is essentially a communication technology based on the HTTP protocol. Therefore, if we want to use SSE to build a connection where the server can push information to the client in real-time, we only need to make some settings to the traditional HTTP response headers. Set the Content-Type to text/event-stream. To ensure that the client displays the latest data, set the Cache-Control to no-cache. In addition, since SSE is essentially a TCP connection, to keep SSE continuously open, set the Connection to keep-alive.

After setting these response headers, we can start writing a simple example based on SSE streaming.

SSE Demo

Server-side code:

const express = require("express");
const app = express();
const PORT = 3000;
app.use(express.static("public"));
app.get("/events", function (req, res) {
  res.setHeader("Content-Type", "text/event-stream");
  res.setHeader("Cache-Control", "no-cache");
  res.setHeader("Connection", "keep-alive");
  let startTime = Date.now();
  const sendEvent = () => {
    // Check if 10 seconds have passed since the start
    if (Date.now() - startTime >= 10000) {
      res.write("event: close\ndata: {}\n\n"); // Send a special event to notify the client to close
      res.end(); // Close the connection
      return;
    }
    const data = { message: "Hello World", timestamp: new Date() };
    res.write(`data: ${JSON.stringify(data)}\n\n`);
    // Send a message every 2 seconds
    setTimeout(sendEvent, 2000);
  };
  sendEvent();
});
app.listen(PORT, () => {
  console.log(`Server running on http://localhost:${PORT}`);
});

Client-side code:

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <title>SSE Example</title>
  </head>
  <body>
    <h1>Server-Sent Events Example</h1>
    <div id="messages"></div>
    <script>
      const evtSource = new EventSource("/events");
      const messages = document.getElementById("messages");
      evtSource.onmessage = function (event) {
        const newElement = document.createElement("p");
        const eventObject = JSON.parse(event.data);
        newElement.textContent = "Message: " + eventObject.message + " at " + eventObject.timestamp;
        messages.appendChild(newElement);
      };
    </script>
  </body>
</html>

When we access the client-side page running on port localhost:3000 in the browser, the page will display the results returned by the server step by step in streaming mode. we can see the results in the browser network panel.

Server-side Basic Response Format

The SSE response consists of a series of events separated by two line breaks. Each event can contain the following fields:

• data: This is the data of the event. If the data has multiple lines, each line must start with data:.
• id: It is the unique identifier of the event. The client can use this ID to resume the event stream.
• event: It is a custom event type. The client can perform different operations based on different event types.
• retry: This is the recommended reconnection time (in milliseconds). If the connection is interrupted, the client will wait for this period of time before attempting to reconnect.

These fields are separated by a single line break, and different events are separated by two line breaks.

Client-side Processing Format

The client uses the EventSource interface to listen for SSE messages. The specific code is as follows:

const evtSource = new EventSource("path/to/sse");
evtSource.onmessage = function (event) {
  console.log(event.data); // Process the received data
};

SSE Application Scenarios

SSE is a communication technology based on the HTTP protocol for one-way message pushing from the server to the client. Therefore, it is particularly suitable for scenarios where the server needs to actively push messages, such as countdown synchronization, real-time inventory updates, real-time stock quotes, real-time weather information, and flash-sale status notifications.

SSE Compatibility

Currently, except for Internet Explorer and some low-version mainstream browsers, most browsers on the market support SSE communication. So, in most cases, we can use this technology with confidence.

Comparison between SSE and WebSocket

Perhaps some careful readers have noticed that the communication method of SSE is quite similar to that of WebSocket, and WebSocket also supports two-way communication. Then why not just use WebSocket? The following table details the differences between the two:

As can be seen from the table, SSE and WebSocket each have their own advantages and disadvantages. For scenarios that require high-frequency interaction between the client and the server, WebSocket is indeed more suitable. However, if only one-way data transmission from the server is needed, SSE has lower energy consumption and requires less client intervention.