Data Center TCP (DCTCP)¶

ABSTRACT¶

Cloud data centers host diverse applications, mixing workloads that require small predictable latency with others requiring large sustained throughput. In this environment, today’s state-of-the-art TCP protocol falls short. We present measurements of a 6000 server production cluster and reveal impairments that lead to high application latencies, rooted in TCP’s demands on the limited buffer space available in data center switches. For example, bandwidth hungry “background” ﬂows build up queues at the switches, and thus impact the performance of latency sensitive “foreground” trafﬁc.

To address these problems, we propose DCTCP, a TCP-like protocol for data center networks. DCTCP leverages Explicit Congestion Notiﬁcation (ECN) in the network to provide multi-bit feedback to the end hosts. We evaluate DCTCP at 1 and 10Gbps speeds using commodity, shallow buffered switches. We ﬁnd DCTCP delivers the same or better throughput than TCP, while using 90% less buffer space. Unlike TCP, DCTCP also provides high burst tolerance and low latency for short ﬂows. In handling workloads derived from operational measurements, we found DCTCP enables the applications to handle 10X the current background trafﬁc, without impacting foreground trafﬁc. Further, a 10X increase in foreground trafﬁc does not cause any timeouts, thus largely eliminating incast problems.

INTRODUCTION¶

In recent years, data centers have transformed computing, with large scale consolidation of enterprise IT into data center hubs, and with the emergence of cloud computing service providers like Amazon, Microsoft and Google. A consistent theme in data center design has been to build highly available, highly performant computing and storage infrastructure using low cost, commodity components [16]. A corresponding trend has also emerged in data center networks. In particular, low-cost switches are common at the top of the rack, providing up to 48 ports at 1Gbps, at a price point under $2000 — roughly the price of one data center server. Several recent research proposals envision creating economical, easyto-manage data centers using novel architectures built atop these commodity switches [2, 12, 15].

Is this vision realistic? The answer depends in large part on how well the commodity switches handle the trafﬁc of real data center applications. In this paper, we focus on soft real-time applications, supporting web search, retail, advertising, and recommendation systems that have driven much data center construction. These applications generate a diverse mix of short and long ﬂows, and require three things from the data center network: low latency for short ﬂows, high burst tolerance, and high utilization for long ﬂows.

The ﬁrst two requirements stem from the Partition/Aggregate (described in § 2.1) workﬂow pattern that many of these applications use. The near real-time deadlines for end results translate into latency targets for the individual tasks in the workﬂow. These targets vary from ∼10ms to ∼100ms, and tasks not completed before their deadline are cancelled, affecting the ﬁnal result. Thus, application requirements for low latency directly impact the quality of the result returned and thus revenue. Reducing network latency allows application developers to invest more cycles in the algorithms that improve relevance and end user experience.

The third requirement, high utilization for large ﬂows, stems from the need to continuously update internal data structures of these applications, as the freshness of the data also affects the quality of the results. Thus, high throughput for these long ﬂows is as essential as low latency and burst tolerance.

In this paper, we make two major contributions. First, we measure and analyze production trafﬁc (>150TB of compressed data), collected over the course of a month from ∼6000 servers ( § 2), extracting application patterns and needs (in particular, low latency needs), from data centers whose network is comprised of commodity switches. Impairments that hurt performance are identiﬁed, and linked to properties of the trafﬁc and the switches.

Second, we propose Data Center TCP (DCTCP), which addresses these impairments to meet the needs of applications ( § 3). DCTCP uses Explicit Congestion Notiﬁcation (ECN), a feature already available in modern commodity switches. We evaluate DCTCP at 1 and 10Gbps speeds on ECN-capable commodity switches ( § 4). We ﬁnd DCTCP successfully supports 10X increases in application foreground and background trafﬁc in our benchmark studies.

The measurements reveal that 99.91% of trafﬁc in our data center is TCP trafﬁc. The trafﬁc consists of query trafﬁc (2KB to 20KB in size), delay sensitive short messages (100KB to 1MB), and throughput sensitive long ﬂows (1MB to 100MB). The query trafﬁc experiences the incast impairment, discussed in [32, 13] in the context of storage networks. However, the data also reveal new impairments unrelated to incast. Query and delay-sensitive short messages experience long latencies due to long ﬂows consuming some or all of the available buffer in the switches. Our key learning from these measurements is that to meet the requirements of such a diverse mix of short and long ﬂows, switch buffer occupancies need to be persistently low, while maintaining high throughput for the long ﬂows. DCTCP is designed to do exactly this.

DCTCP combines Explicit Congestion Notiﬁcation (ECN) with a novel control scheme at the sources. It extracts multibit feedback on congestion in the network from the single bit stream of ECN marks. Sources estimate the fraction of marked packets, and use that estimate as a signal for the extent of congestion. This allows DCTCP to operate with very low buffer occupancies while still achieving high throughput. Figure 1 illustrates the effectiveness of DCTCP in achieving full throughput while taking up a very small footprint in the switch packet buffer, as compared to TCP.

While designing DCTCP, a key requirement was that it be implementable with mechanisms in existing hardware — meaning our evaluation can be conducted on physical hardware, and the solution can be deployed to our data centers. Thus, we did not consider solutions such as RCP [6], which are not implemented in any commercially-available switches.

We stress that DCTCP is designed for the data center environment. In this paper, we make no claims about suitability of DCTCP for wide area networks. The data center environment [19] is significantly different from wide area networks. For example, round trip times (RTTs) can be less than 250 µ s, in absence of queuing. Applications simultaneously need extremely high bandwidths and very low latencies. Often, there is little statistical multiplexing: a single ﬂow can dominate a particular path. At the same time, the data center environment offers certain luxuries. The network is largely homogeneous and under a single administrative control. Thus, backward compatibility, incremental deployment and fairness to legacy protocols are not major concerns. Connectivity to the external Internet is typically managed through load balancers and application proxies that effectively separate internal trafﬁc from external, so issues of fairness with conventional TCP are irrelevant.

We do not address the question of how to apportion data center bandwidth between internal and external (at least one end point outside the data center) ﬂows. The simplest class of solutions involve using Ethernet priorities (Class of Service) to keep internal and external ﬂows separate at the switches, with ECN marking in the data center carried out strictly for internal ﬂows.

The TCP literature is vast, and there are two large families of congestion control protocols that attempt to control queue lengths: (i) Delay-based protocols use increases in RTT measurements as a sign of growing queueing delay, and hence of congestion. These protocols rely heavily on accurate RTT measurement, which is susceptible to noise in the very low latency environment of data centers. Small noisy ﬂuctuations of latency become indistinguishable from congestion and the algorithm can over-react. (ii) Active Queue Management (AQM) approaches use explicit feedback from congested switches. The algorithm we propose is in this family.

Having measured and analyzed the trafﬁc in the cluster and associated impairments in depth, we ﬁnd that DCTCP provides all the beneﬁts we seek. DCTCP requires only 30 lines of code change to TCP, and the setting of a single parameter on the switches.