Lecture 2 Internet and Datacenter¶
Internet¶
What's "Internet"¶
The Internet (or internet) is the global system of interconnected computer networks that uses the Internet protocol suite (TCP/IP) to communicate between networks and devices.
Internet is a "Network of network"
- Campus Networks
- Enterprise Networks
- Internet Service Providers (ISP)
Each network is owned and managed by some organization, and connected together
Autonomous Systems (AS)¶
Internet is divided into Autonomous Systems
- Node: Autonomous System (AS)
- Edge: Two ASes connect to each other
Autonomous System Numbers¶
AS Numbers are 16bit values.
Internet Service Provider (ISP)¶
- An Internet service provider(ISP) is an organization that provides services for connecting to the Internet
- An ISP can have multiple ASes
- In fact, there are grades for ISPs in Internet
Ties of ISPs¶
The Relation Between Neighboring Nodes¶
- Neighboring ASes have business contracts
- How much traffic to carry
- Which destination to reach
- How much
- Common business relationships
- Customer-Provider (上下关系)
- Peer-Peer (同层级关系)
Customer-Provider Relationship¶
Customer needs to be reachable from everyone
- Provider tells all neighbors how to reach the customer
Customer doesn't want to provide transit service
- Customer doesn't let its providers route through it
Multi-Homing: Two or More Providers¶
- Motivations for multi-homing
- Higher reliability, survive single ISP failure
- Better performance by selecting better path
- Financial leverage through competition
- Gaming the 95th-percentile billing
Peer-Peer Relationship¶
Some networks have an incentive to connect directly, to reduce their bill with their own provider
Peers exchange traffic between customers
- AS exports only customer routes to a peer
- AS exports a peer's routes only to its customers
- Often the relationship is settlement-free (免费结算)
How to Connect ASes?¶
Internet eXchange Points (IXPs)¶
- In fact, the connection between ISPs is not \(C_{n}^{2}\)
- Interconnecting each network to its neighbors one-by-one is not cost effective
- Many networks connect in one location => IXP
Tier-1 Providers¶
Tier-1 Provider
- Has no upstream provider of its own
- Typically has a national or international backbone
- Usually no fee between each Tier-1 Providers
Top of the Internet Hierarchy of 12-20 ASes
- Full peer-peer connections between tier-1 providers
Tier-2 Providers¶
- Provide transit service to downstream customers
- But, need at least one provider of their own
- Typically have national or regional scope
Characteristics of AS Paths¶
- AS Path may be longer than shortest path
- Router Path may be longer than shortest path
Routing¶
- inter-domain routing: Find paths between networks => BGP
- intra-domain routing: Find paths within a network => OSPF / RIP / ISIS
Intra Domain Routing¶
......
Inter Domain Routing - BGP¶
=> BGP => very complex !
BGP is Path Vector Protocol
- BGP announcement carry complete path information instead of distances.
- Every "node" inherits the "Prefix" and put its own Node_Num into it, so that it can realize the "complete path information"
- Complete path enables
- Each AS is free to select and use any path preferably, maybe the cheapest one.
- BGP is Policy Based (very flexible)
The Common Used BGP Policy
- Prefer the path with the highest WEIGHT.
- Prefer the path with the highest LOCAL_PREF.
- Prefer the path that was locally originated via a network or aggregate BGP subcommand or through redistribution from an IGP. Local paths that are sourced by the network or redistribute commands are preferred over local aggregates that are sourced by the aggregate-address command.
- Prefer the path with the shortest AS_PATH. (judged by length of AS_PATH)
- Prefer the path with the lowest origin type. (IGP / EGP)
- Prefer the path with the lowest multi-exit discriminator (MED). ......
More information is listed in CISCO_BGAlggorithm
It's so complex that many issues are caused by BGP
- Globally issues
- The configuration is of great significance and is hard to config for people.
- SDN is excellent but we still use old-system mostly now.
- Therefore, if SDN is commonly used in social life, the BGP can be discarded !
Data Center Networks¶
Traditional Architecture¶
Traits¶
- Three Layers: core / distribution (aggregation, 聚合/汇总) / access
- Layer 2 => MAC => STP solves the Loop problem
- Layer 3 => IP
Limitations¶
- Waste of Bandwidth
- STP block ports except one
- Latency
- Server to server traffic: need many hops
- No for the cloud
- rise of east-west traffic
- VM migration（虚拟机迁移）
Solution: Fat-Tree¶
基础特点：上面的链路带宽大，下层的链路带宽小
Clos Network¶
=> multi-stage switching
Theorem 1
If m ≥ 2n-1, the Clos Network is strict-sense nonblocking
Theorem 2
If m ≥ n, the Clos Network is Rearrangeably nonblocking
Folded 3-Stage Clos Network¶
Traits¶
- Spine - Leaf Structure
- Equal-cost multi-path (ECMP): IP and Routing => All traffic is evenly distributed！
- Server to server traffic: 3 hops
- Large bandwidth for east-west traffic
- VM migration
Limitation¶
Problem: each switch still needs a lot of ports
Benes Network¶
Solution: Adding more stages¶
=> reducing an N x N cross-bar switch to two N/2 x N/2 crossbar switches and two N-input exchange switches
N x N Benes Network¶
2 log2N−1 stages, each containing N/2 2x2 crossbar switches, and use a total of N log2N − N/2 2x2 crossbar switches.
Top-of-Rack (机柜顶部) Architecture¶
- Rack of servers
- top-of-rack switch
- commodity servers
- Modular Design
- Preconfigured racks
- Power, Network, and storage cabling
Fat Tree¶
Structure Traits¶
- Each switch has k ports
- There are k pods
- Each pod has k/2 aggr.Switches and k/2 edge.Switches
- Each edge.Switch is connected to k/2 hosts and k/2 aggr.Switches
- Each core.Switch is connected to an aggr.Switch for each pod
Some Details¶
- How many switches are needed?
- k * (k/2 + k/2) + \((1/4)k^2\)
- How many servers that it can hold?
- (k/2) * (k/2) *k = \((1/4)k^3\)
- How many hops between each pair of hosts at different pods?
- ≥ 6
- How many 6-hop paths between hosts at different pods?
- (k/2)*(k/2) = (1/4)\(k^3\)
Extensions¶
Method: Add more cheap switches!
