Peering

Peering is the practice of voluntarily interconnecting distinctly separate data networks on the Internet, for the purposes of exchanging traffic between the customers of the peered networks. Peering is also known as settlement-free interconnection, which indicates that neither party pays the other for the traffic being exchanged.

Peering typically involves the following elements:

The physical interconnection of the networks involved.
The exchange of routing information through the BGP routing protocol.
Commercial and contractual peering contracts or agreements.

How peering works

The Internet is a collection of separate and distinct networks, each one operating under a common framework of globally unique IP addressing and global Border Gateway Protocol (BGP) routing.

The relationships between these networks are generally described by one of the following three categories:

Transit (or pay) - You pay money (or settlement) to another network for Internet access (or transit).
Peer (or swap) - Two networks exchange traffic between each other's customers freely, and for mutual benefit.
Customer (or sell) - Another network pays you money to provide them with Internet access.

Furthermore, in order for a network to reach any specific other network on the Internet, it must either:

Sell transit (or Internet access) service to that network (making them a 'customer'),
Peer directly with that network, or with a network who sells transit service to that network, or
Pay another network for transit service, where that other network must in turn also sell, peer, or pay for access.

Often, a network can be very grossly classified based on how it connects to other networks, roughly defined by Internet Tiers:

Tier 1 - A network who only sells to, or peers with, every other network on the Internet.
Tier 2 - A network who peers with other networks, but still pays for transit to reach some portion of the Internet.
Tier 3 - A network who solely purchases transit from other networks to reach the Internet.

The Internet is based on the principle of global reachability (sometimes called end-to-end reachability), which means that any Internet user can reach any other Internet user as though they were on the same network. Therefore, any Internet connected network must by definition either pay another network for transit, or peer with every other network who also does not purchase transit.

Motivations for peering

Peering involves two networks coming together to exchange traffic with each other freely, and for mutual benefit. This 'mutual benefit' is most often the motivation behind peering, which is often described solely by "Reduced costs for Transit services". Other less tangible motivations can include:

Increased capacity for extremely large amounts of traffic (distributing traffic across many networks).
Increased control over your traffic (reducing dependence on one or more transit providers).
Improved performance (attempting to bypass potential bottlenecks with a "direct" path).
Improved perception of your network (being able to claim a "Higher Tier").
Government regulations, or the desire to avoid the appearance of being a monopoly.

Physical interconnections for peering

The physical interconnections used for peering are categorized into two types:

Public peering - Interconnection through a shared fabric such as an Ethernet, ATM, or FDDI switch.
Private peering - Interconnection through a direct or dedicated circuit, either Telco or Dark.

Public peering

Public peering is accomplished across a Layer 2 access technology, generally called a shared fabric. At these locations, multiple carriers interconnect with one or more other carriers across a single physical port. Historically public peering locations were known as network access points (NAPs), today they are most often called Exchange Points or Internet Exchanges (IXs for short). Many of the largest exchange points in the world can have hundreds of participants, and some span multiple buildings and colocation facilities across a city.

Since public peering allows networks interested in peering to interconnect with many other networks through a single port, it is often considered to offer "less capacity" than private peering, but to a larger number of networks. Many smaller networks, or networks who are just beginning to peer, find that public peering exchange points provide an excellent way to meet and interconnect with other networks who may be open to peering with them. Some larger networks utilize public peering as a way to aggregate a large number of "smaller peers", or as a location for conducting low-cost "trial peering" without the expense of provisioning private peering on a temporary basis, while other larger networks are not willing to participate at public exchanges at all.

Today, many exchange points are operated by carrier neutral third parties, who make money by promoting the interconnection of their customers. Many of these operators will often go to great lengths to promote communication and encourage new peering, and will often arrange social events for these purposes.

Private peering

Private peering is the direct interconnection between only two networks, across a Layer 1 or 2 media that offers dedicated capacity that is not shared by any other parties. Early in the history of the Internet, many private peers occurred across 'telco' provisioned SONET circuits between individual carrier-owned facilities. Today, most private interconnections occur at carrier hotels or carrier neutral colocation facilities, where a direct crossconnect can be provisioned between participants within the same building, usually for a much lower cost than telco circuits.

Most of the traffic on the Internet, especially traffic between the largest networks, occurs via private peering. However, because of the resources required to provision each private peer, many networks are unwilling to provide private peering to "small" networks, or to "new" networks who have not yet proven that they will provide a mutual benefit.

Peering agreements/contracts

Most of the original peering on the Internet began with handshake agreements between operators of two networks. Today, the majority of peering between networks is governed by a peering agreement which may be required by one or both sides. This is essentially a legal contract which sets forth the exact details of how traffic is to be exchanged, along with a list of expected activities which may be necessary to maintain the peering relationship, a list of activities which may be considered abusive and result in termination of the relationship, and details concerning how the relationship can be terminated.

History of peering

Early in the history of the Internet, most peering was implemented as public interconnections, and occurred at official sanctioned locations chosen by NSFNET. In the United States, the primary locations for such peering were:

MAE-East - Located in Tysons Corner, VA, and later relocated to Ashburn, VA.
MAE-West - Located in San Jose, CA
AADS - Located in Chicago, IL
SprintNAP - Located in Pennsauken, NJ
PacBell NAP - Located in San Francisco, CA

As the Internet grew, and traffic levels increased, these NAPs became increasingly less able to handle the load required. Most of the early NAPs utilized FDDI technology, which provided only 100 Million bits per second of capacity to each participant. Some of these exchanges upgraded to ATM technology, which provided OC-3 (155 Million bits per second) and OC-12 (622 Million bits per second) of capacity.

Other prospective Exchange Point operators moved directly into offering Ethernet technology, such as Gigabit Ethernet (1000 Million bits per second), which quickly became the predominant choice for Internet Exchange Points due to the reduced cost and increased capacity offered. Today, almost all significant Exchange Points operate solely over Ethernet, and most of the largest Exchange Points offer Ten Gigabit Ethernet (10000 Million bits per second) service.

During the dot-com boom, many Exchange Point and carrier neutral colocation providers had plans to build as many as 50 locations to promote carrier interconnection in the United States alone. Following the dot-com bust essentially all of these plans were abandoned, and today it is considered both economically and technically infeasible to support this level of interconnection among even the largest of networks.

Depeering

By definition, peering is the voluntary and free exchange of traffic between two networks, for mutual benefit. If one or both networks believes that there is no longer a mutual benefit, they may decide to cease peering: this is known as depeering. Some of the reasons why one network may wish to depeer another include:

A desire that the other network pay settlement, either in exchange for continued peering or for Transit services.
A belief that the other network is "profiting unduly" from the settlement free interconnection.
Concern over traffic ratios, which related to the fair sharing of cost for the interconnection.
A desire to peer with the upstream Transit provider of the peered network.
Abuse of the interconnection by the other party, such as pointing default or utilizing the peer for Transit.
Instability of the peered network, repeated routing leaks, lack of response to network abuse issues, etc.
The inability or unwillingness of the peered network to provision additional capacity for peering.
The belief that the peered network is unduly peering with your customers.
Various external political factors (including personal conflicts between individuals at each network).
Mentioning "PIN-SM" as a peering requirement.
MZIMA

In some situations, networks who are being depeered have been known to attempt to fight to keep the peering by intentionally breaking the connectivity between the two networks when the peer is removed, either through a deliberate act or an act of omission. The goal is to force the depeering network to have so many customer complaints that they are willing to restore peering. Examples of this include forcing traffic via a path that does not have enough capacity to handle the load, or intentionally blocking alternate routes to or from the other network. Some very notable examples of these situations have included:

BBNPlanet vs Exodus
PSI vs Cable and Wireless
AOL/ATDN vs Cogent
Teleglobe vs Cogent
France Telecom vs Cogent
France Telecom vs Proxad
Level(3) vs XO
Level(3) vs Cogent

Modern peering

Peering locations

The modern Internet operates with significantly more peering locations than at any time in the past, resulting in improved performance and better routing for the majority of the traffic on the Internet. However, in the interests of reducing costs and improving efficiency, most networks have attempted to standardize on relatively few locations within these individual regions where they will be able to quickly and efficiently interconnect with their peering partners.

The primary locations for peering within the United States are generally considered to be:

San Francisco Bay Region (San Jose CA, Palo Alto CA, Santa Clara CA, San Francisco CA)
Washington DC / Northern Virginia Region (Ashburn VA, Reston VA, Vienna VA)
New York City Region (New York NY, Newark NJ)
Chicago Region (Chicago IL)
Los Angeles Region (Los Angeles, CA)
Dallas/Fort Worth Region (Dallas, TX)

Internationally, the most important locations for peering are generally considered to be:

Amsterdam, Netherlands
London, United Kingdom
Frankfurt, Germany
Tokyo, Japan
Hong Kong, China
Seoul, South Korea
Miami, FL, USA

Exchange Points and Colocation Facilities

The largest individual Exchange Points in the world are AMS-IX in Amsterdam, followed closely by LINX in London. The next largest Exchange Point is generally considered to be JPIX in Tokyo, Japan. The United States, with a historically larger focus on private peering and commercial public peering, has a much smaller amount of traffic on public peers compared to other regions which operate non-profit Exchange Points. The combined Exchange Points in multiple cities operated by Equinix are generally considered to be the largest and most important, followed by the PAIX facilities which are currently owned and operated by Switch and Data. Other important but smaller exchange points include LIPEX and LoNAP in London UK, DE-CIX in Frankfurt Germany, NYIIX in New York, and Nap of the Americas (or NOTA) in Miami, Florida.

URLs to some public traffic statistics of exchange points include:

Peering and BGP

A great deal of the complexity in the BGP routing protocol exists to aid the enforcement and fine-tuning of peering and transit agreements. BGP allows operators to define a policy that determines where traffic is routed. Three things commonly used to determine routing are local-preference, multi exit discriminators (MEDs) and AS-Path. Local-preference is used internally within a network to differentiate classes of networks. For example, a particular network will have a higher preference set on internal and customer advertisements. Settlement free peering is then configured to be preferred over paid IP transit.

Networks that speak BGP to each other can (although most do not) engage in multi exit discriminator exchange with each other. When networks interconnect in several locations, MEDs can be used to reference that network's interior gateway protocol cost. (??) This results in both networks sharing the burden of transporting each others traffic on their own network (or cold potato). Hot-potato or nearest-exit routing, which is typically the normal behavior on the Internet, is where traffic destined to another network is delivered to the closest interconnection point.

External links