*“Two roads diverged in a wood, and I—
I took the one less traveled by,
And that has made all the difference.”
- Robert Frost, The Road Not Taken
Traffic engineering (TE) is the art of precisely controlling traffic flow in a network for better user experience. Its goals include:
#1 Efficient utilization of network resources
As networks get denser, traditional destination-based forwarding fails to explore all topological paths, leading to under-utilization of some parts of the network.
#2 Application awareness
Each application puts unique requirements on the infrastructure and the network needs to be flexible to steer traffic through the desired path for that application.
#3 Better packet economics and enforcement of business logic and policies
Packet economics is all about forwarding each packet with the lowest cost per bit, while offsetting for application needs and the business policy.
In practice, TE comprises of a set of components to override IP destination-based routing. See the figure below for an example of how a traffic-engineered path is different from the best path based on destination lookup.
The TE components fall broadly into three layers:
Forwarding: A mechanism to forward packets along a computed path that’s different from destination-based forwarding. For example, a tunnel.
Control: Protocols and algorithms with the end goal of deriving a path from source to destination, given a set of requirements and constraints. It involves mechanisms to gather network state (bandwidth, metric, and other attributes) and algorithms to compute paths using that state.
Adaptation: Subsystems that monitor the network and re-trigger computation if the current computation doesn’t satisfy requirements anymore.
The word art is quite apt here: just like how The road not taken poem so beautifully blends individualism with uncertainty, traffic engineering needs to be practiced carefully to avoid forwarding loops, to avoid network overload, and to maintain good user experience at all times.
In these series of posts, we’ll take a look at various traffic engineering concepts and scenarios, and how these concepts apply to SD-WAN.
Source routing is one of the oldest mechanisms of traffic engineering in the IP world. Published in 1981, the IP specification defined two options, LSRR and SSRR, to prescribe paths from source to destination. The original motivation was to avoid then-expensive forwarding lookups on the routers. Nonetheless, it provided a way to pin a source-to-destination path, either as a contiguous or non-contiguous set of hops. It didn’t gain much traction though, and was finally deprecated, primarily for security reasons [IPOPT]. Indeed, as the Internet continued to expand with many players, ISPs, business interests, and peering relationships, it became apparent that pinning down a path encompassing all such entities would be harder. Moreover, the IP specification didn’t evolve to define Control and Adaptation layers for it to be a complete TE system.
Internet is a large collection of networks, each under different administrative control. BGP denotes the networks as autonomous systems (AS) and defines the protocol machinery for the networks to connect to each other and exchange routing information. As the overall network graph gets denser with multiple connections between any pair of ASes, the obvious question that arises is: how does one network influence which of the connections should receive traffic from the other network.
BGP offers a flexible decision process comprising of several tie-breaking rules to arrive at the result of which path should be chosen for a given prefix. It thus provides a rudimentary tool of traffic engineering within the realm of destination-based forwarding. Some examples include:
Longest match. This is typically used when an end customer site is multi-homed to same or different ISPs and has a provider-independent IP space to advertise. Let’s assume that IP space is 126.96.36.199/24. To influence a particular connection to be used for return traffic, it announces 188.8.131.52/25 on that connection (and 184.108.40.206/24 on the other connection).
Though this works well and is widely used in practice, it has the side effect of increasing the routing table size of routers [IAB report].
AS path length. BGP is a path vector protocol. When a prefix is advertised from one AS to another through the network graph, the sender AS# gets added to an attribute called AS_PATH that is then included along with the prefix announcement. This is primarily used to detect routing loops. One of the tie-breaking rules in BGP is to choose a path with the shortest AS_PATH length. Thus, to influence a connection to be used for return traffic, the sender AS prepends its own AS multiple times when it announces the routes on other connections.
Stay tuned for part 2 of this post, where we continue the discussion with a focus on MPLS networks.