A networked-FPGA platform o ff ering fm exible Ethernet switching - - PowerPoint PPT Presentation

Networked FPGA platforms Opening up network packet visibility Increasing network flexibility

A networked-FPGA platform offering fmexible Ethernet switching from Layer 1 all the way to full SDN via P4

Matthew Knight and Marc Durrenberger

Who we are …

HEADQUARTERED IN SYDNEY GLOBAL OPERATIONS FOUNDED

2013

Clients Globally

100+

Major Verticals

Financial Services Data Centers Defense & Government Telecoms Gaming & Media

Patents & Trademarks

Protected by patented technology

• Layer 1 switches – patching, broadcast and tapping
• Low latency order-entry -- Muxing
• Nanosecond-precision aggregation tap
• FPGA platform for application development

AND

Develop Network Applications

AND

Leverage FPGA Technology

AND

Build Switching Hardware

What we do…

• An overview of Layer 1 switching
• Metamako device architecture and telemetry
• Software-defined networking
• The network data plane - now programmable
• A high-level overview of P4 packet processing directives
• The Metamako P4/SDNet sample application
• The SDNet data plane implementation
• SDNet user-engines and scalability
• Summary

Agenda

• Physical layer switching acting at the Physical Medium Dependent Sublayer (PMD)
• Switching function has no knowledge of byte encoding, framing etc.
• Why is it useful?
• Allows ports to be replicated - tapping, port redistribution
• Extremely fast (5 ns port-to-port) and low-jitter
• Can perform media conversion e.g.:
• Fibre to copper
• multi-mode to single-mode fibre
• multi/single-mode/copper to DWDM - no transceiver vendor-locking
• In Metamako devices, full Ethernet signal and clock regeneration on every port

An overview of Layer 1 switching

Metamako device architecture

• Integrated Layer 1+ switch
• 1 or 3 Xilinx UltraScale™ or Ultrascale+™ FPGAs in 1 or 2 RU
• FPGA Development Kit available
• 48 ports per RU
• x86_64 management processor
• Runs Linux
• Leverages the InfluxData open-source TICK Stack for

telemetry

• L1+ switch ports status/statistics
• Ethernet-connected to L1 switch fabric
• Direct PCIe connections to FPGA(s)
• Toolchain can generate rpm which loads FPGA, drivers,

apps

CPU Layer 1 Switch

x2 MMP x4 MMP x4 MMP

FPGA FPGA FPGA

x14 10GbE x14 10GbE x56 10GbE 1 or 2 RU Chassis

Preconfjgured on every device

• Real-time streaming counters from:
• Linux
• L1+ Port Counters
• Easily extensible to custom SDNet

implementation Feeding local InfmuxData Stack

• Telegraf - collection agent
• InfluxDB - time-series database
• Kapacitor - alerting engine

Accessible in real-time via Web Apps e.g.

• Grafana
• Chronograf

Layer 1+ Port Statistics e.g. SDNet Linux

cpu mem net disk swap …

Telegraf InfluxDB

custom control plane

Kapacitor

Telemetry: Actionable real-time counters and alerting

custom data plane

Software-defined networking

What is SDN?

• The separation of the network control plane

from the data plane and allowing the control plane to serve more than one device What are the key benefjts?

• Abstraction of the data plane

implementation

• Programmatic dynamic control and

management of the network via a single interface

Data Plane Control Plane Data Plane Control Plane

e.g. OpenFlow, VMware NSX, Cisco ACI

Data Plane Control Plane Data Plane Control Plane

The network data plane

What does it do?

• Moves packets around networks
• Logic usually implemented in an ASIC
• Knows how to parse standard network protocols and act upon

fields within them via tables usually controlled by the control plane How does this fjt in to SDN?

• As long as the ASIC supports all the required functions within the

SDN control plane and can interface with it, all is good How does new networking functionality become available?

• Often driven by one or more of the ASIC/switch vendors
• ASIC design-to-market becoming more flexible but release cycles

are still measured in years

The network data plane - now programmable

P4

• Network-specific programming language allowing network data

plane functionality to be defined

• Open-source standard maintained by the P4 Language Consortium
• Complete flexibility in defining packet protocols e.g.
• Support a completely new type of overlay network
• Packet headers can be rewritten on-the-fly

Xilinx SDNet

• The SDNet language is conceptually similar to P4 and has the same

benefits

• SDNet is far more than a functional specification as it also provides a

compiler to translate specifications into working FPGA programmable logic

• Furthermore, SDNet provides a P4 to SDNet translator

Fixed Data Plane Software-defined Control Plane

e.g. OpenFlow, VMware NSX, Cisco ACI

Software-defined Data Plane Software-defined Control Plane

Existing or Custom API

A high-level overview of P4 packet processing directives

Parser

• Receives incoming packets
• Follows rules telling it how to parse multiple levels of packet header
• Parsed fields written to a struct which is passed to the next block

Match-action pipeline

• Permits fields and derived fields from the parsed headers to be used

to make packet forwarding decisions

• Provides access to lookup tables
• Tables can be:
• exact, ternary or lpm (longest-prefix match)
• Allows information to be sent to the control plane
• Fields in headers may be rewritten

Deparser

• Prepends headers to the packet data
• Headers may be unchanged, changed or entirely new

Parser Match-action pipeline Deparser

Arbiter

Demux

Xilinx FPGA

The Metamako P4/SDNet sample application

Based on the P4/NetFPGA project

• 8 port reference design

The data plane

• 16 10GbE interfaces connected to physical FPGA

transceivers in-turn, connected to the Layer 1 switch The control plane API

• 1 virtual interface connected via PCIe and SDNet

drivers to the management processor

• Data is exchanged via raw frames across this

interface

• Python and C++ APIs available on the management

processor

Management Processor SDNet

Virtual Ethernet interface for control plane

x16

10GbE 10GbE

PCI Express

The SDNet data plane implementation

Packet ingress

• Packets from each 10GbE port are muxed into the

packet bus connected to the SDNet Block The packet bus

• The SDNet block receives and (optionally) forwards

packets over a single packet bus

• The width is configurable up to 1 024 bits
• The Metamako example SDNet implementation

uses a 256 bit packet bus Packet egress

• Packets are consumed from the packet bus leaving

the SDNet block and demuxed to the relevant 10GbE port(s)

Management Processor SDNet Block

Virtual Ethernet interface for control plane

x16

10GbE Out 10GbE Out Output Demux

x16

10GbE In 10GbE In Input Mux

PCI Express Packet bus Packet bus

The control plane

Per-packet metadata

• The P4/SDNet functional specification includes the ability

to define per-packet metadata

• This metadata accompanies the packet through the

SDNet block

• It can be logically split into:
• The metadata required by the mux/SDNet/demux

pipeline to process the packet e.g. the input port and following a lookup, the output port(s)

• Any set of fields that can be populated with data or state

e.g. key not found in table destined for the control plane

• Implemented in SDNet as a separate bus

SDNet Block

x16

10GbE Out 10GbE Out Output Demux

x16

10GbE In 10GbE In Input Mux

Packet bus Packet bus Control Bus

Management Processor

Virtual Ethernet interface for control plane

PCI Express

SDNet user engines and scalability

The Concept

• User engines are essentially custom RTL modules

that interface with the SDNet block and can be inserted in the packet processing pipeline

• They allow the implementor to perform custom

processing of every frame e.g.

• Encryption/decryption
• Deep packet inspection
• Traffic Shaping/Quality of Service

P4 Support

• The P4 language supports extern objects which

Xilinx’s translator maps to SDNet user engines

SDNet Block

x16

10GbE Out 10GbE Out Output Demux

x16

10GbE In 10GbE In Input Mux

Packet bus Packet bus

User Engine Core User Engine Core Management Processor

Virtual Ethernet interface for control plane

PCI Express

Raw throughput

• Heavily dependent upon the design meeting timing
• With a relatively straightforward processing pipeline

running on a large UltraScale+™ FPGA, somewhere between 0.5 and 1 Tbps could well be possible Fundamental Trade-offs

• An FPGA is typically clocked an order of magnitude

lower than an ASIC

• The latest and greatest switching ASICs are apparently

capable of 12.8 Tbps

• No ASIC today provides the programability of a P4/

SDNet impementation

SDNet Block

x64

10GbE Out 10GbE Out AXI Crossbar

x64

10GbE In 10GbE In

Packet busses Packet busses

SDNet Block SDNet Block SDNet Block AXI Crossbar

SDNet user engines and scalability

• Software-defined networking is here to stay
• Though SDN introduces a programmable network control plane, the

data plane is still largely defined by ASIC functionality

• With P4 and SDNet, a programmable switching data plane is now

possible leveraging FPGA programmable logic

• Metamako devices provide a powerful, production-ready platform for

switching

• Metamako has built and is demonstrating a sample P4/SDNet

switching application

Summary

Networked FPGA platforms Opening up network packet visibility Increasing network flexibility

Questions?

Networked FPGA platforms Opening up network packet visibility Increasing network flexibility

For more detail and a live P4/SDNet demo, stop by booth #670