The Flowgrammable SDN Stack is a full-featured stack implementation of the OpenFlow specification, versions 1.0, 1.1, 1.2, 1.3, and 1.3.1. The stack provides the following abstractions: message representation, message abstraction, protocol state machine, state machine configuration, system reaction, and application interface. We have over 4,000 directed unit tests running against the message layer, python bindings, and state machine components providing both negative and positive test coverage. The stack is implemented in approximately 80K lines of C++11, a modern systems programming language, for both expressivity and runtime efficiency. Our stack has few external dependencies for greater portability and simpler programmer reasoning. Our library dependencies are: OpenSSL for certificate management and SSL/TLS support and SWIG for cross language binding support.
The OpenFlow architecture starts from the reference of a typical ethernet switch or router. Such a device has two major components. First, the data plane which receives packets on input ports, consults some form of data structures to decide the packets fate, and then enacts the appropriate actions. Second, the control plane performs more high-level functions such as running dynamic protocols (BGP, OSPF, STP, etc), supporting a management interface (SNMP, CLI, SOAP, etc), and modifying the data plane data structures as necessary.
OpenFlow standardizes this architecture in the following way. First the data plane is decomposed into two main components: packet processing pipeline (1), and packet processing data-structures (2). Next, the communication between the control-plane and the data-plane is formalized through the OpenFlow protocol (3). This allows for the data plane and the control plane to live on separate physical entities in a network. The data plane is integrated into OpenFlow by having a piece of switch agent software (4) manipulate the packet processing data structures according to the OpenFlow protocol messages its receives. The control-plane lives on the other end of the OpenFlow Protocol's connection. An application (7) may implement a specific control plane function by either directly interfacing with the OpenFlow protocol's API, or by using an intermediary library (5), that provides higher level abstractions than is present with the protocol API. Finally, applications need not be written in the same language as the OpenFlow protocol, they may use alternative language bindings (6) such as Ruby or Python.
The message layer of the protocol stack supports message construction for all message types of all five OpenFlow versions (1.0, 1.1, 1.2, 1.3, and 1.3.1). A key feature of this component is the prevention of structural and semantic constraint violations in messages. The stack prevents users from creating, both explicitly and from byte arrays, messages that are structurally malformed or are semantically meaningless. This feature may be bypassed in order to build monitoring and testing tools, however, in normal operation, the protection frees the programmer from having to deal with these large classes of typical vulnerabilities.
The message layer provides type definitions for all of the messages and their subcomponents. In addition, there is a standard interface that operates over all of these types:
to_string - convert a message into a readable string
to_buffer - serialize a component into a buffer
from_buffer - construct a message from a buffer
bytes - return the number of bytes a component will consume in a buffer
is_valid - test whether semantic constraints hold
equal, not_equal - test for strict message equality
The message classes can be extended easily, with new proprietary messages or message components, using the necessary interfaces above. Additionally, proprietary extension can be achieved through the extensive set of vendor/experimenter components available for many of the OpenFlow messages.
We also have an extensive positive and negative message test suite that covers all five versions of the protocol, and contains several thousand messages. This suite is included and exercised by the ‘make test’ suite.
The state machine layer provides support for both the explicit and implicit state changes necessary to manage OpenFlow channels across all five versions of OpenFlow (1.0, 1.1, 1.2, 1.3, and 1.3.1). This functionality is provided through two composable state machines, the first manages version negotiation, while the second manages normal established operations of an OpenFlow channel. Version negotiation is supported between the full cross product of possibilities in the version set from above. Not all version sets will have a successful channel establishment; however, we do support the proper error termination in those cases.
The state machine was designed to be system agnostic and to allow for fully isolated unit test. For instance, it should be possible to simply test version negotiation or a state machine that has loaded an application. This ability is demonstrated with our directed unit tests in the ‘make test’ suite.
The primary interface to the state machines are:
constructor - constructs a state machine in its initial state
init - the initializing action for the state machine
fini - the final action of the state machine
recv - a message receive event targeting the state machine
time - a time event targeting the state machine
All interfaces have the possibility of returning a set of messages that must be transmitted over the OpenFlow channel that is associated with this particular state machine. While in established mode the state machines pass the payloads of unconsumed OpenFlow messages up to the application layer. Each state machine is constructed with a reference to the application it should proxy relevant OpenFlow messages towards.
Our library ships with several abstractions necessary to simplify the development of both the switch agent and the controller. This includes things like file descriptor based reactor, uniform system time, system logging, pipe management, reactor managers, etc. The facilities are typical system facilities present in many network systems and are presented as helpful abstractions to simplify the process of constructing actual network systems with the Flowgrammable stack.
A controller application utilizing the Flowgrammable SDN Stack can be written according to the application interface provided. The application interface follows a plugin model. A controller application written by a third party does not run itself. Instead, Flowgrammable SDN Stack provides notification service to the controller application via application interface, and the controller application is written based on these services. The service that is provided in current version is the notification of occurrence of the following events:
Connection with the switch is established.
Connection with the switch is terminated.
An OpenFlow message from the switch is received.
The application interface is provided as a base class for each version of OpenFlow. Controller applications are implemented by inheriting the class of the version they need to support, and reimplementing the virtual functions in the base class. The reimplemented virtual functions are invoked when the driver sends a corresponding notification to the controller application. If any virtual function is not implemented in the derived class (the controller application), the default behavior is to do nothing with the corresponding notification. One exception is the init() interface, which is for notifying that connection with the switch is established, must be reimplemented in the derived class.
Learning Bridge Application
Learning bridge is a sample OpenFlow application included in our submission. It resembles the functionality of an Ethernet bridge. The learning table lives in the controller. The table is updated through Packet_in messages from the switch since the source addresses in those messages can be associated with their ingress ports. Based upon the updates, the application installs flow entries into the switch through Flow_mod messages.
The testing framework is run through CTest, the unit testing and reporting facility that accompanies the CMake toolset. Our goal is to provide comprehensive testing for the Flowgrammable SDN Stack in the form of unit tests, message tests, and interoperability tests.
It is our intent to provide a complete set of unit tests to support regression testing. Each module in the library has an accompanying test directory. These can be configured to be built with the Stack.
The message test suite comprises the bulk of our Stack testing effort. The testing is done by reading, writing, and comparing binary files containing OpenFlow messages. We divide the testing effort into two phases: representation testing and abstraction testing.
The representation tests ensure that the Stack can safely construct message objects from well-formed messages. The abstraction tests ensure that Stack can distinguish between well-formed messages with appropriate values and those without (e.g., a v1.0 flow mod message whose command field is not one of those required by the specification).
The primary emphasis of testing has been the handcrafting of custom messages that exercise representation and abstraction failures. We currently test over 4000 such messages, spanning all versions of the OpenFlow protocol. These tests are executed using the ofp_test tool, provided by the distribution.
The system is also designed to support random testing and trace data.
State Machine Testing
State machine testing covers the handshake sequence between the switch and controller after the version has been negotiated. Positive and negative sequence tests were used to verify that specific OpenFlow messages would trigger the correct state.
Testing Language Bindings
The language bindings are evaluated by providing a language-specific reimplementation of the ofp_test program. For example, the distribution includes a program, ofp_test.py that is also run against the message test set.
The configuration language is designed to have a simple interface for configuring the controller and switches. This language is implemented using a compiler front-end that checks for valid syntax and typing. The main structures in the language are bindings between stages and identifiers, and the staging of those identifiers. The types in the language are predefined, with literals for IPv4, IPv6, integers, time, protocols, strings, and OpenFlow protocol versions. These literals can be combined into predefined types, which form stages. The stages include Realm, Authentication, Authorization, Initialization, but, for a switch agent, Realm is replaced with Connector. These stages are designed to be run in the order specified by the symbol “->” in the language. Each stage describes the behavior a controller should exhibit when making or accepting new connections, or for a switch agent describes its connection behavior. The advantage of using a compiler front-end and a more robust language for configuration is that the configuration can be checked for validity and more advanced configuration features can be implemented.
The configuration language is supported by a utility that will check grammar and typing, and then generate commands that can be run to configure the controller or switch agent. This utility runs in four stages: tokenizing, parsing, elaboration, and evaluation. The parsing stage will check that the input is properly formatted using a grammar designed to be simple to read and write. The elaboration stage checks that all input matches types defined by the language, preventing incorrect input for specific fields. After this stage, guarantees can be made that settings that should hold specific kinds of values are only holding these types of values. Lastly, the input is evaluated and transformed into commands that can be run by the controller or switch agent. Tokenizing is not mentioned because it occurs concurrently with parsing.
The Flowgrammable SDN Stack is implemented using the C++11 programming language. The language provides a number of benefits for performance critical infrastructure like an OpenFlow stack. In particular, the Flowgrammable SDN Stack relies on the following C++11 features to provide an abstract, efficient, and secure implementation of the specifications.
Generalized constant expressions (constexpr) are used to efficiently compute the required number of bytes of statically sized messages.
Move semantics are used to minimize the cost of unavoidable copies when moving messages into containers.
Unrestricted unions provide the ability to define the OpenFlow messages in a compact and reliable way.
Default operations allow compiler generated implementations of trivial default constructors when other constructors are present.
The implementation relies on a number of other features (inheriting and delegating constructors, auto, range-based for loop, etc) are also used throughout. These features reduce program complexity and improve readability.
Language bindings expose the core functionality of the OpenFlow protocol stack in different programming languages (e.g., Python or Ruby). Language bindings provide the opportunity for programmers to quickly build applications to query or interact with the network via the OpenFlow stack.
The implementation uses SWIG to generate language bindings from the interface specifications of the OpenFlow library. SWIG allows us to generate multiple language bindings from a single specification. We plan to support Python, Ruby and Java. However, at the time of release only the Python bindings have been fully implemented.
The Python bindings define a collection of modules, where each module wraps a subset of the OpenFlow library. Currently, there are three modules:
flog - Contains the basic facilities in the flog namespace.
flog_ofp_v1_0 - Wraps the message structures in v1.0.
flog_ofp_v1_1 - Wraps the message structures in v1.1
The classes and functions in these modules are identical to those in the OpenFlow library. The reference documentation can be used as a reference for the Python bindings as well. Below is a simple program showing how to read a message from a file using the Python bindings.
import sys import flog def get_ofp_version(ver): """Returns the OFP module that implements the given version.""" if ver == 1: import flog_ofp_v1_0
elif ver == 2:
import flog_ofp_v1_1 return flog_ofp_v1_1 else: raise "Unsupported protocol version" # Read a message from the buffer b = flog.buffer_from_file(sys.argv) ofp = get_ofp_version(b) v = ofp.Buffer_view(b) m = ofp.Message() # Read the message err = ofp.from_buffer(v, m) if not err: sys.exit(err.code) # Validate the contents err = ofp.is_valid(m) if not err: sys.exit(err.code) print m
In order to use the Python bindings, the compiled modules must be in the interpreter’s search path. These are currently not installed by the build system and must be manually configured.