Distributed Flow Measurements

As per my understanding the NetConf with the help of Yang data model makes it easy to configure distributed network monitoring probes. These probes can export data in IPFIX / PSAMP export formats. All of these standards together help perform distributed flow measurements.

On the other hand, languages / data type definitions such as P4, PacketC, and PacketTypes specify the structure and semantics of the protocol packet formats. The format/semantic specifications are very useful for packet parsing. I understand that IPFIX / PSAMP may have some role to play even in a stand-alone packet analyzer software, but for packet analyzers, the emphasis is likely to be on the packet parsing side. Hence, the specifications like P4 / PacketC have substantial role in influencing the packet analyzer architecture. One idea that we can borrow from flow measurement standards is the specification of flow selection criterion. We can map the Wireshark capture and display filters to selection on flows/sub-flows.

So the real question is this. If we are to build a stand-alone packet analyzer like Wireshark, would the Yang data model be of any help at all? How would the answer change if we were to build a distributed packet analyzer?

Our present understanding is that we can safely skip the NetConf and Yang RFC standards for stand alone packet analyzer. The IPFIX / PSAMP algorithms can be implemented, but there is no need for implementing the export-side of these standards. All the collected information can directly go to a database. In case of distributed packet analysis, NetConf and Yang can help in the distribution of configuration for remote probes. For distributed packet analysis, we may need to add the packet analyzer configuration information to Yang model. On this hybrid flow and packet parsing foundation, a distributed network monitoring framework can be built. Such a framework can perform aggregate flow kind of measurements or a refined packet analysis for selected flows.

References

1. Versatile Monitoring Toolkit (VERMONT) - A C/C++ network monitoring toolkit compliant with IPFIX/PSAMP standards that can be configured using NetConf, see https://github.com/tumi8/vermont
2. A Java project for configuring VERMONT using NetConf exists, see https://github.com/tumi8/netconf/tree/master/vermontconfig
3. A Java collector node code for VERMONT probes exists at https://github.com/tumi8/nasty
4. See all other repositories of https://github.com/tumi8
5. RFC 6728: Configuration Data Model for the IP Flow Information Export (IPFIX) and Packet Sampling (PSAMP) Protocols, see https://tools.ietf.org/html/rfc6728
6. Dr. Gerhard Munz worked a lot on IPFIX, PSAMP, NetConf areas, see the publications available at http://www.net.in.tum.de/en/members/muenz
7. HIgh Speed neTwork mOnitoRing and analYsis (HISTORY) project - distributed flow-based network monitoring and analyis framework, http://www.history-project.net/index.php
8. Münz, G., Antony, A., Dressler, F. and Carle, G., 2006, April. Using netconf for configuring monitoring probes. In Network Operations and Management Symposium, 2006. NOMS 2006. 10th IEEE/IFIP (pp. 1-4). IEEE.