The latest and greatest

Data Model-Driven Management:

Latest Industry and Tool Developments

Distinguished Engineer, Cisco

Operations and Management Area Director, IETF

Benoit Claise

• Data Model-driven Management

• Industry Developments

• Yangcatalog.org

• Conclusion

• References

Agenda

• Human readable and easy to learn

• Hierarchical configuration data models

• Reusable types and groupings (structured

types)

• Extensibility through augmentation

• Formal constraints for configuration

validation

• Data modularity through modules and sub-

modules

• Well defined versioning rules

Why you should care:

YANG is a full, formal contract language

with rich syntax and semantics to build

applications on

YANG – A Data Modeling Language

• APIs derived from the data models:

• Data models = definitions and constraints

• The protocol: NETCONF, RESTCONF, GRPC

• The encoding: JSON, XML, protobuf

• The programming language: Python, Ruby, Java, C, Erlang, ...

• Industry focusing on YANG as the data modeling language for

services and devices

Why Data Model-Driven Management?

Encoding

(serialization)

Data Modeling Language

(schema language)

Data Modeling

(schema)

Protocol

YANG

YANG Data Model

XML JSON

RESTCONF

NETCONF

ProtoBuf

GRPC (HTTP/2)

Thrift

Non Standard

Possible links

YANG Development Kit

Application

Prog. Language

Python C++ Any language

• RESTCONF: no notion of transaction

• RESTCONF: no notion of lock

• RESTCONF: no notion of candidate config and commit

• RESTCONF: so no notion of two phase commit

• RESTCONF: no <copy-config>

• RESTCONF: some more granularity for query => "config", "nonconfig“, "all".

• RESTCONF: XML or JSON (while NETCONF is XML only)

RESTCONF versus NETCONF: Summary

NETCONF might be better for router and switches

RESTCONF might be better for controller north-bound interface

Data Model-Driven Management: Example

Acting on resources

GET : Gets a resource

POST : Creates a resource or invoke operation

PUT : Replaces a resource

DELETE : Removes a resource

Module my-interfaces {

{

namespace ”com.my-interfaces”;

container interfaces {

list interface {

key name;

leaf name { type string; }

leaf admin-status { type enum;}

rpc flap-interface {

input {

leaf name { type string; }

}

output {

leaf result { type boolean; }

}

POST /restconf/operations/my-interfaces:flap-interface

+ JSON/XML Form Data (including name)

Response will have JSON/XML result

GET /restconf/data/my-interfaces:interfaces

GET /restconf/data/my-interfaces:interfaces/interface/<some

name>

PUT /restconf/data/my-interfaces:interfaces/interface/<some

name> + JSON/XML Form Data (name, admin-status)

DELETE /restconf/data/my-

interfaces:interfaces/interface/<some name>

• Scripting: easy to create, hard to maintain/clean-up

=> Data model-driven set of APIs

• However,

Data Model-Driven Management

Data Models = APIs

Automation is as good as your data

models and your toolchains

Data Model-Driven Set of APIs

• Must be thinking of data models first

• even before CLI, then deduce the CLI, the APIs, the documentation, the

feature support

• Testing through models, not only CLI

Why Should You Care?

« If a feature can’t be automated, it doesn’t exist »

« CLI is so 1990’s »

• Data Model-driven Management

• Industry Developments

• Yangcatalog.org

• Conclusion

• References

Agenda

Internet Engineering Task Force

• Open process for Internet Standards

• Produce the RFCs

• The Standard Development Organization that specifies

• NETCONF, RESTCONF, and YANG,

• but also SNMP and MIBs

• Foresaw a « tsunami of YANG models »

• The IESG redistributed workload in order to allow for resources to be focused on YANG model

coordination (Dec 2014)

• “Primary oversight responsibility and coordination of this work across areas (AD document

ownership) becomes the responsibility of Benoit Claise”

IETF: Timeline of Important Specifications

‘10

‘11 ‘12

‘13

‘14 ‘15 ‘16

YANG 1.0

RFC6020

October 2010

Interface and IP Modules

RFC7223, RFC7277

May, June 2014

Common YANG Data Types

RFC6991

July 2013

JSON Encoding

RFC7951

August 2016

Routing Management

RFC8022

November 2016

NETCONF 1.0, SSH Mapping

RFC4741, RFC4742

December 2006

NETCONF 1.1

RFC6241

June 2011

NETCONF Access Control

RFC6536

March 2012

RESTCONF Protocol

RESTCONF

January 2017

NETCONF over TLS + x.509

RFC7589

October 2016

‘17

MIB Modules versus YANG Data Models

• Writable MIB Module IESG Statement (March 2014):

“IETF working groups are therefore encouraged to use the NETCONF/YANG standards for configuration,

especially in new charters

”

• RFC 6643: Translation of Structure of Management Information Version 2 (SMIv2) MIB Modules

to YANG Modules.

• YANG data models for configuration and monitoring of new features

• Will SNMP disappear?

• No: SNMP and MIB models do a good job for monitoring

• SNMP MIBs are configuration and state information, but represented in a way that is unsuitable for

configuration

https://www.ietf.org/iesg/statement/writable-mib-module.html

IETF: YANG Models Growth

http://claise.be/IETFYANGPageCompilation.png

Coordination

is Really

Required,

now!

YANG dependency graph

Coordination is Really Required, Now!

• Previous picture is about the IETF YANG models

• New dimensions: different SDOs/Opensource projects

• New dimension: versioning

• These YANG models must work together to create services

• Good problem to have: All YANG models arrive at the same time

• As opposed to MIB modules in the past

• Standard Development Organizations (SDOs) can’t work in isolation: industry wide coordination is

required

• Openconfig:

• Pro: a few editors, for all YANG modules

• Con: YANG modules change on regular basis

OPENCONFIG

• Operators-led YANG models

• Google, AT&T, British Telecom, Microsoft, Facebook, Comcast, Verizon, Level3, Cox

Communications, Yahoo!, Apple, Jive Communications, Deutsche Telekom / TeraStream, Bell

Canada

• Focus: 123 network elements YANG models

• Routing (BGP, ISIS, RIB, network-instance), routing policy, interfaces

• Layer2 (vlan, spanning tree), ACL, optical transport, MPLS, etc.

• YANG models not aligned with the IETF

• Location: https://github.com/openconfig/public

www.openconfig.net

OPENCONFIG

• Streaming Telemetry specifications and configuration

• gRPC Network Management Interface (gNMI)

• Protocol: gRPC

• Encoding: protobuf

• Network management paradigm:

• config without transaction,

• then telemetry to check when applied

www.openconfig.net

YANG Tsunami in the Industry

TREND

Controller

Server

Client

Orchestrator

Server

Client

Controller

Server

Client

Server

Client

Server

Service Delivery

YANG Modules

Network

YANG modules

Network Element

YANG modules

Different YANG

Module Types

SDOs Alignment and Trajectory

draft-ietf-netmod-yang-model-classification-00.txt

RFC 8199

Data Model Location and Type (Network

Element)

- Native

models

- Standard

models

- Proprietary

extensions

to standard

models

draft-ietf-netmod-yang-model-classification-00.txt

Network Element

Standard YANG Model

Proprietary Extension

to Standard YANG

Model

Proprietary YANG

Model

(also called « native »

models)

RFC 8199

Open Source and SDOs Landscape

Automation of Network + Infrastructure + Cloud + Apps + IOT

Linux Foundation

Hosted

Outside Linux

Foundation

Svcs

Management & Control

Infrastructure

Product, Services & Workloads

CI/CD

Disaggregated Hardware

Network Control

Operating Systems

Cloud & Virtual Management

Orchestration, Management, Policy

Application Layer / App Server

IO Abstraction & Data Path

System Integration & Test Automation

Network Data Analytics

Standards

Numbers

• IETF YANG modules

• Total from RFC: 50

• Total in drafts: 237

• Openconfig

• Total: 123

• Number of YANG data models in my VM

• Total : 11510

• Duplicates removed: 2591

• Operational removed: 2423

• Vendors removed: 1140

This becomes an

industry problem!

How to Organize the Industry?

• With a YANG catalog, which contains all the modules

• The related metadata regarding maturity level, model type, implementation (which ones are

important?), etc

• Based on the draft-clacla-netmod-model-catalog-02

• The inventory of all YANG modules, cross SDOs, cross vendors

• SDOs on board: IETF, BBF, IEEE, ONF… some under discussion

• Some vendors on board: Cisco, Huawei… some under discussion (Juniper)

• Openconfig

• Started as IETF hackathons

www.yangcatalog.org

• Data Model-driven Management

• Industry Developments

• Yangcatalog.org

• Conclusion

• References

Agenda

https://yangcatalog.org

A repository of YANG tools and

the metadata around YANG

models with the purpose of

driving collaboration between

authors and adoption with

consumers.

Programming the Networks

1. Understand the data model-driven management concept

2. Learn the NETCONF/RESTCONF/YANG basics

3. Where are the YANG data models?

4. Finding the right YANG data model

5. Experiment from a GUI and Code Generation

6. Testing

7. What about upgrading a device?

8. What’s missing? PubSub / Telemetry

3. Where are the Supported YANG Data

Models?

YANG models for all platforms;

• common - across NX-OS, IOS-

XE, and IOS-XR

• nx – NX-OS specific models

• xe - IOS-XE specific models

• xr - IOS-XR specific models

Each subdirectory has OS/platform-

specific info in a README file

https://github.com/YangModels/yang/tree/master/vendor/cisco

3. Where are the Other YANG Data Models?

• IETF RFC:

• https://github.com/YangModels/yang/tree/master/standard/ietf

• http://www.claise.be/2018/01/ietf-yang-modules-statistiques/ (tar file)

• IETF drafts:

• http://www.claise.be/2018/01/ietf-yang-modules-statistiques/ (tar file)

• Much statistic information on www.claise.be (daily cron job)

• And don’t forget the discovery capability

• Now all loaded in www.yangcatalog.org

• See next slide

4. Finding the Right

YANG Data Model

• http://www.yangcatalog.org

• Busy including all the YANG modules

from the industry

• Contains a YANG DB search

• Allow one to find out what YANG

modules and features are supported by

a given platform, OS, license, etc.

• Demo: YANG search + YANG

metadata + YANG tree

www.yangcatalog.org

YANG Catalog: Search

4. Finding the Right YANG Data Model

• Organization: contact, maturity level

• Module: name, prefix, version, type, category, dependencies, document uri, submodules

• Implementation: status, platform, software release, opensource, contact

• And new one all the time. Ex: tree structure, generated from MIB, expired

Metadata are Important => Notion of Health Metric

Automation is as good as your data models, their

metadata, and your toolchain

Yangcatalorg.org is API driven

Impact Analysis

• Demo

• https://www.yangcatalog.org/yang-search/impact_analysis.php?modules[]=ietf-tls-

client&modules[]=ietf-tls-server&modules[]=ietf-ssh-client&modules[]=ietf-ssh-

server&modules[]=ietf-restconf-client&modules[]=ietf-restconf-server&modules[]=ietf-

keystore&modules[]=ietf-netconf-client&modules[]=ietf-netconf-

server&orgs[]=ietf&recurse=&rfcs=1

• https://www.yangcatalog.org/yang-search/impact_analysis.php?modules[]=ietf-

interfaces&recurse=0&rfcs=1&show_subm=1&show_dir=both

Impact Analysis

Tracking Dependencies and Dependents

5. Experiment from a GUI and Code Generation

Demo:

• YANG Suite

• YDK

5. Generation of Model-Driven APIs Using

YANG Development Kit (YDK)

YAN

YANG

Data

Models

(YANG)

API

Generator

Docs

Python

C++

Ruby

Docs

C♯

Docs

YDK-gen

YDK-Py

YDK

7. What About Upgrading?

• YANG modules backward compatibility

• What if I upgrade my router? Will my automation/automated service break?

• Note: the native models might not be backward compatible…

• Demo: check-semantic-version API

Semantic Versioning: Tracking Module Changes

BRK

7. What About Upgrading?

Semantic Versioning: Tracking Module Changes

ietf-interfaces@2013-12-23

ietf-interfaces@2017-08-17

• Derived-semantic-version is determined using:

1. Order all modules of the same name by revision from oldest to newest.

2. If module A, revision N+1 has failed compilation, bump its derived semantic MAJOR version.

3. Else, run "pyang --check-update-from" on module A, revision N and revision N+1 to see if backward-incompatible

changes exist.

4. If backward-incompatible changes exist, bump module A, revision N+1's derived MAJOR semantic version.

5. If no backward-incompatible changes exist, compare the pyang trees of module A, revision N and revision N+1.

6. If there are structural differences (e.g., new nodes), bump module A, revision N+1's derived MINOR semantic

version.

7. If no structural differences exist, bump module A, revision N+1's derived PATCH semantic version.

7. What About Upgrading the IOS?

Semantic Versioning

7. What About Upgrading the IOS?

Semantic Version Diffs

1.0.0 2.0.0

The Catalog

can provide

links to diff

the module’s

tree and full

structure

Eating our own Dog Food: yangcatalorg.org is

API driven

• Yangcatalog is Confd based, with the YANG module in draft-clacla-netmod-model-catalog-02

• All APIs are automatically generated

• Demo: POSTMAN collection

• Ex: https://yangcatalog.org:8443/search/name/Cisco-IOS-XR-ipv4-bgp-cfg

• Ex: all openconfig YANG modules on OS/platform

• We want operators to integrate the APIs in their toolchain

• Download yangcatalog postman collection here

Module Sub-tree

Vendor Sub-tree

Eating our own Dog Food: yangcatalorg.org is API driven

Who Should Know about YANG?

• Even if I push YANG everywhere, not everybody should (have to) know about YANG.

• Operator: Not really, as YANG modules = APIs.

• Operator service designer: should know YANG

• Architect: should design your (new) features from a YANG point of view

• Involved in opensource or SDOs: you should care about YANG toolchain

BRK

8. What’s Missing?

Data Model-driven Telemetry

Solving a SNMP Polling, Data Modeling, and OPEX Issues

Models

APIs

Apps

Protocol TransportEncoding

Model-Driven APIs

YANG Development Kit (YDK)

XR Data Models

(native, open)

App1 App2 App3

Configuration

Streaming

Telemetry

Network

Data

IOS-XR

• Data Model-driven Management

• Industry Developments

• Yangcatalog.org

• Conclusion

• References

Agenda

Summary and Key Messages

• Automation and programmability are required these days

• Data Modeling-driven set of APIs is key for automation

• The key is the data models

• YANG is the data modeling language for configuration and monitoring: a full, formal contract

language with rich syntax and semantics to build applications on.

• Many YANG data model developments

• In different standard development organizations (but primarly at the IETF),

• In open source

• The YANG catalog set of data models, metadata, and tools is here to help

Data Model-Driven Management:

Latest Industry and Tool Development

Benoit Claise

References

• Link to IETF 94 Recording: NETCONF, YANG, pyang

• Link to slides at IETF 94: NETCONF Slides, YANG Slides,

• RFC 6244, An Architecture for Network Management Using

NETCONF and YANG

General Training

Tooling – Exploring and using NETCONF/YANG

• Editor plug-ins

• emacs (yang-mode.el)

• vim (yang.vim)

• sublime text (sublime-yang-syntax)

• pyang

• an extensible YANG validator written in Python. (Video trainining: pyang)

• can be used standalone as a validator of YANG modules, or to generate YIN, YANG, DSDL and XSD

from YANG and YIN.

• https://github.com/mbj4668/pyang

• http://www.yangvalidator.com/

• libsmi

• A library allowing the generation of YANG models from SMI/SMIv2 compliant MIBs

http://www.yang-central.org

Tooling – Exploring and using NETCONF/YANG

• ncclient

• a Python library that facilitates client-side scripting and application development around the

NETCONF protocol (only supports NETCONF 1.0)

• Postman

• a Chrome plugin for RESTCONF, allowing for customized sets of REST snippets to be easily built,

maintain and shared. Useful for NETCONF via RESTCONF, for example Open Daylight

• OpenDaylight

• enables auto-generation of RESTconf APIs from YANG models, with NETCONF client support

• APIdocs feature provides a way to explore both local and mounted YANG models