draft-ietf-anima-stable-connectivity-07.txt   draft-ietf-anima-stable-connectivity-08.txt 
ANIMA T. Eckert, Ed. ANIMA T. Eckert, Ed.
Internet-Draft Huawei Internet-Draft Huawei
Intended status: Informational M. Behringer Intended status: Informational M. Behringer
Expires: April 21, 2018 October 18, 2017 Expires: July 20, 2018 January 16, 2018
Using Autonomic Control Plane for Stable Connectivity of Network OAM Using Autonomic Control Plane for Stable Connectivity of Network OAM
draft-ietf-anima-stable-connectivity-07 draft-ietf-anima-stable-connectivity-08
Abstract Abstract
OAM (Operations, Administration and Maintenance - as per BCP161, OAM (Operations, Administration and Maintenance - as per BCP161,
(RFC6291) processes for data networks are often subject to the (RFC6291) processes for data networks are often subject to the
problem of circular dependencies when relying on connectivity problem of circular dependencies when relying on connectivity
provided by the network to be managed for the OAM purposes. provided by the network to be managed for the OAM purposes.
Provisioning while bringing up devices and networks tends to be more Provisioning while bringing up devices and networks tends to be more
difficult to automate than service provisioning later on, changes in difficult to automate than service provisioning later on, changes in
core network functions impacting reachability cannot be automated core network functions impacting reachability cannot be automated
because of ongoing connectivity requirements for the OAM equipment because of ongoing connectivity requirements for the OAM equipment
itself, and widely used OAM protocols are not secure enough to be itself, and widely used OAM protocols are not secure enough to be
carried across the network without security concerns. carried across the network without security concerns.
This document describes how to integrate OAM processes with the This document describes how to integrate OAM processes with an
autonomic control plane (ACP) in Autonomic Networks (AN) in order to autonomic control plane in order to provide stable and secure
provide stable and secure connectivity for those OAM processes. This connectivity for those OAM processes. This connectivity is not
connectivity is not subject to aforementioned circular dependencies. subject to aforementioned circular dependencies.
Status of This Memo Status of This Memo
This Internet-Draft is submitted in full conformance with the This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79. provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/. Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on April 21, 2018. This Internet-Draft will expire on July 20, 2018.
Copyright Notice Copyright Notice
Copyright (c) 2017 IETF Trust and the persons identified as the Copyright (c) 2018 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(https://trustee.ietf.org/license-info) in effect on the date of (https://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License. described in the Simplified BSD License.
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2
1.1. Self dependent OAM Connectivity . . . . . . . . . . . . . 2 1.1. Self-dependent OAM Connectivity . . . . . . . . . . . . . 3
1.2. Data Communication Networks (DCNs) . . . . . . . . . . . 3 1.2. Data Communication Networks (DCNs) . . . . . . . . . . . 3
1.3. Leveraging the ACP . . . . . . . . . . . . . . . . . . . 4 1.3. Leveraging a generalized autonomic control plane . . . . 4
2. Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2. Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.1. Stable Connectivity for Centralized OAM . . . . . . . . . 4 2.1. Stable Connectivity for Centralized OAM . . . . . . . . . 5
2.1.1. Simple Connectivity for Non-ACP capable NMS Hosts . . 5 2.1.1. Simple Connectivity for Non-GACP capable NMS Hosts . 6
2.1.2. Challenges and Limitation of Simple Connectivity . . 6 2.1.2. Challenges and Limitation of Simple Connectivity . . 7
2.1.3. Simultaneous ACP and Data Plane Connectivity . . . . 7 2.1.3. Simultaneous GACP and data-plane Connectivity . . . . 8
2.1.4. IPv4-only NMS Hosts . . . . . . . . . . . . . . . . . 8 2.1.4. IPv4-only NMS Hosts . . . . . . . . . . . . . . . . . 9
2.1.5. Path Selection Policies . . . . . . . . . . . . . . . 11 2.1.5. Path Selection Policies . . . . . . . . . . . . . . . 12
2.1.6. Autonomic NOC Device/Applications . . . . . . . . . . 12 2.1.6. Autonomic NOC Device/Applications . . . . . . . . . . 14
2.1.7. Encryption of data-plane connections . . . . . . . . 13 2.1.7. Encryption of data-plane connections . . . . . . . . 15
2.1.8. Long Term Direction of the Solution . . . . . . . . . 14 2.1.8. Long Term Direction of the Solution . . . . . . . . . 15
2.2. Stable Connectivity for Distributed Network/OAM . . . . . 15 2.2. Stable Connectivity for Distributed Network/OAM . . . . . 16
3. Architectural Considerations . . . . . . . . . . . . . . . . 15 3. Architectural Considerations . . . . . . . . . . . . . . . . 16
3.1. No IPv4 for ACP . . . . . . . . . . . . . . . . . . . . . 15 3.1. No IPv4 for GACP . . . . . . . . . . . . . . . . . . . . 16
4. Security Considerations . . . . . . . . . . . . . . . . . . . 16 4. Security Considerations . . . . . . . . . . . . . . . . . . . 17
5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 17 5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 19
6. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 17 6. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 19
7. Change log [RFC Editor: Please remove] . . . . . . . . . . . 18 7. Change log [RFC Editor: Please remove] . . . . . . . . . . . 19
8. Informative References . . . . . . . . . . . . . . . . . . . 19 8. References . . . . . . . . . . . . . . . . . . . . . . . . . 22
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 21 8.1. Normative References . . . . . . . . . . . . . . . . . . 22
8.2. Informative References . . . . . . . . . . . . . . . . . 23
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 24
1. Introduction 1. Introduction
1.1. Self-dependent OAM Connectivity
1.1. Self dependent OAM Connectivity
OAM (Operations, Administration and Maintenance - as per BCP161, OAM (Operations, Administration and Maintenance - as per BCP161,
[RFC6291]) for data networks is often subject to the problem of [RFC6291]) for data networks is often subject to the problem of
circular dependencies when relying on the connectivity service circular dependencies when relying on the connectivity service
provided by the network to be managed. OAM can easily but provided by the network to be managed. OAM can easily but
unintentionally break the connectivity required for its own unintentionally break the connectivity required for its own
operations. Avoiding these problems can lead to complexity in OAM. operations. Avoiding these problems can lead to complexity in OAM.
This document describes this problem and how to use the Autonomic This document describes this problem and how to use an autonomic
Control Plane (ACP) to solve it without further OAM complexity: control plane to solve it without further OAM complexity:
The ability to perform OAM on a network device requires first the The ability to perform OAM on a network device requires first the
execution of OAM necessary to create network connectivity to that execution of OAM necessary to create network connectivity to that
device in all intervening devices. This typically leads to device in all intervening devices. This typically leads to
sequential, 'expanding ring configuration' from a NOC (Network sequential, 'expanding ring configuration' from a NOC (Network
Operations Center). It also leads to tight dependencies between Operations Center). It also leads to tight dependencies between
provisioning tools and security enrollment of devices. Any process provisioning tools and security enrollment of devices. Any process
that wants to enroll multiple devices along a newly deployed network that wants to enroll multiple devices along a newly deployed network
topology needs to tightly interlock with the provisioning process topology needs to tightly interlock with the provisioning process
that creates connectivity before the enrollment can move on to the that creates connectivity before the enrollment can move on to the
next device. next device.
When performing change operations on a network, it likewise is When performing change operations on a network, it likewise is
necessary to understand at any step of that process that there is no necessary to understand at any step of that process that there is no
interruption of connectivity that could lead to removal of interruption of connectivity that could lead to removal of
connectivity to remote devices. This includes especially change connectivity to remote devices. This includes especially change
provisioning of routing, forwarding, security and addressing policies provisioning of routing, forwarding, security and addressing policies
in the network that often occur through mergers and acquisitions, the in the network that often occur through mergers and acquisitions, the
introduction of IPv6 or other mayor re-hauls in the infrastructure introduction of IPv6 or other mayor re-hauls in the infrastructure
design. Examples include change of an IGP or areas, PA (Provider design. Examples include change of an IGP or areas, PA (Provider
Aggregatabe) to PI (Provider Independent) addressing, or systematic Aggregatable) to PI (Provider Independent) addressing, or systematic
topology changes (such as L2 to L3 changes). topology changes (such as L2 to L3 changes).
All these circular dependencies make OAM complex and potentially All these circular dependencies make OAM complex and potentially
fragile. When automation is being used, for example through fragile. When automation is being used, for example through
provisioning systems, this complexity extends into that automation provisioning systems, this complexity extends into that automation
software. software.
1.2. Data Communication Networks (DCNs) 1.2. Data Communication Networks (DCNs)
In the late 1990'th and early 2000, IP networks became the method of In the late 1990s and early 2000, IP networks became the method of
choice to build separate OAM networks for the communications choice to build separate OAM networks for the communications
infrastructure within Network Providers. This concept was infrastructure within Network Providers. This concept was
standardized in ITU-T G.7712/Y.1703 [ITUT] and called "Data standardized in ITU-T G.7712/Y.1703 [ITUT] and called "Data
Communications Networks" (DCN). These where (and still are) Communications Networks" (DCN). These were (and still are)
physically separate IP(/MPLS) networks that provide access to OAM physically separate IP(/MPLS) networks that provide access to OAM
interfaces of all equipment that had to be managed, from PSTN (Public interfaces of all equipment that had to be managed, from PSTN (Public
Switched Telephone Network) switches over optical equipment to Switched Telephone Network) switches over optical equipment to
nowadays Ethernet and IP/MPLS production network equipment. nowadays Ethernet and IP/MPLS production network equipment.
Such DCN provide stable connectivity not subject to aforementioned Such DCN provide stable connectivity not subject to aforementioned
problems because they are separate network entirely, so change problems because they are a separate network entirely, so change
configuration of the production IP network is done via the DCN but configuration of the production IP network is done via the DCN but
never affects the DCN configuration. Of course, this approach comes never affects the DCN configuration. Of course, this approach comes
at a cost of buying and operating a separate network and this cost is at a cost of buying and operating a separate network and this cost is
not feasible for many providers, most notably smaller providers, most not feasible for many providers, most notably smaller providers, most
enterprises and typical IoT networks (Internet of Things). enterprises and typical IoT networks (Internet of Things).
1.3. Leveraging the ACP 1.3. Leveraging a generalized autonomic control plane
One of the goals of the Autonomic Networks Autonomic Control Plane One of the goals of the IETF ANIMA (Autonomic Networking Integrated
(ACP as defined in [I-D.ietf-anima-autonomic-control-plane] ) is to Model and Approach ) working group is the specification of a secure
provide similar stable connectivity as a DCN, but without having to and automatically built inband management plane that provides similar
build a separate DCN. It is clear that such 'in-band' approach can stable connectivity as a DCN, but without having to build a separate
never achieve fully the same level of separation, but the goal is to DCN. It is clear that such 'in-band' approach can never achieve
get as close to it as possible. fully the same level of separation, but the goal is to get as close
to it as possible.
This solution approach has several aspects. One aspect is designing This goal of this document is to discuss how such an inband
the implementation of the ACP in network devices to make it actually management plane can be used to support the DCN-like OAM use-case,
perform without interruption by changes in what we will call in this leverage its stable connectivity and details the options of deploying
document the "data-plane", a.k.a: the operator or controller it incrementally - short and long term.
configured services planes of the network equipment. This aspect is
not currently covered in this document.
Another aspect is how to leverage the stable IPv6 connectivity The evolving ANIMA working groups specification
provided by the ACP for OAM purposes. This is the current scope of [I-D.ietf-anima-autonomic-control-plane] ) calls this inband
this document. management plane the "Autonomic Control Plane" (ACP). The
discussions in this document are not depending on the specification
of that ACP, but only on a set of high level constraints decided
early on in the work for the ACP. Unless being specific about
details of the ACP, this document uses the term "Generalized ACP"
(GACP) and is applicable to any designs that meet those high level
constraints. For example - but not limited to - variations of the
ACP protocol choices.
The high level constraints of a GACP assumed and discussed in this
document are as follows:
VRF Isolation: The GACP is a virtual network ("VRF") across network
devices - its routing and forwarding are separate from other
routing and forwarding in the network devices. Non-GACP routing/
forwarding is called the "data-plane".
IPv6 only addressing: The GACP provides only IPv6 reachability. It
uses ULA addresses ([RFC4193]) that are routed in a location
independent fashion for example through per network device subnet
prefixes. Automatic addressing in the GACP is therefore simple &
stable: it does not require allocation by address registries,
addresses are identifiers, they do not change when devices move,
and no engineering of the address space to the network topology is
necessary.
NOC connectivity: NOC equipment (controlling OAM operations) either
has access to the GACP directly or has an IP subnet connection to
a GACP-edge device.
Closed Group Security: GACP devices have cryptographic credentials
to mutually authenticate each other as members of a GACP. Traffic
across the GACP is authenticated with these credentials and then
encrypted. The only traffic permitted in & out of the GACP that
is not authenticated by these credentials is through explicit
configuration the traffic from/to the aforementioned non-GACP NOC
equipment with subnet connections to a GACP-edge device (as a
transition method).
The GACP must be built autonomic and its function must not be
disruptable by operator or automated (NMS/SDN) configuration/
provisioning actions. These are allowed to only impact the "data-
plane". This aspect is not currently covered in this document.
Instead, it focusses on the impact of the above constraints: IPv6
only, dual connectivity and security.
2. Solutions 2. Solutions
2.1. Stable Connectivity for Centralized OAM 2.1. Stable Connectivity for Centralized OAM
The ANI is the "Autonomic Networking Infrastructure" consisting of The ANI is the "Autonomic Networking Infrastructure" consisting of
secure zero touch Bootstrap (BRSKI - secure zero touch Bootstrap (BRSKI -
[I-D.ietf-anima-bootstrapping-keyinfra]), GeneRic Autonomic Signaling [I-D.ietf-anima-bootstrapping-keyinfra]), GeneRic Autonomic Signaling
Protocol (GRASP - [I-D.ietf-anima-grasp]), and Autonomic Control Protocol (GRASP - [I-D.ietf-anima-grasp]), and Autonomic Control
Plane (ACP - [I-D.ietf-anima-autonomic-control-plane]). Refer to Plane (ACP - [I-D.ietf-anima-autonomic-control-plane]). Refer to
[I-D.ietf-anima-reference-model] for an overview of the ANI and how [I-D.ietf-anima-reference-model] for an overview of the ANI and how
its components interact and [RFC7575] for concepts and terminology of its components interact and [RFC7575] for concepts and terminology of
ANI and autonomic networks. ANI and autonomic networks.
This section describes stable connectivity for centralized OAM via This section describes stable connectivity for centralized OAM via
ACP/ANI starting by what we expect to be the most easy to deploy the GACP, for example via the ACP with or without a complete ANI,
short-term option. It then describes limitation and challenges of starting by what we expect to be the most easy to deploy short-term
that approach and their solutions/workarounds to finish with the option. It then describes limitation and challenges of that approach
preferred target option of autonomic NOC devices in Section 2.1.6. and their solutions/workarounds to finish with the preferred target
option of autonomic NOC devices in Section 2.1.6.
This order was chosen because it helps to explain how simple initial This order was chosen because it helps to explain how simple initial
use of ACP can be, how difficult workarounds can become (and use of a GACP can be, how difficult workarounds can become (and
therefore what to avoid), and finally because one very promising therefore what to avoid), and finally because one very promising
long-term solution alternative is exactly like the most easy short- long-term solution alternative is exactly like the most easy short-
term solution only virtualized and automated. term solution only virtualized and automated.
In the most common case, OAM will be performed by one or more In the most common case, OAM will be performed by one or more
applications running on a variety of centralized NOC systems that applications running on a variety of centralized NOC systems that
communicate with network devices. We describe differently advanced communicate with network devices. We describe differently advanced
approaches to leverage the ACP for stable connectivity. There is a approaches to leverage a GACP for stable connectivity. There is a
wide range of options, some of which are simple, some more complex. wide range of options, some of which are simple, some more complex.
Three stages can be considered: Three stages can be considered:
o There are simple options described in sections Section 2.1.1 o There are simple options described in sections Section 2.1.1
through Section 2.1.3 that we consider to be good starting points through Section 2.1.3 that we consider to be good starting points
to operationalize the use of the ACP for stable connectivity to operationalize the use of a GACP for stable connectivity today.
today. These options require only network and OAN/NOC device These options require only network and OAN/NOC device
configuration. configuration.
o The are workarounds to connect the ACP to non-IPv6 capable NOC o The are workarounds to connect a GACP to non-IPv6 capable NOC
devices through the use of IPv4/IPv6 NAT (Network Address devices through the use of IPv4/IPv6 NAT (Network Address
Translation) as described in section Section 2.1.4. These Translation) as described in section Section 2.1.4. These
workarounds are not recommended but if such non-IPv6 capable NOC workarounds are not recommended but if such non-IPv6 capable NOC
devices need to be used longer term, then this is the only option devices need to be used longer term, then this is the only option
to connect them to the ACP. to connect them to a GACP.
o Near to long term options can provide all the desired operational, o Near to long term options can provide all the desired operational,
zero touch and security benefits of an autonomic network, but a zero touch and security benefits of an autonomic network, but a
range of details for this still have to be worked out and range of details for this still have to be worked out and
development work on NOC/OAM equipment is necessary. These options development work on NOC/OAM equipment is necessary. These options
are discussed in sections Section 2.1.5 through Section 2.1.8. are discussed in sections Section 2.1.5 through Section 2.1.8.
2.1.1. Simple Connectivity for Non-ACP capable NMS Hosts 2.1.1. Simple Connectivity for Non-GACP capable NMS Hosts
In the most simple candidate deployment case, the ACP extends all the In the most simple candidate deployment case, the GACP extends all
way into the NOC via one or more "ACP edge devices" as defined in the way into the NOC via one or more "GACP-edge-devices". See also
section 6.1 of [I-D.ietf-anima-autonomic-control-plane]. These section 6.1 of [I-D.ietf-anima-autonomic-control-plane]. These
devices "leak" the (otherwise encrypted) ACP natively to NMS hosts. devices "leak" the (otherwise encrypted) GACP natively to NMS hosts.
They acts as the default router to those NMS hosts and provide them They act as the default routers to those NMS hosts and provide them
with IPv6 connectivity into the ACP. NMS hosts with this setup need with IPv6 connectivity into the GACP. NMS hosts with this setup need
to support IPv6 (see e.g. [RFC6434]) but require no other to support IPv6 (see e.g. [RFC6434]) but require no other
modifications to leverage the ACP. modifications to leverage the GACP.
Note that even though the ACP only uses IPv6, it can of course Note that even though the GACP only uses IPv6, it can of course
support OAM for any type of network deployment as long as the network support OAM for any type of network deployment as long as the network
devices support the ACP: The Data Plane can be IPv4 only, dual-stack devices support the GACP: The data-plane can be IPv4 only, dual-stack
or IPv6 only. It is always spearate from the ACP, therefore there is or IPv6 only. It is always separate from the GACP, therefore there
no dependency between the ACP and the IP version(s) used in the Data is no dependency between the GACP and the IP version(s) used in the
Plane. data-plane.
This setup is sufficient for troubleshooting such as SSH into network This setup is sufficient for troubleshooting such as SSH into network
devices, NMS that performs SNMP read operations for status checking, devices, NMS that performs SNMP read operations for status checking,
software downloads into autonomic devices, provisioning of devices software downloads into autonomic devices, provisioning of devices
via NETCONF and so on. In conjunction with otherwise unmodified OAM via NETCONF and so on. In conjunction with otherwise unmodified OAM
via separate NMS hosts it can provide a good subset of the stable via separate NMS hosts it can provide a good subset of the stable
connectivity goals. The limitations of this approach are discussed connectivity goals. The limitations of this approach are discussed
in the next section. in the next section.
Because the ACP provides 'only' for IPv6 connectivity, and because Because the GACP provides 'only' for IPv6 connectivity, and because
addressing provided by the ACP does not include any topological addressing provided by the GACP does not include any topological
addressing structure that operations in a NOC often relies on to addressing structure that operations in a NOC often relies on to
recognize where devices are on the network, it is likely highly recognize where devices are on the network, it is likely highly
desirable to set up DNS (Domain Name System - see [RFC1034]) so that desirable to set up DNS (Domain Name System - see [RFC1034]) so that
the ACP IPv6 addresses of autonomic devices are known via domain the GACP IPv6 addresses of autonomic devices are known via domain
names that include the desired structure. For example, if DNS in the names that include the desired structure. For example, if DNS in the
network was set up with names for network devices as network was set up with names for network devices as
devicename.noc.example.com, and the well known structure of the Data devicename.noc.example.com, and the well-known structure of the data-
Plane IPv4 addresses space was used by operators to infer the region plane IPv4 addresses space was used by operators to infer the region
where a device is located in, then the ACP address of that device where a device is located in, then the GACP address of that device
could be set up as devicename_<region>.acp.noc.example.com, and could be set up as devicename_<region>.acp.noc.example.com, and
devicename.acp.noc.example.com could be a CNAME to devicename.acp.noc.example.com could be a CNAME to
devicename_<region>.acp.noc.example.com. Note that many networks devicename_<region>.acp.noc.example.com. Note that many networks
already use names for network equipment where topological information already use names for network equipment where topological information
is included, even without an ACP. is included, even without a GACP.
2.1.2. Challenges and Limitation of Simple Connectivity 2.1.2. Challenges and Limitation of Simple Connectivity
This simple connectivity of non-autonomic NMS hosts suffers from a This simple connectivity of non-autonomic NMS hosts suffers from a
range of challenges (that is, operators may not be able to do it this range of challenges (that is, operators may not be able to do it this
way) or limitations (that is, operator cannot achieve desired goals way) or limitations (that is, operator cannot achieve desired goals
with this setup). The following list summarizes these challenges and with this setup). The following list summarizes these challenges and
limitations. The following sections describe additional mechanisms limitations. The following sections describe additional mechanisms
to overcome them. to overcome them.
Note that these challenges and limitations exist because ACP is Note that these challenges and limitations exist because GACP is
primarily designed to support distributed ASA (Autonomic Service primarily designed to support distributed ASA (Autonomic Service
Agent, a piece of autonomic software) in the most lightweight Agent, a piece of autonomic software) in the most lightweight
fashion, but not mandatorily require support for additional fashion, but not mandatorily require support for additional
mechanisms to best support centralized NOC operations. It is this mechanisms to best support centralized NOC operations. It is this
document that describes additional (short term) workarounds and (long document that describes additional (short term) workarounds and (long
term) extensions. term) extensions.
1. (Limitation) NMS hosts cannot directly probe whether the desired 1. (Limitation) NMS hosts cannot directly probe whether the desired
so called 'data-plane' network connectivity works because they do so called 'data-plane' network connectivity works because they do
not directly have access to it. This problem is similar to not directly have access to it. This problem is similar to
probing connectivity for other services (such as VPN services) probing connectivity for other services (such as VPN services)
that they do not have direct access to, so the NOC may already that they do not have direct access to, so the NOC may already
employ appropriate mechanisms to deal with this issue (probing employ appropriate mechanisms to deal with this issue (probing
proxies). See Section 2.1.3 for candidate solutions. proxies). See Section 2.1.3 for candidate solutions.
2. (Challenge) NMS hosts need to support IPv6 which often is still 2. (Challenge) NMS hosts need to support IPv6 which often is still
not possible in enterprise networks. See Section 2.1.4 for some not possible in enterprise networks. See Section 2.1.4 for some
workarounds. workarounds.
3. (Limitation) Performance of the ACP will be limited versus normal 3. (Limitation) Performance of the GACP may be limited versus normal
'data-plane' connectivity. The setup of the ACP will often 'data-plane' connectivity. The setup of the GACP will often
support only non-hardware accelerated forwarding. Running a support only non-hardware accelerated forwarding. Running a
large amount of traffic through the ACP, especially for tasks large amount of traffic through the GACP, especially for tasks
where it is not necessary will reduce its performance/ where it is not necessary will reduce its performance/
effectiveness for those operations where it is necessary or effectiveness for those operations where it is necessary or
highly desirable. See Section 2.1.5 for candidate solutions. highly desirable. See Section 2.1.5 for candidate solutions.
4. (Limitation) Security of the ACP is reduced by exposing the ACP 4. (Limitation) Security of the GACP is reduced by exposing the GACP
natively (and unencrypted) into a subnet in the NOC where the NOC natively (and unencrypted) into a subnet in the NOC where the NOC
devices are attached to it. See Section 2.1.7 for candidate devices are attached to it. See Section 2.1.7 for candidate
solutions. solutions.
These four problems can be tackled independently of each other by These four problems can be tackled independently of each other by
solution improvements. Combining some of these solutions solution improvements. Combining some of these solutions
improvements together can lead towards a candiate long term solution. improvements together can lead towards a candidate long term
solution.
2.1.3. Simultaneous ACP and Data Plane Connectivity 2.1.3. Simultaneous GACP and data-plane Connectivity
Simultaneous connectivity to both ACP and data-plane can be achieved Simultaneous connectivity to both GACP and data-plane can be achieved
in a variety of ways. If the data-plane is IPv4-only, then any in a variety of ways. If the data-plane is IPv4-only, then any
method for dual-stack attachment of the NOC device/application will method for dual-stack attachment of the NOC device/application will
suffice: IPv6 connectivity from the NOC provides access via the ACP, suffice: IPv6 connectivity from the NOC provides access via the GACP,
IPv4 will provide access via the data-plane. If as explained above IPv4 will provide access via the data-plane. If as explained above
in the simple case, an autonomic device supports native attachment to in the simple case, an autonomic device supports native attachment to
the ACP, and the existing NOC setup is IPv4 only, then it could be the GACP, and the existing NOC setup is IPv4 only, then it could be
sufficient to attach the ACP device(s) as the IPv6 default router to sufficient to attach the GACP device(s) as the IPv6 default router to
the NOC subnet and keep the existing IPv4 default router setup the NOC subnet and keep the existing IPv4 default router setup
unchanged. unchanged.
If the data-plane of the network is also supporting IPv6, then the If the data-plane of the network is also supporting IPv6, then the
most compatible setup for NOC devices is to have two IPv6 interfaces. most compatible setup for NOC devices is to have two IPv6 interfaces.
One virtual ((e.g. via IEEE 802.1Q [IEEE802.1Q]) or physical One virtual ((e.g. via IEEE 802.1Q [IEEE802.1Q]) or physical
interface connecting to a data-plane subnet, and another into an ACP interface connecting to a data-plane subnet, and another into an GACP
connect subnet as specified in the ACP connection section of connect subnet. See section 8.1 of
[I-D.ietf-anima-autonomic-control-plane]. That document also [I-D.ietf-anima-autonomic-control-plane] for more details. That
specifies how the NOC devices can receive autoconfigured addressing document also specifies how the NOC devices can receive auto
and routes towards the ACP connect subnet if it supports [RFC6724] configured addressing and routes towards the ACP connect subnet if it
and [RFC4191]. supports [RFC6724] and [RFC4191].
Configuring a second interface on a NOC host may be impossible or be Configuring a second interface on a NOC host may be impossible or be
seen as undesired complexity. In that case the ACP edge device needs seen as undesired complexity. In that case the GACP edge device
to provide support for a "Combined ACP and Data Plane interface" as needs to provide support for a "Combined ACP and data-plane
also described in the ACP connect section of interface" as also described in section 8.1 of
[I-D.ietf-anima-autonomic-control-plane]. This setup may not work [I-D.ietf-anima-autonomic-control-plane]. This setup may not work
with autoconfiguration and all NOC host network stacks due to with auto configuration and all NOC host network stacks due to
limitations in those network stacks. They need to be able to perform limitations in those network stacks. They need to be able to perform
RFC6724 source address selection rule 5.5 including caching of next- RFC6724 source address selection rule 5.5 including caching of next-
hop information. See the ACP document text for more details. hop information.
For security reasons, it is not considered appropriate in the ACP For security reasons, it is not considered appropriate to connect a
document to connect a non-ACP router to an ACP connect interface. non-GACP router to a GACP connect interface. The reason is that the
The reason is that the ACP is a secured network domain and all NOC GACP is a secured network domain and all NOC devices connecting via
devices connecting via ACP connect interfaces are also part of that GACP connect interfaces are also part of that secure domain - the
secure domain - the main difference is that the physical link between main difference is that the physical link between the GACP edge
the ACP edge device and the NOC devices is not authenticated/ device and the NOC devices is not authenticated/encrypted and
encrypted and therefore needs to be physically secured. If the therefore, needs to be physically secured. If the secure GACP was
secure ACP was extendable via untrusted routers then it would be a extendable via untrusted routers then it would be a lot more
lot more verify the secure domain assertion. Therefore the ACP edge difficult to verify the secure domain assertion. Therefore the GACP
devices are not supposed to redistribute routes from non-ACP routers edge devices are not supposed to redistribute routes from non-GACP
into the ACP. routers into the GACP.
2.1.4. IPv4-only NMS Hosts 2.1.4. IPv4-only NMS Hosts
ACP does not support IPv4: Single stack IPv6 management of the One architectural expectation for the GACP as described in
network via ACP and (as needed) data plane. Independent of whether Section 1.3 is that all devices that want to use the GACP do support
the data plane is dual-stack, has IPv4 as a service or is single IPv6. Including NMS hosts. Note that this expectation does not
stack IPv6. Dual plane management, IPv6 for ACP, IPv4 for the data imply any requirements against the data-plane, especially no need to
plane is likewise an architecturally simple option. support IPv6 in it. The data-plane could be IPv4 only, IPv6 only,
dual stack or it may not need to have any IP host stack on the
network devices.
The implication of this architectural decision is the potential need The implication of this architectural decision is the potential need
for short-term workarounds when the operational practices in a for short-term workarounds when the operational practices in a
network do not yet meet these target expectations. This section network do not yet meet these target expectations. This section
explains when and why these workarounds may be operationally explains when and why these workarounds may be operationally
necessary and describes them. However, the long term goal is to necessary and describes them. However, the long term goal is to
upgrade all NMS hosts to native IPv6, so the workarounds described in upgrade all NMS hosts to native IPv6, so the workarounds described in
this section should not be considered permanent. this section should not be considered permanent.
Most network equipment today supports IPv6 but it is by far not Most network equipment today supports IPv6 but it is by far not
ubiquitously supported in NOC backend solutions (HW/SW), especially ubiquitously supported in NOC backend solutions (HW/SW), especially
not in the product space for enterprises. Even when it is supported, not in the product space for enterprises. Even when it is supported,
there are often additional limitations or issues using it in a dual there are often additional limitations or issues using it in a dual
stack setup or the operator mandates for simplicity single stack for stack setup or the operator mandates for simplicity single stack for
all operations. For these reasons an IPv4 only management plane is all operations. For these reasons an IPv4 only management plane is
still required and common practice in many enterprises. Without the still required and common practice in many enterprises. Without the
desire to leverage the ACP, this required and common practice is not desire to leverage the GACP, this required and common practice is not
a problem for those enterprises even when they run dual stack in the a problem for those enterprises even when they run dual stack in the
network. We discuss these workarounds here because it is a short network. We discuss these workarounds here because it is a short
term deployment challenge specific to the operations of the ACP. term deployment challenge specific to the operations of a GACP.
To connect IPv4 only management plane devices/applications with the To connect IPv4 only management plane devices/applications with a
ACP, some form of IP/ICMP translation of packets IPv4<->IPv6 is GACP, some form of IP/ICMP translation of packets IPv4<->IPv6 is
necessary. The basic mechanisms for this are defined in SIIT necessary. The basic mechanisms for this are defined in SIIT
([RFC7915]). There are multiple solutions using this mechanisms. To ([RFC7915]). There are multiple solutions using this mechanism. To
understand the possible solutions, we consider the requirements: understand the possible solutions, we consider the requirements:
1. NMS hosts need to be able to initiate connections to any ACP 1. NMS hosts need to be able to initiate connections to any GACP
device for management purposes. Examples include provisioning device for management purposes. Examples include provisioning
via Netconf/(SSH), SNMP poll operations or just diagnostics via via Netconf/(SSH), SNMP poll operations or just diagnostics via
SSH connections from operators. Every ACP device/function that SSH connections from operators. Every GACP device/function that
needs to be reachable from NMS hosts needs to have a separate needs to be reachable from NMS hosts needs to have a separate
IPv4 address. IPv4 address.
2. ACP devices need to be able to initiate connections to NMS hosts 2. GACP devices need to be able to initiate connections to NMS hosts
for example to initiate NTP or radius/diameter connections, send for example to initiate NTP or radius/diameter connections, send
syslog or SNMP trap or initiate Netconf Call Home connections syslog or SNMP trap or initiate Netconf Call Home connections
after bootstrap. Every NMS host needs to have a separate IPv6 after bootstrap. Every NMS host needs to have a separate IPv6
address reachable from the ACP. When connections from ACP address reachable from the GACP. When connections from GACP
devices are made to NMS hosts, the IPv4 source address of these devices are made to NMS hosts, the IPv4 source address of these
connections as seen by the NMS Host must also be unique per ACP connections as seen by the NMS Host must also be unique per GACP
device and the same address as in (1) to maintain the same device and the same address as in (1) to maintain the same
addressing simplicity as in a native IPv4 deployment. For addressing simplicity as in a native IPv4 deployment. For
example in syslog, the source-IP address of a logging device is example in syslog, the source-IP address of a logging device is
used to identify it, and if the device shows problems, an used to identify it, and if the device shows problems, an
operator might want to SSH into the device to diagnose it. operator might want to SSH into the device to diagnose it.
Because of these requirements, the necessary and sufficient set of Because of these requirements, the necessary and sufficient set of
solutions are those that provide 1:1 mapping of IPv6 ACP addresses solutions are those that provide 1:1 mapping of IPv6 GACP addresses
into IPv4 space and 1:1 mapping of IPv4 NMS host space into IPv6 (for into IPv4 space and 1:1 mapping of IPv4 NMS host space into IPv6 (for
use in the ACP). This means that stateless SIIT based solutions are use in the GACP). This means that stateless SIIT based solutions are
sufficient and preferred. sufficient and preferred.
Note that ACP devices may use multiple IPv6 addresses in the ACP Note that GACP devices may use multiple IPv6 addresses in the GACP.
based on which Sub-Scheme they use. For example in the Zone Sub- For example, [I-D.ietf-anima-autonomic-control-plane] section 6.10
Scheme, an ACP device could use two addresses, one with the last defines multiple useful addressing sub-schemes supporting this
address bit (V-bit) 0 and one with 1. Both addresses may need to be option. All those addresses may then need to be reachable through
reachable thought the IPv6/IPv4 address translation. the IPv6/IPv4 address translation.
The need to allocate for every ACP device one or multiple IPv4 The need to allocate for every GACP device one or multiple IPv4
addresses should not be a problem if - as we assume - the NMS hosts addresses should not be a problem if - as we assume - the NMS hosts
can use private IPv4 address space ([RFC1918]). Nevertheless even can use private IPv4 address space ([RFC1918]). Nevertheless even
with RFC1918 address space it is important that the ACP IPv6 with RFC1918 address space it is important that the GACP IPv6
addresses can efficiently be mapped into IPv4 address space without addresses can efficiently be mapped into IPv4 address space without
too much waste. too much waste.
The currently most flexible mapping scheme to achieve this is The currently most flexible mapping scheme to achieve this is
[RFC7757] because it allows configured IPv4 <-> IPv6 prefix mapping. [RFC7757] because it allows configured IPv4 <-> IPv6 prefix mapping.
Assume the ACP uses the Zone Addressing Sub-Scheme and there are 3 Assume the GACP uses the ACP "Zone Addressing" Sub-Scheme and there
registrars. In the Zone Addressing Sub-Scheme, there is for each are 3 registrars. In the Zone Addressing Sub-Scheme, there is for
registrar a constant /112 prefix for which in RFC7757 an EAM each registrar a constant /112 prefix for which in RFC7757 an EAM
(Explicit Address Mapping) into a /16 (eg: RFC1918) prefix into IPv4 (Explicit Address Mapping) into a /16 (e.g.: RFC1918) prefix into
can be configured. Within the registrars /112 prefix, Device-Numbers IPv4 can be configured. Within the registrars /112 prefix, Device-
for devices are sequentially assigned: with V-bit effectively two Numbers for devices are sequentially assigned: with V-bit effectively
numbers are assigned per ACP device. This also means that if IPv4 two numbers are assigned per GACP device. This also means that if
address space is even more constrained, and it is known that a IPv4 address space is even more constrained, and it is known that a
registrar will never need the full /15 extent of Device-Numbers, then registrar will never need the full /15 extent of Device-Numbers, then
a longer than /112 prefix can be configured into the EAM to use less a longer than /112 prefix can be configured into the EAM to use less
IPv4 space. IPv4 space.
When using the Vlong Addressing Sub-Scheme, it is unlikely that one When using the ACP "Vlong Addressing" Sub-Scheme, it is unlikely that
wants or need to translate the full /8 or /16 bits of addressing one wants or need to translate the full /8 or /16 bits of addressing
space per ACP device into IPv4. In this case, the EAM rules of space per GACP device into IPv4. In this case, the EAM rules of
dropping trailing bits can be used to map only N bits of the V-bits dropping trailing bits can be used to map only N bits of the V-bits
into IPv4. This does imply though that only V-addresses that differ into IPv4. This does imply though that only V-addresses that differ
in those high-order N V-bits can be distinguished on the IPv4 side. in those high-order N V-bits can be distinguished on the IPv4 side.
Likewise, the IPv4 address space used for NMS hosts can easily be Likewise, the IPv4 address space used for NMS hosts can easily be
mapped into an ACP prefix assigned to an ACP connect interface. mapped into an address prefix assigned to a GACP connect interface.
A full specification of a solution to perform SIIT in conjunction A full specification of a solution to perform SIIT in conjunction
with ACP connect following the considerations below is outside the with GACP connect following the considerations below is outside the
scope of this document. scope of this document.
To be in compliance with security expectations, SIIT has to to happen To be in compliance with security expectations, SIIT has to happen on
on the ACP edge device itself so that ACP security considerations can the GACP edge device itself so that GACP security considerations can
be taken into account. Eg: that IPv4 only NMS hosts can be dealt be taken into account. E.g.: that IPv4 only NMS hosts can be dealt
with exactly like IPv6 hosts connected to an ACP connect interface. with exactly like IPv6 hosts connected to a GACP connect interface.
Note that prior solutions such as NAT64 ([RFC6146]) may equally be Note that prior solutions such as NAT64 ([RFC6146]) may equally be
useable to translate between ACP IPv6 address space and NMS Hosts useable to translate between GACP IPv6 address space and NMS Hosts
IPv4 address space, and that as workarounds this can also be done on IPv4 address space, and that as workarounds this can also be done on
non ACP Edge Devices connected to an ACP connect interface. The non GACP Edge Devices connected to a GACP connect interface. The
details vary depending on implementation because the options to details vary depending on implementation because the options to
configure address mappings vary widely. Outside of EAM, there are no configure address mappings vary widely. Outside of EAM, there are no
standardized solutions that allow for mapping of prefixes, so it will standardized solutions that allow for mapping of prefixes, so it will
most likely be necessary to explicitly map every individual (/128) most likely be necessary to explicitly map every individual (/128)
ACP device address to an IPv4 address. Such an approach should use GACP device address to an IPv4 address. Such an approach should use
automation/scripting where these address translation entries are automation/scripting where these address translation entries are
created dynamically whenever an ACP device is enrolled or first created dynamically whenever a GACP device is enrolled or first
connected to the ACP network. connected to the GACP network.
Overall, the use of NAT is especially subject to the ROI (Return On Overall, the use of NAT is especially subject to the ROI (Return On
Investment) considerations, but the methods described here may not be Investment) considerations, but the methods described here may not be
too different from the same problems encountered totally independent too different from the same problems encountered totally independent
of AN/ACP when some parts of the network are to introduce IPv6 but of GACP when some parts of the network are to introduce IPv6 but NMS
NMS hosts are not (yet) upgradeable. hosts are not (yet) upgradeable.
2.1.5. Path Selection Policies 2.1.5. Path Selection Policies
As mentioned above, the ACP is not expected to have high performance As mentioned above, a GACP is not expected to have high performance
because its primary goal is connectivity and security, and for because its primary goal is connectivity and security, and for
existing network device platforms this often means that it is a lot existing network device platforms this often means that it is a lot
more effort to implement that additional connectivity with hardware more effort to implement that additional connectivity with hardware
acceleration than without - especially because of the desire to acceleration than without - especially because of the desire to
support full encryption across the ACP to achieve the desired support full encryption across the GACP to achieve the desired
security. security.
Some of these issues may go away in the future with further adoption Some of these issues may go away in the future with further adoption
of the ACP and network device designs that better tender to the needs of a GACP and network device designs that better tender to the needs
of a separate OAM plane, but it is wise to plan for even long-term of a separate OAM plane, but it is wise to plan for even long-term
designs of the solution that does NOT depend on high-performance of designs of the solution that does NOT depend on high-performance of
the ACP. This is opposite to the expectation that future NMS hosts the GACP. This is opposite to the expectation that future NMS hosts
will have IPv6, so that any considerations for IPv4/NAT in this will have IPv6, so that any considerations for IPv4/NAT in this
solution are temporary. solution are temporary.
To solve the expected performance limitations of the ACP, we do To solve the expected performance limitations of the GACP, we do
expect to have the above describe dual-connectivity via both ACP and expect to have the above describe dual-connectivity via both GACP and
data-plane between NOC application devices and AN devices with ACP. data-plane between NOC application devices and devices with GACP.
The ACP connectivity is expected to always be there (as soon as a The GACP connectivity is expected to always be there (as soon as a
device is enrolled), but the data-plane connectivity is only present device is enrolled), but the data-plane connectivity is only present
under normal operations but will not be present during e.g. early under normal operations but will not be present during e.g. early
stages of device bootstrap, failures, provisioning mistakes or during stages of device bootstrap, failures, provisioning mistakes or during
network configuration changes. network configuration changes.
The desired policy is therefore as follows: In the absence of further The desired policy is therefore as follows: In the absence of further
security considerations (see below), traffic between NMS hosts and AN security considerations (see below), traffic between NMS hosts and
devices should prefer data-plane connectivity and resort only to GACP devices should prefer data-plane connectivity and resort only to
using the ACP when necessary, unless it is an operation known to be using the GACP when necessary, unless it is an operation known to be
so much tied to the cases where the ACP is necessary that it makes no so much tied to the cases where the GACP is necessary that it makes
sense to try using the data plane. An example here is of course the no sense to try using the data-plane. An example are SSH connections
SSH connection from the NOC into a network device to troubleshoot from the NOC into a network device to troubleshoot network
network connectivity. This could easily always rely on the ACP. connectivity. This could easily always rely on the GACP. Likewise,
Likewise, if an NMS host is known to transmit large amounts of data, if an NMS host is known to transmit large amounts of data, and it
and it uses the ACP, then its performance need to be controlled so uses the GACP, then its performance need to be controlled so that it
that it will not overload the ACP performance. Typical examples of will not overload the GACP performance. Typical examples of this are
this are software downloads. software downloads.
There is a wide range of methods to build up these policies. We There is a wide range of methods to build up these policies. We
describe a few: describe a few:
Ideally, a NOC system would learn and keep track of all addresses of Ideally, a NOC system would learn and keep track of all addresses of
a device (ACP and the various data plane addresses). Every action of a device (GACP and the various data-plane addresses). Every action
the NOC system would indicate via a "path-policy" what type of of the NOC system would indicate via a "path-policy" what type of
connection it needs (e.g. only data-plane, ACP-only, default to data- connection it needs (e.g. only data-plane, GACP-only, default to
plane, fallback to ACP,...). A connection policy manager would then data-plane, fallback to GACP,...). A connection policy manager would
build connection to the target using the right address(es). Shorter then build connection to the target using the right address(es).
term, a common practice is to identify different paths to a device Shorter term, a common practice is to identify different paths to a
via different names (e.g. loopback vs. interface addresses). This device via different names (e.g. loopback vs. interface addresses).
approach can be expanded to ACP uses, whether it uses NOC system This approach can be expanded to GACP uses, whether it uses NOC
local names or DNS. We describe example schemes using DNS: system local names or DNS. We describe example schemes using DNS:
DNS can be used to set up names for the same network devices but with DNS can be used to set up names for the same network devices but with
different addresses assigned: One name (name.noc.example.com) with different addresses assigned: One name (name.noc.example.com) with
only the data-plane address(es) (IPv4 and/or IPv6) to be used for only the data-plane address(es) (IPv4 and/or IPv6) to be used for
probing connectivity or performing routine software downloads that probing connectivity or performing routine software downloads that
may stall/fail when there are connectivity issues. One name (name- may stall/fail when there are connectivity issues. One name (name-
acp.noc.example.com) with only the ACP reachable address of the acp.noc.example.com) with only the GACP reachable address of the
device for troubleshooting and probing/discovery that is desired to device for troubleshooting and probing/discovery that is desired to
always only use the ACP. One name with data plane and ACP addresses always only use the GACP. One name with data-plane and GACP
(name-both.noc.example.com). addresses (name-both.noc.example.com).
Traffic policing and/or shaping of at the ACP edge in the NOC can be Traffic policing and/or shaping at the GACP edge in the NOC can be
used to throttle applications such as software download into the ACP. used to throttle applications such as software download into the
GACP.
MPTCP (Multipath TCP -see [RFC6824]) is a very attractive candidate MPTCP (Multipath TCP -see [RFC6824]) with additional future work
to automate the use of both data-plane and ACP and minimize or fully could be a very attractive candidate to automate the use of both
avoid the need for the above mentioned logical names to pre-set the data-plane and GACP and minimize or fully avoid the need for the
desired connectivity (data-plane-only, ACP only, both). For example, above mentioned logical names to pre-set the desired connectivity
a set-up for non MPTCP aware applications would be as follows: (data-plane-only, GACP only, both).
DNS naming is set up to provide the ACP IPv6 address of network MPTCP supports signaling between the peers with which additional
devices. Unbeknownst to the application, MPTCP is used. MPTCP available addresses can be exchanged and then additional subflows
mutually discovers between the NOC and network device the data-plane between those addresses can be built. MPTCP also allows to declare
address and caries all traffic across it when that MPTCP subflow subflows to be backup so they do not carry traffic unless non-backup
across the data-plane can be built. subflows fail. Taken together, this could be used to automatically
leverage the stable connectivity of the GACP and the potentially
higher throughput of the data plane and simplify the DNS setup:
In the Autonomic network devices where data-plane and ACP are in (Only) the GACP address of GACP devices could to be put into DNS.
separate VRFs, it is clear that this type of MPTCP subflow creation When MPTCP builds the first subflow across the GACP, it exchanges the
across different VRFs is new/added functionality. Likewise, the data-plane address. A data-plane subflow is built. Once this is
policies of preferring a particular address (NOC-device) or VRF (AN successful, the peers declare the GACP subflow to be a backup flow
device) for the traffic is potentially also a policy not provided as and only the data-plane subflow is used to carry traffic.
a standard.
Making this idea work requires additional specification work outside
the scope of this document. This includes but is not necessarily
limited to defining the following aspects:
The above described behavior/policy for MPTCP must be controllable by
applications or libraries acting on behalf of applications. APIs
and/or data models for this control need to be defined. As outlined
above, applications for example may choose to only perform transfers
if the data-plane is actually available because of performance
limitations of the GACP, so the application needs to be made aware if
the setup of the data-plane subflow fails. Or transfers may want to
only use the GACP because the connection performs configuration
changes that are likely known to bring down the data-plane. It
should be sufficient to make these policies work on a per connection
basis, and not change during the lifetime of a connection for
different data items.
GACP and data-plane addresses exist in different VRFs. The MPTCP
stack needs to be able to access multiple VRFs and know which local
address is existing in which VRF. There also needs to be a logic to
determine which peer address is expected to be reachable via which
local VRF/address. Because the GACP is an isolated network, data-
plane addresses are not reachable via the GACP therefore devices can
determine if a peer address is a data-plane address.
2.1.6. Autonomic NOC Device/Applications 2.1.6. Autonomic NOC Device/Applications
Setting up connectivity between the NOC and autonomic devices when Setting up connectivity between the NOC and autonomic devices when
the NOC device itself is non-autonomic is as mentioned in the the NOC device itself is non-autonomic is as mentioned in the
beginning a security issue. It also results as shown in the previous beginning a security issue. It also results as shown in the previous
paragraphs in a range of connectivity considerations, some of which paragraphs in a range of connectivity considerations, some of which
may be quite undesirable or complex to operationalize. may be quite undesirable or complex to operationalize.
Making NMS hosts autonomic and having them participate in the ACP is Making NMS hosts autonomic and having them participate in the GACP is
therefore not only a highly desirable solution to the security therefore not only a highly desirable solution to the security
issues, but can also provide a likely easier operationalization of issues, but can also provide a likely easier operationalization of
the ACP because it minimizes NOC-special edge considerations - the the GACP because it minimizes NOC-special edge considerations - the
ACP is simply built all the way automatically, even inside the NOC GACP is simply built all the way automatically, even inside the NOC
and only authorized and authenticate NOC devices/applications will and only authorized and authenticate NOC devices/applications will
have access to it. have access to it.
Supporting the ACP all the way into an application device requires Supporting the ACP according to
implementing the following aspects in it: AN bootstrap/enrollment [I-D.ietf-anima-autonomic-control-plane] all the way into an
mechanisms, the secure channel for the ACP and at least the host side application device requires implementing the following aspects in it:
of IPv6 routing setup for the ACP. Minimally this could all be AN bootstrap/enrollment mechanisms, the secure channel for the ACP
implemented as an application and be made available to the host OS and at least the host side of IPv6 routing setup for the ACP.
via e.g. a tap driver to make the ACP show up as another IPv6 enabled Minimally this could all be implemented as an application and be made
interface. available to the host OS via e.g. a tap driver to make the ACP show
up as another IPv6 enabled interface.
Having said this: If the structure of NMS hosts is transformed Having said this: If the structure of NMS hosts is transformed
through virtualization anyhow, then it may be considered equally through virtualization anyhow, then it may be considered equally
secure and appropriate to construct (physical) NMS host system by secure and appropriate to construct (physical) NMS host system by
combining a virtual AN/ACP enabled router with non-AN/ACP enabled combining a virtual GACP enabled router with non-GACP enabled NOC-
NOC-application VMs via a hypervisor, leveraging the configuration application VMs via a hypervisor, leveraging the configuration
options described in the previous sections but just virtualizing options described in the previous sections but just virtualizing
them. them.
2.1.7. Encryption of data-plane connections 2.1.7. Encryption of data-plane connections
When combining ACP and data-plane connectivity for availability and When combining GACP and data-plane connectivity for availability and
performance reasons, this too has an impact on security: When using performance reasons, this too has an impact on security: When using
the ACP, the traffic will be mostly encryption protected, especially the GACP, the traffic will be mostly encryption protected, especially
when considering the above described use of AN application devices. when considering the above described use of application devices with
If instead the data-plane is used, then this is not the case anymore GACP. If instead the data-plane is used, then this is not the case
unless it is done by the application. anymore unless it is done by the application.
The simplest solution for this problem exists when using AN capable The simplest solution for this problem exists when using GACP capable
NMS hosts, because in that case the communicating AN capable NMS host NMS hosts, because in that case the communicating GACP capable NMS
and the AN network device have certificates through the AN enrollment host and the GACP network device have credentials they can mutually
process that they can mutually trust (same AN domain). In result, trust (same GACP domain). In result, data-plane connectivity that
data-plane connectivity that does support this can simply leverage does support this can simply leverage TLS/DTLS
TLS/DTLS ([RFC5246])/([RFC6347]) with mutual AN-domain certificate ([RFC5246])/([RFC6347]) with those GACP credentials for mutual
authentication - and does not incur new key management. authentication - and does not incur new key management.
If this automatic security benefit is seen as most important, but a If this automatic security benefit is seen as most important, but a
"full" ACP stack into the NMS host is unfeasible, then it would still "full" GACP stack into the NMS host is unfeasible, then it would
be possible to design a stripped down version of AN functionality for still be possible to design a stripped down version of GACP
such NOC hosts that only provides enrollment of the NOC host into the functionality for such NOC hosts that only provides enrollment of the
AN domain to the extent that the host receives an AN domain NOC host with the GACP cryptographic credentials but without directly
certificate, but without directly participating in the ACP participating in the GACP encryption method. Instead, the host would
afterwards. Instead, the host would just leverage TLS/DTLS using its just leverage TLS/DTLS using its GACP credentials via the data-plane
AN certificate via the data-plane with AN network devices as well as with GACP network devices as well as indirectly via the GACP with the
indirectly via the ACP with the above mentioned in-NOC network edge above mentioned GACP connect into the GACP.
connectivity into the ACP.
When using the ACP itself, TLS/DTLS for the transport layer between When using the GACP itself, TLS/DTLS for the transport layer between
NMS hosts and network device is somewhat of a double price to pay NMS hosts and network device is somewhat of a double price to pay
(ACP also encrypts) and could potentially be optimized away, but (GACP also encrypts) and could potentially be optimized away, but
given the assumed lower performance of the ACP, it seems that this is given the assumed lower performance of the GACP, it seems that this
an unnecessary optimization. is an unnecessary optimization.
2.1.8. Long Term Direction of the Solution 2.1.8. Long Term Direction of the Solution
If we consider what potentially could be the most lightweight and If we consider what potentially could be the most lightweight and
autonomic long term solution based on the technologies described autonomic long term solution based on the technologies described
above, we see the following direction: above, we see the following direction:
1. NMS hosts should at least support IPv6. IPv4/IPv6 NAT in the 1. NMS hosts should at least support IPv6. IPv4/IPv6 NAT in the
network to enable use of ACP is long term undesirable. Having network to enable use of a GACP is long term undesirable. Having
IPv4 only applications automatically leverage IPv6 connectivity IPv4 only applications automatically leverage IPv6 connectivity
via host-stack translation may be an option but this has not been via host-stack translation may be an option but this has not been
investigated yet. investigated yet.
2. Build the ACP as a lightweight application for NMS hosts so ACP 2. Build the GACP as a lightweight application for NMS hosts so GACP
extends all the way into the actual NMS hosts. extends all the way into the actual NMS hosts.
3. Leverage and as necessary enhance MPTCP with automatic dual- 3. Leverage and as necessary enhance MPTCP with automatic dual-
connectivity: If an MPTCP unaware application is using ACP connectivity: If an MPTCP unaware application is using GACP
connectivity, the policies used should add subflow(s) via the connectivity, the policies used should add subflow(s) via the
data-plane and prefer them. data-plane and prefer them.
4. Consider how to best map NMS host desires to underlying transport 4. Consider how to best map NMS host desires to underlying transport
mechanisms: With the above mentioned 3 points, not all options mechanisms: With the above mentioned 3 points, not all options
are covered. Depending on the OAM, one may still want only ACP, are covered. Depending on the OAM, one may still want only GACP,
only data-plane, or automatically prefer one over the other and/ only data-plane, or automatically prefer one over the other and/
or use the ACP with low performance or high-performance (for or use the GACP with low performance or high-performance (for
emergency OAM such as countering DDoS). It is as of today not emergency OAM such as countering DDoS). It is as of today not
clear what the simplest set of tools is to enable explicitly the clear what the simplest set of tools is to enable explicitly the
choice of desired behavior of each OAM. The use of the above choice of desired behavior of each OAM. The use of the above
mentioned DNS and MPTCP mechanisms is a start, but this will mentioned DNS and MPTCP mechanisms is a start, but this will
require additional thoughts. This is likely a specific case of require additional thoughts. This is likely a specific case of
the more generic scope of TAPS. the more generic scope of TAPS.
2.2. Stable Connectivity for Distributed Network/OAM 2.2. Stable Connectivity for Distributed Network/OAM
The ANI (ACP, Bootstrap, GRASP) can provide via the GRASP protocol Today, many distributed protocols implement their own unique security
common direct-neighbor discovery and capability negotiation (GRASP mechanisms.
via ACP and/or data-plane) and stable and secure connectivity for
functions running distributed in network devices (GRASP via ACP). It
can therefore eliminate the need to re-implement similar functions in
each distributed function in the network. Today, every distributed
protocol does this with functional elements usually called "Hello"
mechanisms and with often protocol specific security mechanisms.
KARP (Keying and Authentication for Routing Protocols, see [RFC6518]) KARP (Keying and Authentication for Routing Protocols, see [RFC6518])
has tried to start provide common directions and therefore reduce the has tried to start to provide common directions and therefore reduce
re-invention of at least some of the security aspects, but it only the re-invention of at least some of the security aspects, but it
covers routing-protocols and it is unclear how well it applicable to only covers routing-protocols and it is unclear how well it is
a potentially wider range of network distributed agents such as those applicable to a potentially wider range of network distributed agents
performing distributed OAM. The ACP can help in these cases. such as those performing distributed OAM. The common security of a
GACP can help in these cases.
Furthermore, GRASP ([I-D.ietf-anima-grasp]) can run on top of a GACP
as a security and transport substrate and provide common local &
remote neighbor discovery and peer negotiation mechanism, further
allowing to unifying & reuse future protocol designs.
3. Architectural Considerations 3. Architectural Considerations
3.1. No IPv4 for ACP 3.1. No IPv4 for GACP
The ACP is targeted to be IPv6 only, and the prior explanations in The GACP is intended to be IPv6 only, and the prior explanations in
this document show that this can lead to some complexity when having this document show that this can lead to some complexity when having
to connect IPv4 only NOC solutions, and that it will be impossible to to connect IPv4 only NOC solutions, and that it will be impossible to
leverage the ACP when the OAM agents on an ACP network device do not leverage the GACP when the OAM agents on a GACP network device do not
support IPv6. Therefore, the question was raised whether the ACP support IPv6. Therefore, the question was raised whether the GACP
should optionally also support IPv4. should optionally also support IPv4.
The decision not to include IPv4 for ACP as something that is The decision not to include IPv4 for GACP as something that is
considered in the use cases in this document is because of the considered in the use cases in this document is because of the
following reasons: following reasons:
In SP networks that have started to support IPv6, often the next In SP networks that have started to support IPv6, often the next
planned step is to consider moving out IPv4 from a native transport planned step is to consider moving out IPv4 from a native transport
as just a service on the edge. There is no benefit/need for multiple to just a service on the edge. There is no benefit/need for multiple
parallel transport families within the network, and standardizing on parallel transport families within the network, and standardizing on
one reduces OPEX and improves reliability. This evolution in the one reduces OPEX and improves reliability. This evolution in the
data plane makes it highly unlikely that investing development cycles data-plane makes it highly unlikely that investing development cycles
into IPv4 support for ACP will have a longer term benefit or enough into IPv4 support for GACP will have a longer term benefit or enough
critical short-term use-cases. Support for IPv4-only for ACP is critical short-term use-cases. Support for IPv6-only for GACP is
purely a strategic choice to focus on the known important long term purely a strategic choice to focus on the known important long term
goals. goals.
In other type of networks as well, we think that efforts to support In other types of networks as well, we think that efforts to support
autonomic networking is better spent in ensuring that one address autonomic networking is better spent in ensuring that one address
family will be support so all use cases will long-term work with it, family will be supported so all use cases will long-term work with
instead of duplicating effort into IPv4. Especially because auto- it, instead of duplicating effort into IPv4. Especially because
addressing for the ACP with IPv4 would be more complex than in IPv6 auto-addressing for the GACP with IPv4 would be more complex than in
due to the IPv4 addressing space. IPv6 due to the IPv4 addressing space.
4. Security Considerations 4. Security Considerations
In this section, we discuss only security considerations not covered In this section, we discuss only security considerations not covered
in the appropriate sub-sections of the solutions described. in the appropriate sub-sections of the solutions described.
Even though ACPs are meant to be isolated, explicit operator Even though GACPs are meant to be isolated, explicit operator
misconfiguration to connect to insecure OAM equipment and/or bugs in misconfiguration to connect to insecure OAM equipment and/or bugs in
ACP devices may cause leakage into places where it is not expected. GACP devices may cause leakage into places where it is not expected.
Mergers/Acquisitions and other complex network reconfigurations Mergers/Acquisitions and other complex network reconfigurations
affecting the NOC are typical examples. affecting the NOC are typical examples.
ACP prefix addresses are ULA addresses. Using these addresses also GACP addresses are ULA addresses. Using these addresses also for NOC
for NOC devices as proposed in this document is not only necessary devices as proposed in this document is not only necessary for above
for above explained simple routing functionality but it is also more explained simple routing functionality but it is also more secure
secure than global IPv6 addresses. ULA addresses are not routed in than global IPv6 addresses. ULA addresses are not routed in the
the global Internet and will therefore be subject to more filtering global Internet and will therefore be subject to more filtering even
even in places where specific ULA addresses are being used. Packets in places where specific ULA addresses are being used. Packets are
are therefore less likely to leak to be successfully injected into therefore less likely to leak to be successfully injected into the
the isolated ACP environment. isolated GACP environment.
The random nature of a ULA prefix provides strong protection against The random nature of a ULA prefix provides strong protection against
address collision even though there is no central assignment address collision even though there is no central assignment
authority. This is helped by the expectation, that ACPs are never authority. This is helped by the expectation that GACPs are never
expected to connect all together, but only few ACPs may ever need to expected to connect all together, but only few GACPs may ever need to
connect together, e.g. when mergers and aquisitions occur. connect together, e.g. when mergers and acquisitions occur.
The ACP specification demands that only packets from configured ACP Note that the GACP constraints demands that only packets from
prefixes are permitted from ACP connect interfaces. It also requires connected subnet prefixes are permitted from GACP connect interfaces,
that RPL root ACP devices need to be able to diagnose unknown ACP limiting the scope of non-cryptographically secured transport to a
destination addresses. subnet within a NOC that instead has to rely on physical security
(only connect trusted NOC devices to it).
To help diagnose packets that unexpectedly leaked for example from To help diagnose packets that unexpectedly leaked for example from
another ACP (that was meant to be deployed separately), it can be another GACP (that was meant to be deployed separately), it can be
useful to voluntarily list your own the ULA ACP prefixes in one of useful to voluntarily list your own the ULA GACP prefixes on some
the sites on the Internet, for example site(s) on the Internet and hope that other users of GACPs do the
https://www.sixxs.net/tools/grh/ula/. Note that this does not same so that you can look up unknown ULA prefix packets seen in your
constitute registration and if you want to ensure that your leaked network. Note that this does not constitute registration.
ACP packets can be recognized to come from you, you may need to list https://www.sixxs.net/tools/grh/ula/ was a site to list ULA prefixes
your prefixes in multiple of those sites. but it is not open for new listings anymore since the mid of 2017.
The authors are not aware of other active Internet sites to list ULA
use.
Note that there is a provision in [RFC4193] for non-locally assigned Note that there is a provision in [RFC4193] for non-locally assigned
address space (L bit = 0), but there is no existing standardization address space (L bit = 0), but there is no existing standardization
for this, so these ULA prefixes must not be used. for this, so these ULA prefixes must not be used.
According to [RFC4193] section 4.4, PTR records for ULA addresses According to [RFC4193] section 4.4, PTR records for ULA addresses
should not be installed into the global DNS (no guaranteed should not be installed into the global DNS (no guaranteed
ownership). Hence also the need to rely on voluntary lists (and in ownership). Hence also the need to rely on voluntary lists (and in
prior paragraph) to make the use of an ULA prefix globally known. prior paragraph) to make the use of an ULA prefix globally known.
Nevertheless, some legacy OAM applications running across the ACP may Nevertheless, some legacy OAM applications running across the GACP
rely on reverse DNS lookup for authentication of requests (eg: TFTP may rely on reverse DNS lookup for authentication of requests (e.g.:
for download of network firmware/config/software). Operators may TFTP for download of network firmware/config/software). Operators
therefore need to use a private DNS setup for the ACP ULA addresses. may therefore need to use a private DNS setup for the GACP ULA
This is the same setup that would be necessary for using RFC1918 addresses. This is the same setup that would be necessary for using
addresses in DNS. See for example [RFC1918] section 5, last RFC1918 addresses in DNS. See for example [RFC1918] section 5, last
paragraph. In [RFC6950] section 4, these setups are discussed in paragraph. In [RFC6950] section 4, these setups are discussed in
more detail. more detail.
Any current and future protocols must rely on secure end-to-end Any current and future protocols must rely on secure end-to-end
communications (TLS/DTLS) and identification and authentication via communications (TLS/DTLS) and identification and authentication via
the certificates assigned to both ends. This is enabled by the the certificates assigned to both ends. This is enabled by the
certificate mechanisms of the ACP. cryptographic credentials mechanisms of the GACP.
If DNS and especially reverse DNS are set up, then it should be set If DNS and especially reverse DNS are set up, then it should be set
up in an automated fashion, linked to the autonomic registrar backend up in an automated fashion when the GACP address for devices are
so that the DNS and reverse DNS records are actually derived from the assigned. In the case of the ACP, DNS resource record creation can
subject name elements of the ACP device certificates in the same way be linked to the autonomic registrar backend so that the DNS and
as the autonomic devices themselves will derive their ULA addresses reverse DNS records are actually derived from the subject name
from their certificates to ensure correct and consistent DNS entries. elements of the ACP device certificates in the same way as the
autonomic devices themselves will derive their ULA addresses from
their certificates to ensure correct and consistent DNS entries.
If an operator feels that reverse DNS records are beneficial to its If an operator feels that reverse DNS records are beneficial to its
own operations but that they should not be made available publically own operations but that they should not be made available publically
for "security" by concealment reasons, then the case of ACP DNS for "security" by concealment reasons, then the case of GACP DNS
entries is probably one of the least problematic use cases for split- entries is probably one of the least problematic use cases for split-
DNS: The ACP DNS names are only needed for the NMS hosts intending to DNS: The GACP DNS names are only needed for the NMS hosts intending
use the ACP - but not network wide across the enterprise. to use the GACP - but not network wide across the enterprise.
5. IANA Considerations 5. IANA Considerations
This document requests no action by IANA. This document requests no action by IANA.
6. Acknowledgements 6. Acknowledgements
This work originated from an Autonomic Networking project at cisco This work originated from an Autonomic Networking project at cisco
Systems, which started in early 2010 including customers involved in Systems, which started in early 2010 including customers involved in
the design and early testing. Many people contributed to the aspects the design and early testing. Many people contributed to the aspects
described in this document, including in alphabetical order: BL described in this document, including in alphabetical order: BL
Balaji, Steinthor Bjarnason, Yves Herthoghs, Sebastian Meissner, Ravi Balaji, Steinthor Bjarnason, Yves Herthoghs, Sebastian Meissner, Ravi
Kumar Vadapalli. The author would also like to thank Michael Kumar Vadapalli. The author would also like to thank Michael
Richardson, James Woodyatt and Brian Carpenter for their review and Richardson, James Woodyatt and Brian Carpenter for their review and
comments. Special thanks to Sheng Jiang and Mohamed Boucadair for comments. Special thanks to Sheng Jiang and Mohamed Boucadair for
their thorough review. their thorough review.
7. Change log [RFC Editor: Please remove] 7. Change log [RFC Editor: Please remove]
08: IESG review fixes.
* Spell check.
* https://raw.githubusercontent.com/anima-wg/autonomic-control-
plane/01908364cfc7259009603bf2b261354b0cc26913/draft-ietf-
anima-stable-connectivity/draft-ietf-anima-stable-connectivity-
08.txt
* Eric Rescorla (comments):Typos, listing ULA on internet
benefits. Other comments from Eric where addressed via commits
for other reviewers already.
* https://raw.githubusercontent.com/anima-wg/autonomic-control-
plane/c02252710fbd7aea15aff550fb393eb36584658b/draft-ietf-
anima-stable-connectivity/draft-ietf-anima-stable-connectivity-
08.txt
* Mirja Kuehlewind (discuss) / Yoshifumi Nishida: Fixed and
Rewrote MPTCP text to be more explanatory, answering questions
raised in disucss.
* https://raw.githubusercontent.com/anima-wg/autonomic-control-
plane/14d5f9b66b8318bc160cee74ad152c0b926b4042/draft-ietf-
anima-stable-connectivity/draft-ietf-anima-stable-connectivity-
08.txt
* Matthew Miller/Alissa Cooper: syntactic nits.
* https://raw.githubusercontent.com/anima-wg/autonomic-control-
plane/9bff109281e8b3c22522c3144cbf0f13e5000498/draft-ietf-
anima-stable-connectivity/draft-ietf-anima-stable-connectivity-
08.txt
* Suresh Krishnan (comment): rewrote first paragraph of 2.1.4
(incomprehensible).
* https://raw.githubusercontent.com/anima-wg/autonomic-control-
plane/f2d8a85c2cc65ca7f823abac0f57d19c744f9b65/draft-ietf-
anima-stable-connectivity/draft-ietf-anima-stable-connectivity-
08.txt
* Alvaro Retana: Made references normative where the authors
think is is important to understand all or parts for the
mechanisms described in this document.
* Alvaro Retana: Clarified that the discussions in this document
are not specific to the ANI ACP, but instead rely primarily on
a set of design constraints for any type of autonomic inband
management network. Called this the GACP (generalized ACP).
Mayor add: explanation of those design constraints in section
1.3. Textual fixes ACP -> GACP throughout the document, but
without semantic changes.
* https://raw.githubusercontent.com/anima-wg/autonomic-control-
plane/d26df831da2953779c3b3ac41ec118cbbe43373e/draft-ietf-
anima-stable-connectivity/draft-ietf-anima-stable-connectivity-
08.txt
* Co-author organization fix.
07: Fixed ID nits. 07: Fixed ID nits.
06: changed "split-horizon" term to "private-DNS" and reworded the 06: changed "split-horizon" term to "private-DNS" and reworded the
paragraph about it. paragraph about it.
05: Integrated fixes from Brian Carpenters review. See github 05: Integrated fixes from Brian Carpenters review. See github
draft-ietf-anima-stable-connectivity/04-brian-carpenter-review- draft-ietf-anima-stable-connectivity/04-brian-carpenter-review-
reply.txt. Details on semantic/structural changes: reply.txt. Details on semantic/structural changes:
* Folded most comments back into draft-ietf-anima-autonomic- * Folded most comments back into draft-ietf-anima-autonomic-
control-plane-09 because this stable connectivity draft was control-plane-09 because this stable connectivity draft was
suggesting things that are better written out and standardized suggesting things that are better written out and standardized
in the ACP document. in the ACP document.
* Section on simultaneous ACP and data plane connectivity section * Section on simultaneous ACP and data-plane connectivity section
reduced/rewritten because of this. reduced/rewritten because of this.
* Re-emphasized security model of ACP - ACP-connect can not * Re-emphasized security model of ACP - ACP-connect can not
arbitrarily extend ACP routing domain. arbitrarily extend ACP routing domain.
* Re-wrote much of NMS section to be less suggestive and more * Re-wrote much of NMS section to be less suggestive and more
descriptive, avoiding the term NAT and referring to relevant descriptive, avoiding the term NAT and referring to relevant
RFCs (SIIT etc.). RFCs (SIIT etc.).
* Main additional text in IPv4 section is about explaining how we * Main additional text in IPv4 section is about explaining how we
skipping to change at page 19, line 22 skipping to change at page 22, line 16
better anymore, but still mention it. better anymore, but still mention it.
individual-02: Added explanation why no IPv4 for ACP. individual-02: Added explanation why no IPv4 for ACP.
individual-01: Added security section discussing the role of individual-01: Added security section discussing the role of
address prefix selection and DNS for ACP. Title change to address prefix selection and DNS for ACP. Title change to
emphasize focus on OAM. Expanded abstract. emphasize focus on OAM. Expanded abstract.
individual-00: Initial version. individual-00: Initial version.
8. Informative References 8. References
8.1. Normative References
[RFC1918] Rekhter, Y., Moskowitz, B., Karrenberg, D., de Groot, G.,
and E. Lear, "Address Allocation for Private Internets",
BCP 5, RFC 1918, DOI 10.17487/RFC1918, February 1996,
<https://www.rfc-editor.org/info/rfc1918>.
[RFC4191] Draves, R. and D. Thaler, "Default Router Preferences and
More-Specific Routes", RFC 4191, DOI 10.17487/RFC4191,
November 2005, <https://www.rfc-editor.org/info/rfc4191>.
[RFC4193] Hinden, R. and B. Haberman, "Unique Local IPv6 Unicast
Addresses", RFC 4193, DOI 10.17487/RFC4193, October 2005,
<https://www.rfc-editor.org/info/rfc4193>.
[RFC6724] Thaler, D., Ed., Draves, R., Matsumoto, A., and T. Chown,
"Default Address Selection for Internet Protocol Version 6
(IPv6)", RFC 6724, DOI 10.17487/RFC6724, September 2012,
<https://www.rfc-editor.org/info/rfc6724>.
[RFC6824] Ford, A., Raiciu, C., Handley, M., and O. Bonaventure,
"TCP Extensions for Multipath Operation with Multiple
Addresses", RFC 6824, DOI 10.17487/RFC6824, January 2013,
<https://www.rfc-editor.org/info/rfc6824>.
[RFC7575] Behringer, M., Pritikin, M., Bjarnason, S., Clemm, A.,
Carpenter, B., Jiang, S., and L. Ciavaglia, "Autonomic
Networking: Definitions and Design Goals", RFC 7575,
DOI 10.17487/RFC7575, June 2015,
<https://www.rfc-editor.org/info/rfc7575>.
[RFC7757] Anderson, T. and A. Leiva Popper, "Explicit Address
Mappings for Stateless IP/ICMP Translation", RFC 7757,
DOI 10.17487/RFC7757, February 2016,
<https://www.rfc-editor.org/info/rfc7757>.
[RFC7915] Bao, C., Li, X., Baker, F., Anderson, T., and F. Gont,
"IP/ICMP Translation Algorithm", RFC 7915,
DOI 10.17487/RFC7915, June 2016,
<https://www.rfc-editor.org/info/rfc7915>.
8.2. Informative References
[I-D.ietf-anima-autonomic-control-plane] [I-D.ietf-anima-autonomic-control-plane]
Behringer, M., Eckert, T., and S. Bjarnason, "An Autonomic Eckert, T., Behringer, M., and S. Bjarnason, "An Autonomic
Control Plane (ACP)", draft-ietf-anima-autonomic-control- Control Plane (ACP)", draft-ietf-anima-autonomic-control-
plane-10 (work in progress), September 2017. plane-13 (work in progress), December 2017.
[I-D.ietf-anima-bootstrapping-keyinfra] [I-D.ietf-anima-bootstrapping-keyinfra]
Pritikin, M., Richardson, M., Behringer, M., Bjarnason, Pritikin, M., Richardson, M., Behringer, M., Bjarnason,
S., and K. Watsen, "Bootstrapping Remote Secure Key S., and K. Watsen, "Bootstrapping Remote Secure Key
Infrastructures (BRSKI)", draft-ietf-anima-bootstrapping- Infrastructures (BRSKI)", draft-ietf-anima-bootstrapping-
keyinfra-08 (work in progress), October 2017. keyinfra-09 (work in progress), October 2017.
[I-D.ietf-anima-grasp] [I-D.ietf-anima-grasp]
Bormann, C., Carpenter, B., and B. Liu, "A Generic Bormann, C., Carpenter, B., and B. Liu, "A Generic
Autonomic Signaling Protocol (GRASP)", draft-ietf-anima- Autonomic Signaling Protocol (GRASP)", draft-ietf-anima-
grasp-15 (work in progress), July 2017. grasp-15 (work in progress), July 2017.
[I-D.ietf-anima-reference-model] [I-D.ietf-anima-reference-model]
Behringer, M., Carpenter, B., Eckert, T., Ciavaglia, L., Behringer, M., Carpenter, B., Eckert, T., Ciavaglia, L.,
Pierre, P., Liu, B., Nobre, J., and J. Strassner, "A Pierre, P., Liu, B., Nobre, J., and J. Strassner, "A
Reference Model for Autonomic Networking", draft-ietf- Reference Model for Autonomic Networking", draft-ietf-
anima-reference-model-04 (work in progress), July 2017. anima-reference-model-05 (work in progress), October 2017.
[IEEE802.1Q] [IEEE802.1Q]
International Telecommunication Union, "802.1Q-2014 - IEEE International Telecommunication Union, "802.1Q-2014 - IEEE
Standard for Local and metropolitan area networks - Standard for Local and metropolitan area networks -
Bridges and Bridged Networks", 2014. Bridges and Bridged Networks", 2014.
[ITUT] International Telecommunication Union, "Architecture and [ITUT] International Telecommunication Union, "Architecture and
specification of data communication network", specification of data communication network",
ITU-T Recommendation G.7712/Y.1703, Noevember 2001. ITU-T Recommendation G.7712/Y.1703, Noevember 2001.
This is the earliest but superceeded version of the This is the earliest but superceeded version of the
series. See "https://www.itu.int/rec/T-REC-G.7712/en" for series. See "https://www.itu.int/rec/T-REC-G.7712/en" for
current versions. current versions.
[RFC1034] Mockapetris, P., "Domain names - concepts and facilities", [RFC1034] Mockapetris, P., "Domain names - concepts and facilities",
STD 13, RFC 1034, DOI 10.17487/RFC1034, November 1987, STD 13, RFC 1034, DOI 10.17487/RFC1034, November 1987,
<https://www.rfc-editor.org/info/rfc1034>. <https://www.rfc-editor.org/info/rfc1034>.
[RFC1918] Rekhter, Y., Moskowitz, B., Karrenberg, D., de Groot, G.,
and E. Lear, "Address Allocation for Private Internets",
BCP 5, RFC 1918, DOI 10.17487/RFC1918, February 1996,
<https://www.rfc-editor.org/info/rfc1918>.
[RFC4191] Draves, R. and D. Thaler, "Default Router Preferences and
More-Specific Routes", RFC 4191, DOI 10.17487/RFC4191,
November 2005, <https://www.rfc-editor.org/info/rfc4191>.
[RFC4193] Hinden, R. and B. Haberman, "Unique Local IPv6 Unicast
Addresses", RFC 4193, DOI 10.17487/RFC4193, October 2005,
<https://www.rfc-editor.org/info/rfc4193>.
[RFC5246] Dierks, T. and E. Rescorla, "The Transport Layer Security [RFC5246] Dierks, T. and E. Rescorla, "The Transport Layer Security
(TLS) Protocol Version 1.2", RFC 5246, (TLS) Protocol Version 1.2", RFC 5246,
DOI 10.17487/RFC5246, August 2008, DOI 10.17487/RFC5246, August 2008,
<https://www.rfc-editor.org/info/rfc5246>. <https://www.rfc-editor.org/info/rfc5246>.
[RFC6146] Bagnulo, M., Matthews, P., and I. van Beijnum, "Stateful [RFC6146] Bagnulo, M., Matthews, P., and I. van Beijnum, "Stateful
NAT64: Network Address and Protocol Translation from IPv6 NAT64: Network Address and Protocol Translation from IPv6
Clients to IPv4 Servers", RFC 6146, DOI 10.17487/RFC6146, Clients to IPv4 Servers", RFC 6146, DOI 10.17487/RFC6146,
April 2011, <https://www.rfc-editor.org/info/rfc6146>. April 2011, <https://www.rfc-editor.org/info/rfc6146>.
skipping to change at page 21, line 10 skipping to change at page 24, line 34
[RFC6434] Jankiewicz, E., Loughney, J., and T. Narten, "IPv6 Node [RFC6434] Jankiewicz, E., Loughney, J., and T. Narten, "IPv6 Node
Requirements", RFC 6434, DOI 10.17487/RFC6434, December Requirements", RFC 6434, DOI 10.17487/RFC6434, December
2011, <https://www.rfc-editor.org/info/rfc6434>. 2011, <https://www.rfc-editor.org/info/rfc6434>.
[RFC6518] Lebovitz, G. and M. Bhatia, "Keying and Authentication for [RFC6518] Lebovitz, G. and M. Bhatia, "Keying and Authentication for
Routing Protocols (KARP) Design Guidelines", RFC 6518, Routing Protocols (KARP) Design Guidelines", RFC 6518,
DOI 10.17487/RFC6518, February 2012, DOI 10.17487/RFC6518, February 2012,
<https://www.rfc-editor.org/info/rfc6518>. <https://www.rfc-editor.org/info/rfc6518>.
[RFC6724] Thaler, D., Ed., Draves, R., Matsumoto, A., and T. Chown,
"Default Address Selection for Internet Protocol Version 6
(IPv6)", RFC 6724, DOI 10.17487/RFC6724, September 2012,
<https://www.rfc-editor.org/info/rfc6724>.
[RFC6824] Ford, A., Raiciu, C., Handley, M., and O. Bonaventure,
"TCP Extensions for Multipath Operation with Multiple
Addresses", RFC 6824, DOI 10.17487/RFC6824, January 2013,
<https://www.rfc-editor.org/info/rfc6824>.
[RFC6950] Peterson, J., Kolkman, O., Tschofenig, H., and B. Aboba, [RFC6950] Peterson, J., Kolkman, O., Tschofenig, H., and B. Aboba,
"Architectural Considerations on Application Features in "Architectural Considerations on Application Features in
the DNS", RFC 6950, DOI 10.17487/RFC6950, October 2013, the DNS", RFC 6950, DOI 10.17487/RFC6950, October 2013,
<https://www.rfc-editor.org/info/rfc6950>. <https://www.rfc-editor.org/info/rfc6950>.
[RFC7575] Behringer, M., Pritikin, M., Bjarnason, S., Clemm, A.,
Carpenter, B., Jiang, S., and L. Ciavaglia, "Autonomic
Networking: Definitions and Design Goals", RFC 7575,
DOI 10.17487/RFC7575, June 2015,
<https://www.rfc-editor.org/info/rfc7575>.
[RFC7757] Anderson, T. and A. Leiva Popper, "Explicit Address
Mappings for Stateless IP/ICMP Translation", RFC 7757,
DOI 10.17487/RFC7757, February 2016,
<https://www.rfc-editor.org/info/rfc7757>.
[RFC7915] Bao, C., Li, X., Baker, F., Anderson, T., and F. Gont,
"IP/ICMP Translation Algorithm", RFC 7915,
DOI 10.17487/RFC7915, June 2016,
<https://www.rfc-editor.org/info/rfc7915>.
Authors' Addresses Authors' Addresses
Toerless Eckert (editor) Toerless Eckert (editor)
Futurewei Technologies Inc. Futurewei Technologies Inc.
2330 Central Expy 2330 Central Expy
Santa Clara 95050 Santa Clara 95050
USA USA
Email: tte+ietf@cs.fau.de Email: tte+ietf@cs.fau.de
Michael H. Behringer Michael H. Behringer
Email: michael.h.behringer@gmail.com Email: michael.h.behringer@gmail.com
 End of changes. 132 change blocks. 
364 lines changed or deleted 510 lines changed or added

This html diff was produced by rfcdiff 1.46. The latest version is available from http://tools.ietf.org/tools/rfcdiff/