ALTO WG                                                           K. Gao
Internet-Draft                                       Tsinghua University
Intended status: Standards Track                                  Y. Lee
Expires: September 12, December 20, 2019                                        Huawei
                                                          S. Randriamasy
                                                         Nokia Bell Labs
                                                                 Y. Yang
                                                         Yale University
                                                                J. Zhang
                                                       Tongji University
                                                          March 11,
                                                           June 18, 2019

                 ALTO Extension: Path Vector Cost Type
                     draft-ietf-alto-path-vector-05
                     draft-ietf-alto-path-vector-06

Abstract

   The Application-Layer Traffic Optimization (ALTO) protocol [RFC7285]
   has defined cost maps and endpoint cost maps to provide basic network
   information.  However, they provide only scalar (numerical or
   ordinal) cost mode values, which are insufficient to satisfy the
   demands of solving more complex network optimization problems.  This
   document introduces an extension to the base ALTO protocol, namely
   the path-vector extension, which allows ALTO clients to query
   information such as the capacity region for a given set of flows
   (called co-flows).  A non-normative example called co-flow scheduling
   is presented to illustrate the limitations of existing ALTO endpoint
   cost maps.  After that, details of the extension are defined.

Requirements Language

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in RFC 2119 [RFC2119].

Status of This Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at http://datatracker.ietf.org/drafts/current/. https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   This Internet-Draft will expire on September 12, December 20, 2019.

Copyright Notice

   Copyright (c) 2019 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info)
   (https://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   3
   2.  Terminology . . . . . . . . . . . . . . . . . . . . . . . . .   5
   3.  Use Case: Capacity Region for Co-Flow Scheduling  . . . . . .   5
   4.  Overview of Path Vector Extensions  . . . . . . . . . . . . .   7
     4.1.  New Cost Type Mode to Encode Path Vectors  . . . . . . . . . .   7
     4.2.  New ALTO Entity Domain to Provide for ANE Properties . . . . . . . .   8
     4.3.  Extended Cost Map/Endpoint Cost Service  Multipart/Related Resource for Compound
           Resources Consistency  . . . . . . .   8
   5.  Path-Vector Cost Type . . . . . . . . . . . . . . . . .   8
   5. . . .   9
     5.1.  Cost Type Mode: path-vector  . . . . . . . . . . . . . . . . .  10
     5.2.  Cost Metric: Link Maximum Reservable Bandwidth  . . . . .  10
   6.  ANE Domain  . . . .   8
     5.1.  Cost Mode: array . . . . . . . . . . . . . . . . . . . .   9
     5.2.  Cost Metric: ane-path .  10
     6.1.  Domain Name . . . . . . . . . . . . . . . . .   9
     5.3.  Path Vector Cost Type Semantics . . . . . .  11
     6.2.  Domain-Specific Entity Identifier . . . . . . .   9
   6.  ANE Domain . . . . .  11
     6.3.  Hierarchy and Inheritance . . . . . . . . . . . . . . . .  11
   7.  Multipart Filtered Cost Map for Path Vector . . . .  10
     6.1.  Domain Name . . . . .  11
     7.1.  Media Type  . . . . . . . . . . . . . . . . . .  10
     6.2.  Domain-Specific Entity Addresses . . . . .  11
     7.2.  HTTP Method . . . . . . .  10
     6.3.  Hierarchy and Inheritance . . . . . . . . . . . . . . . .  10
   7.  Protocol Extensions for Path Vector  11
     7.3.  Accept Input Parameters . . . . . . . . . . . . .  10
     7.1.  Filtered Cost Map Extensions . . . .  12
     7.4.  Capabilities  . . . . . . . . . .  11
       7.1.1.  Accept Input Parameters . . . . . . . . . . . .  12
     7.5.  Uses  . . .  11
       7.1.2.  Capabilities . . . . . . . . . . . . . . . . . . . .  11
       7.1.3. . . .  12
     7.6.  Response  . . . . . . . . . . . . . . . . . . . . . . . .  12
     7.2.
   8.  Multipart Endpoint Cost Service Extensions for Path Vector . . . . . . .  13
     8.1.  Media Type  . . . . . . . .  12
       7.2.1.  Accept Input Parameters . . . . . . . . . . . . . . .  13
       7.2.2.  Capabilities
     8.2.  HTTP Method . . . . . . . . . . . . . . . . . . . . . . .  13
       7.2.3.  Response
     8.3.  Accept Input Parameters . . . . . . . . . . . . . . . . .  13
     8.4.  Capabilities  . . . . .  13
   8.  Examples . . . . . . . . . . . . . . . . .  13
     8.5.  Uses  . . . . . . . . .  13
     8.1.  Workflow . . . . . . . . . . . . . . . . .  14
     8.6.  Response  . . . . . . . . . . . . . . .  13
     8.2.  Information Resource Directory Example . . . . . . . . .  14
     8.3.  Example # 1
   9.  Examples  . . . . . . . . . . . . . . . . . . . . . . .  16
     8.4. . . .  14
     9.1.  Information Resource Directory Example # 2  . . . . . . . . .  14
     9.2.  Example #1  . . . . . . . . . . . . . . .  18
     8.5. . . . . . . . .  16
     9.3.  Example #3 #2  . . . . . . . . . . . . . . . . . . . . . . .  20
   9.  17
     9.4.  Example for Incremental Update  . . . . . . . . . . . . .  19
   10. Compatibility . . . . . . . . . . . . . . . . . . . . . . . .  22
     9.1.  20
     10.1.  Compatibility with Base ALTO Clients/Servers . . . . . .  22
     9.2.  20
     10.2.  Compatibility with Multi-Cost Extension  . . . . . . . . .  23
     9.3.  21
     10.3.  Compatibility with Incremental Update  . . . . . . . . . .  23
   10.  21
   11. General Discussions . . . . . . . . . . . . . . . . . . . . .  23
     10.1.  21
     11.1.  Provide Calendar for Property Map  . . . . . . . . . . .  23
     10.2.  21
     11.2.  Constraint Tests for General Cost Types  . . . . . . . .  24
     10.3.  22
     11.3.  General Compound Multipart Resources Query  . . . . . . . . . . . .  24
   11.  22
   12. Security Considerations . . . . . . . . . . . . . . . . . . .  24
   12.  22
   13. IANA Considerations . . . . . . . . . . . . . . . . . . . . .  25
     12.1.  23
     13.1.  ALTO Cost Mode Registry  . . . . . . . . . . . . . . . .  25
     12.2.  23
     13.2.  ALTO Cost Metric Entity Domain Registry  . . . . . . . . . . . . . . .  26
     12.3.  23
     13.3.  ALTO Entity Domain Property Type Registry  . . . . . . . . . . . . . .  26
     12.4.  ALTO Network Element Property Type Registry  . . . . . .  26
   13.  24
   14. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . .  26
   14.  24
   15. References  . . . . . . . . . . . . . . . . . . . . . . . . .  27
     14.1.  24
     15.1.  Normative References . . . . . . . . . . . . . . . . . .  27
     14.2.  24
     15.2.  Informative References . . . . . . . . . . . . . . . . .  27  25
   Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  27  25

1.  Introduction

   The base ALTO protocol [RFC7285] is designed to expose network
   information through services such as cost maps and endpoint cost
   service.  These services use an extreme "single-node" network
   abstraction, which represents a whole network as a single node node, and
   hosts as "endpoint groups" directly connected to the node.

   Although the "single-node" abstraction works well in many settings,
   it lacks the ability to support emerging use cases, such as co-flow
   scheduling for large-scale data analytics.  For such a use case,
   applications require a more powerful network view abstraction beyond
   the "single-node" abstraction.

   To support capabilities like co-flow scheduling, this document uses a
   "path vector" abstraction to represent more detailed network graph
   information like capacity regions.  The path vector abstraction uses
   path vectors with abstract network elements to provide network graph
   view for applications.  A path vector consists of is a sequence of
   abstract network elements (ANEs) (ANEs), and each ANE represents a network
   device that end-to-end traffic goes
   through.  Example ANEs include through, such as links, switches,
   middleboxes, and their aggregations.  An ANE can have properties such
   as "bandwidth", and "delay".  Providing such information can help
   both applications to achieve better application performance and
   networks to avoid network congestion.

   Providing path vector abstraction using ALTO introduces the following
   additional requirements (ARs):

   AR-1:  The path vector abstraction requires the encoding of array-
      like cost values rather than scalar cost values in cost maps or
      endpoint cost maps.

      Specifically, the path vector abstraction requires the
      specification of the sequence of ANEs between sources and
      destinations.  Such a sequence, however, cannot be encoded by the
      scalar types (numerical or ordinal) which the base ALTO protocol
      supports.

   AR-2:  The path vector abstraction requires the encoding of the
      properties of aforementioned ANEs.

      Specifically, only the sequences of ANEs are not enough for
      existing use cases.  Properties of ANEs such as "bandwidth" and
      "delay" are needed by applications to properly construct capacity
      regions. network
      constraints or states.

   AR-3:  The path vector abstraction requires consistent encoding of
      path vectors (AR-1) and the properties of the elements ANEs in a path
      vector (AR-2).

      Specifically, path vectors and the properties of abstract network
      elements ANEs in the
      vectors are dependent.  A mechanism to query both of them
      consistently is necessary.

   This document proposes the path vector extension which satisfies
   these additional requirements to the ALTO protocol. protocol
   to satisfy these additional requirements .

   Specifically, the ALTO protocol extension encodes the array (AR-1) of ANEs over an
   end-to-end path using a new cost type, and conveys the properties of
   ANEs (AR-2) using unified property map
   [I-D.ietf-alto-unified-props-new].  We also
   provide an optional solution to query separated  The path vectors vector and ANE
   properties of ANEs are conveyed in a consistent way.  But querying general
   separated resources consistently is not the scope in this document. single message encoded as a multipart/
   related message to satisfy AR-3.

   The rest of this document is organized as follows.  Section 3 gives
   an example of co-flow scheduling and illustrates the limitations of
   the base ALTO protocol in such a use case.  Section 4 gives an
   overview of the path vector extension.  Section 5 introduces a new
   cost type.  Section 6 registers a new domain in Domain Registry.
   Section 7 extends and Section 8 define new ALTO resources to support Path
   Vector query by using the request format of Filtered Cost Map and
   Endpoint Cost Service to
   support the compound resource query. Service.  Section 8 9 presents several examples.
   Section 9 10 and Section 10 11 discusses compatibility issues with other
   existing ALTO extensions and design decisions.  Section 11 12 and
   Section 12 13 review the security and IANA considerations.

2.  Terminology

   Besides the terms defined in [RFC7285] and
   [I-D.ietf-alto-unified-props-new], this document also uses the
   following additional terms: Abstract Network Element, Element and Path Vector.

   o  Abstract Network Element (ANE): An abstract network element is an
      abstraction of network components; it components.  It can be an aggregation of
      links, middle boxes, middleboxes, virtualized network function (VNF), etc.  An
      abstract network element has two types of attributes: a name and a
      set of properties.

   o  Path Vector: A path vector is an array of ANEs.  It presents an
      abstract network path between source/destination points such as
      PIDs or endpoints.

3.  Use Case: Capacity Region for Co-Flow Scheduling

   Assume that an application has control over a set of flows, which may
   go through shared links or switches and share a bottleneck.  The
   application hopes to schedule the traffic among multiple flows to get
   better performance.  The capacity region information for those flows
   will benefit the scheduling.  However, existing cost maps can not cannot
   reveal such information.

   Specifically, consider a network as shown in Figure 1.  The network
   has 7 switches (sw1 to sw7) forming a dumb-bell topology.  Switches
   sw1/sw3 provide access on one side, sw2/sw4 provide access on the
   other side, and sw5-sw7 form the backbone.  Endhosts eh1 to eh4 are
   connected to access switches sw1 to sw4 respectively.  Assume that
   the bandwidth of link eh1 -> sw1 and link sw1 -> sw5 are 150 Mbps,
   and the bandwidth of the rest all links are 100 Mbps.

                                  +------+
                                  |      |
                                --+ sw6  +--
                              /   |      |  \
        PID1 +-----+         /    +------+   \          +-----+  PID2
        eh1__|     |_       /                 \     ____|     |__eh2
             | sw1 | \   +--|---+         +---|--+ /    | sw2 |
             +-----+  \  |      |         |      |/     +-----+
                       \_| sw5  +---------+ sw7  |
        PID3 +-----+   / |      |         |      |\     +-----+  PID4
        eh3__|     |__/  +------+         +------+ \____|     |__eh4
             | sw3 |                                    | sw4 |
             +-----+                                    +-----+

                      Figure 1: Raw Network Topology.

   The single-node ALTO topology abstraction of the network is shown in
   Figure 2.

                          +----------------------+
                 {eh1}    |                      |     {eh2}
                 PID1     |                      |     PID2
                   +------+                      +------+
                          |                      |
                          |                      |
                 {eh3}    |                      |     {eh4}
                 PID3     |                      |     PID4
                   +------+                      +------+
                          |                      |
                          +----------------------+

             Figure 2: Base Single-Node Topology Abstraction.

   Consider an application overlay (e.g., a large data analysis system)
   which wants to schedule the traffic among a set of end host source-
   destination pairs, say eh1 -> eh2 and eh1 eh3 -> eh4.  The application
   can request a cost map providing end-to-end available bandwidth,
   using 'availbw' "availbw" as cost-metric and 'numerical' "numerical" as cost-mode.

   The application will receive from ALTO server that the bandwidth of
   eh1 -> eh2 and eh1 eh3 -> eh4 are both 100 Mbps.  But this information is
   not enough.  Consider the following two cases:

   o  Case 1: If eh1 -> eh2 uses the path eh1 -> sw1 -> sw5 -> sw6 ->
      sw7 -> sw2 -> eh2 and eh1 eh3 -> eh4 uses path eh1 eh3 -> sw1 sw3 -> sw5 ->
      sw7 -> sw4 -> eh4, then the application will obtain 150 Mbps at
      most. 200 Mbps.

   o  Case 2: If eh1 -> eh2 uses the path eh1 -> sw1 -> sw5 -> sw7 ->
      sw2 -> eh2 and eh1 eh3 -> eh4 uses the path eh1 eh3 -> sw1 sw3 -> sw5 -> sw7
      -> sw4 -> eh4, then the application will obtain only 100 Mbps at
      most. due
      to the shared link from sw5 to sw7.

   To allow applications to distinguish the two aforementioned cases,
   the network needs to provide more details.  In particular:

   o  The network needs to expose more detailed routing information to
      show the shared bottlenecks. bottlenecks;

   o  The network needs to provide the necessary abstraction to hide the
      real topology information while providing enough information to
      applications.

   The path vector extension defined in this document propose provides a
   solution to provide these details. address the preceding issue.

   See [I-D.bernstein-alto-topo] for a more comprehensive survey of use
   cases where extended network topology information is needed.

4.  Overview of Path Vector Extensions

   This section presents an overview of approaches adopted by the path
   vector extension.  It assumes that the readers are familiar with cost
   map and endpoint cost service defined in [RFC7285].  The path vector
   extension also requires the support of Filtered Property Map defined
   in [I-D.ietf-alto-unified-props-new].

   The path vector extension is composed of three building blocks: (1) a
   new cost type mode to encode path vectors; vectors in a cost map or an endpoint
   cost map; (2) a new ALTO entity domain
   for to enable ANE property
   encoding using the unified property extension [I-D.ietf-alto-unified-props-new] to
   encode properties of ANEs;
   [I-D.ietf-alto-unified-props-new]; and (3) an extension to the cost map and
   endpoint cost resource a generic mechanism to provide path vectors and properties of ANEs put
   multiple ALTO information objects in a single response. response to enforce
   consistency, to preserve modularity and to avoid complex linking of
   multiple responses.

4.1.  New Cost Type Mode to Encode Path Vectors

   Existing cost types modes defined in [RFC7285] allow only scalar cost
   values.  However, the "path vector" abstraction requires to convey
   vector format information. information (AR-1).  To achieve fulfill this requirement, this
   document defines a new cost mode "cost-mode" named path vector to enable indicate that
   the cost value to carry an
   array of elements, and a new cost metric to take names of ANEs as
   elements in the array.  We call such is an array of ANEs a ANEs.  A path vector.
   In vector abstraction should
   be computed for a specific performance metric, and this way, is achieved
   using the existing "cost-metric" component of cost map and endpoint cost service can convey the
   path vector to represent type.  The details
   of the routing information.  Detailed
   information and specifications are new "cost-mode" is given in Section 5.1 and
   Section 5.2. 5.

4.2.  New ALTO Entity Domain to Provide for ANE Properties

   The

   A path vector can of ANEs contains only represent the route abstracted routing elements
   between the a source and
   the a destination.  Although the  Hence, an application can find the
   shared ANEs
   among of different paths, it is not enough for most source-destination pairs but cannot know the
   shared ANEs' properties.  For the capacity region use cases, which
   requires case in
   Section 3, knowing that eh1->eh2 and eh3->eh4 share ANEs but not the
   available bandwidth or delay information of the ANEs.  So shared ANEs, is not enough.

   To encode ANE properties like the available bandwidth in a path
   vector query response, this document adopts uses the unified property map
   extension defined in
   [I-D.ietf-alto-unified-props-new] [I-D.ietf-alto-unified-props-new].
   Specifically, for each path vector query, the ALTO server generates a
   property map associated to provide the general properties
   of ANEs.  The document registers (endpoint) cost map as follows:

   o  a new dynamic entity domain called "ane" to
   represent the ANE.  The address of the ANE an entity domain type "ane" is just
      generated to contain the generated ANEs.  Each ANE
   name used by the path vector.  By requesting has the property map of
   entities same
      unique identifier in the "ane" domain, path vectors and in the client can retrieve dynamic entity
      domain;

   o  each entity in this dynamic entity domain has the property defined
      by the "cost-metric" that generated the properties
   of ANEs in path vectors.

4.3.  Extended Cost Map/Endpoint Cost Service for Compound Resources

   Providing the path vector query.

   Detailed information and specifications are given in Section 6.

4.3.  Multipart/Related Resource for Consistency

   Path vectors and the ANE properties by
   separated resources have several known benefits: (1) can be better
   compatible with the base ALTO protocol; (2) can make different property map resources reuse containing the same cost map or endpoint cost
   resource.  However, it conducts ANEs are two issues:

   o  Efficiency: The separated resources will
   different types of objects, but they require strong consistency.  One
   approach to achieving strong consistency is to define a new media
   type to contain both objects, but this violates modular design.

   Another approach is to provide the objects in two different
   information resources.  Thus, an ALTO client needs to invoke multiple requests/responses make separate
   queries to collect all needed
      information.  It increases get the communication overhead.

   o  Consistency: The path vectors and properties information of ANEs are
      correlated.  So querying them one by one related services.  This may conduct consistency
      issue.  Once cause a
   data synchronization problem between two queries.  Also, as the path vector changes during
   generation of ANE is dynamic, an ALTO server must cache the results
   of a query before a client requests fully retrieves all related resources,
   which hurts the ANE properties, scalability and security of an ALTO server.

   This document uses standard-conforming usage of "multipart/related"
   media type defined in [RFC2387] to elegantly solve the ANE properties may be inconsistent with problem.

   Specifically, using "multipart/related" needs to address two issues:

   o  ALTO uses media type to indicate the previous path vector.

   To solve these issues, this document introduces type of an extension to entry in the
      information resource directory (IRD) (e.g., "application/alto-
      costmap+json" for cost map and "application/alto-
      endpointcostmap+json" for endpoint cost service, which allows map).  Simply putting
      "multipart/related" as the media type, however, makes it
      impossible for an ALTO server client to attach
   a property map in identify the data entry type of a cost map or an endpoint cost service response.

   These issues may exist in all general cases for querying separated
   ALTO information resources.  But solving this general problem
      provided by related entries.

   o  The ALTO SSE extension (see [I-D.ietf-alto-incr-update-sse])
      depends on resource-id to identify push updates, but resource-id
      is not provided only in IRD and hence each entry in the scope IRD has only
      one resource-id.

   This design addresses the two issues as follows:

   o  To address the first issue, the multipart/related media type
      includes the type parameter to allow type indication of this document. the root
      object.  For a cost map service, the "media-type" will be
      "multipart/related" with the parameter "type=application/alto-
      costmap+json"; for an endpoint cost map service, the parameter
      will be "type=application/alto-endpointcostmap+json".  This design
      is highly extensible.  The entries can still use "application/
      alto-costmapfilter+json" or "application/alto-
      endpointcostparams+json" as the accept input parameters, and hence
      an ALTO client still sends the filtered cost map request or
      endpoint cost service request.  The ALTO server sends the response
      as a "multipart/related" message.  The body of the response
      includes two parts: the first one is of the media type specified
      by the "type" parameter; the second one is a property map
      associated to the first map.

   o  To address the second issue, each part of the "multipart/related"
      response message has the MIME part header information including
      "Content-Type" and "Resource-Id".  An ALTO server MAY generate
      incremental updates (see [I-D.ietf-alto-incr-update-sse]) for each
      part separately using the "Resource-Id" header.

   By applying the design above, for each path vector query, an ALTO
   server returns the path vectors and the associated property map
   modularly and consistently.  An ALTO server can reuse the data models
   of the existing information resources.  And an ALTO client can
   subscribe to the incremental updates for the dynamic generated
   information resources without any changes, if th ALTO server provides
   incremental updates for them.

5.  Path-Vector Cost Type

   This document extends the cost types defined in Section 6.1 of
   [RFC7285] by introducing a new cost mode "array" and a new cost
   metric "ane-path". "path-vector".  In the rest content, this document uses "path-
   vector"
   of the document, we use "path-vector" to indicate the combination cost type of the cost mode
   "array" and with
   the cost metric "ane-path". cost-mode "path-vector" for short.

5.1.  Cost Mode: array path-vector

   This document extends the CostMode defined in Section 10.5 of
   [RFC7285] with a new cost mode: "array". "path-vector".  This cost mode
   indicates that every cost value in a cost map represents an array of
   ANEs which are defined in Section 6.2, rather than a
   simple value.  The values are arrays of JSONValue. JSON number or a
   ranking order.

   The specific type
   of each element in ANEs computed by the array depends on ALTO server associate to the cost metric.

5.2.  Cost Metric: ane-path

   This document specifies a new cost metric: "ane-path".  This cost metric indicates that for
   the "path-vector" cost value is a list of ANEs which the path
   from a source to a destination goes across.  The values are arrays of
   ANE names mode.  This document re-defines some cost
   metrics for "path-vector", which are defined in Section 6.2. motivated by the co-flow
   scheduling use case.  The cost metric "ane-path" ALTO client SHOULD NOT be used when ignore the "path-vector"
   cost mode is
   not "array" with any other cost metrics, unless it is explicitly specified by a the future extension.
   If an ALTO client send queries with documents
   define other cost metrics or specify the semantics of existing cost metric "ane-path" and a
   non "array"
   metrics for "path-vector" cost mode, the ALTO server SHOULD return an error with
   the error code "E_INVALID_FIELD_VALUE"; If an ALTO server declares mode for some additional requirements.

5.2.  Cost Metric: Link Maximum Reservable Bandwidth

   This document uses the support same metric name, units of a cost type measurement and
   measurement point(s) with the cost potential measurement domain defined by
   section 4.1 of [I-D.ietf-alto-performance-metrics], but specifies
   different metric "ane-path" description and a non
   "array" method of measurement or calculation
   for "path-vector" cost mode, the ALTO client SHOULD assume such a mode only.

   Metric Description:  When used with "path-vector" cost type mode, it is
   invalid to
      specify the path vector computed by using the spatial and ignore it.

5.3.  Path Vector Cost Type Semantics temporal
      maximum reservable bandwidth over each network link.  The new cost type follows value of
      the convention maximum reservable bandwidth of each ANE in the cost types path vector is
      specified in the
   base ALTO protocol.  Table 1 lists some associated property map.

   Method of Measurement or Calculation:  The value of Maximum
      Reservable Bandwidth is the current defined cost
   types and their semantics.

   +------------+--------------+---------------------------------------+
   | Cost Mode  | Cost Metric  | Semantics                             |
   +------------+--------------+---------------------------------------+
   | numerical  | routingcost  | a number representing the routing     |
   |            |              | cost                                  |
   | numerical  | hopcount     | a number representing the hop count   |
   | ordinal    | routingcost  | a ranking representing the routing    |
   |            |              | cost                                  |
   | ordinal    | hopcount     | a ranking representing the hop count  |
   | array      | ane-path     | a list representing the ane path      |
   +------------+--------------+---------------------------------------+

                  Table 1: Cost Types and Their Semantics

   The "routingcost" and "hopcount" can encoded in "numerical" bandwidth measured between two
      directly connected IS-IS neighbors, OSPF neighbors or
   "ordinal", however, the cost metric "ane-path" can only be applied to
   the cost mode "array" defined in this document to convey path vector
   information. BGP
      neighbors.  The cost metric "ane-path" associated ANEs are computed by some algorithm
      which can not be used in
   "numerical" or "ordinal" unless it is defined in future extensions.
   If the ALTO server declares that it support cost type with cost
   metric being "ane-path" and cost mode not being "array", guarantee the ALTO
   client SHOULD ignore them. equivalent Maximum Reservable Bandwidth
      constraints.

6.  ANE Domain

   This document specifies a new ALTO entity domain called "ane" in
   addition to the ones in [I-D.ietf-alto-unified-props-new].  The ANE
   domain associates property values with the ANEs in a network.  The
   entity in ANE domain is often used in the path vector by cost maps or
   endpoint cost resources.  Accordingly, the ANE domain always depends
   on a cost map or an endpoint cost map.

6.1.  Domain Name

   ane

6.2.  Domain-Specific Entity Addresses Identifier

   The entity address identifier of ane domain is encoded as a JSON string.  The
   string MUST be no more than 64 characters, and it MUST NOT contain
   characters other than US-ASCII alphanumeric characters
   (U+0030-U+0039, U+0041-U+005A, and U+0061-U+007A), the hyphen ("-",
   U+002D), the colon (":", U+003A), the at sign ("@", code point
   U+0040), the low line ("_", U+005F), or the "." separator (U+002E).
   The "." separator is reserved for future use and MUST NOT be used
   unless specifically indicated in this document, or an extension
   document.

   To simplify the description, we use "ANE name" to indicate the
   address
   identifier of an entity in ANE domain in this document.

   The ANE name is usually unrelated to the physical device information.
   It is usually generated by the ALTO server on demand and used to
   distinguish from other ANEs in its dependent cost map or endpoint
   cost map.

6.3.  Hierarchy and Inheritance

   There is no hierarchy or inheritance for properties associated with
   ANEs.

7.  Protocol Extensions  Multipart Filtered Cost Map for Path Vector

   To make the ALTO client query the path vectors and properties of ANEs
   efficiently and consistently, this

   This document extends the introduces a new ALTO resource called Multipart
   Filtered Cost Map and Endpoint Cost Service.

7.1.  Filtered resource, which allows an ALTO server to provide
   other ALTO resources associated to the Cost Map Extensions

   This document extends Filtered Cost Map, as defined resource in Section 11.3.2
   of [RFC7285], by adding new input parameters and capabilities, and by
   augmenting the property map into the data entry of the same
   response.

7.1.  Media Type

   The "media type", "HTTP method", and "uses" specifications (described
   in Sections 11.3.2.1, 11.3.2.2, and 11.3.2.5 media type of [RFC7285],
   respectively) remain the same.

7.1.1. Multipart Filtered Cost Map Resource is
   "multipart/related;type=application/alto-costmap+json".

7.2.  HTTP Method

   The Multipart Filtered Cost Map is requested using the HTTP POST
   method.

7.3.  Accept Input Parameters

   The ReqFilteredCostMap object in Section 11.3.2.3 input parameters of [RFC7285] is
   extended the Multipart Filtered Cost Map MUST be
   encoded as follows: a JSON object {
       [PropertyName compound-properties<1..*>;]
     } ReqPVFilteredCostMap : ReqFilteredCostMap;

   compound-properties:  If the capability "allow-compound-response" is
      false, in the ALTO client MUST NOT specify this field, and body of an HTTP POST request.  The
   media type of the ALTO
      server request MUST reject be one of "application/alto-
   costmapfilter+json".  The format of the request and return "E_INVALID_FILED_VALUE"
      error when it receives a request including this field.  If this
      field is specified and accepted, the ALTO server body MUST augment the
      dependent property map with the properties in this field into be the
      response automatically.

7.1.2. same
   type as defined by section 11.3.2.3 of [RFC7285].

7.4.  Capabilities

   The Multipart Filtered Cost Map capabilities are extended with two new members:

   o  dependent-property-map

   o  allow-compound-response

   The capability "dependent-property-map" indicates which property map
   this resource depends on, and the capability "allow-compound-
   response" indicates whether the ALTO server supports the resource to
   compound the property map with its own response data.  With these two
   additional members, uses the FilteredCostMapCapabilities object in
   Section same capabilities
   as defined by section 11.3.2.4 of [RFC7285] is extended as follows:

     object {
       [ResourceID dependent-property-map;]
       [JSONBool   allow-compound-response;]
     } PVFCMCapabilities : FilteredCostMapCapabilities;

   dependent-property-map:  This field MUST be specified when [RFC7285].  But the "cost-
      type-names" "cost-type-
   names" field SHOULD only includes a cost type name indicating a "ane-path"
      metric.  Its value MUST be a resource id indicating a property map
      including "ane" domain.  If not, types in "path-vector" cost
   mode.  Otherwise, the ALTO client SHOULD consider
      this resource is invalid.

   allow-compound-response:  If present, the true value means ignore a cost type in other
   cost mode, unless additional documents define the ALTO
      client can request specification of it
   for the Multipart Filtered Cost Map resource.

7.5.  Uses

   The resource to augment its dependent property
      map into ID of the response automatically; network map based on which the false value means PIDs in the
   returned cost map will be defined.

7.6.  Response

   The response MUST indicate an error, using ALTO client cannot request protocol error
   handling, as defined in Section 8.5 of [RFC7285], if the compound response.  If omitted, request is
   invalid.

   The response to a valid request MUST be a "multipart/related" message
   as defined by [RFC2387].  The body consists of two parts:

   o  the
      default first part MUST include "Resource-Id" and "Content-Type" in
      its header.  The value is false;

   To of "Resource-Id" MUST be noticed that prefixed by the capability "cost-constraints" is unexpected
   for
      resource id of the "array" cost mode. Multipart Filtered Cost Map appended by a "."
      character.  The syntax and semantics body of constraint
   tests on the "array" cost mode depends on the implementation and can this part MUST be a JSON object with the
      same format as defined in Section 11.2.3.6 of [RFC7285]; The JSON
      object MUST include the future documents.  But it is not "vtag" field in the scope "meta" field, which
      provides the version tag of
   this document.

7.1.3.  Response

   If the ALTO client specifies returned cost map.  The resource
      id of the "cost-type" input parameter with
   "ane-path" metric, version tag MUST be as same as the "dependent-vtags" field in value of the
      "Resource-Id" header.  The "meta" field of
   the response MUST also include the
      "dependent-vtags" field, whose value is a single-element array to
      indicate the version tag of its dependent property
   map following its dependent network map.

   If the ALTO client specifies network map used, where the "compound-properties" input
   parameter which
      network map is accepted by specified in the ALTO server, "uses" attribute of the response Multipart
      Cost Map resource in IRD.

   o  the second part MUST also include a "property-map" field following the "cost-map" field, "Resource-Id" and "Content-Type"
      in its header.  The value of "Resource-Id" MUST be a PropertyMapData object.  This PropertyMapData
   object MUST be equivalent to the result when query prefixed by the dependent
   property map
      resource using the following request: id of the "entities"
   field includes all Multipart Filtered Cost Map appended by a "."
      character.  The body of this part MUST be a JSON object with the ANE names appearing
      same format as defined in the cost values Section 4.6 of
      [I-D.ietf-alto-unified-props-new].  The JSON object MUST include
      the
   "cost-map" field, the "properties" "dependent-vtags" field has in the same "meta" field.  The value as of the
   "compound-properties"
      "dependent-vtags" field does. MUST be an array with a single VersionTag
      object as defined by section 10.3 of [RFC7285].  The properties shown in "resource-id"
      of this VersionTag MUST be the
   "compound-properties" input parameter but are not supported by value of "Resource-Id" header of
      the
   dependent property map SHOULD first part.  The "tag" of this VersionTag MUST be omitted from the response.

7.2. "tag" of
      "vtag" of the first part body.

8.  Multipart Endpoint Cost Service Extensions for Path Vector

   This document extends introduces a new ALTO resource called Multipart
   Endpoint Cost resource, which allows an ALTO server to provide other
   ALTO resources associated to the Endpoint Cost Service, as defined resource in
   Section 11.5.1 of [RFC7285], by adding new input parameters and
   capabilities and by augmenting the property map into the data entry
   of the same
   response.

8.1.  Media Type

   The media type, HTTP method, and "uses" specifications (described in
   Sections 11.5.1.1, 11.5.1.2, and 11.5.1.5 type of [RFC7285], respectively)
   are unchanged.

7.2.1. the Multipart Endpoint Cost Resource is
   "multipart/related;type=application/alto-endpointcostmap+json".

8.2.  HTTP Method

   The Multipart Endpoint Cost resource is requested using the HTTP POST
   method.

8.3.  Accept Input Parameters

   The ReqEndpointCostMap object in Section 11.5.1.3 input parameters of [RFC7285] is
   extended the Multipart Endpoint Cost resource MUST be
   encoded as follows: a JSON object {
       [PropertyName compound-properties<1..*>;]
     } ReqPVEndpointCostMap : ReqEndpointCostMap; in the body of an HTTP POST request.  The "compound-properties" has
   media type of the request MUST be one of "application/alto-
   endpointcostparams+json".  The format of the request body MUST be the
   same interpretation type as defined in
   Section 7.1.1.

7.2.2. by section 11.5.1.3 of [RFC7285].

8.4.  Capabilities

   The extensions to the Multipart Endpoint Cost Service capabilities are
   identical to resource uses the extensions to same capabilities as
   defined by section 11.3.2.4 of [RFC7285].  But the Filtered Cost Map (see
   Section 7.1.2).

7.2.3.  Response

   If "cost-type-names"
   field SHOULD only includes cost types in "path-vector" cost mode.
   Otherwise, the ALTO client specifies the "cost-type" input parameter with
   "ane-path" metric, the response MUST include the "meta" field with
   the "dependent-vtags" SHOULD ignore a cost type in it, and the "dependent-vtags" field MUST
   include other cost
   mode, unless additional documents define the version tag specification of its dependent property map.

   If the ALTO client specifies it for
   the "compound-properties" input
   parameter which is accepted by Multipart Endpoint Cost resource.

8.5.  Uses

   The Multipart Endpoint Cost resource MUST NOT specify the "uses"
   attribute.

8.6.  Response

   The response MUST indicate an error, using ALTO server, protocol error
   handling, as defined in Section 8.5 of [RFC7285], if the request is
   invalid.

   The response to a valid request MUST
   include be a "property-map" field following "multipart/related" message
   as defined by [RFC2387].  The body consists of two parts:

   o  the "endpoint-cost-map"
   field, first part MUST include "Resource-Id" and "Content-Type" in
      its header.  The value of "Resource-Id" MUST be a PropertyMapData object.  This
   PropertyMapData object MUST be equivalent to the result when query prefixed by the dependent property map
      resource using the following request: the
   "entities" field includes all the ANE names appearing in the cost
   values id of the "endpoint-cost-map" field, the "properties" field has Multipart Filtered Cost Map appended by a "."
      character (U+002E).  The body of this part MUST be a JSON object
      with the same value format as defined in Section 11.5.1.6 of [RFC7285];
      The JSON object MUST include the "compound-properties" "vtag" field does.  The
   properties shown in the "compound-properties" input parameter but are
   not supported by the dependent property map SHOULD be omitted from "meta" field,
      which provides the response.

8.  Examples

   This section lists some examples version tag of path vector queries and the
   corresponding responses.

8.1.  Workflow

   This section gives a typical workflow returned endpoint cost map.
      The resource id of how an ALTO client query
   path vectors using the extension.

   1.  Send a GET request for the whole Information Resource Directory.

   2.  Look for version tag MUST be as same as the resource value of
      the (Filtered) Cost Map/Endpoint Cost
       Service which supports "Resource-Id" header.

   o  the "ane-path" cost metric second part MUST also include "Resource-Id" and get the
       resource ID "Content-Type"
      in its header.  The value of "Resource-Id" MUST be prefixed by the dependent property map.

   3.  Check whether the capabilities
      resource id of the property map includes the
       desired "prop-types".

   4.  Check whether the (Filtered) Cost Map/Endpoint Multipart Filtered Cost Service
       allows the compound response.

   5.  If allowed, the ALTO client can send Map appended by a request including the
       desired ANE properties to the ALTO server and receive "."
      character (U+002E).  The body of this part MUST be a compound
       response JSON object
      with the cost map/endpoint cost map and same format as defined in Section 4.6 of
      [I-D.ietf-alto-unified-props-new].  The JSON object MUST include
      the property
       map.

   6.  If not allowed, "dependent-vtags" field in the ALTO client sends a query for "meta" field.  The value of the cost map/
       endpoint cost map first.  After receiving
      "dependent-vtags" field MUST be an array with a single VersionTag
      object as defined by section 10.3 of [RFC7285].  The "resource-id"
      of this VersionTag MUST be the response, value of "Resource-Id" header of
      the ALTO
       client interprets all first part.  The "tag" of this VersionTag MUST be the ANE names appearing in "tag" of
      "vtag" of the response first part body.

9.  Examples

   This section lists some examples of path vector queries and
       sends another query for the property map on those ANE names.

8.2.
   corresponding responses.

9.1.  Information Resource Directory Example

   Here is an example of an Information Resource Directory.  In this
   example, filtered cost map the "cost-map-pv" doesn't support information resource provides a Multipart
   Cost Map resource for path-vector; the multi-
   cost extension but "endpoint-cost-pv" information
   resource provides a MultipartEndpoint Cost resource for path-vector.

   Both of them support the path-vector extension, "endpoint-
   multicost-map" supports both multi-cost extension and path-vector
   extension.  Filtered Property Map "propmap-availbw-delay" supports
   properties "availbw" and "delay". Maximum Reservable Bandwidth ("maxresbw")
   cost metric in "path-vector" cost mode.

     {
       "meta": {
         "cost-types": {
           "path-vector": {
             "cost-mode": "array",
             "cost-metric": "ane-path"
           },
           "num-routingcost": {
             "cost-mode": "numerical",
             "cost-metric": "routingcost"
           },
           "num-hopcount":
           "pv-maxresbw": {
             "cost-mode": "numerical", "path-vector",
             "cost-metric": "hopcount" "maxresbw"
           }
         }
       },
       "resources": {
         "my-default-networkmap": {
           "uri" : "http://alto.example.com/networkmap",
           "media-type" : "application/alto-networkmap+json"
         },
         "my-default-cost-map": {
           "uri": "http://alto.example.com/costmap/pv",
           "media-type": "application/alto-costmap+json",
           "accepts": "application/alto-costmapfilter+json",
           "capabilities": {
             "cost-type-names": [ "num-hopcount",
                                  "num-routingcost" ]
           },
           "uses": [ "my-default-networkmap" ]
         },
         "cost-map-pv": {
           "uri": "http://alto.example.com/costmap/pv",
           "media-type": "application/alto-costmap+json", `multipart/related;
                          type=application/alto-costmap+json`,
           "accepts": "application/alto-costmapfilter+json",
           "capabilities": {
             "cost-type-names": [ "path-vector" ],
             "dependent-property-map": "propmap-availbw-delay" "pv-maxresbw" ]
           },
           "uses": [ "my-default-networkmap" ]
         },
         "endpoint-cost-pv": {
           "uri": "http://alto.exmaple.com/endpointcost/pv",
           "media-type": "application/alto-endpointcost+json", `multipart/related;
                          type=application/alto-endpointcost+json`,
           "accepts": "application/alto-endpointcostparams+json",
           "capabilities": {
             "cost-type-names": [ "path-vector" ],
             "dependent-property-map": "propmap-availbw-delay",
             "allow-compound-response": true "pv-maxresbw" ]
           }
         },
         "invalid-cost-map" :
         "update-pv": {
           "uri": "http://alto.example.com/costmap/invalid", "http://alto.example.com/updates/pv",
           "media-type": "application/alto-costmap+json",
           "accepts": "application/alto-costmapfilter+json",
           "capabilities": {
             "cost-type-names": [ "path-vector" ],
             "allow-compound-response": true
           }, "text/event-stream",
           "uses": [ "my-default-networkmap" ]
         },
         "propmap-availbw-delay": {
           "uri": "http://alto.exmaple.com/propmap/ane-prop",
           "media-type": "application/alto-propmap+json", "endpoint-cost-pv" ],
           "accepts": "application/alto-propmapparams+json", "application/alto-updatestreamparams+json",
           "capabilities": {
             "domain-types": [ "ane" ],
             "prop-types": [ "availbw", "delay" ]
           },
           "uses": [ "cost-map-pv", "endpoint-cost-pv" ]
             "support-stream-control": true
           }
         }
       }

8.3.
     }

9.2.  Example # 1 #1

   Query filtered cost map to get the path vectors.

   POST /costmap/pv HTTP/1.1
   Host: alto.example.com
   Accept: application/alto-costmap+json, multipart/related;
           type=application/alto-costmap+json,
           application/alto-error+json
   Content-Length: [TBD]
   Content-Type: application/alto-costmapfilter+json

   {
     "cost-type": {
       "cost-mode": "array", "path-vector",
       "cost-metric": "ane-path" "maxresbw"
     },
     "pids": {
       "srcs": [ "PID1" ],
       "dsts": [ "PID2", "PID3" ]
     }
   }

   HTTP/1.1 200 OK
   Content-Length: [TBD]
   Content-Type: multipart/related; boundary=example-1;
                 start=cost-map-pv.costmap
                 type=application/alto-costmap+json

   --example-1
   Resource-Id: cost-map-pv.costmap
   Content-Type: application/alto-costmap+json

   {
     "meta": {
       "vtag": {
         "resource-id": "cost-map-pv.costmap",
         "tag": "d827f484cb66ce6df6b5077cb8562b0a"
       },
       "dependent-vtags": [
         {
           "resource-id": "my-default-networkmap",
           "tag": "75ed013b3cb58f896e839582504f622838ce670f" "75ed013b3cb58f896e839582504f6228"
         }
       ],
       "cost-type": {
         "cost-mode": "array", "path-vector",
         "cost-metric": "ane-path" "maxresbw"

       }
     },
     "cost-map": {
       "PID1": {
         "PID2": [ "ane:L001", "ane:L003" ],
         "PID3": [ "ane:L001", "ane:L004" ]
       }
     }
   }

   Then query the properties of ANEs in path vectors.

   POST /propmap/ane-prop HTTP/1.1
   Host: alto.example.com
   Accept: application/alto-propmap+json,
           application/alto-error+json
   Content-Length: [TBD]
   Content-Type: application/alto-propmapparams+json

   {
     "entities": [ "ane:L001", "ane:L003", "ane:L004" ],
     "properties": [ "delay" ]
   }
   HTTP/1.1 200 OK
   Content-Length: [TBD]
   --example-1
   Resource-Id: cost-map-pv.propmap
   Content-Type: application/alto-propmap+json

   {
     "meta": {
       "dependent-vtags": [
         {
           "resource-id": "cost-map-pv", "cost-map-pv.costmap",
           "tag": "a7d57e120ab63124e3c9a82f7a54bc120fc96216" "d827f484cb66ce6df6b5077cb8562b0a"
         }
       ]
     },
     "property-map": {
       "ane:L001": { "delay": 46}, "maxresbw": 100000000},
       "ane:L003": { "delay": 50}, "maxresbw": 150000000},
       "ane:L004": { "delay": 70} "maxresbw": 50000000}
     }
   }

8.4.

9.3.  Example # 2 #2
   POST /endpointcost/pv HTTP/1.1
   Host: alto.example.com
   Accept: application/alto-endpointcost+json, multipart/related;
           type=application/alto-endpointcost+json,
           application/alto-error+json
   Content-Length: [TBD]
   Content-Type: application/alto-endpointcostparams+json

   {
     "multi-cost-types": [
     "cost-type": {
       "cost-mode": "array", "path-vector",
       "cost-metric": "ane-path" "maxresbw"
     },
       {
         "cost-mode": "numerical",
         "cost-metric": "routingcost"
       }
     ],
     "endpoints": {
       "srcs": [ "ipv4:192.0.2.2" ],
       "dsts": [ "ipv4:192.0.2.89",
                 "ipv4:203.0.113.45",
                 "ipv6:2001:db8::10" ]
     }
   }

   HTTP/1.1 200 OK
   Content-Length: [TBD]
   Content-Type: multipart/related; boundary=example-2;
                 start=endpoint-cost-pv.ecs
                 type=application/alto-endpointcost+json

   --example-2
   Resource-Id: endpoint-cost-pv.ecs
   Content-Type: application/alto-endpointcost+json

   {
     "meta": {
       "vtags": {
         "resource-id": "endpoint-cost-pv.ecs",
         "tag": "bb6bb72eafe8f9bdc4f335c7ed3b10822a391cef"
       },
       "cost-type": [
         {"cost-mode": "array", {
         "cost-mode": "path-vector",
         "cost-metric": "ane-path"}
       ] "maxresbw"
       }
     },
     "endpoint-cost-map": {
       "ipv4:192.0.2.2": {
         "ipv4:192.0.2.89":   [ "ane:L001", "ane:L003",
                                "ane:L004" ],
         "ipv4:203.0.113.45": [ "ane:L001", "ane:L004",
                                "ane:L005" ],

         "ipv6:2001:db8::10": [ "ane:L001", "ane:L005",
                                "ane:L007" ]
       }
     }
   }

   POST /endpointcost/pv HTTP/1.1
   Host: alto.example.com
   Accept: application/alto-endpointcost+json,
           application/alto-error+json
   Content-Length: [TBD]
   Content-Type: application/alto-endpointcostparams+json

   {
     "entities": [ "ane:L001", "ane:L003", "ane:L004",
                   "ane:L005", "ane:L007" ],
     "properties": [ "availbw" ]
   }
   HTTP/1.1 200 OK
   Content-Length: [TBD]
   --example-2
   Resource-Id: endpoint-cost-pv.propmap
   Content-Type: application/alto-propmap+json

   {
     "meta": {
       "dependent-vtags": [
         {
           "resource-id": "endpoint-cost-pv", "endpoint-cost-pv.ecs",
           "tag": "12c0889c3c0892bb67df561ed16d93f5d1fa75cf" "bb6bb72eafe8f9bdc4f335c7ed3b10822a391cef"
         }
       ]
     },
     "property-map": {
       "ane:L001": { "availbw": 50 "maxresbw": 50000000 },
       "ane:L003": { "availbw": 48 "maxresbw": 48000000 },
       "ane:L004": { "availbw": 55 "maxresbw": 55000000 },
       "ane:L005": { "availbw": 60 "maxresbw": 60000000 },
       "ane:L007": { "availbw": 35 "maxresbw": 35000000 }
     }
   }

8.5.

9.4.  Example #3 for Incremental Update

   In this example, an ALTO client subscribe the incremental update for
   the Multipart Endpoint Cost resource "endpoint-cost-pv".

   POST /endpointcost/pv /updates/pv HTTP/1.1
   Host: alto.example.com
   Accept: application/alto-endpointcost+json,
           application/alto-error+json text/event-stream
   Content-Type: application/alto-updatestreamparams+json
   Content-Length: [TBD]
   Content-Type: application/alto-endpointcostparams+json

   {
     "multi-cost-types": [
     "add": {
         "cost-mode": "array",
         "cost-metric": "ane-path"
       },
       "ecspvsub1": {
         "cost-mode": "numerical",
         "cost-metric": "routingcost"
         "resource-id": "endpoint-cost-pv",
         "input": <ecs-input>
       }
     ],
     "endpoints": {
       "srcs": [ "ipv4:192.0.2.2" ],
       "dsts": [ "ipv4:192.0.2.89",
                 "ipv4:203.0.113.45",
                 "ipv6:2001:db8::10" ]
     },
     "properties": [ "delay", "availbw" ]
     }
   }
   Based on the server process defined in
   [I-D.ietf-alto-incr-update-sse], the ALTO server will send the
   control-uri first using Server-Sent Event (SSE), and follow the full
   response of the multipart message.

   HTTP/1.1 200 OK
   Content-Length: [TBD]
   Connection: keep-alive
   Content-Type: text/event-stream

   event: application/alto-updatestreamcontrol+json
   data: {"control-uri": "http://alto.example.com/updates/streams/1414"}

   event: multipart/related;boundary=example-3;start=pvmap;
          type=application/alto-endpointcost+json,ecspvsub1
   data: --example-3
   data: Content-ID: pvmap
   data: Content-Type: application/alto-endpointcost+json

   {
     "meta": {
       "dependent-vtags": [
         {
           "resource-id": "propmap-availbw-delay",
           "tag": "bb6bb72eafe8f9bdc4f335c7ed3b10822a391cef"
         }
       ],
       "cost-type": [
         {"cost-mode": "array", "cost-metric": "ane-path"}
       ]
     },
     "endpoint-cost-map": {
       "ipv4:192.0.2.2": {
         "ipv4:192.0.2.89":   [ "ane:L001", "ane:L003",
                                "ane:L004" ],
         "ipv4:203.0.113.45": [ "ane:L001", "ane:L004",
                                "ane:L005" ],
         "ipv6:2001:db8::10": [ "ane:L001", "ane:L005",
                                "ane:L007" ]
       }
     },
     "property-map": {
       "ane:L001": { "availbw": 50, "delay": 46 },
       "ane:L003": { "availbw": 48, "delay": 50 },
       "ane:L004": { "availbw": 55, "delay": 70 },
       "ane:L005": { "availbw": 60, "delay": 100 },
       "ane:L007": { "availbw": 35, "delay": 100 }
     }
   }

9.
   data:
   data: <endpoint-cost-map-entry>
   data: --example-3
   data: Content-ID: nepmap
   data: Content-Type: application/alto-propmap+json
   data:
   data: <property-map-entry>
   data: --example-3--

   Then, the ALTO server will subscribe the whole tree of the multipart
   message automatically.

   When the data updated, the ALTO server will publish the data updates
   for each node in this tree separately.

   event: application/merge-patch+json,ecspvsub1.pvmap
   data: <Merge patch for endpoint-cost-map-update>

   event: application/merge-patch+json,ecspvsub2.nepmap
   data: <Merge patch for property-map-update>

10.  Compatibility

9.1.

10.1.  Compatibility with Base ALTO Clients/Servers

   The path vector extension on Multipart Filtered Cost Map resource and the Multipart Endpoint
   Cost
   Service is resource has no backward compatible compatibility issue with the base ALTO protocol:

   o
   clients and servers.  Although these two types of resources reuse the
   media types defined in the base ALTO protocol for the accept input
   parameters, they have different media types for responses.  If the
   ALTO server provides extended capabilities "dependent-
      property-map" and "allow-compound-response" for Filtered Cost Map
      or Endpoint Cost Service, these two types of resources, but the ALTO
   client only supports does not support them, the base ALTO protocol, then the client will ignore those capabilities the
   resources without conducting any incompatibility.

   o  If the client sends a request with the input parameter
      "properties", but the server only supports the base ALTO protocol,
      the server will ignore this field.

9.2.

10.2.  Compatibility with Multi-Cost Extension

   This document does not specify how to integrate the "array" "path-vector"
   cost mode
   and the "ane-path" cost metric with the multi-cost extension [RFC8189].  Although there is
   no reason why somebody has to compound the path vectors with other
   cost types in a single query, there is no compatible issue doing it
   without constraint tests.

   As Section 7.1.2 mentions, the syntax and semantics of whether
   "constraints" or "or-constraints" field for the "array" cost mode is
   not specified in this document.  So if an ALTO server provides a
   resource with the "array" cost mode and the capability "cost-
   constraints" or "testable-cost-types-names", the ALTO client MAY
   ignore the capability "cost-constraints" or "testable-cost-types-
   names" unless the implementation or future documents specify the
   behavior.

9.3.

10.3.  Compatibility with Incremental Update

   As this document still follows the basic request/response protocol
   with JSON encoding, it is surely compatible with the incremental
   update service as defined by [I-D.ietf-alto-incr-update-sse].  But
   the following details are to be noticed:

   o  When using the compound response, updates on both cost map and
      property map SHOULD be notified.

   o  When not using the compound response, because the cost map is in
      the "uses" attribute of the property map, once the path vectors in
      the cost map change, the ALTO server MUST send the updates of the
      cost map before the updates of the property map.

10.

11.  General Discussions

10.1.

11.1.  Provide Calendar for Property Map

   Fetching the historical network information is useful for many
   traffic optimization problem.  [I-D.ietf-alto-cost-calendar] already
   proposes an ALTO extension called Cost Calendar which provides the
   historical cost values using Filtered Cost Map and Endpoint Cost
   Service.  However, the calendar for only path costs is not enough.

   For example, as the properties of ANEs (e.g., available bandwidth and
   link delay) are usually the real-time network states, they change
   frequently in the real network.  It is very helpful to get the
   historical value of these properties.  Applications may predicate the
   network status using these information to better optimize their
   performance.

   So the coming requirement may be a general calendar service for the
   ALTO information resources.

10.2.

11.2.  Constraint Tests for General Cost Types

   The constraint test is a simple approach to query the data.  It
   allows users to filter the query result by specifying some boolean
   tests.  This approach is already used in the ALTO protocol.
   [RFC7285] and [RFC8189] allow ALTO clients to specify the
   "constraints" and "or-constraints" tests to better filter the result.

   However, the current defined syntax is too simple and can only be
   used to test the scalar cost value.  For more complex cost types,
   like the "array" mode defined in this document, it does not work
   well.  It will be helpful to propose more general constraint tests to
   better perform the query.

   In practice, it is too complex to customize a language for the
   general-purpose boolean tests, and can be a duplicated work.  So it
   may be a good idea to integrate some already defined and widely used
   query languages (or their subset) to solve this problem.  The
   candidates can be XQuery and JSONiq.

10.3.

11.3.  General Compound Multipart Resources Query

   As the last paragraph of Section 4.3 mentions, querying

   Querying multiple ALTO information resources continuously is MAY be a
   general requirement.  And the coming issues like inefficiency and
   inconsistency are also general.  There is no standard solving these
   issues yet.  So we need some approach to make the ALTO client request
   the compound ALTO information resources in a single query.

11.

12.  Security Considerations

   This document is an extension of the base ALTO protocol, so the
   Security Considerations [RFC7285] of the base ALTO protocol fully
   apply when this extension is provided by an ALTO server.

   The path vector extension requires additional considerations on two
   security considerations discussed in the base protocol:
   confidentiality of ALTO information (Section 15.3 of [RFC7285]) and
   availability of ALTO service (Section 15.5 of [RFC7285]).

   For confidentiality of ALTO information, a network operator should be
   aware of that this extension may introduce a new risk: the path
   vector information may make network attacks easier.  For example, as
   the path vector information may reveal more network internal
   structures than the more abstract single-node abstraction, an ALTO
   client may detect the bottleneck link and start a distributed denial-
   of-service (DDoS) attack involving minimal flows to conduct the in-
   network congestion.

   To mitigate this risk, the ALTO server should consider protection
   mechanisms to reduce information exposure or obfuscate the real
   information, in particular, in settings where the network and the
   application do not belong to the same trust domain.  But the
   implementation of path vector extension involving reduction or
   obfuscation should guarantees the constraints on the requested
   properties are still accurate.

   For availability of ALTO service, an ALTO server should be cognizant
   that using path vector extension might have a new risk: frequent
   requesting for path vectors might conduct intolerable increment of
   the server-side storage and break the ALTO server.  It is known that
   the computation of path vectors is unlikely to be cacheable, in that
   the results will depend on the particular requests (e.g., where the
   flows are distributed).  Hence, the service providing path vectors
   may become an entry point for denial-of-service attacks on the
   availability of an ALTO server.  To avoid this risk, authenticity and
   authorization of this ALTO service may need to be better protected.

   Even if there is no intentional attack, the dependent property map of
   path vector might be still dynamically enriched, in that every new
   request for path vectors will make the ALTO server generate a new
   property map.  So the properties of the abstract network elements can
   consume a large amount of resources when cached.  To avoid this, the
   ALTO server providing the path vector extension should support a
   time-to-live configuration for the property map, so that the outdated
   entries can be removed from the property map resource.

12.

13.  IANA Considerations

12.1.

13.1.  ALTO Cost Mode Registry

   This document specifies a new cost mode "array". "path-vector".  However, the
   base ALTO protocol does not have a Cost Mode Registry where new cost
   mode can be registered.  This new cost mode will be registered once
   the registry is defined either in a revised version of [RFC7285] or
   in another future extension.

12.2.  ALTO Cost Metric Registry

   A new cost metric needs to be registered in the "ALTO Cost Metric
   Registry", listed in Table 2.

                   +-------------+---------------------+
                   | Identifier  | Intended Semantics  |
                   +-------------+---------------------+
                   | ane-path    | See Section 5.2     |
                   +-------------+---------------------+

                        Table 2: ALTO Cost Metrics

12.3.

13.2.  ALTO Entity Domain Registry

   As proposed in Section 9.2 of [I-D.ietf-alto-unified-props-new],
   "ALTO Domain Entity Registry" is requested.  Besides, a new domain is
   to be registered, listed in Table 3. 1.

   +-------------+--------------------------+--------------------------+
   | Identifier  | Entity Address Encoding  | Hierarchy & Inheritance  |
   +-------------+--------------------------+--------------------------+
   | ane         | See Section 6.2          | None                     |
   +-------------+--------------------------+--------------------------+

                        Table 3: 1: ALTO Entity Domain

12.4.

13.3.  ALTO Network Element Property Type Registry

   The "ALTO Abstract Network Element Property Type Registry" is required by the ALTO Domain
   "ane", listed in Table 4.

                +-------------+--------------------------+ 2.

   +-------------+------------+----------------------------------------+
   | Identifier  | Intended   | Dependencies and Interpretation        |
   |             | Semantics  |
                +-------------+--------------------------+                                        | availbw
   +-------------+------------+----------------------------------------+
   | ane:maxresb | The available        | application/alto-costmap+json, or      |
   | w           | maximum    | application/alto-endpointcostmap+json, |
   |             | reservable | where the ANE names are used.          |
   |             | bandwidth  |                                        | delay
   | The transmission delay             |
                +-------------+--------------------------+ for the    |                                        |
   |             | ANE        |                                        |
   +-------------+------------+----------------------------------------+

           Table 4: 2: ALTO Abstract Network Element Property Types

13.

14.  Acknowledgments

   The authors would like to thank discussions with Andreas Voellmy,
   Erran Li, Haibin Son, Haizhou Du, Jiayuan Hu, Qiao Xiang, Tianyuan
   Liu, Xiao Shi, Xin Wang, and Yan Luo. The authors thank Greg
   Bernstein (Grotto Networks), Dawn Chen (Tongji University), Wendy
   Roome, and Michael Scharf for their contributions to earlier drafts.

14.

15.  References

14.1.

15.1.  Normative References

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119,
              DOI 10.17487/
              RFC2119, 10.17487/RFC2119, March 1997, <https://www.rfc-editor.org/info/
              rfc2119>.

14.2.
              <https://www.rfc-editor.org/info/rfc2119>.

15.2.  Informative References

   [I-D.bernstein-alto-topo]
              Bernstein, G., Yang, Y., and Y. Lee, "ALTO Topology
              Service: Uses Cases, Requirements, and Framework", draft-
              bernstein-alto-topo-00 (work in progress), October 2013.

   [I-D.ietf-alto-cost-calendar]
              Randriamasy, S., Yang, Y., Wu, Q., Lingli, D., and N.
              Schwan, "ALTO Cost Calendar", draft-ietf-alto-cost-
              calendar-01 (work in progress), February 2017.

   [I-D.ietf-alto-incr-update-sse]
              Roome, W., Yang, Y., W. and S. Chen, Y. Yang, "ALTO Incremental Updates Using
              Server-Sent Events (SSE)", draft-ietf-alto-incr-update-
              sse-16 (work in progress), March 2019.

   [I-D.ietf-alto-performance-metrics]
              Wu, Q., Yang, Y., Lee, Y., Dhody, D., and S. Randriamasy,
              "ALTO Performance Cost Metrics", draft-ietf-alto-
              incr-update-sse-15
              performance-metrics-06 (work in progress), December November 2018.

   [I-D.ietf-alto-unified-props-new]
              Roome, W., Chen, Randriamasy, S., xinwang2014@hotmail.com, x., Yang, Y., and J. Zhang, "Extensible Property Maps
              "Unified Properties for the ALTO Protocol", draft-ietf-alto-unified-props-new-01 draft-ietf-
              alto-unified-props-new-07 (work in progress), December 2017. March 2019.

   [RFC2387]  Levinson, E., "The MIME Multipart/Related Content-type",
              RFC 2387, DOI 10.17487/RFC2387, August 1998,
              <https://www.rfc-editor.org/info/rfc2387>.

   [RFC7285]  Alimi, R., Ed., Penno, R., Ed., Yang, Y., Ed., Kiesel, S.,
              Previdi, S., Roome, W., Shalunov, S., and R. Woundy,
              "Application-Layer Traffic Optimization (ALTO) Protocol",
              RFC 7285, DOI 10.17487/RFC7285, September 2014,
              <https://www.rfc-editor.org/info/rfc7285>.

   [RFC8189]  Randriamasy, S., Roome, W., and N. Schwan, "Multi-Cost
              Application-Layer Traffic Optimization (ALTO)", RFC 8189,
              DOI 10.17487/RFC8189, October 2017, <https://www.rfc-
              editor.org/info/rfc8189>.
              <https://www.rfc-editor.org/info/rfc8189>.

Authors' Addresses
   Kai Gao
   Tsinghua University
   Beijing  Beijing
   China

   Email: gaok12@mails.tsinghua.edu.cn

   Young Lee
   Huawei
   TX
   USA

   Email: leeyoung@huawei.com

   Sabine Randriamasy
   Nokia Bell Labs
   Route de Villejust
   NOZAY  91460
   FRANCE

   Email: Sabine.Randriamasy@nokia-bell-labs.com

   Y. Richard Yang
   Yale University
   51 Prospect St
   New Haven  CT
   USA

   Email: yry@cs.yale.edu

   Jingxuan Jensen Zhang
   Tongji University
   4800 Caoan Road
   Shanghai  201804
   China

   Email: jingxuan.n.zhang@gmail.com