Internet Working Group
BESS WorkGroup                                                A. Sajassi
Internet Draft
Internet-Draft                                              P. Brissette
Category:
Intended status: Standards Track                           Cisco Systems
Expires: September 3, 2020                                     R. Schell
                                                                 Verizon
                                                                J. Drake
                                                                 Juniper
                                                              J. Rabadan
                                                                   Nokia

Expires: July 18, 2019                                  January 18, 2019
                                                           March 2, 2020

                     EVPN Virtual Ethernet Segment
              draft-ietf-bess-evpn-virtual-eth-segment-04
              draft-ietf-bess-evpn-virtual-eth-segment-05

Abstract

   EVPN and PBB-EVPN introduce a family of solutions for multipoint
   Ethernet services over MPLS/IP network with many advanced features
   among which their multi-homing capabilities.  These solutions define
   two types of multi-homing for an Ethernet Segment (ES): 1) Single-
   Active
   introduce Single-Active and 2) All-Active, where All-Active for an Ethernet Segment is (ES),
   itself defined as a set of physical links between the multi-homed
   device/network and a set of PE devices that they are connected to.

   Some Service Providers want to extend
   This document extends the Ethernet Segment concept of the physical
   links in so that an ES to Ethernet Virtual Circuits (EVCs) where many of such
   EVCs can
   be aggregated on a single physical External Network-to-
   Network Interface (ENNI). An ES that consists of associated to a set of EVCs
   instead of physical links is (e.g., VLANs) or other objects such as
   MPLS Label Switch Paths (LSPs) or Pseudowires (PWs), referred to as a virtual ES
   Virtual Ethernet Segments (vES).  This draft describes the
   requirements and the extensions needed to support vES in EVPN and
   PBB-EVPN.

Requirements Language

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in RFC 2119 [RFC2119] and
   RFC 8174 [RFC8174].

Status of this This Memo

   This Internet-Draft is submitted to IETF in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups. (IETF).  Note that other groups may also distribute
   working documents as Internet-Drafts.  The list of current Internet-
   Drafts is at https://datatracker.ietf.org/drafts/current/.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."
   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/1id-abstracts.html

   The list of

   This Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html will expire on September 3, 2020.

Copyright and License Notice

   Copyright (c) 2014 2020 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info)
   (https://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

Conventions

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
   "OPTIONAL" in this document are to be interpreted as described in BCP
   14 [RFC2119] [RFC8174] when, and only when, they appear in all
   capitals, as shown here.

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . . .  4
     1.1   3
     1.1.  Virtual Ethernet Segments in Access Ethernet Networks . . .  4
     1.2   3
     1.2.  Virtual Ethernet Segments in Access MPLS Networks . . . . .   5
   2.  Terminology . . . . . . . . . . . . . . . . . . . . . . . . . .  7   6
   3.  Requirements  . . . . . . . . . . . . . . . . . . . . . . . . .  8   7
     3.1.  Single-Homed & and Multi-Homed Virtual Ethernet Segments vES  . . .  8 . . . . . . . . .   7
     3.2.  Scalability . . . . . . . . . . . . . . . . . . . . . . . .   8
     3.3.  Local Switching . . . . . . . . . . . . . . . . . . . . . .  9   8
     3.4.  EVC Service Types . . . . . . . . . . . . . . . . . . . . .   9
     3.5.  Designated Forwarder (DF) Election  . . . . . . . . . . . . 10   9
     3.6.  OAM . . . . . . . . . . . . . . . . . . . . . . . . . . . .  10
     3.7.  Failure & and Recovery  . . . . . . . . . . . . . . . . . . . . 11  10
     3.8.  Fast Convergence  . . . . . . . . . . . . . . . . . . . . . 11  10
   4.  Solution Overview . . . . . . . . . . . . . . . . . . . . . . .  11
     4.1.  EVPN DF Election for vES  . . . . . . . . . . . . . . . . . 13  12
   5.  Failure Handling & and Recovery . . . . . . . . . . . . . . . . . . 14  13
     5.1.  Failure Handling for Single-Active vES in EVPN  . . . . . . 15  14
     5.2.  EVC Failure Handling for Single-Active vES in PBB-EVPN  . . 16  15
     5.3.  Port Failure Handling for Single-Active vES's vESes in EVPN . . . 17  16
     5.4.  Port Failure Handling for Single-Active vES's vESes in PBB-EVPN . 18  17
     5.5.  Fast Convergence in PBB-EVPN  . . (PBB-)EVPN  . . . . . . . . . . . . .  18
   6.  BGP Encoding  . . . . . . . . . . . . . . . . . . . . . . . . .  20
     6.1.  I-SID Extended Community  . . . . . . . . . . . . . . . . . 21  20
   7.  Acknowledgements  . . . . . . . . . . . . . . . . . . . . . . .  21
   8.  Security Considerations . . . . . . . . . . . . . . . . . . . .  21
   9.  IANA Considerations . . . . . . . . . . . . . . . . . . . . . .  21
   10. Intellectual Property Considerations  . . . . . . . . . . . . . 22  21
   11. Normative References  . . . . . . . . . . . . . . . . . . . . . 22
   12. Informative . . . .  21
     11.1.  Normative References . . . . . . . . . . . . . . . . . .  21
     11.2.  Informative References . . . . . . . . . . . . . . . . .  22
   13.
   Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  22

1.  Introduction

   [RFC7432] and [RFC7623] introduce a family of solutions for
   multipoint Ethernet services over MPLS/IP network with many advanced
   features among which their multi-homing capabilities.  These
   solutions
   define two types of multi-homing for an Ethernet Segment (ES): 1) introduce Single-Active and 2) All-Active, where All-Active for an Ethernet
   Segment is (ES), itself defined as a set of links between the multi-homed multi-
   homed device/network and a set of PE devices that they are connected
   to.

   This document extends the Ethernet Segment concept so that an ES can
   be associated to a set of EVCs (e.g., VLANs) or other objects such as
   MPLS Label Switch Paths (LSPs) or Pseudowires (PWs).

1.1 (PWs), referred to as
   Virtual Ethernet Segments (vES).  This draft describes the
   requirements and the extensions needed to support vES in EVPN and
   PBB-EVPN.

1.1.  Virtual Ethernet Segments in Access Ethernet Networks

   Some Service Providers (SPs) want to extend the concept of the
   physical links in an ES to Ethernet Virtual Circuits (EVCs) where
   many of such EVCs (e.g., VLANs) can be aggregated on a single
   physical External Network-to-Network Interface (ENNI).  An ES that
   consists of a set of EVCs instead of physical links is referred to as
   a virtual ES (vES).  Figure 1 depicts two PE devices (PE1 and PE2)
   each with an ENNI where a number of vES's vESes are aggregated on - each of
   which through its associated EVC.

                     Carrier
                     Ethernet
       +-----+       Network
       | CE11|EVC1   +---------+     <---- EVPN Network  ----->
       +-----+   \  |         |       +---+
       Cust. A    \-0=========0--ENNI1|   |
       +-----+      |         |  ENNI1|   |   +-------+   +---+
       | CE12|EVC2--0=========0--ENNI1|PE1|---|       |   |   |
       +-----+      |         |  ENNI1|   |   |       |---|PE3|-
                    |       ==0--ENNI1|   |   |IP/MPLS|   |   | \  +---+
       +-----+      |      /  |       +---+   |Network|   +---+  \-|   |
       | CE22|EVC3--0==== /   |               |       |            |CE4|
       +-----+      |    X    |               |       |   +---+    |   |
       Cust. B      |   / \   |       +---+   |       |   |   |  /-|   |
       +-----+     -0===   ===0--ENNI2|   |   |       |---|PE4|-/  +---+
       | CE3 |EVC4/ |         |  ENNI2|PE2|---|       |   |   |
       |     |EVC5--0=========0--ENNI2|   |   +-------+   +---+
       +-----+      |         |       +---+
       Cust. C      +---------+   /\
              /\                  ||
              ||                  ENNI
              EVCs             Interface
       <--------802.1Q---------->  <---- EVPN Network -----> <-802.1Q->

   Figure 1: DHD/DHN (both SA/AA) and SH on same ENNI

   ENNIs are commonly used to reach off-network / out-of-franchise
   customer sites via independent Ethernet access networks or third-
   party Ethernet Access Providers (EAP) (see Figure 1).  ENNIs can
   aggregate traffic from hundreds to thousands of vES's; where, vESes, where each vES
   is represented by its associated EVC on that ENNI.  As a result,
   ENNIs and their associated EVCs are a key element of SP off-networks
   that are carefully designed and closely monitored.

   In order to meet customer's customers' Service Level Agreements (SLA), SPs build
   redundancy via multiple EVPN PEs and across multiple ENNIs (as shown
   in Figure 1) where a given vES can be multi-homed to two or more EVPN
   PE devices (on two or more ENNIs) via their associated EVCs.  Just
   like physical ES's in [RFC7432] and [RFC7623] solutions, these vES's vESes
   can be single-homed or multi-homed ES's and when multi-homed, then
   can operate in either Single-Active or All-Active redundancy modes.
   In a typical SP off-network scenario, an ENNI can be associated with
   several thousands of single-homed vES's, vESes, several hundreds of Single-
   Active vES's vESes and it may also be associated with tens or hundreds of
   All-Active vES's.

1.2 vESes.

1.2.  Virtual Ethernet Segments in Access MPLS Networks

   Other Service Providers (SPs) want to extend the concept of the
   physical links in an ES to individual Pseudowires (PWs) or to MPLS
   Label Switched Paths (LSPs) in Access MPLS networks - i.e., a vES
   consisting of a set of PWs or a set of LSPs.  Figure 2 illustrates
   this concept.

                    MPLS Aggregation
                    Network
      +-----+      +----------------+ <---- EVPN Network ----->      +-----------------+
      | CE11|EVC1  |                 |
      +-----+   \+AG1--+  PW1      +-----+
      Cust. A    -0----|===========|     |
      +-----+     | ---+===========|     |   +-------+   +---+
      | CE12|EVC2-0/   |  PW2   /\ | PE1 +---+       |   |   |
      +-----+     ++---+      ==||=|     |   |       +---+PE3+-
                   |         //=||=|     |   |IP/MPLS|   |   | \  +---+
                   |        //  \/ ++----+   |Network|   +---+  \-+   |
      +-----+EVC3  |    PW3//  LSP1  |       |       |            |CE4|
      | CE13|    +AG2--+===/PW4      |       |       |   +---+    |   |
      +-----+     0    |===     /\ ++----+   |       |   |   |  /-+   |
                  0    |==PW5===||=|     |   |       +---+PE4+-/  +---+
      +-----+    /++---+==PW6===||=| PE2 +---+       |   |   |
      | CE14|EVC4  |            \/ |     |   +-------+   +---+
      +-----+      |           LSP2+-----+
      Cust. C      +----------------+      +-----------------+
             /\
             ||
             EVCs
   <--802.1Q---><------MPLS------->
      <--802.1Q---><------MPLS------> <---- EVPN Network ---> <-802.1Q->

           Figure 2: DHN and SH on Access MPLS networks

   In some cases, Service Providers use Access MPLS Networks that belong
   to separate administrative entities or third parties as a way to get
   access to the their own IP/MPLS network infrastructure.  This is the
   case illustrated in Figure 2.

   In such scenarios, a virtual ES (vES) is defined as a set of
   individual PWs if they cannot be aggregated into a common LSP.  If
   the aggregation of PWs is possible, the vES can be associated to an
   LSP in a given PE.  In the example of Figure 2, EVC3 is connected to
   a VPWS instance in AG2 that is connected to PE1 and PE2 via PW3 and
   PW5 respectively.  EVC4 is connected to a separate VPWS instance on
   AG2 that gets connected to an EVI on PE1 and PE2 via PW4 and PW6,
   respectively.  Since the PWs for the two VPWS instances can be
   aggregated into the same LSPs going to the EVPN MPLS network, a common
   virtual ES can be defined for LSP1 and LSP2.  This vES will be shared
   by two separate EVIs in the EVPN network.

   In some cases, this aggregation of PWs into common LSPs may not be
   possible.  For instance, if PW3 were terminated into a third PE, e.g.
   PE3, instead of PE1, the vES would need to be defined on a per
   individual PW on each PE, i.e. PW3 and PW5 would belong to ES-1,
   whereas PW4 and PW6 would be associated to ES-2.

   For MPLS/IP access networks where a vES represents a set of PWs or
   LSPs, this document extends Single-Active multi-homing procedures of
   [RFC7432] and [7623] [RFC7623] to vES.  The vES extension to All-Active
   multi- homing is outside of the scope of this document for MPLS/IP
   access networks.

   This draft describes requirements and the extensions needed to
   support a vES in [RFC7432] and [RFC7623].  Section 3 lists the set of
   requirements for vES's. a vES.  Section 4 describes extensions for a vES
   that are applicable to EVPN solutions including [RFC7432], [RFC7623], [RFC7432] and
   [RFC8214].
   [RFC7209].  Furthermore, these extensions meet the requirements
   described in section 3.  Section 4 gives solution overview and
   section 5 describes the failure handling handling, recovery, scalability, and
   recovery for vES's in fast
   convergence of [RFC7432] and [RFC7623]. [RFC7623] for vESes.  Section 6 covers
   scalability and fast convergence required for vES's in [RFC7432]
   introduces a new BGP extended community and
   [RFC7623]. its encoding.

2.  Terminology

   AC: Attachment Circuit

   BEB: Backbone Edge Bridge

   B-MAC: Backbone MAC Address

   CE: Customer Edge

   CFM: Connectivity Fault Management (802.1ag)

   C-MAC: Customer/Client MAC Address

   DHD: Dual-homed Device

   DHN: Dual-homed Network

   ENNI: External Network-Network Interface
   ES: Ethernet Segment

   ESI: Ethernet-Segment Ethernet Segment Identifier

   EVC: Ethernet Virtual Circuit

   EVPN: Ethernet VPN

   I-SID: Service Instance Identifier (24 bits and global within a PBB
   network see [RFC7080])

   LACP: Link Aggregation Control Protocol

   PBB: Provider Backbone Bridge

   PBB-EVPN: Provider Backbone Bridge EVPN

   PE: Provider Edge

   SH: Single-Homed

   Single-Active Redundancy Mode (SA): When only a single PE, among a
   group of PEs attached to an Ethernet-Segment, Ethernet Segment, is allowed to forward
   traffic to/from that Ethernet Segment, then the Ethernet segment Segment is
   defined to be operating in Single-Active redundancy mode.

   All-Active Redundancy Mode (AA): When all PEs attached to an Ethernet
   segment are allowed to forward traffic to/from that Ethernet-Segment, Ethernet Segment,
   then the Ethernet segment Segment is defined to be operating in All-Active
   redundancy mode.

3.  Requirements

   This section describes the requirements specific to virtual Ethernet
   Segment (vES) for (PBB-)EVPN solutions.  These requirements are in
   addition to the ones described in [RFC7209], [RFC8214], [RFC7432], and
   [RFC7623].

3.1.  Single-Homed & and Multi-Homed Virtual Ethernet Segments vES

   A PE needs to support the following types of vES's: vESes:

   (R1a) A PE MUST handle single-homed vES's vESes on a single physical port
   (e.g., single ENNI)

   (R1b) A PE MUST handle a mix of Single-Homed vES's vESes and Single-Active
   multi-homed vES's vESes simultaneously on a single physical port (e.g.,
   single ENNI).  Single-Active multi-homed vES's vESes will be simply
   referred to as Single-Active vES's vESes through the rest of this document.

   (R1c) A PE MAY handle All-Active multi-homed vES's vESes on a single
   physical port.  All-Active multi-homed vES's vESes will be simply referred
   to as All-Active vES's vESes through the rest of this document.

   (R1d) A PE MAY handle a mixed of All-Active vES's vESes along with other
   types of vES's vESes on a single physical port port.

   (R1e) A Multi-Homed vES (Single-Active or All-Active) can be spread
   across any two or more PEs  (on ENNIs, on any two or more ENNIs) PEs.

3.2.  Scalability

   A single physical port (e.g., ENNI) can be associated with many
   vES's.
   vESes.  The following requirements give a quantitative measure for
   each vES type.

   (R2a) A PE SHOULD handle very large number of Single-Homed vES's vESes on a
   single physical port (e.g., thousands of vES's vESes on a single ENNI) ENNI).

   (R2b) A PE SHOULD handle large number of Single-Active vES's vESes on a
   single physical port (e.g., hundreds of vES's vESes on a single ENNI) ENNI).

   (R2c) A PE MAY handle large number of All-Active Multi-Homed vES's vESes on a single
   physical port (e.g., hundreds of vES's vESes on a single ENNI) ENNI).

   (R2d) A PE SHOULD handle the above scale for a mix of Single-homed
   vES's
   vESes and Single-Active vES's vESes simultaneously on a single physical
   port (e.g., single ENNI)

   (R4e) ENNI).

   (R2e) A PE MAY handle the above sale for a mixed of All-Active Multi-
   Homed vES's vESes
   along with other types of vES's vESes on a single physical port port.

3.3.  Local Switching

   Many vES's vESes of different types can be aggregated on a single physical
   port on a PE device and some of these vES can belong to the same
   service instance (or customer).  This translates into the need for
   supporting local switching among the vES's vESes of the same service
   instance on the same physical port (e.g., ENNI) of the PE.

   (R3a) A PE MUST support local switching among different vES's vESes
   belonging to the same service instance (or customer) on a single
   physical port.  For example, in Figure 1, PE1 MUST support local
   switching between CE11 and CE12 (both belonging to customer A) that
   are mapped to two Single-homed vES's vESes on ENNI1.  In case of Single-Active vES's, Single-
   Active vESes, the local switching is performed among active EVCs
   belonging to the same service instance on the same ENNI.

3.4.  EVC Service Types

   A physical port (e.g., ENNI) of a PE can aggregate many EVCs each of
   which is associated with a vES.  Furthermore, an EVC may carry one or
   more VLANs.  Typically, an EVC carries a single VLAN and thus it is
   associated with a single broadcast domain.  However, there is no
   restriction on an EVC to carry more than one VLAN.

   (R4a) An EVC can be associated with a single broadcast domain - e.g.,
   VLAN-based service or VLAN bundle service service.

   (R4b) An EVC MAY be associated with several broadcast domains - e.g.,
   VLAN-aware bundle service service.

   In the same way, a PE can aggregate many LSPs and PWs.  In the case
   of individual PWs per vES, typically a PW is associated with a single
   broadcast domain, but there is no restriction on the PW to carry more
   than one VLAN if the PW is of type Raw mode.

   (R4c) A PW can be associated with a single broadcast domain - e.g.,
   VLAN-based service or VLAN bundle service.

   (R4d) An PW MAY be associated with several broadcast domains - e.g.,
   VLAN-aware bundle service." service.

3.5.  Designated Forwarder (DF) Election

   Section 8.5 of [RFC7432] describes the default procedure for DF
   election in EVPN which is also used in [RFC7623] and [RFC8214].  This
   default DF election procedure is performed at the granularity of
   <ESI,
   (ESI, Ethernet Tag>. Tag).  In case of a vES, the same EVPN default
   procedure for DF election also applies; however, at the granularity
   of <vESI, (vESI, Ethernet Tag>; Tag); where vESI is the virtual Ethernet Segment
   Identifier.
   Identifier and the Ethernet Tag field is represented by and I-SID in
   PBB-EVPN and by a VLAN ID (VID) in EVPN.  As in [RFC7432], this default
   defult procedure for DF election at the granularity of <vESI, (vESI,
   Ethernet Tag> Tag) is also referred to as "service carving"; where, Ethernet Tag is represented by an I-SID in
   PBB-EVPN and by a VLAN ID (VID) in EVPN. carving".  With service
   carving, it is
   possible desireable to evenly distribute partition the DFs for different
   vES's among different PEs, thus evenly distributing the traffic among
   different PEs.  The following list the requirements apply to DF
   election of vES's for
   EVPN. (PBB-)EVPN.

   (R5a) A vES with m EVCs can be distributed among n ENNIs belonging to
   p PEs in any arbitrary oder; order; where n >= p >= m.  For example, if
   there is an vES with 2 EVCs and there are 5 ENNIs on 5 PEs (PE1
   through PE5), then vES can be dual-homed to PE2 and PE4 and the DF
   election must be performed between PE2 and PE4.

   (R5b) Each vES MUST be identified by its own virtual ESI (vESI) (vESI).

3.6.  OAM

   In order to detect the failure of an individual EVC and perform DF
   election for its associated vES as the result of this failure, each
   EVC should be monitored independently.

   (R6a) Each EVC SHOULD be monitored for its health independently independently.

   (R6b) A single EVC failure (among many aggregated on a single
   physical port/ENNI) MUST trigger DF election for its associated vES.

3.7.  Failure & and Recovery

   (R7a) Failure and failure recovery of an EVC for a Single-homed vES
   SHALL NOT impact any other EVCs for within its own service instance or any
   other service instances.  In other words, for PBB-EVPN, it SHALL NOT
   trigger any MAC flushing both within its own I-SID as well as other
   I-SIDs.  (R7b) In case of All-Active Multi-Homed vES, failure and failure
   recovery of an EVC for that vES SHALL NOT impact any other EVCs for
   within its own service instance or any other service instances.  In other
   words, for PBB-EVPN, it SHALL NOT trigger any MAC flushing both
   within its own I-SID as well as other I-SIDs.  (R7c) Failure & and
   failure recovery of an EVC for a Single-Active vES SHALL only impact only
   its own service instance.  In other words, for PBB- EVPN, MAC
   flushing SHALL be limited to the associated I-SID only and SHALL NOT
   impact any other I-SIDs.  (R7d) Failure & and failure recovery of an
   EVC for a Single-Active vES MAY only impact C-MACs associated with
   MHD/MHNs for that service instance.  In other words, MAC flushing
   SHOULD be limited to single service instance (I-SID in the case of
   PBB-EVPN) and only CMACs for Single-Active MHD/MHNs.

3.8.  Fast Convergence

   Since large number of EVCs (and their associated vES's) vESes) are
   aggregated via a single physical port (e.g., ENNI), then the failure
   of that physical port impacts large number of vES's vESes and triggers
   large number of ES route withdrawals.  Formulating, sending,
   receiving, and processing such large number of BGP messages can
   introduce delay in DF election and convergence time.  As such, it is
   highly desirable to have a mass-withdraw mechanism similar to the one
   in the [RFC7432] for withdrawing large number of Ethernet A-D routes.
   (R8a) There SHOULD be a mechanism equivalent to EVPN mass-withdraw
   such that upon an ENNI failure, only a single BGP message is needed
   to indicate to the remote PEs to trigger DF election for all impacted
   vES associated with that ENNI.

4.  Solution Overview

   The solutions described in [RFC7432] and [RFC7623] are leveraged as as-
   is with one simple the modification and that is the ESI assignment is performed for an
   EVC or a group of EVCs or LSPs/PWs instead of a link or a group of
   physical links.  In other words, the ESI is associated with a virtual
   ES (vES) and that's why it will be (vES), hereby referred to as vESI.  For the EVPN solution,
   everything basically remains the same except for the handling of
   physical port failure where many vES's vESes can be impacted.  Section 5.1
   and 5.3 below describe the handling of physical port/link failure for
   EVPN.  In a typical multi-homed operation, MAC addresses are learned
   behind a vES and are advertised with the ESI corresponding to the vES
   (i.e., vESI).  EVPN aliasing and mass- withdraw operations are
   performed with respect to vES.  In other words, the Ethernet A-D
   routes for these operations are advertised with vESI instead of ESI.
   For PBB-EVPN solution, the main change is with respect to the BMAC
   address assignment which is performed similar to what is described in
   section 7.2.1.1 of [RFC7623] with the following refinements:

   -

   o  One shared BMAC address SHOULD used per PE for the single-homed
   vES's.
      vESes.  In other words, a single BMAC is shared for all single-homed
   vES's single-
      homed vESes on that PE.

   -

   o  One shared BMAC address SHOULD be used per PE per physical port
      (e.g., ENNI) for the Single-Active vES's. vESes.  In other words, a
      single BMAC is shared for all Single-Active vES's vESes that share the
      same ENNI.

   -

   o  One shared BMAC address MAY be used for all Single-Active vES's vESes on
      that PE.

   -

   o  One BMAC address SHOULD be used per set of EVCs representing an
      All-Active multi-homed vES.  In other words, a single BMAC address is used per
      vES for All-Active multi-homing scenarios.

   -

   o  A single BMAC address MAY also be used per vES per PE for Single-
      Active  multi-homing scenarios.

                       BEB   +--------------+  BEB
                        ||   |              |   ||
                        \/   |              |   \/
          +----+ EVC1 +----+ |              | +----+   +----+
          | CE1|------|    | |              | |    |---| CE2|
          +----+\     | PE1| |   IP/MPLS    | | PE3|   +----+
                 \    +----+ |   Network    | +----+
                  \          |              |
               EVC2\  +----+ |              |
                    \ |    | |              |
                     \| PE2| |              |
                      +----+ |              |
                        /\   +--------------+
                        ||
                        BEB
          <--802.1Q--><---------- PBB-EVPN --------><--802.1Q->

      Figure 3: PBB-EVPN Network

4.1.  EVPN DF Election for vES

   The procedure for service carving for virtual Ethernet Segments is
   the same as the one outlined in section 8.5 of [RFC7432] except for
   the fact that ES is replaced with vES.  For the sake of clarity and
   completeness, this procedure is repeated below:

   1.  When a PE discovers the vESI or is configured with the vESI
       associated with its attached vES, it advertises an Ethernet
       Segment route with the associated ES-Import extended community
       attribute.

   2.  The PE then starts a timer (default value = 3 seconds) to allow
       the reception of Ethernet Segment routes from other PE nodes
       connected to the same vES.  This timer value MUST be same across
       all PEs connected to the same vES.

   3.  When the timer expires, each PE builds an ordered list of the IP
       addresses of all the PE nodes connected to the vES (including
       itself), in increasing numeric value.  Each IP address in this
       list is extracted from the "Originator Router's IP address" field
       of the advertised Ethernet Segment route.  Every PE is then given
       an ordinal indicating its position in the ordered list, starting
       with 0 as the ordinal for the PE with the numerically lowest IP
       address.  The ordinals are used to determine which PE node will
       be the DF for a given EVPN instance on the vES using the
       following rule: Assuming a redundancy group of N PE nodes, the PE
       with ordinal i is the DF for an EVPN instance with an associated
       Ethernet Tag value of V when (V mod N) = i.  It should be noted
       that using "Originator Router's IP address" field in the Ethernet
       Segment route to get the PE IP address needed for the ordered
       list, allows for a CE to be multi-homed across different ASes if
       such need ever arises.

   4.  The PE that is elected as a DF for a given EVPN instance will
       unblock traffic for that EVPN instance.  Note that the DF PE
       unblocks all traffic in both ingress and egress directions for
       Single-Active vES and unblocks multi-destination in egress
       direction for All-Active Multi-homed vES.  All non-DF PEs block
       all traffic in both ingress and egress directions for Single-Active Single-
       Active vES and block multi-destination traffic in the egress
       direction for All-Active multi-homed vES.

   In the case of an EVC failure, the affected PE withdraws its Virtual
   Ethernet Segment route if there are no more EVCs associated to the
   vES in the PE.  This will re-trigger the DF Election procedure on all
   the PEs in the Redundancy Group.  For PE node failure, or upon PE
   commissioning or decommissioning, the PEs re-trigger the DF Election
   Procedure across all affected vES's. vESes.  In case of a Single-Active multi-homing, Single-Active,
   when a service moves from one PE in the Redundancy Group to another
   PE as a result of DF re-election, the PE, which ends up being the
   elected DF for the service, SHOULD trigger a MAC address flush
   notification towards the associated vES.  This can be done, for e.g.
   using IEEE 802.1ak MVRP 'new' declaration.

   For LSP LSP-baesd and PW based PW-based vES, the non-DF PE SHOULD signal PW-status
   'standby' signaling to the Aggregation PE (e.g., AG PE in Figure 2), and the a new
   DF PE MAY send an LDP MAC withdraw message as a MAC address flush
   notification.  It should be noted that the PW-status is signaled for
   the scenarios where there is a one-to-one mapping between EVI/BD and
   the PW.

5.  Failure Handling & and Recovery

   There are a number of failure scenarios to consider such as:

   A: CE Uplink Port Failure uplink port failure

   B: Ethernet Access Network Failure failure

   C: PE Access-facing Port access-facing port or link Failure failure

   D: PE Node Failure node failure

   E: PE isolation from IP/MPLS network

   [RFC7432], [RFC7623], and [RFC8214] solutions provide protection
   against such failures as described in the corresponding references.
   In the presence of virtual Ethernet Segments (vES's) (vESes) in these
   solutions, besides the above failure scenarios, there EVC failure is one more an
   additional scenario to consider and that is EVC failure. This consider.  Handing vES failure scenarios
   implies that individual EVCs or PWs need to be monitored and upon their
   detection of failure
   detection, or restoration of services, appropriate DF
   election procedures and failure recovery
   mechanism need to be mechanisms are executed.

   [ETH-OAM] is used for monitoring EVCs and upon failure detection of a
   given EVC, DF election procedure per section [4.1] is executed.  For
   PBB-EVPN, some extensions are needed to handle the failure and
   recovery procedures of [RFC7623] in order to meet the above
   requirements.  These extensions are describe described in the next section.

   [MPLS-OAM] and [PW-OAM] are used for monitoring the status of LSPs
   and/or PWs associated to vES.

                         B            D
                         ||           ||
                         \/           \/
                       +-----+
          +-----+      |     |       +---+
          | CE1 |EVC2--0=====0--ENNI1|   |   +-------+
          +-----+      |    =0--ENNI1|PE1|---|       |  +---+  +---+
          Cust. A      |   / |       |   |   |IP/MPLS|--|PE3|--|CE4|
          +-----+      |  /  |       +---+   |Network|  |   |  +---+
          |     |EVC2--0==   |               |       |  +---+
          | CE2 |      |     |       +---+   |       |
          |     |EVC3--0=====0--ENNI2|PE2|---|       |
          +-----+      |     |       |   |   +-------+
                       +-----+       +---+
                    /\             /\     /\
                    ||             ||     ||
                    A              C      E

      Figure 4: Failure Scenarios A,B,C,D and E

5.1.  Failure Handling for Single-Active vES in EVPN

   When

   In RFC7432, when a DF PE connected to a Single-Active multi-homed
   Ethernet Segment loses connectivity to the segment, due to link or
   port failure, it signals to the remote PEs to withdraw all MAC
   addresses associated with that Ethernet Segment.  This is done by
   advertising a mass-withdraw message using Ethernet A-D per-ES route.

   It should be noted that for dual-homing use cases where there is only
   a single backup path, MAC withdraw can be avoided by the remote PEs
   as they can simply update their nexthop associated with the affected
   MAC entries to the backup path per procedure described in section 8.2
   of [RFC7432].

   In case of an EVC failure which impacts a single vES, the exact same
   EVPN procedure is used.  In this case, the message using Ethernet A-D
   per ES vES route carries the vESI representing the vES which in turn is
   associated with the failed EVC.  The remote PEs upon receiving this
   message perform the same procedures outlined in section 8.2 of
   [RFC7432].

5.2.  EVC Failure Handling for Single-Active vES in PBB-EVPN

   When a PE connected to a Single-Active multi-homed Ethernet Segment loses
   connectivity to the segment, due to link or port failure, it signals
   the remote PE to flush all CMAC addresses associated with that
   Ethernet Segment.  This is done by advertising a BMAC route along
   with MAC Mobility Extended community.

   In case of an EVC failure that impacts a single vES, if the above
   PBB-EVPN procedure is used, it results in excessive CMAC flushing
   because a single physical port can support large number of EVCs (and
   their associated vES's) vESes) and thus advertising a BMAC corresponding to
   the physical port with MAC mobility Extended community will result in
   flushing CMAC addresses not just for the impacted EVC but for all
   other EVCs on that port.

   In order to reduce the scope of CMAC flushing to only the impacted
   service instances (the service instance(s) impacted by the EVC
   failure), the BGP flush message is sent along with a list of impacted
   I-SID(s) represented by the new EVPN I-SID Extended Community as
   defined in section 6.  Since typically an EVC maps to a single
   broadcast domain and thus a single service instance, the list only
   contains a single I-SID.  However, if the failed EVC carries multiple
   VLANs each with its own broadcast domain, then the list contains
   several I-SIDs - one for each broadcast domain.  This new BGP flush
   message basically instructs the remote PE to perform flushing for
   CMACs corresponding to the advertised BMAC only across the advertised
   list of I-ISIDs.

   The new I-SID Extended Community provides a way to encode upto 24 I-
   SIDs in each Extended Community if the impacted I-SIDs are sequential
   (the base I-SID value plus the next 23 I-SID values).  If the number
   of I-SIDs associated with a failed EVC is large or if the affected I-
   SIDs are not sequential, then multiple I-SID Extended Communities can
   be sent along with the flush message.  However, if the number of
   affected I-SIDs is very large such that the corresponding I-SID
   Extended Communities cannot be fitted in a single BGP attribute, then
   the EVC failure can be treated as a port failure and the procedures
   of section 5.4 can be exercised (i.e., a single BGP flush message
   without the I-SID list can be transmitted).  When the BGP flush
   message is transmitted without the I-SID list, then it instructs the
   receiving PEs to flush CMACs associated with that BMAC across all I-
   SIDs.

   There can be scenarios (although unlikely) where multiple EVCs within
   the same physical port can fail within a short time resulting in the
   PE advertising multiple BGP flush messages each with their own list
   of I-SIDs; however, the route reflector receiving these messages will
   only send the last flush message.  This results in PEs receiving such
   flush messages not to properly flush all the affected I-SIDs.  In
   order to address such scenarios, a timer T1 is started upon an EVC1
   failure on the advertising PE.  If there is another EVC2 failure
   within T1, affected I-SIDs are aggregated for both EVC1 and EVC2 to
   be sent along the new flush message.  Furthermore when EVC2 failure
   occurs, another timer T2 (with the same value as T1) is started to
   keep track of the affected I-SIDs for EVC2.  Such I-SID aggregation
   may result in multiple flushing for the same I-SID(s) on the
   receiving PEs.  The default value for this timer T is 10 seconds.

   The I-SID dependent flushing mechanism described in this section is
   also backward compatible for the PEs supporting [RFC7623] such that
   the PEs that don't understand the I-SID list (i.e., the new I-SID
   Extended Community) simply ignore it and default to flushing all the
   I-SIDs for the B-MAC - i.e., the PEs default to per-port flushing
   described in section 5.4.

   The above BMAC route that is advertised with the MAC Mobility
   Extended Community, can either represent the MAC address of the
   physical port that the failed EVC is associated with, or it can
   represent the MAC address of the PE.  In the latter case, this is the
   dedicated per-PE MAC address used for all Single-Active vES's vESes on that
   PE.  The former one performs better than the latter one in terms of
   reducing the scope of flushing and thus it is the recommended
   approach because only CMAC addresses for the impacted service
   instances on the failed EVC are flushed.

5.3.  Port Failure Handling for Single-Active vES's vESes in EVPN

   When a large number of EVCs are aggregated via a single physical port
   on a PE; where each EVC corresponds to a vES, then the port failure
   impacts all the associated EVCs and their corresponding vES's. vESes.  If
   the number of EVCs corresponding to the Single-Active vES's vESes for that
   physical port is in thousands, then thousands of service instances
   are impacted.  Therefore, the BGP flush message need to be inclusive
   of all these impacted service instances.  In order to achieve this,
   the following extensions are added to the baseline EVPN mechanism:

   1)

   1.  When a PE advertises an Ether-AD Ethernet A-D per ES route for a given
       vES, it colors it with the MAC address of the physical port which
       is associated with that vES using EVPN Router's MAC Extended
       Community per [EVPN-IRB].  The receiving PEs take note of this
       color and create a list of vES's vESes for this color.

   2)

   2.  Upon a port failure (e.g., ENNI failure), the PE advertise a
       special mass-withdraw message with the MAC address of the failed
       port (i.e., the color of the port) encoded in the ESI field.  For
       this encoding, type 3 ESI (RFC7432 section 5) is used with the
       MAC field set to the MAC address of the port and the 3-octet
       local discriminator field set to 0xFFFFFF.  This mass-withdraw
       route is advertised with a list of Route Targets corresponding to
       the impacted service instances.  If the number of Route Targets
       is more than they can fit into a single attribute, then a set of
       Ethernet A-D per ES routes are advertised.

   3.  Upon a port failure (e.g., ENNI failure), the PE advertise a
       special mass-withdraw message with the MAC address of the failed
       port.

   4.  The remote PEs upon receiving this message, realize that this is a based on ESI Type 3
       and 0xFFFFFF Local Discrimnator values, detect the special vES
       mass-withdraw message and they message.  The remote PEs then access the list of
       the vES's for the specified color. Next, they initiate color created in (1) and initialte
       locally mass-withdraw procedure procedures for each of the vES's in the
       list.

   In scenarios where a logical ENNI is used the above procedure equally
   applies.  The logical ENNI is represented by type a Type 3 ESI and the MAC
   address used in the ENNI's ESI is used as a color for vES's vESes as
   described above.

5.4.  Port Failure Handling for Single-Active vES's vESes in PBB-EVPN

   When a large number of EVCs are aggregated via a single physical port
   on a PE; PE, where each EVC corresponds to a vES, then the port failure
   impacts all the associated EVCs and their corresponding vES's. vESes.  If
   the number of EVCs corresponding to the Single-Active vES's vESes for that
   physical port is in thousands, then thousands of service instances
   (I-SIDs) are impacted.  In such failure scenarios, the following two
   MAC flushing mechanisms per [RFC7623] can be performed.

   1)

   1.  If the MAC address of the physical port is used for PBB
       encapsulation as BMAC SA, then upon the port failure, the PE MUST
       use the EVPN MAC route withdrawal message to signal the flush.

   2)

   2.  If the PE shared MAC address is used for PBB encapsulation as
       BMAC SA, then upon the port failure, the PE MUST re-advertise
       this MAC route with the MAC Mobility Extended Community to signal
       the flush.

   The first method is recommended because it reduces the scope of
   flushing the most.

5.5.  Fast Convergence in PBB-EVPN (PBB-)EVPN

   As described above, when a large number of EVCs are aggregated via a
   physical port on a PE; PE, and where each EVC corresponds to a vES, then
   the port failure impacts all the associated EVCs and their
   corresponding
   vES's. vESes.  Two actions must be taken as the result of such
   port failure:

   - Flushing of

   o  For EVPN initiate mass-withdraw procedure for all vESes associated
      with the failed port and for PBB-EVPN flush all CMACs associated
      with the BMAC of the failed port
   for across all the impacted I-SIDs

   -

   o  DF election for all impacted vES's vESes associated with the failed port

   Section 5.3 already describes how perform mass-withdraw for all
   affected vESes using a single BGP advertisment.  Section 5.4
   describes how to flush CMAC address in the most optimum way - e.g.,
   to flush least number of CMAC addresses for the impacted I-SIDs.
   This section describes how to perform DF election in the most optimum
   way - e.g., to trigger DF election for all impacted vES's vESes (which can
   be in thousands) among the participating PEs via a single BGP message
   as opposed to sending thousands of BGP messages - one per vES.  This
   section assumes that the MAC flushing mechanism described in section
   5.4, bullet (1) is used.

   In order to devise such fast convergence mechanism that can be
   triggered via a single BGP message, all vES's vESes associated with a given
   physical port (e.g., ENNI) are colored with the same color
   representing that physical port.  The MAC address of the physical
   port is used for this coloring purposes and when the PE advertises an
   ES route for a vES associated with that physical port, it advertises
   it with an EVPN Router's MAC Extended Community indicating the color
   of that port.

   The receiving PEs take note of this color and for each such color,
   they create a list of vES's vESes associated with this color (i.e.,
   associated with this MAC address).  Now, when a port failure occurs,
   the impacted PE needs to notify the other PEs of this color so that
   these PEs can identify all the impacted vES's vESes associated with this
   color (from the above list) and re-execute DF election procedures for
   all the impacted vES's. vESes.  This is done by withdrawing the BMAC address
   associated with the failed port.

                     +-----+
          +----+     |     |       +---+
          | CE1|AC1--0=====0--ENNI1|   |  +-------+
          |    |AC2--0     |       |PE1|--|       |
          +----+     |\  ==0--ENNI2|   |  |       |
                     | \/  |       +---+  |       |
                     | /\  |              |IP/MPLS|
          +----+     |/  \ |       +---+  |Network|   +---+  +---+
          | CE2|AC4--0    =0--ENNI3|   |  |       |---|PE4|--|CE4|
          |    |AC4--0=====0--ENNI3|PE2|--|       |   +---+  +---+
          +----+     | ====0--ENNI3|   |  |       |
                     |/    |       +---+  |       |
                     0     |              |       |
          +----+    /|     |       +---+  |       |
          | CE3|AC5- |     |       |PE3|--|       |
          |    |AC6--0=====0--ENNI4|   |  +-------+
          +----+     |     |       +---+
                     +-----+

      Figure 5: Fast Convergence Upon ENNI Failure

   The following describes the procedure for coloring vES's vESes and fast
   convergence using this color in more details:

   1-

   1.  When a vES is configured, the PE colors the vES with the MAC
       address of the corresponding physical port and advertises the
       Ethernet Segment route for this vES with this color.

   2-

   2.  All other PEs (in the redundancy group) take note of this color
       and add the vES to the list for this color.

   3-

   3.  Upon the occurrence of a port failure (e.g., an ENNI failure),
       for PBB-EVPN the PE sends the flush message by withdrawing the
       BMAC address associated with the failed port and for EVPN, the PE
       withdraws Ethernet Segment route associated with the failed port.
       The PE should prioritize sending this flush message over ES route
       withdrawal messages of impacted vES's.

   4- vESes.

   4.  On reception of the flush message, other PEs use this info to
       flush their impacted CMACs MACs and to initiate DF election procedures
       across all their affected vES's.

   5- vESes.

   5.  The PE with the physical port failure (ENNI failure), also sends
   ES
       vES route withdrawal for every impacted vES's. vESes.  The other PEs
       upon receiving these messages, clear up their BGP tables.  It
       should be noted the ES vES route withdrawal messages are not used
       for executing DF election procedures by the receiving PEs.

6.  BGP Encoding

   This document defines one new BGP Extended Community for EVPN.

6.1.  I-SID Extended Community

   A new EVPN BGP Extended Community called I-SID is introduced.  This
   new extended community is a transitive extended community with the
   Type field of 0x06 (EVPN) and the Sub-Type of 0x07.

   The I-SID Extended Community is encoded as an 8-octet value as
   follows:

        0                   1                   2                   3
        0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       | Type=0x06     | Sub-Type=0x07 |        Base I-SID             |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |    Cont.      |            Bit Map (24 bits)                  |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

       Figure 6: I-SID Extended Community

   This extended community is used to indicate the list of I-SIDs
   associated with a given Ethernet Segment.

   24-bit map represents the next 24 I-SID after the base I-SID.  For
   example based I-SID of 10025 with 24-bit map of zero means, only a
   single I-SID of 10025.  I-SID of 10025 with bit map of 0x000001 means
   there are two I-SIDs, 10025 and 10026.

7.  Acknowledgements

   The authors would like to thanks Mei Zhang and Zhang, Jose Liste Liste, and Luc Andre
   Burdet for their reviews and feedbacks of this document.

8.  Security Considerations

   All the security considerations in [RFC7432] and [RFC7623] apply
   directly to this document because this document leverages the control
   and data plane procedures described in those documents.

   This document does not introduce any new security considerations
   beyond that of [RFC7432] and [RFC7623] because advertisements and
   processing of Ethernet Segment route for vES in this document follows
   that of physical ES in those RFCs.

9.  IANA Considerations

   IANA has allocated sub-type value 7 in the "EVPN Extended Community
   Sub-Types" registry defined in "https://www.iana.org/assignments/bgp-
   extended-communities/bgp-extended-communities.xhtml#evpn" as follows:

      SUB-TYPE   NAME           Reference
      ----   --------------   -------------
      0x07   I-SID Ext Comm   [draft-sajassi-bess-evpn-virtual-eth-segment]   [draft-ietf-bess-evpn-virtual-eth-segment]

   It is requested from IANA to update the reference to this document.

10.  Intellectual Property Considerations

   This document is being submitted for use in IETF standards
   discussions.

11.  References

11.1.  Normative References

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119, March 1997.

   [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC2119
   Key Words", BCP 14, RFC 8174,
              DOI 10.17487/RFC8174, May 2017. 10.17487/RFC2119, March 1997,
              <https://www.rfc-editor.org/info/rfc2119>.

   [RFC7432]  Sajassi, et al., A., Ed., Aggarwal, R., Bitar, N., Isaac, A.,
              Uttaro, J., Drake, J., and W. Henderickx, "BGP MPLS-Based
              Ethernet VPN", RFC 7432, DOI 10.17487/RFC7432, February 2015.
              2015, <https://www.rfc-editor.org/info/rfc7432>.

   [RFC7623]  Sajassi, et al., A., Ed., Salam, S., Bitar, N., Isaac, A., and W.
              Henderickx, "Provider Backbone Bridging Combined with
              Ethernet VPN (PBB-EVPN)", RFC 7623, DOI 10.17487/RFC7623,
              September 2015. 2015, <https://www.rfc-editor.org/info/rfc7623>.

   [RFC8174]  Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
              2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
              May 2017, <https://www.rfc-editor.org/info/rfc8174>.

   [RFC8214] Boutrus, et al.,  Boutros, S., Sajassi, A., Salam, S., Drake, J., and J.
              Rabadan, "Virtual Private Wire Service Support in Ethernet
              VPN", RFC 8214, DOI 10.17487/RFC8214, August 2017.

   [EVPN-IRB] Sajassi, et al., "Integrated Routing and Bridging in
   EVPN", draft-ietf-bess-evpn-inter-subnet-forwarding-05, July 2018.

12. 2017,
              <https://www.rfc-editor.org/info/rfc8214>.

11.2.  Informative References

   [RFC7209] Sajassi, et al., "Requirements for Ethernet VPN (EVPN)",
              RFC 7209, May 2014.

   [RFC7080]  Sajassi, A., Salam, S., Bitar, N., and F. Balus, "Virtual
              Private LAN Service (VPLS) Interoperability with Provider
              Backbone Bridges", RFC 7080, DOI 10.17487/RFC7080,
              December 2013.

13. 2013, <https://www.rfc-editor.org/info/rfc7080>.

   [RFC7209]  Sajassi, A., Aggarwal, R., Uttaro, J., Bitar, N.,
              Henderickx, W., and A. Isaac, "Requirements for Ethernet
              VPN (EVPN)", RFC 7209, DOI 10.17487/RFC7209, May 2014,
              <https://www.rfc-editor.org/info/rfc7209>.

Authors' Addresses

   Ali Sajassi
   Cisco Systems
   MILPITAS, CALIFORNIA 95035
   UNITED STATES

   Email: sajassi@cisco.com

   Patrice Brissette
   Cisco Systems

   Email: pbrisset@cisco.com

   Rick Schell
   Verizon

   Email: richard.schell@verizon.com
   John E Drake
   Juniper

   Email: jdrake@juniper.net

   Jorge Rabadan
   Nokia

   Email: jorge.rabadan@nokia.com