draft-ietf-ippm-reporting-metrics-05.txt   draft-ietf-ippm-reporting-metrics-06.txt 
Network Working Group A. Morton Network Working Group A. Morton
Internet-Draft G. Ramachandran Internet-Draft G. Ramachandran
Intended status: Informational G. Maguluri Intended status: Informational G. Maguluri
Expires: January 8, 2012 AT&T Labs Expires: July 10, 2012 AT&T Labs
July 7, 2011 January 7, 2012
Reporting Metrics: Different Points of View Reporting Metrics: Different Points of View
draft-ietf-ippm-reporting-metrics-05 draft-ietf-ippm-reporting-metrics-06
Abstract Abstract
Consumers of IP network performance metrics have many different uses Consumers of IP network performance metrics have many different uses
in mind. The memo provides "long-term" reporting considerations in mind. The memo provides "long-term" reporting considerations
(e.g, days, weeks or months, as opposed to 10 seconds), based on (e.g, days, weeks or months, as opposed to 10 seconds), based on
analysis of the two key audience points-of-view. It describes how analysis of the two key audience points-of-view. It describes how
the audience categories affect the selection of metric parameters and the audience categories affect the selection of metric parameters and
options when seeking info that serves their needs. options when seeking info that serves their needs.
skipping to change at page 1, line 42 skipping to change at page 1, line 42
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/. Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on January 8, 2012. This Internet-Draft will expire on July 10, 2012.
Copyright Notice Copyright Notice
Copyright (c) 2011 IETF Trust and the persons identified as the Copyright (c) 2012 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
skipping to change at page 3, line 36 skipping to change at page 3, line 36
5.3. Summary for Delay . . . . . . . . . . . . . . . . . . . . 15 5.3. Summary for Delay . . . . . . . . . . . . . . . . . . . . 15
6. Effect of POV on Raw Capacity Metrics . . . . . . . . . . . . 15 6. Effect of POV on Raw Capacity Metrics . . . . . . . . . . . . 15
6.1. Type-P Parameter . . . . . . . . . . . . . . . . . . . . . 15 6.1. Type-P Parameter . . . . . . . . . . . . . . . . . . . . . 15
6.2. a priori Factors . . . . . . . . . . . . . . . . . . . . . 16 6.2. a priori Factors . . . . . . . . . . . . . . . . . . . . . 16
6.3. IP-layer Capacity . . . . . . . . . . . . . . . . . . . . 16 6.3. IP-layer Capacity . . . . . . . . . . . . . . . . . . . . 16
6.4. IP-layer Utilization . . . . . . . . . . . . . . . . . . . 17 6.4. IP-layer Utilization . . . . . . . . . . . . . . . . . . . 17
6.5. IP-layer Available Capacity . . . . . . . . . . . . . . . 17 6.5. IP-layer Available Capacity . . . . . . . . . . . . . . . 17
6.6. Variability in Utilization and Avail. Capacity . . . . . . 18 6.6. Variability in Utilization and Avail. Capacity . . . . . . 18
6.6.1. General Summary of Variability . . . . . . . . . . . . 18 6.6.1. General Summary of Variability . . . . . . . . . . . . 18
7. Effect of POV on Restricted Capacity Metrics . . . . . . . . . 19 7. Effect of POV on Restricted Capacity Metrics . . . . . . . . . 19
7.1. Type-P Parameter and Type-C Parameter . . . . . . . . . . 19 7.1. Type-P Parameter and Type-C Parameter . . . . . . . . . . 20
7.2. a priori Factors . . . . . . . . . . . . . . . . . . . . . 20 7.2. a priori Factors . . . . . . . . . . . . . . . . . . . . . 20
7.3. Measurement Interval . . . . . . . . . . . . . . . . . . . 20 7.3. Measurement Interval . . . . . . . . . . . . . . . . . . . 20
7.4. Bulk Transfer Capacity Reporting . . . . . . . . . . . . . 21 7.4. Bulk Transfer Capacity Reporting . . . . . . . . . . . . . 21
7.5. Variability in Bulk Transfer Capacity . . . . . . . . . . 21 7.5. Variability in Bulk Transfer Capacity . . . . . . . . . . 22
8. Test Streams and Sample Size . . . . . . . . . . . . . . . . . 22 8. Test Streams and Sample Size . . . . . . . . . . . . . . . . . 22
8.1. Test Stream Characteristics . . . . . . . . . . . . . . . 22 8.1. Test Stream Characteristics . . . . . . . . . . . . . . . 22
8.2. Sample Size . . . . . . . . . . . . . . . . . . . . . . . 22 8.2. Sample Size . . . . . . . . . . . . . . . . . . . . . . . 23
9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 23 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 23
10. Security Considerations . . . . . . . . . . . . . . . . . . . 23 10. Security Considerations . . . . . . . . . . . . . . . . . . . 23
11. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 23 11. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 23
12. References . . . . . . . . . . . . . . . . . . . . . . . . . . 23 12. References . . . . . . . . . . . . . . . . . . . . . . . . . . 24
12.1. Normative References . . . . . . . . . . . . . . . . . . . 23 12.1. Normative References . . . . . . . . . . . . . . . . . . . 24
12.2. Informative References . . . . . . . . . . . . . . . . . . 24 12.2. Informative References . . . . . . . . . . . . . . . . . . 25
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 25 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 25
1. Introduction 1. Introduction
When designing measurements of IP networks and presenting the When designing measurements of IP networks and presenting the
results, knowledge of the audience is a key consideration. To results, knowledge of the audience is a key consideration. To
present a useful and relevant portrait of network conditions, one present a useful and relevant portrait of network conditions, one
must answer the following question: must answer the following question:
"How will the results be used?" "How will the results be used?"
skipping to change at page 18, line 20 skipping to change at page 18, line 20
capacity. capacity.
6.6. Variability in Utilization and Avail. Capacity 6.6. Variability in Utilization and Avail. Capacity
As with most metrics and measurements, assessing the consistency or As with most metrics and measurements, assessing the consistency or
variability in the results gives a the user an intuitive feel for the variability in the results gives a the user an intuitive feel for the
degree (or confidence) that any one value is representative of other degree (or confidence) that any one value is representative of other
results, or the underlying distribution from which these singleton results, or the underlying distribution from which these singleton
measurements have come. measurements have come.
Two questions are raised here for further discussion:
What ways can Utilization be measured and summarized to describe the What ways can Utilization be measured and summarized to describe the
potential variability in a useful way? potential variability in a useful way?
How can the variability in Available Capacity estimates be reported, How can the variability in Available Capacity estimates be reported,
so that the confidence in the results is also conveyed? so that the confidence in the results is also conveyed?
Proposal for Discussion: We suggest some methods below:
6.6.1. General Summary of Variability 6.6.1. General Summary of Variability
With a set of singleton Utilization or Available Capacity estimates, With a set of singleton Utilization or Available Capacity estimates,
each representing a minimum time to ascertain the estimate, we each representing a time interval needed to ascertain the estimate,
propose to describe the variation over the set of singletons as we seek to describe the variation over the set of singletons as
though reporting summary statistics of a distribution. Four useful though reporting summary statistics of a distribution. Three useful
summary statistics are: summary statistics are:
o Minimum, Maximum, and the Range they define o Minimum,
o Mode o Maximum,
For an on-going series of singleton estimates, we propose a moving o Range
average of n estimates to provide a single value estimate to more
easily distinguish substantial changes in performance over time. For An alternate way to represent the Range is as ratio of Maximum to
Minimum value. This enables an easily understandable statistic to
describe the range observed. For example, when Maximum = 3*Minimum,
then the Max/Min Ratio is 3 and users may see variability of this
order. On the other hand, Capacity estimates with a Max/Min Ratio
near 1 are quite consistent and near the central measure or statistic
reported.
For an on-going series of singleton estimates, a moving average of n
estimates may provide a single value estimate to more easily
distinguish substantial changes in performance over time. For
example, in a window of n singletons observed in time interval, t, a example, in a window of n singletons observed in time interval, t, a
percentage change of x% is declared to be a submstantial change and percentage change of x% is declared to be a substantial change and
reported as an exception. reported as an exception.
Often, the most informative summary of the results is a two-axis plot
rather than a table of statistics, where time is plotted on the
x-axis and the singleton value on the y-axis. The time-series plot
can illustrate sudden changes in an otherwise stable range, identify
bi-modality easily, and help quickly assess correlation with other
time-series. Plots of frequency of the singleton values are likewise
useful tools to visualize the variation.
7. Effect of POV on Restricted Capacity Metrics 7. Effect of POV on Restricted Capacity Metrics
This section describes the ways that restricted capacity metrics can This section describes the ways that restricted capacity metrics can
be tuned to reflect the preferences of the two audiences, or be tuned to reflect the preferences of the two audiences, or
different Points-of-View (POV). Raw capacity refers to the metrics different Points-of-View (POV). Raw capacity refers to the metrics
defined in [RFC3148] which include restrictions such as data defined in [RFC3148] which include restrictions such as data
uniqueness or flow-control response to congestion. uniqueness or flow-control response to congestion.
In primary metric considered is Bulk Transfer Capacity (BTC) for In primary metric considered is Bulk Transfer Capacity (BTC) for
complete paths. [RFC3148] defines complete paths. [RFC3148] defines
skipping to change at page 21, line 50 skipping to change at page 22, line 15
long as the measurable maximum exceeds the target capacity. long as the measurable maximum exceeds the target capacity.
7.5. Variability in Bulk Transfer Capacity 7.5. Variability in Bulk Transfer Capacity
As with most metrics and measurements, assessing the consistency or As with most metrics and measurements, assessing the consistency or
variability in the results gives a the user an intuitive feel for the variability in the results gives a the user an intuitive feel for the
degree (or confidence) that any one value is representative of other degree (or confidence) that any one value is representative of other
results, or the underlying distribution from which these singleton results, or the underlying distribution from which these singleton
measurements have come. measurements have come.
Two questions are raised here for further discussion: With two questions looming:
What ways can BTC be measured and summarized to describe the 1. What ways can BTC be measured and summarized to describe the
potential variability in a useful way? potential variability in a useful way?
How can the variability in BTC estimates be reported, so that the 2. How can the variability in BTC estimates be reported, so that the
confidence in the results is also conveyed? confidence in the results is also conveyed?
we suggest the methods of Section 6.6.1 above, and the additional
results presentations given in [RFC6349].
8. Test Streams and Sample Size 8. Test Streams and Sample Size
This section discusses two key aspects of measurement that are This section discusses two key aspects of measurement that are
sometimes omitted from the report: the description of the test stream sometimes omitted from the report: the description of the test stream
on which the measurements are based, and the sample size. on which the measurements are based, and the sample size.
8.1. Test Stream Characteristics 8.1. Test Stream Characteristics
Network Characterization has traditionally used Poisson-distributed Network Characterization has traditionally used Poisson-distributed
skipping to change at page 25, line 5 skipping to change at page 25, line 25
Shalunov, S. and M. Swany, "Reporting IP Performance Shalunov, S. and M. Swany, "Reporting IP Performance
Metrics to Users", draft-ietf-ippm-reporting-06 (work in Metrics to Users", draft-ietf-ippm-reporting-06 (work in
progress), March 2011. progress), March 2011.
[RFC5481] Morton, A. and B. Claise, "Packet Delay Variation [RFC5481] Morton, A. and B. Claise, "Packet Delay Variation
Applicability Statement", RFC 5481, March 2009. Applicability Statement", RFC 5481, March 2009.
[RFC5835] Morton, A. and S. Van den Berghe, "Framework for Metric [RFC5835] Morton, A. and S. Van den Berghe, "Framework for Metric
Composition", RFC 5835, April 2010. Composition", RFC 5835, April 2010.
[RFC6349] Constantine, B., Forget, G., Geib, R., and R. Schrage,
"Framework for TCP Throughput Testing", RFC 6349,
August 2011.
[Y.1540] ITU-T Recommendation Y.1540, "Internet protocol data [Y.1540] ITU-T Recommendation Y.1540, "Internet protocol data
communication service - IP packet transfer and communication service - IP packet transfer and
availability performance parameters", December 2002. availability performance parameters", December 2002.
[Y.1541] ITU-T Recommendation Y.1540, "Network Performance [Y.1541] ITU-T Recommendation Y.1540, "Network Performance
Objectives for IP-Based Services", February 2006. Objectives for IP-Based Services", February 2006.
Authors' Addresses Authors' Addresses
Al Morton Al Morton
 End of changes. 20 change blocks. 
28 lines changed or deleted 51 lines changed or added

This html diff was produced by rfcdiff 1.41. The latest version is available from http://tools.ietf.org/tools/rfcdiff/