headers
Al Morton | 9 Apr 2012 21:08
Picon
Favicon

Comments on draft-ietf-bmwg-ca-bench-meth-01

Mike and Sarah,

Comments below, some are purely editorial.

Al
(as a participant)

-=-=-=-=-=-=-

Section 1 Intro
...
    The end goal of this methodology is to generate performance metrics
    in a lab environment that will more closely relate to actual observed
s/more closely/closely/

    performance on production networks.  By utilizing dynamic traffic
    patterns relevant to modern networks, this methodology should be able
    to more closely tie laboratory and production metrics.
s/more closely/closely/

and a question about:
   ... It should be
    further noted than any metrics acquired from production networks
    SHOULD be captured according to the policies and procedures of the
    IPPM or PMOL working groups.

What metrics are you thinking about here?

-=-=-=-=-=-=-
(Continue reading)

Permalink | Reply |
headers
Barry Constantine | 10 Apr 2012 23:08

Comments on draft-ietf-bmwg-ca-bench-meth-01.txt

Hi Mike / Sarah,

 

The document is really coming along nicely.

 

Here are some comments / suggestions I had:

 

4.1.4.3.  Packet Loss

 

   The test tool SHOULD report the number of flow packets lost or

   dropped from source to destination.

 

>>>>>>>>>>>>

 

   The TCP Efficiency metric from RFC 6349 could be beneficial in this area.

   Specifically TCP Efficiency is the ratio of:

 

   total transmitted bytes - retransmitted bytes

   ------------------------------------------------------  x 100

                total transmitted bytes

  

  Ideal score is 100%.

 

   With TCP based application flows, it is the number of retransmitted bytes

   that is more telling than the number of lost packets, and the efficiency

   metric helps to quantify the overall performance.

  

   So I would consider measuring packet loss and TCP Efficiency (and on a per

   flow basis would be very useful as well).

  

   

 4.1.4.4.  Application Flow Latency

 

   The test tool SHOULD report the minimum, maximum and average amount

   of time an application flow member takes to traverse the DUT, as

   defined by RFC 1242 [3], Section 3.8.  This rate SHOULD be reported

   individually for each application protocol present within the traffic

   mix.

 

>>>>>>>>>>>>>>

 

   Similarly, the Buffer Delay Percent from RFC 6349 could be useful here.

   Buffer Delay =

  

   RTT (under load) - RTT baseline

   --------------------------------------- x 100

            RTT baseline

                                            

Ideal = 0%

 

Versus just min, max, and average, Buffer Delay percent (in combination with

TCP Efficiency) can segment performance issues into the loss related issues versus

device buffering performance issues.

              

I was involved with a benchmark of WAN Acceleration devices recently, and the

vendor found these metrics to be very beneficial for the reasons I stated above.

 

Thank you,

Barry Constantine

 

 

_______________________________________________
bmwg mailing list
bmwg <at> ietf.org
https://www.ietf.org/mailman/listinfo/bmwg
Permalink | Reply |

Gmane