OpenVswitch developers ()

Ben Pfaff | 30 Jun 2012 07:40

[flow monitor 01/11] ofproto: Fix typos in comments.

This corrects the spelling of the OFOPERATION_* constants.

Signed-off-by: Ben Pfaff <blp@...>
---
 ofproto/ofproto.c |    6 +++---
 1 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/ofproto/ofproto.c b/ofproto/ofproto.c
index ce4da9d..5221318 100644
--- a/ofproto/ofproto.c
+++ b/ofproto/ofproto.c
 <at>  <at>  -117,9 +117,9  <at>  <at>  struct ofoperation {
     struct hmap_node hmap_node; /* In ofproto's "deletions" hmap. */
     struct rule *rule;          /* Rule being operated upon. */
     enum ofoperation_type type; /* Type of operation. */
-    struct rule *victim;        /* OFOPERATION_ADDING: Replaced rule. */
-    union ofp_action *actions;  /* OFOPERATION_MODIFYING: Replaced actions. */
-    int n_actions;              /* OFOPERATION_MODIFYING: # of old actions. */
+    struct rule *victim;        /* OFOPERATION_ADD: Replaced rule. */
+    union ofp_action *actions;  /* OFOPERATION_MODIFY: Replaced actions. */
+    int n_actions;              /* OFOPERATION_MODIFY: # of old actions. */
     ovs_be64 flow_cookie;       /* Rule's old flow cookie. */
 };

--

-- 
1.7.2.5

headers

Ben Pfaff | 30 Jun 2012 07:40

[flow monitor 03/11] nicira-ext: Fix wrong information in comment.

The priority of exact-match flows is ignored only for the OF1.0 flow match
format, never for NXM.

Signed-off-by: Ben Pfaff <blp@...>
---
 include/openflow/nicira-ext.h |    3 +--
 1 files changed, 1 insertions(+), 2 deletions(-)

diff --git a/include/openflow/nicira-ext.h b/include/openflow/nicira-ext.h
index a5d1b11..82deeb0 100644
--- a/include/openflow/nicira-ext.h
+++ b/include/openflow/nicira-ext.h
 <at>  <at>  -1887,8 +1887,7  <at>  <at>  struct nx_flow_stats {
     ovs_be32 duration_sec;    /* Time flow has been alive in seconds. */
     ovs_be32 duration_nsec;   /* Time flow has been alive in nanoseconds
                                  beyond duration_sec. */
-    ovs_be16 priority;        /* Priority of the entry. Only meaningful
-                                 when this is not an exact-match entry. */
+    ovs_be16 priority;        /* Priority of the entry. */
     ovs_be16 idle_timeout;    /* Number of seconds idle before expiration. */
     ovs_be16 hard_timeout;    /* Number of seconds before expiration. */
     ovs_be16 match_len;       /* Length of nx_match. */
--

-- 
1.7.2.5

headers

Justin Pettit | 6 Jul 2012 00:59

Re: [flow monitor 03/11] nicira-ext: Fix wrong information in comment.

Looks good.

--Justin

On Jun 29, 2012, at 10:40 PM, Ben Pfaff wrote:

> The priority of exact-match flows is ignored only for the OF1.0 flow match
> format, never for NXM.
> 
> Signed-off-by: Ben Pfaff <blp@...>
> ---
> include/openflow/nicira-ext.h |    3 +--
> 1 files changed, 1 insertions(+), 2 deletions(-)
> 
> diff --git a/include/openflow/nicira-ext.h b/include/openflow/nicira-ext.h
> index a5d1b11..82deeb0 100644
> --- a/include/openflow/nicira-ext.h
> +++ b/include/openflow/nicira-ext.h
>  <at>  <at>  -1887,8 +1887,7  <at>  <at>  struct nx_flow_stats {
>     ovs_be32 duration_sec;    /* Time flow has been alive in seconds. */
>     ovs_be32 duration_nsec;   /* Time flow has been alive in nanoseconds
>                                  beyond duration_sec. */
> -    ovs_be16 priority;        /* Priority of the entry. Only meaningful
> -                                 when this is not an exact-match entry. */
> +    ovs_be16 priority;        /* Priority of the entry. */
>     ovs_be16 idle_timeout;    /* Number of seconds idle before expiration. */
>     ovs_be16 hard_timeout;    /* Number of seconds before expiration. */
>     ovs_be16 match_len;       /* Length of nx_match. */
> -- 
> 1.7.2.5

(Continue reading)

headers

Ben Pfaff | 30 Jun 2012 07:40

[flow monitor 02/11] ovs-ofctl: Fix typos in manpage.

Signed-off-by: Ben Pfaff <blp@...>
---
 utilities/ovs-ofctl.8.in |    6 +++---
 1 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/utilities/ovs-ofctl.8.in b/utilities/ovs-ofctl.8.in
index 9c4ea0c..3ca217b 100644
--- a/utilities/ovs-ofctl.8.in
+++ b/utilities/ovs-ofctl.8.in
 <at>  <at>  -276,7 +276,7  <at>  <at>  If a switch has no controller configured, or if
 the configured controller is disconnected, no traffic is sent, so
 monitoring will not show any traffic.
 .
-.IP "\fBmonitor \fIswitch\fR [\fImiss-len\fR] [\fIinvalid_ttl\fR]"
+.IP "\fBmonitor \fIswitch\fR [\fImiss-len\fR] [\fBinvalid_ttl\fR]"
 Connects to \fIswitch\fR and prints to the console all OpenFlow
 messages received.  Usually, \fIswitch\fR should specify the name of a
 bridge in the \fBovs\-vswitchd\fR database.
 <at>  <at>  -292,8 +292,8  <at>  <at>  specified, very little traffic will ordinarily be printed.)
 .IP
 If \fBinvalid_ttl\fR is passed, \fBovs\-ofctl\fR sends an OpenFlow ``set
 configuration'' message at connection setup time that requests
-\fIINVALID_TTL_TO_CONTROLLER\fR, so that \fBovs\-ofctl monitor\fR can
-receive ``packets-in'' messages when TTL reaches zero on \fBdec_ttl\fR action.
+\fBINVALID_TTL_TO_CONTROLLER\fR, so that \fBovs\-ofctl monitor\fR can
+receive ``packet-in'' messages when TTL reaches zero on \fBdec_ttl\fR action.
 .IP

 This command may be useful for debugging switch or controller
--

--

(Continue reading)

headers

Justin Pettit | 6 Jul 2012 00:58

Re: [flow monitor 02/11] ovs-ofctl: Fix typos in manpage.

Looks good.

--Justin

On Jun 29, 2012, at 10:40 PM, Ben Pfaff wrote:

> Signed-off-by: Ben Pfaff <blp@...>
> ---
> utilities/ovs-ofctl.8.in |    6 +++---
> 1 files changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/utilities/ovs-ofctl.8.in b/utilities/ovs-ofctl.8.in
> index 9c4ea0c..3ca217b 100644
> --- a/utilities/ovs-ofctl.8.in
> +++ b/utilities/ovs-ofctl.8.in
>  <at>  <at>  -276,7 +276,7  <at>  <at>  If a switch has no controller configured, or if
> the configured controller is disconnected, no traffic is sent, so
> monitoring will not show any traffic.
> .
> -.IP "\fBmonitor \fIswitch\fR [\fImiss-len\fR] [\fIinvalid_ttl\fR]"
> +.IP "\fBmonitor \fIswitch\fR [\fImiss-len\fR] [\fBinvalid_ttl\fR]"
> Connects to \fIswitch\fR and prints to the console all OpenFlow
> messages received.  Usually, \fIswitch\fR should specify the name of a
> bridge in the \fBovs\-vswitchd\fR database.
>  <at>  <at>  -292,8 +292,8  <at>  <at>  specified, very little traffic will ordinarily be printed.)
> .IP
> If \fBinvalid_ttl\fR is passed, \fBovs\-ofctl\fR sends an OpenFlow ``set
> configuration'' message at connection setup time that requests
> -\fIINVALID_TTL_TO_CONTROLLER\fR, so that \fBovs\-ofctl monitor\fR can
> -receive ``packets-in'' messages when TTL reaches zero on \fBdec_ttl\fR action.

(Continue reading)

headers

Ben Pfaff | 30 Jun 2012 07:40

[flow monitor 04/11] ofproto: Make ofoperation_create() return the new operation.

This seems like sensible return value semantics to me, even though the new
operation is also available through rule->pending.

This is a code cleanup only that should not affect behavior.

Signed-off-by: Ben Pfaff <blp@...>
---
 ofproto/ofproto.c |   27 ++++++++++++++++++---------
 1 files changed, 18 insertions(+), 9 deletions(-)

diff --git a/ofproto/ofproto.c b/ofproto/ofproto.c
index 5221318..0e3009a 100644
--- a/ofproto/ofproto.c
+++ b/ofproto/ofproto.c
 <at>  <at>  -123,8 +123,9  <at>  <at>  struct ofoperation {
     ovs_be64 flow_cookie;       /* Rule's old flow cookie. */
 };

-static void ofoperation_create(struct ofopgroup *, struct rule *,
-                               enum ofoperation_type);
+static struct ofoperation *ofoperation_create(struct ofopgroup *,
+                                              struct rule *,
+                                              enum ofoperation_type);
 static void ofoperation_destroy(struct ofoperation *);

 /* oftable. */
 <at>  <at>  -2851,6 +2852,7  <at>  <at>  add_flow(struct ofproto *ofproto, struct ofconn *ofconn,
     } else if (victim && victim->pending) {
         error = OFPROTO_POSTPONE;
     } else {

(Continue reading)

headers

Justin Pettit | 6 Jul 2012 01:34

Re: [flow monitor 04/11] ofproto: Make ofoperation_create() return the new operation.

Looks good.

--Justin

On Jun 29, 2012, at 10:40 PM, Ben Pfaff wrote:

> This seems like sensible return value semantics to me, even though the new
> operation is also available through rule->pending.
> 
> This is a code cleanup only that should not affect behavior.
> 
> Signed-off-by: Ben Pfaff <blp@...>
> ---
> ofproto/ofproto.c |   27 ++++++++++++++++++---------
> 1 files changed, 18 insertions(+), 9 deletions(-)
> 
> diff --git a/ofproto/ofproto.c b/ofproto/ofproto.c
> index 5221318..0e3009a 100644
> --- a/ofproto/ofproto.c
> +++ b/ofproto/ofproto.c
>  <at>  <at>  -123,8 +123,9  <at>  <at>  struct ofoperation {
>     ovs_be64 flow_cookie;       /* Rule's old flow cookie. */
> };
> 
> -static void ofoperation_create(struct ofopgroup *, struct rule *,
> -                               enum ofoperation_type);
> +static struct ofoperation *ofoperation_create(struct ofopgroup *,
> +                                              struct rule *,
> +                                              enum ofoperation_type);
> static void ofoperation_destroy(struct ofoperation *);

(Continue reading)

headers

Ben Pfaff | 6 Jul 2012 19:21

Re: [flow monitor 04/11] ofproto: Make ofoperation_create() return the new operation.

Thanks.

I pushed these first four patches.

On Thu, Jul 05, 2012 at 04:34:12PM -0700, Justin Pettit wrote:
> Looks good.
> 
> --Justin
> 
> 
> On Jun 29, 2012, at 10:40 PM, Ben Pfaff wrote:
> 
> > This seems like sensible return value semantics to me, even though the new
> > operation is also available through rule->pending.
> > 
> > This is a code cleanup only that should not affect behavior.
> > 
> > Signed-off-by: Ben Pfaff <blp@...>
> > ---
> > ofproto/ofproto.c |   27 ++++++++++++++++++---------
> > 1 files changed, 18 insertions(+), 9 deletions(-)
> > 
> > diff --git a/ofproto/ofproto.c b/ofproto/ofproto.c
> > index 5221318..0e3009a 100644
> > --- a/ofproto/ofproto.c
> > +++ b/ofproto/ofproto.c
> >  <at>  <at>  -123,8 +123,9  <at>  <at>  struct ofoperation {
> >     ovs_be64 flow_cookie;       /* Rule's old flow cookie. */
> > };
> >

(Continue reading)

headers

Ben Pfaff | 30 Jun 2012 07:40

[flow monitor 05/11] ofproto: Finalize all ofoperations in a given ofgroup at the same time.

An upcoming commit will add support for flow table monitoring by
controllers.  One feature of this upcoming support is that a controller's
own changes to the flow table can be abbreviated to a summary, since the
controller presumably know what it has already sent to the switch.
However, the summary only makes sense if a set of flow table changes
completely succeeds or completely fails.  If it partially fails, the
switch must not attempt to summarize it, because the controller needs
to know the details.  Given that, we have to wait for all of the
operations in an ofgroup to either succeed or fail before the switch
can send its flow table update report to the controllers.  This
commit makes that change.

Signed-off-by: Ben Pfaff <blp@...>
---
 ofproto/ofproto.c |  184 +++++++++++++++++++++++++++++------------------------
 1 files changed, 102 insertions(+), 82 deletions(-)

diff --git a/ofproto/ofproto.c b/ofproto/ofproto.c
index 0e3009a..9f67fe0 100644
--- a/ofproto/ofproto.c
+++ b/ofproto/ofproto.c
 <at>  <at>  -86,6 +86,7  <at>  <at>  struct ofopgroup {
     struct ofproto *ofproto;    /* Owning ofproto. */
     struct list ofproto_node;   /* In ofproto's "pending" list. */
     struct list ops;            /* List of "struct ofoperation"s. */
+    int n_running;              /* Number of ops still pending. */

     /* Data needed to send OpenFlow reply on failure or to send a buffered
      * packet on success.
 <at>  <at>  -100,7 +101,6  <at>  <at>  struct ofopgroup {

(Continue reading)

headers

Ben Pfaff | 30 Jun 2012 07:40

[flow monitor 07/11] ofp-print: Slightly extend ofp_print_bit_names().

In an upcoming commit I want to print comma-separated, not space-separated,
names for a set of bits.

Signed-off-by: Ben Pfaff <blp@...>
---
 lib/ofp-print.c |   20 +++++++++++---------
 1 files changed, 11 insertions(+), 9 deletions(-)

diff --git a/lib/ofp-print.c b/lib/ofp-print.c
index 5103c3e..e6d9151 100644
--- a/lib/ofp-print.c
+++ b/lib/ofp-print.c
 <at>  <at>  -461,7 +461,8  <at>  <at>  compare_ports(const void *a_, const void *b_)

 static void
 ofp_print_bit_names(struct ds *string, uint32_t bits,
-                    const char *(*bit_to_name)(uint32_t bit))
+                    const char *(*bit_to_name)(uint32_t bit),
+                    char separator)
 {
     int n = 0;
     int i;
 <at>  <at>  -478,7 +479,7  <at>  <at>  ofp_print_bit_names(struct ds *string, uint32_t bits,
             const char *name = bit_to_name(bit);
             if (name) {
                 if (n++) {
-                    ds_put_char(string, ' ');
+                    ds_put_char(string, separator);
                 }
                 ds_put_cstr(string, name);

(Continue reading)

headers

Ben Pfaff | 30 Jun 2012 07:40

[flow monitor 06/11] classifier: New function cls_rule_is_loose_match().

This function will be useful in an upcoming commit.

Signed-off-by: Ben Pfaff <blp@...>
---
 lib/classifier.c |   76 ++++++++++++++++++++++++++++++++----------------------
 lib/classifier.h |    3 ++
 2 files changed, 48 insertions(+), 31 deletions(-)

diff --git a/lib/classifier.c b/lib/classifier.c
index 0541b5c..8a57b21 100644
--- a/lib/classifier.c
+++ b/lib/classifier.c
 <at>  <at>  -934,6 +934,46  <at>  <at>  classifier_rule_overlaps(const struct classifier *cls,

     return false;
 }
+
+/* Returns true if 'rule' exactly matches 'criteria' or are more specific than
+ * 'criteria'.  That is, 'rule' matches 'criteria' and this function returns
+ * true if, for every field:
+ *
+ *   - 'criteria' and 'rule' specify the same (non-wildcarded) value for the
+ *     field, or
+ *
+ *   - 'criteria' wildcards the field,
+ *
+ * but not if:
+ *
+ *   - 'criteria' and 'rule' specify different values for the field, or
+ *

(Continue reading)

headers

Ben Pfaff | 30 Jun 2012 07:40

[flow monitor 08/11] ofp-util: New function ofputil_postappend_stats_reply().

It's easier to make up stats replies if you don't have to accurately
predict the number of bytes that will be necessary.

This immediately allows us to rewrite ofputil_append_flow_stats_reply()
without using NXM_MAX_LEN, which in turn allows us to deleted NXM_MAX_LEN
entirely, which is nice since keeping it up-to-date is error-prone.

An upcoming commit will introduce a second user.

Signed-off-by: Ben Pfaff <blp@...>
---
 lib/nx-match.h |   40 ----------------------------------------
 lib/ofp-util.c |   45 +++++++++++++++++++++++++++++++++++++--------
 lib/ofp-util.h |    1 +
 3 files changed, 38 insertions(+), 48 deletions(-)

diff --git a/lib/nx-match.h b/lib/nx-match.h
index c814275..2b34804 100644
--- a/lib/nx-match.h
+++ b/lib/nx-match.h
 <at>  <at>  -90,46 +90,6  <at>  <at>  void nxm_decode(struct mf_subfield *, ovs_be32 header, ovs_be16 ofs_nbits);
 void nxm_decode_discrete(struct mf_subfield *, ovs_be32 header,
                          ovs_be16 ofs, ovs_be16 n_bits);
 
-BUILD_ASSERT_DECL(FLOW_WC_SEQ == 12);
-/* Upper bound on the length of an nx_match.  The longest nx_match (an
- * IPV6 neighbor discovery message using all the registers) would be:
- *
- *                   header  value  mask  total
- *                   ------  -----  ----  -----

(Continue reading)

headers

Ben Pfaff | 30 Jun 2012 07:40

[flow monitor 09/11] ofp-util: Make put_stats__() public as ofputil_put_stats_header().

An upcoming commit will introduce a user outside of ofp-util.c.

The change to put_stats_reply__() is larger than strictly necessary to
avoid making really ugly line breaks.

Signed-off-by: Ben Pfaff <blp@...>
---
 lib/ofp-util.c |   25 ++++++++++++++-----------
 lib/ofp-util.h |    4 ++++
 2 files changed, 18 insertions(+), 11 deletions(-)

diff --git a/lib/ofp-util.c b/lib/ofp-util.c
index 2c1dd47..b749389 100644
--- a/lib/ofp-util.c
+++ b/lib/ofp-util.c
 <at>  <at>  -3241,10 +3241,10  <at>  <at>  update_openflow_length(struct ofpbuf *buffer)
     oh->length = htons(buffer->size);
 }

-static void
-put_stats__(ovs_be32 xid, uint8_t ofp_type,
-            ovs_be16 ofpst_type, ovs_be32 nxst_subtype,
-            struct ofpbuf *msg)
+void
+ofputil_put_stats_header(ovs_be32 xid, uint8_t ofp_type,
+                         ovs_be16 ofpst_type, ovs_be32 nxst_subtype,
+                         struct ofpbuf *msg)
 {
     if (ofpst_type == htons(OFPST_VENDOR)) {
         struct nicira_stats_msg *nsm;

(Continue reading)

headers

Ben Pfaff | 30 Jun 2012 07:40

[flow monitor 10/11] ofproto: Refactor rule_has_out_port() in terms of new ofp-util function.

An upcoming commit will introduce a second user for the new function
actions_output_to_port().

Signed-off-by: Ben Pfaff <blp@...>
---
 lib/ofp-util.c    |   15 +++++++++++++++
 lib/ofp-util.h    |    3 +++
 ofproto/ofproto.c |   15 +++------------
 3 files changed, 21 insertions(+), 12 deletions(-)

diff --git a/lib/ofp-util.c b/lib/ofp-util.c
index b749389..6db215c 100644
--- a/lib/ofp-util.c
+++ b/lib/ofp-util.c
 <at>  <at>  -4058,6 +4058,21  <at>  <at>  action_outputs_to_port(const union ofp_action *action, ovs_be16 port)
     }
 }

+bool
+actions_output_to_port(const union ofp_action *actions, size_t n_actions,
+                       ovs_be16 port)
+{
+    const union ofp_action *oa;
+    size_t left;
+
+    OFPUTIL_ACTION_FOR_EACH_UNSAFE (oa, left, actions, n_actions) {
+        if (action_outputs_to_port(oa, port)) {
+            return true;
+        }
+    }

(Continue reading)

headers

Ben Pfaff | 30 Jun 2012 07:40

[flow monitor 11/11] ofproto: New feature to notify controllers of flow table changes.

OpenFlow switching monitoring and controller coordination can be made more
efficient if the switch can notify a controller of flow table changes as
they occur, rather than periodically polling for changes.  This commit
implements such a feature.

This commit is preliminary.  It lacks:

    - Unit tests.

    - An implementation of the NXT_FLOW_MONITOR_PAUSED and
      NXT_FLOW_MONITOR_RESUMED notifications, and the underlying buffer
      tracking (documented in nicira-ext.h).

    - Reason code reporting for removed flows.

However it does work in my simple manual testing, so it should be good
enough for initial review.

Feature #6633.
CC: Natasha Gude <natasha-l0M0P4e3n4LQT0dZR+AlfA <at> public.gmane.org>
Signed-off-by: Ben Pfaff <blp-l0M0P4e3n4LQT0dZR+AlfA <at> public.gmane.org>
---
 NEWS                          |    2 +
 include/openflow/nicira-ext.h |  238 +++++++++++++++++++++++++++++++++++-
 lib/learning-switch.c         |    5 +
 lib/ofp-errors.h              |   15 ++-
 lib/ofp-parse.c               |   62 ++++++++++
 lib/ofp-parse.h               |    4 +
 lib/ofp-print.c               |  142 +++++++++++++++++++++
 lib/ofp-util.c                |  271 +++++++++++++++++++++++++++++++++++++++++

(Continue reading)

headers

Justin Pettit | 6 Jul 2012 00:56

Re: [flow monitor 01/11] ofproto: Fix typos in comments.

Looks good.

--Justin

On Jun 29, 2012, at 10:40 PM, Ben Pfaff wrote:

> This corrects the spelling of the OFOPERATION_* constants.
> 
> Signed-off-by: Ben Pfaff <blp@...>
> ---
> ofproto/ofproto.c |    6 +++---
> 1 files changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/ofproto/ofproto.c b/ofproto/ofproto.c
> index ce4da9d..5221318 100644
> --- a/ofproto/ofproto.c
> +++ b/ofproto/ofproto.c
>  <at>  <at>  -117,9 +117,9  <at>  <at>  struct ofoperation {
>     struct hmap_node hmap_node; /* In ofproto's "deletions" hmap. */
>     struct rule *rule;          /* Rule being operated upon. */
>     enum ofoperation_type type; /* Type of operation. */
> -    struct rule *victim;        /* OFOPERATION_ADDING: Replaced rule. */
> -    union ofp_action *actions;  /* OFOPERATION_MODIFYING: Replaced actions. */
> -    int n_actions;              /* OFOPERATION_MODIFYING: # of old actions. */
> +    struct rule *victim;        /* OFOPERATION_ADD: Replaced rule. */
> +    union ofp_action *actions;  /* OFOPERATION_MODIFY: Replaced actions. */
> +    int n_actions;              /* OFOPERATION_MODIFY: # of old actions. */
>     ovs_be64 flow_cookie;       /* Rule's old flow cookie. */
> };
>

(Continue reading)

headers

Ben Pfaff | 6 Jul 2012 01:00

Re: [flow monitor 01/11] ofproto: Fix typos in comments.

Thanks.

By the way, I'm going to send out a new version of patches 10 and 11,
possibly today, so don't bother reviewing those yet.  Patch 8 already
got committed earlier today (there was another series that depended on
it), so you can skip that one too.  The rest of the series won't
really change.

On Thu, Jul 05, 2012 at 03:56:45PM -0700, Justin Pettit wrote:
> Looks good.
> 
> --Justin
> 
> 
> On Jun 29, 2012, at 10:40 PM, Ben Pfaff wrote:
> 
> > This corrects the spelling of the OFOPERATION_* constants.
> > 
> > Signed-off-by: Ben Pfaff <blp@...>
> > ---
> > ofproto/ofproto.c |    6 +++---
> > 1 files changed, 3 insertions(+), 3 deletions(-)
> > 
> > diff --git a/ofproto/ofproto.c b/ofproto/ofproto.c
> > index ce4da9d..5221318 100644
> > --- a/ofproto/ofproto.c
> > +++ b/ofproto/ofproto.c
> >  <at>  <at>  -117,9 +117,9  <at>  <at>  struct ofoperation {
> >     struct hmap_node hmap_node; /* In ofproto's "deletions" hmap. */
> >     struct rule *rule;          /* Rule being operated upon. */

(Continue reading)