David Koscinski | 30 Jan 18:22 2012
Picon

MARK accounting packet counts do not match mangle or tc

I am using complex traffic shaping and marking traffic with MARK 1 through 5.
Then I am using accounting to detect the MARKs and keep counts of each so that I can see that my traffic shaping is doing what I want.

I am finding that the accounting packet count is often 0 when the corresponding "shorewall show tc" piority does have a packet count.

Here is my tcclasses:
#INTERFACE      MARK    RATE            CEIL            PRIORITY        OPTIONS
#$NET_IF = eth0

$NET_IF         1       400kbit         full            1               tos=0x68/0xfc,tos=0xb8/0xfc     # voip: N trunks <at> 80kbit per trunk : at least 400kbit for 5 trunks.  Here 5% of 10mbit is 500kbit.
$NET_IF         2       full*10/100     full            2               tcp-ack,tos-minimize-delay      # interactive traffic
$NET_IF         3       full*10/100     full            3                                               # vpn traffic (encrypted)
$NET_IF         4       full*60/100     full            4               default                         # default
$NET_IF         5       full*10/100     full*95/100     5                                               # backups and other low priority stuff

Here are my tc and accounting results.  Notice how the tc packet count for priority 3 (which is mark 3) is 23477 whereas the accounting packet count for mark 3 is 0.  Conversely notice how tc packet count for priority 5 (which is mark 5) is 0 whereas the accounting packet count for mark 5 is 17130.  The counts for priority 1 pretty closely match the accounting counts for mark 1.


# shorewall show tc | tail -55 | head -35;shorewall show tc_0 tc_1 tc_2 tc_3 tc_4 tc_5
class htb 1:11 parent 1:1 leaf 2: prio 1 quantum 2000 rate 400000bit ceil 5000Kbit burst 1800b/8 mpu 0b overhead 0b cburst 4Kb/8 mpu 0b overhead 0b level 0
 Sent 7884354 bytes 37911 pkt (dropped 0, overlimits 0 requeues 0)
 rate 240bit 0pps backlog 0b 0p requeues 0
 lended: 37911 borrowed: 0 giants: 0
 tokens: 34720 ctokens: 6458

class htb 1:1 root rate 5000Kbit ceil 5000Kbit burst 4Kb/8 mpu 0b overhead 0b cburst 4Kb/8 mpu 0b overhead 0b level 7
 Sent 31397414 bytes 251481 pkt (dropped 0, overlimits 0 requeues 0)
 rate 107728bit 71pps backlog 0b 0p requeues 0
 lended: 1727 borrowed: 0 giants: 0
 tokens: 5959 ctokens: 5959

class htb 1:13 parent 1:1 leaf 4: prio 3 quantum 2500 rate 500000bit ceil 5000Kbit burst 1850b/8 mpu 0b overhead 0b cburst 4Kb/8 mpu 0b overhead 0b level 0
 Sent 7805243 bytes 23477 pkt (dropped 0, overlimits 0 requeues 0)
 rate 22088bit 13pps backlog 0b 0p requeues 0
 lended: 21781 borrowed: 1696 giants: 0
 tokens: 23584 ctokens: 5959

class htb 1:12 parent 1:1 leaf 3: prio 2 quantum 2500 rate 500000bit ceil 5000Kbit burst 1850b/8 mpu 0b overhead 0b cburst 4Kb/8 mpu 0b overhead 0b level 0
 Sent 9658166 bytes 169508 pkt (dropped 0, overlimits 0 requeues 0)
 rate 16104bit 33pps backlog 0b 0p requeues 0
 lended: 169493 borrowed: 4 giants: 0
 tokens: 28064 ctokens: 6407

class htb 1:15 parent 1:1 leaf 6: prio 5 quantum 2500 rate 500000bit ceil 4750Kbit burst 1850b/8 mpu 0b overhead 0b cburst 3974b/8 mpu 0b overhead 0b level 0
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 rate 0bit 0pps backlog 0b 0p requeues 0
 lended: 0 borrowed: 0 giants: 0
 tokens: 29600 ctokens: 6694

class htb 1:14 parent 1:1 leaf 5: prio 4 quantum 15000 rate 3000Kbit ceil 5000Kbit burst 3099b/8 mpu 0b overhead 0b cburst 4Kb/8 mpu 0b overhead 0b level 0
 Sent 6049651 bytes 20585 pkt (dropped 0, overlimits 0 requeues 0)
 rate 69296bit 25pps backlog 0b 0p requeues 0
 lended: 20558 borrowed: 27 giants: 0
 tokens: 8138 ctokens: 6484
Shorewall 4.4.12.1 Chains tc_0 tc_1 tc_2 tc_3 tc_4 tc_5 at gw-cary.corp.ibcengineering.com - Mon Jan 30 11:10:59 CST 2012

Counters reset Mon Jan 30 10:14:52 CST 2012

Chain tc_0 (2 references)
 pkts bytes target     prot opt in     out     source               destination
 389K  495M            all  --  eth0   *       0.0.0.0/0            0.0.0.0/0           MARK match 0x0/0xff
 213K   21M            all  --  *      eth0    0.0.0.0/0            0.0.0.0/0           MARK match 0x0/0xff

Chain tc_1 (2 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0            all  --  eth0   *       0.0.0.0/0            0.0.0.0/0           MARK match 0x1/0xff
37909 7353K            all  --  *      eth0    0.0.0.0/0            0.0.0.0/0           MARK match 0x1/0xff

Chain tc_2 (2 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0            all  --  eth0   *       0.0.0.0/0            0.0.0.0/0           MARK match 0x2/0xff
   49  8504            all  --  *      eth0    0.0.0.0/0            0.0.0.0/0           MARK match 0x2/0xff

Chain tc_3 (2 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0            all  --  eth0   *       0.0.0.0/0            0.0.0.0/0           MARK match 0x3/0xff
    0     0            all  --  *      eth0    0.0.0.0/0            0.0.0.0/0           MARK match 0x3/0xff

Chain tc_4 (2 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0            all  --  eth0   *       0.0.0.0/0            0.0.0.0/0           MARK match 0x4/0xff
    0     0            all  --  *      eth0    0.0.0.0/0            0.0.0.0/0           MARK match 0x4/0xff

Chain tc_5 (2 references)
 pkts bytes target     prot opt in     out     source               destination
17130 2652K            all  --  eth0   *       0.0.0.0/0            0.0.0.0/0           MARK match 0x5/0xff
    0     0            all  --  *      eth0    0.0.0.0/0            0.0.0.0/0           MARK match 0x5/0xff


Here are my shorewall capabilities that are lacking:
# shorewall show capabilities|grep Not
   Extended Connection Tracking Match Support: Not available
   IPP2P Match: Not available
   Repeat match: Not available
   Extended MARK Target 2: Not available
   Time Match: Not available
   LOGMARK Target: Not available
   IPMARK Target: Not available
   Persistent SNAT: Not available
   TPROXY Target: Not available
   FLOW Classifier: Not available
   fwmark route mask: Not available

Do I misunderstand the capabilities of the MARK column in the accounting table?  Or have I misconfigured something?

Thanks for the help.

------------------------------------------------------------------------------
Try before you buy = See our experts in action!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-dev2
------------------------------------------------------------------------------
Try before you buy = See our experts in action!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-dev2
Tom Eastep | 30 Jan 19:19 2012
Picon

Re: MARK accounting packet counts do not match mangle or tc

On Mon, 2012-01-30 at 11:22 -0600, David Koscinski wrote:

> Do I misunderstand the capabilities of the MARK column in the
> accounting table?  Or have I misconfigured something?

It's not possible to say, given what you have told us. 

1. Which chain(s) are you doing your TC marking in?
2. It appears that you are doing your accounting in the filter table, is
that correct? (Shorewall also allows you to do accounting in the
mangle).

I suspect that you are marking packets after they have been through
accounting; that would explain what you are seeing. You may wish to
refer to the diagram at http://www.shorewall.net/NetfilterOverview.html.

-Tom
-- 
Tom Eastep        \ When I die, I want to go like my Grandfather who
Shoreline,         \ died peacefully in his sleep. Not screaming like
Washington, USA     \ all of the passengers in his car
http://shorewall.net \________________________________________________

------------------------------------------------------------------------------
Try before you buy = See our experts in action!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-dev2
------------------------------------------------------------------------------
Try before you buy = See our experts in action!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-dev2
David Koscinski | 30 Jan 19:59 2012
Picon

Re: MARK accounting packet counts do not match mangle or tc

Thanks Tom.

I have MARK_IN_FORWARD_CHAIN=Yes


In the case of mark/priority 3 I am marking in the POSTROUTING chain:
3:T     0.0.0.0/0       0.0.0.0/0       udp     1194   # openvpn

For mark/priorty 5 I am marking in the FORWARD chains:
5:F     67.52.58.192/28 0.0.0.0/0       tcp     22,10000:10099  # NATed ssh
5:F     0.0.0.0/0       67.52.58.192/28 tcp     -       22,10000:10099  # NATed ssh

I don't understand you comment about doing accounting in the filter table vs. the mangle table.   Perhaps this information is what you are looking for:
I configure my accounting rules using the /etc/shorewall/accounting file and I have no SECTION directives.  I use the shorewall show accounting and shorewall show tc_0 tc_1 tc_2 tc_3 tc_5 tc_5 commands to check the counters.  Here are my accounting rules:
tc_0:COUNT      -               $NET_IF                 -                               -       -       -       -       0
tc_0:COUNT      -               -                       $NET_IF                         -       -       -       -       0

tc_1:COUNT      -               $NET_IF                 -                               -       -       -       -       1
tc_1:COUNT      -               -                       $NET_IF                         -       -       -       -       1

tc_2:COUNT      -               $NET_IF                 -                               -       -       -       -       2
tc_2:COUNT      -               -                       $NET_IF                         -       -       -       -       2

tc_3:COUNT      -               $NET_IF                 -                               -       -       -       -       3
tc_3:COUNT      -               -                       $NET_IF                         -       -       -       -       3

tc_4:COUNT      -               $NET_IF                 -                               -       -       -       -       4
tc_4:COUNT      -               -                       $NET_IF                         -       -       -       -       4

tc_5:COUNT      -               $NET_IF                 -                               -       -       -       -       5
tc_5:COUNT      -               -                       $NET_IF                         -       -       -       -       5

tc_6:COUNT      -               $NET_IF                 -                               -       -       -       -       6
tc_6:COUNT      -               -                       $NET_IF                         -       -       -       -       6

I also use shorewall show mangle to see how my tcrules are being applied, but since mangle includes intermediate results, I am trying to use shorewall show accounting to see the final mark/priority results.

I realize that the final results are in the shorewall show tc output, but I currently use accounting data to generate graphs showing how traffic is being used.  I am trying to add another type of graph that shows the traffic per priority.  So I am hoping to prepare /etc/shorewall/accounting rules that let me see the same numbers I would get from shorewall show tc.

Cheers,

david.

On Mon, Jan 30, 2012 at 12:19 PM, Tom Eastep <teastep <at> shorewall.net> wrote:
On Mon, 2012-01-30 at 11:22 -0600, David Koscinski wrote:

> Do I misunderstand the capabilities of the MARK column in the
> accounting table?  Or have I misconfigured something?

It's not possible to say, given what you have told us.

1. Which chain(s) are you doing your TC marking in?
2. It appears that you are doing your accounting in the filter table, is
that correct? (Shorewall also allows you to do accounting in the
mangle).

I suspect that you are marking packets after they have been through
accounting; that would explain what you are seeing. You may wish to
refer to the diagram at http://www.shorewall.net/NetfilterOverview.html.

-Tom
--
Tom Eastep        \ When I die, I want to go like my Grandfather who
Shoreline,         \ died peacefully in his sleep. Not screaming like
Washington, USA     \ all of the passengers in his car
http://shorewall.net \________________________________________________



------------------------------------------------------------------------------
Try before you buy = See our experts in action!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-dev2
_______________________________________________
Shorewall-users mailing list
Shorewall-users <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/shorewall-users


------------------------------------------------------------------------------
Try before you buy = See our experts in action!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-dev2
------------------------------------------------------------------------------
Try before you buy = See our experts in action!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-dev2
David Koscinski | 30 Jan 20:34 2012
Picon

Re: MARK accounting packet counts do not match mangle or tc


On Mon, Jan 30, 2012 at 12:19 PM, Tom Eastep <teastep <at> shorewall.net> wrote:
On Mon, 2012-01-30 at 11:22 -0600, David Koscinski wrote:

> Do I misunderstand the capabilities of the MARK column in the
> accounting table?  Or have I misconfigured something?

It's not possible to say, given what you have told us.

1. Which chain(s) are you doing your TC marking in?
2. It appears that you are doing your accounting in the filter table, is
that correct? (Shorewall also allows you to do accounting in the
mangle).

I suspect that you are marking packets after they have been through
accounting; that would explain what you are seeing. You may wish to
refer to the diagram at http://www.shorewall.net/NetfilterOverview.html.

-Tom
--
Tom Eastep        \ When I die, I want to go like my Grandfather who
Shoreline,         \ died peacefully in his sleep. Not screaming like
Washington, USA     \ all of the passengers in his car
http://shorewall.net \________________________________________________



------------------------------------------------------------------------------
Try before you buy = See our experts in action!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-dev2
_______________________________________________
Shorewall-users mailing list
Shorewall-users <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/shorewall-users


Sorry for the top post last time.

I've been thinking some more about your reply and I've been studying the netfilter diagram you referenced and the shorewall-accounting documentation.

From that I can definitely say that I am doing accounting in the netfilter table.

According to the diagram the last chain that /etc/shorewall/accounting would see is FORWARD.  So my tcrules that apply mark 3 cannot be accounted for because they have not been applied yet.
3:T     0.0.0.0/0       0.0.0.0/0       udp     1194   # openvpn

So then to mark the openvpn traffic that is generated on the firewall (since it hosts openvpn) I would need a tcrule like this:
3     fw     0.0.0.0/0     udp     1194 #openvpn
As I understand it, this would mark in the OUTPUT chain, which is part of the filter table.

Is that reasoning correct?

Thanks again.

david.
------------------------------------------------------------------------------
Try before you buy = See our experts in action!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-dev2
------------------------------------------------------------------------------
Try before you buy = See our experts in action!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-dev2
Tom Eastep | 30 Jan 20:43 2012
Picon

Re: MARK accounting packet counts do not match mangle or tc

On Mon, 2012-01-30 at 13:34 -0600, David Koscinski wrote:

> 
> I've been thinking some more about your reply and I've been studying
> the netfilter diagram you referenced and the shorewall-accounting
> documentation.
> 
> From that I can definitely say that I am doing accounting in the
> netfilter table.
> 
> According to the diagram the last chain that /etc/shorewall/accounting
> would see is FORWARD.  So my tcrules that apply mark 3 cannot be
> accounted for because they have not been applied yet.
> 3:T     0.0.0.0/0       0.0.0.0/0       udp     1194   # openvpn
> 
> So then to mark the openvpn traffic that is generated on the firewall
> (since it hosts openvpn) I would need a tcrule like this:
> 3     fw     0.0.0.0/0     udp     1194 #openvpn
> As I understand it, this would mark in the OUTPUT chain, which is part
> of the filter table.
> 
> Is that reasoning correct?

It will mark the traffic in the mangle table's OUTPUT chain. So it will
be visible to the output accounting rules that are jumped to from the
filter table's OUTPUT chain.

-Tom
-- 
Tom Eastep        \ When I die, I want to go like my Grandfather who
Shoreline,         \ died peacefully in his sleep. Not screaming like
Washington, USA     \ all of the passengers in his car
http://shorewall.net \________________________________________________

------------------------------------------------------------------------------
Try before you buy = See our experts in action!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-dev2
------------------------------------------------------------------------------
Try before you buy = See our experts in action!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-dev2
David Koscinski | 30 Jan 20:45 2012
Picon

Re: MARK accounting packet counts do not match mangle or tc



On Mon, Jan 30, 2012 at 1:34 PM, David Koscinski <dmkoscinski <at> gmail.com> wrote:

On Mon, Jan 30, 2012 at 12:19 PM, Tom Eastep <teastep <at> shorewall.net> wrote:
On Mon, 2012-01-30 at 11:22 -0600, David Koscinski wrote:

> Do I misunderstand the capabilities of the MARK column in the
> accounting table?  Or have I misconfigured something?

It's not possible to say, given what you have told us.

1. Which chain(s) are you doing your TC marking in?
2. It appears that you are doing your accounting in the filter table, is
that correct? (Shorewall also allows you to do accounting in the
mangle).

I suspect that you are marking packets after they have been through
accounting; that would explain what you are seeing. You may wish to
refer to the diagram at http://www.shorewall.net/NetfilterOverview.html.

-Tom
--
Tom Eastep        \ When I die, I want to go like my Grandfather who
Shoreline,         \ died peacefully in his sleep. Not screaming like
Washington, USA     \ all of the passengers in his car
http://shorewall.net \________________________________________________



------------------------------------------------------------------------------
Try before you buy = See our experts in action!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-dev2
_______________________________________________
Shorewall-users mailing list
Shorewall-users <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/shorewall-users


Sorry for the top post last time.

I've been thinking some more about your reply and I've been studying the netfilter diagram you referenced and the shorewall-accounting documentation.

From that I can definitely say that I am doing accounting in the netfilter table.

According to the diagram the last chain that /etc/shorewall/accounting would see is FORWARD.  So my tcrules that apply mark 3 cannot be accounted for because they have not been applied yet.

3:T     0.0.0.0/0       0.0.0.0/0       udp     1194   # openvpn

So then to mark the openvpn traffic that is generated on the firewall (since it hosts openvpn) I would need a tcrule like this:
3     fw     0.0.0.0/0     udp     1194 #openvpn
As I understand it, this would mark in the OUTPUT chain, which is part of the filter table.

Is that reasoning correct?

Thanks again.

david.

Well that change did the trick for mark 3.  But is exposed a flaw in my plans.  Since /etc/accounting is only seeing MARK values prior to POSTROUTING, then my stats may not reflect the reality of what is going out eth0 since MARK could change.

So you mentioned that accounting can be done in mangle.  A quick google search revealed the ACCOUNTING_TABLE=mangle directive.  Looks like I need a shorewall upgrade to take advantage of that.

Even though I think I've found the answers based on your comments, please do reply if you can.  I'd like to be sure I am understanding this correctly.

david.


------------------------------------------------------------------------------
Try before you buy = See our experts in action!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-dev2
------------------------------------------------------------------------------
Try before you buy = See our experts in action!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-dev2
Tom Eastep | 30 Jan 21:04 2012
Picon

Re: MARK accounting packet counts do not match mangle or tc

On Mon, 2012-01-30 at 13:45 -0600, David Koscinski wrote:

>         
> 
> Well that change did the trick for mark 3.  But is exposed a flaw in
> my plans.  Since /etc/accounting is only seeing MARK values prior to
> POSTROUTING, then my stats may not reflect the reality of what is
> going out eth0 since MARK could change.

I'm not following you.

> 
> So you mentioned that accounting can be done in mangle.  A quick
> google search revealed the ACCOUNTING_TABLE=mangle directive.  Looks
> like I need a shorewall upgrade to take advantage of that.

That isn't going to work. When ACCOUNTING_TABLE=mangle, accounting
occurs before marking.

> 
> Even though I think I've found the answers based on your comments,
> please do reply if you can.  I'd like to be sure I am understanding
> this correctly.

-Tom
-- 
Tom Eastep        \ When I die, I want to go like my Grandfather who
Shoreline,         \ died peacefully in his sleep. Not screaming like
Washington, USA     \ all of the passengers in his car
http://shorewall.net \________________________________________________

------------------------------------------------------------------------------
Try before you buy = See our experts in action!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-dev2
------------------------------------------------------------------------------
Try before you buy = See our experts in action!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-dev2
David Koscinski | 30 Jan 21:18 2012
Picon

Re: MARK accounting packet counts do not match mangle or tc



On Mon, Jan 30, 2012 at 2:04 PM, Tom Eastep <teastep <at> shorewall.net> wrote:
On Mon, 2012-01-30 at 13:45 -0600, David Koscinski wrote:

>
>
> Well that change did the trick for mark 3.  But is exposed a flaw in
> my plans.  Since /etc/accounting is only seeing MARK values prior to
> POSTROUTING, then my stats may not reflect the reality of what is
> going out eth0 since MARK could change.

I'm not following you.

>
> So you mentioned that accounting can be done in mangle.  A quick
> google search revealed the ACCOUNTING_TABLE=mangle directive.  Looks
> like I need a shorewall upgrade to take advantage of that.

That isn't going to work. When ACCOUNTING_TABLE=mangle, accounting
occurs before marking.

>
> Even though I think I've found the answers based on your comments,
> please do reply if you can.  I'd like to be sure I am understanding
> this correctly.

-Tom
--
Tom Eastep        \ When I die, I want to go like my Grandfather who
Shoreline,         \ died peacefully in his sleep. Not screaming like
Washington, USA     \ all of the passengers in his car
http://shorewall.net \________________________________________________



------------------------------------------------------------------------------
Try before you buy = See our experts in action!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-dev2
_______________________________________________
Shorewall-users mailing list
Shorewall-users <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/shorewall-users


>>I'm not following you.
I mean that when I using /etc/shorewall/accounting I am seeing stats based on what the MARK was before POSTROUTING.  Since I want to know what the final MARK was as the packets leave eth0, I cannot use /etc/shorewall/accounting.  

>>That isn't going to work. When ACCOUNTING_TABLE=mangle, accounting
occurs before marking.
So it appears that I cannot use /etc/shorewall/accounting to track what the final MARK was on outgoing packets regardless of whether I do accounting in filter or mangle.  POSTROUTING tcrules can conceivably change the MARK after accounting has been done.
------------------------------------------------------------------------------
Try before you buy = See our experts in action!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-dev2
------------------------------------------------------------------------------
Try before you buy = See our experts in action!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-dev2
Tom Eastep | 30 Jan 21:36 2012
Picon

Re: MARK accounting packet counts do not match mangle or tc

On Mon, 2012-01-30 at 14:18 -0600, David Koscinski wrote:

> >>I'm not following you.
> I mean that when I using /etc/shorewall/accounting I am seeing stats
> based on what the MARK was before POSTROUTING.  Since I want to know
> what the final MARK was as the packets leave eth0, I cannot
> use /etc/shorewall/accounting. 

Why will the mark change? So long as you don't use :T marks, your
marking will occur before accounting.

> 
> >>That isn't going to work. When ACCOUNTING_TABLE=mangle, accounting
> occurs before marking.
> So it appears that I cannot use /etc/shorewall/accounting to track
> what the final MARK was on outgoing packets regardless of whether I do
> accounting in filter or mangle.  POSTROUTING tcrules can conceivably
> change the MARK after accounting has been done.

Not if you don't have such rules.

-Tom
-- 
Tom Eastep        \ When I die, I want to go like my Grandfather who
Shoreline,         \ died peacefully in his sleep. Not screaming like
Washington, USA     \ all of the passengers in his car
http://shorewall.net \________________________________________________

------------------------------------------------------------------------------
Try before you buy = See our experts in action!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-dev2
------------------------------------------------------------------------------
Try before you buy = See our experts in action!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-dev2
David Koscinski | 30 Jan 22:47 2012
Picon

Re: MARK accounting packet counts do not match mangle or tc



On Mon, Jan 30, 2012 at 2:36 PM, Tom Eastep <teastep <at> shorewall.net> wrote:
On Mon, 2012-01-30 at 14:18 -0600, David Koscinski wrote:

> >>I'm not following you.
> I mean that when I using /etc/shorewall/accounting I am seeing stats
> based on what the MARK was before POSTROUTING.  Since I want to know
> what the final MARK was as the packets leave eth0, I cannot
> use /etc/shorewall/accounting.

Why will the mark change? So long as you don't use :T marks, your
marking will occur before accounting.

>
> >>That isn't going to work. When ACCOUNTING_TABLE=mangle, accounting
> occurs before marking.
> So it appears that I cannot use /etc/shorewall/accounting to track
> what the final MARK was on outgoing packets regardless of whether I do
> accounting in filter or mangle.  POSTROUTING tcrules can conceivably
> change the MARK after accounting has been done.

Not if you don't have such rules.

-Tom
--
Tom Eastep        \ When I die, I want to go like my Grandfather who
Shoreline,         \ died peacefully in his sleep. Not screaming like
Washington, USA     \ all of the passengers in his car
http://shorewall.net \________________________________________________



------------------------------------------------------------------------------
Try before you buy = See our experts in action!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-dev2
_______________________________________________
Shorewall-users mailing list
Shorewall-users <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/shorewall-users


Yes, I see.  I just have to remember that distinction.  I guess I can just put a comment in tcrules to remind myself that :T marks can't be seen in accounting. 

Thank you very much for the info today.
------------------------------------------------------------------------------
Try before you buy = See our experts in action!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-dev2
------------------------------------------------------------------------------
Try before you buy = See our experts in action!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-dev2

Gmane