Harika Tandra | 13 Aug 2012 16:50

country and city code with ralabel and GeoIP (argus-client-3.0.6.2)

Hi Carter,

I am using argus-clients-3.0.6.2. I see that ralabel is not working with GeoIPCity database. 
I am able to get AS information but not City related information. I am using the 
following commands: 

/usr/local/bin/ralabel -f /etc/ralabel.conf -S localhost -s sas das sco dco scity dcity

and 

/usr/local/bin/ralabel -f /etc/ralabel.conf -S localhost -w - | /usr/local/bin/ra -s sco dco sas das

And my ralabel.conf file is :

RALABEL_GEOIP_ASN=yes
RALABEL_GEOIP_ASN_FILE="/usr/local/share/GeoIP/GeoIPASNum.dat"
RALABEL_GEOIP_CITY="saddr,daddr:cco,cco3,city,lat,lon"
RALABEL_GEOIP_CITY_FILE="/usr/local/share/GeoIP/GeoIPCity.dat"

Please let me know if I you are observing the same or maybe something wrong at my end.

Thanks,
Harika Tandra.

----------------------------------------------------------
Harika Tandra
Research Associate (Software Engineer)
GLORIAD, ISSE
311 Conference Center Building
University of Tennessee 
(Continue reading)

Carter Bullard | 13 Aug 2012 17:04

Re: country and city code with ralabel and GeoIP (argus-client-3.0.6.2)

Hey Harika,
The generic city related information is added to the flow record's label as an ascii metadata string,
so there aren't specific city, zip or state fields to print, at least not today.  To filter on the field contents,
you use the " -e <regex> " option to specify the field contents you're looking for.

We do have support for country codes, which can come from various databases, and support
for  AS numbers, which comes from the GEOIP library, right now (if you have the right databases
in place.  As a result, you should get values when you printout the sco, dco, sas, and das
independent of the extended city data.

What output are you getting when you print out these fields and the labels?

   ra -s sco dco sas das label:64

Carter



On Aug 13, 2012, at 10:50 AM, Harika Tandra wrote:

Hi Carter,

I am using argus-clients-3.0.6.2. I see that ralabel is not working with GeoIPCity database.
I am able to get AS information but not City related information. I am using the
following commands:

/usr/local/bin/ralabel -f /etc/ralabel.conf -S localhost -s sas das sco dco scity dcity

and

/usr/local/bin/ralabel -f /etc/ralabel.conf -S localhost -w - | /usr/local/bin/ra -s sco dco sas das


And my ralabel.conf file is :

RALABEL_GEOIP_ASN=yes
RALABEL_GEOIP_ASN_FILE="/usr/local/share/GeoIP/GeoIPASNum.dat"
RALABEL_GEOIP_CITY="saddr,daddr:cco,cco3,city,lat,lon"
RALABEL_GEOIP_CITY_FILE="/usr/local/share/GeoIP/GeoIPCity.dat"


Please let me know if I you are observing the same or maybe something wrong at my end.

Thanks,
Harika Tandra.




----------------------------------------------------------
Harika Tandra
Research Associate (Software Engineer)
GLORIAD, ISSE
311 Conference Center Building
University of Tennessee
htandra <at> gloriad.org
htandra <at> utk.edu





Attachment (smime.p7s): application/pkcs7-signature, 4367 bytes
Harika Tandra | 13 Aug 2012 17:38

Re: country and city code with ralabel and GeoIP (argus-client-3.0.6.2)

Hi Carter,

Thank you, its good to know about the label metadata string. I can grep the needed information from it.
I am not getting sco and dco directly though. This is the output I get from the below command :

# /usr/local/bin/ralabel -f /etc/ralabel.conf -S localhost -w - | /usr/local/bin/ra -s sco dco sas das label:64
         1781       scity=KR,KR,(null),37.000000,127.500000:dcity=US,US,(null),38.0*
          137  6879 scity=IT,IT,Direzione,40.799999,9.016700:dcity=EG,EG,Cairo,30.0*
         8075 20928 scity=US,US,(null),38.000000,-97.000000:dcity=EG,EG,Cairo,30.04*
         8075 20928 scity=US,US,Redmond,47.670601,-122.068497:dcity=EG,EG,(null),27*
         3512  7472 scity=US,US,Atlanta,33.795200,-84.324799:dcity=SG,SG,(null),1.3*
          137  6879 scity=IT,IT,Direzione,40.799999,9.016700:dcity=EG,EG,(null),27.*
         9488    91 scity=KR,KR,Seoul,37.566399,126.999702:dcity=US,US,Troy,42.7495*
          137  2561 scity=IT,IT,Direzione,40.799999,9.016700:dcity=EG,EG,G?za,30.00*
        22950  4538 scity=CA,CA,Saskatoon,52.133301,-106.666801:dcity=CN,CN,Beijing*
          239  4538 scity=CA,CA,Toronto,43.666698,-79.416801:dcity=CN,CN,Guangzhou,*
        36441  4538 scity=US,US,Athens,33.949902,-83.375000:dcity=CN,CN,Changchun,4*

When I query the GeoIPCity database separately, I do get the expected output. So everything on that end seems right. 

Thanks,
Harika.


On Aug 13, 2012, at 11:04 AM, Carter Bullard wrote:

Hey Harika,
The generic city related information is added to the flow record's label as an ascii metadata string,
so there aren't specific city, zip or state fields to print, at least not today.  To filter on the field contents,
you use the " -e <regex> " option to specify the field contents you're looking for.

We do have support for country codes, which can come from various databases, and support
for  AS numbers, which comes from the GEOIP library, right now (if you have the right databases
in place.  As a result, you should get values when you printout the sco, dco, sas, and das
independent of the extended city data.

What output are you getting when you print out these fields and the labels?

   ra -s sco dco sas das label:64

Carter



On Aug 13, 2012, at 10:50 AM, Harika Tandra wrote:

Hi Carter,

I am using argus-clients-3.0.6.2. I see that ralabel is not working with GeoIPCity database.
I am able to get AS information but not City related information. I am using the
following commands:

/usr/local/bin/ralabel -f /etc/ralabel.conf -S localhost -s sas das sco dco scity dcity

and

/usr/local/bin/ralabel -f /etc/ralabel.conf -S localhost -w - | /usr/local/bin/ra -s sco dco sas das


And my ralabel.conf file is :

RALABEL_GEOIP_ASN=yes
RALABEL_GEOIP_ASN_FILE="/usr/local/share/GeoIP/GeoIPASNum.dat"
RALABEL_GEOIP_CITY="saddr,daddr:cco,cco3,city,lat,lon"
RALABEL_GEOIP_CITY_FILE="/usr/local/share/GeoIP/GeoIPCity.dat"


Please let me know if I you are observing the same or maybe something wrong at my end.

Thanks,
Harika Tandra.




----------------------------------------------------------
Harika Tandra
Research Associate (Software Engineer)
GLORIAD, ISSE
311 Conference Center Building
University of Tennessee
htandra <at> gloriad.org
htandra <at> utk.edu






Carter Bullard | 13 Aug 2012 18:14

Re: country and city code with ralabel and GeoIP (argus-client-3.0.6.2)

Hey Harika,
You aren't inserting the country codes with your configuration, you are
specifying that the extended city information should include the country codes,
but that just inserts the cco into the metadata string, it doesn't populate the
sco or dco values.

Currently, to insert country codes so that sco and dco are populated, you need
to use RALABEL_ARIN_COUNTRY_CODES and set an ARIN style data file
for the encodings.  I'll look to change this, but currently, you should set both
label strategies.

Remember, the " * " at the end of the string indicates that you didn't provide enough
space to print the values, so the 64 should be larger, or use comma separated output,
which doesn't truncate the fields.


Carter 


On Aug 13, 2012, at 11:38 AM, Harika Tandra wrote:

Hi Carter,

Thank you, its good to know about the label metadata string. I can grep the needed information from it.
I am not getting sco and dco directly though. This is the output I get from the below command :

# /usr/local/bin/ralabel -f /etc/ralabel.conf -S localhost -w - | /usr/local/bin/ra -s sco dco sas das label:64
         1781       scity=KR,KR,(null),37.000000,127.500000:dcity=US,US,(null),38.0*
          137  6879 scity=IT,IT,Direzione,40.799999,9.016700:dcity=EG,EG,Cairo,30.0*
         8075 20928 scity=US,US,(null),38.000000,-97.000000:dcity=EG,EG,Cairo,30.04*
         8075 20928 scity=US,US,Redmond,47.670601,-122.068497:dcity=EG,EG,(null),27*
         3512  7472 scity=US,US,Atlanta,33.795200,-84.324799:dcity=SG,SG,(null),1.3*
          137  6879 scity=IT,IT,Direzione,40.799999,9.016700:dcity=EG,EG,(null),27.*
         9488    91 scity=KR,KR,Seoul,37.566399,126.999702:dcity=US,US,Troy,42.7495*
          137  2561 scity=IT,IT,Direzione,40.799999,9.016700:dcity=EG,EG,G?za,30.00*
        22950  4538 scity=CA,CA,Saskatoon,52.133301,-106.666801:dcity=CN,CN,Beijing*
          239  4538 scity=CA,CA,Toronto,43.666698,-79.416801:dcity=CN,CN,Guangzhou,*
        36441  4538 scity=US,US,Athens,33.949902,-83.375000:dcity=CN,CN,Changchun,4*

When I query the GeoIPCity database separately, I do get the expected output. So everything on that end seems right. 

Thanks,
Harika.


On Aug 13, 2012, at 11:04 AM, Carter Bullard wrote:

Hey Harika,
The generic city related information is added to the flow record's label as an ascii metadata string,
so there aren't specific city, zip or state fields to print, at least not today.  To filter on the field contents,
you use the " -e <regex> " option to specify the field contents you're looking for.

We do have support for country codes, which can come from various databases, and support
for  AS numbers, which comes from the GEOIP library, right now (if you have the right databases
in place.  As a result, you should get values when you printout the sco, dco, sas, and das
independent of the extended city data.

What output are you getting when you print out these fields and the labels?

   ra -s sco dco sas das label:64

Carter



On Aug 13, 2012, at 10:50 AM, Harika Tandra wrote:

Hi Carter,

I am using argus-clients-3.0.6.2. I see that ralabel is not working with GeoIPCity database.
I am able to get AS information but not City related information. I am using the
following commands:

/usr/local/bin/ralabel -f /etc/ralabel.conf -S localhost -s sas das sco dco scity dcity

and

/usr/local/bin/ralabel -f /etc/ralabel.conf -S localhost -w - | /usr/local/bin/ra -s sco dco sas das


And my ralabel.conf file is :

RALABEL_GEOIP_ASN=yes
RALABEL_GEOIP_ASN_FILE="/usr/local/share/GeoIP/GeoIPASNum.dat"
RALABEL_GEOIP_CITY="saddr,daddr:cco,cco3,city,lat,lon"
RALABEL_GEOIP_CITY_FILE="/usr/local/share/GeoIP/GeoIPCity.dat"


Please let me know if I you are observing the same or maybe something wrong at my end.

Thanks,
Harika Tandra.




----------------------------------------------------------
Harika Tandra
Research Associate (Software Engineer)
GLORIAD, ISSE
311 Conference Center Building
University of Tennessee
htandra <at> gloriad.org
htandra <at> utk.edu







Attachment (smime.p7s): application/pkcs7-signature, 4367 bytes
Harika Tandra | 13 Aug 2012 18:48

Re: country and city code with ralabel and GeoIP (argus-client-3.0.6.2)

Hi Carter,

I understand the configuration now. Thank you. 
I downloaded and set the ARIN country codes file and uncommented RALABEL_ARIN_COUNTRY_CODES
in ralabel.conf.
Now I am getting all the information needed with this command :

/usr/local/bin/ralabel -f /etc/ralabel.conf -S localhost -w - | /usr/local/bin/ra -c ',' -s sco dco sas das label

Thanks again.

Regards,
Harika.


On Aug 13, 2012, at 12:14 PM, Carter Bullard wrote:

Hey Harika,
You aren't inserting the country codes with your configuration, you are
specifying that the extended city information should include the country codes,
but that just inserts the cco into the metadata string, it doesn't populate the
sco or dco values.

Currently, to insert country codes so that sco and dco are populated, you need
to use RALABEL_ARIN_COUNTRY_CODES and set an ARIN style data file
for the encodings.  I'll look to change this, but currently, you should set both
label strategies.

Remember, the " * " at the end of the string indicates that you didn't provide enough
space to print the values, so the 64 should be larger, or use comma separated output,
which doesn't truncate the fields.


Carter 


On Aug 13, 2012, at 11:38 AM, Harika Tandra wrote:

Hi Carter,

Thank you, its good to know about the label metadata string. I can grep the needed information from it.
I am not getting sco and dco directly though. This is the output I get from the below command :

# /usr/local/bin/ralabel -f /etc/ralabel.conf -S localhost -w - | /usr/local/bin/ra -s sco dco sas das label:64
         1781       scity=KR,KR,(null),37.000000,127.500000:dcity=US,US,(null),38.0*
          137  6879 scity=IT,IT,Direzione,40.799999,9.016700:dcity=EG,EG,Cairo,30.0*
         8075 20928 scity=US,US,(null),38.000000,-97.000000:dcity=EG,EG,Cairo,30.04*
         8075 20928 scity=US,US,Redmond,47.670601,-122.068497:dcity=EG,EG,(null),27*
         3512  7472 scity=US,US,Atlanta,33.795200,-84.324799:dcity=SG,SG,(null),1.3*
          137  6879 scity=IT,IT,Direzione,40.799999,9.016700:dcity=EG,EG,(null),27.*
         9488    91 scity=KR,KR,Seoul,37.566399,126.999702:dcity=US,US,Troy,42.7495*
          137  2561 scity=IT,IT,Direzione,40.799999,9.016700:dcity=EG,EG,G?za,30.00*
        22950  4538 scity=CA,CA,Saskatoon,52.133301,-106.666801:dcity=CN,CN,Beijing*
          239  4538 scity=CA,CA,Toronto,43.666698,-79.416801:dcity=CN,CN,Guangzhou,*
        36441  4538 scity=US,US,Athens,33.949902,-83.375000:dcity=CN,CN,Changchun,4*

When I query the GeoIPCity database separately, I do get the expected output. So everything on that end seems right. 

Thanks,
Harika.


On Aug 13, 2012, at 11:04 AM, Carter Bullard wrote:

Hey Harika,
The generic city related information is added to the flow record's label as an ascii metadata string,
so there aren't specific city, zip or state fields to print, at least not today.  To filter on the field contents,
you use the " -e <regex> " option to specify the field contents you're looking for.

We do have support for country codes, which can come from various databases, and support
for  AS numbers, which comes from the GEOIP library, right now (if you have the right databases
in place.  As a result, you should get values when you printout the sco, dco, sas, and das
independent of the extended city data.

What output are you getting when you print out these fields and the labels?

   ra -s sco dco sas das label:64

Carter



On Aug 13, 2012, at 10:50 AM, Harika Tandra wrote:

Hi Carter,

I am using argus-clients-3.0.6.2. I see that ralabel is not working with GeoIPCity database.
I am able to get AS information but not City related information. I am using the
following commands:

/usr/local/bin/ralabel -f /etc/ralabel.conf -S localhost -s sas das sco dco scity dcity

and

/usr/local/bin/ralabel -f /etc/ralabel.conf -S localhost -w - | /usr/local/bin/ra -s sco dco sas das


And my ralabel.conf file is :

RALABEL_GEOIP_ASN=yes
RALABEL_GEOIP_ASN_FILE="/usr/local/share/GeoIP/GeoIPASNum.dat"
RALABEL_GEOIP_CITY="saddr,daddr:cco,cco3,city,lat,lon"
RALABEL_GEOIP_CITY_FILE="/usr/local/share/GeoIP/GeoIPCity.dat"


Please let me know if I you are observing the same or maybe something wrong at my end.

Thanks,
Harika Tandra.




----------------------------------------------------------
Harika Tandra
Research Associate (Software Engineer)
GLORIAD, ISSE
311 Conference Center Building
University of Tennessee
htandra <at> gloriad.org
htandra <at> utk.edu









Gmane