gmane.comp.apache.mod-security.user

problem with my regex and single line HTML comment in RESPONSE_BODY

Hi,

I'm having a problem with a regex.

I want to prevent content in an HTTP response being returned as in:
<!--
	TODO admin:adminpw
  -->

My regular expression:
<!--[ \r\n\t]*(.?)*(Admin|admin|administrator|pwd|password)(.*)[ \r\n\t]* -->

works both in Regex Coach and Expresso 3.0 and on variations such as:
    <!--
	Use Admin to regenerate database
  -->

which is what I want.

To try to make a long story short - without a bunch of debug stuff -
it works as designed on the 'bad' pages with multiline HTML comments
like above but when it processes a page with any HTML comment on one
line (e.g. <!-- Stop Instructions -->) it chokes and stops processing
the page.

My rule:
SecRule RESPONSE_BODY "<!--[
\r\n\t]*(.?)*(Admin|admin|administrator|pwd|password)(.*)[ \r\n\t]*
-->"
"phase:4,ctl:auditLogParts=+E,log,deny,status:501,auditlog,msg:'HTML

(Continue reading)

headers

Ryan Barnett | 27 Jun 16:45

Re: problem with my regex and single line HTMLcomment in RESPONSE_BODY

Stephen,
First off, I am sending you a "Virtual High-Five" as you are
demonstrating that you are using many of the various tools and
techniques that we have been discussing for trouble-shooting!  Using
RegEx Coach/Expresso and also leveraging the debug log is outstanding.

To your exact issues, I don't believe that Mod was "choking" on anything
but rather that your RegEx was not working correctly for the single line
comment.  Here are a few comments -

1) While RegEx tools (such as Expresso) are invaluable for helping to
create or trouble-shoot regex issues, beware that you can also run into
issues if the tool is not configured identically to how ModSecurity's
pcre/transformations will be used.  I just ran into this recently where
Expresso had the option checked to "Ignore Whitespace" and it screwed up
some of my tests.  In Expresso 3.0, click on the "Design Mode" tab and
check at the bottom to see how it is handling some of these issues.

2) In order to account for both single or multi-line comments, I had to
update your RegEx to add a few quantifiers to allow for the
presence/absence of CR, LF, etc...  I also made a few changes to catch
other "admin" or "password" variations.  One other item, in order to
handle mixed-case issues you can use "t:lowercase" however keep in mind
that transformation functions against the RESPONSE_BODY are rather
expensive so if you can specify all of the variations in the RegEx you
might want to for performance reasons.  You could also try and use the
regex embedded ignore case flag (?i:).  Here is an updated rule that
works for me -

SecRule RESPONSE_BODY "<!--[

(Continue reading)

headers

Stephen Craig Evans | 27 Jun 18:01

Re: problem with my regex and single line HTMLcomment in RESPONSE_BODY

Thanks a lot, Ryan. I'll try it out tomorrow morning and I'm 97% sure
that it will work, based on your previous success rate of suggestions.

I didn't mean to be offensive with the ModSecurity "choking" part; I
believe it uses the Apache PCRE engine.

In addition to those tools, I find the
http://www.regular-expressions.info/tutorial.html site useful and the
Friedl regex book is my current bathroom book 

Cheers,
Stephen

On Fri, Jun 27, 2008 at 10:45 PM, Ryan Barnett <Ryan.Barnett <at> breach.com> wrote:
> Stephen,
> First off, I am sending you a "Virtual High-Five" as you are
> demonstrating that you are using many of the various tools and
> techniques that we have been discussing for trouble-shooting!  Using
> RegEx Coach/Expresso and also leveraging the debug log is outstanding.
>
> To your exact issues, I don't believe that Mod was "choking" on anything
> but rather that your RegEx was not working correctly for the single line
> comment.  Here are a few comments -
>
> 1) While RegEx tools (such as Expresso) are invaluable for helping to
> create or trouble-shoot regex issues, beware that you can also run into
> issues if the tool is not configured identically to how ModSecurity's
> pcre/transformations will be used.  I just ran into this recently where
> Expresso had the option checked to "Ignore Whitespace" and it screwed up
> some of my tests.  In Expresso 3.0, click on the "Design Mode" tab and

(Continue reading)

headers

Stephen Craig Evans | 28 Jun 13:29

Re: problem with my regex and single line HTMLcomment in RESPONSE_BODY

Hi Ryan,

Thanks. That's very close. Your regex works great except when the
'bad' words are between 2 HTML comments like this:

password

I was getting a false match and it took me hours to get it whittled
down to this; just put those 3 lines between the <body> tags of the
barest HTML file and a false match will result. Both Expresso and
RegEx Coach say that it is not a match.

I might have to use regex capturing or something like that but I don't
have the regex skills yet to do that. It's good enough for now and
once I get the regex skills to figure it out, I'll revisit this.

Thanks,
Stephen

On Fri, Jun 27, 2008 at 10:45 PM, Ryan Barnett <Ryan.Barnett <at> breach.com> wrote:
> Stephen,
> First off, I am sending you a "Virtual High-Five" as you are
> demonstrating that you are using many of the various tools and
> techniques that we have been discussing for trouble-shooting!  Using
> RegEx Coach/Expresso and also leveraging the debug log is outstanding.
>
> To your exact issues, I don't believe that Mod was "choking" on anything
> but rather that your RegEx was not working correctly for the single line
> comment.  Here are a few comments -

(Continue reading)