Micah Anderson | 24 Jan 2012 22:22

scaling issues with security_tokens


I am having a problem with scaling squirrelmail because of the update
frequency of the security_tokens. It seems as though a new
security_token is generated and put into the database (via a REPLACE
INTO userprefs) everytime a squirrelmail user does anything in
squirrelmail... perhaps with some exceptions (I did not look that
closely).

This may not be a big deal when you only have a few users, but when you
have 39k users, this starts to really hammer the database. It also makes
replication a major pain because every single one of those userprefs
updates results in a query written to the binary log that then needs to
be replayed on the replication slave.

I've set $max_token_age_days = 1; but that only affects how much data
accumulates in the security_tokens, but it doesn't affect the number of
updates. 

Besides turning these off, which does not seem like a good idea, could
there be a better way to handle these?

thanks!
micah

--

-- 

------------------------------------------------------------------------------
Keep Your Developer Skills Current with LearnDevNow!
(Continue reading)

Paul Lesniewski | 25 Jan 2012 04:08
Favicon

Re: scaling issues with security_tokens

On Tue, Jan 24, 2012 at 1:22 PM, Micah Anderson <micah <at> riseup.net> wrote:
>
> I am having a problem with scaling squirrelmail because of the update
> frequency of the security_tokens. It seems as though a new
> security_token is generated and put into the database (via a REPLACE
> INTO userprefs) everytime a squirrelmail user does anything in
> squirrelmail... perhaps with some exceptions (I did not look that
> closely).
>
> This may not be a big deal when you only have a few users, but when you
> have 39k users, this starts to really hammer the database. It also makes
> replication a major pain because every single one of those userprefs
> updates results in a query written to the binary log that then needs to
> be replayed on the replication slave.
>
> I've set $max_token_age_days = 1; but that only affects how much data
> accumulates in the security_tokens, but it doesn't affect the number of
> updates.
>
> Besides turning these off, which does not seem like a good idea, could
> there be a better way to handle these?

You could turn it off and enable referrer checking.  Referrer checking
is a good alternative safety mechanism as long as you understand that
any users whose browsers do not send referrer information will not
have the benefits of this security check (such users would be
vulnerable to CSRF attacks, although it may be reasonable to argue
that users who know enough to turn off referrer sending in their
browsers are savvy enough to know such attacks when they see them).

(Continue reading)

Micah Anderson | 6 Feb 2012 20:34

Re: scaling issues with security_tokens


Hi Paul!

Paul Lesniewski <paul <at> squirrelmail.org> writes:
> On Tue, Jan 24, 2012 at 1:22 PM, Micah Anderson <micah <at> riseup.net> wrote:
>>
>> I am having a problem with scaling squirrelmail because of the update
>> frequency of the security_tokens. It seems as though a new
>> security_token is generated and put into the database (via a REPLACE
>> INTO userprefs) everytime a squirrelmail user does anything in
>> squirrelmail... perhaps with some exceptions (I did not look that
>> closely).
>>
>> This may not be a big deal when you only have a few users, but when you
>> have 39k users, this starts to really hammer the database. It also makes
>> replication a major pain because every single one of those userprefs
>> updates results in a query written to the binary log that then needs to
>> be replayed on the replication slave.
>>
>> I've set $max_token_age_days = 1; but that only affects how much data
>> accumulates in the security_tokens, but it doesn't affect the number of
>> updates.
>>
>> Besides turning these off, which does not seem like a good idea, could
>> there be a better way to handle these?
>
> You could turn it off and enable referrer checking.  Referrer checking
> is a good alternative safety mechanism as long as you understand that
> any users whose browsers do not send referrer information will not
> have the benefits of this security check (such users would be
(Continue reading)

Paul Lesniewski | 7 Feb 2012 23:53
Favicon

Re: scaling issues with security_tokens

>> See attached patch - if I can get feedback, I will add it for version 1.4.23.
>
> We applied this patch to the version we have installed (from Debian
> Squeeze), with only minor modifications to get things to line up
> right.
>
> We did it about 5 days ago and have been monitoring the situation since,
> and I am happy to say that it is completely solved our problem!  Our
> disk space usage from the binary replication logs has gone away. We went
> from 95% usage to 25% usage as the binary logs were rotated out
> throughout the day.
>
> We also looked at the tokens that were added and can confirm its working
> as expected with the patch. The number of token updates now is
> significantly less, which has a pretty large impact on our resource
> usage overall.

Thanks for the feedback.  Committed (with one small addition) to SVN.

--

-- 
Paul Lesniewski
SquirrelMail Team
Please support Open Source Software by donating to SquirrelMail!
http://squirrelmail.org/donate_paul_lesniewski.php

------------------------------------------------------------------------------
Keep Your Developer Skills Current with LearnDevNow!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
(Continue reading)


Gmane