Apache web server development

Nick Edwards | 20 Jun 2012 14:25

md5crypt passwords

Hello,

I posted this to users list last week but no-one bit, so I'm trying here.

With md5crypt no longer recommended for use by its author, will Apache
soon support sha256/sha512 in basic authentication via MySQL.

I understand the apr version is different to plain md5crypt, but it is
based on the same thing from what I can tell, so its pointless
upgrading our database passwords to use sha512 if Apache's still the
weak link.

All admin scripts run in perl, and we are currently doing this with
apache_md5_crypt($password); using Crypt::PasswdMD5

For Mail and FTP, we are _now_ successfully using  crypt($password,
'$6$' . $16charsalt) for sha512, be nice if Apache basic auth would
too!

Apache currently only offers SHA1 which is about as secure (can be
read as , as hopeless as) MD5.

Can the project devs/team leaders indicate if there are future plans
to mnprove the basic auth security methods up to SHA512?

nik

headers

Stefan Fritsch | 20 Jun 2012 22:52

Re: md5crypt passwords

On Wed, 20 Jun 2012, Nick Edwards wrote:
> I posted this to users list last week but no-one bit, so I'm trying here.
>
> With md5crypt no longer recommended for use by its author, will Apache
> soon support sha256/sha512 in basic authentication via MySQL.

Note that it does not really matter that much which hash algorithm is 
used. The number of rounds is more important. APR-MD5 ("$apr1$") does 1000 
times recursive md5 (which is 1000 times more secure in terms of brute 
forcing than plain md5). We should switch to something that needs more 
processing time so that it is more difficult to brute force.

> I understand the apr version is different to plain md5crypt, but it is
> based on the same thing from what I can tell, so its pointless
> upgrading our database passwords to use sha512 if Apache's still the
> weak link.
>
> All admin scripts run in perl, and we are currently doing this with
> apache_md5_crypt($password); using Crypt::PasswdMD5
>
> For Mail and FTP, we are _now_ successfully using  crypt($password,
> '$6$' . $16charsalt) for sha512, be nice if Apache basic auth would
> too!

APR passes everything it doesn't know to the system's crypt() function. So 
chances are good that using $6$... already works for you. However, there 
is currently no way to create such hashes with htpasswd.

> Apache currently only offers SHA1 which is about as secure (can be
> read as , as hopeless as) MD5.

(Continue reading)

headers

Reindl Harald | 20 Jun 2012 23:19

Re: md5crypt passwords


Am 20.06.2012 22:52, schrieb Stefan Fritsch:
> On Wed, 20 Jun 2012, Nick Edwards wrote:
>> I posted this to users list last week but no-one bit, so I'm trying here.
>>
>> With md5crypt no longer recommended for use by its author, will Apache
>> soon support sha256/sha512 in basic authentication via MySQL.
> 
> Note that it does not really matter that much which hash algorithm is used. The number of rounds is more important.
> APR-MD5 ("$apr1$") does 1000 times recursive md5 (which is 1000 times more secure in terms of brute
forcing than
> plain md5). 

jesus christ do not tell this any crypto specialist!
this is completly wrong and the opposite true

you do NOT NEED the right password
you ONLY need a hash-collision

in the worst case md5(password(md5(password)) is much more
unsecure as md5(password) alone! why?

because if my password is longer than a hash and you are
hasing the hash again the original password will no
longer matter - the collsion is based on the shorter one

headers

Reindl Harald | 20 Jun 2012 23:29

Re: md5crypt passwords


Am 20.06.2012 23:19, schrieb Reindl Harald:
> 
> 
> Am 20.06.2012 22:52, schrieb Stefan Fritsch:
>> On Wed, 20 Jun 2012, Nick Edwards wrote:
>>> I posted this to users list last week but no-one bit, so I'm trying here.
>>>
>>> With md5crypt no longer recommended for use by its author, will Apache
>>> soon support sha256/sha512 in basic authentication via MySQL.
>>
>> Note that it does not really matter that much which hash algorithm is used. The number of rounds is more important.
>> APR-MD5 ("$apr1$") does 1000 times recursive md5 (which is 1000 times more secure in terms of brute
forcing than
>> plain md5). 
> 
> jesus christ do not tell this any crypto specialist!
> this is completly wrong and the opposite true
> 
> you do NOT NEED the right password
> you ONLY need a hash-collision
> 
> in the worst case md5(password(md5(password)) is much more
> unsecure as md5(password) alone! why?
> 
> because if my password is longer than a hash and you are
> hasing the hash again the original password will no
> longer matter - the collsion is based on the shorter one

one more reason:

(Continue reading)

headers

Stefan Fritsch | 20 Jun 2012 23:52

Re: md5crypt passwords

On Wednesday 20 June 2012, Reindl Harald wrote:
> Am 20.06.2012 23:19, schrieb Reindl Harald:
> > Am 20.06.2012 22:52, schrieb Stefan Fritsch:
> >> On Wed, 20 Jun 2012, Nick Edwards wrote:
> >>> I posted this to users list last week but no-one bit, so I'm
> >>> trying here.
> >>> 
> >>> With md5crypt no longer recommended for use by its author, will
> >>> Apache soon support sha256/sha512 in basic authentication via
> >>> MySQL.
> >> 
> >> Note that it does not really matter that much which hash
> >> algorithm is used. The number of rounds is more important.
> >> APR-MD5 ("$apr1$") does 1000 times recursive md5 (which is 1000
> >> times more secure in terms of brute forcing than plain md5).
> > 
> > jesus christ do not tell this any crypto specialist!
> > this is completly wrong and the opposite true
> > 
> > you do NOT NEED the right password
> > you ONLY need a hash-collision
> > 
> > in the worst case md5(password(md5(password)) is much more
> > unsecure as md5(password) alone! why?
> > 
> > because if my password is longer than a hash and you are
> > hasing the hash again the original password will no
> > longer matter - the collsion is based on the shorter one

I should have written "it does not really matter that much *which of

(Continue reading)

headers

Reindl Harald | 20 Jun 2012 23:59

Re: md5crypt passwords


Am 20.06.2012 23:52, schrieb Stefan Fritsch:
>> you do not need the original password!
>> you only need a hash-collision and can leave out
>> special chars completly to find one
> 
> You need a password that gives the same value after 1000 rounds of 
> md5(password md5(password md5(password ...))). This is much more 
> expensive to find with brute force than a password that gives a 
> collision for a single md5

everybody with crypto knowledge will explain you that you
are totally wrong - i can only try in my words!

in the context of a hash-collision and rainbow-tables
you need any string producing the same hash, no matter
if 1, 10 or 1000 times md5() recursion

there is a reason why even the developer of md5crypt
saw the need for a offical statement that md5crypt
should never again be considered as secure in any case!

-------- Original-Nachricht --------
Betreff: CVE-2012-3287: md5crypt is no longer considered safe
Datum: Fri, 8 Jun 2012 00:04:49 GMT
Von: phk <at> FreeBSD.org
An: bugtraq <at> securityfocus.com

The LinkedIn password incompetence has resulted in a number of "just use md5crypt and you'll be fine"
pieces of

(Continue reading)

headers

Stefan Fritsch | 21 Jun 2012 00:14

Re: md5crypt passwords

On Wednesday 20 June 2012, Reindl Harald wrote:
> there is a reason why even the developer of md5crypt
> saw the need for a offical statement that md5crypt
> should never again be considered as secure in any case!

> http://phk.freebsd.dk/sagas/md5crypt_eol.html

Follow the link in his statement:

http://2012.sharcs.org/slides/sprengers.pdf

They can try around 1 million md5crypt operations per second (md5crypt 
is basically the same as APR-MD5). For plain md5 (one round) there are 
programs that do more than 200 million operations per second. That's a 
rather big difference. And plain sha1 or even sha512 is much closer to 
plain md5 than to md5crypt.

I agree that we should use something more secure, really soon. But 
there is no reason to panic, yet.

Cheers,
Stefan

headers

Reindl Harald | 21 Jun 2012 00:27

Re: md5crypt passwords


Am 21.06.2012 00:14, schrieb Stefan Fritsch:
> On Wednesday 20 June 2012, Reindl Harald wrote:
>> there is a reason why even the developer of md5crypt
>> saw the need for a offical statement that md5crypt
>> should never again be considered as secure in any case!
> 
> 
>> http://phk.freebsd.dk/sagas/md5crypt_eol.html
> 
> Follow the link in his statement:
> 
> http://2012.sharcs.org/slides/sprengers.pdf
> 
> They can try around 1 million md5crypt operations per second (md5crypt 
> is basically the same as APR-MD5). For plain md5 (one round) there are 
> programs that do more than 200 million operations per second. That's a 
> rather big difference. And plain sha1 or even sha512 is much closer to 
> plain md5 than to md5crypt.
> 
> I agree that we should use something more secure, really soon. But 
> there is no reason to panic, yet.

here we are agree
no reason for panic now

i only needed to point out that weakhash(weakhash(weakhash()))
does not result in stronghash() no matter how often you wrap

(Continue reading)

headers

André Malo | 21 Jun 2012 20:19

Re: md5crypt passwords

* Reindl Harald wrote:

> i only needed to point out that weakhash(weakhash(weakhash()))
> does not result in stronghash() no matter how often you wrap

I'm not sure, why the topic drifted there anyway. md5crypt does not actually 
nest hashes like this.

nd
--

-- 
package Hacker::Perl::Another::Just;print
qq~ <at> {[reverse split/::/ =>__PACKAGE__]}~;

#  André Malo  #  http://www.perlig.de  #

headers

Noel Butler | 21 Jun 2012 05:42

Re: md5crypt passwords

On Wed, 2012-06-20 at 22:52 +0200, Stefan Fritsch wrote:

On Wed, 20 Jun 2012, Nick Edwards wrote: > I posted this to users list last week but no-one bit, so I'm trying here. > > With md5crypt no longer recommended for use by its author, will Apache > soon support sha256/sha512 in basic authentication via MySQL. Note that it does not really matter that much which hash algorithm is used. The number of rounds is more important. APR-MD5 ("$apr1$") does 1000 times recursive md5 (which is 1000 times more secure in terms of brute forcing than plain md5). We should switch to something that needs more processing time so that it is more difficult to brute force.

yup, I'm not a crypto expert but IIRC sha512 by default uses rounds=5000 (if not rounds= is not specified)

I brought this up with (I think it was) Bill, a year ago (using SHA2) and at that time there were no plans, however, in light of recent events, I'd agree it needs to be revisited.

> > For Mail and FTP, we are _now_ successfully using crypt($password, > '$6$' . $16charsalt) for sha512, be nice if Apache basic auth would > too! APR passes everything it doesn't know to the system's crypt() function. So chances are good that using $6$... already works for you. However, there is currently no way to create such hashes with htpasswd.

If the OP is using crypt() correctly with a modern nix OS it certainly will, I've been using sha512 for a while.
I'm surprised he did not just "try it", he might have had his answer a week ago.

in perl I've been using

$psalt= join '', ('.', '/', 0..9, 'A'..'Z', 'a'..'z')[map {rand 64} (0..15)];
$epass = crypt($PASS, '$6$' . $psalt);

the end result will be readable by httpd to auth users

In php, I've used

$psalt = uniqid(16); <-- yes i know a bad cheat ... but I'm far from knowledgeable with php
$epass = crypt($PASS,'$6$'.$csalt);

headers

Nick Edwards | 22 Jun 2012 02:59

Re: md5crypt passwords

Hi Stefan,

On 6/21/12, Stefan Fritsch <sf <at> sfritsch.de> wrote:
> On Wed, 20 Jun 2012, Nick Edwards wrote:
>> I posted this to users list last week but no-one bit, so I'm trying here.
>>
>> With md5crypt no longer recommended for use by its author, will Apache
>> soon support sha256/sha512 in basic authentication via MySQL.
>
> Note that it does not really matter that much which hash algorithm is
> used. The number of rounds is more important. APR-MD5 ("$apr1$") does 1000
> times recursive md5 (which is 1000 times more secure in terms of brute
> forcing than plain md5). We should switch to something that needs more
> processing time so that it is more difficult to brute force.
>
>> I understand the apr version is different to plain md5crypt, but it is
>> based on the same thing from what I can tell, so its pointless
>> upgrading our database passwords to use sha512 if Apache's still the
>> weak link.
>>
>> All admin scripts run in perl, and we are currently doing this with
>> apache_md5_crypt($password); using Crypt::PasswdMD5
>>
>> For Mail and FTP, we are _now_ successfully using  crypt($password,
>> '$6$' . $16charsalt) for sha512, be nice if Apache basic auth would
>> too!
>
> APR passes everything it doesn't know to the system's crypt() function. So
> chances are good that using $6$... already works for you. However, there
> is currently no way to create such hashes with htpasswd.
>

Thanks, it does work, I did not realise this, because some time ago I
tried to use our default md5cryprt password used for ftp and mail, but
Apache did not like it, after much googling I found reference saying
it was different and needed apr1, if it falls back why would that have
failed for us? or is this fallback only something new?

>
> I would be for importing some state of the art scheme from some BSD. Good
> schemes allow to change the number of rounds without loosing backward
> compatibility. I guess bcrypt is a candidate. This new algorithm should
> then be the new default.
>

Don't know much about bcrypt sorry.

Thanks again for your help, all is now happy

headers

Ben Laurie | 21 Jun 2012 11:40

Re: md5crypt passwords

On Wed, Jun 20, 2012 at 1:25 PM, Nick Edwards <nick.z.edwards <at> gmail.com> wrote:
> Hello,
>
> I posted this to users list last week but no-one bit, so I'm trying here.
>
> With md5crypt no longer recommended for use by its author, will Apache
> soon support sha256/sha512 in basic authentication via MySQL.
>
> I understand the apr version is different to plain md5crypt, but it is
> based on the same thing from what I can tell, so its pointless
> upgrading our database passwords to use sha512 if Apache's still the
> weak link.
>
> All admin scripts run in perl, and we are currently doing this with
> apache_md5_crypt($password); using Crypt::PasswdMD5
>
> For Mail and FTP, we are _now_ successfully using  crypt($password,
> '$6$' . $16charsalt) for sha512, be nice if Apache basic auth would
> too!
>
> Apache currently only offers SHA1 which is about as secure (can be
> read as , as hopeless as) MD5.
>
> Can the project devs/team leaders indicate if there are future plans
> to mnprove the basic auth security methods up to SHA512?

This thread is all over the place!

So, a summary...

* SHA-1 is not as broken as MD5 - no way to generate collisions has
been found that is practical.

* MD5 is still pre-image resistant even though it is possible to
generate collisions. This means that repeated applications of MD5 are
also pre-image resistant.

* The threat against md5crypt is the same as it has always been:
rainbow tables. The current attacks have not changed this.

* htpasswd uses an 8 byte salt with 6 bits per byte. So, a rainbow
table will have 2^48 entries per password. Assuming you can somehow
manage 1 byte per entry, that's still nearly 300 TB per password.

So, I'd say there's no need for panic, but there's also no reason that
I can see to not move to a more modern scheme.

Then the question is: what scheme? Here are some design criteria I
think would be useful.

1. Use something from the SHA-2 family - SHA-512 would seem fine to me.

2. Use a very large salt - disk space is not at a premium for password stores!

3. Use quite a lot of rounds,.

4. Use something that is hard to optimise in hardware (ideally).

headers

Issac Goldstand | 21 Jun 2012 11:53

Re: md5crypt passwords

On 21/06/2012 12:40, Ben Laurie wrote:
> 4. Use something that is hard to optimise in hardware (ideally). 
And what about massive sites that need the crypto HW to manage the
concurrent logins?

Yes, you're making it harder on the hackers, but also potentially on our
users. 

...Or did I just put my foot in my crypto-clueless-mouth?

  Issac

headers

Ben Laurie | 21 Jun 2012 11:57

Re: md5crypt passwords

On Thu, Jun 21, 2012 at 10:53 AM, Issac Goldstand <margol <at> beamartyr.net> wrote:
> On 21/06/2012 12:40, Ben Laurie wrote:
>> 4. Use something that is hard to optimise in hardware (ideally).
> And what about massive sites that need the crypto HW to manage the
> concurrent logins?

I have never come across a site where login causes significant load.
They may exist, but I figure they can do their own thing for
passwords...

> Yes, you're making it harder on the hackers, but also potentially on our
> users.
>
> ...Or did I just put my foot in my crypto-clueless-mouth?
>
>  Issac
>

headers

Noel Butler | 21 Jun 2012 13:12

Re: md5crypt passwords

On Thu, 2012-06-21 at 10:57 +0100, Ben Laurie wrote:

On Thu, Jun 21, 2012 at 10:53 AM, Issac Goldstand <margol <at> beamartyr.net> wrote: > On 21/06/2012 12:40, Ben Laurie wrote: >> 4. Use something that is hard to optimise in hardware (ideally). > And what about massive sites that need the crypto HW to manage the > concurrent logins? I have never come across a site where login causes significant load.

That's because all the commonly used methods are cryptographic hash's that are designed for speed, unlike password hash's, designed to makes things crawl for the norti peoples.

headers

Noel Butler | 21 Jun 2012 13:09

Re: md5crypt passwords

On Thu, 2012-06-21 at 10:40 +0100, Ben Laurie wrote:

4. Use something that is hard to optimise in hardware (ideally).

5. Only hire web developers who know what they're doing, who know what security is, and how to audit their code

If they cant get the database, then it wouldn't mater if they were in *gulp* plain text

headers

Stefan Fritsch | 23 Jun 2012 22:42

Choosing a stronger password hash algorithm

On Thursday 21 June 2012, Ben Laurie wrote:
> Then the question is: what scheme? Here are some design criteria I
> think would be useful.
> 
> 1. Use something from the SHA-2 family - SHA-512 would seem fine to
> me.

The sha message digests are designed to be fast and to be easily 
implemented in hardware. In general, the speed-up when going from 
general purpose CPU implementations to parallel implementations using 
GPU or specialized hardware seems to be relatively high. Therefore I 
think what is generally used as sha256_crypt or sha512_crypt is not 
the optimal choice. [1] has some thoughts about choosing a password 
hashing algorithm. 

> 
> 2. Use a very large salt - disk space is not at a premium for
> password stores!

Agreed. 64bit should be absolute minimum, more would be better.

> 3. Use quite a lot of rounds,.

The number should be configurable so that we can adjust the cost 
without breaking backward compatibility.

> 4. Use something that is hard to optimise in hardware (ideally).

bcrypt [1] and scrypt [2] seem to be much more difficult to port to 
hardware or GPUs than sha256/512_crypt [3-5]. We can't make the 
operation too expensive on normal CPUs or we create a DoS problem if 
someone does lots of requests with wrong passwords. Therefore I think 
choosing an algorithm that does not scale well on more specialized 
hardware is good.

Both bcrypt and scrypt can be adjusted in how much CPU time to use. 
scrypt can also be adjusted in how much RAM it uses. bcrypt uses a 
128bit salt, AFAICS scrypt can use arbitrary salt lengths.

Bcrypt has seen a lot more review than scrypt, therefore I am somewhat 
in favor of bcrypt. Crypt_blowfish [6] is an implementation with a 
very liberal license that we could use. Scrypt has a 2-clause BSD 
license which would also be OK.

Opinions?

Cheers,
Stefan

[1] http://static.usenix.org/event/usenix99/provos.html
[2] http://www.tarsnap.com/scrypt.html
[3] http://stackoverflow.com/a/6807375
[4] http://www.openwall.com/lists/john-dev/2012/05/14/1
[5] http://www.openwall.com/lists/john-dev/2012/05/14/4
[6] http://www.openwall.com/crypt/

headers

William A. Rowe Jr. | 23 Jun 2012 23:38

Re: Choosing a stronger password hash algorithm

On 6/23/2012 3:42 PM, Stefan Fritsch wrote:
> 
> bcrypt [1] and scrypt [2] seem to be much more difficult to port to 
> hardware or GPUs than sha256/512_crypt [3-5]. We can't make the 
> operation too expensive on normal CPUs or we create a DoS problem if 
> someone does lots of requests with wrong passwords. Therefore I think 
> choosing an algorithm that does not scale well on more specialized 
> hardware is good.
> 
> Both bcrypt and scrypt can be adjusted in how much CPU time to use. 
> scrypt can also be adjusted in how much RAM it uses. bcrypt uses a 
> 128bit salt, AFAICS scrypt can use arbitrary salt lengths.
> 
> Bcrypt has seen a lot more review than scrypt, therefore I am somewhat 
> in favor of bcrypt. Crypt_blowfish [6] is an implementation with a 
> very liberal license that we could use. Scrypt has a 2-clause BSD 
> license which would also be OK.
> 
> Opinions?

We already have integration points to openssl, why add yet another
dependency?  Seems foolish.

headers

Stefan Fritsch | 24 Jun 2012 00:01

Re: Choosing a stronger password hash algorithm

On Saturday 23 June 2012, William A. Rowe Jr. wrote:
> On 6/23/2012 3:42 PM, Stefan Fritsch wrote:
> > bcrypt [1] and scrypt [2] seem to be much more difficult to port
> > to hardware or GPUs than sha256/512_crypt [3-5]. We can't make
> > the operation too expensive on normal CPUs or we create a DoS
> > problem if someone does lots of requests with wrong passwords.
> > Therefore I think choosing an algorithm that does not scale well
> > on more specialized hardware is good.
> > 
> > Both bcrypt and scrypt can be adjusted in how much CPU time to
> > use. scrypt can also be adjusted in how much RAM it uses. bcrypt
> > uses a 128bit salt, AFAICS scrypt can use arbitrary salt
> > lengths.
> > 
> > Bcrypt has seen a lot more review than scrypt, therefore I am
> > somewhat in favor of bcrypt. Crypt_blowfish [6] is an
> > implementation with a very liberal license that we could use.
> > Scrypt has a 2-clause BSD license which would also be OK.
> > 
> > Opinions?
> 
> We already have integration points to openssl, why add yet another
> dependency?  Seems foolish.

Openssl is not required, neither for apr nor for httpd. I propose to 
import either crypt_blowfish or scrypt into apr, just like apr 
contains some foreign sha1 and md5 code. This way we would not get an 
additional external dependency.

headers

Graham Leggett | 24 Jun 2012 00:51

Re: Choosing a stronger password hash algorithm

On 24 Jun 2012, at 12:01 AM, Stefan Fritsch wrote:

> Openssl is not required, neither for apr nor for httpd. I propose to 
> import either crypt_blowfish or scrypt into apr, just like apr 
> contains some foreign sha1 and md5 code. This way we would not get an 
> additional external dependency.

APR-util has a crypto library to deal with this exact problem - the need for low level crypto functions
without having to tightly bind ourselves to one toolkit over another, or import code. With the formal move
by the Redhat people towards NSS as a shared crypto API, this becomes more important as time goes by.

Ideally, like we have a generic synchronous encryption API, we should have a generic hash API too, so that
the user can use whatever hash that the underlying toolkit provides.

Regards,
Graham
--

Attachment (smime.p7s): application/pkcs7-signature, 4365 bytes

headers

Stefan Fritsch | 24 Jun 2012 10:34

Re: Choosing a stronger password hash algorithm

On Sunday 24 June 2012, Graham Leggett wrote:
> On 24 Jun 2012, at 12:01 AM, Stefan Fritsch wrote:
> > Openssl is not required, neither for apr nor for httpd. I propose
> > to import either crypt_blowfish or scrypt into apr, just like
> > apr contains some foreign sha1 and md5 code. This way we would
> > not get an additional external dependency.
> 
> APR-util has a crypto library to deal with this exact problem - the
> need for low level crypto functions without having to tightly bind
> ourselves to one toolkit over another, or import code. With the
> formal move by the Redhat people towards NSS as a shared crypto
> API, this becomes more important as time goes by.
> 
> Ideally, like we have a generic synchronous encryption API, we
> should have a generic hash API too, so that the user can use
> whatever hash that the underlying toolkit provides.

I rather like the fact that you can use htpasswd on one system and use 
the result on another system, regardless of the operating system. If 
we are willing to give that up, we may just make htpasswd use the more 
advanced schemes offered by the system's crypt() function.

Also, using what is available in the crypto libraries would limit us 
to PKCS5/PBKDF2, which is the only password hashing algorithm 
supported by openssl (AFAICS). Since PBKDF2 is based on sha256, 
though, it scales relatively well on GPUs. Of course, we could also 
implement other schemes on top of some cryptographhic hash or cipher 
provided by the crypto libraries. But I would rather avoid that 
because it's a lot of work to verify the result.

headers

Steve Marquess | 25 Jun 2012 00:45

Re: Choosing a stronger password hash algorithm

On 06/24/2012 04:34 AM, Stefan Fritsch wrote:
> On Sunday 24 June 2012, Graham Leggett wrote:
>> On 24 Jun 2012, at 12:01 AM, Stefan Fritsch wrote:
>>> Openssl is not required, neither for apr nor for httpd. I propose
>>> to import either crypt_blowfish or scrypt into apr, just like
>>> apr contains some foreign sha1 and md5 code. This way we would
>>> not get an additional external dependency.
>>
>> APR-util has a crypto library to deal with this exact problem - the
>> need for low level crypto functions without having to tightly bind
>> ourselves to one toolkit over another, or import code. With the
>> formal move by the Redhat people towards NSS as a shared crypto
>> API, this becomes more important as time goes by.
>>
>> Ideally, like we have a generic synchronous encryption API, we
>> should have a generic hash API too, so that the user can use
>> whatever hash that the underlying toolkit provides.
> 
> I rather like the fact that you can use htpasswd on one system and use 
> the result on another system, regardless of the operating system. If 
> we are willing to give that up, we may just make htpasswd use the more 
> advanced schemes offered by the system's crypt() function.

One complication to keep in mind: when you don't do all your
cryptography via a specific crypto library (OpenSSL, NSS, etc.) then
FIPS 140-2 compliance goes from trivial (for 2.4) to messy. Not
generally a problem outside of the U.S., but it very much matters
anywhere in the U.S. government market.

-Steve M.

--

-- 
Steve Marquess
OpenSSL Software Foundation, Inc.
1829 Mount Ephraim Road
Adamstown, MD  21710
USA
+1 877 673 6775 s/b
+1 301 874 2571 direct
marquess <at> opensslfoundation.com
marquess <at> openssl.com

headers

Stefan Fritsch | 25 Jun 2012 21:37

Re: Choosing a stronger password hash algorithm

On Monday 25 June 2012, Steve Marquess wrote:
> > I rather like the fact that you can use htpasswd on one system
> > and use  the result on another system, regardless of the
> > operating system. If we are willing to give that up, we may just
> > make htpasswd use the more advanced schemes offered by the
> > system's crypt() function.
> 
> One complication to keep in mind: when you don't do all your
> cryptography via a specific crypto library (OpenSSL, NSS, etc.)
> then FIPS 140-2 compliance goes from trivial (for 2.4) to messy.
> Not generally a problem outside of the U.S., but it very much
> matters anywhere in the U.S. government market.

The APR-MD5 password hashing is already implemented in apr-util and 
does not use an external crypto library. Would another password hashin 
algorithm chang anything? Or is it already necessary for FIPS 
compliance to patch apr-util or httpd?

Cheers,
Stefan

headers

Steve Marquess | 25 Jun 2012 22:34

Re: Choosing a stronger password hash algorithm

On 06/25/2012 03:37 PM, Stefan Fritsch wrote:
> ...
>>
>> One complication to keep in mind: when you don't do all your
>> cryptography via a specific crypto library (OpenSSL, NSS, etc.)
>> then FIPS 140-2 compliance goes from trivial (for 2.4) to messy.
>> Not generally a problem outside of the U.S., but it very much
>> matters anywhere in the U.S. government market.
> 
> The APR-MD5 password hashing is already implemented in apr-util and 
> does not use an external crypto library. Would another password hashin 
> algorithm chang anything? Or is it already necessary for FIPS 
> compliance to patch apr-util or httpd?

MD5 isn't allowed in FIPS mode (with the peculiar exception of use for
TLS proper).

Note that the "FIPS capable" OpenSSL (OpenSSL built with the "fips"
build time option in the presence of a validated FIPS module) will
automatically disable use of disallowed cryptography when the FIPS mode
of operation is enabled.

The awkward thing about FIPS 140-2 validated cryptography is that it
offers absolutely no tangible advantage over otherwise comparable
non-validated cryptographic implementations -- it isn't more secure,
performance isn't better, and so forth. There is only one reason to use
it: because such use is mandated for certain environments. But, one of
those environments (U.S. government) is a huge market. As a consultant
I've been paid good money for many years to hack FIPS 140-2 (and other
kinds) of compliance into open source products like mod_ssl and OpenSSH.
That patching is getting easier over time but is still necessary for
many such products.

An open source based FIPS validated module is available for use by
anyone at no cost (the OpenSSL FIPS Object Module 1.2, soon to be joined
by 2.0). But, the need to patch many OSS products to use it is a
deterrent to many end users. Make it easy to build a FIPS compliant
httpd without patches and help put consultants like me out of business 

-Steve M.

--

-- 
Steve Marquess
OpenSSL Software Foundation, Inc.
1829 Mount Ephraim Road
Adamstown, MD  21710
USA
+1 877 673 6775 s/b
+1 301 874 2571 direct
marquess <at> opensslfoundation.com
marquess <at> openssl.com

headers

William A. Rowe Jr. | 26 Jun 2012 00:13

Re: Choosing a stronger password hash algorithm

On 6/24/2012 3:34 AM, Stefan Fritsch wrote:
> On Sunday 24 June 2012, Graham Leggett wrote:
>>
>> Ideally, like we have a generic synchronous encryption API, we
>> should have a generic hash API too, so that the user can use
>> whatever hash that the underlying toolkit provides.
> 
> I rather like the fact that you can use htpasswd on one system and use 
> the result on another system, regardless of the operating system. If 
> we are willing to give that up, we may just make htpasswd use the more 
> advanced schemes offered by the system's crypt() function.

You misunderstand.  The algorithms need to exist in the apr crypt
implementation.

We might choose to provide 'fallback' implementations if the are absent.

This gets you to things like FIPS-140-2 implementations when APR is
correctly configured and built, without the hassle of validating our
own implementation.  We aren't put in a position of implementing the
assembly language optimized flavor, because the library vendor has
already done this.

Nobody is talking about crypt(), although generic implementations of
crypt() are offered by openssl and likely from gnutls etc.

> Also, using what is available in the crypto libraries would limit us 
> to PKCS5/PBKDF2, which is the only password hashing algorithm 
> supported by openssl (AFAICS). Since PBKDF2 is based on sha256, 
> though, it scales relatively well on GPUs. Of course, we could also 
> implement other schemes on top of some cryptographhic hash or cipher 
> provided by the crypto libraries. But I would rather avoid that 
> because it's a lot of work to verify the result.

Take a quick glance through these two drafts;

http://csrc.nist.gov/publications/drafts/800-107/Draft_Revised_SP800-107.pdf
http://csrc.nist.gov/publications/drafts/800-118/draft-sp800-118.pdf

I think we may be dwelling too much on computation time.  The larger issue
is better seeding.  An illicitly obtained password file of hashed p/w's
will always be a risk to the accounts, irrespective of computation time.
What is critical is the seeding such that duplicate passwords are not
obvious, and that passwords recycled on other services don't demonstrate
the same hash (ergo, collision) value.

headers

Stefan Fritsch | 29 Jun 2012 11:23

Re: Choosing a stronger password hash algorithm

On Tuesday 26 June 2012, William A. Rowe Jr. wrote:
> On 6/24/2012 3:34 AM, Stefan Fritsch wrote:
> > On Sunday 24 June 2012, Graham Leggett wrote:
> >> Ideally, like we have a generic synchronous encryption API, we
> >> should have a generic hash API too, so that the user can use
> >> whatever hash that the underlying toolkit provides.
> > 
> > I rather like the fact that you can use htpasswd on one system
> > and use the result on another system, regardless of the
> > operating system. If we are willing to give that up, we may just
> > make htpasswd use the more advanced schemes offered by the
> > system's crypt() function.
> 
> You misunderstand.  The algorithms need to exist in the apr crypt
> implementation.

Passing some salt string like "$6$" to crypt() is just a different way 
to call a library. And on FIPS compliant systems, the system's 
password hashes are likely also FIPS compliant. But this approach 
would not work for platforms like Windows that don't have crypt().

> We might choose to provide 'fallback' implementations if the are
> absent.
> 
> This gets you to things like FIPS-140-2 implementations when APR is
> correctly configured and built, without the hassle of validating
> our own implementation.  We aren't put in a position of
> implementing the assembly language optimized flavor, because the
> library vendor has already done this.
>
> Nobody is talking about crypt(), although generic implementations
> of crypt() are offered by openssl and likely from gnutls etc.

There is some confusion here between crypt() the function and crypt 
the algorithm (i.e. DES-crypt). On Linux and BSDs, crypt() supports 
more secure algorithms than DES-crypt.

> 
> > Also, using what is available in the crypto libraries would limit
> > us to PKCS5/PBKDF2, which is the only password hashing algorithm
> > supported by openssl (AFAICS). Since PBKDF2 is based on sha256,
> > though, it scales relatively well on GPUs. Of course, we could
> > also implement other schemes on top of some cryptographhic hash
> > or cipher provided by the crypto libraries. But I would rather
> > avoid that because it's a lot of work to verify the result.
> 
> Take a quick glance through these two drafts;
> 
> 
http://csrc.nist.gov/publications/drafts/800-107/Draft_Revised_SP800-107.pdf
> http://csrc.nist.gov/publications/drafts/800-118/draft-sp800-118.pdf
>
> I think we may be dwelling too much on computation time.  The
> larger issue is better seeding.  An illicitly obtained password
> file of hashed p/w's will always be a risk to the accounts,
> irrespective of computation time. What is critical is the seeding
> such that duplicate passwords are not obvious, and that passwords
> recycled on other services don't demonstrate the same hash (ergo,
> collision) value.

The first paper doesn't mention password hashing at all. The second 
one mentions stretching only to make creation of rainbow tables more 
difficult. But increasing computation time by stretching is also 
important for simple brute force attacks without rainbow tables. And 
the reason why the md5crypt author has recently declared his algorithm 
insecure is the too low computation time and nothing else.

The salting definitely can be improved. htpasswd only uses 2^32 
different values on a given libc. htdbm only uses the timestamp as 
salt.

Apart from that I think that we should do both things: Add a good non-
FIPS compliant algorithm that is built into APR and a FIPS compliant 
variant (whatever that is). The former is straightforward but the 
latter isn't. For example with PBKDF2, you have to define an output 
format because none is defined yet. Also, one must choose a suitable 
size of the resulting key. And I don't really know if PBKDF2 is 
actually FIPS compliant. Then there is the question of the API: Would 
we add a replacement function for apr_password_validate() that also 
takes an apr_crypto_driver_t argument? Or would we make some 
apr_password_set_driver() function that sets the driver to be used by 
apr_password_validate()? Do we also need a way to disable non-FIPS 
compliant algorithms?

Therefore I will go ahead and add bcrypt support to apr-util which 
will be a big improvement for the 95% of users who don't need FIPS. I 
would be willing to help anyone who wants to add a FIPS compliant 
scheme, but I am not going to do the necessary research myself. And I 
don't think we should delay the release of a new apr-util with bcrypt 
support for a FIPS compliant scheme that may or may not be added in 
the future.

Cheers,
Stefan

headers

William A. Rowe Jr. | 1 Jul 2012 21:48

Re: Choosing a stronger password hash algorithm

On 6/29/2012 4:23 AM, Stefan Fritsch wrote:
> 
> Therefore I will go ahead and add bcrypt support to apr-util which 
> will be a big improvement for the 95% of users who don't need FIPS.

It sounds to me that after we spent a great deal of time to make APR
largely library agnostic, you are insisting on binding us to a specific
library.  I would be very hostile to that.

If you are talking about a plugable, client-agnostic approach which
the user can ignore the details of how apr was built, that would be
fine.

Before throwing code at APR, please post a patch, because the group
might largely be in agreement.  The delta to configure.in and the
headers is probably sufficient for now.  But if your solution is to
add more mandatory dependencies or make apr less flexible, it would
be met with resistance.

The user should remain largely oblivious whether the pw crypt used
lives in bcrypt or openssl or the kernel, and the resulting pw data
should be always portable between different builds of apr.  This was
the logic behind the sha1, md5 and other common password schemes.
The crypt scheme was an exception, one which can be resolved even
on win32 and hpux with the use of openssl's crypt implementation
or linking to fcrypt.

Platform or library binding dependent pw data takes us back to 1999.

headers

Stefan Fritsch | 2 Jul 2012 21:02

Re: Choosing a stronger password hash algorithm

On Sunday 01 July 2012, William A. Rowe Jr. wrote:
> On 6/29/2012 4:23 AM, Stefan Fritsch wrote:
> > Therefore I will go ahead and add bcrypt support to apr-util
> > which will be a big improvement for the 95% of users who don't
> > need FIPS.
> 
> It sounds to me that after we spent a great deal of time to make
> APR largely library agnostic, you are insisting on binding us to a
> specific library.  I would be very hostile to that.
> 

No. As I have mentioned in one of the other mails, I prefer importing 
a suitably licensed version of the source code.

What I currently have is

http://people.apache.org/~sf/apr_bcrypt_password_support.diff

diffstat:

 build.conf              |    2 
 crypto/apr_md5.c        |  145 -------
 crypto/apr_passwd.c     |  196 ++++++++++
 crypto/crypt_blowfish.c |  902 ++++++++++++++++++++++++++++++++++++++
 crypto/crypt_blowfish.h |   27 +
 include/apr_md5.h       |    5 

> If you are talking about a plugable, client-agnostic approach which
> the user can ignore the details of how apr was built, that would be
> fine.
> 
> Before throwing code at APR, please post a patch, because the group
> might largely be in agreement.  The delta to configure.in and the
> headers is probably sufficient for now.  But if your solution is to
> add more mandatory dependencies or make apr less flexible, it would
> be met with resistance.

No new dependency, no configure.in change. Instead I add two files 
from the crypt_blowfish distribution. I intend to make some benchmarks 
with the assembler version, but it says it was written for Pentium 1, 
so I am not so sure that it will greatly improve performance on 
current processors. If it is actually significantly faster than the 
pure C version, I would add it plus the necessary build logic. That 
would mean one additional file (there is only an assembler version for 
i386).

The diff of crypt_blowfish.c versus upstream is one line. So importing 
new versions if there happen to be any upgrades should be easy.

> 
> The user should remain largely oblivious whether the pw crypt used
> lives in bcrypt or openssl or the kernel, and the resulting pw data
> should be always portable between different builds of apr.  This
> was the logic behind the sha1, md5 and other common password
> schemes. The crypt scheme was an exception, one which can be
> resolved even on win32 and hpux with the use of openssl's crypt
> implementation or linking to fcrypt.
> 
> Platform or library binding dependent pw data takes us back to
> 1999.

Yes. That's why I was sceptical about using apr_crypto for this task, 
too. After all, apr_crypto is still optional.

Cheers,
Stefan

headers

Ben Laurie | 2 Jul 2012 21:15

Re: Choosing a stronger password hash algorithm

FWIW, I am not super-keen on this particular move. Whilst bcrypt is
certainly an improvement, I am wary of relying on Blowfish - it is not
a mainstream cipher and is not subject to the kind of scrutiny that,
say, AES or SHA-2/3 are.

If we are going to change, then we should change to a mechanism that
is subject to ongoing cryptanalysis.

On Mon, Jul 2, 2012 at 8:02 PM, Stefan Fritsch <sf <at> sfritsch.de> wrote:
> On Sunday 01 July 2012, William A. Rowe Jr. wrote:
>> On 6/29/2012 4:23 AM, Stefan Fritsch wrote:
>> > Therefore I will go ahead and add bcrypt support to apr-util
>> > which will be a big improvement for the 95% of users who don't
>> > need FIPS.
>>
>> It sounds to me that after we spent a great deal of time to make
>> APR largely library agnostic, you are insisting on binding us to a
>> specific library.  I would be very hostile to that.
>>
>
> No. As I have mentioned in one of the other mails, I prefer importing
> a suitably licensed version of the source code.
>
> What I currently have is
>
> http://people.apache.org/~sf/apr_bcrypt_password_support.diff
>
> diffstat:
>
>  build.conf              |    2
>  crypto/apr_md5.c        |  145 -------
>  crypto/apr_passwd.c     |  196 ++++++++++
>  crypto/crypt_blowfish.c |  902 ++++++++++++++++++++++++++++++++++++++
>  crypto/crypt_blowfish.h |   27 +
>  include/apr_md5.h       |    5
>
>
>> If you are talking about a plugable, client-agnostic approach which
>> the user can ignore the details of how apr was built, that would be
>> fine.
>>
>> Before throwing code at APR, please post a patch, because the group
>> might largely be in agreement.  The delta to configure.in and the
>> headers is probably sufficient for now.  But if your solution is to
>> add more mandatory dependencies or make apr less flexible, it would
>> be met with resistance.
>
> No new dependency, no configure.in change. Instead I add two files
> from the crypt_blowfish distribution. I intend to make some benchmarks
> with the assembler version, but it says it was written for Pentium 1,
> so I am not so sure that it will greatly improve performance on
> current processors. If it is actually significantly faster than the
> pure C version, I would add it plus the necessary build logic. That
> would mean one additional file (there is only an assembler version for
> i386).
>
> The diff of crypt_blowfish.c versus upstream is one line. So importing
> new versions if there happen to be any upgrades should be easy.
>
>>
>> The user should remain largely oblivious whether the pw crypt used
>> lives in bcrypt or openssl or the kernel, and the resulting pw data
>> should be always portable between different builds of apr.  This
>> was the logic behind the sha1, md5 and other common password
>> schemes. The crypt scheme was an exception, one which can be
>> resolved even on win32 and hpux with the use of openssl's crypt
>> implementation or linking to fcrypt.
>>
>> Platform or library binding dependent pw data takes us back to
>> 1999.
>
> Yes. That's why I was sceptical about using apr_crypto for this task,
> too. After all, apr_crypto is still optional.
>
> Cheers,
> Stefan

headers

Stefan Fritsch | 2 Jul 2012 21:46

Re: Choosing a stronger password hash algorithm

On Monday 02 July 2012, Ben Laurie wrote:
> FWIW, I am not super-keen on this particular move. Whilst bcrypt is
> certainly an improvement, I am wary of relying on Blowfish - it is
> not a mainstream cipher and is not subject to the kind of scrutiny
> that, say, AES or SHA-2/3 are.
> 
> If we are going to change, then we should change to a mechanism
> that is subject to ongoing cryptanalysis.

bcrypt has the advantage that it currently does not give much speed-up 
of GPUs versus CPUs. So brute-forcing is more difficult than e.g. for 
glibc's sha256 or sha512 based algorithms. And we can't just 
arbitrarily increase the number of rounds because that would make 
httpd prone to DoS attacks. My rationale for bcrypt is here:

http://mail-archives.apache.org/mod_mbox/apr-
dev/201206.mbox/%3C201206232242.42669.sf%40sfritsch.de%3E

Your comments on that would be welcome.

Due to Poul-Henning Kamp's declaration that md5crypt is insecure, 
there is some renewed interest in this field. Maybe there will be a 
new algorithm soon that is both difficult to brute-force on GPUs and 
based on something standard like AES or SHA*.

Maybe we could add bcrypt for now and if something better appears, 
then add that as well?

Cheers,
Stefan

headers

Ben Laurie | 2 Jul 2012 21:54

Re: Choosing a stronger password hash algorithm

On Mon, Jul 2, 2012 at 8:46 PM, Stefan Fritsch <sf <at> sfritsch.de> wrote:
> On Monday 02 July 2012, Ben Laurie wrote:
>> FWIW, I am not super-keen on this particular move. Whilst bcrypt is
>> certainly an improvement, I am wary of relying on Blowfish - it is
>> not a mainstream cipher and is not subject to the kind of scrutiny
>> that, say, AES or SHA-2/3 are.
>>
>> If we are going to change, then we should change to a mechanism
>> that is subject to ongoing cryptanalysis.
>
>
> bcrypt has the advantage that it currently does not give much speed-up
> of GPUs versus CPUs. So brute-forcing is more difficult than e.g. for
> glibc's sha256 or sha512 based algorithms. And we can't just
> arbitrarily increase the number of rounds because that would make
> httpd prone to DoS attacks. My rationale for bcrypt is here:
>
> http://mail-archives.apache.org/mod_mbox/apr-
> dev/201206.mbox/%3C201206232242.42669.sf%40sfritsch.de%3E
>
> Your comments on that would be welcome.

I don't have any response beyond what I said above. I agree about the
GPU vs CPU thing, though I'd really advocate for sufficient salt and
good passwords!

> Due to Poul-Henning Kamp's declaration that md5crypt is insecure,
> there is some renewed interest in this field. Maybe there will be a
> new algorithm soon that is both difficult to brute-force on GPUs and
> based on something standard like AES or SHA*.
>
> Maybe we could add bcrypt for now and if something better appears,
> then add that as well?

I guess. I admit I find it hard to imagine that bcrypt would be broken
any time soon. I wish there was a better answer.

headers

Stefan Fritsch | 29 Jun 2012 11:23

Re: Choosing a stronger password hash algorithm

On Tuesday 26 June 2012, William A. Rowe Jr. wrote:
> On 6/24/2012 3:34 AM, Stefan Fritsch wrote:
> > On Sunday 24 June 2012, Graham Leggett wrote:
> >> Ideally, like we have a generic synchronous encryption API, we
> >> should have a generic hash API too, so that the user can use
> >> whatever hash that the underlying toolkit provides.
> > 
> > I rather like the fact that you can use htpasswd on one system
> > and use the result on another system, regardless of the
> > operating system. If we are willing to give that up, we may just
> > make htpasswd use the more advanced schemes offered by the
> > system's crypt() function.
> 
> You misunderstand.  The algorithms need to exist in the apr crypt
> implementation.

Passing some salt string like "$6$" to crypt() is just a different way 
to call a library. And on FIPS compliant systems, the system's 
password hashes are likely also FIPS compliant. But this approach 
would not work for platforms like Windows that don't have crypt().

> We might choose to provide 'fallback' implementations if the are
> absent.
> 
> This gets you to things like FIPS-140-2 implementations when APR is
> correctly configured and built, without the hassle of validating
> our own implementation.  We aren't put in a position of
> implementing the assembly language optimized flavor, because the
> library vendor has already done this.
>
> Nobody is talking about crypt(), although generic implementations
> of crypt() are offered by openssl and likely from gnutls etc.

There is some confusion here between crypt() the function and crypt 
the algorithm (i.e. DES-crypt). On Linux and BSDs, crypt() supports 
more secure algorithms than DES-crypt.

> 
> > Also, using what is available in the crypto libraries would limit
> > us to PKCS5/PBKDF2, which is the only password hashing algorithm
> > supported by openssl (AFAICS). Since PBKDF2 is based on sha256,
> > though, it scales relatively well on GPUs. Of course, we could
> > also implement other schemes on top of some cryptographhic hash
> > or cipher provided by the crypto libraries. But I would rather
> > avoid that because it's a lot of work to verify the result.
> 
> Take a quick glance through these two drafts;
> 
> 
http://csrc.nist.gov/publications/drafts/800-107/Draft_Revised_SP800-107.pdf
> http://csrc.nist.gov/publications/drafts/800-118/draft-sp800-118.pdf
>
> I think we may be dwelling too much on computation time.  The
> larger issue is better seeding.  An illicitly obtained password
> file of hashed p/w's will always be a risk to the accounts,
> irrespective of computation time. What is critical is the seeding
> such that duplicate passwords are not obvious, and that passwords
> recycled on other services don't demonstrate the same hash (ergo,
> collision) value.

The first paper doesn't mention password hashing at all. The second 
one mentions stretching only to make creation of rainbow tables more 
difficult. But increasing computation time by stretching is also 
important for simple brute force attacks without rainbow tables. And 
the reason why the md5crypt author has recently declared his algorithm 
insecure is the too low computation time and nothing else.

The salting definitely can be improved. htpasswd only uses 2^32 
different values on a given libc. htdbm only uses the timestamp as 
salt.

Apart from that I think that we should do both things: Add a good non-
FIPS compliant algorithm that is built into APR and a FIPS compliant 
variant (whatever that is). The former is straightforward but the 
latter isn't. For example with PBKDF2, you have to define an output 
format because none is defined yet. Also, one must choose a suitable 
size of the resulting key. And I don't really know if PBKDF2 is 
actually FIPS compliant. Then there is the question of the API: Would 
we add a replacement function for apr_password_validate() that also 
takes an apr_crypto_driver_t argument? Or would we make some 
apr_password_set_driver() function that sets the driver to be used by 
apr_password_validate()? Do we also need a way to disable non-FIPS 
compliant algorithms?

Therefore I will go ahead and add bcrypt support to apr-util which 
will be a big improvement for the 95% of users who don't need FIPS. I 
would be willing to help anyone who wants to add a FIPS compliant 
scheme, but I am not going to do the necessary research myself. And I 
don't think we should delay the release of a new apr-util with bcrypt 
support for a FIPS compliant scheme that may or may not be added in 
the future.

Cheers,
Stefan

headers

William A. Rowe Jr. | 26 Jun 2012 00:13

Re: Choosing a stronger password hash algorithm

On 6/24/2012 3:34 AM, Stefan Fritsch wrote:
> On Sunday 24 June 2012, Graham Leggett wrote:
>>
>> Ideally, like we have a generic synchronous encryption API, we
>> should have a generic hash API too, so that the user can use
>> whatever hash that the underlying toolkit provides.
> 
> I rather like the fact that you can use htpasswd on one system and use 
> the result on another system, regardless of the operating system. If 
> we are willing to give that up, we may just make htpasswd use the more 
> advanced schemes offered by the system's crypt() function.

You misunderstand.  The algorithms need to exist in the apr crypt
implementation.

We might choose to provide 'fallback' implementations if the are absent.

This gets you to things like FIPS-140-2 implementations when APR is
correctly configured and built, without the hassle of validating our
own implementation.  We aren't put in a position of implementing the
assembly language optimized flavor, because the library vendor has
already done this.

Nobody is talking about crypt(), although generic implementations of
crypt() are offered by openssl and likely from gnutls etc.

> Also, using what is available in the crypto libraries would limit us 
> to PKCS5/PBKDF2, which is the only password hashing algorithm 
> supported by openssl (AFAICS). Since PBKDF2 is based on sha256, 
> though, it scales relatively well on GPUs. Of course, we could also 
> implement other schemes on top of some cryptographhic hash or cipher 
> provided by the crypto libraries. But I would rather avoid that 
> because it's a lot of work to verify the result.

Take a quick glance through these two drafts;

http://csrc.nist.gov/publications/drafts/800-107/Draft_Revised_SP800-107.pdf
http://csrc.nist.gov/publications/drafts/800-118/draft-sp800-118.pdf

I think we may be dwelling too much on computation time.  The larger issue
is better seeding.  An illicitly obtained password file of hashed p/w's
will always be a risk to the accounts, irrespective of computation time.
What is critical is the seeding such that duplicate passwords are not
obvious, and that passwords recycled on other services don't demonstrate
the same hash (ergo, collision) value.

headers

Stefan Fritsch | 24 Jun 2012 10:34

Re: Choosing a stronger password hash algorithm

On Sunday 24 June 2012, Graham Leggett wrote:
> On 24 Jun 2012, at 12:01 AM, Stefan Fritsch wrote:
> > Openssl is not required, neither for apr nor for httpd. I propose
> > to import either crypt_blowfish or scrypt into apr, just like
> > apr contains some foreign sha1 and md5 code. This way we would
> > not get an additional external dependency.
> 
> APR-util has a crypto library to deal with this exact problem - the
> need for low level crypto functions without having to tightly bind
> ourselves to one toolkit over another, or import code. With the
> formal move by the Redhat people towards NSS as a shared crypto
> API, this becomes more important as time goes by.
> 
> Ideally, like we have a generic synchronous encryption API, we
> should have a generic hash API too, so that the user can use
> whatever hash that the underlying toolkit provides.

I rather like the fact that you can use htpasswd on one system and use 
the result on another system, regardless of the operating system. If 
we are willing to give that up, we may just make htpasswd use the more 
advanced schemes offered by the system's crypt() function.

Also, using what is available in the crypto libraries would limit us 
to PKCS5/PBKDF2, which is the only password hashing algorithm 
supported by openssl (AFAICS). Since PBKDF2 is based on sha256, 
though, it scales relatively well on GPUs. Of course, we could also 
implement other schemes on top of some cryptographhic hash or cipher 
provided by the crypto libraries. But I would rather avoid that 
because it's a lot of work to verify the result.

headers

Graham Leggett | 24 Jun 2012 00:51

Re: Choosing a stronger password hash algorithm

On 24 Jun 2012, at 12:01 AM, Stefan Fritsch wrote:

> Openssl is not required, neither for apr nor for httpd. I propose to 
> import either crypt_blowfish or scrypt into apr, just like apr 
> contains some foreign sha1 and md5 code. This way we would not get an 
> additional external dependency.

APR-util has a crypto library to deal with this exact problem - the need for low level crypto functions
without having to tightly bind ourselves to one toolkit over another, or import code. With the formal move
by the Redhat people towards NSS as a shared crypto API, this becomes more important as time goes by.

Ideally, like we have a generic synchronous encryption API, we should have a generic hash API too, so that
the user can use whatever hash that the underlying toolkit provides.

Regards,
Graham
--

Attachment (smime.p7s): application/pkcs7-signature, 4365 bytes

headers

Nick Edwards | 12 Jul 2012 13:03

Re: Choosing a stronger password hash algorithm

Away on holidays, late reply, apologies.

I would like to see bcrypt introduced, a number of sites seem to be
moving that way, would be nice if apache did too!

On 6/24/12, Stefan Fritsch <sf <at> sfritsch.de> wrote:
> On Saturday 23 June 2012, William A. Rowe Jr. wrote:
>> On 6/23/2012 3:42 PM, Stefan Fritsch wrote:
>> > bcrypt [1] and scrypt [2] seem to be much more difficult to port
>> > to hardware or GPUs than sha256/512_crypt [3-5]. We can't make
>> > the operation too expensive on normal CPUs or we create a DoS
>> > problem if someone does lots of requests with wrong passwords.
>> > Therefore I think choosing an algorithm that does not scale well
>> > on more specialized hardware is good.
>> >
>> > Both bcrypt and scrypt can be adjusted in how much CPU time to
>> > use. scrypt can also be adjusted in how much RAM it uses. bcrypt
>> > uses a 128bit salt, AFAICS scrypt can use arbitrary salt
>> > lengths.
>> >
>> > Bcrypt has seen a lot more review than scrypt, therefore I am
>> > somewhat in favor of bcrypt. Crypt_blowfish [6] is an
>> > implementation with a very liberal license that we could use.
>> > Scrypt has a 2-clause BSD license which would also be OK.
>> >
>> > Opinions?
>>
>> We already have integration points to openssl, why add yet another
>> dependency?  Seems foolish.
>
> Openssl is not required, neither for apr nor for httpd. I propose to
> import either crypt_blowfish or scrypt into apr, just like apr
> contains some foreign sha1 and md5 code. This way we would not get an
> additional external dependency.
>

headers

Stefan Fritsch | 24 Jun 2012 00:01

Re: Choosing a stronger password hash algorithm

On Saturday 23 June 2012, William A. Rowe Jr. wrote:
> On 6/23/2012 3:42 PM, Stefan Fritsch wrote:
> > bcrypt [1] and scrypt [2] seem to be much more difficult to port
> > to hardware or GPUs than sha256/512_crypt [3-5]. We can't make
> > the operation too expensive on normal CPUs or we create a DoS
> > problem if someone does lots of requests with wrong passwords.
> > Therefore I think choosing an algorithm that does not scale well
> > on more specialized hardware is good.
> > 
> > Both bcrypt and scrypt can be adjusted in how much CPU time to
> > use. scrypt can also be adjusted in how much RAM it uses. bcrypt
> > uses a 128bit salt, AFAICS scrypt can use arbitrary salt
> > lengths.
> > 
> > Bcrypt has seen a lot more review than scrypt, therefore I am
> > somewhat in favor of bcrypt. Crypt_blowfish [6] is an
> > implementation with a very liberal license that we could use.
> > Scrypt has a 2-clause BSD license which would also be OK.
> > 
> > Opinions?
> 
> We already have integration points to openssl, why add yet another
> dependency?  Seems foolish.

Openssl is not required, neither for apr nor for httpd. I propose to 
import either crypt_blowfish or scrypt into apr, just like apr 
contains some foreign sha1 and md5 code. This way we would not get an 
additional external dependency.

headers

Stefan Fritsch | 23 Jun 2012 22:42

Choosing a stronger password hash algorithm

On Thursday 21 June 2012, Ben Laurie wrote:
> Then the question is: what scheme? Here are some design criteria I
> think would be useful.
> 
> 1. Use something from the SHA-2 family - SHA-512 would seem fine to
> me.

The sha message digests are designed to be fast and to be easily 
implemented in hardware. In general, the speed-up when going from 
general purpose CPU implementations to parallel implementations using 
GPU or specialized hardware seems to be relatively high. Therefore I 
think what is generally used as sha256_crypt or sha512_crypt is not 
the optimal choice. [1] has some thoughts about choosing a password 
hashing algorithm. 

> 
> 2. Use a very large salt - disk space is not at a premium for
> password stores!

Agreed. 64bit should be absolute minimum, more would be better.

> 3. Use quite a lot of rounds,.

The number should be configurable so that we can adjust the cost 
without breaking backward compatibility.

> 4. Use something that is hard to optimise in hardware (ideally).

bcrypt [1] and scrypt [2] seem to be much more difficult to port to 
hardware or GPUs than sha256/512_crypt [3-5]. We can't make the 
operation too expensive on normal CPUs or we create a DoS problem if 
someone does lots of requests with wrong passwords. Therefore I think 
choosing an algorithm that does not scale well on more specialized 
hardware is good.

Both bcrypt and scrypt can be adjusted in how much CPU time to use. 
scrypt can also be adjusted in how much RAM it uses. bcrypt uses a 
128bit salt, AFAICS scrypt can use arbitrary salt lengths.

Bcrypt has seen a lot more review than scrypt, therefore I am somewhat 
in favor of bcrypt. Crypt_blowfish [6] is an implementation with a 
very liberal license that we could use. Scrypt has a 2-clause BSD 
license which would also be OK.

Opinions?

Cheers,
Stefan

[1] http://static.usenix.org/event/usenix99/provos.html
[2] http://www.tarsnap.com/scrypt.html
[3] http://stackoverflow.com/a/6807375
[4] http://www.openwall.com/lists/john-dev/2012/05/14/1
[5] http://www.openwall.com/lists/john-dev/2012/05/14/4
[6] http://www.openwall.com/crypt/