List about the Sun Java Enterprise System, Sun ONE, and iPlanet communication and directory software products

Rolf E. Sonneveld | 25 Apr 2012 23:44

CRAM-MD5 and storage of password

Hi, all

running Messaging Server 7.0 update 4 patch level 18 (soon patchlevel 24).

I wonder whether the following can be done. In http://docs.oracle.com/cd/E19566-01/819-4428/bgbau/index.html it is described that, when one wants to use one of the challenge response mechanisms, the password must be stored in cleartext in the directory. An alternative can be, to use the PLAIN mechanism and protect the password by using a TLS encrypted transport.

Is it absolutely impossible to use CRAM-MD5 and still store passwords in a hashed format in the directory?

The server sends a challenge to the client. Suppose the client would not use a function:

hashfunction(challenge,password)

but instead would use:

hashfunction(challenge,MD5(password))

and the MD5 hash of the password would be stored in the directory. For MD5, read any hash function that would fit. Is this setup feasible?

Or more in general: do I have only two choices:

password send in cleartext, password not in cleartext stored
password send in hash form, password stored in cleartext

or is there an alternative (assuming there are clients that can do SASL, but that can't do TLS)?

/rolf

headers

Ned Freed | 26 Apr 2012 02:35

Re: CRAM-MD5 and storage of password

> Hi, all

> running Messaging Server  7.0 update 4 patch level 18 (soon patchlevel 24).

> I wonder whether the following can be done. In
> http://docs.oracle.com/cd/E19566-01/819-4428/bgbau/index.html it is
> described that, when one wants to use one of the challenge response
> mechanisms, the password must be stored in cleartext in the directory.
> An alternative can be, to use the PLAIN mechanism and protect the
> password by using a TLS encrypted transport.

> Is it absolutely impossible to use CRAM-MD5 and still store passwords in
> a hashed format in the directory?

Yes. As defined, CRAM-MD5 operates by computing a hash of the cleartext
password including a server-supplied hash.

Algorithmically you could substitute a hash of the password for the password.
But this raises a bunch of issues:

(1) The client and the server have to agree or they won't interoperate.
    CRAM-MD5 is widely deployed; if you make this change and keep on
    calling it CRAM-MD5 you're going to have major problems.

(2) You can call it something else, say CRAM-MD5-HASHED. But now you have
    a different problem: Nothing out there supports it.

(3) If all you need to log in is the hash of your password, you've effectively
    promoted to the same role as the cleartext password. If, say, I steal
    you password database, the fact that the passwords are hashed won't
    prevent me from using them to log in to all your users accounts and
    do bad things. 

    The only thing this really prevents is someone stealing your password
    database and using the content directly with a regular client that
    supports your CRAM-MD5-HASH scheme (assuming such a client exists). But
    I rather suspect that anyone who does this won't have a problem finding
    or creating a specialized client to do this.

In any case, this entire discussion is moot, because there's already a
well-defined mechanism that is as good as it gets without resorting to public
key operations: SCRAM (RFC 5802).

In particular, SCRAM doesn't require storage of plaintext passwords and it
doesn't directly expose things on the wire. You can still mount a brute-force
password guessing attack on either an observed exchange or a stolen password
file, but SCRAM mitigates this to the greatest extent possible by iterating
hash functions.

> The server sends a challenge to the client. Suppose the client would not
> use a function:

> hashfunction(challenge,password)

> but instead would use:

> hashfunction(challenge,MD5(password))

> and the MD5 hash of the password would be stored in the directory. For
> MD5, read any hash function that would fit. Is this setup feasible?

It's feasible but not practical. See above.

> Or more in general: do I have only two choices:

>  1. password send in cleartext, password not in cleartext stored
>  2. password send in hash form, password stored in cleartext

> or is there an alternative (assuming there are clients that can do SASL,
> but that can't do TLS)?

SCRAM is the alternative you're looking for. Unfortunately, I don't believe
we've implemented support for it yet, although I suppose you could do it
yourself with the authentication plugin. But that doesn't get you client
support, of course.

				Ned

headers

Chris Newman | 26 Apr 2012 18:56

Re: CRAM-MD5 and storage of password

--On April 25, 2012 17:35:42 -0700 Ned Freed <ned.freed@...> wrote:
>> or is there an alternative (assuming there are clients that can do SASL,
>> but that can't do TLS)?
>
> SCRAM is the alternative you're looking for. Unfortunately, I don't 
believe
> we've implemented support for it yet, although I suppose you could do it
> yourself with the authentication plugin. But that doesn't get you client
> support, of course.

It would be possible (and not too difficult) to add SCRAM support (RFC 
5802/5803) to Messaging Server (7u4 or later) using our third-party 
authentication server interface (although the optional channel binding 
feature would require some enhancements to the interface). As I'm 
personally interested in seeing SCRAM succeed, I'd be willing to write the 
server code (and I'm willing to do so in my free time if necessary as my 
work time is very constrained). But that still leaves the problem of client 
support. If you managed to get one or two client vendors on board, we could 
get it working.

Another option is Kerberos/GSSAPI. We've had one customer get that working 
through the third-party authentication server interface (by implementing 
RFC 4752). I find Kerberos expensive in terms of management and client 
configuration costs, but at least there are client implementations 
available.

CRAM-MD5 has too many design flaws, so it's basically considered useless 
from a security perspective these days. Ditto for APOP.

		- Chris

headers

Rolf E. Sonneveld | 26 Apr 2012 23:26

Re: CRAM-MD5 and storage of password

Hi, Chris,

On 4/26/12 6:56 PM, Chris Newman wrote:
> --On April 25, 2012 17:35:42 -0700 Ned Freed <ned.freed@...> 
> wrote:
>>> or is there an alternative (assuming there are clients that can do 
>>> SASL,
>>> but that can't do TLS)?
>>
>> SCRAM is the alternative you're looking for. Unfortunately, I don't 
> believe
>> we've implemented support for it yet, although I suppose you could do it
>> yourself with the authentication plugin. But that doesn't get you client
>> support, of course.
>
> It would be possible (and not too difficult) to add SCRAM support (RFC 
> 5802/5803) to Messaging Server (7u4 or later) using our third-party 
> authentication server interface (although the optional channel binding 
> feature would require some enhancements to the interface). 

Can you elaborate on this channel binding feature. Do you mean the 
mailSMTPSubmitChannel feature?

> As I'm personally interested in seeing SCRAM succeed, I'd be willing 
> to write the server code (and I'm willing to do so in my free time if 
> necessary as my work time is very constrained). But that still leaves 
> the problem of client support. If you managed to get one or two client 
> vendors on board, we could get it working.

Anyone contacts within the MUA industry? If SCRAM was added to the Java 
libraries, it would be a major step forward. Well, Java is owned by 
Oracle, so...

>
> Another option is Kerberos/GSSAPI. We've had one customer get that 
> working through the third-party authentication server interface (by 
> implementing RFC 4752). I find Kerberos expensive in terms of 
> management and client configuration costs, but at least there are 
> client implementations available.

When using third-party authentication, what does it mean for 
patches/upgrades of the messaging server? Will it keep working?

> CRAM-MD5 has too many design flaws, so it's basically considered 
> useless from a security perspective these days. Ditto for APOP.

So to summarize: the best thing for now is: use SASL PLAIN and encrypt 
the transport layer, is that correct?

/rolf

headers

Rolf E. Sonneveld | 26 Apr 2012 23:01

Re: CRAM-MD5 and storage of password

On 4/26/12 2:35 AM, Ned Freed wrote:
>> Hi, all
>
>> running Messaging Server  7.0 update 4 patch level 18 (soon 
>> patchlevel 24).
>
>> I wonder whether the following can be done. In
>> http://docs.oracle.com/cd/E19566-01/819-4428/bgbau/index.html it is
>> described that, when one wants to use one of the challenge response
>> mechanisms, the password must be stored in cleartext in the directory.
>> An alternative can be, to use the PLAIN mechanism and protect the
>> password by using a TLS encrypted transport.
>
>> Is it absolutely impossible to use CRAM-MD5 and still store passwords in
>> a hashed format in the directory?
>
> Yes. As defined, CRAM-MD5 operates by computing a hash of the cleartext
> password including a server-supplied hash.
>
> Algorithmically you could substitute a hash of the password for the 
> password.
> But this raises a bunch of issues:
>
> (1) The client and the server have to agree or they won't interoperate.
>    CRAM-MD5 is widely deployed; if you make this change and keep on
>    calling it CRAM-MD5 you're going to have major problems.
>
> (2) You can call it something else, say CRAM-MD5-HASHED. But now you have
>    a different problem: Nothing out there supports it.
>
> (3) If all you need to log in is the hash of your password, you've 
> effectively
>    promoted to the same role as the cleartext password. If, say, I steal
>    you password database, the fact that the passwords are hashed won't
>    prevent me from using them to log in to all your users accounts and
>    do bad things.
>    The only thing this really prevents is someone stealing your password
>    database and using the content directly with a regular client that
>    supports your CRAM-MD5-HASH scheme (assuming such a client exists). 
> But
>    I rather suspect that anyone who does this won't have a problem 
> finding
>    or creating a specialized client to do this.

Yep, so at the end of the day it will not solve the problem.

>
> In any case, this entire discussion is moot, because there's already a
> well-defined mechanism that is as good as it gets without resorting to 
> public
> key operations: SCRAM (RFC 5802).
>
> In particular, SCRAM doesn't require storage of plaintext passwords 
> and it
> doesn't directly expose things on the wire. You can still mount a 
> brute-force
> password guessing attack on either an observed exchange or a stolen 
> password
> file, but SCRAM mitigates this to the greatest extent possible by 
> iterating
> hash functions.
>
>> The server sends a challenge to the client. Suppose the client would not
>> use a function:
>
>> hashfunction(challenge,password)
>
>> but instead would use:
>
>> hashfunction(challenge,MD5(password))
>
>> and the MD5 hash of the password would be stored in the directory. For
>> MD5, read any hash function that would fit. Is this setup feasible?
>
> It's feasible but not practical. See above.
>
>> Or more in general: do I have only two choices:
>
>>  1. password send in cleartext, password not in cleartext stored
>>  2. password send in hash form, password stored in cleartext
>
>> or is there an alternative (assuming there are clients that can do SASL,
>> but that can't do TLS)?
>
> SCRAM is the alternative you're looking for. Unfortunately, I don't 
> believe
> we've implemented support for it yet, although I suppose you could do it
> yourself with the authentication plugin. But that doesn't get you client
> support, of course.

Thanks for your answer. I understand and agree it's the infamous chicken 
and egg problem (as usual with standards ), but the minimum I can do 
is to file an RFE for this customer and hope it will get sufficient 
'momentum' to get it implemented, hoping that client support will sooner 
or later follow.

If cleartext storage of passwords in the directory is enabled (and 
configured in the mail server), will it be possible to support 
simultaneously CRAM-MD5 and PLAIN?

/rolf

headers

Chris Newman | 27 Apr 2012 02:12

Re: CRAM-MD5 and storage of password

--On April 26, 2012 23:01:33 +0200 "Rolf E. Sonneveld" 
<R.E.Sonneveld@...> wrote:
> Thanks for your answer. I understand and agree it's the infamous chicken
> and egg problem (as usual with standards ), but the minimum I can do
> is to file an RFE for this customer and hope it will get sufficient
> 'momentum' to get it implemented, hoping that client support will sooner
> or later follow.

I'd personally appreciate if you get the RFE created.

> If cleartext storage of passwords in the directory is enabled (and
> configured in the mail server), will it be possible to support
> simultaneously CRAM-MD5 and PLAIN?

Yes.

--On April 26, 2012 23:26:11 +0200 "Rolf E. Sonneveld" 
<R.E.Sonneveld@...> wrote:

> Hi, Chris,
>
> On 4/26/12 6:56 PM, Chris Newman wrote:
>> --On April 25, 2012 17:35:42 -0700 Ned Freed <ned.freed@...>
>> wrote:
>>>> or is there an alternative (assuming there are clients that can do
>>>> SASL,
>>>> but that can't do TLS)?
>>>
>>> SCRAM is the alternative you're looking for. Unfortunately, I don't
>> believe
>>> we've implemented support for it yet, although I suppose you could do it
>>> yourself with the authentication plugin. But that doesn't get you client
>>> support, of course.
>>
>> It would be possible (and not too difficult) to add SCRAM support (RFC
>> 5802/5803) to Messaging Server (7u4 or later) using our third-party
>> authentication server interface (although the optional channel binding
>> feature would require some enhancements to the interface).
>
> Can you elaborate on this channel binding feature. Do you mean the
> mailSMTPSubmitChannel feature?

No. I'm talking about RFC 5802's "Channel binding" feature described in 
section 6 (unrelated to MTA channels). It allows a SCRAM authentication to 
be bound to an SSL session so that SCRAM+SSL is significantly more secure 
than passwords+SSL and provides true mutual authentication that can't be 
intercepted. SCRAM was designed to be useful with or without SSL, raising 
the security level significantly in both cases relative to plaintext 
passwords.

>> As I'm personally interested in seeing SCRAM succeed, I'd be willing
>> to write the server code (and I'm willing to do so in my free time if
>> necessary as my work time is very constrained). But that still leaves
>> the problem of client support. If you managed to get one or two client
>> vendors on board, we could get it working.
>
> Anyone contacts within the MUA industry? If SCRAM was added to the Java
> libraries, it would be a major step forward. Well, Java is owned by
> Oracle, so...

I have a couple contacts, but none who would be able to spend 
implementation time on a client authentication feature at my request.

>> Another option is Kerberos/GSSAPI. We've had one customer get that
>> working through the third-party authentication server interface (by
>> implementing RFC 4752). I find Kerberos expensive in terms of
>> management and client configuration costs, but at least there are
>> client implementations available.
>
> When using third-party authentication, what does it mean for
> patches/upgrades of the messaging server? Will it keep working?

Yes, the inter-process protocol is a stable documented protocol.

>> CRAM-MD5 has too many design flaws, so it's basically considered
>> useless from a security perspective these days. Ditto for APOP.
>
> So to summarize: the best thing for now is: use SASL PLAIN and encrypt
> the transport layer, is that correct?

For most usage scenarios, PLAIN+SSL/TLS is the right choice today (and 
that's a sad statement about the importance that's placed on authentication 
security by the industry). Client certificate authentication is also 
possible and more secure, but it tends to be suitable mostly for high 
security environments that can afford to spend a lot on PKI management.

		- Chris

headers

Rolf E. Sonneveld | 27 Apr 2012 22:20

Re: CRAM-MD5 and storage of password

Hi, Chris,

On 4/27/12 2:12 AM, Chris Newman wrote:
> --On April 26, 2012 23:01:33 +0200 "Rolf E. Sonneveld" 
> <R.E.Sonneveld@...> wrote:
>> Thanks for your answer. I understand and agree it's the infamous chicken
>> and egg problem (as usual with standards ), but the minimum I can do
>> is to file an RFE for this customer and hope it will get sufficient
>> 'momentum' to get it implemented, hoping that client support will sooner
>> or later follow.
>
> I'd personally appreciate if you get the RFE created.

The RFE I filed today got ERB (Enhancement Request Bug) number 14012027. 
If anyone on the list would be interested in this functionality, please 
add your name/support ID to this ERB.

>
>> If cleartext storage of passwords in the directory is enabled (and
>> configured in the mail server), will it be possible to support
>> simultaneously CRAM-MD5 and PLAIN?
>
> Yes.

OK, thanks.

>
> --On April 26, 2012 23:26:11 +0200 "Rolf E. Sonneveld" 
> <R.E.Sonneveld@...> wrote:
>
>> Hi, Chris,
>>
>> On 4/26/12 6:56 PM, Chris Newman wrote:
>>> --On April 25, 2012 17:35:42 -0700 Ned Freed <ned.freed@...>
>>> wrote:
>>>>> or is there an alternative (assuming there are clients that can do
>>>>> SASL,
>>>>> but that can't do TLS)?
>>>>
>>>> SCRAM is the alternative you're looking for. Unfortunately, I don't
>>> believe
>>>> we've implemented support for it yet, although I suppose you could 
>>>> do it
>>>> yourself with the authentication plugin. But that doesn't get you 
>>>> client
>>>> support, of course.
>>>
>>> It would be possible (and not too difficult) to add SCRAM support (RFC
>>> 5802/5803) to Messaging Server (7u4 or later) using our third-party
>>> authentication server interface (although the optional channel binding
>>> feature would require some enhancements to the interface).
>>
>> Can you elaborate on this channel binding feature. Do you mean the
>> mailSMTPSubmitChannel feature?
>
> No. I'm talking about RFC 5802's "Channel binding" feature described 
> in section 6 (unrelated to MTA channels). It allows a SCRAM 
> authentication to be bound to an SSL session so that SCRAM+SSL is 
> significantly more secure than passwords+SSL and provides true mutual 
> authentication that can't be intercepted. SCRAM was designed to be 
> useful with or without SSL, raising the security level significantly 
> in both cases relative to plaintext passwords.

OK.

>
>>> As I'm personally interested in seeing SCRAM succeed, I'd be willing
>>> to write the server code (and I'm willing to do so in my free time if
>>> necessary as my work time is very constrained). But that still leaves
>>> the problem of client support. If you managed to get one or two client
>>> vendors on board, we could get it working.
>>
>> Anyone contacts within the MUA industry? If SCRAM was added to the Java
>> libraries, it would be a major step forward. Well, Java is owned by
>> Oracle, so...
>
> I have a couple contacts, but none who would be able to spend 
> implementation time on a client authentication feature at my request.

In the past there were companies (like Innosoft ) that implemented 
technologies to anticipate on customer requests. Supporting SCRAM in 
Comms Suite while other MTA vendors do not yet support it is a 
competitive advantage (well, let's be realistic, it is one of a list of 
features that can make up a competitive advantage).

>
>>> Another option is Kerberos/GSSAPI. We've had one customer get that
>>> working through the third-party authentication server interface (by
>>> implementing RFC 4752). I find Kerberos expensive in terms of
>>> management and client configuration costs, but at least there are
>>> client implementations available.
>>
>> When using third-party authentication, what does it mean for
>> patches/upgrades of the messaging server? Will it keep working?
>
> Yes, the inter-process protocol is a stable documented protocol.

Excuse me, I'm not sure my question was clear and I'm not sure how to 
interpret your answer. What I intended to say is the following: suppose 
a customer implements Kerberos/GSSAPI using the 3rd party authentication 
server interface for the current version (7.0u4-24) and it works. Now, 
suppose later this year version 7.1 of Messaging Server is released and 
the customer decides to upgrade, then my question is: will the 
Kerberos/GSSAPI software written by the customer against the 3rd party 
auth server interface keep working, or will it require modifications and 
re-testing. I suppose your answer means, that an upgrade of the mail 
server will have no impact on the customer written code, but I'm not 
sure if that is what I should read in it.

>
>>> CRAM-MD5 has too many design flaws, so it's basically considered
>>> useless from a security perspective these days. Ditto for APOP.
>>
>> So to summarize: the best thing for now is: use SASL PLAIN and encrypt
>> the transport layer, is that correct?
>
> For most usage scenarios, PLAIN+SSL/TLS is the right choice today (and 
> that's a sad statement about the importance that's placed on 
> authentication security by the industry). Client certificate 
> authentication is also possible and more secure, but it tends to be 
> suitable mostly for high security environments that can afford to 
> spend a lot on PKI management.

Agreed.

/rolf

headers

Chris Newman | 28 Apr 2012 00:00

Re: CRAM-MD5 and storage of password

--On April 27, 2012 22:20:01 +0200 "Rolf E. Sonneveld" 
<R.E.Sonneveld@...> wrote:
> The RFE I filed today got ERB (Enhancement Request Bug) number 14012027.
> If anyone on the list would be interested in this functionality, please
> add your name/support ID to this ERB.

Thank you.

>>>> Another option is Kerberos/GSSAPI. We've had one customer get that
>>>> working through the third-party authentication server interface (by
>>>> implementing RFC 4752). I find Kerberos expensive in terms of
>>>> management and client configuration costs, but at least there are
>>>> client implementations available.
>>>
>>> When using third-party authentication, what does it mean for
>>> patches/upgrades of the messaging server? Will it keep working?
>>
>> Yes, the inter-process protocol is a stable documented protocol.
>
> Excuse me, I'm not sure my question was clear and I'm not sure how to
> interpret your answer. What I intended to say is the following: suppose a
> customer implements Kerberos/GSSAPI using the 3rd party authentication
> server interface for the current version (7.0u4-24) and it works. Now,
> suppose later this year version 7.1 of Messaging Server is released and
> the customer decides to upgrade, then my question is: will the
> Kerberos/GSSAPI software written by the customer against the 3rd party
> auth server interface keep working, or will it require modifications and
> re-testing. I suppose your answer means, that an upgrade of the mail
> server will have no impact on the customer written code, but I'm not sure
> if that is what I should read in it.

Yes, that's correct. The third party authentication interface works over a 
socket rather than via a shared library. That means the software written by 
the customer is built as a separate binary that does not link against 
Messaging Server software. So a change in Messaging Server software has no 
impact on the customer binary. As the protocol is documented/stable, the 
customer binary will continue to work with new versions.

Of course, if we add new features to the interface, the customer code would 
need to be rebuilt to take advantage of those new features. But new 
features would be added such that they're optional and the present 
functionality will continue to work.

		- Chris