gmane.comp.lang.jython.devel

Leo Soto M. | 1 Aug 00:43

On String, PyString and PyUnicode

Hi folks,

Short History:

I don't think that java Strings should be mapped to PyString. They
should be mapped to PyUnicode. Any obvious reason why I may be wrong?

Whole History:

I was trying to implement unicodedata.normalize. It ought to be
straightforward, by delegating to java.text.Normalizer.normalize.

Here is the code:

from java.text import Normalizer;

_forms = {
    'NFC':  Normalizer.Form.NFC,
    'NFKC': Normalizer.Form.NFKC,
    'NFD':  Normalizer.Form.NFD,
    'NFKD': Normalizer.Form.NFKD
}
def normalize(form, unistr):
    """
    Return the normal form 'form' for the Unicode string unistr.  Valid
    values for form are 'NFC', 'NFKC', 'NFD', and 'NFKD'.
    """
    return Normalizer.normalize(unistr, _forms[form])

But it isn't right. Normalizer.normalize returns a String, which is

(Continue reading)

headers

Frank Wierzbicki | 1 Aug 02:46

Re: On String, PyString and PyUnicode

On Thu, Jul 31, 2008 at 6:46 PM, Leo Soto M. <leo.soto <at> gmail.com> wrote:
> Hi folks,
>
> Short History:
>
> I don't think that java Strings should be mapped to PyString. They
> should be mapped to PyUnicode. Any obvious reason why I may be wrong?
Ah now I understand what you where getting at in IRC.  I think you are
right.  The only reason for the current behavior is that PyString is
much older than PyUnicode, and so the mapping is just outdated.  Not
sure if there are backwards compatibility issues that will cause big
problems -- it may be a pretty big change... but I am all for giving
it a try.  It certainly makes sense.

-Frank

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/

headers

Jim Baker | 1 Aug 05:42

Re: On String, PyString and PyUnicode

I agree, it makes more sense to map java.lang.String to PyUnicode. PyUnicode is of course just a wrapper around String, sharing the same UTF-16 encoding.

Is there any code that depends on this? The only artificial usage of PyString to represent 8-bit strings should be in Jython itself, such as in PyFile or cStringIO. And there's the rub, our own internals return String from a PyFile#readline, etc., depending on Jython to make the conversion to PyString. I suspect the damage is limited to this, however, simply because the distinction is generally preserved for other code via how TYPE interacts with constructors or factories.

We may also see user-level Jython code depending on this behavior, but I suspect it would actually make it work better. Just looked at this old doc, http://jython.sourceforge.net/docs/differences.html : "The u"" string modifier is optional and completely ignored if specified." Not anymore, and that's true of much of the rest of the doc. BTW, 2.5 final must include a purging of all such obsolete docs, or at least qualifying them to a prior release.

- Jim

So if we were to refactor PyFile and similar modules to return PyString instead, it's quite likely

On Thu, Jul 31, 2008 at 6:46 PM, Frank Wierzbicki <fwierzbicki <at> gmail.com> wrote:

On Thu, Jul 31, 2008 at 6:46 PM, Leo Soto M. <leo.soto <at> gmail.com> wrote:
> Hi folks,
>
> Short History:
>
> I don't think that java Strings should be mapped to PyString. They
> should be mapped to PyUnicode. Any obvious reason why I may be wrong?

Ah now I understand what you where getting at in IRC. I think you are
right. The only reason for the current behavior is that PyString is
much older than PyUnicode, and so the mapping is just outdated. Not
sure if there are backwards compatibility issues that will cause big
problems -- it may be a pretty big change... but I am all for giving
it a try. It certainly makes sense.

-Frank

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Jython-dev mailing list
Jython-dev <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/jython-dev

--
Jim Baker
jbaker <at> zyasoft.com

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/

_______________________________________________
Jython-dev mailing list
Jython-dev <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/jython-dev