headers
Eli Zaretskii | 4 Aug 08:25
Picon

Re: merge emacs-bidi into the main tree

> From: Alex Schroeder <alex <at> emacswiki.org>
> Date: Mon, 04 Aug 2003 03:39:31 +0200
> 
> "Eli Zaretskii" <eliz <at> elta.co.il> writes:
> 
> > I'd rather ask on emacs-devel for volunteers to carry on the original
> > plan.  I can coordinate the effort if that would help (assuming people
> > who'd like to work on this agree for me to be the coordinator).
> 
> Sadly I don't have much hope at the moment -- it hasn't worked
> before, why should it work now?

Perhaps for the same reason you now think the m17n.org code is a
viable solution--because lots of time have passed, and a bidi-capable
Emacs is now more important than before.  Meanwile, some of the
developers have more experience in hacking redisplay code, so their
response might be different this time, especially if the plea sounds
convincing.

> Merging the current solution will give users something to work with
> while we wait for the volunteer we need to emerge.

Someone will have to invest a lot of work to make the m17n.org
solution be satisfactory on the user level.  AFAIK, no one is doing
that now, so we still need volunteers to work on issues like keyboard
input and various aspects of encoding bidirectional text.

Please don't misunderstand me: this is nowhere close to the NIH thing.
I know everything about the virtues of existing solutions.  I saw the
m17n.org code in action when I was in Japan several years ago and had
(Continue reading)

Permalink | Reply |
headers
Alex Schroeder | 4 Aug 16:54
Face

merge emacs-bidi into the main tree

Some people have been waiting for a long time for a bidi-enabled Emacs
-- at least I have been waiting for more than a year.  The strange
thing is that we have a working implementation:  emacs-bidi is
available from www.m17n.org, and has been since Emacs 20.3 or earlier.

There are a few imperfections, or so I hear, but when I -- as a
non-Arabic-speaker -- saw the Arabic glyphs rendered correctly on IRC
(using ERC, the Emacs IRC client) -- I was surprised!  Yes, there may
be problems with the existing solution, but at least there *is* a
solution.

I would therefore like to see the existing emacs-bidi implementation
merged with our CVS version.

Back at in the days when Gerd was maintainer, I heard he had issues
with the emacs-bidi implementation and didn't want to use it.  Perhaps
performance was bad.  Perhaps he thought changes to the redisplay code
would be necessary, and he felt that these changes had to wait.
Perhaps you could not turn bidi off at the time.

Well, for somebody like me who just wants to learn Arabic, copy Arabic
song titles from old tapes onto his website, or read the occasional
Arabic greeting on IRC, the current solution is good enough.  And I
don't feel we should wait any longer.  We have waited long enough.

If the reasons for rejecting the current emacs-bidi are still valid,
let's list them and discuss them again.  In the mean time a few people
like Jan D and Kim Storm have had quite some exposure to the redisplay
code, so perhaps we do have the necessary know-how now to fix the
remaining issues.  That would be wonderful.  Eli has already done
(Continue reading)

Permalink | Reply |
headers
Richard Stallman | 7 Aug 08:05
Picon
Picon

Re: merge emacs-bidi into the main tree

    Some people have been waiting for a long time for a bidi-enabled Emacs
    -- at least I have been waiting for more than a year.  The strange
    thing is that we have a working implementation:  emacs-bidi is
    available from www.m17n.org, and has been since Emacs 20.3 or earlier.

I would expect that it needs merging to be able to be installed
into the current sources.  Is someone interested in doing that work?

    There are a few imperfections, or so I hear, but when I -- as a
    non-Arabic-speaker -- saw the Arabic glyphs rendered correctly on IRC
    (using ERC, the Emacs IRC client) -- I was surprised!  Yes, there may
    be problems with the existing solution, but at least there *is* a
    solution.

What do Hebrew and Arab users think of it?
Eli Z, what is your opinion?

Handa, what do you think?
Permalink | Reply |
headers
Nadim Shaikli | 9 Aug 20:43
Picon
Favicon

Re: merge emacs-bidi into the main tree

--- Richard Stallman wrote:
>
[snip snip]
>   --- Alex Schroeder wrote:
>     There are a few imperfections, or so I hear, but when I -- as a
>     non-Arabic-speaker -- saw the Arabic glyphs rendered correctly on IRC
>     (using ERC, the Emacs IRC client) -- I was surprised!  Yes, there may
>     be problems with the existing solution, but at least there *is* a
>     solution.
> 
> What do Hebrew and Arab users think of it?

Arabic support in m17n's emacs code was very usable and almost complete.
There weren't that many issues left with it (most were minor) from a user's
prospective, our only concern was making sure that excellent effort made
its way into emacs' main trunk and I can see that there is movement in
that direction which I too applaud and highly encourage (for whatever
it's worth :-)

For those that have not received all the emails, please read the following
thread (courtesy of the emacs' mailing-lists) --

  http://mail.gnu.org/archive/html/emacs-devel/2003-08/msg00039.html

Any other Arabic user comments (ie. Arabeyes people) ?

PS: Thanks Alex again for getting the ball rolling...

Regards,
(Continue reading)

Permalink | Reply |
headers
Richard Stallman | 11 Aug 14:54
Picon
Picon

Re: merge emacs-bidi into the main tree

    > What do Hebrew and Arab users think of it?

    Arabic support in m17n's emacs code was very usable and almost complete.

We were talking about the emacs-bidi code; your response
is about m17n's Emacs code.  Are you talking about the same
emacs-bidi code with a different name, or are you talking
about a different version?

Is the m17n's Emacs code installed in the Emacs CVS repository?
If yes, is it in the trunk or in a branch?
Permalink | Reply |
headers
Nadim Shaikli | 11 Aug 18:32
Picon
Favicon

Re: merge emacs-bidi into the main tree

--- Richard Stallman <rms <at> gnu.org> wrote:
>     > What do Hebrew and Arab users think of it?
> 
>     Arabic support in m17n's emacs code was very usable and almost complete.
> 
> We were talking about the emacs-bidi code; your response
> is about m17n's Emacs code.  Are you talking about the same
> emacs-bidi code with a different name, or are you talking
> about a different version?

Sorry, I was talking about the code that was developed and is
currently hosted by m17n,

  http://www.m17n.org/emacs-bidi/index.html

> Is the m17n's Emacs code installed in the Emacs CVS repository?
> If yes, is it in the trunk or in a branch?

As far as I know the code developed by m17n is _not_ in Emacs' CVS
repository (my info might be dated though).  What Alex had commented
on in his initial post, I believe, was with regard to m17n's code
(his usage with ERC, etc) and my comments were to that end as well.
m17n.org's additions are _very_ usable and accurate from a user's
prospective and they (or similar :-) ought to get folded-in.  As
there has been a plethora of posts on this topic and there is
movement in that direction, this post/opinion is non-consequential.

Sorry for any possible confusion.

 - Nadim
(Continue reading)

Permalink | Reply |
headers
Richard Stallman | 13 Aug 01:22
Picon
Picon

Re: merge emacs-bidi into the main tree

It sounds like there are three different bidi efforts for Emacs: the
emacs-bidi, the m17n, and Eli Z's code (not ready yet).  Is that
correct?

Has anyone looked at all three?
Permalink | Reply |
headers
Nadim Shaikli | 13 Aug 05:44
Picon
Favicon

Re: merge emacs-bidi into the main tree

--- Richard Stallman <rms <at> gnu.org> wrote:
> It sounds like there are three different bidi efforts for Emacs: the
> emacs-bidi, the m17n, and Eli Z's code (not ready yet).  Is that
> correct?

I believe there are only 2 efforts -- m17n's working code and Eli's
code now being looked into (emacs-bidi is a mailing-list ;-)

Regards,

 - Nadim

__________________________________
Do you Yahoo!?
Yahoo! SiteBuilder - Free, easy-to-use web site design software
http://sitebuilder.yahoo.com
Permalink | Reply |
headers
Kenichi Handa | 13 Aug 02:33

Re: merge emacs-bidi into the main tree

In article <E19miSn-0004AG-IU <at> fencepost.gnu.org>, Richard Stallman <rms <at> gnu.org> writes:
> It sounds like there are three different bidi efforts for Emacs: the
> emacs-bidi, the m17n, and Eli Z's code (not ready yet).  Is that
> correct?

No.  The emacs-bidi and the m17n are the same thing.  As
emacs-bidi is now distributed at www.m17n.org, it is
referred as "the m17n".

---
Ken'ichi HANDA
handa <at> m17n.org
Permalink | Reply |
headers
Richard Stallman | 14 Aug 05:08
Picon
Picon

Re: merge emacs-bidi into the main tree

    > It sounds like there are three different bidi efforts for Emacs: the
    > emacs-bidi, the m17n, and Eli Z's code (not ready yet).  Is that
    > correct?

    No.  The emacs-bidi and the m17n are the same thing.  As
    emacs-bidi is now distributed at www.m17n.org, it is
    referred as "the m17n".

So it sounds like some people are saying that the emacs-bidi code
is usable as it stands and is worth installing, and nobody thinks
it is useless.  Meanwhile, it looks like you are ready to make
use of Eli's code in the short term.

If we can use Eli's code soon enough, then it's not worth installing
the Emacs-bidi code; it would just be extra work.  On the other hand,
if it looks like the difficulty of using Eli's code is enough to make
it hard, then I think we should install the Emacs-bidi code now.
Permalink | Reply |
headers
Kenichi Handa | 15 Aug 07:51

Re: merge emacs-bidi into the main tree

In article <E19n8T9-00031X-S0 <at> fencepost.gnu.org>, Richard Stallman <rms <at> gnu.org> writes:
> So it sounds like some people are saying that the emacs-bidi code
> is usable as it stands and is worth installing, and nobody thinks
> it is useless.  Meanwhile, it looks like you are ready to make
> use of Eli's code in the short term.

> If we can use Eli's code soon enough, then it's not worth installing
> the Emacs-bidi code; it would just be extra work.  On the other hand,
> if it looks like the difficulty of using Eli's code is enough to make
> it hard, then I think we should install the Emacs-bidi code now.

Synchronizing emacs-unicode to HEAD will take a few weeks
more.  After that, I'll take a look at Eli's code and decide
what to do.  In any cases, I will install the changes for
BIDI in the branch emacs-unicode-2 because the current codes
of emacs-bidi depends on automatic-composition facitliy to
display Arabic and Hebrew, and that exists only in
emacs-unicode (and in emacs-bidi) version.  I'd like to
avoid the extra work of installing automatic-composition for
the current HEAD.

FYI, the automatic-composition is to call various character
composition functions from the dipslay routine dynamically
so that we don't have to run XXX-compose-region on file
reading and character inputting.

---
Ken'ichi HANDA
handa <at> m17n.org
(Continue reading)

Permalink | Reply |
headers
Richard Stallman | 17 Aug 02:36
Picon
Picon

Re: merge emacs-bidi into the main tree

    Synchronizing emacs-unicode to HEAD will take a few weeks
    more.  After that, I'll take a look at Eli's code and decide
    what to do.  In any cases, I will install the changes for
    BIDI in the branch emacs-unicode-2 because the current codes
    of emacs-bidi depends on automatic-composition facitliy to
    display Arabic and Hebrew, and that exists only in
    emacs-unicode (and in emacs-bidi) version.

I am not sure what this means.  What is the difference between the
branch emacs-unicode-2 and the branch emacs-unicode?
Permalink | Reply |
headers
Kenichi Handa | 18 Aug 02:24

Re: merge emacs-bidi into the main tree

In article <E19oBXI-0000Ro-TE <at> fencepost.gnu.org>, Richard Stallman <rms <at> gnu.org> writes:
>     Synchronizing emacs-unicode to HEAD will take a few weeks
>     more.  After that, I'll take a look at Eli's code and decide
>     what to do.  In any cases, I will install the changes for
>     BIDI in the branch emacs-unicode-2 because the current codes
>     of emacs-bidi depends on automatic-composition facitliy to
>     display Arabic and Hebrew, and that exists only in
>     emacs-unicode (and in emacs-bidi) version.

> I am not sure what this means.  What is the difference between the
> branch emacs-unicode-2 and the branch emacs-unicode?

To reflect the changes in HEAD to the Unicode version of
emacs, I made a branch emacs-unicode-2.  When I finish the
work, emacs-unicode branch will be an obsolete branch.

---
Ken'ichi HANDA
handa <at> m17n.org
Permalink | Reply |
headers
Alex Schroeder | 15 Aug 13:51
Face

Re: merge emacs-bidi into the main tree

Kenichi Handa <handa <at> m17n.org> writes:

> Synchronizing emacs-unicode to HEAD will take a few weeks
> more.  After that, I'll take a look at Eli's code and decide
> what to do.  In any cases, I will install the changes for
> BIDI in the branch emacs-unicode-2...

Well, too bad if it takes longer.  I am happy that things are finally
moving on the bidi subject.  Thanks for putting so much work into
the multilingualization of Emacs.  I really appreciate your efforts!

Alex.
--

-- 
http://www.emacswiki.org/alex/
There is no substitute for experience.
Permalink | Reply |
headers
Nadim Shaikli | 15 Aug 17:38
Picon
Favicon

Re: merge emacs-bidi into the main tree

--- Alex Schroeder <alex <at> emacswiki.org> wrote:
> Kenichi Handa <handa <at> m17n.org> writes:
> 
> > Synchronizing emacs-unicode to HEAD will take a few weeks
> > more.  After that, I'll take a look at Eli's code and decide
> > what to do.  In any cases, I will install the changes for
> > BIDI in the branch emacs-unicode-2...
> 
> Well, too bad if it takes longer.  I am happy that things are finally
> moving on the bidi subject.  Thanks for putting so much work into
> the multilingualization of Emacs.  I really appreciate your efforts!

We all appreciate these outstanding efforts :-)

Keep up the great work.

 - Nadim

__________________________________
Do you Yahoo!?
Yahoo! SiteBuilder - Free, easy-to-use web site design software
http://sitebuilder.yahoo.com
Permalink | Reply |
headers
Janusz S. Bień | 8 Aug 14:57
Picon
Picon

Re: merge emacs-bidi into the main tree

On Thu, 07 Aug 2003  emacs-bidi-request <at> gnu.org wrote:

[...]

> Date: 07 Aug 2003 16:29:32 +0200
> From: Eli Zaretskii <eliz <at> elta.co.il>
> Subject: [emacs-bidi] Re: merge emacs-bidi into the main tree

[...]

> I'd prefer to call for volunteers to continue the work I started,
> namely, change the display engine to support bidirectional editing
> without processing batches of buffer's text.  (I can elaborate if
> there's enough interest.)  I have some unpolished stand-alone code
> that does bidi reordering sequentially, which should fit very well
> into the Emacs display engine, but someone needs to take that code and
> integrate it into Emacs, since I probably won't have time any time
> soon enough for this to be practical.
> 
> If anyone steps forward willing to begin hacking the display code, I
> promise to help in any way I can.

I think the number of developers is a monotonic function of the number
of users, and emacs-bidi is already usable!

The best way to widen the users base is to include emacs-bidi in
popular Linux distributions. Therefore I would like to draw your attention to 
the Debian package which I descovered just today.

It is available at
(Continue reading)

Permalink | Reply |
headers
Tzafrir Cohen | 12 Aug 00:38
Picon
Favicon

Re: Re: merge emacs-bidi into the main tree

On Fri, Aug 08, 2003 at 02:57:51PM +0200, Janusz S. Bień wrote:
> On Thu, 07 Aug 2003  emacs-bidi-request <at> gnu.org wrote:
> 
> I think the number of developers is a monotonic function of the number
> of users, and emacs-bidi is already usable!
> 
> The best way to widen the users base is to include emacs-bidi in
> popular Linux distributions. Therefore I would like to draw your attention to 
> the Debian package which I descovered just today.
> 
> It is available at
> 
> http://www.pipapo.org/www/www/Files/emacsbidi

$ grep emacs /etc/apt/sources.list 
# bidi emacs
deb http://www.pipapo.org/www/www/Files/emacsbidi ./

I figured I am not an addicted emacs user, but being a debian user makes
me the perfect test-subject for the package

> 
> and can be installed in the standard way with apt-get,
> aptitude etc.
> 
> Here is its description:
> 
>  Package: emacsbidi
>  Version: 0.9-1
>  Section: editors
(Continue reading)

Permalink | Reply |
headers
Kenichi Handa | 12 Aug 07:26

Re: Re: merge emacs-bidi into the main tree

At first, I have no idea how the package is created.

In article <20030811223825.GH8406 <at> dira.dyndns.org>, Tzafrir Cohen <tzafrir <at> technion.ac.il> writes:
>   emacsbidi -fn heb8x13

> And typed some Hebrew chars using the X winodws's Hebrew keyboard. No
> bidi rendering sighted.

If you at least see correct Hebrew chars, please load
bidi.el by M-x load-library RET bidi RET, then you'll see
the new menu "Bidi" in the top menu-bar.  Then, turn on
menu-bar-≥Bidi->"Enable bidi display".

If you are using Arabic chars, please also turn on
menu-bar-≥Bidi->"Enable auto composition".

---
Ken'ichi HANDA
handa <at> m17n.org
Permalink | Reply |
headers
Tzafrir Cohen | 13 Aug 22:17
Picon
Favicon

Re: Re: merge emacs-bidi into the main tree

On Tue, Aug 12, 2003 at 02:26:16PM +0900, Kenichi Handa wrote:
> At first, I have no idea how the package is created.
> 
> In article <20030811223825.GH8406 <at> dira.dyndns.org>, Tzafrir Cohen <tzafrir <at> technion.ac.il> writes:
> >   emacsbidi -fn heb8x13
> 
> > And typed some Hebrew chars using the X winodws's Hebrew keyboard. No
> > bidi rendering sighted.
> 
> If you at least see correct Hebrew chars, please load
> bidi.el by M-x load-library RET bidi RET, then you'll see
> the new menu "Bidi" in the top menu-bar.  Then, turn on
> menu-bar-≥Bidi->"Enable bidi display".

Works now. 

Some other notes:

* bidi-ehud (or selecting the "ehud" menu item seems to have the same
  effect as turning off bidi display)

* Rant: Shouldn't it be built with gtk support? I got used to decent menus...

* I built a crued rpm package for RH7.3 at work. I have not yet found
  any tester there, and I have some reasons to suspect it is not good
  so I din't upload it for now. But if anybody's interested...

* Generally the process for building an rpm is to adapt an existing rpm,
  instead of writing one from scratch. If someone needs help on any
  other distro, let me know.
(Continue reading)

Permalink | Reply |
headers
TAKAHASHI Naoto | 22 Aug 12:00

emacs-bidi-0.9.1

Tzafrir Cohen writes:

> * bidi-ehud (or selecting the "ehud" menu item seems to have the same
>   effect as turning off bidi display)

I fixed this problem.  The new version is available from our ftp site.

  ftp://ftp.m17n.org/pub/mule/.notready/emacs-bidi-0.9.1.tar.gz

Please give it a try.

--

-- 
TAKAHASHI Naoto
ntakahas <at> m17n.org
http://www.m17n.org/ntakahas/
Permalink | Reply |
headers
TAKAHASHI Naoto | 15 Aug 10:27

Re: Re: merge emacs-bidi into the main tree

Tzafrir Cohen writes:

> * bidi-ehud (or selecting the "ehud" menu item seems to have the same
>   effect as turning off bidi display)

It works if you turn on "Reverse displaying orientation" in the Bidi
menu.  Give me some time to make it work in left-aligned windows.

--

-- 
TAKAHASHI Naoto
ntakahas <at> m17n.org
http://www.m17n.org/ntakahas/
Permalink | Reply |
headers
Janusz S. Bień | 12 Aug 06:52
Picon
Picon

emacsbidi package


I am afraid I don't know Debian well enough to be able to help, so I
forward the letter to Debian Bug Tracking System (and Joshua M. Kwan
who sent already some comments to BTS) and debian-emacsen list.

Best regards

Janusz

On Tue, 12 Aug 2003  Tzafrir Cohen <tzafrir <at> technion.ac.il> wrote:

> On Fri, Aug 08, 2003 at 02:57:51PM +0200, Janusz S. Bień wrote:
> > On Thu, 07 Aug 2003  emacs-bidi-request <at> gnu.org wrote:
> > 
> > I think the number of developers is a monotonic function of the number
> > of users, and emacs-bidi is already usable!
> > 
> > The best way to widen the users base is to include emacs-bidi in
> > popular Linux distributions. Therefore I would like to draw your attention to 
> > the Debian package which I descovered just today.
> > 
> > It is available at
> > 
> > http://www.pipapo.org/www/www/Files/emacsbidi
> 
> $ grep emacs /etc/apt/sources.list 
> # bidi emacs
> deb http://www.pipapo.org/www/www/Files/emacsbidi ./
> 
> I figured I am not an addicted emacs user, but being a debian user makes
(Continue reading)

Permalink | Reply |
headers
Janusz S. Bień | 12 Aug 06:52
Picon
Picon

Bug#184831: emacsbidi package


I am afraid I don't know Debian well enough to be able to help, so I
forward the letter to Debian Bug Tracking System (and Joshua M. Kwan
who sent already some comments to BTS) and debian-emacsen list.

Best regards

Janusz

On Tue, 12 Aug 2003  Tzafrir Cohen <tzafrir <at> technion.ac.il> wrote:

> On Fri, Aug 08, 2003 at 02:57:51PM +0200, Janusz S. Bień wrote:
> > On Thu, 07 Aug 2003  emacs-bidi-request <at> gnu.org wrote:
> > 
> > I think the number of developers is a monotonic function of the number
> > of users, and emacs-bidi is already usable!
> > 
> > The best way to widen the users base is to include emacs-bidi in
> > popular Linux distributions. Therefore I would like to draw your attention to 
> > the Debian package which I descovered just today.
> > 
> > It is available at
> > 
> > http://www.pipapo.org/www/www/Files/emacsbidi
> 
> $ grep emacs /etc/apt/sources.list 
> # bidi emacs
> deb http://www.pipapo.org/www/www/Files/emacsbidi ./
> 
> I figured I am not an addicted emacs user, but being a debian user makes
(Continue reading)

Permalink | Reply |
headers
Chahine M. HAMILA | 8 Aug 20:58

Re: merge emacs-bidi into the main tree

> I would expect that it needs merging to be able to be installed
> into the current sources.  Is someone interested in doing that work?

Hi, the screenshots look great, but I personally haven't used that one (and
don't know if I can right now since I'm not at my usual place and only have
a win32 machine for the few coming weeks).
Our resources on Arabeyes in terms of developers are very scarce, and my own
time resources are scarce too. But since I use Emacs a lot, I am definitely
interested in this, so if someone tells me what work is involved and where
it should be started, I could consider it if it fits within my time frame.
The problem is the code base of Emacs is huge and I certainly don't have the
necessary time to get into a fine understanding of it or even of every
detail of its architecture, but if the steps for merging the existing
patches are simpler than that I can undertake the job...
Permalink | Reply |
headers
Eli Zaretskii | 7 Aug 16:29
Picon

Re: merge emacs-bidi into the main tree

> From: Richard Stallman <rms <at> gnu.org>
> Date: Thu, 07 Aug 2003 02:05:07 -0400
> 
> What do Hebrew and Arab users think of it?
> Eli Z, what is your opinion?

I feel uneasy about using something about which Gerd was most unhappy
at the time this was discussed.

I'd prefer to call for volunteers to continue the work I started,
namely, change the display engine to support bidirectional editing
without processing batches of buffer's text.  (I can elaborate if
there's enough interest.)  I have some unpolished stand-alone code
that does bidi reordering sequentially, which should fit very well
into the Emacs display engine, but someone needs to take that code and
integrate it into Emacs, since I probably won't have time any time
soon enough for this to be practical.

If anyone steps forward willing to begin hacking the display code, I
promise to help in any way I can.
Permalink | Reply |
headers
Kenichi Handa | 8 Aug 13:44

Re: Re: merge emacs-bidi into the main tree

In article <ufzkd8rsj.fsf <at> elta.co.il>, Eli Zaretskii <eliz <at> elta.co.il> writes:
> I'd prefer to call for volunteers to continue the work I started,
> namely, change the display engine to support bidirectional editing
> without processing batches of buffer's text.  (I can elaborate if
> there's enough interest.)  I have some unpolished stand-alone code
> that does bidi reordering sequentially, which should fit very well
> into the Emacs display engine, but someone needs to take that code and
> integrate it into Emacs, since I probably won't have time any time
> soon enough for this to be practical.

> If anyone steps forward willing to begin hacking the display code, I
> promise to help in any way I can.

I'm now working to synchronize the emacs-unicode branch to
the latest HEAD code.  As there were many cosmetic and
restructuring changes in HEAD, it's not easy to adjust the
code of emacs-unicode.  I think it takes two or three weeks
more.

After that, I can start working on bidi.  I'd like to
incorporate your reordering routine and my display routine
of emacs-bidi into the emacs-unicode branch instead of
directly into HEAD to avoid another synching labor.

---
Ken'ichi HANDA
handa <at> m17n.org
Permalink | Reply |
headers
Alex Schroeder | 8 Aug 19:48
Face

Re: Re: merge emacs-bidi into the main tree

Kenichi Handa <handa <at> m17n.org> writes:

> I'm now working to synchronize the emacs-unicode branch to
> the latest HEAD code.  As there were many cosmetic and
> restructuring changes in HEAD, it's not easy to adjust the
> code of emacs-unicode.  I think it takes two or three weeks
> more.
>
> After that, I can start working on bidi.  I'd like to
> incorporate your reordering routine and my display routine
> of emacs-bidi into the emacs-unicode branch instead of
> directly into HEAD to avoid another synching labor.

That sounds like awesome news!  :)

Alex.
--

-- 
http://www.emacswiki.org/alex/
I was on holidays from 2003-07-01 to 2003-07-29
and have a lot of catching up to do.
Permalink | Reply |
headers
Eli Zaretskii | 8 Aug 18:04
Picon

Re: Re: merge emacs-bidi into the main tree

> Date: Fri, 8 Aug 2003 20:44:50 +0900 (JST)
> From: Kenichi Handa <handa <at> m17n.org>
> 
> I'd like to
> incorporate your reordering routine and my display routine
> of emacs-bidi into the emacs-unicode branch instead of
> directly into HEAD to avoid another synching labor.

That's just great.  I'll happily send you my code and provide any
other help you might need.

Thanks!
Permalink | Reply |
headers
Gerd Moellmann | 8 Aug 13:23
Picon
Favicon

Re: merge emacs-bidi into the main tree

Eli Zaretskii <eliz <at> elta.co.il> writes:

> I feel uneasy about using something about which Gerd was most unhappy
> at the time this was discussed.

IIRC, I was worried that it might be a dead end, sort of a hack, which
users of r2l languages wouldn't be happy with in the long term.  But I
have to confess that I don't remember the details.

[...]

> If anyone steps forward willing to begin hacking the display code, I
> promise to help in any way I can.

And maybe I'll recover from my burn-out at some point :).
Permalink | Reply |
headers
Richard Stallman | 9 Aug 16:21
Picon
Picon

Re: merge emacs-bidi into the main tree

    IIRC, I was worried that it might be a dead end, sort of a hack, which
    users of r2l languages wouldn't be happy with in the long term.

That may be true--but even if that is true, but might be a good thing
to install it now.  Sure, we would like something better such as Eli
is working on, but I don't think he will have it ready soon.
If that will take another two years, we may as well have the emacs-bidi
code in the mean time, rather than nothing.
Permalink | Reply |
headers
Eli Zaretskii | 10 Aug 08:36
Picon

Re: merge emacs-bidi into the main tree

> From: Richard Stallman <rms <at> gnu.org>
> Date: Sat, 09 Aug 2003 10:21:06 -0400
> 
> That may be true--but even if that is true, but might be a good thing
> to install it now.  Sure, we would like something better such as Eli
> is working on, but I don't think he will have it ready soon.
> If that will take another two years, we may as well have the emacs-bidi
> code in the mean time, rather than nothing.

As Handa-san says he will try to use what I did, I think this is not
an issue anymore.  If what I wrote has any value, I'm sure it will be
merged with Emacs.
Permalink | Reply |
headers
Gerd Moellmann | 9 Aug 17:04
Picon
Favicon

Re: merge emacs-bidi into the main tree

Richard Stallman <rms <at> gnu.org> writes:

>     IIRC, I was worried that it might be a dead end, sort of a hack, which
>     users of r2l languages wouldn't be happy with in the long term.
> 
> That may be true--but even if that is true, but might be a good thing
> to install it now.  Sure, we would like something better such as Eli
> is working on, but I don't think he will have it ready soon.
> If that will take another two years, we may as well have the emacs-bidi
> code in the mean time, rather than nothing.

That's true, no argument, if nobody steps up to finish Eli's
implementation.

I think it would be unfortunate though, and that's why I mentioned the
dead end, if users really get unhappy and redisplay is hacked to
improve performance in the right-to-left case based on a presumable
temporary solution.
Permalink | Reply |
headers
Eli Zaretskii | 8 Aug 18:02
Picon

Re: merge emacs-bidi into the main tree

> From: gerd.moellmann <at> t-online.de (Gerd Moellmann)
> Date: 08 Aug 2003 13:23:28 +0200
> 
> IIRC, I was worried that it might be a dead end, sort of a hack, which
> users of r2l languages wouldn't be happy with in the long term.  But I
> have to confess that I don't remember the details.

I still have the mail we exchanged, and can dig out the details.

IIRC, you were worried about the performance hit: the code as written
in the display engine part required to disable all the optimizations
and shortcuts that exist for the left-to-right languages, and you felt
this would hamper redisplay performance even in the simplest cases,
like cursor motion and insertion of a single character.
Permalink | Reply |
headers
Gerd Moellmann | 8 Aug 18:13
Picon
Favicon

Re: merge emacs-bidi into the main tree

"Eli Zaretskii" <eliz <at> elta.co.il> writes:

> IIRC, you were worried about the performance hit: the code as written
> in the display engine part required to disable all the optimizations
> and shortcuts that exist for the left-to-right languages, and you felt
> this would hamper redisplay performance even in the simplest cases,
> like cursor motion and insertion of a single character.

Yes, that matches the little I remember.
Permalink | Reply |
headers
Richard Stallman | 9 Aug 16:21
Picon
Picon

Re: merge emacs-bidi into the main tree

    > IIRC, you were worried about the performance hit: the code as written
    > in the display engine part required to disable all the optimizations
    > and shortcuts that exist for the left-to-right languages, 

Does it disable the optimizations always, or only when there is
right-to-left text?

								and you felt
    > this would hamper redisplay performance even in the simplest cases,
    > like cursor motion and insertion of a single character.

    Yes, that matches the little I remember.

If the optimizations are disabled only when there is right-to-left text,
and if the users of emacs-bidi think it is better than nothing, we may
as well include the feature anyway.

If the change requires disabling optimizations always, even for people
who don't use right-to-left text, that is potentially a serious
problem.  Maybe it can be solved by adding a flag that people must set
in order to use right-to-left text.  Of course, an automatic solution
would be better, but the flag would be enough to make it acceptable.
Permalink | Reply |
headers
Eli Zaretskii | 10 Aug 08:34
Picon

Re: merge emacs-bidi into the main tree

> From: Richard Stallman <rms <at> gnu.org>
> Date: Sat, 09 Aug 2003 10:21:17 -0400
> 
>     > IIRC, you were worried about the performance hit: the code as written
>     > in the display engine part required to disable all the optimizations
>     > and shortcuts that exist for the left-to-right languages, 
> 
> Does it disable the optimizations always, or only when there is
> right-to-left text?

Handa-san should give the definitive answer, but I think it was
always, as long as Emacs was told that the buffer _could_ contain
right-to-left text.  This is because Emacs doesn't know whether there
actually is right-to-left text in the buffer, and cannot do so easily
without getting a significant performance hit (what would we do?
search the buffer for certain ranges of characters after each change
to buffer's text?).

> If the change requires disabling optimizations always, even for people
> who don't use right-to-left text, that is potentially a serious
> problem.  Maybe it can be solved by adding a flag that people must set
> in order to use right-to-left text.

Such a flag indeed existed in the implementation I saw a few years
ago, when I was in Japan.  But perhaps things have changed since then.

However, it doesn't seem right to me to have an Emacs that cannot
scroll fast enough just because I've set such a flag, assuming that
Gerd's intuition is correct.  IMHO, of course.
(Continue reading)

Permalink | Reply |
headers
Richard Stallman | 11 Aug 14:53
Picon
Picon

Re: merge emacs-bidi into the main tree

    Handa-san should give the definitive answer, but I think it was
    always, as long as Emacs was told that the buffer _could_ contain
    right-to-left text.  This is because Emacs doesn't know whether there
    actually is right-to-left text in the buffer, and cannot do so easily
    without getting a significant performance hit (what would we do?
    search the buffer for certain ranges of characters after each change
    to buffer's text?).

That is acceptable, in my opinion, because it means most users
won't lose anything.

    However, it doesn't seem right to me to have an Emacs that cannot
    scroll fast enough just because I've set such a flag, assuming that
    Gerd's intuition is correct.  IMHO, of course.

It is a major, but supporting your language with slower scrolling is
better than failing to support your language at all.

    As Handa-san says he will try to use what I did, I think this is not
    an issue anymore.  If what I wrote has any value, I'm sure it will be
    merged with Emacs.

That would be very good.  Handa-san, can you tell us more about what
you plan to do with this code, and how much work you think it will be?
Permalink | Reply |
headers
Kenichi Handa | 12 Aug 03:49

Re: merge emacs-bidi into the main tree

Sorry for lazy responses on this thread.

In article <E19mCAl-0003yH-V7 <at> fencepost.gnu.org>, Richard Stallman <rms <at> gnu.org> writes:
>     Handa-san should give the definitive answer, but I think it was
>     always, as long as Emacs was told that the buffer _could_ contain
>     right-to-left text.  This is because Emacs doesn't know whether there
>     actually is right-to-left text in the buffer, and cannot do so easily
>     without getting a significant performance hit (what would we do?
>     search the buffer for certain ranges of characters after each change
>     to buffer's text?).

> That is acceptable, in my opinion, because it means most users
> won't lose anything.

>     However, it doesn't seem right to me to have an Emacs that cannot
>     scroll fast enough just because I've set such a flag, assuming that
>     Gerd's intuition is correct.  IMHO, of course.

As to optimization, what I disabled are, as far as I remeber:

  o direct_output_xxx
  o partial update of a line.  A line is always updated fully.

The other optimizatios should be still effective even if the
buffer local variable `enable-bidi-display' or
`orientation-reversed' are non-nil.

So, for instance, just scrolling should not be delayed.

One strong objection point I remember is about caching
(Continue reading)

Permalink | Reply |
headers
Richard Stallman | 13 Aug 19:13
Picon
Picon

Re: merge emacs-bidi into the main tree

      o partial update of a line.  A line is always updated fully.

It should not be to hard to make this optimization work again
when the line in question contains no rtl characters.
Permalink | Reply |
headers
Eli Zaretskii | 12 Aug 08:59
Picon

Re: merge emacs-bidi into the main tree

> Date: Tue, 12 Aug 2003 10:49:26 +0900 (JST)
> From: Kenichi Handa <handa <at> m17n.org>
> 
> In brief, what I did in emacs-bidi are:
> 
> (1) change xdisp.c to call get_next_display_element_visually
>   and set_iterator_to_next_visually instead of
>   get_next_display_element and set_iterator_to_next.
> 
> (2) make a new file bidi.c that implements
>   get_next_display_element_visually and
>   set_iterator_to_next_visually.
> 
> (3) make a new file bidi.el that implements simple
>   bidi-reordering function that is called from
>   get_next_display_element_visually to create a cache in
>   (struct it).
> 
> (4) change xterm.c to display glyphs flushing to right when
>   orientation-reversed is non-nil.
> 
> My current plan is to replace (2) and (3) with Eli's code.

It's not gonna be that simple ;-)

The code I wrote is designed to be plugged into the part of the
display engine that walks the buffer and, for each character it
encounters, generates a `struct glyph' element of the glyph matrix.
Currently, ignoring overlays, text properties, etc., this buffer scan
is linear, from left to right, and going to the next buffer position
(Continue reading)

Permalink | Reply |
headers
Kenichi Handa | 12 Aug 09:18

Re: Re: merge emacs-bidi into the main tree

In article <9743-Tue12Aug2003085951+0300-eliz <at> elta.co.il>, "Eli Zaretskii" <eliz <at> elta.co.il> writes:
>>  In brief, what I did in emacs-bidi are:
>>  
>>  (1) change xdisp.c to call get_next_display_element_visually
>>    and set_iterator_to_next_visually instead of
>>    get_next_display_element and set_iterator_to_next.
>>  
>>  (2) make a new file bidi.c that implements
>>    get_next_display_element_visually and
>>    set_iterator_to_next_visually.
>>  
>>  (3) make a new file bidi.el that implements simple
>>    bidi-reordering function that is called from
>>    get_next_display_element_visually to create a cache in
>>    (struct it).
>>  
>>  (4) change xterm.c to display glyphs flushing to right when
>>    orientation-reversed is non-nil.
>>  
>>  My current plan is to replace (2) and (3) with Eli's code.

> It's not gonna be that simple ;-)

[...]
> (To save future work, some crucial information about the characters
> over which we scan is cached inside the bidi_it struct, to facilitate
> processing later when we need to go back to those characters and
> generate the glyph matrix elements from them.  So, in the above
> example, going back from 'D' to 'A' boils down to delivering
> information already cached during the forward scan.)
(Continue reading)

Permalink | Reply |
headers
Eli Zaretskii | 12 Aug 10:48
Picon

Re: Re: merge emacs-bidi into the main tree

> Date: Tue, 12 Aug 2003 16:18:12 +0900 (JST)
> From: Kenichi Handa <handa <at> m17n.org>
> 
> Then, it seems that what you've done is not that different
> from the current emacs-bidi.  In most cases, we anyway move
> the iterator all over "abcdABCDefg".  Your code caches only
> some crucial information, so get_next_display_element will
> need some work at C B A e f g.  My code caches all
> information given by get_next_display_element, so
> get_next_display_element will work fast at C B A e f g.

Right.  IIRC, Gerd did not like the caching of glyph matrix elements.
I think he felt that producing the information stored in the glyph is
expensive, and so doing that unnecessarily would hamper performance.
The information my code caches is only relevant to the bidi properties
of the characters, which need to be computed anyway to decide what is
the next character in the visual order.
Permalink | Reply |
headers
Gerd Moellmann | 10 Aug 12:42
Picon
Favicon

Re: merge emacs-bidi into the main tree

Eli Zaretskii <eliz <at> elta.co.il> writes:

> However, it doesn't seem right to me to have an Emacs that cannot
> scroll fast enough just because I've set such a flag, assuming that
> Gerd's intuition is correct.

One way to see how working with disabled redisplay optimizations feels
might be to intentionally disable the optimizations in the source
code.

(Which is, BTW, where my intuition comes from, only that I didn't
disable any optimizations, but implement one after the other until the
result was "fast enough", for me anyway.  And, another BTW, redisplay
optimization was _the_ major time sink and _the_ primary source of
complexity, when rewriting redisplay, esp. under the ancillary
condition of being compatible with Emacs 19's redisplay.  I'm
mentioning that only in case that r2l redisplay optimization will
prove necessary.)

In any case, I think it might be worth the experiment.  Computers got
noticeable faster, maybe the picture is different now than it was in
the late '90s.
Permalink | Reply |
headers
Eli Zaretskii | 10 Aug 18:56
Picon

Re: merge emacs-bidi into the main tree

> From: gerd.moellmann <at> t-online.de (Gerd Moellmann)
> Date: 10 Aug 2003 12:42:37 +0200
> 
> (Which is, BTW, where my intuition comes from, only that I didn't
> disable any optimizations, but implement one after the other until the
> result was "fast enough", for me anyway.

Just so we have a reference: what kind of a machine did you use at
the time you did that?

> In any case, I think it might be worth the experiment.  Computers got
> noticeable faster, maybe the picture is different now than it was in
> the late '90s.

True.

Btw--and this is just for completeness' sake--one of the design goals
of what I've written was to allow the reordering code to be called
from a variety of places, like if we need to encode text in visual
order so it could be sent to a printer.
Permalink | Reply |
headers
Gerd Moellmann | 10 Aug 18:33
Picon
Favicon

Re: merge emacs-bidi into the main tree

"Eli Zaretskii" <eliz <at> elta.co.il> writes:

> Just so we have a reference: what kind of a machine did you use at
> the time you did that?

I think I mainly used a P2 400 MHz.
Permalink | Reply |
headers
Stefan Monnier | 10 Aug 18:45
Picon
Favicon

Re: merge emacs-bidi into the main tree

> > Just so we have a reference: what kind of a machine did you use at
> > the time you did that?
> 
> I think I mainly used a P2 400 MHz.

I.e. something faster than what I (and my girlfriend, and my officemate)
use every day.

	Stefan
Permalink | Reply |
headers
Gerd Moellmann | 9 Aug 16:58
Picon
Favicon

Re: merge emacs-bidi into the main tree

Richard Stallman <rms <at> gnu.org> writes:

>     > IIRC, you were worried about the performance hit: the code as written
>     > in the display engine part required to disable all the optimizations
>     > and shortcuts that exist for the left-to-right languages, 
> 
> Does it disable the optimizations always, or only when there is
> right-to-left text?

AFAIR, most optimizations were disabled in buffers containing
right-to-left text, so that this wouldn't be a problem for users who
don't use right-to-left text in the first place.
Permalink | Reply |
headers
Oliver Scholz | 8 Aug 11:05
X-Face
Picon
Picon

Re: merge emacs-bidi into the main tree

Eli Zaretskii <eliz <at> elta.co.il> writes:

>> From: Richard Stallman <rms <at> gnu.org>
>> Date: Thu, 07 Aug 2003 02:05:07 -0400
>> 
>> What do Hebrew and Arab users think of it?
>> Eli Z, what is your opinion?
>
> I feel uneasy about using something about which Gerd was most unhappy
> at the time this was discussed.
>
> I'd prefer to call for volunteers to continue the work I started,
> namely, change the display engine to support bidirectional editing
> without processing batches of buffer's text.  (I can elaborate if
> there's enough interest.)  I have some unpolished stand-alone code
> that does bidi reordering sequentially, which should fit very well
> into the Emacs display engine, but someone needs to take that code and
> integrate it into Emacs, since I probably won't have time any time
> soon enough for this to be practical.
>
> If anyone steps forward willing to begin hacking the display code, I
> promise to help in any way I can.

Well, I am probably not yet skilled enough to be of any real help. And
I still have making fontsets customizable as the `car' of my
TODO-list, while the university does not leave much time for me.

But since I have been fascinated by m18n issues, since I started to
use Emacs. And since I'd like to become proficient in display hacking
some time in the future, I would appreciate it, if you could add me to
(Continue reading)

Permalink | Reply |
headers
Eli Zaretskii | 8 Aug 17:57
Picon

Re: merge emacs-bidi into the main tree

> From: Oliver Scholz <epameinondas <at> gmx.de>
> Date: Fri, 08 Aug 2003 11:05:26 +0200
> 
> But since I have been fascinated by m18n issues, since I started to
> use Emacs. And since I'd like to become proficient in display hacking
> some time in the future, I would appreciate it, if you could add me to
> the emacs-bidi mailing list.

You don't need me for that: emacs-bidi is a public list, so you can
add yourself via this page:

  http:/savannah.gnu.org/projects/emacs

(follow the link to mailing lists).

And thanks for volunteering.
Permalink | Reply |
headers
Kenichi Handa | 8 Aug 13:47

Re: merge emacs-bidi into the main tree

In article <ullu4msdl.fsf <at> ID-87814.user.dfncis.de>, Oliver Scholz <epameinondas <at> gmx.de> writes:
> Well, I am probably not yet skilled enough to be of any real help. And
> I still have making fontsets customizable as the `car' of my
> TODO-list, while the university does not leave much time for me.

> But since I have been fascinated by m18n issues, since I started to
> use Emacs. And since I'd like to become proficient in display hacking
> some time in the future, I would appreciate it, if you could add me to
> the emacs-bidi mailing list. Maybe I could do some minor tasks, at
> least.

I think you can subscribe to emacs-bidi mailing list from
this page:
	<http://mail.gnu.org/mailman/listinfo/emacs-bidi>

---
Ken'ichi HANDA
handa <at> m17n.org
Permalink | Reply |
headers
Alex Schroeder | 4 Aug 16:34
Face

Re: merge emacs-bidi into the main tree

Eli Zaretskii <eliz <at> elta.co.il> writes:

> Perhaps for the same reason you now think the m17n.org code is a
> viable solution--because lots of time have passed, and a bidi-capable
> Emacs is now more important than before.  Meanwile, some of the
> developers have more experience in hacking redisplay code, so their
> response might be different this time, especially if the plea sounds
> convincing.

Well, we can always try.  I will post a mail on emacs-devel and ask
for volunteers to do it, but I will also suggest the inclusion of
emacs-bidi into current CVS if no volunteer can be found.

The reason I have not much hope is that whenever tricky redisplay
problems used to come up a few month ago, RMS used to CC Gerd and ask
his opinion.  Perhaps Jan D and Kim Storm have had a lot more exposure
to the redisplay code in the meantime.  Having them onboard would
certainly be a good thing.

Alex.
--

-- 
http://www.emacswiki.org/alex/
I was on holidays from 2003-07-01 to 2003-07-29
and have a lot of catching up to do.
Permalink | Reply |
headers
Eli Zaretskii | 4 Aug 18:22
Picon

Re: merge emacs-bidi into the main tree

> From: Alex Schroeder <alex <at> emacswiki.org>
> Date: Mon, 04 Aug 2003 16:34:24 +0200
> 
> I will post a mail on emacs-devel and ask
> for volunteers to do it, but I will also suggest the inclusion of
> emacs-bidi into current CVS if no volunteer can be found.

Please do.

> Perhaps Jan D and Kim Storm have had a lot more exposure to the
> redisplay code in the meantime.

Exactly.
Permalink | Reply |

Gmane