headers
TAKAHASHI Naoto | 5 Mar 06:26

Re: emacs-bidi - funtional/ready, now what ?

Sorry for my obscure statement in the last email.

Nadim Shaikli writes:

>> The current version never goes into the main tree.  Actually, it uses
>> algorithms that have already been rejected to be included.

> Any further details ?  Are there competing algorithms out there
> that are deemed "acceptable" ?

Eli Zaretskii, one of the most important Emacs developers, is writing
such a code.  I heard he has done to some extent and his code is
expected to be included in the main trunk once finished.

>> Advertising the current version is greatly welcome.

> I'm a bit confused.  If the algorithm has been rejected and passed
> over what's the plan then ?  Advertising the current code will
> certainly get testers (or we all hope so), but then what ?

First of all, we need to learn what kinds of functionalities people
want to have and what kind of user interface is preferable.  On the
other hand, the algorithm (namely, how they are implemented) does not
concern ordinary users; only Emacs developers care about it.

>> A "pollished" version will not do; a drastic change is required.

> Could you please elaborate on the "change".

Developers feel that the current emacs-bidi is not efficient enough.
(Continue reading)

Permalink | Reply |
headers
Eli Zaretskii | 5 Mar 18:47
Picon

Re: Re: emacs-bidi - funtional/ready, now what ?

> Date: Wed, 5 Mar 2003 14:26:43 +0900 (JST)
> From: TAKAHASHI Naoto <ntakahas <at> m17n.org>
> 
> > Any further details ?  Are there competing algorithms out there
> > that are deemed "acceptable" ?
> 
> Eli Zaretskii, one of the most important Emacs developers, is writing
> such a code.  I heard he has done to some extent and his code is
> expected to be included in the main trunk once finished.

Unfortunately, a series of personal disasters (well, that's an
exaggeration, actually) and daytime job committments have all but
stopped my work on this during the last year.

> Developers feel that the current emacs-bidi is not efficient enough.

Right.

> We need to find an efficient way to integrate a bidi reordering
> routine into Emacs' display engine.  I do not know well about Emacs'
> display routine so I cannot explain more.  Sorry.

Some explanation of the proposed better solution can be found in the
discussion whose pointer was posted here by Handa-san, and in the
archives of the emacs-bidi list.  Feel free to ask questions if
something there is unclear.

> > Has RMS or the emacs developers given a clear statement on what it
> > is that they'd like to see or not (URL) ?  Is there a means for us
> > to elevate Bidi's inclusion on their list of things to do ?
(Continue reading)

Permalink | Reply |
headers
Alex Schroeder | 5 Mar 10:45
Face

Re: Re: emacs-bidi - funtional/ready, now what ?

TAKAHASHI Naoto <ntakahas <at> m17n.org> writes:

> I do not think any URL is available for now.  If someone writes a
> summary, I can put it one the emacs-bidi web page.

At the time the problem was this:  The display engine moves through
the buffer character by character and decides how to display it.  This
is the current architecture, and the display engine is a very delicate
piece of work, so this is not going to change unless somebody rewrites
alot of code.

Therefore, we cannot use existing bidi algorithms, because these
expect a string instead of a stream of characters.  Using a string,
one could implement the Unicode or a similar reordering algorithm
which returned a "visual order" string to display.  But this
straightforward route is not acceptable if we want to integrate bidi
at the display engine level.

We need a piece of code that hooks into the display engine and that
can return the next character to display, given a position in a
buffer.  (The important part here is that this will probably not be
the character at that position when bidi reordering is in effect.)
This will require looking ahead, looking behind, applying algorithms,
and caching the result, so that when the display engine moves to the
next position in the buffer and wants to know what character to
display, less work is required.

Anyway, that's the basic problem description.

It seems that Eli has a stand-alone solution of this piece of code and
(Continue reading)

Permalink | Reply |
headers
Kamal Dalal | 7 Mar 20:15

Re: Re: emacs-bidi - funtional/ready, now what ?

On Wed, Mar 05, 2003 at 10:45:46AM +0100, Alex Schroeder wrote:

> 
> At the time the problem was this:  The display engine moves through
> the buffer character by character and decides how to display it.  This
> is the current architecture, and the display engine is a very delicate
> piece of work, so this is not going to change unless somebody rewrites
> alot of code.
> 
> Therefore, we cannot use existing bidi algorithms, because these
> expect a string instead of a stream of characters.  Using a string,
> one could implement the Unicode or a similar reordering algorithm
> which returned a "visual order" string to display.  But this
> straightforward route is not acceptable if we want to integrate bidi
> at the display engine level.
> 
> We need a piece of code that hooks into the display engine and that
> can return the next character to display, given a position in a
> buffer.  (The important part here is that this will probably not be
> the character at that position when bidi reordering is in effect.)
> This will require looking ahead, looking behind, applying algorithms,
> and caching the result, so that when the display engine moves to the
> next position in the buffer and wants to know what character to
> display, less work is required.
> 
> Anyway, that's the basic problem description.
> 
> It seems that Eli has a stand-alone solution of this piece of code and
> needs to integrate it into the display engine.  That is hairy work.
> If you are interested, ask Eli -- last time he was very willing to
(Continue reading)

Permalink | Reply |
headers
Eli Zaretskii | 9 Mar 17:09
Picon

Re: Re: emacs-bidi - funtional/ready, now what ?

> Date: Fri, 7 Mar 2003 11:15:09 -0800
> From: Kamal Dalal <kamal+arabeyes <at> banoora.net>
> 
> An incremental bidi algorithm is very interesting. I went through the
> archives and could not find more information that shed more light on
> such an algorithm. Did I miss it ?

I described several aspects of that implementation several times, but
I don't know whether my explanations ``shed more light'' on it ;-)

> Eli, perhaps you will be willing to share this algorithm.

As I said several times here in the past, I'm uneasy about releasing
unfinished code.  I did say that I will gladly pass whatever I have to
individuals who will demonstrate that they are determined to work on
bidi Emacs.

> I am researching possible solutions for a bidi aware terminal for
> Windows (GPL) and such an algorithm may be very useful.

Really?  I'd be surprised: terminals display text line by line, so a
batch-style reordering such as in FriBidi would be more than adequate,
I think.  I think I've read somewhere that there are a couple of
bidi-aware terminal emulators out there which use FriBidi.

Also, Windows inherently supports bidi, so it sounds like you don't
need anything to have what you want.

Can you tell why do you think a sequential algorithm would be useful
for a terminal?
(Continue reading)

Permalink | Reply |
headers
Kamal Dalal | 9 Mar 19:54

Re: Re: emacs-bidi - funtional/ready, now what ?

On Sun, Mar 09, 2003 at 06:09:07PM +0200, Eli Zaretskii wrote:
> 
> I described several aspects of that implementation several times, but
> I don't know whether my explanations ``shed more light'' on it ;-)

Nope! Still dark ;)

> As I said several times here in the past, I'm uneasy about releasing
> unfinished code.  I did say that I will gladly pass whatever I have to
> individuals who will demonstrate that they are determined to work on
> bidi Emacs.

I understand. Hope that person arrives really soon.

> Also, Windows inherently supports bidi, so it sounds like you don't
> need anything to have what you want.

Correct, but I'd rather not use Windows services to do bidi.

> Can you tell why do you think a sequential algorithm would be useful
> for a terminal?

A terminal receives characters on the wire as they arrive. I was
wondering whether an incremental approach to bidi would allow one to
apply it on the characters on the wire. 

I know, an incremental bidi may turn out to be completely useless to
terminals, but I am still interested ;)

# Kamal
(Continue reading)

Permalink | Reply |
headers
TAKAHASHI Naoto | 6 Mar 03:55

Re: Re: emacs-bidi - funtional/ready, now what ?

Thank you for your explanation, Eli.

"Eli Zaretskii" writes:

>> > Has RMS or the emacs developers given a clear statement on what it
>> > is that they'd like to see or not (URL) ?  Is there a means for us
>> > to elevate Bidi's inclusion on their list of things to do ?
>> 
>> I do not think any URL is available for now.  If someone writes a
>> summary, I can put it one the emacs-bidi web page.

> I wrote such summaries a few times in the past, see the archives.

Oh, yes, of course ML archives are available.  I was thinking of a web
page specially designed to explain "why is Emacs bidi-illiterate?"

> P.S.  Btw, I didn't see the original messages from Nadim Shaikli, and
> neither did I see the message from Alex to which you replied in
> another mail.  These messages are also not in the emacs-bidi archives.
> Does someone know what happened with them?  Did they bounce off the
> list or something?

Strange.  Does the emacs-bidi list reject posts from non-members?  Are
Nadim and Alex on the list?

--

-- 
TAKAHASHI Naoto
ntakahas <at> m17n.org
http://www.m17n.org/ntakahas/
(Continue reading)

Permalink | Reply |
headers
Eli Zaretskii | 6 Mar 05:29
Picon

Re: Re: emacs-bidi - funtional/ready, now what ?

> Date: Thu, 6 Mar 2003 11:55:50 +0900 (JST)
> From: TAKAHASHI Naoto <ntakahas <at> m17n.org>
> 
> Strange.  Does the emacs-bidi list reject posts from non-members?

Yes, I think it does.
Permalink | Reply |
headers
TAKAHASHI Naoto | 5 Mar 13:50

Re: Re: emacs-bidi - funtional/ready, now what ?

Thank you for the explanation, Alex.  I made a link from the
emacs-bidi web page to the DisplayEngineForBidi section in the Emacs
Wiki site.

--

-- 
TAKAHASHI Naoto
ntakahas <at> m17n.org
http://www.m17n.org/ntakahas/
Permalink | Reply |
headers
Kenichi Handa | 5 Mar 08:15

Re: Re: emacs-bidi - funtional/ready, now what ?

In article <200303050526.h255Qhv03588 <at> mule.m17n.org>, TAKAHASHI Naoto <ntakahas <at> m17n.org> writes:
>>  Has RMS or the emacs developers given a clear statement on what it
>>  is that they'd like to see or not (URL) ?  Is there a means for us
>>  to elevate Bidi's inclusion on their list of things to do ?

Please visit this page:
    http://mail.gnu.org/archive/html/emacs-devel/
and search string "bidi".  Then, you'll find, for instance,
this page:
    http://mail.gnu.org/archive/html/emacs-devel/2001-07/msg00042.html

You'll see the flow of discussions by reading previous or
next mails on the same thread.

> Don't worry.  The core developers are aware of the necessity of bidi.
> All we need is a programmer who
>   1. knows Emacs' display engine, and
>   2. is familiar with bidi concepts, and
>   3. has time to write a code.

> Yes, there are a few men who satisfy 1. and 2. above, but they are all
> busy.  (You know how such people are rare and precious.)  If you
> really want bidi support right now, you need to hire them.  Otherwise,
> you need to wait for their voluntary work or do it yourself.

If I can get a budget of, at least, 40,000 USD, I can
persuade our boss that I spend my time on it as a side job.
:-)

---
(Continue reading)

Permalink | Reply |
headers
Eli Zaretskii | 5 Mar 18:42
Picon

Re: Re: emacs-bidi - funtional/ready, now what ?

> Date: Wed, 5 Mar 2003 16:15:06 +0900 (JST)
> From: Kenichi Handa <handa <at> m17n.org>
> 
> Please visit this page:
>     http://mail.gnu.org/archive/html/emacs-devel/
> and search string "bidi".  Then, you'll find, for instance,
> this page:
>     http://mail.gnu.org/archive/html/emacs-devel/2001-07/msg00042.html
> 
> You'll see the flow of discussions by reading previous or
> next mails on the same thread.

Thank you for this pointer, Handa-san.  Yes, this discussion is a
useful starting point to learn about the issues.

Additional discussions were held on the emacs-bidi mailing list, so
its archives (circa 2001) are also a useful reading.

> If I can get a budget of, at least, 40,000 USD, I can
> persuade our boss that I spend my time on it as a side job.
> :-)

Same here.
Permalink | Reply |

Gmane