Brian D. Burns | 7 Jul 2012 02:28
Gravatar

if_xcmdsrv.c patches

Recently, I have been looking into different methods for using Vim
to syntax highlight small blocks of code. In the process, I found
runtime/tools/xcmdsrv_client.c, which worked but seemed a little
outdated. So, after working with this a bit, I decided to create a
Ruby extension from src/if_xcmdsrv.c.
https://github.com/burns/vim_client-ruby
This is the most I've ever done in C, so it was quite the learning
experience. So, comments/suggestions are certainly welcome :)

While working with this, I found a few things I wanted to submit
for changes in if_xcmdsrv.c. I've attached 3 patches, which include
my comments.

if_xcmdsrv.patch.1 was a duplicate call to ga_init2.

if_xcmdsrv.patch.2 deals with the fact that the server is actually
sending a response to clients making input key requests (--remote-send),
even though they don't expect one, and some other thoughts.

if_xcmdsrv.patch.3 suggests some changes to the ServerWait() function,
which speeds up response time, handles an issue where the XEvent queue
is not being checked when 'localLoop' is true, and a few other thoughts.

I appreciate your time - and Vim! :)

- Brian

--

-- 
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
(Continue reading)

Bram Moolenaar | 7 Jul 2012 15:57
Picon

Re: if_xcmdsrv.c patches


Brian Burns wrote:

> Recently, I have been looking into different methods for using Vim
> to syntax highlight small blocks of code. In the process, I found
> runtime/tools/xcmdsrv_client.c, which worked but seemed a little
> outdated. So, after working with this a bit, I decided to create a
> Ruby extension from src/if_xcmdsrv.c.
> https://github.com/burns/vim_client-ruby
> This is the most I've ever done in C, so it was quite the learning
> experience. So, comments/suggestions are certainly welcome :)
> 
> While working with this, I found a few things I wanted to submit
> for changes in if_xcmdsrv.c. I've attached 3 patches, which include
> my comments.
> 
> if_xcmdsrv.patch.1 was a duplicate call to ga_init2.
> 
> if_xcmdsrv.patch.2 deals with the fact that the server is actually
> sending a response to clients making input key requests (--remote-send),
> even though they don't expect one, and some other thoughts.
> 
> if_xcmdsrv.patch.3 suggests some changes to the ServerWait() function,
> which speeds up response time, handles an issue where the XEvent queue
> is not being checked when 'localLoop' is true, and a few other thoughts.
> 
> I appreciate your time - and Vim! :)

It looks like you know what you are doing.  I appreciate taking the time
to improve this code, it hasn't been worked on for a while.  I'll look
(Continue reading)

Brian D. Burns | 9 Jul 2012 01:49
Gravatar

Re: if_xcmdsrv.c patches

On 07/07/2012 09:57 AM, Bram Moolenaar wrote:
> 
> Brian Burns wrote:
> 
>> Recently, I have been looking into different methods for using Vim
>> to syntax highlight small blocks of code. In the process, I found
>> runtime/tools/xcmdsrv_client.c, which worked but seemed a little
>> outdated. So, after working with this a bit, I decided to create a
>> Ruby extension from src/if_xcmdsrv.c.
>> https://github.com/burns/vim_client-ruby
>> This is the most I've ever done in C, so it was quite the learning
>> experience. So, comments/suggestions are certainly welcome :)
>>
>> While working with this, I found a few things I wanted to submit
>> for changes in if_xcmdsrv.c. I've attached 3 patches, which include
>> my comments.
>>
>> if_xcmdsrv.patch.1 was a duplicate call to ga_init2.
>>
>> if_xcmdsrv.patch.2 deals with the fact that the server is actually
>> sending a response to clients making input key requests (--remote-send),
>> even though they don't expect one, and some other thoughts.
>>
>> if_xcmdsrv.patch.3 suggests some changes to the ServerWait() function,
>> which speeds up response time, handles an issue where the XEvent queue
>> is not being checked when 'localLoop' is true, and a few other thoughts.
>>
>> I appreciate your time - and Vim! :)
> 
> It looks like you know what you are doing.  I appreciate taking the time
(Continue reading)

Brian D. Burns | 9 Jul 2012 01:57
Gravatar

Re: if_xcmdsrv.c patches

On 07/08/2012 07:49 PM, Brian D. Burns wrote:
> On 07/07/2012 09:57 AM, Bram Moolenaar wrote:
>>
>> Brian Burns wrote:
>>
>>> Recently, I have been looking into different methods for using Vim
>>> to syntax highlight small blocks of code. In the process, I found
>>> runtime/tools/xcmdsrv_client.c, which worked but seemed a little
>>> outdated. So, after working with this a bit, I decided to create a
>>> Ruby extension from src/if_xcmdsrv.c.
>>> https://github.com/burns/vim_client-ruby
>>> This is the most I've ever done in C, so it was quite the learning
>>> experience. So, comments/suggestions are certainly welcome :)
>>>
>>> While working with this, I found a few things I wanted to submit
>>> for changes in if_xcmdsrv.c. I've attached 3 patches, which include
>>> my comments.
>>>
>>> if_xcmdsrv.patch.1 was a duplicate call to ga_init2.
>>>
>>> if_xcmdsrv.patch.2 deals with the fact that the server is actually
>>> sending a response to clients making input key requests (--remote-send),
>>> even though they don't expect one, and some other thoughts.
>>>
>>> if_xcmdsrv.patch.3 suggests some changes to the ServerWait() function,
>>> which speeds up response time, handles an issue where the XEvent queue
>>> is not being checked when 'localLoop' is true, and a few other thoughts.
>>>
>>> I appreciate your time - and Vim! :)
>>
(Continue reading)

Bram Moolenaar | 9 Jul 2012 20:48
Picon

Re: if_xcmdsrv.c patches


Brain D. Burns wrote:

> On 07/08/2012 07:49 PM, Brian D. Burns wrote:
> > On 07/07/2012 09:57 AM, Bram Moolenaar wrote:
> >>
> >> Brian Burns wrote:
> >>
> >>> Recently, I have been looking into different methods for using Vim
> >>> to syntax highlight small blocks of code. In the process, I found
> >>> runtime/tools/xcmdsrv_client.c, which worked but seemed a little
> >>> outdated. So, after working with this a bit, I decided to create a
> >>> Ruby extension from src/if_xcmdsrv.c.
> >>> https://github.com/burns/vim_client-ruby
> >>> This is the most I've ever done in C, so it was quite the learning
> >>> experience. So, comments/suggestions are certainly welcome :)
> >>>
> >>> While working with this, I found a few things I wanted to submit
> >>> for changes in if_xcmdsrv.c. I've attached 3 patches, which include
> >>> my comments.
> >>>
> >>> if_xcmdsrv.patch.1 was a duplicate call to ga_init2.
> >>>
> >>> if_xcmdsrv.patch.2 deals with the fact that the server is actually
> >>> sending a response to clients making input key requests (--remote-send),
> >>> even though they don't expect one, and some other thoughts.
> >>>
> >>> if_xcmdsrv.patch.3 suggests some changes to the ServerWait() function,
> >>> which speeds up response time, handles an issue where the XEvent queue
> >>> is not being checked when 'localLoop' is true, and a few other thoughts.
(Continue reading)

Bram Moolenaar | 10 Jul 2012 12:31
Picon

Re: if_xcmdsrv.c patches


Brian Burns wrote:

> > While you are digging into this, could you find some way to test this
> > code?  This might be complicated, since it requires starting two Vim
> > instances talking to each other, but it would be very useful to have at
> > least basic testing for this code.
> > 
> > 
> 
> Hey,
> 
> I've attached some basic tests.
> I'm not sure if what I'm doing here would work on all systems,
> but it's a start. Let me know what you think.

Thanks.  It's nice to have a test for this, there was nothing before.

It is rather Unix-specific, but that's hard to avoid.  We can only run
this test from the Unix Makefile so that we don't even try on other
systems.

Even the Unix version may not have the clientserver feature.  We can
check that at the start:

STARTTEST
:so small.vim
:" drop out when the clientserver features is not supported
:if !has("clientserver")
: e! test.ok
(Continue reading)

Brian D. Burns | 13 Jul 2012 15:09
Gravatar

Re: if_xcmdsrv.c patches

On 07/10/2012 06:31 AM, Bram Moolenaar wrote:
> 
> Brian Burns wrote:
> 
>> I've attached some basic tests.
>> I'm not sure if what I'm doing here would work on all systems,
>> but it's a start. Let me know what you think.
> 
> Thanks.  It's nice to have a test for this, there was nothing before.
> 
> It is rather Unix-specific, but that's hard to avoid.  We can only run
> this test from the Unix Makefile so that we don't even try on other
> systems.
> 
> Even the Unix version may not have the clientserver feature.  We can
> check that at the start:
> 
> STARTTEST
> :so small.vim
> :" drop out when the clientserver features is not supported
> :if !has("clientserver")
> : e! test.ok
> : w! test.out
> : qa!
> :endif
> 
> If the test fails halfway then the test server keeps running.  I think
> it's better to kill it from the Makefile.
> 
> You can use v:progname for the Vim program name.  It doesn't include the
(Continue reading)

Bram Moolenaar | 14 Jul 2012 12:21
Picon

Re: if_xcmdsrv.c patches


Brain Burns wrote:

> On 07/10/2012 06:31 AM, Bram Moolenaar wrote:
> > 
> > Brian Burns wrote:
> > 
> >> I've attached some basic tests.
> >> I'm not sure if what I'm doing here would work on all systems,
> >> but it's a start. Let me know what you think.
> > 
> > Thanks.  It's nice to have a test for this, there was nothing before.
> > 
> > It is rather Unix-specific, but that's hard to avoid.  We can only run
> > this test from the Unix Makefile so that we don't even try on other
> > systems.
> > 
> > Even the Unix version may not have the clientserver feature.  We can
> > check that at the start:
> > 
> > STARTTEST
> > :so small.vim
> > :" drop out when the clientserver features is not supported
> > :if !has("clientserver")
> > : e! test.ok
> > : w! test.out
> > : qa!
> > :endif
> > 
> > If the test fails halfway then the test server keeps running.  I think
(Continue reading)

Brian D. Burns | 17 Jul 2012 12:46
Gravatar

Re: if_xcmdsrv.c patches

On 07/14/2012 06:21 AM, Bram Moolenaar wrote:
>
>> Brian Burns wrote:
>>
>>
>> I've attached a patch which includes an updated test and Makefile, with
>> the server being started and stopped in the Makefile. Also included is a
>> change to serverPeekReply() and an added note for ServerWait().
> 
> [...]
> 
> Thanks!  I'll look into it later.
> 

I've attached a new patch, with an updated test and Makefile which
includes tests using --remote-send and --remote-expr.

Also, I've spent some time with gdb looking into this "missing event"
issue and I've determined what the root cause is.

When ServerWait is called with localLoop FALSE, ui_delay is used and
ultimately calls RealWaitForChar. This function is using select/poll to
determine if there is an event waiting in order to call xterm_update.
However, this is not an accurate test, since the X server reads that fd.
In this case, it's doing so before RealWaitForChar has a chance to poll
the fd. So, even if select/poll returns 0 or FD_ISSET() returns false,
events may already be in the X queue, and a call to XtAppPending would
find it. In light of this, I was thinking that RealWaitForChar needs to
calls xterm_update regardless. Something like:

(Continue reading)

James McCoy | 17 Jul 2012 15:30
Gravatar

Re: if_xcmdsrv.c patches


On Jul 17, 2012 6:47 AM, "Brian D. Burns" <burns <at> iosctr.com> wrote:
>
> On 07/14/2012 06:21 AM, Bram Moolenaar wrote:
> >
> >> Brian Burns wrote:
> >>
> >>
> >> I've attached a patch which includes an updated test and Makefile, with
> >> the server being started and stopped in the Makefile. Also included is a
> >> change to serverPeekReply() and an added note for ServerWait().
> >
> > [...]
> >
> > Thanks!  I'll look into it later.
> >
>
> I've attached a new patch, with an updated test and Makefile which
> includes tests using --remote-send and --remote-expr.
>
> Also, I've spent some time with gdb looking into this "missing event"
> issue and I've determined what the root cause is.

This looks very similar to an issue xscreensaver used to have.  Maybe the solution[0] I found for that can provide some inspiration.

[0]: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=486603#45

Cheers,
James

--
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Brian D. Burns | 18 Jul 2012 18:09
Gravatar

Re: if_xcmdsrv.c patches

> On Tuesday, July 17, 2012 9:30:42 AM UTC-4, James McCoy wrote:
>
> > On Jul 17, 2012 6:47 AM, "Brian D. Burns" <burns <at> iosctr.com> wrote:
> >
> > I've attached a new patch, with an updated test and Makefile which
> > includes tests using --remote-send and --remote-expr.
> >
> > Also, I've spent some time with gdb looking into this "missing event"
> > issue and I've determined what the root cause is.
> 
> This looks very similar to an issue xscreensaver used to have.  Maybe the solution[0] I found for that can
provide some inspiration.
> 
> [0]: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=486603#45
> 
> Cheers,
> James

Thanks James, it does appear similar.

Given that the current code expects FD_ISSET to be true if events are
waiting, the following should be equivalent to the commented portion:

if (ret >= 0 && xterm_Shell != (Widget)0)
{
    if (XtAppPending(app_context))
    {
        xterm_update();
        if (((ret == 1 && FD_ISSET(ConnectionNumber(xterm_dpy), &rfds))
            || ret == 0) && !input_available())
        {
            finished = FALSE;
        }
    }
}

/*
if (ret > 0 && xterm_Shell != (Widget)0
        && FD_ISSET(ConnectionNumber(xterm_dpy), &rfds))
{
    xterm_update();
    if (--ret == 0 && !input_available())
    {
        finished = FALSE;
    }
}
*/

However, when I use this, and remove the other "fixes":
- XCheckWindowEvent in ServerWait (when localLoop is FALSE)
- XCheckWindowEvent in serverPeekReply
(i.e. letting the call to xterm_update/XtAppPending trigger serverEventProc)
the test passes, but you have to tap ESC or something to get them to continue running.
The same is true for several of the other tests.

These are two new backtraces for calls to RealWaitForChar I see when applying these changes:

#0  RealWaitForChar (fd=0, msec=4000, check_for_gpm=0x0) at os_unix.c:4917
#1  0x0813f33b in WaitForChar (msec=4000) at os_unix.c:4870
#2  0x0813b5a7 in mch_inchar (buf=0x8245be7 "", maxlen=85, wtime=-1, tb_change_cnt=8) at os_unix.c:406
#3  0x081a57ce in ui_inchar (buf=0x8245be7 "", maxlen=85, wtime=-1, tb_change_cnt=8) at ui.c:193
#4  0x080d4914 in inchar (buf=0x8245be7 "", maxlen=257, wait_time=-1, tb_change_cnt=8) at getchar.c:3025
#5  0x080d45af in vgetorpeek (advance=1) at getchar.c:2800
#6  0x080d2bba in vgetc () at getchar.c:1582
#7  0x080d30d4 in safe_vgetc () at getchar.c:1787
#8  0x08115739 in normal_cmd (oap=0xbfffe968, toplevel=1) at normal.c:665
#9  0x081d5b3a in main_loop (cmdwin=0, noexmode=0) at main.c:1294
#10 0x081d5637 in main (argc=5, argv=0xbfffeb84) at main.c:998

#0  RealWaitForChar (fd=0, msec=-1, check_for_gpm=0x0) at os_unix.c:5244
#1  0x0813f33b in WaitForChar (msec=-1) at os_unix.c:4870
#2  0x0813b614 in mch_inchar (buf=0x8245be6 "", maxlen=86, wtime=-1, tb_change_cnt=7) at os_unix.c:436
#3  0x081a57ce in ui_inchar (buf=0x8245be6 "", maxlen=86, wtime=-1, tb_change_cnt=7) at ui.c:193
#4  0x080d4914 in inchar (buf=0x8245be6 "", maxlen=258, wait_time=-1, tb_change_cnt=7) at getchar.c:3025
#5  0x080d45af in vgetorpeek (advance=1) at getchar.c:2800
#6  0x080d2bba in vgetc () at getchar.c:1582
#7  0x080d30d4 in safe_vgetc () at getchar.c:1787
#8  0x08115739 in normal_cmd (oap=0xbfffe968, toplevel=1) at normal.c:665
#9  0x081d5b3a in main_loop (cmdwin=0, noexmode=0) at main.c:1294
#10 0x081d5637 in main (argc=5, argv=0xbfffeb84) at main.c:998

I'm guessing the extra events being dispatched must be causing this.
Stepping through the test doesn't work too well, but when I use the
commented (i.e. current) code above, along with the "fixes" in
ServerWait and serverPeekReply, I never see RealWaitForChar called with
anything other than msec=0.
The calls look more like this:

#0  RealWaitForChar (fd=0, msec=0, check_for_gpm=0x0) at os_unix.c:5254
#1  0x0813f2af in mch_breakcheck () at os_unix.c:4811
#2  0x081a593d in ui_breakcheck () at ui.c:367
#3  0x080d330d in vgetorpeek (advance=1) at getchar.c:2014
#4  0x080d2bba in vgetc () at getchar.c:1582
#5  0x080d30d4 in safe_vgetc () at getchar.c:1787
#6  0x080b5162 in getcmdline (firstc=58, count=1, indent=0) at ex_getln.c:360
#7  0x080b7fb2 in getexline (c=58, cookie=0x0, indent=0) at ex_getln.c:2155
#8  0x080a239d in do_cmdline (cmdline=0x0, fgetline=0x80b7f7a <getexline>, cookie=0x0, flags=0) at ex_docmd.c:1018
#9  0x0811ccb2 in nv_colon (cap=0xbfffe8c0) at normal.c:5404
#10 0x0811620b in normal_cmd (oap=0xbfffe968, toplevel=1) at normal.c:1193
#11 0x081d5b22 in main_loop (cmdwin=0, noexmode=0) at main.c:1294
#12 0x081d561f in main (argc=9, argv=0xbfffeb84) at main.c:998

--

-- 
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php


Gmane