Development discussion of the internals of the Cygwin DLL

Corinna Vinschen | 16 Feb 2012 13:46

Cygwin's spawn/exec, mintty and the "Program Compatibility Assistant"

Hi guys, and especially
Hi Andy,

A couple of weeks ago we noticed this strange problem with mintty.  The
"Program Compatibility Assistant" (PCA) on W7 took it under its wings
for no apparent reason, and if the session was long enough you would
notice that a specific svchost process, the one running the Pcasvc
service, would take more and more memory and CPU time.

The solution for this problem was either to ask the user to switch off
the PCA service, or a change in Cygwin's spawn/exec code.  in short, PCA
sets up a Job for a controlled application, but allows to breakaway from
the job.  That's what Cygwin now does in *every* invocation of
spawn/exec:

  JOBOBJECT_BASIC_LIMIT_INFORMATION jobinfo;
  if (QueryInformationJobObject (NULL, JobObjectBasicLimitInformation,
                                 &jobinfo, sizeof jobinfo, NULL)
      && (jobinfo.LimitFlags & (JOB_OBJECT_LIMIT_BREAKAWAY_OK
                                | JOB_OBJECT_LIMIT_SILENT_BREAKAWAY_OK)))
    {     
      /* Add CREATE_BREAKAWAY_FROM_JOB to the CreateProcess flags. */
      c_flags |= CREATE_BREAKAWAY_FROM_JOB;
    }       

While this works, what bugs me is that this additional system call takes
extra time and we all know that Cygwin is slow.  Ask the users.

I have still no idea why mintty is affected.  It doesn't show up in the
registry-based PCA database.  Yesterday I found this on MSDN:

(Continue reading)

Andy Koppe | 16 Feb 2012 14:15

Picon

Re: Cygwin's spawn/exec, mintty and the "Program Compatibility Assistant"

On 16 February 2012 12:46, Corinna Vinschen wrote:
> Hi guys, and especially
> Hi Andy,
>
>
> A couple of weeks ago we noticed this strange problem with mintty.  The
> "Program Compatibility Assistant" (PCA) on W7 took it under its wings
> for no apparent reason, and if the session was long enough you would
> notice that a specific svchost process, the one running the Pcasvc
> service, would take more and more memory and CPU time.
>
> The solution for this problem was either to ask the user to switch off
> the PCA service, or a change in Cygwin's spawn/exec code.  in short, PCA
> sets up a Job for a controlled application, but allows to breakaway from
> the job.  That's what Cygwin now does in *every* invocation of
> spawn/exec:
>
>  JOBOBJECT_BASIC_LIMIT_INFORMATION jobinfo;
>  if (QueryInformationJobObject (NULL, JobObjectBasicLimitInformation,
>                                 &jobinfo, sizeof jobinfo, NULL)
>      && (jobinfo.LimitFlags & (JOB_OBJECT_LIMIT_BREAKAWAY_OK
>                                | JOB_OBJECT_LIMIT_SILENT_BREAKAWAY_OK)))
>    {
>      /* Add CREATE_BREAKAWAY_FROM_JOB to the CreateProcess flags. */
>      c_flags |= CREATE_BREAKAWAY_FROM_JOB;
>    }
>
> While this works, what bugs me is that this additional system call takes
> extra time and we all know that Cygwin is slow.  Ask the users.
>

(Continue reading)

Corinna Vinschen | 16 Feb 2012 14:39

Re: Cygwin's spawn/exec, mintty and the "Program Compatibility Assistant"

On Feb 16 13:15, Andy Koppe wrote:
> On 16 February 2012 12:46, Corinna Vinschen wrote:
> > Hi guys, and especially
> > Hi Andy,
> >
> >
> > A couple of weeks ago we noticed this strange problem with mintty.  The
> > "Program Compatibility Assistant" (PCA) on W7 took it under its wings
> > for no apparent reason, and if the session was long enough you would
> > notice that a specific svchost process, the one running the Pcasvc
> > service, would take more and more memory and CPU time.
> >
> > The solution for this problem was either to ask the user to switch off
> > the PCA service, or a change in Cygwin's spawn/exec code.  in short, PCA
> > sets up a Job for a controlled application, but allows to breakaway from
> > the job.  That's what Cygwin now does in *every* invocation of
> > spawn/exec:
> >
> >  JOBOBJECT_BASIC_LIMIT_INFORMATION jobinfo;
> >  if (QueryInformationJobObject (NULL, JobObjectBasicLimitInformation,
> >                                 &jobinfo, sizeof jobinfo, NULL)
> >      && (jobinfo.LimitFlags & (JOB_OBJECT_LIMIT_BREAKAWAY_OK
> >                                | JOB_OBJECT_LIMIT_SILENT_BREAKAWAY_OK)))
> >    {
> >      /* Add CREATE_BREAKAWAY_FROM_JOB to the CreateProcess flags. */
> >      c_flags |= CREATE_BREAKAWAY_FROM_JOB;
> >    }
> >
> > While this works, what bugs me is that this additional system call takes
> > extra time and we all know that Cygwin is slow.  Ask the users.

(Continue reading)

Corinna Vinschen | 17 Feb 2012 11:34

Re: Cygwin's spawn/exec, mintty and the "Program Compatibility Assistant"

On Feb 16 14:39, Corinna Vinschen wrote:
> On Feb 16 13:15, Andy Koppe wrote:
> > On 16 February 2012 12:46, Corinna Vinschen wrote:
> > > I have still no idea why mintty is affected.  It doesn't show up in the
> > > registry-based PCA database.  Yesterday I found this on MSDN:
> > > http://msdn.microsoft.com/en-us/library/bb756937.aspx but I don't see
> > > anything there which would explain the effect for mintty.  There's a
> > > section called "Detecting Program Failures Due to Deprecated Windows
> > > Components", maybe that's a hint?
> > [...]
> > > Apart from that, apparently there are multiple other ways to disable PCA
> > > from meddling with mintty.  One of them is the (incorrectly documented)
> > > registry value
> > >
> > >  HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows
NT\CurrentVersion\AppCompatFlags\Compatibility Assistant\ExecutablesToExclude
> > >
> > > which is a REG_MULTI_SZ value you can fill with full paths to the
> > > exeutables to exclude.  I tried that and it actually works.  So, instead
> > > of calling QueryInformationJobObject for each spawn/exec, we could just
> > > add mintty.exe to that registry key.
> > >
> > > Or, alternatively, mintty could come with a builtin manifest.  I just
> > > don't know how to include a manifest into a binary...
> > 
> > Mintty already has a manifest, namely the res.mft in its sources, so
> > I'd just need to know what needs to go in there.
> 
> I'm not sure.  Per the articel, it should be sufficient to include an
> "asInvoker" manifest, see the section "Excluding Programs from PCA".

(Continue reading)

Corinna Vinschen | 17 Feb 2012 13:18

Re: Cygwin's spawn/exec, mintty and the "Program Compatibility Assistant"

On Feb 17 11:34, Corinna Vinschen wrote:
> This is not only about the manifest apparently.  [...]

Nope, this is *ALL* about the manifest, apparently.

I still don't understand what's going on, but I finally puzzled out that
it's not only mintty.

In fact, if I start *any* self-built console process directly from
Windows Explorer I find the process being part of a PCA job!  If I
start CMD first, and then the console application from there, there'
no job object involved.

This is NOT Cygwin-only.  This also occurs with self-built non-Cygwin
(mingw) applications.  And what's even more puzzeling is the fact that
adding an "asInvoker" manifest, internal or external, doesn't help
anything...

On W7!  On Vista, the manifest helps.

Some more digging on MSDN turned up two interesting blogs from a guy
calling himself "the App Compat Guy":

First, here's the blog entry which describes why this doesn't work on
W7.  It's a funny one, since he's not very happy with what happens on
W7, apparently:

http://blogs.msdn.com/b/cjacks/archive/2009/06/18/pca-changes-for-windows-7-how-to-tell-us-you-are-not-an-installer-take-2-because-we-changed-the-rules-on-you.aspx

While he talks about installers, the "per OS" manifest problem

(Continue reading)

Earnie Boyd | 17 Feb 2012 15:27

Picon

Re: Cygwin's spawn/exec, mintty and the "Program Compatibility Assistant"

On Fri, Feb 17, 2012 at 7:18 AM, Corinna Vinschen wrote:
> Bottom line:
>
> - From observation I assume that it's all about having a runlevel manifest
>  or not.  On W7 the binary additionally needs a special W7 compatible
>  manifest.
>
> - Console applications are only affected when started directly from the
>  GUI.  If they are started from a correctly-manifested console app (CMD),
>  they and their children are not affected anymore.
>
> - So we would have to create all executables with a W7-compatible
>  manifest.  That is quite certainly not an option.
>
> - So we have to keep the CREATE_BREAKAWAY_FROM_JOB stuff in Cygwin
>  for the time being.

Or, create a specially crafted mintty.bat file that uses ``%COMSPEC%
/c start mintty.exe''.

--

-- 
Earnie
-- https://sites.google.com/site/earnieboyd

Charles Wilson | 17 Feb 2012 21:26

Re: Cygwin's spawn/exec, mintty and the "Program Compatibility Assistant"

On 2/17/2012 7:18 AM, Corinna Vinschen wrote:
> Bottom line:
> 
> - From observation I assume that it's all about having a runlevel manifest
>   or not.  On W7 the binary additionally needs a special W7 compatible
>   manifest.
> 
> - Console applications are only affected when started directly from the
>   GUI.  If they are started from a correctly-manifested console app (CMD),
>   they and their children are not affected anymore.
> 
> - So we would have to create all executables with a W7-compatible
>   manifest.  That is quite certainly not an option.
> 
> - So we have to keep the CREATE_BREAKAWAY_FROM_JOB stuff in Cygwin
>   for the time being.

So, should run.exe (and run2.exe) be modified to do something similar to
what Cygwin does, since they both use CreateProcess directly?

--
Chuck

Corinna Vinschen | 17 Feb 2012 21:56

Re: Cygwin's spawn/exec, mintty and the "Program Compatibility Assistant"

On Feb 17 15:26, Charles Wilson wrote:
> On 2/17/2012 7:18 AM, Corinna Vinschen wrote:
> > Bottom line:
> > 
> > - From observation I assume that it's all about having a runlevel manifest
> >   or not.  On W7 the binary additionally needs a special W7 compatible
> >   manifest.
> > 
> > - Console applications are only affected when started directly from the
> >   GUI.  If they are started from a correctly-manifested console app (CMD),
> >   they and their children are not affected anymore.
> > 
> > - So we would have to create all executables with a W7-compatible
> >   manifest.  That is quite certainly not an option.
> > 
> > - So we have to keep the CREATE_BREAKAWAY_FROM_JOB stuff in Cygwin
> >   for the time being.
> 
> So, should run.exe (and run2.exe) be modified to do something similar to
> what Cygwin does, since they both use CreateProcess directly?

Just tested.  Yes, that should be helpful.  For run and run2 it should be
sufficient to do what the former code in Cygwin did:

  JOBOBJECT_BASIC_LIMIT_INFORMATION jobinfo;
  if (QueryInformationJobObject (NULL, JobObjectBasicLimitInformation,
  				 &jobinfo, sizeof jobinfo, NULL)
      & (jobinfo.LimitFlags & (JOB_OBJECT_LIMIT_BREAKAWAY_OK
			       | JOB_OBJECT_LIMIT_SILENT_BREAKAWAY_OK)))
    create_flags |= CREATE_BREAKAWAY_FROM_JOB;

(Continue reading)

Corinna Vinschen | 17 Feb 2012 22:04

Re: Cygwin's spawn/exec, mintty and the "Program Compatibility Assistant"

On Feb 17 21:56, Corinna Vinschen wrote:
> On Feb 17 15:26, Charles Wilson wrote:
> > On 2/17/2012 7:18 AM, Corinna Vinschen wrote:
> > > Bottom line:
> > > 
> > > - From observation I assume that it's all about having a runlevel manifest
> > >   or not.  On W7 the binary additionally needs a special W7 compatible
> > >   manifest.
> > > 
> > > - Console applications are only affected when started directly from the
> > >   GUI.  If they are started from a correctly-manifested console app (CMD),
> > >   they and their children are not affected anymore.
> > > 
> > > - So we would have to create all executables with a W7-compatible
> > >   manifest.  That is quite certainly not an option.
> > > 
> > > - So we have to keep the CREATE_BREAKAWAY_FROM_JOB stuff in Cygwin
> > >   for the time being.
> > 
> > So, should run.exe (and run2.exe) be modified to do something similar to
> > what Cygwin does, since they both use CreateProcess directly?
> 
> Just tested.  Yes, that should be helpful.  For run and run2 it should be
> sufficient to do what the former code in Cygwin did:
> 
>   JOBOBJECT_BASIC_LIMIT_INFORMATION jobinfo;
>   if (QueryInformationJobObject (NULL, JobObjectBasicLimitInformation,
>   				 &jobinfo, sizeof jobinfo, NULL)
>       & (jobinfo.LimitFlags & (JOB_OBJECT_LIMIT_BREAKAWAY_OK
> 			       | JOB_OBJECT_LIMIT_SILENT_BREAKAWAY_OK)))

(Continue reading)

Andy Koppe | 19 Feb 2012 07:00

Picon

Re: Cygwin's spawn/exec, mintty and the "Program Compatibility Assistant"

On 17 February 2012 12:18, Corinna Vinschen wrote:
> On Feb 17 11:34, Corinna Vinschen wrote:
>> This is not only about the manifest apparently.  [...]
>
> Nope, this is *ALL* about the manifest, apparently.
>
> I still don't understand what's going on, but I finally puzzled out that
> it's not only mintty.
>
> In fact, if I start *any* self-built console process directly from
> Windows Explorer I find the process being part of a PCA job!  If I
> start CMD first, and then the console application from there, there'
> no job object involved.
>
> This is NOT Cygwin-only.  This also occurs with self-built non-Cygwin
> (mingw) applications.  And what's even more puzzeling is the fact that
> adding an "asInvoker" manifest, internal or external, doesn't help
> anything...
>
> On W7!  On Vista, the manifest helps.
>
> Some more digging on MSDN turned up two interesting blogs from a guy
> calling himself "the App Compat Guy":
>
> First, here's the blog entry which describes why this doesn't work on
> W7.  It's a funny one, since he's not very happy with what happens on
> W7, apparently:
>
> http://blogs.msdn.com/b/cjacks/archive/2009/06/18/pca-changes-for-windows-7-how-to-tell-us-you-are-not-an-installer-take-2-because-we-changed-the-rules-on-you.aspx
>

(Continue reading)

Corinna Vinschen | 19 Feb 2012 14:45

Re: Cygwin's spawn/exec, mintty and the "Program Compatibility Assistant"

On Feb 19 06:00, Andy Koppe wrote:
> On 17 February 2012 12:18, Corinna Vinschen wrote:
> > Bottom line:
> >
> > - From observation I assume that it's all about having a runlevel manifest
> >  or not.  On W7 the binary additionally needs a special W7 compatible
> >  manifest.
> >
> > - Console applications are only affected when started directly from the
> >  GUI.  If they are started from a correctly-manifested console app (CMD),
> >  they and their children are not affected anymore.
> >
> > - So we would have to create all executables with a W7-compatible
> >  manifest.  That is quite certainly not an option.
> >
> > - So we have to keep the CREATE_BREAKAWAY_FROM_JOB stuff in Cygwin
> >  for the time being.
> 
> Sorry to be a bit dense, but does this mean for mintty? Do I need to
> add something to the manifest, and if so, how urgent is it?

For mintty this means you *can* extend its manifest if you like.  It
will avoid that PCA takes control of mintty.  Here's what works for me,
even on W7.  The GUIDs identify the operating system versions.  The
first one is for W7, the second one for Vista.  The GUID for W8 has not
been released yet, afaics.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.0">
  [...]

(Continue reading)

Andy Koppe | 17 Feb 2012 12:51

Picon

Re: Cygwin's spawn/exec, mintty and the "Program Compatibility Assistant"

On 16 February 2012 13:39, Corinna Vinschen wrote:
> On Feb 16 13:15, Andy Koppe wrote:
>> On 16 February 2012 12:46, Corinna Vinschen wrote:
>> > Hi guys, and especially
>> > Hi Andy,
>> >
>> >
>> > A couple of weeks ago we noticed this strange problem with mintty.  The
>> > "Program Compatibility Assistant" (PCA) on W7 took it under its wings
>> > for no apparent reason, and if the session was long enough you would
>> > notice that a specific svchost process, the one running the Pcasvc
>> > service, would take more and more memory and CPU time.
>> >
>> > The solution for this problem was either to ask the user to switch off
>> > the PCA service, or a change in Cygwin's spawn/exec code.  in short, PCA
>> > sets up a Job for a controlled application, but allows to breakaway from
>> > the job.  That's what Cygwin now does in *every* invocation of
>> > spawn/exec:
>> >
>> >  JOBOBJECT_BASIC_LIMIT_INFORMATION jobinfo;
>> >  if (QueryInformationJobObject (NULL, JobObjectBasicLimitInformation,
>> >                                 &jobinfo, sizeof jobinfo, NULL)
>> >      && (jobinfo.LimitFlags & (JOB_OBJECT_LIMIT_BREAKAWAY_OK
>> >                                | JOB_OBJECT_LIMIT_SILENT_BREAKAWAY_OK)))
>> >    {
>> >      /* Add CREATE_BREAKAWAY_FROM_JOB to the CreateProcess flags. */
>> >      c_flags |= CREATE_BREAKAWAY_FROM_JOB;
>> >    }
>> >
>> > While this works, what bugs me is that this additional system call takes

(Continue reading)

Return

Return to gmane.os.cygwin.devel.

Project Web Page

Development discussion of the internals of the Cygwin DLL

Search Archive

Language

Change language

Options

Current view: Threads only / Showing only 20 lines / Not hiding cited text.
Change to All messages, whole messages, or hide cited text.

Post a message
NNTP Newsgroup
Classic Gmane web interface
XML

XML

RSS Feed
List Information

About Gmane