headers
Matthew Wilcox | 13 May 19:45

Re: glibc hppa build failure - ulimit

On Fri, May 13, 2005 at 01:21:48PM -0400, Daniel Jacobowitz wrote:
> On Fri, May 13, 2005 at 05:38:33PM +0100, Matthew Wilcox wrote:
> > On Fri, May 13, 2005 at 12:17:18PM -0400, Daniel Jacobowitz wrote:
> > > 27319 mmap(NULL, 1073741824, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0
<unfinished ...>
> > > 27319 <... mmap resumed> )              = -1 ENOMEM (Cannot allocate memory)
> > > 
> > > Right now I don't think we could even rebuild glibc -21.  The hppa
> > > machines are configured with ulimit -s set to 1GB.  This makes
> > > LinuxThreads use 1GB thread stacks.  Which is, um, pretty bad.
> > 
> > PA machines grow the stack upwards, starting at 0xffffffff - hard
> > stack limit.  glibc never used to pay attention to the stack limit,
> > choosing always to use 4MB stacks (iirc).  When did glibc change that?
> 
> Probably when Carlos added a patch to glibc which defined
> FLOATING_STACKS.  Glibc throttles the size to 8MB if the rlimit is
> infinity, but trusts the rlimit if it is explicitly larger than 8MB.

Ugh.  Can we change the logic there to throttle to 8MB if the rlimit is
larger than 1GB and we're building a 32-bit libc?

--

-- 
"Next the statesmen will invent cheap lies, putting the blame upon 
the nation that is attacked, and every man will be glad of those
conscience-soothing falsities, and will diligently study them, and refuse
to examine any refutations of them; and thus he will by and by convince 
himself that the war is just, and will thank God for the better sleep 
he enjoys after this process of grotesque self-deception." -- Mark Twain
(Continue reading)

Permalink | Reply |
headers
Daniel Jacobowitz | 13 May 20:10
Picon
Favicon

Re: glibc hppa build failure - ulimit

On Fri, May 13, 2005 at 06:45:35PM +0100, Matthew Wilcox wrote:
> On Fri, May 13, 2005 at 01:21:48PM -0400, Daniel Jacobowitz wrote:
> > On Fri, May 13, 2005 at 05:38:33PM +0100, Matthew Wilcox wrote:
> > > On Fri, May 13, 2005 at 12:17:18PM -0400, Daniel Jacobowitz wrote:
> > > > 27319 mmap(NULL, 1073741824, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANONYMOUS, -1,
0 <unfinished ...>
> > > > 27319 <... mmap resumed> )              = -1 ENOMEM (Cannot allocate memory)
> > > > 
> > > > Right now I don't think we could even rebuild glibc -21.  The hppa
> > > > machines are configured with ulimit -s set to 1GB.  This makes
> > > > LinuxThreads use 1GB thread stacks.  Which is, um, pretty bad.
> > > 
> > > PA machines grow the stack upwards, starting at 0xffffffff - hard
> > > stack limit.  glibc never used to pay attention to the stack limit,
> > > choosing always to use 4MB stacks (iirc).  When did glibc change that?
> > 
> > Probably when Carlos added a patch to glibc which defined
> > FLOATING_STACKS.  Glibc throttles the size to 8MB if the rlimit is
> > infinity, but trusts the rlimit if it is explicitly larger than 8MB.
> 
> Ugh.  Can we change the logic there to throttle to 8MB if the rlimit is
> larger than 1GB and we're building a 32-bit libc?

In the future?  Probably, ask Carlos.  But for sarge, I'd much rather
not make additional changes.

--

-- 
Daniel Jacobowitz
CodeSourcery, LLC
(Continue reading)

Permalink | Reply |
headers
Matthew Wilcox | 13 May 21:32

Re: glibc hppa build failure - ulimit

On Fri, May 13, 2005 at 02:10:49PM -0400, Daniel Jacobowitz wrote:
> On Fri, May 13, 2005 at 06:45:35PM +0100, Matthew Wilcox wrote:
> > On Fri, May 13, 2005 at 01:21:48PM -0400, Daniel Jacobowitz wrote:
> > > On Fri, May 13, 2005 at 05:38:33PM +0100, Matthew Wilcox wrote:
> > > > On Fri, May 13, 2005 at 12:17:18PM -0400, Daniel Jacobowitz wrote:
> > > > > 27319 mmap(NULL, 1073741824, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANONYMOUS,
-1, 0 <unfinished ...>
> > > > > 27319 <... mmap resumed> )              = -1 ENOMEM (Cannot allocate memory)
> > > > > 
> > > > > Right now I don't think we could even rebuild glibc -21.  The hppa
> > > > > machines are configured with ulimit -s set to 1GB.  This makes
> > > > > LinuxThreads use 1GB thread stacks.  Which is, um, pretty bad.
> > > > 
> > > > PA machines grow the stack upwards, starting at 0xffffffff - hard
> > > > stack limit.  glibc never used to pay attention to the stack limit,
> > > > choosing always to use 4MB stacks (iirc).  When did glibc change that?
> > > 
> > > Probably when Carlos added a patch to glibc which defined
> > > FLOATING_STACKS.  Glibc throttles the size to 8MB if the rlimit is
> > > infinity, but trusts the rlimit if it is explicitly larger than 8MB.
> > 
> > Ugh.  Can we change the logic there to throttle to 8MB if the rlimit is
> > larger than 1GB and we're building a 32-bit libc?
> 
> In the future?  Probably, ask Carlos.  But for sarge, I'd much rather
> not make additional changes.

Well, how can we fix this for sarge then?  Either we need to change glibc
or the kernel, both of which are extremely deeply frozen.  I'd rather
fix this by reverting to our previous behaviour of assuming 8MB stacks
(Continue reading)

Permalink | Reply |
headers
Daniel Jacobowitz | 13 May 21:55
Picon
Favicon

Re: glibc hppa build failure - ulimit

On Fri, May 13, 2005 at 08:32:53PM +0100, Matthew Wilcox wrote:
> Well, how can we fix this for sarge then?  Either we need to change glibc
> or the kernel, both of which are extremely deeply frozen.  I'd rather
> fix this by reverting to our previous behaviour of assuming 8MB stacks
> for threads than change the kernel's behaviour to limit stacks to 8MB
> by default.  That would seem to risk breaking much more stuff.

I'm not following.  Are you saying that the changed ulimit is the
result of a kernel change?  That seems very strange, since sarti has a
134 day uptime and did not have this problem building glibc a month
ago for -21.

If you want to change glibc at this point, discuss with Carlos; I can't
take care of it.  Just getting -22 built has taken most of my free time
for this week.

--

-- 
Daniel Jacobowitz
CodeSourcery, LLC
Permalink | Reply |
headers
Matthew Wilcox | 13 May 22:26

Re: glibc hppa build failure - ulimit

On Fri, May 13, 2005 at 03:55:39PM -0400, Daniel Jacobowitz wrote:
> On Fri, May 13, 2005 at 08:32:53PM +0100, Matthew Wilcox wrote:
> > Well, how can we fix this for sarge then?  Either we need to change glibc
> > or the kernel, both of which are extremely deeply frozen.  I'd rather
> > fix this by reverting to our previous behaviour of assuming 8MB stacks
> > for threads than change the kernel's behaviour to limit stacks to 8MB
> > by default.  That would seem to risk breaking much more stuff.
> 
> I'm not following.  Are you saying that the changed ulimit is the
> result of a kernel change?  That seems very strange, since sarti has a
> 134 day uptime and did not have this problem building glibc a month
> ago for -21.

No, the kernel's had a 1GB ulimit for years.  Something must've changed
in glibc.

> If you want to change glibc at this point, discuss with Carlos; I can't
> take care of it.  Just getting -22 built has taken most of my free time
> for this week.

Carlos?  This seems like your fault, how do *you* want to fix it?  ;-)

--

-- 
"Next the statesmen will invent cheap lies, putting the blame upon 
the nation that is attacked, and every man will be glad of those
conscience-soothing falsities, and will diligently study them, and refuse
to examine any refutations of them; and thus he will by and by convince 
himself that the war is just, and will thank God for the better sleep 
he enjoys after this process of grotesque self-deception." -- Mark Twain
(Continue reading)

Permalink | Reply |
headers
Carlos O'Donell | 14 May 21:15
Picon

Re: glibc hppa build failure - ulimit

On Fri, May 13, 2005 at 09:26:10PM +0100, Matthew Wilcox wrote:
> > If you want to change glibc at this point, discuss with Carlos; I can't
> > take care of it.  Just getting -22 built has taken most of my free time
> > for this week.
> 
> Carlos?  This seems like your fault, how do *you* want to fix it?  ;-)

We have always had floating stacks in hppa, that's not the issue.
The issue here is that someone set the ulimit to 1GB. What does that
have to do with me? :)

If glibc sees a ulimit that isn't inifinty it attempts to set the limit
instead of enforcing the maximum limit (8MB).

This behaviour has been around forever, why is the kernel setting 1GB
limits instead of infinity?

c.
Permalink | Reply |
headers
Matthew Wilcox | 15 May 15:52

Re: glibc hppa build failure - ulimit

On Sat, May 14, 2005 at 03:15:36PM -0400, Carlos O'Donell wrote:
> On Fri, May 13, 2005 at 09:26:10PM +0100, Matthew Wilcox wrote:
> > > If you want to change glibc at this point, discuss with Carlos; I can't
> > > take care of it.  Just getting -22 built has taken most of my free time
> > > for this week.
> > 
> > Carlos?  This seems like your fault, how do *you* want to fix it?  ;-)
> 
> We have always had floating stacks in hppa, that's not the issue.
> The issue here is that someone set the ulimit to 1GB. What does that
> have to do with me? :)

The ulimit's been 1GB since 2000.  Actually, I think that's only true
if you ssh into a machine.  ssh tries to set the ulimit to infinity
and we throttle that to 1GB.  So if you were always building locally,
you'd never notice the problem, the stack would be 80MB.

--

-- 
"Next the statesmen will invent cheap lies, putting the blame upon 
the nation that is attacked, and every man will be glad of those
conscience-soothing falsities, and will diligently study them, and refuse
to examine any refutations of them; and thus he will by and by convince 
himself that the war is just, and will thank God for the better sleep 
he enjoys after this process of grotesque self-deception." -- Mark Twain
Permalink | Reply |
headers
Carlos O'Donell | 15 May 22:17
Picon

Re: glibc hppa build failure - ulimit

On Sun, May 15, 2005 at 02:52:58PM +0100, Matthew Wilcox wrote:
> On Sat, May 14, 2005 at 03:15:36PM -0400, Carlos O'Donell wrote:
> > On Fri, May 13, 2005 at 09:26:10PM +0100, Matthew Wilcox wrote:
> > > > If you want to change glibc at this point, discuss with Carlos; I can't
> > > > take care of it.  Just getting -22 built has taken most of my free time
> > > > for this week.
> > > 
> > > Carlos?  This seems like your fault, how do *you* want to fix it?  ;-)
> > 
> > We have always had floating stacks in hppa, that's not the issue.
> > The issue here is that someone set the ulimit to 1GB. What does that
> > have to do with me? :)
> 
> The ulimit's been 1GB since 2000.  Actually, I think that's only true
> if you ssh into a machine.  ssh tries to set the ulimit to infinity
> and we throttle that to 1GB.  So if you were always building locally,
> you'd never notice the problem, the stack would be 80MB.

Where is the stack limit set to 1GB in the kernel? I don't see any
throttling code to that effect.

I see that the default process limit is set to 10 * the _STK_LIM, so 80MB
as you say. I don't see any sort of 1GB throttling besides the VM code
which has to throttle to [RLIMIT_STACK].rlim_cur.

If ssh sets the rlimit to infinity then new threads will have 8MB
stacks, which is reasonable.

I never build locally, I always ssh into by build system (and run screen
though).
(Continue reading)

Permalink | Reply |

Gmane