Harald Hoyer | 6 Sep 12:52
Favicon
Gravatar

Boot speedup with readahead

Hello fellow Fedora developers,

recently readahead was modified to adapt to system file changes and to start very early in the boot process 
via upstart.

I would like to encourage you to test readahead-1.4.5-3.fc10 from rawhide (even possible on F9), which I
built 
some minutes ago. It may take a day to reach your local mirror.

# yum --enablerepo=rawhide install readahead
or
# yum --enablerepo=rawhide update readahead

With the next reboot readahead-collector runs and collects the information which files are used during
the 
boot process. The next reboot then, readahead read ahead those files and the boot process (from init start
to 
gdm login screen) should be approx. 10% faster.

Hope it speeds up your boot process a little bit. Please report any changes in your boot time (can be measured 
with a stop clock or bootchart).

You can modify /etc/sysconfig/readahead to turn readahead on/off.

--

-- 
fedora-devel-list mailing list
fedora-devel-list <at> redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list

(Continue reading)

Eric Sandeen | 6 Sep 17:32
Favicon
Gravatar

Re: Boot speedup with readahead

Harald Hoyer wrote:
> Hello fellow Fedora developers,
> 
> recently readahead was modified to adapt to system file changes and to start very early in the boot process 
> via upstart.

Glad to see that!  Sorry I helped put the hammer down on older readahead
in the F8 era... but it was pretty broken back then ... ;)

> I would like to encourage you to test readahead-1.4.5-3.fc10 from rawhide (even possible on F9), which I
built 
> some minutes ago. It may take a day to reach your local mirror.
> 
> # yum --enablerepo=rawhide install readahead
> or
> # yum --enablerepo=rawhide update readahead
> 
> With the next reboot readahead-collector runs and collects the information which files are used during
the 
> boot process. The next reboot then, readahead read ahead those files and the boot process (from init start
to 
> gdm login screen) should be approx. 10% faster.

So when / how often does readahead-collector run now?

Thanks,
-Eric

> Hope it speeds up your boot process a little bit. Please report any changes in your boot time (can be
measured 
(Continue reading)

Harald Hoyer | 8 Sep 15:48
Favicon
Gravatar

Re: Boot speedup with readahead

Eric Sandeen schrieb:
> Harald Hoyer wrote:
>> Hello fellow Fedora developers,
>>
>> recently readahead was modified to adapt to system file changes and to start very early in the boot
process 
>> via upstart.
> 
> Glad to see that!  Sorry I helped put the hammer down on older readahead
> in the F8 era... but it was pretty broken back then ... ;)
> 
>> I would like to encourage you to test readahead-1.4.5-3.fc10 from rawhide (even possible on F9), which I
built 
>> some minutes ago. It may take a day to reach your local mirror.
>>
>> # yum --enablerepo=rawhide install readahead
>> or
>> # yum --enablerepo=rawhide update readahead
>>
>> With the next reboot readahead-collector runs and collects the information which files are used during
the 
>> boot process. The next reboot then, readahead read ahead those files and the boot process (from init
start to 
>> gdm login screen) should be approx. 10% faster.
> 
> So when / how often does readahead-collector run now?
> 
> Thanks,
> -Eric

(Continue reading)

Eric Sandeen | 8 Sep 16:49
Favicon
Gravatar

Re: Boot speedup with readahead

Harald Hoyer wrote:
> Eric Sandeen schrieb:

>> So when / how often does readahead-collector run now?
>>
>> Thanks,
>> -Eric
> 
> Every month.
> 
> /etc/cron.monthly/readahead-monthly.cron

It'd be interesting to do a daily yum update / reboot and see how the
boot times look, graphed for a couple months.

Will things degrade until the next collection run?  I wonder if some
inotify magic might be interesting; if more than X% of the
previously-collected files have changed, then kick the collector on again?

-Eric

--

-- 
fedora-devel-list mailing list
fedora-devel-list <at> redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list

Harald Hoyer | 8 Sep 17:01
Favicon
Gravatar

Re: Boot speedup with readahead

Eric Sandeen schrieb:
> Harald Hoyer wrote:
>> Eric Sandeen schrieb:
> 
>>> So when / how often does readahead-collector run now?
>>>
>>> Thanks,
>>> -Eric
>> Every month.
>>
>> /etc/cron.monthly/readahead-monthly.cron
> 
> It'd be interesting to do a daily yum update / reboot and see how the
> boot times look, graphed for a couple months.
> 
> Will things degrade until the next collection run?  I wonder if some
> inotify magic might be interesting; if more than X% of the
> previously-collected files have changed, then kick the collector on again?
> 
> -Eric
> 

hehe, yes, but how would you implement that? :) I don't think this is doable :)

--

-- 
fedora-devel-list mailing list
fedora-devel-list <at> redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list

(Continue reading)

Panu Matilainen | 8 Sep 17:07

Re: Boot speedup with readahead

On Mon, 8 Sep 2008, Harald Hoyer wrote:

> Eric Sandeen schrieb:
>> Harald Hoyer wrote:
>>> Eric Sandeen schrieb:
>> 
>>>> So when / how often does readahead-collector run now?
>>>> 
>>>> Thanks,
>>>> -Eric
>>> Every month.
>>> 
>>> /etc/cron.monthly/readahead-monthly.cron
>> 
>> It'd be interesting to do a daily yum update / reboot and see how the
>> boot times look, graphed for a couple months.
>> 
>> Will things degrade until the next collection run?  I wonder if some
>> inotify magic might be interesting; if more than X% of the
>> previously-collected files have changed, then kick the collector on again?
>> 
>> -Eric
>> 
>
> hehe, yes, but how would you implement that? :) I don't think this is doable 
> :)

A yum-readahead plugin could look at the readahead list after an upgrade 
and calculate if refreshing is needed.

(Continue reading)

Seth Vidal | 8 Sep 17:12
Favicon

Re: Boot speedup with readahead

On Mon, 2008-09-08 at 18:07 +0300, Panu Matilainen wrote:
> On Mon, 8 Sep 2008, Harald Hoyer wrote:
> 
> > Eric Sandeen schrieb:
> >> Harald Hoyer wrote:
> >>> Eric Sandeen schrieb:
> >> 
> >>>> So when / how often does readahead-collector run now?
> >>>> 
> >>>> Thanks,
> >>>> -Eric
> >>> Every month.
> >>> 
> >>> /etc/cron.monthly/readahead-monthly.cron
> >> 
> >> It'd be interesting to do a daily yum update / reboot and see how the
> >> boot times look, graphed for a couple months.
> >> 
> >> Will things degrade until the next collection run?  I wonder if some
> >> inotify magic might be interesting; if more than X% of the
> >> previously-collected files have changed, then kick the collector on again?
> >> 
> >> -Eric
> >> 
> >
> > hehe, yes, but how would you implement that? :) I don't think this is doable 
> > :)
> 
> A yum-readahead plugin could look at the readahead list after an upgrade 
> and calculate if refreshing is needed.
(Continue reading)

Jeremy Katz | 8 Sep 17:27
Favicon

Re: Boot speedup with readahead

On Mon, 2008-09-08 at 11:12 -0400, Seth Vidal wrote:
> On Mon, 2008-09-08 at 18:07 +0300, Panu Matilainen wrote:
> > On Mon, 8 Sep 2008, Harald Hoyer wrote:
> > > Eric Sandeen schrieb:
> > >> Harald Hoyer wrote:
> > >>> Eric Sandeen schrieb:
> > >>>> So when / how often does readahead-collector run now?
> > >>>> 
> > >>>> Thanks,
> > >>>> -Eric
> > >>> Every month.
> > >>> 
> > >>> /etc/cron.monthly/readahead-monthly.cron
> > >> 
> > >> It'd be interesting to do a daily yum update / reboot and see how the
> > >> boot times look, graphed for a couple months.
> > >> 
> > >> Will things degrade until the next collection run?  I wonder if some
> > >> inotify magic might be interesting; if more than X% of the
> > >> previously-collected files have changed, then kick the collector on again?
> > >> 
> > >> -Eric
> > >> 
> > >
> > > hehe, yes, but how would you implement that? :) I don't think this is doable 
> > > :)
> > 
> > A yum-readahead plugin could look at the readahead list after an upgrade 
> > and calculate if refreshing is needed.
> > 
(Continue reading)

Seth Vidal | 8 Sep 18:02
Favicon

Re: Boot speedup with readahead

On Mon, 2008-09-08 at 11:27 -0400, Jeremy Katz wrote:

> And doing it this way would be more effective than cron as there are
> plenty of people (especially with laptops/desktops which aren't on all
> the time) for whom cron really isn't appropriate for anything that we
> actually care to have done
> 

What package does the readahead cron-monthly job live in?

-sv

--

-- 
fedora-devel-list mailing list
fedora-devel-list <at> redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list

Eric Sandeen | 8 Sep 18:08
Favicon
Gravatar

Re: Boot speedup with readahead

Seth Vidal wrote:
> On Mon, 2008-09-08 at 11:27 -0400, Jeremy Katz wrote:
> 
>> And doing it this way would be more effective than cron as there are
>> plenty of people (especially with laptops/desktops which aren't on all
>> the time) for whom cron really isn't appropriate for anything that we
>> actually care to have done
>>
> 
> What package does the readahead cron-monthly job live in?
> 
> -sv
> 
> 

it's in readahead:

[root <at> box]# rpm -ql readahead | grep cron
/etc/cron.daily/readahead.cron
/etc/cron.monthly/readahead-monthly.cron

But we should probably wait a week to see what magic Arjan has before we
go off re-architecting any of this :)

-Eric

--

-- 
fedora-devel-list mailing list
fedora-devel-list <at> redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list
(Continue reading)

Seth Vidal | 8 Sep 18:15
Favicon

Re: Boot speedup with readahead

On Mon, 2008-09-08 at 11:08 -0500, Eric Sandeen wrote:
> Seth Vidal wrote:
> > On Mon, 2008-09-08 at 11:27 -0400, Jeremy Katz wrote:
> > 
> >> And doing it this way would be more effective than cron as there are
> >> plenty of people (especially with laptops/desktops which aren't on all
> >> the time) for whom cron really isn't appropriate for anything that we
> >> actually care to have done
> >>
> > 
> > What package does the readahead cron-monthly job live in?
> > 
> > -sv
> > 
> > 
> 
> it's in readahead:
> 
> [root <at> box]# rpm -ql readahead | grep cron
> /etc/cron.daily/readahead.cron
> /etc/cron.monthly/readahead-monthly.cron
> 
> But we should probably wait a week to see what magic Arjan has before we
> go off re-architecting any of this :)

okay
:)

-sv

(Continue reading)

Callum Lerwick | 8 Sep 22:28

Re: Boot speedup with readahead

On Mon, 2008-09-08 at 11:27 -0400, Jeremy Katz wrote:
> And doing it this way would be more effective than cron as there are
> plenty of people (especially with laptops/desktops which aren't on all
> the time) for whom cron really isn't appropriate for anything that we
> actually care to have done

We still need to trigger pre-linking after yum transactions rather than
by a cron job...
--

-- 
fedora-devel-list mailing list
fedora-devel-list <at> redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list
Jesse Keating | 8 Sep 22:59
Favicon

Re: Boot speedup with readahead

On Mon, 2008-09-08 at 15:28 -0500, Callum Lerwick wrote:
> 
> We still need to trigger pre-linking after yum transactions rather than
> by a cron job...

So that we can have an "Optimizing harddrive" phase during updates like
OSX does?  that'd be awesome...

-- 
Jesse Keating
Fedora -- Freedom² is a feature!
identi.ca: http://identi.ca/jkeating
--

-- 
fedora-devel-list mailing list
fedora-devel-list <at> redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list
Eric Sandeen | 8 Sep 23:11
Favicon
Gravatar

Re: Boot speedup with readahead

Jesse Keating wrote:
> On Mon, 2008-09-08 at 15:28 -0500, Callum Lerwick wrote:
>> We still need to trigger pre-linking after yum transactions rather than
>> by a cron job...
> 
> So that we can have an "Optimizing harddrive" phase during updates like
> OSX does?  that'd be awesome...

I guess you can wait now, or wait later.

But yeah I agree with osx it's:

boots up fast, awesome!
background download of updates, awesome!
updates install quickly (usually) - awesome!
updates force reboot - not awesome!
reboot takes AGES that first time - not aweseome!

-Eric

--

-- 
fedora-devel-list mailing list
fedora-devel-list <at> redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list

Jesse Keating | 8 Sep 23:16
Favicon

Re: Boot speedup with readahead

On Mon, 2008-09-08 at 16:11 -0500, Eric Sandeen wrote:
> reboot takes AGES that first time - not aweseome!

At least they moved it to the reboot.  It used to be tacked on to the
tail end of each update run, so you'd download fast, install fast, and
then sit there for unknown amount of time waiting for "optimizations"
that are likely not going to make up for the lost time.

-- 
Jesse Keating
Fedora -- Freedom² is a feature!
identi.ca: http://identi.ca/jkeating
--

-- 
fedora-devel-list mailing list
fedora-devel-list <at> redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list
Ola Thoresen | 9 Sep 12:59
Favicon

Re: Boot speedup with readahead

Jesse Keating wrote:
> On Mon, 2008-09-08 at 15:28 -0500, Callum Lerwick wrote:
>> We still need to trigger pre-linking after yum transactions rather than
>> by a cron job...
> 
> So that we can have an "Optimizing harddrive" phase during updates like
> OSX does?  that'd be awesome...
> 

Prelink is fine, but please don't "optimize" my brand new SSD-drive!
=;-)

Rgds.

Ola Thhoresen

-- 
         _,--',   _._.--._____
  .--.--';_'-.', ";_      _.,-'   Ola Thoresen
 .'--'.  _.'    {`'-;_ .-.>.'
       '-:_      )  / `' '=.      It is easier to fix Unix
         ) >     {_/,     /~)     than to live with Windows
         |/               `^ .'

--

-- 
fedora-devel-list mailing list
fedora-devel-list <at> redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list

(Continue reading)

Ville Skyttä | 8 Sep 23:24
Favicon

Re: Boot speedup with readahead

On Monday 08 September 2008, Callum Lerwick wrote:
> On Mon, 2008-09-08 at 11:27 -0400, Jeremy Katz wrote:
> > And doing it this way would be more effective than cron as there are
> > plenty of people (especially with laptops/desktops which aren't on all
> > the time) for whom cron really isn't appropriate for anything that we
> > actually care to have done
>
> We still need to trigger pre-linking after yum transactions rather than
> by a cron job...

Please keep users of other package managers than yum in mind, too.  Can things 
like these be hooked to rpm level transactions rather than yum 
("%triggerpostrans -- *" or something)?

--

-- 
fedora-devel-list mailing list
fedora-devel-list <at> redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list

Jeremy Katz | 8 Sep 23:33
Favicon

Re: Boot speedup with readahead

On Tue, 2008-09-09 at 00:24 +0300, Ville Skyttä wrote:
> On Monday 08 September 2008, Callum Lerwick wrote:
> > On Mon, 2008-09-08 at 11:27 -0400, Jeremy Katz wrote:
> > > And doing it this way would be more effective than cron as there are
> > > plenty of people (especially with laptops/desktops which aren't on all
> > > the time) for whom cron really isn't appropriate for anything that we
> > > actually care to have done
> >
> > We still need to trigger pre-linking after yum transactions rather than
> > by a cron job...
> 
> Please keep users of other package managers than yum in mind, too.  Can things 
> like these be hooked to rpm level transactions rather than yum 
> ("%triggerpostrans -- *" or something)?

Not really -- for one thing, there isn't functionality for transaction
level triggers.  And even if there were, using them would introduce
other new and different problems.  It really boils down to the fact that
the Fedora package manager is yum.  If users of other package managers
want to get their package manager to do similar things, we won't stop
them -- but trying to make everything work for such a small minority is
going to be increasingly difficult.  Much like if you wanted to use dpkg
+ alien to install packages rather than using rpm

Jeremy

--

-- 
fedora-devel-list mailing list
fedora-devel-list <at> redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list
(Continue reading)

Ville Skyttä | 10 Sep 20:15
Favicon

Re: Boot speedup with readahead

On Tuesday 09 September 2008, Jeremy Katz wrote:

> It really boils down to the fact that
> the Fedora package manager is yum.  If users of other package managers
> want to get their package manager to do similar things, we won't stop
> them -- but trying to make everything work for such a small minority is
> going to be increasingly difficult.

It's not really about _making_ everything work everywhere I'm worried about.  
It's about people seemingly being very eager to _move_ existing, working 
functionality to yum or its plugins when someone finds a case that could be 
optimized by doing that, and dropping the existing generic 
functionality.  "We won't stop them to make their favourite package manager 
to do similar things" does not really cover it at all well, it's closer to 
actively making things harder for them.

Keeping other package manager (including plain rpm) users in mind does not 
mean that time should be invested to make everything work for them as well as 
the optimized case does for yum users.  It's very much enough to just not 
drop the existing functionality and do the yum-optimized cases so that they 
play nice together with that.  With the right mindset, I don't think that's 
going to be at all difficult to do in the vast majority of cases.

Just a couple of examples from this thread: readahead and prelink 
updates/re-runs.  Both are things that should Just Work out of the box even 
without any package manager (even rpm!) installed.   

If I misunderstood and the optimized cases are meant to be done so that 
they're just optimized for yum users, and the existing generic functionality 
is preserved for others, then no worries.
(Continue reading)

Seth Vidal | 9 Sep 15:51
Favicon

Re: Boot speedup with readahead

On Mon, 2008-09-08 at 15:28 -0500, Callum Lerwick wrote:
> On Mon, 2008-09-08 at 11:27 -0400, Jeremy Katz wrote:
> > And doing it this way would be more effective than cron as there are
> > plenty of people (especially with laptops/desktops which aren't on all
> > the time) for whom cron really isn't appropriate for anything that we
> > actually care to have done
> 
> We still need to trigger pre-linking after yum transactions rather than
> by a cron job...

So we really need a generic yum-post-transaction trigger script.

something that says:

pkgname: action: command

ie:

glibc: *: /run/something/for/a/glibc/


How do we determine which pkgs changing need a new prelink run?

I can put this together probably today if that's what's needed.

-sv

--

-- 
fedora-devel-list mailing list
fedora-devel-list <at> redhat.com
(Continue reading)

Colin Walters | 9 Sep 16:01
Gravatar

Re: Boot speedup with readahead

On Tue, Sep 9, 2008 at 9:51 AM, Seth Vidal <skvidal <at> fedoraproject.org> wrote:
>
> So we really need a generic yum-post-transaction trigger script.
>
> something that says:
>
> pkgname: action: command
>
> ie:
>
> glibc: *: /run/something/for/a/glibc/
>
>
> How do we determine which pkgs changing need a new prelink run?

This would be great and we've really needed it for a long time.
Ideally it would be structured to recognize packages based on their
content; for example, if a package installs an icon in
/usr/share/icons, we know to invoke gtk-update-icon-cache.

--

-- 
fedora-devel-list mailing list
fedora-devel-list <at> redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list

Arjan van de Ven | 9 Sep 16:23
Favicon

Re: Boot speedup with readahead

On Tue, 9 Sep 2008 10:01:09 -0400
"Colin Walters" <walters <at> verbum.org> wrote:
> This would be great and we've really needed it for a long time.
> Ideally it would be structured to recognize packages based on their
> content; for example, if a package installs an icon in
> /usr/share/icons, we know to invoke gtk-update-icon-cache.

that makes it a question if this should be in yum or in rpm....

> 

-- 
Arjan van de Ven 	Intel Open Source Technology Centre
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org

--

-- 
fedora-devel-list mailing list
fedora-devel-list <at> redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list

Seth Vidal | 9 Sep 16:29
Favicon

Re: Boot speedup with readahead

On Tue, 2008-09-09 at 07:23 -0700, Arjan van de Ven wrote:
> On Tue, 9 Sep 2008 10:01:09 -0400
> "Colin Walters" <walters <at> verbum.org> wrote:
> > This would be great and we've really needed it for a long time.
> > Ideally it would be structured to recognize packages based on their
> > content; for example, if a package installs an icon in
> > /usr/share/icons, we know to invoke gtk-update-icon-cache.
> 
> that makes it a question if this should be in yum or in rpm....
> 

I'm fairly confident it'll be easier to do in yum.

-sv

--

-- 
fedora-devel-list mailing list
fedora-devel-list <at> redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list

Arjan van de Ven | 9 Sep 16:40
Favicon

Re: Boot speedup with readahead

On Tue, 09 Sep 2008 10:29:46 -0400
Seth Vidal <skvidal <at> fedoraproject.org> wrote:

> On Tue, 2008-09-09 at 07:23 -0700, Arjan van de Ven wrote:
> > On Tue, 9 Sep 2008 10:01:09 -0400
> > "Colin Walters" <walters <at> verbum.org> wrote:
> > > This would be great and we've really needed it for a long time.
> > > Ideally it would be structured to recognize packages based on
> > > their content; for example, if a package installs an icon in
> > > /usr/share/icons, we know to invoke gtk-update-icon-cache.
> > 
> > that makes it a question if this should be in yum or in rpm....
> > 
> 
> I'm fairly confident it'll be easier to do in yum.
> 

I absolutely do not doubt that ;)

the question is more complex than 'what is easiest' if "correctness"
type things move into these checks rather than just optimization things.

-- 
Arjan van de Ven 	Intel Open Source Technology Centre
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org

--

-- 
fedora-devel-list mailing list
fedora-devel-list <at> redhat.com
(Continue reading)

seth vidal | 9 Sep 16:43
Favicon

Re: Boot speedup with readahead

On Tue, 2008-09-09 at 07:40 -0700, Arjan van de Ven wrote:
> On Tue, 09 Sep 2008 10:29:46 -0400
> Seth Vidal <skvidal <at> fedoraproject.org> wrote:
> 
> > On Tue, 2008-09-09 at 07:23 -0700, Arjan van de Ven wrote:
> > > On Tue, 9 Sep 2008 10:01:09 -0400
> > > "Colin Walters" <walters <at> verbum.org> wrote:
> > > > This would be great and we've really needed it for a long time.
> > > > Ideally it would be structured to recognize packages based on
> > > > their content; for example, if a package installs an icon in
> > > > /usr/share/icons, we know to invoke gtk-update-icon-cache.
> > > 
> > > that makes it a question if this should be in yum or in rpm....
> > > 
> > 
> > I'm fairly confident it'll be easier to do in yum.
> > 
> 
> I absolutely do not doubt that ;)
> 
> the question is more complex than 'what is easiest' if "correctness"
> type things move into these checks rather than just optimization things.

I'll work on a simple plugin to handle this and we can see if it'll work
semi-correctly. If we decide it needs to be at the rpm-level then so be
it. :)

-sv

--

-- 
(Continue reading)

Doug Ledford | 10 Sep 17:35
Favicon

Re: Boot speedup with readahead

On Tue, 2008-09-09 at 10:29 -0400, Seth Vidal wrote:
> On Tue, 2008-09-09 at 07:23 -0700, Arjan van de Ven wrote:
> > On Tue, 9 Sep 2008 10:01:09 -0400
> > "Colin Walters" <walters <at> verbum.org> wrote:
> > > This would be great and we've really needed it for a long time.
> > > Ideally it would be structured to recognize packages based on their
> > > content; for example, if a package installs an icon in
> > > /usr/share/icons, we know to invoke gtk-update-icon-cache.
> > 
> > that makes it a question if this should be in yum or in rpm....
> > 
> 
> I'm fairly confident it'll be easier to do in yum.

It doesn't matter where it's easy to do, what matters is it's done in
the right place.  Core system infrastructure should never be a case of
done the easy way, always the right way.

-- 
Doug Ledford <dledford <at> redhat.com>
              GPG KeyID: CFBFF194
              http://people.redhat.com/dledford

Infiniband specific RPMs available at
              http://people.redhat.com/dledford/Infiniband

--

-- 
fedora-devel-list mailing list
(Continue reading)

Jesse Keating | 10 Sep 18:51
Favicon

Re: Boot speedup with readahead

On Wed, 2008-09-10 at 11:35 -0400, Doug Ledford wrote:
> > I'm fairly confident it'll be easier to do in yum.
> 
> It doesn't matter where it's easy to do, what matters is it's done in
> the right place.  Core system infrastructure should never be a case of
> done the easy way, always the right way.

Ever heard of prototyping? 

-- 
Jesse Keating
Fedora -- Freedom² is a feature!
identi.ca: http://identi.ca/jkeating
--

-- 
fedora-devel-list mailing list
fedora-devel-list <at> redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list
Seth Vidal | 9 Sep 16:36
Favicon

Re: Boot speedup with readahead

On Tue, 2008-09-09 at 10:01 -0400, Colin Walters wrote:
> On Tue, Sep 9, 2008 at 9:51 AM, Seth Vidal <skvidal <at> fedoraproject.org> wrote:
> >
> > So we really need a generic yum-post-transaction trigger script.
> >
> > something that says:
> >
> > pkgname: action: command
> >
> > ie:
> >
> > glibc: *: /run/something/for/a/glibc/
> >
> >
> > How do we determine which pkgs changing need a new prelink run?
> 
> This would be great and we've really needed it for a long time.
> Ideally it would be structured to recognize packages based on their
> content; for example, if a package installs an icon in
> /usr/share/icons, we know to invoke gtk-update-icon-cache.

Does it need to be run if the pkg removed an icon?

-sv

--

-- 
fedora-devel-list mailing list
fedora-devel-list <at> redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list

(Continue reading)

Colin Walters | 9 Sep 17:10
Gravatar

Re: Boot speedup with readahead

On Tue, Sep 9, 2008 at 10:36 AM, Seth Vidal <skvidal <at> fedoraproject.org> wrote:
>
> Does it need to be run if the pkg removed an icon?

Yeah, that would be more correct; but if we don't it just means that a
program will succeed in finding an icon it "shouldn't have", and I
can't offhand think of a reasonable case where that would be a
problem.

--

-- 
fedora-devel-list mailing list
fedora-devel-list <at> redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list

Arjan van de Ven | 9 Sep 16:09
Favicon

Re: Boot speedup with readahead

On Tue, 09 Sep 2008 09:51:37 -0400
Seth Vidal <skvidal <at> fedoraproject.org> wrote:

> On Mon, 2008-09-08 at 15:28 -0500, Callum Lerwick wrote:
> > On Mon, 2008-09-08 at 11:27 -0400, Jeremy Katz wrote:
> > > And doing it this way would be more effective than cron as there
> > > are plenty of people (especially with laptops/desktops which
> > > aren't on all the time) for whom cron really isn't appropriate
> > > for anything that we actually care to have done
> > 
> > We still need to trigger pre-linking after yum transactions rather
> > than by a cron job...
> 
> 
> So we really need a generic yum-post-transaction trigger script.
> 
> something that says:
> 
> pkgname: action: command
> 
> ie:
> 
> glibc: *: /run/something/for/a/glibc/
> 
> 
> How do we determine which pkgs changing need a new prelink run?

anything that puts something in /usr/bin or /usr/lib (and their
variations)
so would be nice if it would glob on package name OR package content.
(Continue reading)

Behdad Esfahbod | 9 Sep 19:51
Favicon
Gravatar

Re: Boot speedup with readahead

Seth Vidal wrote:
> On Mon, 2008-09-08 at 15:28 -0500, Callum Lerwick wrote:
>> On Mon, 2008-09-08 at 11:27 -0400, Jeremy Katz wrote:
>>> And doing it this way would be more effective than cron as there are
>>> plenty of people (especially with laptops/desktops which aren't on all
>>> the time) for whom cron really isn't appropriate for anything that we
>>> actually care to have done
>> We still need to trigger pre-linking after yum transactions rather than
>> by a cron job...
> 
> 
> So we really need a generic yum-post-transaction trigger script.
> 
> something that says:
> 
> pkgname: action: command
> 
> ie:
> 
> glibc: *: /run/something/for/a/glibc/
> 
> 
> How do we determine which pkgs changing need a new prelink run?

Shouldn't this be done on RPM level?  Isn't this pretty much an extension of
RPM triggers?  Or it's so hard to do that that we now want to move stuff to yum?

Similar issues that can be automatically handled also:

  - Run gtk-update-icon-cache if icons have been added/removed.
(Continue reading)

Seth Vidal | 9 Sep 19:57
Favicon

Re: Boot speedup with readahead

On Tue, 2008-09-09 at 13:51 -0400, Behdad Esfahbod wrote:
> Shouldn't this be done on RPM level?  Isn't this pretty much an extension of
> RPM triggers?  Or it's so hard to do that that we now want to move stuff to yum?

a proof of concept plugin for yum is almost finished. If we want to move
it to the rpm layer, fine. The rpm devs are busy and getting this added
to rpm will take some time. I don't really see the problem in working on
it here.

-sv

--

-- 
fedora-devel-list mailing list
fedora-devel-list <at> redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list

Seth Vidal | 9 Sep 23:33
Favicon

Re: Boot speedup with readahead

On Tue, 2008-09-09 at 13:57 -0400, Seth Vidal wrote:
> On Tue, 2008-09-09 at 13:51 -0400, Behdad Esfahbod wrote:
> > Shouldn't this be done on RPM level?  Isn't this pretty much an extension of
> > RPM triggers?  Or it's so hard to do that that we now want to move stuff to yum?
> 
> 
> a proof of concept plugin for yum is almost finished. If we want to move
> it to the rpm layer, fine. The rpm devs are busy and getting this added
> to rpm will take some time. I don't really see the problem in working on
> it here.
> 

Here's a proof of concept plugin:

http://skvidal.fedorapeople.org/misc/post-transaction-actions/

put the .py file in /usr/lib/yum-plugins
put the .conf file in /etc/yum/pluginconf.d/
make an /etc/yum/post-actions/ and put the .action file in there - add
more .action files as you'd like.

run yum transactions and see what happens...

-sv

--

-- 
fedora-devel-list mailing list
fedora-devel-list <at> redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list

(Continue reading)

Colin Walters | 10 Sep 20:14
Gravatar

Re: Boot speedup with readahead

On Tue, Sep 9, 2008 at 5:33 PM, Seth Vidal <skvidal <at> fedoraproject.org> wrote:
>
> Here's a proof of concept plugin:
>
> http://skvidal.fedorapeople.org/misc/post-transaction-actions/

Ok cool - with one small fix to the plugin, this simple action:

# Update the GTK+ icon cache when an icon is installed.
# See: http://www.gtk.org/api/2.6/gtk/gtk-update-icon-cache.html
/usr/share/icons/hicolor/*:any:gtk-update-icon-cache --quiet --force
/usr/share/icons/hicolor

seems to work.  Now we can apply hundreds (thousand?) of patches of
the following form, and probably in many cases entirely eliminate the
single %post and %postun we've been copying and pasting around.  It
will be awesome.

Index: hotssh.spec
===================================================================
RCS file: /cvs/pkgs/rpms/hotssh/devel/hotssh.spec,v
retrieving revision 1.1
diff -u -r1.1 hotssh.spec
--- hotssh.spec	4 Aug 2008 19:01:32 -0000	1.1
+++ hotssh.spec	10 Sep 2008 18:14:01 -0000
@@ -4,7 +4,7 @@
 Summary: Secure Shell Client
 Name: hotssh
 Version: 0.2.5
-Release: 1%{?dist}
(Continue reading)

Seth Vidal | 10 Sep 20:16
Favicon

Re: Boot speedup with readahead

On Wed, 2008-09-10 at 14:14 -0400, Colin Walters wrote:
> On Tue, Sep 9, 2008 at 5:33 PM, Seth Vidal <skvidal <at> fedoraproject.org> wrote:
> >
> > Here's a proof of concept plugin:
> >
> > http://skvidal.fedorapeople.org/misc/post-transaction-actions/
> 
> Ok cool - with one small fix to the plugin, this simple action:
> 

Thank you, I've applied this to the one I uploaded above.

So this seems to work - now the question is - should we push for this to
go into rpm instead?

-sv

--

-- 
fedora-devel-list mailing list
fedora-devel-list <at> redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list

Doug Ledford | 10 Sep 20:24
Favicon

Re: Boot speedup with readahead

On Wed, 2008-09-10 at 14:16 -0400, Seth Vidal wrote:
> On Wed, 2008-09-10 at 14:14 -0400, Colin Walters wrote:
> > On Tue, Sep 9, 2008 at 5:33 PM, Seth Vidal <skvidal <at> fedoraproject.org> wrote:
> > >
> > > Here's a proof of concept plugin:
> > >
> > > http://skvidal.fedorapeople.org/misc/post-transaction-actions/
> > 
> > Ok cool - with one small fix to the plugin, this simple action:
> > 
> 
> 
> Thank you, I've applied this to the one I uploaded above.
> 
> So this seems to work - now the question is - should we push for this to
> go into rpm instead?

Absolutely.  If it's going to exist at all, it needs to be in rpm.
Otherwise this and the corresponding rpm changes fail the litmus test of
"will my system be the same if I download the rpm(2) and run the
transaction manually using rpm as it is if I use yum to install the
rpm".  Any time you fail that litmus test, you've put the code in the
wrong place (at least IMO).

--

-- 
Doug Ledford <dledford <at> redhat.com>
              GPG KeyID: CFBFF194
              http://people.redhat.com/dledford

Infiniband specific RPMs available at
(Continue reading)

Seth Vidal | 10 Sep 20:42
Favicon

Re: Boot speedup with readahead

On Wed, 2008-09-10 at 14:24 -0400, Doug Ledford wrote:
> On Wed, 2008-09-10 at 14:16 -0400, Seth Vidal wrote:
> > On Wed, 2008-09-10 at 14:14 -0400, Colin Walters wrote:
> > > On Tue, Sep 9, 2008 at 5:33 PM, Seth Vidal <skvidal <at> fedoraproject.org> wrote:
> > > >
> > > > Here's a proof of concept plugin:
> > > >
> > > > http://skvidal.fedorapeople.org/misc/post-transaction-actions/
> > > 
> > > Ok cool - with one small fix to the plugin, this simple action:
> > > 
> > 
> > 
> > Thank you, I've applied this to the one I uploaded above.
> > 
> > So this seems to work - now the question is - should we push for this to
> > go into rpm instead?
> 
> Absolutely.  If it's going to exist at all, it needs to be in rpm.
> Otherwise this and the corresponding rpm changes fail the litmus test of
> "will my system be the same if I download the rpm(2) and run the
> transaction manually using rpm as it is if I use yum to install the
> rpm".  Any time you fail that litmus test, you've put the code in the
> wrong place (at least IMO).

unless we bite the bullet and stop falling back on being able to do
everything from the rpm interface.

And just move forward with doing things from the yum-cli and/or yum
modules.
(Continue reading)

Michael Stone | 10 Sep 22:16
Favicon

Re: Boot speedup with readahead

>And just move forward with doing things from the yum-cli and/or yum
>modules.
>
>Using the rpm interface for rescue/recovery functionality only.

Please remember that people use packaging tools for more than simple
system maintenance; for example, I use an image-builder which is now
based on smart (even though it was originally based on yum) because
smart is more convenient for some of my programmatic use cases [1].

Basically, diversity in this area is a Good Thing, in my humble opinion.
Please don't impair it without a really good reason when alternatives
exist (such as pushing this functionality into RPM).

Thanks,

Michael

[1]: In particular, because of its programmable mirror system, priority
system, support for alternate package formats, built-in ability to
download packages, builtin ability to save URLs for the packages it is
going to download, strictness options, etc.)

--

-- 
fedora-devel-list mailing list
fedora-devel-list <at> redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list

Jeff Spaleta | 10 Sep 22:19

Re: Boot speedup with readahead

On Wed, Sep 10, 2008 at 12:16 PM, Michael Stone <michael <at> laptop.org> wrote:
> [1]: In particular, because of its programmable mirror system, priority
> system, support for alternate package formats, built-in ability to
> download packages, builtin ability to save URLs for the packages it is
> going to download, strictness options, etc.)

Hey maybe we should push some of those features down the the rpm layer too.

-jef

--

-- 
fedora-devel-list mailing list
fedora-devel-list <at> redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list

Seth Vidal | 10 Sep 22:46
Favicon

Re: Boot speedup with readahead

On Wed, 2008-09-10 at 16:16 -0400, Michael Stone wrote:

> Please remember that people use packaging tools for more than simple
> system maintenance; for example, I use an image-builder which is now
> based on smart (even though it was originally based on yum) because
> smart is more convenient for some of my programmatic use cases [1].

I think I'd like to look at those use cases in more detail. If there's
anything that can be done in smart's interface that isn't doable in via
the yum python modules I'd be curious to know about it (excluding sudoku
b/c I just don't care)

> [1]: In particular,
> because of its programmable mirror system, 
- how much more programmable do you want than being able to modify the
mirrorlist directly per repo?

> priority system, 
available in a yum plugin - though questionably useful to begin with.

> support for alternate package formats,
not sure how this helps fedora.

> built-in ability to download packages,
yum can clearly download pkgs - not sure why it being built in rather
than a plugin matters though.

Also you can download directly if you're using the yum modules.

> builtin ability to save URLs for the packages it is going to download,
(Continue reading)

Michael Stone | 11 Sep 02:48
Favicon

Re: Boot speedup with readahead

On Wed, Sep 10, 2008 at 04:46:27PM -0400, Seth Vidal wrote:
>On Wed, 2008-09-10 at 16:16 -0400, Michael Stone wrote:
>
>> Please remember that people use packaging tools for more than simple
>> system maintenance; for example, I use an image-builder which is now
>> based on smart (even though it was originally based on yum) because
>> smart is more convenient for some of my programmatic use cases [1].
>
>I think I'd like to look at those use cases in more detail. 

I'm happy to explain in more detail, though I'll do so in my own order.

First, some introduction: my software [1] consists of a family of python
programs (called "compilations", since each is a primitive compiler), designed
to be run in a mock chroot, which convert collections of packages and hacks
into publishables (for example a list of installed packages, a XO JFFS2 disk
image, and a tarball of the filesystem contents). 

   [1]: http://wiki.laptop.org/go/Puritan

Its primary goal is to ease the task of creating releaseable disk images for
OLPC XOs in a reproducible and verifiable fashion by storing all effective
inputs to the build process in git commits. Its secondary goal is to overcome
some of the infelicities of pilgrim with respect to error detection, cleanup,
and debuggability. It's third goal is to be useful to expert Python programmers
who maintain Linux builds for fixed hardware regardless of their preferred
distro and package-format. It therefore exists in contrast to general-purpose
tools like debian-installer, anaconda, revisor, and livecd-tools, which are
distro-specific build tools with an interest in shielding their users from the
tool's implementation's underlying error-detection, analysis, and correction
(Continue reading)

Seth Vidal | 11 Sep 19:20
Favicon

Re: Boot speedup with readahead

On Wed, 2008-09-10 at 20:48 -0400, Michael Stone wrote:
> I'm happy to explain in more detail, though I'll do so in my own order.
> 
> First, some introduction: my software [1] consists of a family of python
> programs (called "compilations", since each is a primitive compiler), designed
> to be run in a mock chroot, which convert collections of packages and hacks
> into publishables (for example a list of installed packages, a XO JFFS2 disk
> image, and a tarball of the filesystem contents). 
> 
>    [1]: http://wiki.laptop.org/go/Puritan
> 
> Its primary goal is to ease the task of creating releaseable disk images for
> OLPC XOs in a reproducible and verifiable fashion by storing all effective
> inputs to the build process in git commits. Its secondary goal is to overcome
> some of the infelicities of pilgrim with respect to error detection, cleanup,
> and debuggability. It's third goal is to be useful to expert Python programmers
> who maintain Linux builds for fixed hardware regardless of their preferred
> distro and package-format. It therefore exists in contrast to general-purpose
> tools like debian-installer, anaconda, revisor, and livecd-tools, which are
> distro-specific build tools with an interest in shielding their users from the
> tool's implementation's underlying error-detection, analysis, and correction
> mechanisms and workflows, often by means of a domain-specific language.
> 
> Smart is convenient to these goals in that it has a thoroughly documented shell
> interface which permits the following useful operations:
> 
> Smart permits me to control my selection of package repositories package more
> succinctly and less distro-specifically than yum; e.g.
> 
>    smart channel -y --remove-all
(Continue reading)

Re: Boot speedup with readahead

On Wednesday, 10 September 2008 at 20:42, Seth Vidal wrote:
> On Wed, 2008-09-10 at 14:24 -0400, Doug Ledford wrote:
> > On Wed, 2008-09-10 at 14:16 -0400, Seth Vidal wrote:
> > > On Wed, 2008-09-10 at 14:14 -0400, Colin Walters wrote:
> > > > On Tue, Sep 9, 2008 at 5:33 PM, Seth Vidal <skvidal <at> fedoraproject.org> wrote:
> > > > >
> > > > > Here's a proof of concept plugin:
> > > > >
> > > > > http://skvidal.fedorapeople.org/misc/post-transaction-actions/
> > > > 
> > > > Ok cool - with one small fix to the plugin, this simple action:
> > > > 
> > > 
> > > 
> > > Thank you, I've applied this to the one I uploaded above.
> > > 
> > > So this seems to work - now the question is - should we push for this to
> > > go into rpm instead?
> > 
> > Absolutely.  If it's going to exist at all, it needs to be in rpm.
> > Otherwise this and the corresponding rpm changes fail the litmus test of
> > "will my system be the same if I download the rpm(2) and run the
> > transaction manually using rpm as it is if I use yum to install the
> > rpm".  Any time you fail that litmus test, you've put the code in the
> > wrong place (at least IMO).
> 
> unless we bite the bullet and stop falling back on being able to do
> everything from the rpm interface.

-1
(Continue reading)

Rahul Sundaram | 11 Sep 08:41
Favicon

Re: Boot speedup with readahead

Dominik 'Rathann' Mierzejewski wrote:
> On Wednesday, 10 September 2008 at 20:42, Seth Vidal wrote:

>> unless we bite the bullet and stop falling back on being able to do
>> everything from the rpm interface.
> 
> -1
> 
> Fedora doesn't need to go where Suse already is.

Instead of voting, it would be better, if you can explain why it is 
wrong in your opinion.  +1/-1 doesn't really convey much useful 
information to assign any weight in a debate.

Rahul

--

-- 
fedora-devel-list mailing list
fedora-devel-list <at> redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list

Nicolas Mailhot | 11 Sep 10:44
Favicon

Re: Boot speedup with readahead


Le Jeu 11 septembre 2008 08:41, Rahul Sundaram a écrit :
>
> Dominik 'Rathann' Mierzejewski wrote:
>> On Wednesday, 10 September 2008 at 20:42, Seth Vidal wrote:
>
>>> unless we bite the bullet and stop falling back on being able to do
>>> everything from the rpm interface.
>>
>> -1
>>
>> Fedora doesn't need to go where Suse already is.
>
> Instead of voting, it would be better, if you can explain why it is
> wrong in your opinion.  +1/-1 doesn't really convey much useful
> information to assign any weight in a debate.

Yum has progressed a lot in past years but it is not significantly
better than competitors (as evidenced by the fact rpm distros have not
adopted in in mass) and thus entrenching yum by relying on
yum-specific features feels to me rather premature. Competition is
healthy.

Plus rpm.org has been resurected so there's no compelling reason not
to do stuff at this level.

-- 
Nicolas Mailhot

--

-- 
(Continue reading)

Re: Boot speedup with readahead

On Thursday, 11 September 2008 at 08:41, Rahul Sundaram wrote:
> Dominik 'Rathann' Mierzejewski wrote:
> >On Wednesday, 10 September 2008 at 20:42, Seth Vidal wrote:
> 
> >>unless we bite the bullet and stop falling back on being able to do
> >>everything from the rpm interface.
> >
> >-1
> >
> >Fedora doesn't need to go where Suse already is.
> 
> Instead of voting, it would be better, if you can explain why it is 
> wrong in your opinion.  +1/-1 doesn't really convey much useful 
> information to assign any weight in a debate.

I believe it is the one who proposes the change that has to explain
why it is necessary and can't be done in any other way. It has already
been said why it is wrong in this thread: people still use rpm command
to install packages. Is there a very good reason to force them to use
yum instead? Yum is still slower than plain rpm. After all, it is
a frontend and I do not think forcing people to use frontends is a good
idea when the tool behind the frontend can do the job just as well.

Regards,
R.

--

-- 
Fedora http://fedoraproject.org/wiki/User:Rathann
Livna http://rpm.livna.org | MPlayer http://mplayerhq.hu
"Faith manages."
(Continue reading)

Panu Matilainen | 11 Sep 10:34

Re: Boot speedup with readahead

On Wed, 10 Sep 2008, Seth Vidal wrote:

> On Wed, 2008-09-10 at 14:14 -0400, Colin Walters wrote:
>> On Tue, Sep 9, 2008 at 5:33 PM, Seth Vidal <skvidal <at> fedoraproject.org> wrote:
>>>
>>> Here's a proof of concept plugin:
>>>
>>> http://skvidal.fedorapeople.org/misc/post-transaction-actions/
>>
>> Ok cool - with one small fix to the plugin, this simple action:
>
> Thank you, I've applied this to the one I uploaded above.
>
> So this seems to work - now the question is - should we push for this to
> go into rpm instead?

Not instead, but to rpm *too*, yes.

Quite obviously an action that's currently done from say %post script 
should stay at rpm level, moving some things to be only done once per 
transaction is just an optimization really. But yum knows a lot more about 
the surrounding world than rpm does, such as which repository a package 
came from which is something rpm has no clue about, and thus it can do 
things in post-transaction that rpm simply could not do. I don't have an 
example use-case in mind atm but I'm sure somebody can come up with some 
relatively sane example ;)

 	- Panu -

--

-- 
(Continue reading)

Re: Boot speedup with readahead

On Thursday, 11 September 2008 at 10:34, Panu Matilainen wrote:
> On Wed, 10 Sep 2008, Seth Vidal wrote:
> 
> >On Wed, 2008-09-10 at 14:14 -0400, Colin Walters wrote:
> >>On Tue, Sep 9, 2008 at 5:33 PM, Seth Vidal <skvidal <at> fedoraproject.org> 
> >>wrote:
> >>>
> >>>Here's a proof of concept plugin:
> >>>
> >>>