Technical discussion about the development of the HotSpot virtual machine ()

Kurt Miller | 15 Mar 2007 16:44

precompiled headers and includeDB*

Hi,

The use of precompiled headers obscures problems with
the includeDB* files and they have become out of date.
Building on linux without defining USE_PRECOMPILED_HEADER
in build/linux/makefiles/gcc.make reveals the problems
with the includeDB* files.

I've taken a stab at updating the includeDB files but
I'm not fully sure I've got it all right so far. compiler2
completes ok, but compiler1 bails like this:

make[4]: Entering directory `/home/truk/openjdk/hotspot/build/linux/linux_i486_compiler1/product'
make[4]: *** No rule to make target `node.hpp', needed by `compactingPermGenGen.o'. Stop.

I could use some guidance at this point. Do the patches
below look correct so far?

Thanks,
-Kurt

--- src/share/vm/includeDB_core.orig	Fri Mar  9 09:52:24 2007
+++ src/share/vm/includeDB_core	Sat Mar 10 19:43:08 2007
 <at>  <at>  -429,7 +429,7  <at>  <at>  cardTableModRefBS.cpp			cardTableRS.hpp
 cardTableModRefBS.cpp			java.hpp
 cardTableModRefBS.cpp			mutexLocker.hpp
 cardTableModRefBS.cpp			sharedHeap.hpp
-cardTableModRefBS.cpp			space.hpp
+cardTableModRefBS.cpp			space.inline.hpp
 cardTableModRefBS.cpp			virtualspace.hpp

(Continue reading)

Kurt Miller | 15 Mar 2007 23:04

Re: precompiled headers and includeDB*

On Thursday 15 March 2007 11:44:45 am Kurt Miller wrote:
> Hi,
> 
> The use of precompiled headers obscures problems with
> the includeDB* files and they have become out of date.
> Building on linux without defining USE_PRECOMPILED_HEADER
> in build/linux/makefiles/gcc.make reveals the problems
> with the includeDB* files.

Ok I see where I went wrong now. Here is the full
update for consideration.

--- src/share/vm/includeDB_core.orig	Thu Mar  1 08:58:41 2007
+++ src/share/vm/includeDB_core	Thu Mar 15 15:02:21 2007
 <at>  <at>  -429,7 +429,7  <at>  <at>  cardTableModRefBS.cpp			cardTableRS.hpp
 cardTableModRefBS.cpp			java.hpp
 cardTableModRefBS.cpp			mutexLocker.hpp
 cardTableModRefBS.cpp			sharedHeap.hpp
-cardTableModRefBS.cpp			space.hpp
+cardTableModRefBS.cpp			space.inline.hpp
 cardTableModRefBS.cpp			virtualspace.hpp
 cardTableModRefBS.cpp			universe.hpp

 <at>  <at>  -706,7 +706,6  <at>  <at>  compactPermGen.hpp			permGen.hpp

 compactingPermGenGen.cpp                compactingPermGenGen.hpp
 compactingPermGenGen.cpp                filemap.hpp
-compactingPermGenGen.cpp                generation.inline.hpp
 compactingPermGenGen.cpp                generationSpec.hpp
 compactingPermGenGen.cpp                genOopClosures.inline.hpp

(Continue reading)

tom fogal | 15 Mar 2007 17:07

Re: precompiled headers and includeDB*

 <200703151144.45721.lists@...>Kurt Miller writes:
> The use of precompiled headers obscures problems with
> the includeDB* files and they have become out of date.
[snip]
> 
> I could use some guidance at this point. Do the patches
> below look correct so far?
[snip]

While we're on the subject, is there a compelling reason for the
includeDB system in general?  I guess I don't really understand that
aspect of the build system very well.  It exists because of a potential
performance gain in build times through the use of precompiled headers,
correct?

As far as I know, the system uses these includeDB files and MakeDep to
somehow build a single giant include file, which gets precompiled.
Then all files end up including this precompiled header.

This seems like a lot of trouble to go through however, and I'm not
convinced of the performance gain.  In this scheme, compilation of
file.C won't have to parse its include of file.h, but it still has to
read 1 giant precompiled header file from disk.  It would seem that
reading + parsing a small number of includes wouldn't be that much
slower (or maybe even faster) then just a read of one large precompiled
header.  Unless most headers really need to get included in a
significant portion of the JDK, which seems unlikely but possible.

So, making the unreasonable assumption that I understand things
correctly, then dependencies && includes could be done in a more

(Continue reading)

steve goldman | 15 Mar 2007 19:12

Picon

Re: precompiled headers and includeDB*

tom fogal wrote:
> 
> While we're on the subject, is there a compelling reason for the
> includeDB system in general?  I guess I don't really understand that
> aspect of the build system very well.  It exists because of a potential
> performance gain in build times through the use of precompiled headers,
> correct?

No that is not the reason it exists at all. The original reason it 
existed was because the original developers of hotspot intended to 
license different parts of the source base as different products. 
includeDB was the database used to manage which pieces were sent with 
which products.

Nowadays it generates the project files for Visual Studio and will 
likely generate the project files for NetBeans in addition to generating 
the dependencies for make.

> 
> As far as I know, the system uses these includeDB files and MakeDep to
> somehow build a single giant include file, which gets precompiled.
> Then all files end up including this precompiled header.
> 
> This seems like a lot of trouble to go through however, and I'm not
> convinced of the performance gain.  In this scheme, compilation of
> file.C won't have to parse its include of file.h, but it still has to
> read 1 giant precompiled header file from disk.  It would seem that
> reading + parsing a small number of includes wouldn't be that much
> slower (or maybe even faster) then just a read of one large precompiled
> header.  Unless most headers really need to get included in a

(Continue reading)

John Rose | 15 Mar 2007 19:35

Picon

Re: precompiled headers and includeDB*

On Mar 15, 2007, at 11:12 AM, steve goldman wrote:

> No that is not the reason it exists at all. The original reason it  
> existed was because the original developers of hotspot intended to  
> license different parts of the source base as different products.  
> includeDB was the database used to manage which pieces were sent  
> with which products.
...
> I think the consensus view is that if we didn't have MakeDeps we  
> wouldn't go out and invent it.

True.  There are other ways to manage this information, some of which  
would be more pleasant or familiar to some people.
But  in the big picture there are many more profitable changes needed.

It's probably a matter of personal preference, but I and others enjoy  
managing the inter-file dependencies in a centralized way.
And (and Ken Russell and others said last time the topic came up)  
checking the dependencies for circularity up front is handy.
This is the more valuable because there are "build flavors" of VM  
(C1, C2).

A final bit of consensus:  This is only a minor part of the learning  
curve for working on Hotspot.

-- John

Florian Weimer | 15 Mar 2007 17:50

Picon

Re: precompiled headers and includeDB*

* tom fogal:

> This seems like a lot of trouble to go through however, and I'm not
> convinced of the performance gain.  In this scheme, compilation of
> file.C won't have to parse its include of file.h, but it still has to
> read 1 giant precompiled header file from disk.

The file is not read, but mapped into the process address space AFAIK.

Keith McGuigan | 15 Mar 2007 17:53

Picon

Re: precompiled headers and includeDB*

Hi Tom,

tom fogal wrote:
> parsing a small number of includes wouldn't be that much
> slower (or maybe even faster) then just a read of one large precompiled
> header.  Unless most headers really need to get included in a
> significant portion of the JDK, which seems unlikely but possible.

It actually is the case that many, many headers get included in each source 
file.  Some are worse than others, of course, and this is aggravated by the fact 
that many headers files require other header files.

I believe that we could still get the precompiled header support without 
MakeDeps/includeDB (i.e, tradition includes), but there's a surprising amount of 
fondness for this system at this point.

--
- Keith

Ivan Krylov | 15 Mar 2007 17:23

Picon

Re: precompiled headers and includeDB*

Hi Tom,

Using precompiled headers does help a lot to reduce hotspot compilation 
time.
The can be clearly observed if you compare with compilation time using 
Sun Studio that we use for Solaris builds.
In the mean time we continue to convince sun studio folks to improve 
precompiled header support. The way it is done now it will not help much 
hotspot compilation.

Now to you points;
Looking at http://gcc.gnu.org/onlinedocs/gcc/Precompiled-Headers.html
there is a restriction "Only one precompiled header can be used in a 
particular compilation"
So we can't have many small precompiled files

Regards,

Ivan.

tom fogal wrote:
>  <200703151144.45721.lists@...>Kurt Miller writes:
>> The use of precompiled headers obscures problems with
>> the includeDB* files and they have become out of date.
> [snip]
>> I could use some guidance at this point. Do the patches
>> below look correct so far?
> [snip]
> 
> While we're on the subject, is there a compelling reason for the

(Continue reading)

Return

Return to gmane.comp.java.openjdk.hotspot.devel.

Project Web Page

Technical discussion about the development of the HotSpot virtual machine ()

Search Archive

Language

Change language

Options

Current view: Threads only / Showing only 20 lines / Not hiding cited text.
Change to All messages, whole messages, or hide cited text.

Post a message
NNTP Newsgroup
Classic Gmane web interface
XML

XML

RSS Feed
List Information

About Gmane