Chris Molozian | 28 May 2011 14:43

A few queries about Jam

Hey all,

I hope the mailing list is still active. I've been through all the documentation for Jam and peeked at the Jambase file in the source repository but there's still a couple of things I can't find the idiomatic way to do.

My build consists of using a D compiler to build a project I'm working on involving the LLVM. The Jamfile I've created looks like this:

DCFLAGS = -fPIC -O -inline -release -w -wi -I./imports/llvm-2.9 ;
C++FLAGS =
    -I/usr/include
    -DNDEBUG
    -D_GNU_SOURCE
    -D__STDC_LIMIT_MACROS
    -D__STDC_CONSTANT_MACROS
    -O3
    -fomit-frame-pointer
    -fno-exceptions
    -fPIC
    -Woverloaded-virtual
    -Wcast-qual ;   # should be generated from `llvm-config --cxxflags`
LINK = $(DC) ;
DFILES =
    src/ob2c/main.d
    src/ob2c/info.d ;    # should be scanned from src dir

# override the Link action to correct for the dmd compiler -o flag
actions Link bind NEEDLIBS {
    $(LINK) $(LINKFLAGS) -of$(<) $(UNDEFS) $(>) $(NEEDLIBS) $(LINKLIBS)
}

MainFromObjects ob2c : $(DFILES:S=.o) ;
Objects $(DFILES) ;
LinkLibraries ob2c : llvm-c-ext ;
Library llvm-c-ext :
    imports/llvm-2.9/llvm/Ext.cpp
    imports/llvm-2.9/llvm/Target.cpp ;

The things I'm not sure how to do are:
  • Can I call an external program (i.e. llvm-config) and store the stdout from it to the C++FLAGS variable?
  • All of my source code is within a src folder, can I tell Jam to scan the directory and subdirectories for source files and store them to the DFILES variable?
  • Is there a way for me to force all target output (e.g. object files, generated libraries, binaries... etc) to be put within a build folder at the same directory level as the Jamfile?
Hope this makes sense, any help and advice would be most appreciated.

Cheers,

Chris

_______________________________________________
jamming mailing list  -  jamming <at> maillist.perforce.com
http://maillist.perforce.com/mailman/listinfo/jamming
Alen Ladavac | 29 May 2011 13:18

Re: A few queries about Jam

Chris wrote at 2011-05-28:
> I hope the mailing list is still active.
Yes. Just that the damn thing is so stable there's nothing interesting to chatter about. ;)

> °       Can I call an external program (i.e. llvm-config) and store the stdout from it to the C++FLAGS variable?
Not in vanilla jam. But IIRC there is a custom patch for that somewhere. Or at least one of the forks that can do
it (BJam, JamPlus, ...). Don't know the details from the top of my head.

> °       All of my source code is within a src folder, can I tell Jam to scan the directory and subdirectories for
source files and store them to the DFILES variable?
Yes. There is this GLOB rule that scans files in a folder. It doesn't handle subdirectories, but you can
write a recursive rule based on it that does. Just watch out for . and .. , as usual.

> °       Is there a way for me to force all target output (e.g. object files, generated libraries, binaries...
etc) to be put within a build folder at the same directory level as the Jamfile?
You can reconfigure everything, even rewrite your own Jambase from scratch. Depends on how compatible
with stock solution you want to be.

HTH,
Alen

_______________________________________________
jamming mailing list  -  jamming <at> maillist.perforce.com
http://maillist.perforce.com/mailman/listinfo/jamming

Chris Molozian | 29 May 2011 18:01

Re: A few queries about Jam

Dear Alen,

Thanks for the suggestions, I agree the stableness of the tool and it's cross-platform support are wonderful. I think for the time being I'll hard-code the llvm-config flags, I thought about directing the output to a file and reading that in to the C++FLAGS variable but wasn't sure how to do that either.

Regarding redirecting the target output to a folder, I want to stay as compatible as possible with the stock Jambase file. The less I have to modify the better. I was hoping there might be a simple trick to it I'd missed, ah well.

Would you happen to have an example of the GLOB rule that I could try to understand. I found the reference rather unclear:

Glob directories : patterns ; builtin

Scans directories for files matching regex patterns, returning the list of matching files with directory prepended. Useful with the [...] construct to change the Glob result into a list.

I don't understand the [...] notation and where I'm supposed to use it, or how to recursively call itself to descend directories.

Kind Regards,

Chris


On 29/05/11 12:18, Alen Ladavac wrote:
Chris wrote at 2011-05-28:
I hope the mailing list is still active.
Yes. Just that the damn thing is so stable there's nothing interesting to chatter about. ;)
° Can I call an external program (i.e. llvm-config) and store the stdout from it to the C++FLAGS variable?
Not in vanilla jam. But IIRC there is a custom patch for that somewhere. Or at least one of the forks that can do it (BJam, JamPlus, ...). Don't know the details from the top of my head.
° All of my source code is within a src folder, can I tell Jam to scan the directory and subdirectories for source files and store them to the DFILES variable?
Yes. There is this GLOB rule that scans files in a folder. It doesn't handle subdirectories, but you can write a recursive rule based on it that does. Just watch out for . and .. , as usual.
° Is there a way for me to force all target output (e.g. object files, generated libraries, binaries... etc) to be put within a build folder at the same directory level as the Jamfile?
You can reconfigure everything, even rewrite your own Jambase from scratch. Depends on how compatible with stock solution you want to be. HTH, Alen
_______________________________________________
jamming mailing list  -  jamming <at> maillist.perforce.com
http://maillist.perforce.com/mailman/listinfo/jamming
Alen Ladavac | 30 May 2011 00:00

Re: A few queries about Jam

One idea for your flags is to execute it on the command line that calls jam, and pass it through its input using
-s cmdline option. A bit clumsy, but might work. 

You can read from a file, if the file looks like:

C++FLAGS = <something> ;

Then you just Include the file. Been using that trick for some things and it works well. :)

As for glob usage, it is something like:

sourcefiles = [ GLOB $(SomeFolder) : *.cpp *.c ] ;

Here is also a RGLOB rule I wrote a while ago. It is clumsy and suboptimal, but works on stock jam. (ListDir is
just a helper rule.)

# list all files in a directory, except ., .. 
# [ ListDir dirname ]
rule ListDir {

  # start with empty list
  local _result =  ;

  # for each file in the directory  
  local _dirlist = [ GLOB $(1) : * ] ;
  for _subdir in $(_dirlist) {

    # if it is not . or ..
    switch $(_subdir) {
        case */.    : _dummy = "" ; # is there some no-op statement?
        case */..   : _dummy = "" ; # is there some no-op statement?
        case * : 
          # add it to the list
          _result += $(_subdir) ;
    }
  }

  # return resulting list
  return $(_result) ;
}

# same as glob, but recurses into subdirs
rule RGLOB {

  # initially use the files in the current directory
  local _files = [ GLOB $(1) : $(2) ] ;

  # filter so that only real files are included (skip current dir, parent dir and all other dirs)
  local _f = ;
  local _result = ;
  for _f in $(_files) {
    if $(_f:BS) != . && $(_f:BS) != .. && ! [ GLOB $(_f) : * ] {
      _result += $(_f) ;
    }
  }

  # list all subdirectories (and files, but it doesn't hurt)
  local _subdirlist = [ ListDir $(1) ] ;

  # for each subdir/file
  for _subdir in $(_subdirlist) {
    # recurse into it
    _result += [ RGLOB $(_subdir) : $(2) ] ;
  }

  # return the resulting list
  return $(_result) ;
}

Sunday, May 29, 2011, 6:01:02 PM, you wrote:

Dear Alen,

Thanks for the suggestions, I agree the stableness of the tool and it's cross-platform support are
wonderful. I think for the time being I'll hard-code the llvm-config flags, I thought about directing the
output to a file and reading that in to the C++FLAGS variable but wasn't sure how to do that either.

Regarding redirecting the target output to a folder, I want to stay as compatible as possible with the stock
Jambase file. The less I have to modify the better. I was hoping there might be a simple trick to it I'd
missed, ah well.

Would you happen to have an example of the GLOB rule that I could try to understand. I found the reference
rather unclear:
Glob directories : patterns ; builtin
Scans directories for files matching regex patterns, returning the list of matching files with directory
prepended. Useful with the [...] construct to change the Glob result into a list.
I don't understand the [...] notation and where I'm supposed to use it, or how to recursively call itself to
descend directories.

Kind Regards,

Chris

On 29/05/11 12:18, Alen Ladavac wrote: 
Chris wrote at 2011-05-28:

I hope the mailing list is still active.

Yes. Just that the damn thing is so stable there's nothing interesting to chatter about. ;)

°       Can I call an external program (i.e. llvm-config) and store the stdout from it to the C++FLAGS variable?

Not in vanilla jam. But IIRC there is a custom patch for that somewhere. Or at least one of the forks that can do
it (BJam, JamPlus, ...). Don't know the details from the top of my head.

°       All of my source code is within a src folder, can I tell Jam to scan the directory and subdirectories for
source files and store them to the DFILES variable?

Yes. There is this GLOB rule that scans files in a folder. It doesn't handle subdirectories, but you can
write a recursive rule based on it that does. Just watch out for . and .. , as usual.

°       Is there a way for me to force all target output (e.g. object files, generated libraries, binaries...
etc) to be put within a build folder at the same directory level as the Jamfile?

You can reconfigure everything, even rewrite your own Jambase from scratch. Depends on how compatible
with stock solution you want to be.

HTH,

Alen

--

-- 
Best regards,
 Alen                            mailto:alenl-ml <at> croteam.com

_______________________________________________
jamming mailing list  -  jamming <at> maillist.perforce.com
http://maillist.perforce.com/mailman/listinfo/jamming

Chris Molozian | 31 May 2011 14:02

Re: A few queries about Jam

Thank you to Alen and John for the great help. I'm using this:

DFILES = [ GLOB src/ob2c src/ob2c/gen : *.d ] ;

to handle gathering up the source files. I decided to keep the build 
script nice and short and avoid defining RGLOB, just listing the 
directories for GLOB is fine by me.

It's a good idea about using an external script (.bat, .sh) to feed jam 
the correct C++FLAGS. I'll probably implement that eventually when I do 
serious testing on windows.

As suggested I've tried:

MkDir build ;
ALL_LOCATE_TARGET = build ;

to relocate all build output to a dedicated build folder but it appears 
to do nothing. The object files are still built alongside source files 
and the libraries and executable build in the same directory as the 
Jamfile. I'm not sure what I'm doing wrong.

On 29/05/11 23:00, Alen Ladavac wrote:
> One idea for your flags is to execute it on the command line that calls jam, and pass it through its input
using -s cmdline option. A bit clumsy, but might work.
>
> You can read from a file, if the file looks like:
>
> C++FLAGS =<something>  ;
>
> Then you just Include the file. Been using that trick for some things and it works well. :)
>
> As for glob usage, it is something like:
>
> sourcefiles = [ GLOB $(SomeFolder) : *.cpp *.c ] ;
>
> Here is also a RGLOB rule I wrote a while ago. It is clumsy and suboptimal, but works on stock jam. (ListDir is
just a helper rule.)
>
> # list all files in a directory, except ., ..
> # [ ListDir dirname ]
> rule ListDir {
>
>    # start with empty list
>    local _result =  ;
>
>    # for each file in the directory
>    local _dirlist = [ GLOB $(1) : * ] ;
>    for _subdir in $(_dirlist) {
>
>      # if it is not . or ..
>      switch $(_subdir) {
>          case */.    : _dummy = "" ; # is there some no-op statement?
>          case */..   : _dummy = "" ; # is there some no-op statement?
>          case * :
>            # add it to the list
>            _result += $(_subdir) ;
>      }
>    }
>
>    # return resulting list
>    return $(_result) ;
> }
>
> # same as glob, but recurses into subdirs
> rule RGLOB {
>
>    # initially use the files in the current directory
>    local _files = [ GLOB $(1) : $(2) ] ;
>
>    # filter so that only real files are included (skip current dir, parent dir and all other dirs)
>    local _f = ;
>    local _result = ;
>    for _f in $(_files) {
>      if $(_f:BS) != .&&  $(_f:BS) != ..&&  ! [ GLOB $(_f) : * ] {
>        _result += $(_f) ;
>      }
>    }
>
>
>    # list all subdirectories (and files, but it doesn't hurt)
>    local _subdirlist = [ ListDir $(1) ] ;
>
>    # for each subdir/file
>    for _subdir in $(_subdirlist) {
>      # recurse into it
>      _result += [ RGLOB $(_subdir) : $(2) ] ;
>    }
>
>    # return the resulting list
>    return $(_result) ;
> }
>
>
>
> Sunday, May 29, 2011, 6:01:02 PM, you wrote:
>
>
> Dear Alen,
>
> Thanks for the suggestions, I agree the stableness of the tool and it's cross-platform support are
wonderful. I think for the time being I'll hard-code the llvm-config flags, I thought about directing the
output to a file and reading that in to the C++FLAGS variable but wasn't sure how to do that either.
>
> Regarding redirecting the target output to a folder, I want to stay as compatible as possible with the
stock Jambase file. The less I have to modify the better. I was hoping there might be a simple trick to it I'd
missed, ah well.
>
> Would you happen to have an example of the GLOB rule that I could try to understand. I found the reference
rather unclear:
> Glob directories : patterns ; builtin
> Scans directories for files matching regex patterns, returning the list of matching files with
directory prepended. Useful with the [...] construct to change the Glob result into a list.
> I don't understand the [...] notation and where I'm supposed to use it, or how to recursively call itself to
descend directories.
>
> Kind Regards,
>
> Chris
>
>
> On 29/05/11 12:18, Alen Ladavac wrote:
> Chris wrote at 2011-05-28:
>
>
> I hope the mailing list is still active.
>
>
> Yes. Just that the damn thing is so stable there's nothing interesting to chatter about. ;)
>
>
>
>
> °       Can I call an external program (i.e. llvm-config) and store the stdout from it to the C++FLAGS variable?
>
>
> Not in vanilla jam. But IIRC there is a custom patch for that somewhere. Or at least one of the forks that can
do it (BJam, JamPlus, ...). Don't know the details from the top of my head.
>
>
>
>
> °       All of my source code is within a src folder, can I tell Jam to scan the directory and subdirectories for
source files and store them to the DFILES variable?
>
>
> Yes. There is this GLOB rule that scans files in a folder. It doesn't handle subdirectories, but you can
write a recursive rule based on it that does. Just watch out for . and .. , as usual.
>
>
>
>
> °       Is there a way for me to force all target output (e.g. object files, generated libraries, binaries...
etc) to be put within a build folder at the same directory level as the Jamfile?
>
>
> You can reconfigure everything, even rewrite your own Jambase from scratch. Depends on how compatible
with stock solution you want to be.
>
>
>
> HTH,
>
> Alen
>
>
>
>
>
>
>
>
_______________________________________________
jamming mailing list  -  jamming <at> maillist.perforce.com
http://maillist.perforce.com/mailman/listinfo/jamming

John Waugh | 1 Jun 2011 21:09
Picon

Re: A few queries about Jam

On Tue, May 31, 2011 at 5:02 AM, Chris Molozian <chris <at> cmoz.me> wrote:
>
> As suggested I've tried:
>
> MkDir build ;
> ALL_LOCATE_TARGET = build ;
>
> to relocate all build output to a dedicated build folder but it appears to do nothing. The object files are
still built alongside source files and the libraries and executable build in the same directory as the
Jamfile. I'm not sure what I'm doing wrong.

Hi Chris,

I haven't used ALL_LOCATE_TARGET that much myself, but looking at
Jambase, it seems to be used by the SubDir rule. Therefore, be sure to
set it before you call SubDir.

Or perhaps you are not using SubDir at all? That could explain it, too.

-John
_______________________________________________
jamming mailing list  -  jamming <at> maillist.perforce.com
http://maillist.perforce.com/mailman/listinfo/jamming

John Waugh | 1 Jun 2011 23:11
Picon

Re: A few queries about Jam

> On Tue, May 31, 2011 at 5:02 AM, Chris Molozian <chris <at> cmoz.me> wrote:
>
> As suggested I've tried:
>
> MkDir build ;
> ALL_LOCATE_TARGET = build ;
>
> to relocate all build output to a dedicated build folder but it appears to do nothing. The object files are
still built alongside source files and the libraries and executable build in the same directory as the
Jamfile. I'm not sure what I'm doing wrong.

I forgot to mention: you can always set LOCATE on targets yourself, of course.
In fact, if you look in Jambase at how ALL_LOCATE_TARGET is used (it's
just 2 lines), it just sets the default LOCATE path.

LOCATE itself is a jam 'special' variable, which is used to bind a
target to a physical path on the file system.
c.f. http://public.perforce.com/public/jam/src/Jam.html#search

(apologies if you already knew this)

-John
_______________________________________________
jamming mailing list  -  jamming <at> maillist.perforce.com
http://maillist.perforce.com/mailman/listinfo/jamming

Diane Holt | 1 Jun 2011 23:46
Picon

Re: A few queries about Jam

SubDir sets:

LOCATE_TARGET = $(ALL_LOCATE_TARGET) $(SUBDIR) ;

So if ALL_LOCATE_TARGET isn't set, it defaults to SUBDIR, which is guaranteed to be set (since SubDir sets it).

Then the Object rules uses MakeLocate (which is much nicer than "LOCATE on", since it not only sets the output location, it mkdir's it for you, if needed) to set the location to $(LOCATE_TARGET).

If you're not using SubDir, I'd suggest you do. If the default rules/actions in Jambase don't suit your needs, you can always override them with your own (of the same name) -- last one in, wins. But if you can use the defaults, it just makes things easier for you -- that much less to write and maintain.

Diane

On Wed, Jun 1, 2011 at 2:11 PM, John Waugh <john.waugh <at> gmail.com> wrote:
> On Tue, May 31, 2011 at 5:02 AM, Chris Molozian <chris <at> cmoz.me> wrote:
>
> As suggested I've tried:
>
> MkDir build ;
> ALL_LOCATE_TARGET = build ;
>
> to relocate all build output to a dedicated build folder but it appears to do nothing. The object files are still built alongside source files and the libraries and executable build in the same directory as the Jamfile. I'm not sure what I'm doing wrong.

I forgot to mention: you can always set LOCATE on targets yourself, of course.
In fact, if you look in Jambase at how ALL_LOCATE_TARGET is used (it's
just 2 lines), it just sets the default LOCATE path.

LOCATE itself is a jam 'special' variable, which is used to bind a
target to a physical path on the file system.
c.f. http://public.perforce.com/public/jam/src/Jam.html#search

(apologies if you already knew this)

-John
_______________________________________________
jamming mailing list  -  jamming <at> maillist.perforce.com
http://maillist.perforce.com/mailman/listinfo/jamming

_______________________________________________
jamming mailing list  -  jamming <at> maillist.perforce.com
http://maillist.perforce.com/mailman/listinfo/jamming
Chris Molozian | 3 Jun 2011 18:57

Re: A few queries about Jam

Dear John,

Thank you very much for your help so far, please no need to apologise for detailing anything about the inner workings of Jam. I'm still very much a novice with this tool but getting very close to the perfect build setup for my (soon to be) open source project. This setup would complete item 2 in the Joel Test ;-) .

I setup homebrew on my brother's Apple Mac OS X and used it to install the necessary LLVM, D Compiler and Jam tools to test the build environment. The Jam build tool in the homebrew repository sets up the Perforce version of Jam. I added the additional rules for building D files to my Jamfile and all is working perfectly. The build script so far is:

# auxiliary rules to support building D code with perforce Jam
rule UserObject {
    switch $(>:S) {
     case .d : Dc $(<) : $(>) ;
     case * :
        Exit "Unknown suffix on " $(>) " - see UserObject rule in Jambase." ;
    }
}

rule Dc {
    Depends $(<) : $(>) ;
    DCFLAGS on $(<) += $(DCFLAGS) $(SUBDIRDCFLAGS) ;
}

actions Dc {
    $(DC) -c -of$(<) $(DCFLAGS) $(DOPTIM) $(>)
}

# override the Link action to correct for the dmd compiler -o flag
actions Link bind NEEDLIBS {
    $(LINK) $(LINKFLAGS) -of$(<) $(UNDEFS) $(>) $(NEEDLIBS) $(LINKLIBS)
}


DC = dmd ;
DCFLAGS = -fPIC -O -inline -release -w -wi -I./imports/llvm-2.9 -I./src/ ;
C++FLAGS =
    -I/usr/include
    -DNDEBUG
    -D_GNU_SOURCE
    -D__STDC_LIMIT_MACROS
    -D__STDC_CONSTANT_MACROS
    -O3
    -fomit-frame-pointer
    -fno-exceptions
    -fPIC
    -Woverloaded-virtual
    -Wcast-qual ;   # should be generated from `llvm-config --cxxflags`
LINK = $(DC) ;
DFILES = [ GLOB src/ob2c : *.d ] ;

MainFromObjects ob2c : $(DFILES:S=.o) ;
Objects $(DFILES) ;
LinkLibraries ob2c : llvm-c-ext ;
Library llvm-c-ext :
    imports/llvm-2.9/llvm/Ext.cpp
    imports/llvm-2.9/llvm/Target.cpp ;

The only remaining addition I'd like to make is to force all build output to a dedicated build folder as you've been helping me with. I don't understand where to apply the SubDir rule and the ALL_LOCATE_TARGET variable. My project directory structure is as follows:
   code/
      |-- src/
      |-- imports/
      |-- build/        <-- folder for all build output
      |-- Jamfile

Cheers,

Chris


On 01/06/11 22:11, John Waugh wrote:
On Tue, May 31, 2011 at 5:02 AM, Chris Molozian <chris <at> cmoz.me> wrote: As suggested I've tried: MkDir build ; ALL_LOCATE_TARGET = build ; to relocate all build output to a dedicated build folder but it appears to do nothing. The object files are still built alongside source files and the libraries and executable build in the same directory as the Jamfile. I'm not sure what I'm doing wrong.
I forgot to mention: you can always set LOCATE on targets yourself, of course. In fact, if you look in Jambase at how ALL_LOCATE_TARGET is used (it's just 2 lines), it just sets the default LOCATE path. LOCATE itself is a jam 'special' variable, which is used to bind a target to a physical path on the file system. c.f. http://public.perforce.com/public/jam/src/Jam.html#search (apologies if you already knew this) -John
_______________________________________________
jamming mailing list  -  jamming <at> maillist.perforce.com
http://maillist.perforce.com/mailman/listinfo/jamming
John Waugh | 3 Jun 2011 23:03
Picon

Re: A few queries about Jam

On Fri, Jun 3, 2011 at 9:57 AM, Chris Molozian <chris <at> cmoz.me> wrote:
>
> The only remaining addition I'd like to make is to force all build output to
> a dedicated build folder as you've been helping me with. I don't understand
> where to apply the SubDir rule and the ALL_LOCATE_TARGET variable. My
> project directory structure is as follows:
>    code/
>       |-- src/
>       |-- imports/
>       |-- build/        <-- folder for all build output
>       |-- Jamfile
>

Here's a Jamfile I created as a demo - the source files referenced are
just for a 'hello world' program.

# --- BEGIN Jamfile ---
ALL_LOCATE_TARGET = build ;
SubDir TOP ;

# Compile will fail if the destionations for ALL_LOCATE_TARGET
# don't exist, so be sure to create them
MkDir build/src ;
Depends proggie : build/src ;

Main myprogram : src/main.c src/foo.c ;
# --- END Jamfile ---

As I mentioned, I haven't used ALL_LOCATE_TARGET much myself, so
perhaps the MkDir/Depends stuff could be handled more automatically,
but that worked for me.

Does it make sense?

-John

_______________________________________________
jamming mailing list  -  jamming <at> maillist.perforce.com
http://maillist.perforce.com/mailman/listinfo/jamming

matthew conte | 4 Jun 2011 00:21

Re: A few queries about Jam

You shouldn't need to explicitly make the build directories, there.


We use ALL_LOCATE_TARGET and it works great for us. Just make sure you have it set up in your Jamrules, then at the top of each Jamfile ensure that you use SubDir. e.g. for a top level directory that has a src1 and src2 subdirectory, you should have this:

[top-level Jamfile]
SubDir TOP ;

# Stuff here

SubInclude $(TOP) src1 ;
SubInclude $(TOP) src2 ;

[src1 Jamfile]
SubDir TOP src1 ;

# stuff here

[src2 Jamfile]
SubDir TOP src2 ;

# stuff


In fact, we go a step further. Because we build for many different platforms, we actually want our build products to go into subdirectories, e.g. "temp/windows-debug" and "temp/macos-release", etc. So our Jamrules looks like this:

ALL_LOCATE_TARGET = temp ;
# Make LocateObjects be called at the end of the SubDir rule.
SUBDIRRULES += LocateObjects ;
rule LocateObjects
{
    # Build flavor is windows-debug, etc.
    LOCATE_TOKENS = $(ALL_LOCATE_TARGET) $(BUILD_FLAVOR) $(SUBDIR_TOKENS) ;

    # See the setup of these in Jambase's SubDir rule for reference.
    LOCATE_TARGET = [ FSubDirPath TOP $(LOCATE_TOKENS) ] ;
    LOCATE_SOURCE = [ FSubDirPath TOP $(LOCATE_TOKENS) ] ;
}

Note that this assumes a TOP, but that's fine for us.

Cheers,
--matt

On Fri, Jun 3, 2011 at 2:03 PM, John Waugh <john.waugh <at> gmail.com> wrote:
On Fri, Jun 3, 2011 at 9:57 AM, Chris Molozian <chris <at> cmoz.me> wrote:
>
> The only remaining addition I'd like to make is to force all build output to
> a dedicated build folder as you've been helping me with. I don't understand
> where to apply the SubDir rule and the ALL_LOCATE_TARGET variable. My
> project directory structure is as follows:
>    code/
>       |-- src/
>       |-- imports/
>       |-- build/        <-- folder for all build output
>       |-- Jamfile
>

Here's a Jamfile I created as a demo - the source files referenced are
just for a 'hello world' program.

# --- BEGIN Jamfile ---
ALL_LOCATE_TARGET = build ;
SubDir TOP ;

# Compile will fail if the destionations for ALL_LOCATE_TARGET
# don't exist, so be sure to create them
MkDir build/src ;
Depends proggie : build/src ;

Main myprogram : src/main.c src/foo.c ;
# --- END Jamfile ---


As I mentioned, I haven't used ALL_LOCATE_TARGET much myself, so
perhaps the MkDir/Depends stuff could be handled more automatically,
but that worked for me.

Does it make sense?

-John

_______________________________________________
jamming mailing list  -  jamming <at> maillist.perforce.com
http://maillist.perforce.com/mailman/listinfo/jamming

_______________________________________________
jamming mailing list  -  jamming <at> maillist.perforce.com
http://maillist.perforce.com/mailman/listinfo/jamming
John Waugh | 4 Jun 2011 01:47
Picon

Re: A few queries about Jam

On Fri, Jun 3, 2011 at 3:21 PM, matthew conte <matt <at> baisoku.org> wrote:
> You shouldn't need to explicitly make the build directories, there.

I found that I had to create the destination directory at least once.
Creating the dirs manually upfront works - any subsequent 'jam clean'
will only kill the files and not the dirs.
But if you ever removed the 'build' directory, AFAIK you'd get errors
the next time you built.

At least that's how it's playing out in my test case I'm toying with -
see below for output from a terminal session.

> We use ALL_LOCATE_TARGET and it works great for us. Just make sure you have
> it set up in your Jamrules...

Good point about putting it in Jamrules, rather than at the top of the
Jamfile. Putting it ahead of SubDir in Jamfile *seemed* unnecessarily
ugly.

Here's the output of a session where I don't have 'build/src' created
beforehand:

$ ls -R
Jamfile  Jamrules build    src

./build:

./src:
foo.c  foo.h  main.c

$ cat Jamrules
# --- BEGIN Jamrules ---
ALL_LOCATE_TARGET = build ;
# --- END Jamrules ---

$ cat Jamfile
# --- BEGIN Jamfile ---
SubDir TOP ;

Main myprog : src/main.c src/foo.c ;
# --- END Jamfile ---

$ ftjam
...found 19 target(s)...
...updating 3 target(s)...
Cc build/src/main.o
Assembler messages:
FATAL: can't create build/src/main.o: No such file or directory

cc -c -o build/src/main.o  -O  -I. src/main.c

...failed Cc build/src/main.o ...
Cc build/src/foo.o
Assembler messages:
FATAL: can't create build/src/foo.o: No such file or directory

cc -c -o build/src/foo.o  -O  -I. src/foo.c

...failed Cc build/src/foo.o ...
...skipped myprog for lack of src/main.o...
...failed updating 2 target(s)...
...skipped 1 target(s)...

$ mkdir build/src

$ ftjam
...found 19 target(s)...
...updating 3 target(s)...
Cc build/src/main.o
Cc build/src/foo.o
Link build/myprog
Chmod1 build/myprog
...updated 3 target(s)...

-John
_______________________________________________
jamming mailing list  -  jamming <at> maillist.perforce.com
http://maillist.perforce.com/mailman/listinfo/jamming

Diane Holt | 4 Jun 2011 02:07
Picon

Re: A few queries about Jam

This is where you went wrong:

> Main myprog : src/main.c src/foo.c ;

You should either have this Jamfile do a SubInclude for src, and put this Main target in src/Jamfile, or you can add $(TOP)/src to SEARCH_SOURCE. I prefer the first one, since then you can build from the src subdir instead of always have to build from $TOP (eg., if you're working down in src on foo.c, after you're done editing, you can just 'jam' from there).

But either of those will get Main to MkDir your build subdir for you, because the rule will do a MakeLocate, and MakeLocate does both the MkDir and the "LOCATE on".

Diane

On Fri, Jun 3, 2011 at 4:47 PM, John Waugh <john.waugh <at> gmail.com> wrote:
On Fri, Jun 3, 2011 at 3:21 PM, matthew conte <matt <at> baisoku.org> wrote:
> You shouldn't need to explicitly make the build directories, there.

I found that I had to create the destination directory at least once.
Creating the dirs manually upfront works - any subsequent 'jam clean'
will only kill the files and not the dirs.
But if you ever removed the 'build' directory, AFAIK you'd get errors
the next time you built.

At least that's how it's playing out in my test case I'm toying with -
see below for output from a terminal session.

> We use ALL_LOCATE_TARGET and it works great for us. Just make sure you have
> it set up in your Jamrules...

Good point about putting it in Jamrules, rather than at the top of the
Jamfile. Putting it ahead of SubDir in Jamfile *seemed* unnecessarily
ugly.

Here's the output of a session where I don't have 'build/src' created
beforehand:

$ ls -R
Jamfile  Jamrules build    src

./build:

./src:
foo.c  foo.h  main.c

$ cat Jamrules
# --- BEGIN Jamrules ---
ALL_LOCATE_TARGET = build ;
# --- END Jamrules ---

$ cat Jamfile
# --- BEGIN Jamfile ---
SubDir TOP ;

Main myprog : src/main.c src/foo.c ;
# --- END Jamfile ---

$ ftjam
...found 19 target(s)...
...updating 3 target(s)...
Cc build/src/main.o
Assembler messages:
FATAL: can't create build/src/main.o: No such file or directory

cc -c -o build/src/main.o  -O  -I. src/main.c

...failed Cc build/src/main.o ...
Cc build/src/foo.o
Assembler messages:
FATAL: can't create build/src/foo.o: No such file or directory

cc -c -o build/src/foo.o  -O  -I. src/foo.c

...failed Cc build/src/foo.o ...
...skipped myprog for lack of src/main.o...
...failed updating 2 target(s)...
...skipped 1 target(s)...

$ mkdir build/src

$ ftjam
...found 19 target(s)...
...updating 3 target(s)...
Cc build/src/main.o
Cc build/src/foo.o
Link build/myprog
Chmod1 build/myprog
...updated 3 target(s)...


-John
_______________________________________________
jamming mailing list  -  jamming <at> maillist.perforce.com
http://maillist.perforce.com/mailman/listinfo/jamming

_______________________________________________
jamming mailing list  -  jamming <at> maillist.perforce.com
http://maillist.perforce.com/mailman/listinfo/jamming
John Waugh | 4 Jun 2011 07:20
Picon

Re: A few queries about Jam

On Fri, Jun 3, 2011 at 5:07 PM, Diane Holt <holt.diane <at> gmail.com> wrote:
> This is where you went wrong:
>
>> Main myprog : src/main.c src/foo.c ;
>
> You should either have this Jamfile do a SubInclude for src, and put this
> Main target in src/Jamfile, or you can add $(TOP)/src to SEARCH_SOURCE.
> ...
> either of those will get Main to MkDir your build subdir for you
>

Good point.
However, after fiddling around with these methods, I will just point
out (correct me if I'm wrong) that neither of these lend themselves to
having duplicately-named source files in different subdirectories.

Not that that's usually an issue, but the purist in me cares on some level (:

-John
_______________________________________________
jamming mailing list  -  jamming <at> maillist.perforce.com
http://maillist.perforce.com/mailman/listinfo/jamming

Alen Ladavac | 5 Jun 2011 11:56

Re: A few queries about Jam

Not jam related, but duplicate named source files are best avoided anyway because may other tools may fail
in different ways. JM2C.

Note that jam can usually do things like that using "grist". I'm not familiar with how it works with default
jambase, since we use a very customized one.

John wrote at 2011-06-04:

> On Fri, Jun 3, 2011 at 5:07 PM, Diane Holt <holt.diane <at> gmail.com> wrote:
>> This is where you went wrong:

>>> Main myprog : src/main.c src/foo.c ;

>> You should either have this Jamfile do a SubInclude for src, and put this
>> Main target in src/Jamfile, or you can add $(TOP)/src to SEARCH_SOURCE.
>> ...
>> either of those will get Main to MkDir your build subdir for you

> Good point.
> However, after fiddling around with these methods, I will just point
> out (correct me if I'm wrong) that neither of these lend themselves to
> having duplicately-named source files in different subdirectories.

> Not that that's usually an issue, but the purist in me cares on some level (:

> -John
> _______________________________________________
> jamming mailing list  -  jamming <at> maillist.perforce.com
> http://maillist.perforce.com/mailman/listinfo/jamming

--

-- 
Alen

_______________________________________________
jamming mailing list  -  jamming <at> maillist.perforce.com
http://maillist.perforce.com/mailman/listinfo/jamming

Diane Holt | 7 Jun 2011 00:41
Picon

Re: A few queries about Jam

That's what grist is for -- it allows you to have any number of files named the same (e.g., a zillion main.c files). If you're building Main targets, the gristing is taken care of for you. If you've got your own type of targets you're building, see the FGristFiles rule.

Diane

On Fri, Jun 3, 2011 at 10:20 PM, John Waugh <john.waugh <at> gmail.com> wrote:
On Fri, Jun 3, 2011 at 5:07 PM, Diane Holt <holt.diane <at> gmail.com> wrote:
> This is where you went wrong:
>
>> Main myprog : src/main.c src/foo.c ;
>
> You should either have this Jamfile do a SubInclude for src, and put this
> Main target in src/Jamfile, or you can add $(TOP)/src to SEARCH_SOURCE.
> ...
> either of those will get Main to MkDir your build subdir for you
>

Good point.
However, after fiddling around with these methods, I will just point
out (correct me if I'm wrong) that neither of these lend themselves to
having duplicately-named source files in different subdirectories.

Not that that's usually an issue, but the purist in me cares on some level (:

-John

_______________________________________________
jamming mailing list  -  jamming <at> maillist.perforce.com
http://maillist.perforce.com/mailman/listinfo/jamming
John Waugh | 7 Jun 2011 19:39
Picon

Re: A few queries about Jam

On Mon, Jun 6, 2011 at 3:41 PM, Diane Holt <holt.diane <at> gmail.com> wrote:
> That's what grist is for ...

Surely - I was just pointing out that in the particular case we were
discussing - using ALL_LOCATE_TARGET - the final resting place of the
.o files would be a single dir when using the method you described.
Grist is so jam can keep same-named files separate "in its head" at
the same time. When those files are written to disk, of course they
need to have different final bound locations.

Normally they would, but using ALL_LOCATE_TARGET puts them all
together in one dir, unless you specify the dir names as part of the
file name, as I originally had suggested -- but then you have to
create the output dirs yourself, which is how we got into this whole
discussion in the first place.

Sorry, making a lot of talk out of not much of an issue (:

I think the takeaway is: use ALL_LOCATE_TARGET in the manner Diane
suggested, and also don't have two identically-named source files in
the same project.

-John
_______________________________________________
jamming mailing list  -  jamming <at> maillist.perforce.com
http://maillist.perforce.com/mailman/listinfo/jamming

Diane Holt | 7 Jun 2011 21:07
Picon

Re: A few queries about Jam

Oh, sorry -- I'd forgotten about the earlier discussion wrt ALL_LOCATE_TARGET. I don't actually use ALL_LOCATE_TARGET -- I set LOCATE_TARGET (based on the path elements of the SubDir target), in one of the rules I set SUBDIRRULES to. So my build tree is a single place (BUILD_DIR), and the object files build out to subdirs under there, based on their own path in the source tree, which is why same-named object files aren't a problem.

Diane

On Tue, Jun 7, 2011 at 10:39 AM, John Waugh <john.waugh <at> gmail.com> wrote:
On Mon, Jun 6, 2011 at 3:41 PM, Diane Holt <holt.diane <at> gmail.com> wrote:
> That's what grist is for ...

Surely - I was just pointing out that in the particular case we were
discussing - using ALL_LOCATE_TARGET - the final resting place of the
.o files would be a single dir when using the method you described.
Grist is so jam can keep same-named files separate "in its head" at
the same time. When those files are written to disk, of course they
need to have different final bound locations.

Normally they would, but using ALL_LOCATE_TARGET puts them all
together in one dir, unless you specify the dir names as part of the
file name, as I originally had suggested -- but then you have to
create the output dirs yourself, which is how we got into this whole
discussion in the first place.

Sorry, making a lot of talk out of not much of an issue (:

I think the takeaway is: use ALL_LOCATE_TARGET in the manner Diane
suggested, and also don't have two identically-named source files in
the same project.

-John

_______________________________________________
jamming mailing list  -  jamming <at> maillist.perforce.com
http://maillist.perforce.com/mailman/listinfo/jamming
Diane Holt | 4 Jun 2011 01:48
Picon

Re: A few queries about Jam

> SubInclude $(TOP) src1 ;

That should be TOP not $(TOP) -- SubDir adds the $( ) for you. Jam will give you an error if you include the $( ), because SubDir is expecting it to be the name of the var, so it will try to expand a variable that doesn't exist (since you've passed it the value not the name).

Diane

On Fri, Jun 3, 2011 at 3:21 PM, matthew conte <matt <at> baisoku.org> wrote:
You shouldn't need to explicitly make the build directories, there.

We use ALL_LOCATE_TARGET and it works great for us. Just make sure you have it set up in your Jamrules, then at the top of each Jamfile ensure that you use SubDir. e.g. for a top level directory that has a src1 and src2 subdirectory, you should have this:

[top-level Jamfile]
SubDir TOP ;

# Stuff here

SubInclude $(TOP) src1 ;
SubInclude $(TOP) src2 ;

[src1 Jamfile]
SubDir TOP src1 ;

# stuff here

[src2 Jamfile]
SubDir TOP src2 ;

# stuff


In fact, we go a step further. Because we build for many different platforms, we actually want our build products to go into subdirectories, e.g. "temp/windows-debug" and "temp/macos-release", etc. So our Jamrules looks like this:

ALL_LOCATE_TARGET = temp ;
# Make LocateObjects be called at the end of the SubDir rule.
SUBDIRRULES += LocateObjects ;
rule LocateObjects
{
    # Build flavor is windows-debug, etc.
    LOCATE_TOKENS = $(ALL_LOCATE_TARGET) $(BUILD_FLAVOR) $(SUBDIR_TOKENS) ;

    # See the setup of these in Jambase's SubDir rule for reference.
    LOCATE_TARGET = [ FSubDirPath TOP $(LOCATE_TOKENS) ] ;
    LOCATE_SOURCE = [ FSubDirPath TOP $(LOCATE_TOKENS) ] ;
}

Note that this assumes a TOP, but that's fine for us.

Cheers,
--matt

On Fri, Jun 3, 2011 at 2:03 PM, John Waugh <john.waugh <at> gmail.com> wrote:
On Fri, Jun 3, 2011 at 9:57 AM, Chris Molozian <chris <at> cmoz.me> wrote:
>
> The only remaining addition I'd like to make is to force all build output to
> a dedicated build folder as you've been helping me with. I don't understand
> where to apply the SubDir rule and the ALL_LOCATE_TARGET variable. My
> project directory structure is as follows:
>    code/
>       |-- src/
>       |-- imports/
>       |-- build/        <-- folder for all build output
>       |-- Jamfile
>

Here's a Jamfile I created as a demo - the source files referenced are
just for a 'hello world' program.

# --- BEGIN Jamfile ---
ALL_LOCATE_TARGET = build ;
SubDir TOP ;

# Compile will fail if the destionations for ALL_LOCATE_TARGET
# don't exist, so be sure to create them
MkDir build/src ;
Depends proggie : build/src ;

Main myprogram : src/main.c src/foo.c ;
# --- END Jamfile ---


As I mentioned, I haven't used ALL_LOCATE_TARGET much myself, so
perhaps the MkDir/Depends stuff could be handled more automatically,
but that worked for me.

Does it make sense?

-John

_______________________________________________
jamming mailing list  -  jamming <at> maillist.perforce.com
http://maillist.perforce.com/mailman/listinfo/jamming


_______________________________________________
jamming mailing list  -  jamming <at> maillist.perforce.com
http://maillist.perforce.com/mailman/listinfo/jamming


_______________________________________________
jamming mailing list  -  jamming <at> maillist.perforce.com
http://maillist.perforce.com/mailman/listinfo/jamming

Gmane