Toddius Zho | 22 Jun 2012 23:50
Favicon

[p4] Version ... Everything?


Posted on behalf of forum user 'Toddius Zho'.

Milestone 1: Version Only Sources

In my organization, we version sources used to create deployable things --
scripts, applications, plugins, etc. We typically then transfer these
"built" artifacts into target machines and destinations. Maybe there
is a formal install, or maybe it's just drag 'n drop. 

Milestone 2: Version Deployables

Lately, we've progressed to putting the deployable artifacts themselves into
Perforce. The target machines now have direct access to our P4PORT and can then
just pull down what it needs. Perforce itself becomes a distribution channel.
P4ROOT on these machines are in a clean staging area. They can install the
artifact from that location to a different destination area.

Milestone 3: Version Configuration

The amount of configuration that must be done outside of an application is
starting to become overwhelming. There is setup of environment variables,
bootstrapping of JVMs, optimizing application servers and daemons, etc. It would
be nice to version these files, too, but they don't exist (nor should they?)
under P4ROOT. How do you version things like:

/etc/profile.d/custom.sh /etc/init.d/my-home-grown-service

We try to make these things as parameterizable as possible, but sooner or later
you have to stick a  machine-specific  value into a file that also exists on
(Continue reading)

Michael Mirman | 23 Jun 2012 02:04
Picon
Favicon

Re: [p4] Version ... Everything?

What we have done may not be exactly what you are trying to do, but maybe it's close enough...

1. Version crontabs.
When the number of the hosts in our Perforce infrastructure (IA) jumped from 20 to 40, it became obvious that
going to every individual host and modify crontabs (typically, some Perforce maintenance and
monitoring) does not scale.
Solution:
All crontabs live in a specific location in the repository.
A post-commit trigger "pushes" all new changes to the appropriate machine and installs the new crontab.
As an additional safety valve, there is a cron job on every machine (obviously, it's listed in every crontab
file in Perforce) that "pulls" the appropriate file and installs it. This is just in case if the trigger
fails, the crontab will not be out-of-date forever.

2. /etc/init.d/ scripts
We cover only startup Perforce-related scripts in Perforce, although technically there is no reason not
to cover something else the same way:

The file under /etc/init.d/ is actually a link to a script, which lives in a particular place on that
machine. The location of the actual script is usually local to that machine, but only because our servers
by and large don't mount NFS disks.
The area where the actual script lives is a Perforce workspace that gets updated when the corresponding
scripts change in the repository.
We used to push by a post-commit trigger, but it didn't scale well for us.
Now a post-commit trigger sets a counter as sort of a flag, and every IA host runs a cron job every five minutes
to check that counter.
If it finds the counter set, it derives the client name from the name of the counter and syncs that client.
If any particular machine is down or has no connection with the Perforce server, it will do that when the
machine is rebooted and/or the connection is fixed.  This works great for all our geographic locations
around the world.

(Continue reading)

Peter Buckley | 23 Jun 2012 02:52
Picon
Gravatar

Re: [p4] Version ... Everything?

What about using something like pallet (http://palletops.com/) (or chef, or puppet) to handle the
configuration of your machines? 

The general idea would be you'd have some templates stored in source control, and they get deployed (and
machine-specific-customizations) on the fly, on demand, and/or on a schedule. 

Tools like pallet make this type of task more manageable. It also makes changes to the configuration
simpler to roll out and keep consistent going forward. 

-----Original Message-----
From: Toddius Zho <perforce-user-forum <at> forums.perforce.com>
Sender: perforce-user-bounces <at> perforce.com
Date: Fri, 22 Jun 2012 14:50:01 
To: <perforce-user <at> perforce.com>
Reply-To: perforce-user <at> perforce.com
Subject: [p4] Version ... Everything?

Posted on behalf of forum user 'Toddius Zho'.

Milestone 1: Version Only Sources

In my organization, we version sources used to create deployable things --
scripts, applications, plugins, etc. We typically then transfer these
"built" artifacts into target machines and destinations. Maybe there
is a formal install, or maybe it's just drag 'n drop. 

Milestone 2: Version Deployables

Lately, we've progressed to putting the deployable artifacts themselves into
Perforce. The target machines now have direct access to our P4PORT and can then
(Continue reading)

Robert Cowham | 25 Jun 2012 10:50
Favicon

Re: [p4] Version ... Everything?


Posted on behalf of forum user 'Robert Cowham'.

[http://forums.perforce.com/index.php?app=forums&module=forums&section=findpost&pid=5514]
Peter Buckley, on 2012/06/23 00:50:03 UTC, said:
>    What about using something like pallet (   http://palletops.com/ ) (or chef, or puppet) to handle the
configuration of your machines? 
>   
>   The general idea would be you'd have some templates stored in source control, and they get deployed (and
machine-specific-customizations) on the fly, on demand, and/or on a schedule. 
>   
>   Tools like pallet make this type of task more manageable. It also makes changes to the configuration
simpler to roll out and keep consistent going forward. 
>   
Tod add to this: I would check out the "DevOps" concept which is
getting increasing traction and mindshare. There are lots of articles and groups
meeting up about this and the tools Peter has mentioned are a good starter.

DevOps is at risk of being a passing "flavour of the month", can
certainly be over-hyped and isn't yet delivering for everyone, but there are
good ideas and approaches in there if you read with your BS detection filter set
appropriately  :)

--
Please click here to see the post in its original format:
  http://forums.perforce.com/index.php?/topic/1694-version-everything
_______________________________________________
perforce-user mailing list  -  perforce-user <at> perforce.com
http://maillist.perforce.com/mailman/listinfo/perforce-user

(Continue reading)

Toddius Zho | 26 Jun 2012 15:55
Favicon

Re: [p4] Version ... Everything?


Posted on behalf of forum user 'Toddius Zho'.

I would like to thank everyone for their responses. Coordinating p4 counter with
submit triggers is something I will definitely look into. I was already
investigating CF Engine 3, but PalletOps looks more palatable (pun intended) at
this stage.

The problem with lookup repositories is there are some weak links in our
solution. Admins might go in and change these settings without versioning them.
I would have to routinely cron some "catch-all" processes that try to
"after-the-fact" capture as many of these rogue changes as possible.

--
Please click here to see the post in its original format:
  http://forums.perforce.com/index.php?/topic/1694-version-everything
_______________________________________________
perforce-user mailing list  -  perforce-user <at> perforce.com
http://maillist.perforce.com/mailman/listinfo/perforce-user

Toddius Zho | 26 Jun 2012 22:45
Favicon

Re: [p4] Version ... Everything?


Posted on behalf of forum user 'Toddius Zho'.

Which is the better practice?

Practice A: Machine-Specific Files Beneath Projects' File Structure

//depot/projectFoo/conf/machine01.settings
//depot/projectFoo/conf/machine02.settings
//depot/projectBar/conf/machine01.settings
//depot/projectBar/conf/machine02.settings 

Practice B: Mirroring Projects' File Structures Underneath Machine-Specific
Roots

//depot/machine01/projectFoo/conf/profile.settings
//depot/machine01/projectBar/conf/profile.settings
//depot/machine02/projectFoo/conf/profile.settings
//depot/machine02/projectBar/conf/profile.settings 

--
Please click here to see the post in its original format:
  http://forums.perforce.com/index.php?/topic/1694-version-everything
_______________________________________________
perforce-user mailing list  -  perforce-user <at> perforce.com
http://maillist.perforce.com/mailman/listinfo/perforce-user

Stephen Vance | 23 Jun 2012 16:01

Re: [p4] Version ... Everything?

Consolidate what you do outside of the app. Yes, you will always have 
start scripts and components where you can't move the configuration, but 
for many things, especially in your own application, you can put the 
configurations into a lookup repository like LDAP or a database. Once 
you do that, your only environment-specific configuration parameter in 
the app is where to look for the settings and you can version the load 
scripts for the settings.

I'm making it sound a bit easier than it is, but it's very do-able.

Steve

On 6/22/12 17:50 PM, Toddius Zho wrote:
>
> Posted on behalf of forum user 'Toddius Zho'.
>
> Milestone 1: Version Only Sources
>
> In my organization, we version sources used to create deployable 
> things --
> scripts, applications, plugins, etc. We typically then transfer these
> "built" artifacts into target machines and destinations. Maybe there
> is a formal install, or maybe it's just drag 'n drop.
> Milestone 2: Version Deployables
>
> Lately, we've progressed to putting the deployable artifacts 
> themselves into
> Perforce. The target machines now have direct access to our P4PORT and 
> can then
> just pull down what it needs. Perforce itself becomes a distribution 
(Continue reading)


Gmane