User interface development mailing list for OLPC. ()

Tomeu Vizoso | 2 Jan 16:00

Re: g.sl.o issues for Karma and perhaps other activities

On Sat, Jan 2, 2010 at 03:50, Bryan Berry <bryan <at> olenepal.org> wrote:
> On Sat, Jan 2, 2010 at 6:16 AM, Wade Brainerd <wadetb <at> gmail.com> wrote:
>>
>> This sounds like the ideal conditions for Git.
>>
>> Just set up a server at your office using any Git related software you
>> want, like Gitorious or even GitWeb.  Developers set their projects up
>> on your local server, and when they reach some level of stability they
>> create public repositories on git.sugarlabs.org.
>
> I would like to set up gitorious but not sure how difficult this would be
>
>>
>> Do all your development on your local server, and every once a while
>> push the changes over to SL.
>>
>>  git push gitorious <at> git.sugarlabs.org:myproject/mainline.git
>>
>> If SL people want to make changes, they clone the repository on
>> git.sugarlabs.org and push to it.
>
> We will have about 60 individual repos by April and hopefully several
> hundred by the end of 2010. It will be impractical and error-prone is we
> have to create each repo twice, once on our local server and once on the SL
> server. Is there a way to automate this?

Guess you can cook a cron job quite easily, but I don't see any way
around having only one writeable instance and the others read only,
unless someone wants to take care of resolving conflicts.

(Continue reading)

headers

Wade Brainerd | 2 Jan 18:01

Re: g.sl.o issues for Karma and perhaps other activities

How badly do you need a user friendly repository creator like Gitorious?

If it's not important, you can just use GitWeb which is trivial to set up.

I guess the important thing to consider is that Git can handle
distributing and merging *code* changes across as many servers as you
want.  But if you want the metadata like project descriptions updated,
you'll have to setup cron or a manual process like that.

Honestly, 60 projects doesn't sound like too much work to set up by
hand on both servers, compared with the amount of work to actually
develop the lessons...  Setting up a project on g.sl.o only takes a
minute or so.

-Wade

On Sat, Jan 2, 2010 at 10:00 AM, Tomeu Vizoso <tomeu <at> sugarlabs.org> wrote:
> On Sat, Jan 2, 2010 at 03:50, Bryan Berry <bryan <at> olenepal.org> wrote:
>> On Sat, Jan 2, 2010 at 6:16 AM, Wade Brainerd <wadetb <at> gmail.com> wrote:
>>>
>>> This sounds like the ideal conditions for Git.
>>>
>>> Just set up a server at your office using any Git related software you
>>> want, like Gitorious or even GitWeb.  Developers set their projects up
>>> on your local server, and when they reach some level of stability they
>>> create public repositories on git.sugarlabs.org.
>>
>> I would like to set up gitorious but not sure how difficult this would be
>>
>>>

(Continue reading)

headers

Bernie Innocenti | 2 Jan 20:37

Re: g.sl.o issues for Karma and perhaps other activities

Sorry it took some time to answer this thread. I'm still recovering from
a new year's eve in Times Square 

On Sat, 2010-01-02 at 12:01 -0500, Wade Brainerd wrote:
> How badly do you need a user friendly repository creator like
> Gitorious?
> 
> If it's not important, you can just use GitWeb which is trivial to set
> up.
> 
> I guess the important thing to consider is that Git can handle
> distributing and merging *code* changes across as many servers as you
> want.  But if you want the metadata like project descriptions updated,
> you'll have to setup cron or a manual process like that.
> 
> Honestly, 60 projects doesn't sound like too much work to set up by
> hand on both servers, compared with the amount of work to actually
> develop the lessons...  Setting up a project on g.sl.o only takes a
> minute or so.

The problem with managing many repositories by hand is not just setting
them up.  Once you have plenty of people accessing these repositories,
you'll need to implement fine-grained access control. The number of
access requests probably grows very quickly, proportionally to the
number of developers and repositories. Soon or later, your gitmaster
will become buried in support requests.

That said, Gitorious is a complex Rails application. Compared to other
web applications, it was quite hard to deploy and maintain. Indefero
(http://www.indefero.net/) sounds like a much simpler alternative that I

(Continue reading)

headers

Bryan Berry | 3 Jan 03:44

Re: g.sl.o issues for Karma and perhaps other activities

On Sun, Jan 3, 2010 at 1:22 AM, Bernie Innocenti <bernie <at> codewiz.org> wrote:

Sorry it took some time to answer this thread. I'm still recovering from
a new year's eve in Times Square

np, hope you had a great time!

The problem with managing many repositories by hand is not just setting
them up. Once you have plenty of people accessing these repositories,
you'll need to implement fine-grained access control. The number of
access requests probably grows very quickly, proportionally to the
number of developers and repositories. Soon or later, your gitmaster
will become buried in support requests.

That said, Gitorious is a complex Rails application. Compared to other
web applications, it was quite hard to deploy and maintain. Indefero
(http://www.indefe ro.net/) sounds like a much simpler alternative that I

would investigate.

I am wary of experimenting when the gslo infrastructure is still in flux. I would rather stick w/ gitweb or cgit and then move to whatever becomes of gslo. Should we start w/ cgit or gitweb?

I have my hands full experimenting w/ SVG and html5 at the moment so I need to keep my focus there for the time being.

Finally, we've been planning to migrate to the Nokia instance of
Gitorious for a while. We're currently waiting for management approval,
for which I have no ETA. We'll ping them again after the holidays.

Meanwhile, I would appreciate if someone would like to experiment with a
fresher installation of Gitorious. Bryan, if you feel like working on
it, I could create a gitorious account for you on Sunjammer.

Again, I would love to but my plate is quite full right now supporting roxan and Vaibhaw's work on Karma.

_______________________________________________
Sugar-devel mailing list
Sugar-devel <at> lists.sugarlabs.org
http://lists.sugarlabs.org/listinfo/sugar-devel

headers

Bernie Innocenti | 3 Jan 21:46

Re: g.sl.o issues for Karma and perhaps other activities

On Sun, 2010-01-03 at 08:29 +0545, Bryan Berry wrote:

> 
> I am wary of experimenting when the gslo infrastructure is still in
> flux. I would rather stick w/ gitweb or cgit and then move to whatever
> becomes of gslo. Should we start w/ cgit or gitweb?
> 

An alternative to installing a local multi-repository interface just
occurred to me.

To solve the data transfer issue, git offers a pletora of
bandwidth-saving options. One possibility is to let the Nepali
developers clone from each other's repository:

 git clone --reference officemate.local:~bryanwb/src/karma \
    git://git.sugarlabs.org/karma/mainline.git karma

Alternatively, you could rsync the files to your local machine:

  rsync -aP officemate.local:~bryanwb/src/karma/.git karma-ref.git
  git clone --reference karma-ref.git \
      git://git.sugarlabs.org/karma/mainline.git karma

It doesn't matter if the reference repository contains unwanted changes
or hasn't been updated for a while. Any good bits will be reused, and
the rest will be fetched remotely.

Other tips:

 * Never use the http protocol, as it is quite inefficient.

 * try passing the --thin option when you pull. The server-side
   will work harder to minimize file transfers.

 * Another way to speed-up initial cloning is to limit the number
   of revisions with --depth. It's usually not a big deal, unless
   your history contains lots of large files that were subsequently
   deleted.

 * If your history contains plenty of similar files, for example
   PNGs that have been merely moved around, you'd benefit by
   repacking your repository with larger --window and --depth
   parameters. You could repack locally and, when you're happy
   with the result, push it to Gitorious with "git push --mirror".

 * If your history is badly messed up, you may "edit" it locally
   with a combination of the advanced git operations such as
   "git rebase --interactive", "git reset --hard" and
   "git commit --amend". When you publish the resulting edited
   history, people who had previously cloned from you will have
   to refetch. Needless to say, git's arsenal of history-rewriting
   commands is as dangerous as it is powerful.

 * the git:// protocol is probably slightly faster than using the
   ssh tunnel. You can easily switch between the two using
   "git remote ..." or by editing .git/config manually.

Free git advice is available from me on #sugar, GMT-5 office hours.
Next business day, on-site support contracts available on demand.
Git is a registered trademark of Stupid Version Control Systems, Inc.

--

-- 
   // Bernie Innocenti - http://codewiz.org/
 \X/  Sugar Labs       - http://sugarlabs.org/

headers

Bernie Innocenti | 3 Jan 21:46

Re: g.sl.o issues for Karma and perhaps other activities

On Sun, 2010-01-03 at 08:29 +0545, Bryan Berry wrote:

> 
> I am wary of experimenting when the gslo infrastructure is still in
> flux. I would rather stick w/ gitweb or cgit and then move to whatever
> becomes of gslo. Should we start w/ cgit or gitweb?
> 

An alternative to installing a local multi-repository interface just
occurred to me.

To solve the data transfer issue, git offers a pletora of
bandwidth-saving options. One possibility is to let the Nepali
developers clone from each other's repository:

 git clone --reference officemate.local:~bryanwb/src/karma \
    git://git.sugarlabs.org/karma/mainline.git karma

Alternatively, you could rsync the files to your local machine:

  rsync -aP officemate.local:~bryanwb/src/karma/.git karma-ref.git
  git clone --reference karma-ref.git \
      git://git.sugarlabs.org/karma/mainline.git karma

It doesn't matter if the reference repository contains unwanted changes
or hasn't been updated for a while. Any good bits will be reused, and
the rest will be fetched remotely.

Other tips:

 * Never use the http protocol, as it is quite inefficient.

 * try passing the --thin option when you pull. The server-side
   will work harder to minimize file transfers.

 * Another way to speed-up initial cloning is to limit the number
   of revisions with --depth. It's usually not a big deal, unless
   your history contains lots of large files that were subsequently
   deleted.

 * If your history contains plenty of similar files, for example
   PNGs that have been merely moved around, you'd benefit by
   repacking your repository with larger --window and --depth
   parameters. You could repack locally and, when you're happy
   with the result, push it to Gitorious with "git push --mirror".

 * If your history is badly messed up, you may "edit" it locally
   with a combination of the advanced git operations such as
   "git rebase --interactive", "git reset --hard" and
   "git commit --amend". When you publish the resulting edited
   history, people who had previously cloned from you will have
   to refetch. Needless to say, git's arsenal of history-rewriting
   commands is as dangerous as it is powerful.

 * the git:// protocol is probably slightly faster than using the
   ssh tunnel. You can easily switch between the two using
   "git remote ..." or by editing .git/config manually.

Free git advice is available from me on #sugar, GMT-5 office hours.
Next business day, on-site support contracts available on demand.
Git is a registered trademark of Stupid Version Control Systems, Inc.

--

-- 
   // Bernie Innocenti - http://codewiz.org/
 \X/  Sugar Labs       - http://sugarlabs.org/

headers

Bryan Berry | 3 Jan 03:44

Re: g.sl.o issues for Karma and perhaps other activities

On Sun, Jan 3, 2010 at 1:22 AM, Bernie Innocenti <bernie <at> codewiz.org> wrote:

Sorry it took some time to answer this thread. I'm still recovering from
a new year's eve in Times Square

np, hope you had a great time!

The problem with managing many repositories by hand is not just setting
them up. Once you have plenty of people accessing these repositories,
you'll need to implement fine-grained access control. The number of
access requests probably grows very quickly, proportionally to the
number of developers and repositories. Soon or later, your gitmaster
will become buried in support requests.

That said, Gitorious is a complex Rails application. Compared to other
web applications, it was quite hard to deploy and maintain. Indefero
(http://www.indefe ro.net/) sounds like a much simpler alternative that I

would investigate.

I am wary of experimenting when the gslo infrastructure is still in flux. I would rather stick w/ gitweb or cgit and then move to whatever becomes of gslo. Should we start w/ cgit or gitweb?

I have my hands full experimenting w/ SVG and html5 at the moment so I need to keep my focus there for the time being.

Finally, we've been planning to migrate to the Nokia instance of
Gitorious for a while. We're currently waiting for management approval,
for which I have no ETA. We'll ping them again after the holidays.

Meanwhile, I would appreciate if someone would like to experiment with a
fresher installation of Gitorious. Bryan, if you feel like working on
it, I could create a gitorious account for you on Sunjammer.

Again, I would love to but my plate is quite full right now supporting roxan and Vaibhaw's work on Karma.

_______________________________________________
Sugar-devel mailing list
Sugar-devel <at> lists.sugarlabs.org
http://lists.sugarlabs.org/listinfo/sugar-devel

headers

Bernie Innocenti | 2 Jan 20:37

Re: g.sl.o issues for Karma and perhaps other activities

Sorry it took some time to answer this thread. I'm still recovering from
a new year's eve in Times Square 

On Sat, 2010-01-02 at 12:01 -0500, Wade Brainerd wrote:
> How badly do you need a user friendly repository creator like
> Gitorious?
> 
> If it's not important, you can just use GitWeb which is trivial to set
> up.
> 
> I guess the important thing to consider is that Git can handle
> distributing and merging *code* changes across as many servers as you
> want.  But if you want the metadata like project descriptions updated,
> you'll have to setup cron or a manual process like that.
> 
> Honestly, 60 projects doesn't sound like too much work to set up by
> hand on both servers, compared with the amount of work to actually
> develop the lessons...  Setting up a project on g.sl.o only takes a
> minute or so.

The problem with managing many repositories by hand is not just setting
them up.  Once you have plenty of people accessing these repositories,
you'll need to implement fine-grained access control. The number of
access requests probably grows very quickly, proportionally to the
number of developers and repositories. Soon or later, your gitmaster
will become buried in support requests.

That said, Gitorious is a complex Rails application. Compared to other
web applications, it was quite hard to deploy and maintain. Indefero
(http://www.indefero.net/) sounds like a much simpler alternative that I
would investigate.

Finally, we've been planning to migrate to the Nokia instance of
Gitorious for a while. We're currently waiting for management approval,
for which I have no ETA. We'll ping them again after the holidays.

Meanwhile, I would appreciate if someone would like to experiment with a
fresher installation of Gitorious. Bryan, if you feel like working on
it, I could create a gitorious account for you on Sunjammer.

I'm open to other possibilities, too. It would be great if we could
share our development infrastructure.

--

-- 
   // Bernie Innocenti - http://codewiz.org/
 \X/  Sugar Labs       - http://sugarlabs.org/

PS: Bryan, you were right: Avatar was fantastic!

headers

Wade Brainerd | 2 Jan 18:01

Re: g.sl.o issues for Karma and perhaps other activities

How badly do you need a user friendly repository creator like Gitorious?

If it's not important, you can just use GitWeb which is trivial to set up.

I guess the important thing to consider is that Git can handle
distributing and merging *code* changes across as many servers as you
want.  But if you want the metadata like project descriptions updated,
you'll have to setup cron or a manual process like that.

Honestly, 60 projects doesn't sound like too much work to set up by
hand on both servers, compared with the amount of work to actually
develop the lessons...  Setting up a project on g.sl.o only takes a
minute or so.

-Wade

On Sat, Jan 2, 2010 at 10:00 AM, Tomeu Vizoso <tomeu <at> sugarlabs.org> wrote:
> On Sat, Jan 2, 2010 at 03:50, Bryan Berry <bryan <at> olenepal.org> wrote:
>> On Sat, Jan 2, 2010 at 6:16 AM, Wade Brainerd <wadetb <at> gmail.com> wrote:
>>>
>>> This sounds like the ideal conditions for Git.
>>>
>>> Just set up a server at your office using any Git related software you
>>> want, like Gitorious or even GitWeb.  Developers set their projects up
>>> on your local server, and when they reach some level of stability they
>>> create public repositories on git.sugarlabs.org.
>>
>> I would like to set up gitorious but not sure how difficult this would be
>>
>>>
>>> Do all your development on your local server, and every once a while
>>> push the changes over to SL.
>>>
>>>  git push gitorious <at> git.sugarlabs.org:myproject/mainline.git
>>>
>>> If SL people want to make changes, they clone the repository on
>>> git.sugarlabs.org and push to it.
>>
>> We will have about 60 individual repos by April and hopefully several
>> hundred by the end of 2010. It will be impractical and error-prone is we
>> have to create each repo twice, once on our local server and once on the SL
>> server. Is there a way to automate this?
>
> Guess you can cook a cron job quite easily, but I don't see any way
> around having only one writeable instance and the others read only,
> unless someone wants to take care of resolving conflicts.
>
> Regards,
>
> Tomeu
>
>
>>
>>>
>>>  git push gitorious <at> git.sugarlabs.org:myproject/wadebs-clone.git
>>>
>>> The SL person lets the author know by email, and the author pulls the
>>> changes to their local repository, merges them, and pushes them to
>>> your internal server.
>>>
>>>  git pull gitorious <at> git.sugarlabs.org:myproject/wadebs-clone.git
>>>  .. do merge work
>>>  git push username <at> local-git-server:myproject.git
>>>
>>> Does this help at all?
>>>
>>> -Wade
>>>
>>>
>>> On Thu, Dec 31, 2009 at 10:37 PM, Bryan Berry <bryan <at> olenepal.org> wrote:
>>> > I want to discuss some issues for managing Karma lessons on glso. Please
>>> > let
>>> > it be clear that I am not criticizing the infrastructure team
>>> > __at_all__. I
>>> > think they are doing a great job. The issues I am encountering have to
>>> > do
>>> > with underlying tools and some issues specific to developers working in
>>> > countries w/ crappy bandwidth, such as Nepal.
>>> > Some of the main goals of the Karma Project are to get more developers
>>> > in
>>> > general involved in creating content for Sugar and to make OLE Nepal's
>>> > content development more accessible and open to developers inside and
>>> > outside Nepal. We have a full-time team of 7 sw engineers, 3 graphic
>>> > designers, and 3 teachers working on content. It would be a crying shame
>>> > if
>>> > we can't work with the larger community.
>>> > One big problem for devs here in Nepal is that international bandwidth
>>> > is
>>> > both lousy and expensive. Conversely, w/in Kathmandu bandwidth is
>>> > relatively
>>> > high-speed and cheap. I have up to 2 Mbps w/in Nepal but get around 30
>>> > kbps
>>> > for a site hosted outside Nepal.
>>> > The Karma repos are big and there will soon be many. The main Karma repo
>>> > will be 10-15 MB and each individual lesson will be in its own repo,
>>> > usually
>>> > 2-4 MB. I hope to have about 60 individual karma activities under source
>>> > control. That will be easily 200 MB. Transferring files of that size
>>> > over
>>> > slow international links will really cramp our development cycle. At the
>>> > same time we need for the Karma lessons to be easily accessible
>>> > internationally.
>>> > A working solution will have to start with a server inside Nepal hosting
>>> > the
>>> > Karma content. OLE Nepal can likely provide the server space. Would it
>>> > be
>>> > possible for us to set up our own instance of gitorious? My impression
>>> > is
>>> > that everyone is waiting to move to the gitorious instance but something
>>> > is
>>> > holding it up. Even if g.sl.o migrates to
>>> > gitorious.org how difficult would it be to set up an instance in Nepal.
>>> > Or
>>> > will it be too hard to set up a gitorious instance and we should just
>>> > use
>>> > something simple for Karma like cgit?
>>> > So say we do set up an instance of gitorious here in Nepal. How could we
>>> > make it easy for others outside Nepal to access the code and contribute
>>> > back? If you are outside Nepal, downloading from a server in Nepal also
>>> > sucks due to the bandwidth issue. Would it be feasible to set up a
>>> > read-only
>>> > mirror of Nepal's repositories on the Sugar infrastructure?
>>> > I would like there to be a writable set of repositories on an
>>> > international
>>> > server but I can't imagine how the this mirror would sync w/ the Nepal
>>> > server without lots of nasty conflicts.
>>> > Sugaristas, please let me know what you think
>>> > _______________________________________________
>>> > Sugar-devel mailing list
>>> > Sugar-devel <at> lists.sugarlabs.org
>>> > http://lists.sugarlabs.org/listinfo/sugar-devel
>>> >
>>> >
>>
>>
>> _______________________________________________
>> Sugar-devel mailing list
>> Sugar-devel <at> lists.sugarlabs.org
>> http://lists.sugarlabs.org/listinfo/sugar-devel
>>
>>
>
>
>
> --
> «Sugar Labs is anyone who participates in improving and using Sugar.
> What Sugar Labs does is determined by the participants.» - David
> Farning
>