Jan Stolarek | 13 Dec 09:56 2012
Picon

How to start with GHC development?

Dear list,

I'm reposting my message from Haskell-cafe here since this seems like a more appropriate place to 
ask this question. I would like to learn about internals of GHC and contribute to its development 
in the future. I read a couple of papers that give a very general overview of GHC (chapter from 
AoS, papers about inliner and multicore support) and I'm thinking what direction should I pursue 
now. I got the GHC sources and started reading commentary on the wiki, but it seems that entry 
barrier is very high.

Aside from the problem of understanding the whole project organization itself, I also don't have 
background on implementation of functional languages. In fact I have a rather basic knowledge of 
compilers in general - I only took Stanford's online course on Compilers at Coursera. I was 
thinking that perhaps I should read SPJs "Implementing Functional Languages: a tutorial" or "The 
Implementation Of Functional Languages" book to learn more about theory? On the other hand I 
don't know if getting stuck in theory for next couple of months is a good idea.

I would greatly appreciate any advice and help.

Janek
Tim Watson | 13 Dec 10:24 2012
Picon

Re: How to start with GHC development?

I'm in a very similar position. Have some background knowledge and would love to contribute to ghc in the
future hut the barrier to entry is pretty high even though I've some familiarity with compiler theory,
have a long history with functional languages like ml/ocaml and a background in c that might mean
contributing to the runtime system in the future could be an area where I make myself useful.

More pointers to resources and research materials would be most helpful.

Tim

On 13 Dec 2012, at 08:56, Jan Stolarek <jan.stolarek <at> p.lodz.pl> wrote:

> Dear list,
> 
> I'm reposting my message from Haskell-cafe here since this seems like a more appropriate place to 
> ask this question. I would like to learn about internals of GHC and contribute to its development 
> in the future. I read a couple of papers that give a very general overview of GHC (chapter from 
> AoS, papers about inliner and multicore support) and I'm thinking what direction should I pursue 
> now. I got the GHC sources and started reading commentary on the wiki, but it seems that entry 
> barrier is very high.
> 
> Aside from the problem of understanding the whole project organization itself, I also don't have 
> background on implementation of functional languages. In fact I have a rather basic knowledge of 
> compilers in general - I only took Stanford's online course on Compilers at Coursera. I was 
> thinking that perhaps I should read SPJs "Implementing Functional Languages: a tutorial" or "The 
> Implementation Of Functional Languages" book to learn more about theory? On the other hand I 
> don't know if getting stuck in theory for next couple of months is a good idea.
> 
> I would greatly appreciate any advice and help.
> 
> Janek
(Continue reading)

Chris Nicholls | 13 Dec 10:41 2012
Picon

Re: How to start with GHC development?

Hi Jan,


The commentary is a great place to start to understand the GHC pipeline. In fact, GHC is an incredibly well documented project. However, being a place where a lot of new theory gets tested, its' also one of the densest and most intimidating (It's perhaps the only source code I've read that frequently  provides references to research papers as comments). So to some extent, to work on it probably does require bashing against some theory for a while. 

It also depends on the part of the pipeline that you're interested in. There's quite a difference between, say, the typechecker, and the codegen stages, and they require rather different knowledge. 

And this is where I'm a little stuck too. I have a reasonably good picture of GHC's pipeline, having worked on it for my master's thesis, and I'd like to get more involved and possibly even contribute something useful. But I'm not sure where to start. There is the 'Notes on proposed or in progress (but out of tree) GHC compiler features' but it's hard to tell what has been started, or even nearly completed (and none of them seem that 'entry level'). 

What's the best way to get started? Bug fixes? Writing a toy plugin? I don't have a huge amount of time to offer, but I would like to learn to help! 


On Thu, Dec 13, 2012 at 8:56 AM, Jan Stolarek <jan.stolarek <at> p.lodz.pl> wrote:
Dear list,

I'm reposting my message from Haskell-cafe here since this seems like a more appropriate place to
ask this question. I would like to learn about internals of GHC and contribute to its development
in the future. I read a couple of papers that give a very general overview of GHC (chapter from
AoS, papers about inliner and multicore support) and I'm thinking what direction should I pursue
now. I got the GHC sources and started reading commentary on the wiki, but it seems that entry
barrier is very high.

Aside from the problem of understanding the whole project organization itself, I also don't have
background on implementation of functional languages. In fact I have a rather basic knowledge of
compilers in general - I only took Stanford's online course on Compilers at Coursera. I was
thinking that perhaps I should read SPJs "Implementing Functional Languages: a tutorial" or "The
Implementation Of Functional Languages" book to learn more about theory? On the other hand I
don't know if getting stuck in theory for next couple of months is a good idea.

I would greatly appreciate any advice and help.

Janek

_______________________________________________
Glasgow-haskell-users mailing list
Glasgow-haskell-users <at> haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

_______________________________________________
Glasgow-haskell-users mailing list
Glasgow-haskell-users <at> haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Yuras Shumovich | 13 Dec 10:54 2012
Picon

Re: How to start with GHC development?

On Thu, 2012-12-13 at 09:41 +0000, Chris Nicholls wrote:

> What's the best way to get started? Bug fixes? Writing a toy plugin? I
> don't have a huge amount of time to offer, but I would like to learn to
> help!
> 

GHC bug sweep is the way I'm trying to start with:
http://hackage.haskell.org/trac/ghc/wiki/BugSweep
(have no idea whether it is the best way or not :) )

Thanks,
Yuras
Simon Marlow | 13 Dec 14:36 2012
Picon

Re: How to start with GHC development?

On 13/12/12 09:54, Yuras Shumovich wrote:
> On Thu, 2012-12-13 at 09:41 +0000, Chris Nicholls wrote:
>
>> What's the best way to get started? Bug fixes? Writing a toy plugin? I
>> don't have a huge amount of time to offer, but I would like to learn to
>> help!
>>
>
> GHC bug sweep is the way I'm trying to start with:
> http://hackage.haskell.org/trac/ghc/wiki/BugSweep
> (have no idea whether it is the best way or not :) )

The BugSweep was a great idea at the time, but I think it stalled.  Do 
feel free to carry on though!  I think it's a great way to learn about 
GHC, because it will send you off in random directions investigating things.

While there are lots of old bugs that are now fixed, or have duplicates, 
or aren't relevant, etc. there are also lots of old bugs that are still 
around because they aren't fixed because they aren't worth the effort. 
Still, making even *some* progress on any old bug is worthwhile, even if 
it is to point out a workaround that didn't exist before, or update the 
test case.  This will generate an email, and since there are people that 
read all the Trac emails, occasionally this prods someone else to make 
further progress or just close the ticket.

Cheers,
	Simon
Jan Stolarek | 13 Dec 11:34 2012
Picon

Re: How to start with GHC development?

> So to some extent, to work on it probably does require bashing against some theory for
> a while.
I don't mind that, but on the other hand I don't want to get stuck *only* in theory.

> It also depends on the part of the pipeline that you're interested in.
> There's quite a difference between, say, the typechecker, and the codegen
> stages, and they require rather different knowledge.
I'm aware of that. Right now I'd like to get a general understanding of the project. Later I was 
thinking about looking into concurrency and parallelism and perhaps at some point in the future 
at the typechecker (I guess this requires lots of theoretical background I don't have).

> And this is where I'm a little stuck too. I have a reasonably good picture
> of GHC's pipeline, having worked on it for my master's thesis, and I'd like
> to get more involved and possibly even contribute something useful. But I'm
> not sure where to start. 
Well, that's a bit discouraging - if you worked on it in master's thesis and still don't know 
where to start then it this is probably even harder than I suspected.

Janek
Chris Nicholls | 13 Dec 13:00 2012
Picon

Re: How to start with GHC development?


> And this is where I'm a little stuck too. I have a reasonably good picture
> of GHC's pipeline, having worked on it for my master's thesis, and I'd like
> to get more involved and possibly even contribute something useful. But I'm
> not sure where to start.
Well, that's a bit discouraging - if you worked on it in master's thesis and still don't know
where to start then it this is probably even harder than I suspected.

Sorry! That wasn't what I intended. I meant more that I have a reasonably, good idea of  *how* to work on ghc but not much of idea of *what* I can usefully do.  The work in my thesis turned out not to be that useful, and I'm not sure what to do next. It sounds like you've already got some ideas there though.

Perhaps I shouldn't have hijacked your question with a 'me too' but I suspect there are quite a lot of people who want to contribute, but don't really know where to start. 

Don't be discouraged, its absolutely possible to get your hands dirty without having to understand all the theory and the community is incredibly friendly. 
 As for getting an overview of the pipeline, I found the videos of the talks the Simons gave at the 2006 Hackathon helpful. Perhaps someone with more knowledge than me can comment on how relevant the content is after six years? 
_______________________________________________
Glasgow-haskell-users mailing list
Glasgow-haskell-users <at> haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Jan Stolarek | 13 Dec 13:56 2012
Picon

Re: How to start with GHC development?

> Sorry! That wasn't what I intended. 
I know, but still... :)

> I meant more that I have a reasonably,  good idea of  *how* to work on ghc 
That's what I'm trying to figure out right now. GHC build system looks very complicated and it 
seems I'll be spending weeks on understanding it before I can get to the compiler code itself.

> The work in my thesis turned out not to be that useful, and I'm not sure what to do next.
Speaking of your thesis - have you worked out internals of GHC by yourself or did you have someone 
with knowledge of GHC supervise your work?

> It sounds like you've already got some ideas there though.
Not really. I know what is interesting for me, but I don't know what exactly I could do.

> Perhaps I shouldn't have hijacked your question with a 'me too' 
You didn't hijack it :)

>  As for getting an overview of the pipeline, I found the videos of the
> talks the Simons gave at the 2006 Hackathon helpful. 
Thanks, I didn't know about these! I wonder if slides are available? The quality of video and 
sound is actually rather low.

Janek
Daniel Trstenjak | 13 Dec 14:40 2012
Picon

Re: How to start with GHC development?


Hi Jan,

On Thu, Dec 13, 2012 at 01:56:13PM +0100, Jan Stolarek wrote:
> That's what I'm trying to figure out right now. GHC build system looks very complicated and it 
> seems I'll be spending weeks on understanding it before I can get to the compiler code itself.

I think it's easier to have a fix point. Look for something you would
like to add to GHC, for a bug you would like to fix.

Starting at this fix point you can spread out and learn all the
necessary things to be able to implement it.

Sure, you will never know if you know enough to implement it in the most
fitting way for GHC. But I would consider this as part of the learning
process and the GHC developers will give you appropriate feedback.

I think that it's quite hard to learn such a big system like GHC in a top to
bottom manner and for myself it's easier to stay motivated if I've
a concrete task, the learning has a concrete reason.

Greetings,
Daniel
Chris Nicholls | 13 Dec 15:52 2012
Picon

Re: How to start with GHC development?


On Thu, Dec 13, 2012 at 12:56 PM, Jan Stolarek <jan.stolarek <at> p.lodz.pl> wrote:
Speaking of your thesis - have you worked out internals of GHC by yourself or did you have someone
with knowledge of GHC supervise your work?

I worked on my own, but the GHC folk were very helpful when I got stuck. That was quite a slow tactic in retrospect, I reckon I could have got much further if I'd had the bottle to ask more questions. One problem with the Haskell community is that everyone else seems so much smarter...
_______________________________________________
Glasgow-haskell-users mailing list
Glasgow-haskell-users <at> haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Daniel Trstenjak | 13 Dec 16:47 2012
Picon

Re: How to start with GHC development?


Hi Chris,

On Thu, Dec 13, 2012 at 02:52:37PM +0000, Chris Nicholls wrote:
> I worked on my own, but the GHC folk were very helpful when I got stuck.
> That was quite a slow tactic in retrospect, I reckon I could have got much
> further if I'd had the bottle to ask more questions. One problem with
> the Haskell community is that everyone else seems so much smarter...

Smart people won't think of you to be dumb because you know less,
because they know how little they know themself. ;)

Fear is such a drawback and the more reserved guys get easily
intimidated by the cocky guys, because they think that the
cocky guys really know what they're doing.

Greetings,
Daniel
Simon Peyton-Jones | 14 Dec 15:44 2012
Picon

RE: How to start with GHC development?

This thread has made it clear that we should do more to help people find a “way in” to GHC.  Here is what I have done:

 

·         Started a GHC Reading List page, giving background reading.  It’s just a start; there are many gaps. I would love it if you would all help fill it out.  It’s a wiki.

 

·         Amplified the Working on GHC page.  Again, please help make it more useful.  It’s a wiki.

 

What else would help?   If the answer is X, can anyone help do X?

 

Thanks

 

Simon

 

 

From: glasgow-haskell-users-bounces <at> haskell.org [mailto:glasgow-haskell-users-bounces <at> haskell.org] On Behalf Of Chris Nicholls
Sent: 13 December 2012 12:00
To: Jan Stolarek
Cc: GHC Users List
Subject: Re: How to start with GHC development?

 

 

> And this is where I'm a little stuck too. I have a reasonably good picture
> of GHC's pipeline, having worked on it for my master's thesis, and I'd like
> to get more involved and possibly even contribute something useful. But I'm
> not sure where to start.

Well, that's a bit discouraging - if you worked on it in master's thesis and still don't know
where to start then it this is probably even harder than I suspected.

 

Sorry! That wasn't what I intended. I meant more that I have a reasonably, good idea of  *how* to work on ghc but not much of idea of *what* I can usefully do.  The work in my thesis turned out not to be that useful, and I'm not sure what to do next. It sounds like you've already got some ideas there though.

 

Perhaps I shouldn't have hijacked your question with a 'me too' but I suspect there are quite a lot of people who want to contribute, but don't really know where to start. 

 

Don't be discouraged, its absolutely possible to get your hands dirty without having to understand all the theory and the community is incredibly friendly. 

 As for getting an overview of the pipeline, I found the videos of the talks the Simons gave at the 2006 Hackathon helpful. Perhaps someone with more knowledge than me can comment on how relevant the content is after six years? 

_______________________________________________
Glasgow-haskell-users mailing list
Glasgow-haskell-users <at> haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Gershom Bazerman | 14 Dec 16:56 2012
Picon

Re: How to start with GHC development?

On 12/14/12 9:44 AM, Simon Peyton-Jones wrote:
<!-- /* Font Definitions */ <at> font-face {font-family:Wingdings; panose-1:5 0 0 0 0 0 0 0 0 0;} <at> font-face {font-family:Wingdings; panose-1:5 0 0 0 0 0 0 0 0 0;} <at> font-face {font-family:Calibri; panose-1:2 15 5 2 2 2 4 3 2 4;} <at> font-face {font-family:Tahoma; panose-1:2 11 6 4 3 5 4 4 2 4;} <at> font-face {font-family:Verdana; panose-1:2 11 6 4 3 5 4 4 2 4;} /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {margin:0cm; margin-bottom:.0001pt; font-size:12.0pt; font-family:"Times New Roman","serif";} a:link, span.MsoHyperlink {mso-style-priority:99; color:blue; text-decoration:underline;} a:visited, span.MsoHyperlinkFollowed {mso-style-priority:99; color:purple; text-decoration:underline;} p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph {mso-style-priority:34; margin-top:0cm; margin-right:0cm; margin-bottom:0cm; margin-left:36.0pt; margin-bottom:.0001pt; font-size:12.0pt; font-family:"Times New Roman","serif";} span.EmailStyle17 {mso-style-type:personal-reply; font-family:"Verdana","sans-serif"; color:#1F497D;} .MsoChpDefault {mso-style-type:export-only; font-family:"Calibri","sans-serif"; mso-fareast-language:EN-US;} <at> page WordSection1 {size:612.0pt 792.0pt; margin:72.0pt 72.0pt 72.0pt 72.0pt;} div.WordSection1 {page:WordSection1;} /* List Definitions */ <at> list l0 {mso-list-id:1541632036; mso-list-type:hybrid; mso-list-template-ids:-1242241946 134807553 134807555 134807557 134807553 134807555 134807557 134807553 134807555 134807557;} <at> list l0:level1 {mso-level-number-format:bullet; mso-level-text:\F0B7; mso-level-tab-stop:none; mso-level-number-position:left; text-indent:-18.0pt; font-family:Symbol;} <at> list l0:level2 {mso-level-number-format:bullet; mso-level-text:o; mso-level-tab-stop:none; mso-level-number-position:left; text-indent:-18.0pt; font-family:"Courier New";} <at> list l0:level3 {mso-level-number-format:bullet; mso-level-text:\F0A7; mso-level-tab-stop:none; mso-level-number-position:left; text-indent:-18.0pt; font-family:Wingdings;} <at> list l0:level4 {mso-level-number-format:bullet; mso-level-text:\F0B7; mso-level-tab-stop:none; mso-level-number-position:left; text-indent:-18.0pt; font-family:Symbol;} <at> list l0:level5 {mso-level-number-format:bullet; mso-level-text:o; mso-level-tab-stop:none; mso-level-number-position:left; text-indent:-18.0pt; font-family:"Courier New";} <at> list l0:level6 {mso-level-number-format:bullet; mso-level-text:\F0A7; mso-level-tab-stop:none; mso-level-number-position:left; text-indent:-18.0pt; font-family:Wingdings;} <at> list l0:level7 {mso-level-number-format:bullet; mso-level-text:\F0B7; mso-level-tab-stop:none; mso-level-number-position:left; text-indent:-18.0pt; font-family:Symbol;} <at> list l0:level8 {mso-level-number-format:bullet; mso-level-text:o; mso-level-tab-stop:none; mso-level-number-position:left; text-indent:-18.0pt; font-family:"Courier New";} <at> list l0:level9 {mso-level-number-format:bullet; mso-level-text:\F0A7; mso-level-tab-stop:none; mso-level-number-position:left; text-indent:-18.0pt; font-family:Wingdings;} ol {margin-bottom:0cm;} ul {margin-bottom:0cm;} -->

This thread has made it clear that we should do more to help people find a “way in” to GHC.  Here is what I have done:

 

·         Started a GHC Reading List page, giving background reading.  It’s just a start; there are many gaps. I would love it if you would all help fill it out.  It’s a wiki.

 

·         Amplified the Working on GHC page.  Again, please help make it more useful.  It’s a wiki.

 

What else would help?   If the answer is X, can anyone help do X?

 


Something else that I think would be useful is a long-term 'wishlist' for ghc, consisting of modest goals from an engineering standpoint. My sense is that these involve factoring subsystems to make them more independent and modular, as well as moving towards parallelizable code, and perhaps the notion of ghc-as-a-service so that one could have a long-running compiler process able to perform individual parts of a pipeline on-demand. The scheme of what needs to change and overall architecture needs to be determined by people with very good knowledge of the compiler as a whole. But individual tasks and areas could be tackled by others wishing to contribute.

I'd also like more of a sense of what work remains to be done to decouple ghci from ghc, such that we could pleasantly produce other ghci-like environments using the same API, and perhaps ultimately treat desktop ghci as one means among many of interacting with ghc-as-a-library. (i.e. how can we get to 'cabal install ghci').

I'm also curious about the state of the plugins pipeline, how comfortable we are with it, and what could potentially be improved (and how that relates to having ghc potentially process externally generated or transformed core).

This is to say, outside of the list of bugs on trac, and outside of potentially researchy new features, what are the sorts of things we would like to make GHC a more dependable, efficient, maintainable, flexible piece of software? The things I'm suggesting a focus on are things that I remember discussion of along these lines. They're perhaps not where core GHC developers feel effort should be focused. But where then *should* it be?

A sense of this might help interested contributors to decide where to direct their efforts.

Thanks,
Gershom
_______________________________________________
Glasgow-haskell-users mailing list
Glasgow-haskell-users <at> haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
wren ng thornton | 15 Dec 07:14 2012

Re: How to start with GHC development?

On 12/14/12 10:56 AM, Gershom Bazerman wrote:
> On 12/14/12 9:44 AM, Simon Peyton-Jones wrote:
>>
>> This thread has made it clear that we should do more to help people
>> find a "way in" to GHC.  Here is what I have done:
>>
>> ·Started a GHC Reading List page
>> <http://hackage.haskell.org/trac/ghc/wiki/ReadingList>, giving
>> background reading.  It's just a start; there are many gaps. I would
>> love it if you would all help fill it out.  It's a wiki.
>>
>> ·Amplified the Working on GHC
>> <http://hackage.haskell.org/trac/ghc/wiki/WorkingConventions> page.
>> Again, please help make it more useful. It's a wiki.
>>
>> What else would help?   If the answer is X, can anyone help do X?
>
> Something else that I think would be useful is a long-term 'wishlist'
> for ghc, consisting of modest goals from an engineering standpoint.
> [...]
> A sense of this might help interested contributors to decide where to
> direct their efforts.

I agree wholeheartedly. A lot of times fixing bugs requires one to 
already have a good idea how everything is structured. And research 
features require one to be interested/able to do research. In my 
experience with other projects, the more mid-term engineering tasks tend 
to be a great place to get started.

Now, I wonder, how should we go about setting this up? Wiki pages are 
good for the more long-term or more visionary sorts of goals, but 
they're too liable to get out of date for the short- and mid-term 
engineering goals. The bug tracker already has a listing for "tasks" 
rather than "bugs"[1]. Perhaps we should be advertising that more? Or 
should we tweak the tracker settings to provide a better venue for these 
sorts of tasks? The latter would make it easy to have a wiki page on 
teaching folks how to search for tasks appropriate to their comfort level...

[1] http://hackage.haskell.org/trac/ghc/report/2

--

-- 
Live well,
~wren
Jan Stolarek | 14 Dec 17:11 2012
Picon

Re: How to start with GHC development?

Dnia piątek, 14 grudnia 2012, Simon Peyton-Jones napisał:
> This thread has made it clear that we should do more to help people find a
> "way in" to GHC.  
Thanks for taking intrest in our efforts Simon :)

After three days of reading through the wiki I came up with some conclusions. The thing that seems 
to be most difficult for me is that wiki has a tree structure: every page links to many other 
pages. For an unexperienced person it is hard to prioritize information. What do I need to know 
first and what can be left out for later? It would be good if this knowledge was somehow 
linearized. I found Hackathon lectures to be slightly helpful. Low quality of sound and video 
makes it difficult to learn from these particular videos but I think the idea itself is great. 
Perhaps it would be possible to create some introductory videos & presentations that would 
introduce GHC's build system and architecture? I know this would be a lot of effort but perhaps 
it would pay off? I'm thinking something like Coursera class. I would love to see such a thing!

Another thing I found difficult is that wiki gives different information on getting the sources. 
This page: http://hackage.haskell.org/trac/ghc/wiki/Building/GettingTheSources gives one set of 
commands one has to issue to check out the source code, but this page: 
http://hackage.haskell.org/trac/ghc/wiki/WorkingConventions/Git gives a different set of 
instructions. For a person that does not know the structure of repositories and the build system 
itself this is confusing. I think that at this point people expect - at least I expected - a list 
of commands to run in order to get the sources. Having two different possibilities is confusing 
as one needs to figure out what these commands actually do. With my current knowledge - or lack 
of thereof - I'm unable to correct this on the wiki

> *         Started a GHC Reading List
> page<http://hackage.haskell.org/trac/ghc/wiki/ReadingList>, giving
> background reading.  It's just a start; there are many gaps. I would love
> it if you would all help fill it out.  It's a wiki.
I read some of the listed papers and while they definitely gave me a nice overview of the compiler 
I still feel a bit helpless when faced with the actual code. Simon, you listed your books 
in "General background" section. I don't doubt they are interesting (I only started reading fisrt 
of the books) and if one wants to impement functional languages they are a must. But they are 
over 700 pages in total - is reading them an optimal way of getting into GHC development? Well, 
if the answer is 'yes' then I'm off to reading, but perhaps there are easier approaches?
Anyway, it's a great idea to have a list of papers and books about architecture of GHC, so I'm 
going to fill out DPH section. BTW. GpH page has a nicely prepared list of papers - GHC deserves 
the same.

To sum up: I think that wiki has almost all the knowledge needed to get started, but that 
knowledge has to be presented to the beginners in a form of some tutorial, presentations or 
anything that is linear and guides people by hand.

I hope that was constructive.

Janek

>
>
> *         Amplified the Working on
> GHC<http://hackage.haskell.org/trac/ghc/wiki/WorkingConventions> page. 
> Again, please help make it more useful.  It's a wiki.
>
> What else would help?   If the answer is X, can anyone help do X?
>
> Thanks
>
> Simon
>
>
> From: glasgow-haskell-users-bounces <at> haskell.org
> [mailto:glasgow-haskell-users-bounces <at> haskell.org] On Behalf Of Chris
> Nicholls Sent: 13 December 2012 12:00
> To: Jan Stolarek
> Cc: GHC Users List
> Subject: Re: How to start with GHC development?
>
> > And this is where I'm a little stuck too. I have a reasonably good
> > picture of GHC's pipeline, having worked on it for my master's thesis,
> > and I'd like to get more involved and possibly even contribute something
> > useful. But I'm not sure where to start.
>
> Well, that's a bit discouraging - if you worked on it in master's thesis
> and still don't know where to start then it this is probably even harder
> than I suspected.
>
> Sorry! That wasn't what I intended. I meant more that I have a reasonably,
> good idea of  *how* to work on ghc but not much of idea of *what* I can
> usefully do.  The work in my thesis turned out not to be that useful, and
> I'm not sure what to do next. It sounds like you've already got some ideas
> there though.
>
> Perhaps I shouldn't have hijacked your question with a 'me too' but I
> suspect there are quite a lot of people who want to contribute, but don't
> really know where to start.
>
> Don't be discouraged, its absolutely possible to get your hands dirty
> without having to understand all the theory and the community is incredibly
> friendly. As for getting an overview of the pipeline, I found the videos of
> the talks the Simons gave at the 2006 Hackathon helpful. Perhaps someone
> with more knowledge than me can comment on how relevant the content is
> after six years?

_______________________________________________
Glasgow-haskell-users mailing list
Glasgow-haskell-users <at> haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Ian Lynagh | 14 Dec 17:32 2012

Re: How to start with GHC development?

On Fri, Dec 14, 2012 at 02:44:19PM +0000, Simon Peyton-Jones wrote:
> This thread has made it clear that we should do more to help people find a "way in" to GHC.

I think the main problem is that it's a very broad question. The answer
to "how should I get started" would be completely different for if you
wanted to implement a type system extension, port GHC to a new platform,
or fix a bug in ghci (just to pick 3 examples; I think that trying to
enumerate on a wiki page all the possible ways in which you could get
started would be infeasible).

I just had a quick look at how easy things are to find from
    http://hackage.haskell.org/trac/ghc/
and they look OK to me. One thing I wonder is how understandable the
"commentary" link is to someone who doesn't already know what it is.
Would something like "A developer's guide to GHC" be better? That might
be too long for the sidebar, though.

Thanks
Ian
Jan Stolarek | 15 Dec 09:41 2012
Picon

Re: How to start with GHC development?

Dnia piątek, 14 grudnia 2012, Ian Lynagh napisał:
> I think the main problem is that it's a very broad question. The answer
> to "how should I get started" would be completely different for if you
> wanted to implement a type system extension, port GHC to a new platform,
> or fix a bug in ghci
All three task have one thing in common though - the GHC build system. Project sources are over 
500MB, there are many subrepositories, build scripts etc - this is really intimidating at first 
and I guess that seeing all that was a bit discouraging. Now that I know the compiler itself is 
about 9MB it doesn't look that bad :) Wiki indeed seems to have all the information required. 
It's just that it would be good to give beginners a _precise_ list of things they should read to 
begin hacking.

Janek

_______________________________________________
Glasgow-haskell-users mailing list
Glasgow-haskell-users <at> haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Ian Lynagh | 15 Dec 14:58 2012

Re: How to start with GHC development?

On Sat, Dec 15, 2012 at 09:41:12AM +0100, Jan Stolarek wrote:
> Dnia piątek, 14 grudnia 2012, Ian Lynagh napisał:
> > I think the main problem is that it's a very broad question. The answer
> > to "how should I get started" would be completely different for if you
> > wanted to implement a type system extension, port GHC to a new platform,
> > or fix a bug in ghci
> All three task have one thing in common though - the GHC build system. Project sources are over 
> 500MB, there are many subrepositories, build scripts etc - this is really intimidating at first 
> and I guess that seeing all that was a bit discouraging. Now that I know the compiler itself is 
> about 9MB it doesn't look that bad :) Wiki indeed seems to have all the information required. 
> It's just that it would be good to give beginners a _precise_ list of things they should read to 
> begin hacking.

OK, so how can we improve it?

I've just made one change that might help: On
    http://hackage.haskell.org/trac/ghc/wiki/Building
"Working Conventions" was mentioned in the opening paragraph of
as something to "also see", which perhaps made it sound
like it should be read first, which I don't think was the intention.
I've moved it to the bottom of the list instead, so you won't see it
before going through "Setting up your system for building GHC", "Getting
the sources" and "Getting started with the build system".

Thanks
Ian

_______________________________________________
Glasgow-haskell-users mailing list
Glasgow-haskell-users <at> haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Jan Stolarek | 15 Dec 15:46 2012
Picon

Re: How to start with GHC development?

> OK, so how can we improve it?
First of all I think that materials in the wiki are well organized for people who already have 
some knowledge and only need to learn about particular things they don't yet know. In this case I 
think it is fine to have wiki pages connected between one another so they form cycles and don't 
have a particular ordering.

However as a beginner I'd expect that pages are ordered in a linear way (sorry for repeating 
myself, but I think this is important), that every concept is explained in only one place and 
that there are no links in the text (even at the end - if the text is structured lineary than I 
don't need to decide what to read next, I just go to next section/chapter). This is of course a 
perfect situation that seems to be achievable in a book, but probably not in a wiki (wikibook 
could be a middle ground?).

Right now there are two pages (I know of...) that instruct how to get the sources:
http://hackage.haskell.org/trac/ghc/wiki/Building/GettingTheSources
http://hackage.haskell.org/trac/ghc/wiki/WorkingConventions/Git
They list two different places containing the sources: github and darcs.haskell.org. Are these 
equivalent? A couple of months ago I played with GHC sources: the ones from github were about 
17MB, the ones from haskell.org were much larger (85MB IIRC) so I guess that these might not be 
exact mirrors.
Second page gives a nice description of how to set up build trees, but it duplicates information 
from the first page (checking out the sources). First page also tells to make a local clone of 
the repo but it does not explain why such a clone should be made (the second page does).
I think that merging these two pages into one might be a good idea. 
I would definitely like more details on the git workflow, especially the branches. Right now I see 
about 40 branches - why so many? Am I required to create my own branch when I start hacking or 
should I just go with master?

It seems that many informations in the wiki are duplicated. There are two pages about 
repositories:
http://hackage.haskell.org/trac/ghc/wiki/Repositories
http://hackage.haskell.org/trac/ghc/wiki/WorkingConventions/Repositories
(after reading the first one source tree started to make much more sense - this is one of the 
informations *I* would like to get first).

In general I think that for beginers it would be good if the wiki had a form of a book divided 
into chapters. I only wonder if it is possible to adjust the wiki for the newbies and at the same 
time keep it useful for more experienced developers.

Janek
Simon Marlow | 18 Dec 14:59 2012
Picon

Re: How to start with GHC development?

On 15/12/12 14:46, Jan Stolarek wrote:
>> OK, so how can we improve it?
> First of all I think that materials in the wiki are well organized for people who already have
> some knowledge and only need to learn about particular things they don't yet know. In this case I
> think it is fine to have wiki pages connected between one another so they form cycles and don't
> have a particular ordering.
>
> However as a beginner I'd expect that pages are ordered in a linear way (sorry for repeating
> myself, but I think this is important), that every concept is explained in only one place and
> that there are no links in the text (even at the end - if the text is structured lineary than I
> don't need to decide what to read next, I just go to next section/chapter). This is of course a
> perfect situation that seems to be achievable in a book, but probably not in a wiki (wikibook
> could be a middle ground?).
>
> Right now there are two pages (I know of...) that instruct how to get the sources:
> http://hackage.haskell.org/trac/ghc/wiki/Building/GettingTheSources
> http://hackage.haskell.org/trac/ghc/wiki/WorkingConventions/Git

So the first page here tells you how to get a single source tree so that 
you can build it.  The second page tells you how to create a 2-tree 
setup for use with GHC's validate; the latter is aimed at people doing 
lots of GHC development (which is why it's under WorkingConventions). 
Both scenarios are important, and I think it's more important to deal 
with the simple case first which is why it is right near the start of 
the Building Guide.

Does that help?  Is there something we could add somewhere that would 
make it clearer?

> It seems that many informations in the wiki are duplicated. There are two pages about
> repositories:
> http://hackage.haskell.org/trac/ghc/wiki/Repositories
> http://hackage.haskell.org/trac/ghc/wiki/WorkingConventions/Repositories
> (after reading the first one source tree started to make much more sense - this is one of the
> informations *I* would like to get first).

The first page lists the repositories and where the upstreams and 
mirrors are.  The second page contains the conventions for working on 
other repositories (which is why it's under WorkingConventions).

> In general I think that for beginers it would be good if the wiki had a form of a book divided
> into chapters. I only wonder if it is possible to adjust the wiki for the newbies and at the same
> time keep it useful for more experienced developers.

The nice thing about a wiki is that you don't have to move content 
around, you can just make new contents pages that contain whatever 
organisation you want. So maybe what you want is a separate page that 
links to things to read in a particular order?

Cheers,
	Simon
Simon Peyton-Jones | 18 Dec 16:51 2012
Picon

RE: How to start with GHC development?

| > It seems that many informations in the wiki are duplicated. There are
| > two pages about
| > repositories:
| > http://hackage.haskell.org/trac/ghc/wiki/Repositories
| > http://hackage.haskell.org/trac/ghc/wiki/WorkingConventions/Repositori
| > es (after reading the first one source tree started to make much more
| > sense - this is one of the informations *I* would like to get first).
| 
| The first page lists the repositories and where the upstreams and
| mirrors are.  The second page contains the conventions for working on
| other repositories (which is why it's under WorkingConventions).

Simon, I don't find that a clear distinction. Looking at the two, I'm a bit confused too!

* The lists on WorkingConventions/Repositories duplicates the table in Repositories.
* I believe that perhaps WorkingConventions/Repositories is solely concerned with how to *modify* a
library; it opening para says as much.  Fine; but it shouldn't duplicate the info.

Maybe the table could do with a column saying "GHC" or "Upstream" to specify the "how to modify" convention? 
(I wish the table could somehow be narrower.  And that the library name was the first column.)  Perhaps the
master table can look like this:

What           GHC repo location                Upstream repo exists?
               http://darcs.haskell.org/

GHC            ghc.git
ghc-tarballs   ghc-tarballs.git
...etc...
binary         binary.git                       YES
...etc...

Then we can deal with the complexities of upstream repos in another page.  I think that might put the info in a
way that's easier to grok.  I can do it if Simon and Ian agree; or Ian could.

Simon
Simon Marlow | 18 Dec 17:40 2012
Picon

Re: How to start with GHC development?

On 18/12/12 15:51, Simon Peyton-Jones wrote:
> | > It seems that many informations in the wiki are duplicated. There are
> | > two pages about
> | > repositories:
> | > http://hackage.haskell.org/trac/ghc/wiki/Repositories
> | > http://hackage.haskell.org/trac/ghc/wiki/WorkingConventions/Repositori
> | > es (after reading the first one source tree started to make much more
> | > sense - this is one of the informations *I* would like to get first).
> |
> | The first page lists the repositories and where the upstreams and
> | mirrors are.  The second page contains the conventions for working on
> | other repositories (which is why it's under WorkingConventions).
>
> Simon, I don't find that a clear distinction. Looking at the two, I'm a bit confused too!

So Repositories is "what repositories there are", and 
WorkingConventions/Repositories is "how to work on them".  Isn't that a 
clear distinction?

> * The lists on WorkingConventions/Repositories duplicates the table in Repositories.

There are two separate workflows, so we have to say which libraries each 
workflow applies to.  I'd be fine with merging this info with the other 
table - it might be slightly more awkward having the info on a separate 
page, but there would be only one list of repositories.

> * I believe that perhaps WorkingConventions/Repositories is solely concerned with how to *modify* a
library; it opening para says as much.  Fine; but it shouldn't duplicate the info.

Right.

> Maybe the table could do with a column saying "GHC" or "Upstream" to specify the "how to modify"
convention?  (I wish the table could somehow be narrower.  And that the library name was the first column.) 
Perhaps the master table can look like this:
>
> What           GHC repo location                Upstream repo exists?
>                 http://darcs.haskell.org/
>
> GHC            ghc.git
> ghc-tarballs   ghc-tarballs.git
> ...etc...
> binary         binary.git                       YES
> ...etc...
>
> Then we can deal with the complexities of upstream repos in another page.  I think that might put the info in a
way that's easier to grok.  I can do it if Simon and Ian agree; or Ian could.

Ok by me.

Cheers,
	Simon
Simon Peyton-Jones | 19 Dec 13:40 2012
Picon

RE: How to start with GHC development?

Simon, Ian

OK, I have spent a much longer time than I intended editing our wiki pages.  Can you look at

http://hackage.haskell.org/trac/ghc/wiki/Repositories
http://hackage.haskell.org/trac/ghc/wiki/Repositories/Upstream

The former is supposed to be the summary; the latter focuses on upstream repos.  

They are supposed to subsume the current
WorkingConventions/Repositories
which I have not yet deleted.

I'm not 100% sure of the yes/no in the columns.

Thanks

Simon

| -----Original Message-----
| From: Simon Marlow [mailto:marlowsd <at> gmail.com]
| Sent: 18 December 2012 16:40
| To: Simon Peyton-Jones
| Cc: Jan Stolarek; Ian Lynagh; glasgow-haskell-users <at> haskell.org
| Subject: Re: How to start with GHC development?
| 
| On 18/12/12 15:51, Simon Peyton-Jones wrote:
| > | > It seems that many informations in the wiki are duplicated. There
| > | > are two pages about
| > | > repositories:
| > | > http://hackage.haskell.org/trac/ghc/wiki/Repositories
| > | > http://hackage.haskell.org/trac/ghc/wiki/WorkingConventions/Reposi
| > | > tori es (after reading the first one source tree started to make
| > | > much more sense - this is one of the informations *I* would like
| > | > to get first).
| > |
| > | The first page lists the repositories and where the upstreams and
| > | mirrors are.  The second page contains the conventions for working
| > | on other repositories (which is why it's under WorkingConventions).
| >
| > Simon, I don't find that a clear distinction. Looking at the two, I'm
| a bit confused too!
| 
| So Repositories is "what repositories there are", and
| WorkingConventions/Repositories is "how to work on them".  Isn't that a
| clear distinction?
| 
| > * The lists on WorkingConventions/Repositories duplicates the table in
| Repositories.
| 
| There are two separate workflows, so we have to say which libraries each
| workflow applies to.  I'd be fine with merging this info with the other
| table - it might be slightly more awkward having the info on a separate
| page, but there would be only one list of repositories.
| 
| > * I believe that perhaps WorkingConventions/Repositories is solely
| concerned with how to *modify* a library; it opening para says as much.
| Fine; but it shouldn't duplicate the info.
| 
| Right.
| 
| > Maybe the table could do with a column saying "GHC" or "Upstream" to
| specify the "how to modify" convention?  (I wish the table could somehow
| be narrower.  And that the library name was the first column.)  Perhaps
| the master table can look like this:
| >
| > What           GHC repo location                Upstream repo exists?
| >                 http://darcs.haskell.org/
| >
| > GHC            ghc.git
| > ghc-tarballs   ghc-tarballs.git
| > ...etc...
| > binary         binary.git                       YES
| > ...etc...
| >
| > Then we can deal with the complexities of upstream repos in another
| page.  I think that might put the info in a way that's easier to grok.
| I can do it if Simon and Ian agree; or Ian could.
| 
| Ok by me.
| 
| Cheers,
| 	Simon
Simon Marlow | 19 Dec 14:05 2012
Picon

Re: How to start with GHC development?

I like it, the table is much easier to understand.

I slightly preferred having the "how to modify" material on a separate 
page and under WorkingConventions, but as long as we link to these pages 
from the right places under WorkingConventions it should be ok.

Cheers,
	Simon

On 19/12/12 12:40, Simon Peyton-Jones wrote:
> Simon, Ian
>
> OK, I have spent a much longer time than I intended editing our wiki pages.  Can you look at
>
> http://hackage.haskell.org/trac/ghc/wiki/Repositories
> http://hackage.haskell.org/trac/ghc/wiki/Repositories/Upstream
>
> The former is supposed to be the summary; the latter focuses on upstream repos.
>
> They are supposed to subsume the current
> WorkingConventions/Repositories
> which I have not yet deleted.
>
> I'm not 100% sure of the yes/no in the columns.
>
>
> Thanks
>
> Simon
>
> | -----Original Message-----
> | From: Simon Marlow [mailto:marlowsd <at> gmail.com]
> | Sent: 18 December 2012 16:40
> | To: Simon Peyton-Jones
> | Cc: Jan Stolarek; Ian Lynagh; glasgow-haskell-users <at> haskell.org
> | Subject: Re: How to start with GHC development?
> |
> | On 18/12/12 15:51, Simon Peyton-Jones wrote:
> | > | > It seems that many informations in the wiki are duplicated. There
> | > | > are two pages about
> | > | > repositories:
> | > | > http://hackage.haskell.org/trac/ghc/wiki/Repositories
> | > | > http://hackage.haskell.org/trac/ghc/wiki/WorkingConventions/Reposi
> | > | > tori es (after reading the first one source tree started to make
> | > | > much more sense - this is one of the informations *I* would like
> | > | > to get first).
> | > |
> | > | The first page lists the repositories and where the upstreams and
> | > | mirrors are.  The second page contains the conventions for working
> | > | on other repositories (which is why it's under WorkingConventions).
> | >
> | > Simon, I don't find that a clear distinction. Looking at the two, I'm
> | a bit confused too!
> |
> | So Repositories is "what repositories there are", and
> | WorkingConventions/Repositories is "how to work on them".  Isn't that a
> | clear distinction?
> |
> | > * The lists on WorkingConventions/Repositories duplicates the table in
> | Repositories.
> |
> | There are two separate workflows, so we have to say which libraries each
> | workflow applies to.  I'd be fine with merging this info with the other
> | table - it might be slightly more awkward having the info on a separate
> | page, but there would be only one list of repositories.
> |
> | > * I believe that perhaps WorkingConventions/Repositories is solely
> | concerned with how to *modify* a library; it opening para says as much.
> | Fine; but it shouldn't duplicate the info.
> |
> | Right.
> |
> | > Maybe the table could do with a column saying "GHC" or "Upstream" to
> | specify the "how to modify" convention?  (I wish the table could somehow
> | be narrower.  And that the library name was the first column.)  Perhaps
> | the master table can look like this:
> | >
> | > What           GHC repo location                Upstream repo exists?
> | >                 http://darcs.haskell.org/
> | >
> | > GHC            ghc.git
> | > ghc-tarballs   ghc-tarballs.git
> | > ...etc...
> | > binary         binary.git                       YES
> | > ...etc...
> | >
> | > Then we can deal with the complexities of upstream repos in another
> | page.  I think that might put the info in a way that's easier to grok.
> | I can do it if Simon and Ian agree; or Ian could.
> |
> | Ok by me.
> |
> | Cheers,
> | 	Simon
>
Ian Lynagh | 5 Jan 16:09 2013

Re: How to start with GHC development?

On Wed, Dec 19, 2012 at 12:40:50PM +0000, Simon Peyton-Jones wrote:
> Simon, Ian
> 
> OK, I have spent a much longer time than I intended editing our wiki pages.  Can you look at
> 
> http://hackage.haskell.org/trac/ghc/wiki/Repositories
> http://hackage.haskell.org/trac/ghc/wiki/Repositories/Upstream
> 
> The former is supposed to be the summary; the latter focuses on upstream repos.  
> 
> They are supposed to subsume the current
> WorkingConventions/Repositories
> which I have not yet deleted.
> 
> I'm not 100% sure of the yes/no in the columns.

Look fine to me. I've updated the tables.

Thanks
Ian
Jan Stolarek | 18 Dec 22:12 2012
Picon

Re: How to start with GHC development?

> So the first page here tells you how to get a single source tree so that
> you can build it.  The second page tells you how to create a 2-tree
> setup for use with GHC's validate; the latter is aimed at people doing
> lots of GHC development (which is why it's under WorkingConventions).
I think that wiki doesn't make distinction between (setup for beginners)/(setup for regular 
developers) clear - see below. It would be good to give some motivation behind each of these 
approaches as right now it seems there are two different instructions to do the same thing.

> (...) I think it's more important to deal
> with the simple case first which is why it is right near the start of
> the Building Guide.
Please notice that one of the first things one reads on "Getting the GHC Sources" page is:

"The first thing to do is install  git. And then read Git Working Conventions for instructions on 
how to use Git with GHC development."

It suggests quite explicitly to read the conventions before proceeding with the rest of the page! 
Personally I find information about dual-tree setup very useful. Assuming one knows how to use 
git it is not as overcomplicated setup.

> The first page lists the repositories and where the upstreams and
> mirrors are.  The second page contains the conventions for working on
> other repositories (which is why it's under WorkingConventions).
I would find it more intuitive to have one page with couple of sections: 1) Structure of 
repositories; 2) Working with external repositories; - or something like that.

> The nice thing about a wiki is that you don't have to move content
> around, you can just make new contents pages that contain whatever
> organisation you want. So maybe what you want is a separate page that
> links to things to read in a particular order?
No, I think not - the main page of "Building Guide" is already such a page. The list of topics is 
very nice and I don't feel it needs improving. The problems begin when one start reading through 
particular topics.

I must add that despite my endless comments on how Commentary could be improved I consider GHC to 
be the best documented project I've seen so far. You did a really fantastic work with putting 
your knowledge on the wiki!

Janek
Simon Peyton-Jones | 19 Dec 13:42 2012
Picon

RE: How to start with GHC development?


| > The first page lists the repositories and where the upstreams and
| > mirrors are.  The second page contains the conventions for working on
| > other repositories (which is why it's under WorkingConventions).
| I would find it more intuitive to have one page with couple of sections:
| 1) Structure of
| repositories; 2) Working with external repositories; - or something like
| that.

Try this:

http://hackage.haskell.org/trac/ghc/wiki/Repositories

Simon
Jan Stolarek | 19 Dec 15:47 2012
Picon

Re: How to start with GHC development?

> Try this:
>
> http://hackage.haskell.org/trac/ghc/wiki/Repositories
Very nice! WorkingConventions/Repositories still duplicates some information but that's not a big 
problem since neither Repositories nor Repositories/Upstream link to it. Perhaps it would be good 
to give a short explanation about darcs mirrors? They are in the diagram in 
Repositories/Upstream. I think there was some explanation on the previous version of these pages. 

Janek
Simon Peyton-Jones | 19 Dec 16:02 2012
Picon

RE: How to start with GHC development?

| > http://hackage.haskell.org/trac/ghc/wiki/Repositories
| Very nice! WorkingConventions/Repositories still duplicates some
| information 

The latter page should disappear; it is 100% covered by the other two.  If you can find any links to it, please
edit them into links to the new pages.

| Repositories/Upstream link to it. Perhaps it would be good to give a
| short explanation about darcs mirrors? They are in the diagram in
| Repositories/Upstream. I think there was some explanation on the
| previous version of these pages.

I think I included everything that was there before. You can look back in time by clicking the "History" link
at the top of each page.

Simon

Gmane