Niklas Hambüchen | 29 Apr 08:00 2013

GSoC proposal: Haskell AST-based refactoring and API upgrading tool

I would like to propose the development of source code refactoring tool
that operates on Haskell source code ASTs and lets you formulate rewrite
rules written in Haskell.

Objective
---------

The goal is to make refactorings easier and allow global code changes
that might be incredibly tedious to do in a non-automated way.
By making these transformations convenient, we can make it easier to
maintain clean code, add new features or clean up leftovers faster, and
reduce the fear and effort to upgrade to newer versions of packages and
APIs.

Transformations
---------------

First, here are a few operations you would use this tool for. Some of
them are common operations you would also do in other programming
languages, some are more specific to Haskell.

* Changing all occurrences of "import Prelude hiding (catch)" to "import
qualified Control.Exception as E"

* Replacing all uses of a function with that function being imported
qualified or the other way around

* Adding a field to data constructor a record, setting user-supplied
defaults for construction and destruction:

(Continue reading)

Malcolm Wallace | 29 Apr 09:36 2013

Re: GSoC proposal: Haskell AST-based refactoring and API upgrading tool


On 29 Apr 2013, at 07:00, Niklas Hambüchen wrote:

> I would like to propose the development of source code refactoring tool
> that operates on Haskell source code ASTs and lets you formulate rewrite
> rules written in Haskell.

Seen this?
http://www.haskell.org/haskellwiki/HaRe

Regards,
    Malcolm
Niklas Hambüchen | 29 Apr 09:59 2013

Re: GSoC proposal: Haskell AST-based refactoring and API upgrading tool

Hello Malcolm,

no, I had indeed not seen this! Thanks for the link.

It goes very much in the direction I was thinking of, but it does not
seem to maintained and does not cabal install either.

It also seems very much focused on interactive editor integration as
compared to written-out transformations.

Do you know to what extent they have built and a modification-friendly AST?
Also, do you know if the people involved in this are still active in the
community and interested in working further in this direction?

Thanks
Niklas

On 29/04/13 15:36, Malcolm Wallace wrote:
> 
> On 29 Apr 2013, at 07:00, Niklas Hambüchen wrote:
> 
>> I would like to propose the development of source code refactoring tool
>> that operates on Haskell source code ASTs and lets you formulate rewrite
>> rules written in Haskell.
> 
> 
> Seen this?
> http://www.haskell.org/haskellwiki/HaRe
> 
> Regards,
(Continue reading)

Alexander Kjeldaas | 29 Apr 10:32 2013
Picon

Re: GSoC proposal: Haskell AST-based refactoring and API upgrading tool

There is another aspect to this:  How do you get maintainers to apply the patches?  How should hackage be changed to accomodate large-scale refactorings?

There was a discussion on this mailing list related to build regressions on GHC 7.6 last year. 

All of the regressions could be fixed using perl regexps, and it was only a few hours of work, much less than the work involved in the discussion itself.  I downloaded all of hackage and did the fixes using perl.

http://www.haskell.org/pipermail/haskell-cafe/2012-August/103155.html

However, without the community infrastructure to actually apply the patches, the problem is not solved.

I think this is mainly a community/organizational issue.  Refactoring is not really the problem, but of course better refactoring abilities are good.

Alexander


On Mon, Apr 29, 2013 at 9:59 AM, Niklas Hambüchen <mail <at> nh2.me> wrote:
Hello Malcolm,

no, I had indeed not seen this! Thanks for the link.

It goes very much in the direction I was thinking of, but it does not
seem to maintained and does not cabal install either.

It also seems very much focused on interactive editor integration as
compared to written-out transformations.

Do you know to what extent they have built and a modification-friendly AST?
Also, do you know if the people involved in this are still active in the
community and interested in working further in this direction?

Thanks
Niklas

On 29/04/13 15:36, Malcolm Wallace wrote:
>
> On 29 Apr 2013, at 07:00, Niklas Hambüchen wrote:
>
>> I would like to propose the development of source code refactoring tool
>> that operates on Haskell source code ASTs and lets you formulate rewrite
>> rules written in Haskell.
>
>
> Seen this?
> http://www.haskell.org/haskellwiki/HaRe
>
> Regards,
>     Malcolm
>

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Roman Cheplyaka | 29 Apr 09:55 2013

Re: GSoC proposal: Haskell AST-based refactoring and API upgrading tool

* Niklas Hambüchen <mail <at> nh2.me> [2013-04-29 14:00:23+0800]
> I would like to propose the development of source code refactoring tool
> that operates on Haskell source code ASTs and lets you formulate rewrite
> rules written in Haskell.

Hi Niklas,

This is a great idea. I talked about it at HIW last year[1] and have
been working on it since then.

[1]: http://www.youtube.com/watch?v=Ae-6uIMQPmU

1. What you call a "full-source" AST is already present in
   haskell-src-exts (under "Annotated" subtree). It /almost/ satisfies

     pretty .  parse = id,

   (one thing it fails to do is to preserve tabs), and fixing it should
   be definitely easier than starting from scratch.

2. HSE has two pretty-printers: an exact one, and one that ignores
   annotations and pretty-prints in its own style. It's important to
   have a mixture of both, so that we can pretty-print generated
   snippets and splice them into an already formatted AST. A friend of
   mine, Pavel Poukh, is working on a pretty-printer that produces an
   annotated tree instead of a flat string, and AFAIK that work is close
   to done.

   [2]: https://github.com/Pnom/haskell-ast-pretty

3. A major thing that I devoted my time to is name resolution and
   package management [3,4], which are necessary for the tool to work.

   [3]: https://github.com/feuerbach/haskell-names
   [4]: https://github.com/feuerbach/haskell-packages

   They are also close to done.

4. As for the tool itself, I have a crude prototype [5], but I
   haven't updated it for a while.

   [5]: https://github.com/feuerbach/hasfix

5. The user interface is, of course, an important topic by itself. I
   was thinking of something less expressive and more declarative, but
   your idea is also interesting. Let's discuss this separately.

Regarding this topic as a GSoC project, I'm not so sure this is a good
idea. As a past GSoC student myself, I feel that new projects (as
opposed to established projects with existing user base and development
team) have much greater risks. (This is also backed up by data [6].)

But if you (or anyone else who happens to read this) would like to get
involved, please get in touch with me!

[6]: http://www.gwern.net/Haskell Summer of Code

Roman
Simon Hengel | 29 Apr 12:09 2013
Picon

Re: GSoC proposal: Haskell AST-based refactoring and API upgrading tool

Hi Niklas,
I haven't read the whole proposal as I'm short of time.  But Alan
Zimmerman is doing a lot of work on integrating HaRe with the GHC API
[1].  He is alanz on freenode and a regular in #hspec.

I haven't looked at the code, but maybe it's of interest to you.

Cheers,
Simon

[1] https://github.com/alanz/HaRe/tree/ghc-api

On Mon, Apr 29, 2013 at 02:00:23PM +0800, Niklas Hambüchen wrote:
> I would like to propose the development of source code refactoring tool
> that operates on Haskell source code ASTs and lets you formulate rewrite
> rules written in Haskell.
> 
> Objective
> ---------
> 
> The goal is to make refactorings easier and allow global code changes
> that might be incredibly tedious to do in a non-automated way.
> By making these transformations convenient, we can make it easier to
> maintain clean code, add new features or clean up leftovers faster, and
> reduce the fear and effort to upgrade to newer versions of packages and
> APIs.
> 
> 
> Transformations
> ---------------
> 
> First, here are a few operations you would use this tool for. Some of
> them are common operations you would also do in other programming
> languages, some are more specific to Haskell.
> 
> * Changing all occurrences of "import Prelude hiding (catch)" to "import
> qualified Control.Exception as E"
> 
> * Replacing all uses of a function with that function being imported
> qualified or the other way around
> 
> * Adding a field to data constructor a record, setting user-supplied
> defaults for construction and destruction:
> 
>     -- Suppose you want to change one of these
>     data User = User { name :: String, age :: Int }
>     data User = User String Int
> 
>     -- into one of these
>     data User = User { name :: String, age :: Int, active :: Bool }
>     data User = User String Int Bool
> 
>     -- the refactoring tool could perform, in all relevant locations:
>     show (User name age) = ...
>     show (User name age _) = ...
> 
>     -- and also this transformation:
>     ... u { name = "deleted" } ...
>     ... u { name = "deleted", active = False } ...
> 
>     -- or equivalently with records.
> 
>     -- Special cases could be taken care of as specified, such as
>     --   "whenever an object of [this User type] has
>     --    of its records passed into some function 'email', do this
>     --    now only if the user is active, so modify all relevant code
>     --        email (name u)
>     --    to
>     --        if (active u) then email (name u) else return ()
> 
>     -- Other examples include adding a position counter to attoparsec.
> 
> * Adding a type parameter to a type
> 
>     -- This happens a lot on monad transformer stacks, e.g.
>     newtype MyMonad a b c = MyMonad (ReaderT a (WriterT b ...
> 
>     -- and as you would probably agree on, this is not the most
>     -- comfortable change to make; in big project this can mean
>     -- hour-long grinding.
> 
>     -- It has also recently happened in the basic underlying types
>     -- of packages like conduit and pipes.
> 
> * Adding a new transformer around a monad
> 
> * Addressing problems like mentioned in
> http://blog.ezyang.com/2012/01/modelling-io/:
>   "There is one last problem with this approach: once the primitives
> have been selected, huge swaths of the standard library have to be
> redefined by “copy pasting” their definitions ..."
> 
> * Extracting a value into a let or where clause
> 
> * Renaming a variable, and all its occurrences that are semantically
> same variable (based on its scope)
> 
> * Changing the way things are done, such as:
> 
>     * Replacing uses of fmap with <$>, also taking care of the
>       corresponding import, and such cases were partial application
>       is involved
> 
>     * Replacing uses of "when (isJust)" to "forM_"
> 
> * Making imports clearer by adding all functions used to the file to the
> import list of the module that gets them in scope
> 
> * Finding all places where an exported function does not have all its
> arguments haddock-documented.
> 
> * Performing whole-project refactorings instead of operating on single
> files only, allowing operations like
> 
>     "Find me all functions of this type, e.g.
>          Maybe a -> (a -> m a) -> m a
>      in the project and extract them into this new module,
>      with the name 'onJust'."
> 
> 
> Some of the problems above can be tried to address using regex-based
> search and replace, but this already fails in the simplest case of
> "import Prelude hiding (catch)" in case there is more than that imported
> from Prelude or newlines involved in the import list.
> 
> Transformation on the AST are much more powerful, and can guarantee that
> the result is, at least syntactically, valid. No text base tool can do that.
> 
> 
> Other uses
> ----------
> 
> In addition to being able to perform transformations as mentioned above,
> the refactoring tool as a library can be leveraged to:
> 
> * Support or be the base of code formatting tools such as
> haskell-stylish, linters, style/convention checkers, static analyzers,
> test coverage tools etc.
> 
> * Implement automatic API upgrades.
> 
>   Imagine the author of a library you use deprecates some functions,
> introduces replacements, adds type parameters. In these cases, it is
> very clear and often well-documented which code has to be replaced by
> what. The library author could, along with the new release, publish
> "upgrade transformations" that you can apply to your code base to save
> most of the manual work.
> 
>   These upgrade transformations could be either parts of the packages
> themselves or be separately maintained and refined on the feedback of
> users applying them to their code bases.
>   This could allow the Haskell community to keep up their fast pace
> while making API breakage a non-problem.
> 
>   Automation is what makes us deal well with test suites, source
> control, error checking. Taking the pain out of this will increase
> people's incentive to upgrade their code an to keep it well-maintained
> by easy refactorings.
> 
>   This concept of automatic code upgrades would, to my knowledge, be
> quite unique in the programming language world, and the possibility to
> have this is, as many things, the result of Haskell's excellent type system.
>   In comparison to dynamic and "unsafe" languages, we actually have a
> lot of information around that we can use to write powerful and
> expressive tools.
> 
> 
> Splitting into GSoC tasks
> -------------------------
> 
> I think that the project is too large for one summer and should be split
> into two parts:
> 
> 1. Implementation of a "full-source" transformation-enabling AST
> 2. The transformation engine and API / DSL
> 
> 
> About 1: Creating a full-source AST
> 
> Existing parsers for Haskell tend to throw away information that is not
> necessary for compiling the code. Of course this is not good for a
> code-to-code transformation tool, stripping comments or whitespace the
> programmer cares about is not an option.
> 
> haskell-src-exts has gained support for comments around version 1.1 (as
> another GSoC project?), but still comments are treated as foreigners,
> given to you in a list detached from the AST; their positions are
> specified with integers, which means that when you modify the AST, you
> have to take care to adjust the comments as well.
> 
> The problem seems to be that haskell-source-exts is made for parsing,
> its AST is made for transformations. I have yet to see a tool that
> actually *modifies* haskell-src-exts's AST - most of them use it for
> parsing, and then apply some form of pretty-printing to get back to code.
> 
> We need an AST that contains *all* information about the original source
> code, which means render . parse = id; this is why I called it "full
> source" AST.
> Also, this AST should have all elements available in a way that
> encourages modification and does not discriminate against operations
> like re-indentation or alignment.
> 
> I like to think of this as something that could eventually be part of
> ghc, in a pipeline like
> 
>     GHC's parser
>  -> full-source AST
>  -> reduced AST that strips parts irrelevant for compilation
>  -> usual compilation pipeline
> 
> I am thinking of this being in a GHC environment because it would
> guarantee that it is up-to-date with the currently supported language
> extensions etc.
> However, I also see that this might slow down development and make
> things harder, so at least for the beginning it might be better to do it
> as as a normal library; I would love to get feedback on this.
> 
> It might be a good idea to start with GHC's or haskell-source-exts
> parser and modify them to obtain the full-source AST. I believe though
> that it would be beneficial if the AST would be to a certain extent be
> decoupled from the parser, such that other people could write other
> parsers (e.g. using other parsing libraries like Parsec, attoparsec,
> uu-parsinglib, trifecta) that produce the same compatible AST type.
> 
> I believe that creating this AST makes a GSoC project for itself, and
> that it would be beneficial for all efforts processing Haskell as a
> source language.
> 
> 
> About 2: Transformation engine and how to write transformations
> 
> Transformations should be Haskell code that operates on the AST.
> 
> They would be similar to how you write TemplateHaskell in a way, yet
> much easier and more intuitive to use in most cases. I suggest the
> creation of a monadic DSL that allows you to select and match those
> parts of the code you are interested in, perform some case analysis and
> computation to determine how you want to transform it, and then give you
> convenient ways to express your changes in that DSL.
> 
> Functions in this DSL would roughly belong to one of the kinds
>     * finding/matching
>     * transforming/rewriting.
> 
> I have not thought about how this DSL would exactly look like, but I
> could imagine myself writing a high-level transformation like this:
> 
>     is           <- getImports
>     let matching = filter (importsFunction "catch")
>                  . filter (hasModuleName "Prelude")
>     mapM_ (rewrite . removeImportFunction "catch") matching
> 
> One of the priority goals of this GSoC project is making this API
> convenient; transformations should read more naturally and be much
> shorter than equivalent TemplateHaskell.
> 
> This extensive, convenience oriented monadic DSL should aim to have a
> reasonably large set of functions, and "do things for you" even if you
> could build them yourself with three or four combinators.
> It should base itself on a minimal set of core transformations.
> I think that the Haskell community has a lot of expertise, especially
> from parsing libraries, in creating this core API + convenience
> combinators duo, that could aid the GSoC student to get this right.
> 
> Along with this way to specify transformations, a transformation engine
> has to be constructed which can perform them somewhat efficiently over
> large code-bases.
> 
> While speed should should not be a main target for this project and
> functionality and expressiveness are the main goals, the transformation
> engine should be designed at least with speed in mind, which can be
> worked on at a later stage outside of the GSoC project.
> 
> This second part of the proposal should result in a library to perform
> transformations and an executable that can apply them to a folder of
> source files.
> 
> 
> Possible extensions and follow-up projects
> ------------------------------------------
> 
> * Interactive refactoring tool
> 
> The refactoring tool could be driven by an interactive application that
> lets you write some transformations, apply them to your code base, view
> the diffs and build the project; on build failures or otherwise
> not-complete transformations it would allow to refine your
> transformations, or write custom special cases for selected files, line
> ranges or functions.
> This way you could quickly upgrade your code base in a fast
> write-check-repeat cycle.
> 
> * An API upgrade infrastructure
> 
> As mentioned above, transformations could be published along with new
> software packages that allow their dependants to easily upgrade their
> code to the latest API version with minimal manual effort.
> An infrastructure or standard way of way of doing this could be established.
> For now, I think that package maintainers would publish their upgrade
> transformations in their projects' or separate code repositories, and
> users would apply them with the refactoring tool as needed.
> I can also imagine a more dedicated infrastructure though, where API
> upgrades are stored on Hackage or a similar database.
> 
> 
> Summary
> -------
> 
> This proposal contains two GSoC projects that I believe to be reasonably
> sized for one summer each.
> 
> I think that they make good projects according to GSoC standards as they
> are focused, feasible, and aimed at creating real-world code that bring
> a clearly visible benefit.
> 
> The entry barrier is not too high since knowledge of compiler or runtime
> internals is not required. However, applicants should probably have a
> fairly good understanding of parsers and how they are dealt with in
> Haskell, and be somewhat familiar with the Haskell community in order to
> find sources of good feedback and for being able to judge whether the
> tools being created would be convenient for the community to use.
> 
> 
> Discussion
> ----------
> 
> Now I would be glad to get some responses on this proposal; I have
> written a bit of text, but it is still a very rough idea and I would
> love to hear your thoughts about it.
> 
> _______________________________________________
> Haskell-Cafe mailing list
> Haskell-Cafe <at> haskell.org
> http://www.haskell.org/mailman/listinfo/haskell-cafe

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Davorak | 29 Apr 21:01 2013
Picon

Re: GSoC proposal: Haskell AST-based refactoring and API upgrading tool

The latest updates on HaRe with GHC API project seem to be posted on the google+ community page:

https://plus.google.com/communities/116266567145785623821



On Monday, April 29, 2013 5:09:56 AM UTC-5, Simon Hengel wrote:
Hi Niklas,
I haven't read the whole proposal as I'm short of time.  But Alan
Zimmerman is doing a lot of work on integrating HaRe with the GHC API
[1].  He is alanz on freenode and a regular in #hspec.

I haven't looked at the code, but maybe it's of interest to you.

Cheers,
Simon

[1] https://github.com/alanz/HaRe/tree/ghc-api

On Mon, Apr 29, 2013 at 02:00:23PM +0800, Niklas Hambüchen wrote:
> I would like to propose the development of source code refactoring tool
> that operates on Haskell source code ASTs and lets you formulate rewrite
> rules written in Haskell.
>
> Objective
> ---------
>
> The goal is to make refactorings easier and allow global code changes
> that might be incredibly tedious to do in a non-automated way.
> By making these transformations convenient, we can make it easier to
> maintain clean code, add new features or clean up leftovers faster, and
> reduce the fear and effort to upgrade to newer versions of packages and
> APIs.
>
>
> Transformations
> ---------------
>
> First, here are a few operations you would use this tool for. Some of
> them are common operations you would also do in other programming
> languages, some are more specific to Haskell.
>
> * Changing all occurrences of "import Prelude hiding (catch)" to "import
> qualified Control.Exception as E"
>
> * Replacing all uses of a function with that function being imported
> qualified or the other way around
>
> * Adding a field to data constructor a record, setting user-supplied
> defaults for construction and destruction:
>
>     -- Suppose you want to change one of these
>     data User = User { name :: String, age :: Int }
>     data User = User String Int
>
>     -- into one of these
>     data User = User { name :: String, age :: Int, active :: Bool }
>     data User = User String Int Bool
>
>     -- the refactoring tool could perform, in all relevant locations:
>     show (User name age) = ...
>     show (User name age _) = ...
>
>     -- and also this transformation:
>     ... u { name = "deleted" } ...
>     ... u { name = "deleted", active = False } ...
>
>     -- or equivalently with records.
>
>     -- Special cases could be taken care of as specified, such as
>     --   "whenever an object of [this User type] has
>     --    of its records passed into some function 'email', do this
>     --    now only if the user is active, so modify all relevant code
>     --        email (name u)
>     --    to
>     --        if (active u) then email (name u) else return ()
>
>     -- Other examples include adding a position counter to attoparsec.
>
> * Adding a type parameter to a type
>
>     -- This happens a lot on monad transformer stacks, e.g.
>     newtype MyMonad a b c = MyMonad (ReaderT a (WriterT b ...
>
>     -- and as you would probably agree on, this is not the most
>     -- comfortable change to make; in big project this can mean
>     -- hour-long grinding.
>
>     -- It has also recently happened in the basic underlying types
>     -- of packages like conduit and pipes.
>
> * Adding a new transformer around a monad
>
> * Addressing problems like mentioned in
> http://blog.ezyang.com/2012/01/modelling-io/:
>   "There is one last problem with this approach: once the primitives
> have been selected, huge swaths of the standard library have to be
> redefined by “copy pasting” their definitions ..."
>
> * Extracting a value into a let or where clause
>
> * Renaming a variable, and all its occurrences that are semantically
> same variable (based on its scope)
>
> * Changing the way things are done, such as:
>
>     * Replacing uses of fmap with <$>, also taking care of the
>       corresponding import, and such cases were partial application
>       is involved
>
>     * Replacing uses of "when (isJust)" to "forM_"
>
> * Making imports clearer by adding all functions used to the file to the
> import list of the module that gets them in scope
>
> * Finding all places where an exported function does not have all its
> arguments haddock-documented.
>
> * Performing whole-project refactorings instead of operating on single
> files only, allowing operations like
>
>     "Find me all functions of this type, e.g.
>          Maybe a -> (a -> m a) -> m a
>      in the project and extract them into this new module,
>      with the name 'onJust'."
>
>
> Some of the problems above can be tried to address using regex-based
> search and replace, but this already fails in the simplest case of
> "import Prelude hiding (catch)" in case there is more than that imported
> from Prelude or newlines involved in the import list.
>
> Transformation on the AST are much more powerful, and can guarantee that
> the result is, at least syntactically, valid. No text base tool can do that.
>
>
> Other uses
> ----------
>
> In addition to being able to perform transformations as mentioned above,
> the refactoring tool as a library can be leveraged to:
>
> * Support or be the base of code formatting tools such as
> haskell-stylish, linters, style/convention checkers, static analyzers,
> test coverage tools etc.
>
> * Implement automatic API upgrades.
>
>   Imagine the author of a library you use deprecates some functions,
> introduces replacements, adds type parameters. In these cases, it is
> very clear and often well-documented which code has to be replaced by
> what. The library author could, along with the new release, publish
> "upgrade transformations" that you can apply to your code base to save
> most of the manual work.
>
>   These upgrade transformations could be either parts of the packages
> themselves or be separately maintained and refined on the feedback of
> users applying them to their code bases.
>   This could allow the Haskell community to keep up their fast pace
> while making API breakage a non-problem.
>
>   Automation is what makes us deal well with test suites, source
> control, error checking. Taking the pain out of this will increase
> people's incentive to upgrade their code an to keep it well-maintained
> by easy refactorings.
>
>   This concept of automatic code upgrades would, to my knowledge, be
> quite unique in the programming language world, and the possibility to
> have this is, as many things, the result of Haskell's excellent type system.
>   In comparison to dynamic and "unsafe" languages, we actually have a
> lot of information around that we can use to write powerful and
> expressive tools.
>
>
> Splitting into GSoC tasks
> -------------------------
>
> I think that the project is too large for one summer and should be split
> into two parts:
>
> 1. Implementation of a "full-source" transformation-enabling AST
> 2. The transformation engine and API / DSL
>
>
> About 1: Creating a full-source AST
>
> Existing parsers for Haskell tend to throw away information that is not
> necessary for compiling the code. Of course this is not good for a
> code-to-code transformation tool, stripping comments or whitespace the
> programmer cares about is not an option.
>
> haskell-src-exts has gained support for comments around version 1.1 (as
> another GSoC project?), but still comments are treated as foreigners,
> given to you in a list detached from the AST; their positions are
> specified with integers, which means that when you modify the AST, you
> have to take care to adjust the comments as well.
>
> The problem seems to be that haskell-source-exts is made for parsing,
> its AST is made for transformations. I have yet to see a tool that
> actually *modifies* haskell-src-exts's AST - most of them use it for
> parsing, and then apply some form of pretty-printing to get back to code.
>
> We need an AST that contains *all* information about the original source
> code, which means render . parse = id; this is why I called it "full
> source" AST.
> Also, this AST should have all elements available in a way that
> encourages modification and does not discriminate against operations
> like re-indentation or alignment.
>
> I like to think of this as something that could eventually be part of
> ghc, in a pipeline like
>
>     GHC's parser
>  -> full-source AST
>  -> reduced AST that strips parts irrelevant for compilation
>  -> usual compilation pipeline
>
> I am thinking of this being in a GHC environment because it would
> guarantee that it is up-to-date with the currently supported language
> extensions etc.
> However, I also see that this might slow down development and make
> things harder, so at least for the beginning it might be better to do it
> as as a normal library; I would love to get feedback on this.
>
> It might be a good idea to start with GHC's or haskell-source-exts
> parser and modify them to obtain the full-source AST. I believe though
> that it would be beneficial if the AST would be to a certain extent be
> decoupled from the parser, such that other people could write other
> parsers (e.g. using other parsing libraries like Parsec, attoparsec,
> uu-parsinglib, trifecta) that produce the same compatible AST type.
>
> I believe that creating this AST makes a GSoC project for itself, and
> that it would be beneficial for all efforts processing Haskell as a
> source language.
>
>
> About 2: Transformation engine and how to write transformations
>
> Transformations should be Haskell code that operates on the AST.
>
> They would be similar to how you write TemplateHaskell in a way, yet
> much easier and more intuitive to use in most cases. I suggest the
> creation of a monadic DSL that allows you to select and match those
> parts of the code you are interested in, perform some case analysis and
> computation to determine how you want to transform it, and then give you
> convenient ways to express your changes in that DSL.
>
> Functions in this DSL would roughly belong to one of the kinds
>     * finding/matching
>     * transforming/rewriting.
>
> I have not thought about how this DSL would exactly look like, but I
> could imagine myself writing a high-level transformation like this:
>
>     is           <- getImports
>     let matching = filter (importsFunction "catch")
>                  . filter (hasModuleName "Prelude")
>     mapM_ (rewrite . removeImportFunction "catch") matching
>
> One of the priority goals of this GSoC project is making this API
> convenient; transformations should read more naturally and be much
> shorter than equivalent TemplateHaskell.
>
> This extensive, convenience oriented monadic DSL should aim to have a
> reasonably large set of functions, and "do things for you" even if you
> could build them yourself with three or four combinators.
> It should base itself on a minimal set of core transformations.
> I think that the Haskell community has a lot of expertise, especially
> from parsing libraries, in creating this core API + convenience
> combinators duo, that could aid the GSoC student to get this right.
>
> Along with this way to specify transformations, a transformation engine
> has to be constructed which can perform them somewhat efficiently over
> large code-bases.
>
> While speed should should not be a main target for this project and
> functionality and expressiveness are the main goals, the transformation
> engine should be designed at least with speed in mind, which can be
> worked on at a later stage outside of the GSoC project.
>
> This second part of the proposal should result in a library to perform
> transformations and an executable that can apply them to a folder of
> source files.
>
>
> Possible extensions and follow-up projects
> ------------------------------------------
>
> * Interactive refactoring tool
>
> The refactoring tool could be driven by an interactive application that
> lets you write some transformations, apply them to your code base, view
> the diffs and build the project; on build failures or otherwise
> not-complete transformations it would allow to refine your
> transformations, or write custom special cases for selected files, line
> ranges or functions.
> This way you could quickly upgrade your code base in a fast
> write-check-repeat cycle.
>
> * An API upgrade infrastructure
>
> As mentioned above, transformations could be published along with new
> software packages that allow their dependants to easily upgrade their
> code to the latest API version with minimal manual effort.
> An infrastructure or standard way of way of doing this could be established.
> For now, I think that package maintainers would publish their upgrade
> transformations in their projects' or separate code repositories, and
> users would apply them with the refactoring tool as needed.
> I can also imagine a more dedicated infrastructure though, where API
> upgrades are stored on Hackage or a similar database.
>
>
> Summary
> -------
>
> This proposal contains two GSoC projects that I believe to be reasonably
> sized for one summer each.
>
> I think that they make good projects according to GSoC standards as they
> are focused, feasible, and aimed at creating real-world code that bring
> a clearly visible benefit.
>
> The entry barrier is not too high since knowledge of compiler or runtime
> internals is not required. However, applicants should probably have a
> fairly good understanding of parsers and how they are dealt with in
> Haskell, and be somewhat familiar with the Haskell community in order to
> find sources of good feedback and for being able to judge whether the
> tools being created would be convenient for the community to use.
>
>
> Discussion
> ----------
>
> Now I would be glad to get some responses on this proposal; I have
> written a bit of text, but it is still a very rough idea and I would
> love to hear your thoughts about it.
>
> _______________________________________________
> Haskell-Cafe mailing list
> Haskel... <at> haskell.org
> http://www.haskell.org/mailman/listinfo/haskell-cafe

_______________________________________________
Haskell-Cafe mailing list
Haskel... <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe

Gmane