Niklas Hambüchen | 10 Mar 18:35 2014

On Haskell IDEs

On 10/03/14 16:22, Michal Antkiewicz wrote:>
>
>     I would say making ghci a full-blown IDE backend akin lisps
>     slime-swank or clojure nrepl would be the best approach.
>
> there's already hdevtools which plays that role - it provides all sorts
> of information to an IDE, like types, location of declarations, etc.
> It's a background process and it's quite responsive (tries to be
> incremental). I don't know how it is implemented and whether it is a
> wrapper around GHCi.
>
> But I agree that all IDEs should simply use GHCi as the official IDE
> backend. Maybe some parts from scion/hdevtools/ghc-mod/etc. could be
> pulled back into GHCi?
>
> Michal

Mateusz just clarified that his proposal is more about improving Yi as
an editor than making it more of a Haskell IDE, so I am replying to this
in a new thread.

Alexandr Ruchkin and me maintain SublimeHaskell, the most-used Haskell
plugin for Sublime Text.

It integrates ghc-mod, hdevtools, cabal etc. and can do things like
showing compile errors, completing imports, displaying the type of an
expression in the buffer or insert it over a function declaration.

When we want to change a Haskell file's content, we get the relevant
info from, say, ghc-mod and then do the actual modification logic in the
(Continue reading)

Vagif Verdi | 10 Mar 19:41 2014
Picon

Re: On Haskell IDEs

>It integrates ghc-mod, hdevtools, cabal
...
> Alexandr started a project called "hsdev"
(https://github.com/mvoidex/hsdev)

This is an excellent illustration of how fragmentation in this specific area hurts community. Not only you are forced to use 3 different tools (ghc-mod, hdevtools, cabal) each of which basically provides the same service but in an incompatible and incomplete way. But you decided to create yet another one instead of contributing to the ones you already use (they are all open source)

The problem here is that most of IDE features require some environment that loads and keeps track of the entire project information. Unfortunately approach of creating separate small utilities not gonna work in this case, because each utility would have to load entire project to be able to provide some useful functionality.

This is clearly the case where concerted community effort is required, similar to cabal.

And we do have a good base for it: ghci

On Monday, March 10, 2014 10:35:19 AM UTC-7, Niklas Hambüchen wrote:
On 10/03/14 16:22, Michal Antkiewicz wrote:>
>
>     I would say making ghci a full-blown IDE backend akin lisps
>     slime-swank or clojure nrepl would be the best approach.
>
> there's already hdevtools which plays that role - it provides all sorts
> of information to an IDE, like types, location of declarations, etc.
> It's a background process and it's quite responsive (tries to be
> incremental). I don't know how it is implemented and whether it is a
> wrapper around GHCi.
>
> But I agree that all IDEs should simply use GHCi as the official IDE
> backend. Maybe some parts from scion/hdevtools/ghc-mod/etc. could be
> pulled back into GHCi?
>
> Michal

Mateusz just clarified that his proposal is more about improving Yi as
an editor than making it more of a Haskell IDE, so I am replying to this
in a new thread.

Alexandr Ruchkin and me maintain SublimeHaskell, the most-used Haskell
plugin for Sublime Text.

It integrates ghc-mod, hdevtools, cabal etc. and can do things like
showing compile errors, completing imports, displaying the type of an
expression in the buffer or insert it over a function declaration.

When we want to change a Haskell file's content, we get the relevant
info from, say, ghc-mod and then do the actual modification logic in the
editor scripting language (in our case Python).
We recognise that this logic is duplicated in pretty much every editor
with such Haskell features, and at the same think that Haskell is a much
nicer language to program this logic. Technically, it is also less error
prone to work with ghc-mod's API than with its command line interface.

That's why we seem to move more and more to do the actual modifications
in an external Haskell program, and have the part in the actual editor
be as minimal as possible. Alexandr started a project called "hsdev"
(https://github.com/mvoidex/hsdev) for that. It is very editor-agnostic
and we think that sharing something like that across editors makes sense
since it could save plugin writers a lot of time. The ideas is that you
would tell this tool to perform an action, e.g. "import missing imports"
or "go to declaration", and it would do it for you instead implementing
the buffer changes in your editor plugin.

There is one problem with that: Editors don't deal well with other
programs editing currently opened files. In Sublime's case, the buffer
will automatically refresh, but it is very well possible that your caret
position and undo history get lost, which totally interrupts the work flow.

So I was wondering what the most minimal interface would be that allows
your editor to be aware of the changes that an external program (HaRE,
hsdev, etc.) would like to make to your code base.

At the moment we consider having hsdev outputting a diff format that
describes the changes, and then each editor would have to implement a
simple interpreter that goes through the files, applying the changes.

That way we can avoid an external file change and the problems that come
with it.

The Sublime-related discussion about this is at
https://github.com/SublimeHaskell/SublimeHaskell/pull/132#issuecomment-33888678.

I think that IDE-style "smart language features" might have a better
place in cross-editor tools than in the editors themselves. ghc-mod is a
successful example.

I also think that most Haskell-IDE maintainers would agree (please
correct me otherwise) that doing program transformation features is much
more pleasant to do in Haskell than in Elisp, Vimscript, Python or Java,
and that specifying out a cross-editor "modification description" and
implementing the smart parts in a shareable tool might save us all a lot
of time.

It looks like Scion (http://code.google.com/p/scion-lib/) followed
similar goals, but doesn't seem to have found wide adoption and no real
development for multiple years.

I would welcome a discussion about why past attempts to this failed,
whether you think that having a monolithic Haskell-IDE-tool output diffs
for any editor to apply would lower the barriers for making editors
Haskell-smart, and if you are an editor plugin maintainer, if you agree
that sharing common functionality is worth it.
_______________________________________________
Haskell-Cafe mailing list
Haskel... <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Niklas Hambüchen | 10 Mar 20:32 2014

Re: On Haskell IDEs

On 10/03/14 18:41, Vagif Verdi wrote:
> This is an excellent illustration of how fragmentation in this specific
> area hurts community. Not only you are forced to use 3 different tools
> (ghc-mod, hdevtools, cabal) each of which basically provides the same
> service but in an incompatible and incomplete way. But you decided to
> create yet another one instead of contributing to the ones you already
> use (they are all open source)

I'm not sure what you mean to say; ghc-mod and cabal certainly do not do
the same. Hsdev doesn't either, and claiming that we don't contribute to
these projects is strange.
Vagif Verdi | 10 Mar 20:44 2014
Picon

Re: On Haskell IDEs

Sorry, i meant ghci not cabal.

As for hsdev, in order to provide its features it will have to load and keep track of entire project information which is already done by ghc-mod and hdevtools separately. So you  ushould see how much effort is wasted on doing the same thing again and again in each of 4 different tools (ghci, ghc-mod, hdevtools and hsdev)

On Monday, March 10, 2014 12:32:25 PM UTC-7, Niklas Hambüchen wrote:
On 10/03/14 18:41, Vagif Verdi wrote:
> This is an excellent illustration of how fragmentation in this specific
> area hurts community. Not only you are forced to use 3 different tools
> (ghc-mod, hdevtools, cabal) each of which basically provides the same
> service but in an incompatible and incomplete way. But you decided to
> create yet another one instead of contributing to the ones you already
> use (they are all open source)

I'm not sure what you mean to say; ghc-mod and cabal certainly do not do
the same. Hsdev doesn't either, and claiming that we don't contribute to
these projects is strange.
_______________________________________________
Haskell-Cafe mailing list
Haskel... <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Andrey Chudnov | 10 Mar 21:59 2014
Picon

Re: On Haskell IDEs

While I do not condone Mr. Verdi's verbal bashing of the efforts of the Haskell community to provide development tools, I do feel his pain. There is quite a bit of divided effort which seems to result in having multiple inferior solutions.

My all-time favorite is ghc-mod because "it just works", but it can be very slow for large projects, especially involving template-haskell (and there are no easy fixes for that).

HDevTools, in theory, is great: a persistent server, allowing incremental builds/checks etc., but it's oblivious to .cabal files, let alone sandboxes, which makes it completely impractical for any degree of serious Haskell development (on that note, how come pull request 30 [1] is still open?! /rant).

Finally, GHCi is supposed to be enough for IDE support in editors. However, as I understand it, the problem is that its output is not easily parsed and, hence, consumed by editors; on top of that, using it with .cabal files and sandboxes requires calling 'cabal repl', so using another tool.

So, maybe, GHCi could be augmented with an easily consumed output format (like JSON), and 'cabal' be given support for that mode as well? Since it's that time of the year again (I mean, the GSoC proposal time :)), maybe, someone would look into this idea?

[1] https://github.com/bitc/hdevtools/pull/30

/Andrey

On 03/10/2014 03:44 PM, Vagif Verdi wrote:
Sorry, i meant ghci not cabal.

As for hsdev, in order to provide its features it will have to load and keep track of entire project information which is already done by ghc-mod and hdevtools separately. So you  ushould see how much effort is wasted on doing the same thing again and again in each of 4 different tools (ghci, ghc-mod, hdevtools and hsdev)

On Monday, March 10, 2014 12:32:25 PM UTC-7, Niklas Hambüchen wrote:
On 10/03/14 18:41, Vagif Verdi wrote:
> This is an excellent illustration of how fragmentation in this specific
> area hurts community. Not only you are forced to use 3 different tools
> (ghc-mod, hdevtools, cabal) each of which basically provides the same
> service but in an incompatible and incomplete way. But you decided to
> create yet another one instead of contributing to the ones you already
> use (they are all open source)

I'm not sure what you mean to say; ghc-mod and cabal certainly do not do
the same. Hsdev doesn't either, and claiming that we don't contribute to
these projects is strange.
_______________________________________________
Haskell-Cafe mailing list
Haskel... <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe <at> haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Michal Antkiewicz | 10 Mar 22:11 2014
Picon
Picon

Re: On Haskell IDEs

Cabal repl is absolutely great! Are there any reasons why it couldn't provide all information that ghc-mod and hdevtools do? It could be run as a background process, right? It knows about sandboxes as well.

That would be a great SoC project - extend cabal repls to subsume ghc-mod and hdevtools.

Michal


On Mon, Mar 10, 2014 at 4:59 PM, Andrey Chudnov <achudnov <at> gmail.com> wrote:
While I do not condone Mr. Verdi's verbal bashing of the efforts of the Haskell community to provide development tools, I do feel his pain. There is quite a bit of divided effort which seems to result in having multiple inferior solutions.

My all-time favorite is ghc-mod because "it just works", but it can be very slow for large projects, especially involving template-haskell (and there are no easy fixes for that).

HDevTools, in theory, is great: a persistent server, allowing incremental builds/checks etc., but it's oblivious to .cabal files, let alone sandboxes, which makes it completely impractical for any degree of serious Haskell development (on that note, how come pull request 30 [1] is still open?! /rant).

Finally, GHCi is supposed to be enough for IDE support in editors. However, as I understand it, the problem is that its output is not easily parsed and, hence, consumed by editors; on top of that, using it with .cabal files and sandboxes requires calling 'cabal repl', so using another tool.

So, maybe, GHCi could be augmented with an easily consumed output format (like JSON), and 'cabal' be given support for that mode as well? Since it's that time of the year again (I mean, the GSoC proposal time :)), maybe, someone would look into this idea?

[1] https://github.com/bitc/hdevtools/pull/30

/Andrey


On 03/10/2014 03:44 PM, Vagif Verdi wrote:
Sorry, i meant ghci not cabal.

As for hsdev, in order to provide its features it will have to load and keep track of entire project information which is already done by ghc-mod and hdevtools separately. So you  ushould see how much effort is wasted on doing the same thing again and again in each of 4 different tools (ghci, ghc-mod, hdevtools and hsdev)

On Monday, March 10, 2014 12:32:25 PM UTC-7, Niklas Hambüchen wrote:
On 10/03/14 18:41, Vagif Verdi wrote:
> This is an excellent illustration of how fragmentation in this specific
> area hurts community. Not only you are forced to use 3 different tools
> (ghc-mod, hdevtools, cabal) each of which basically provides the same
> service but in an incompatible and incomplete way. But you decided to
> create yet another one instead of contributing to the ones you already
> use (they are all open source)

I'm not sure what you mean to say; ghc-mod and cabal certainly do not do
the same. Hsdev doesn't either, and claiming that we don't contribute to
these projects is strange.
_______________________________________________
Haskell-Cafe mailing list
Haskel... <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe <at> haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe


_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe



_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Andrey Chudnov | 10 Mar 22:22 2014
Picon

Re: On Haskell IDEs

If we go down that road, it's not cabal-repl that would need to be extended. AFAIK, it's just a wrapper around GHCi that calls that latter with package info extracted from the cabal file and, optionally, the sandbox. So, really, it's GHCi that would need to be extended. I think, something as simple as adding an option to output JSON (or some other structured format with wide support in editors) instead of plain text would go a long way to allow editors to consume it's output: they could just pipe commands via stdin and read the JSON output via stdout, parse it and display the info in the buffer. Depending on how GHCi looks inside and how amenable it is to such modifications, it might actually be a doable summer project. Am I missing any technical gotchas?

/Andrey

On 03/10/2014 05:11 PM, Michal Antkiewicz wrote:
Cabal repl is absolutely great! Are there any reasons why it couldn't provide all information that ghc-mod and hdevtools do? It could be run as a background process, right? It knows about sandboxes as well.

That would be a great SoC project - extend cabal repls to subsume ghc-mod and hdevtools.

Michal


On Mon, Mar 10, 2014 at 4:59 PM, Andrey Chudnov <achudnov <at> gmail.com> wrote:
While I do not condone Mr. Verdi's verbal bashing of the efforts of the Haskell community to provide development tools, I do feel his pain. There is quite a bit of divided effort which seems to result in having multiple inferior solutions.

My all-time favorite is ghc-mod because "it just works", but it can be very slow for large projects, especially involving template-haskell (and there are no easy fixes for that).

HDevTools, in theory, is great: a persistent server, allowing incremental builds/checks etc., but it's oblivious to .cabal files, let alone sandboxes, which makes it completely impractical for any degree of serious Haskell development (on that note, how come pull request 30 [1] is still open?! /rant).

Finally, GHCi is supposed to be enough for IDE support in editors. However, as I understand it, the problem is that its output is not easily parsed and, hence, consumed by editors; on top of that, using it with .cabal files and sandboxes requires calling 'cabal repl', so using another tool.

So, maybe, GHCi could be augmented with an easily consumed output format (like JSON), and 'cabal' be given support for that mode as well? Since it's that time of the year again (I mean, the GSoC proposal time :)), maybe, someone would look into this idea?

[1] https://github.com/bitc/hdevtools/pull/30

/Andrey


On 03/10/2014 03:44 PM, Vagif Verdi wrote:
Sorry, i meant ghci not cabal.

As for hsdev, in order to provide its features it will have to load and keep track of entire project information which is already done by ghc-mod and hdevtools separately. So you  ushould see how much effort is wasted on doing the same thing again and again in each of 4 different tools (ghci, ghc-mod, hdevtools and hsdev)

On Monday, March 10, 2014 12:32:25 PM UTC-7, Niklas Hambüchen wrote:
On 10/03/14 18:41, Vagif Verdi wrote:
> This is an excellent illustration of how fragmentation in this specific
> area hurts community. Not only you are forced to use 3 different tools
> (ghc-mod, hdevtools, cabal) each of which basically provides the same
> service but in an incompatible and incomplete way. But you decided to
> create yet another one instead of contributing to the ones you already
> use (they are all open source)

I'm not sure what you mean to say; ghc-mod and cabal certainly do not do
the same. Hsdev doesn't either, and claiming that we don't contribute to
these projects is strange.
_______________________________________________
Haskell-Cafe mailing list
Haskel... <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


_______________________________________________ Haskell-Cafe mailing list Haskell-Cafe <at> haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe


_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe




_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Carlo Hamalainen | 11 Mar 07:25 2014
Picon

Re: On Haskell IDEs

On 10/03/14 22:22, Andrey Chudnov wrote:
If we go down that road, it's not cabal-repl that would need to be extended. AFAIK, it's just a wrapper around GHCi that calls that latter with package info extracted from the cabal file and, optionally, the sandbox. So, really, it's GHCi that would need to be extended. I think, something as simple as adding an option to output JSON (or some other structured format with wide support in editors) instead of plain text would go a long way to allow editors to consume it's output: they could just pipe commands via stdin and read the JSON output via stdout, parse it and display the info in the buffer. Depending on how GHCi looks inside and how amenable it is to such modifications, it might actually be a doable summer project. Am I missing any technical gotchas?

Regarding the data format, ghc-mod uses the GHC API to get the de-sugared and type-checked representation of a module, e.g.

https://github.com/kazu-yamamoto/ghc-mod/blob/master/Language/Haskell/GhcMod/Gap.hs#L326-L337

Or, my tool ghc-imported-from needs to get the "guts" from the GHC API to get the global reader environment:

https://github.com/carlohamalainen/ghc-imported-from/blob/master/Language/Haskell/GhcImportedFrom.hs#L343-L373

So my feeling is that we'd need to pass Haskell values back and forth, not just JSON.

Personally I would love it if GHCi was extended in the way that you are suggesting, as it would make my tool much faster. Ditto for ghc-mod.

-- Carlo Hamalainen http://carlo-hamalainen.net
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Maxim Kulkin | 11 Mar 08:19 2014
Picon

Re: On Haskell IDEs

I played a bit with GHC API and Cabal (and even implemented .cabal support for hdevtools) and now I see issues with this approach:

1. GHC depends on Cabal (rly?) and this is most likely some old version of Cabal, not the one you use currently (e.g. I use cabal-install and Cabal 1.18, but GHC 7.6.3 -- the latest stable -- depends on Cabal 1.16). This makes it hard to develop tools that both use GHC API and latest Cabal. Hopefully this dependency will be removed in future versions of GHC.

2. Sometimes you get stuck with types and can't get source code compile and GHC API-based tools are of no help in this situation, because they won't be able to provide type for some variable in source file because this source file contains a syntax/type error in some other part. I'm not sure if GHC API can be tweaked to bypass errors and do the best effort of parsing/desugaring of other parts of sources with errors. Maybe some external parser (haskell-src-exts) should be used for source inspection.


On Tue, Mar 11, 2014 at 10:25 AM, Carlo Hamalainen <carlo <at> carlo-hamalainen.net> wrote:
On 10/03/14 22:22, Andrey Chudnov wrote:
If we go down that road, it's not cabal-repl that would need to be extended. AFAIK, it's just a wrapper around GHCi that calls that latter with package info extracted from the cabal file and, optionally, the sandbox. So, really, it's GHCi that would need to be extended. I think, something as simple as adding an option to output JSON (or some other structured format with wide support in editors) instead of plain text would go a long way to allow editors to consume it's output: they could just pipe commands via stdin and read the JSON output via stdout, parse it and display the info in the buffer. Depending on how GHCi looks inside and how amenable it is to such modifications, it might actually be a doable summer project. Am I missing any technical gotchas?

Regarding the data format, ghc-mod uses the GHC API to get the de-sugared and type-checked representation of a module, e.g.

https://github.com/kazu-yamamoto/ghc-mod/blob/master/Language/Haskell/GhcMod/Gap.hs#L326-L337

Or, my tool ghc-imported-from needs to get the "guts" from the GHC API to get the global reader environment:

https://github.com/carlohamalainen/ghc-imported-from/blob/master/Language/Haskell/GhcImportedFrom.hs#L343-L373

So my feeling is that we'd need to pass Haskell values back and forth, not just JSON.

Personally I would love it if GHCi was extended in the way that you are suggesting, as it would make my tool much faster. Ditto for ghc-mod.

-- Carlo Hamalainen http://carlo-hamalainen.net

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Mateusz Kowalczyk | 11 Mar 08:26 2014
Picon

Re: On Haskell IDEs

On 11/03/14 07:19, Maxim Kulkin wrote:
> I played a bit with GHC API and Cabal (and even implemented .cabal support
> for hdevtools) and now I see issues with this approach:
> 
> 1. GHC depends on Cabal (rly?) and this is most likely some old version of
> Cabal, not the one you use currently (e.g. I use cabal-install and Cabal
> 1.18, but GHC 7.6.3 -- the latest stable -- depends on Cabal 1.16). This
> makes it hard to develop tools that both use GHC API and latest Cabal.
> Hopefully this dependency will be removed in future versions of GHC.

Actually, 1.16 was the bleeding edge release when 7.6.3 came out so
technically it was new. Old compiler depends on an old version, I don't
see why you find this odd. 7.8 will ship with a newer version (whether
we like this or not).

Having said that, I don't think GHC API depends on Cabal API, does it?

> 2. Sometimes you get stuck with types and can't get source code compile and
> GHC API-based tools are of no help in this situation, because they won't be
> able to provide type for some variable in source file because this source
> file contains a syntax/type error in some other part. I'm not sure if GHC
> API can be tweaked to bypass errors and do the best effort of
> parsing/desugaring of other parts of sources with errors. Maybe some
> external parser (haskell-src-exts) should be used for source inspection.
> 

--

-- 
Mateusz K.
Maxim Kulkin | 11 Mar 08:28 2014
Picon

Re: On Haskell IDEs

ghc package depends on exact version of Cabal pacakge. I understand why old package depends on some other old package. I just don't understand why GHC depends on Cabal.


On Tue, Mar 11, 2014 at 11:26 AM, Mateusz Kowalczyk <fuuzetsu <at> fuuzetsu.co.uk> wrote:
On 11/03/14 07:19, Maxim Kulkin wrote:
> I played a bit with GHC API and Cabal (and even implemented .cabal support
> for hdevtools) and now I see issues with this approach:
>
> 1. GHC depends on Cabal (rly?) and this is most likely some old version of
> Cabal, not the one you use currently (e.g. I use cabal-install and Cabal
> 1.18, but GHC 7.6.3 -- the latest stable -- depends on Cabal 1.16). This
> makes it hard to develop tools that both use GHC API and latest Cabal.
> Hopefully this dependency will be removed in future versions of GHC.

Actually, 1.16 was the bleeding edge release when 7.6.3 came out so
technically it was new. Old compiler depends on an old version, I don't
see why you find this odd. 7.8 will ship with a newer version (whether
we like this or not).

Having said that, I don't think GHC API depends on Cabal API, does it?

> 2. Sometimes you get stuck with types and can't get source code compile and
> GHC API-based tools are of no help in this situation, because they won't be
> able to provide type for some variable in source file because this source
> file contains a syntax/type error in some other part. I'm not sure if GHC
> API can be tweaked to bypass errors and do the best effort of
> parsing/desugaring of other parts of sources with errors. Maybe some
> external parser (haskell-src-exts) should be used for source inspection.
>


--
Mateusz K.
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
AlanKim Zimmerman | 11 Mar 09:08 2014
Picon

Re: On Haskell IDEs

HaRe gets around the cabal version mismatch by making sure that the return types from the relevant ghc-mod calls are types that are not exported by cabal.

e.g. http://hackage.haskell.org/package/ghc-mod-3.1.4/docs/Language-Haskell-GhcMod-Internal.html#v:cabalAllTargets returns [String] for the target names.

This could perhaps be extended to either use a separate library of stable types to be used by both cabal and ghc, or to return e.g. json

Alan


On Tue, Mar 11, 2014 at 9:19 AM, Maxim Kulkin <maxim.kulkin <at> gmail.com> wrote:
I played a bit with GHC API and Cabal (and even implemented .cabal support for hdevtools) and now I see issues with this approach:

1. GHC depends on Cabal (rly?) and this is most likely some old version of Cabal, not the one you use currently (e.g. I use cabal-install and Cabal 1.18, but GHC 7.6.3 -- the latest stable -- depends on Cabal 1.16). This makes it hard to develop tools that both use GHC API and latest Cabal. Hopefully this dependency will be removed in future versions of GHC.

2. Sometimes you get stuck with types and can't get source code compile and GHC API-based tools are of no help in this situation, because they won't be able to provide type for some variable in source file because this source file contains a syntax/type error in some other part. I'm not sure if GHC API can be tweaked to bypass errors and do the best effort of parsing/desugaring of other parts of sources with errors. Maybe some external parser (haskell-src-exts) should be used for source inspection.


On Tue, Mar 11, 2014 at 10:25 AM, Carlo Hamalainen <carlo <at> carlo-hamalainen.net> wrote:
On 10/03/14 22:22, Andrey Chudnov wrote:
If we go down that road, it's not cabal-repl that would need to be extended. AFAIK, it's just a wrapper around GHCi that calls that latter with package info extracted from the cabal file and, optionally, the sandbox. So, really, it's GHCi that would need to be extended. I think, something as simple as adding an option to output JSON (or some other structured format with wide support in editors) instead of plain text would go a long way to allow editors to consume it's output: they could just pipe commands via stdin and read the JSON output via stdout, parse it and display the info in the buffer. Depending on how GHCi looks inside and how amenable it is to such modifications, it might actually be a doable summer project. Am I missing any technical gotchas?

Regarding the data format, ghc-mod uses the GHC API to get the de-sugared and type-checked representation of a module, e.g.

https://github.com/kazu-yamamoto/ghc-mod/blob/master/Language/Haskell/GhcMod/Gap.hs#L326-L337

Or, my tool ghc-imported-from needs to get the "guts" from the GHC API to get the global reader environment:

https://github.com/carlohamalainen/ghc-imported-from/blob/master/Language/Haskell/GhcImportedFrom.hs#L343-L373

So my feeling is that we'd need to pass Haskell values back and forth, not just JSON.

Personally I would love it if GHCi was extended in the way that you are suggesting, as it would make my tool much faster. Ditto for ghc-mod.

-- Carlo Hamalainen http://carlo-hamalainen.net

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe



_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe


_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Maxim Kulkin | 11 Mar 09:17 2014
Picon

Re: On Haskell IDEs

The thing is not with types: I would like to use a version of cabal in my tool, but dependency on GHC API (which I also use) dictates me other (older) version of Cabal for an unknown and unobvious reason. I want user of my tool to pick arbitrary Cabal version, the one that he actually uses. I'm even ok to write different implementations to support differencies in Cabal API.

I once googled a mailing discussion on that GHC uses methods for parsing and printing some types like package ids or something and they introduced a dependency for that reason.



On Tue, Mar 11, 2014 at 12:08 PM, AlanKim Zimmerman <alan.zimm <at> gmail.com> wrote:
HaRe gets around the cabal version mismatch by making sure that the return types from the relevant ghc-mod calls are types that are not exported by cabal.

e.g. http://hackage.haskell.org/package/ghc-mod-3.1.4/docs/Language-Haskell-GhcMod-Internal.html#v:cabalAllTargets returns [String] for the target names.

This could perhaps be extended to either use a separate library of stable types to be used by both cabal and ghc, or to return e.g. json

Alan


On Tue, Mar 11, 2014 at 9:19 AM, Maxim Kulkin <maxim.kulkin <at> gmail.com> wrote:
I played a bit with GHC API and Cabal (and even implemented .cabal support for hdevtools) and now I see issues with this approach:

1. GHC depends on Cabal (rly?) and this is most likely some old version of Cabal, not the one you use currently (e.g. I use cabal-install and Cabal 1.18, but GHC 7.6.3 -- the latest stable -- depends on Cabal 1.16). This makes it hard to develop tools that both use GHC API and latest Cabal. Hopefully this dependency will be removed in future versions of GHC.

2. Sometimes you get stuck with types and can't get source code compile and GHC API-based tools are of no help in this situation, because they won't be able to provide type for some variable in source file because this source file contains a syntax/type error in some other part. I'm not sure if GHC API can be tweaked to bypass errors and do the best effort of parsing/desugaring of other parts of sources with errors. Maybe some external parser (haskell-src-exts) should be used for source inspection.


On Tue, Mar 11, 2014 at 10:25 AM, Carlo Hamalainen <carlo <at> carlo-hamalainen.net> wrote:
On 10/03/14 22:22, Andrey Chudnov wrote:
If we go down that road, it's not cabal-repl that would need to be extended. AFAIK, it's just a wrapper around GHCi that calls that latter with package info extracted from the cabal file and, optionally, the sandbox. So, really, it's GHCi that would need to be extended. I think, something as simple as adding an option to output JSON (or some other structured format with wide support in editors) instead of plain text would go a long way to allow editors to consume it's output: they could just pipe commands via stdin and read the JSON output via stdout, parse it and display the info in the buffer. Depending on how GHCi looks inside and how amenable it is to such modifications, it might actually be a doable summer project. Am I missing any technical gotchas?

Regarding the data format, ghc-mod uses the GHC API to get the de-sugared and type-checked representation of a module, e.g.

https://github.com/kazu-yamamoto/ghc-mod/blob/master/Language/Haskell/GhcMod/Gap.hs#L326-L337

Or, my tool ghc-imported-from needs to get the "guts" from the GHC API to get the global reader environment:

https://github.com/carlohamalainen/ghc-imported-from/blob/master/Language/Haskell/GhcImportedFrom.hs#L343-L373

So my feeling is that we'd need to pass Haskell values back and forth, not just JSON.

Personally I would love it if GHCi was extended in the way that you are suggesting, as it would make my tool much faster. Ditto for ghc-mod.

-- Carlo Hamalainen http://carlo-hamalainen.net

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe



_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe



_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
JP Moresmau | 11 Mar 09:20 2014
Picon

Re: On Haskell IDEs

Unifying all the different tools underlying IDEs has been talked again before, because the pain is real. For example, see https://github.com/fpco/haskell-ide/wiki. I've written about the GHC API in that wiki, and some of the issues that it has for such tools.
Currently EclipseFP uses buildwrapper, which is a backend I wrote because I was dissatisfied with scion. It integrates the GHC API and Cabal, so that it aims to give the same results that running cabal configure/build. It usually run as a shortlived executable but for performance reason it is possible to have a long lived session to avoid reloading the modules in GHC all the times.
Yes, GHC depends on the Cabal API for some types, which causes untold woes when you use buildwrapper after upgrading Cabal, for example. There is a bug report to remove that dependency but I had no time yet to work on it, so far I've used dynamic-cabal in buildwrapper to allow a newer version of Cabal, at the price of more complexity. But a complete tool would need to integrate both, so it can't be ghci only, otherwise Cabal and GHC would become really stuck together.
The GHC API uses some global settings, which means you sometimes have no choice (if stcouldatic flags change) than to respawn a new process. This could probably be improved since it's not really the way a well behaved Haskell program should act. Memory usage is also a concern.

So I think there should be two lines of work here: one would be to improve the GHC API and/or the ghci interface, to make it more IDE friendly, the other would be to unify all these tools like scion/buildwrapper/ghc-mod in one tool that would hopefully provide all services an IDE needs, and that would allow easy plugin of additional functionality (so for example HaRe could easily add its refactoring capabilities to it). I would happily switch from buildwrapper to another community supported tool if all the functionality I currently have is kept in one form or another.

JP 


On Tue, Mar 11, 2014 at 9:17 AM, Maxim Kulkin <maxim.kulkin <at> gmail.com> wrote:
The thing is not with types: I would like to use a version of cabal in my tool, but dependency on GHC API (which I also use) dictates me other (older) version of Cabal for an unknown and unobvious reason. I want user of my tool to pick arbitrary Cabal version, the one that he actually uses. I'm even ok to write different implementations to support differencies in Cabal API.

I once googled a mailing discussion on that GHC uses methods for parsing and printing some types like package ids or something and they introduced a dependency for that reason.



On Tue, Mar 11, 2014 at 12:08 PM, AlanKim Zimmerman <alan.zimm <at> gmail.com> wrote:
HaRe gets around the cabal version mismatch by making sure that the return types from the relevant ghc-mod calls are types that are not exported by cabal.

e.g. http://hackage.haskell.org/package/ghc-mod-3.1.4/docs/Language-Haskell-GhcMod-Internal.html#v:cabalAllTargets returns [String] for the target names.

This could perhaps be extended to either use a separate library of stable types to be used by both cabal and ghc, or to return e.g. json

Alan


On Tue, Mar 11, 2014 at 9:19 AM, Maxim Kulkin <maxim.kulkin <at> gmail.com> wrote:
I played a bit with GHC API and Cabal (and even implemented .cabal support for hdevtools) and now I see issues with this approach:

1. GHC depends on Cabal (rly?) and this is most likely some old version of Cabal, not the one you use currently (e.g. I use cabal-install and Cabal 1.18, but GHC 7.6.3 -- the latest stable -- depends on Cabal 1.16). This makes it hard to develop tools that both use GHC API and latest Cabal. Hopefully this dependency will be removed in future versions of GHC.

2. Sometimes you get stuck with types and can't get source code compile and GHC API-based tools are of no help in this situation, because they won't be able to provide type for some variable in source file because this source file contains a syntax/type error in some other part. I'm not sure if GHC API can be tweaked to bypass errors and do the best effort of parsing/desugaring of other parts of sources with errors. Maybe some external parser (haskell-src-exts) should be used for source inspection.


On Tue, Mar 11, 2014 at 10:25 AM, Carlo Hamalainen <carlo <at> carlo-hamalainen.net> wrote:
On 10/03/14 22:22, Andrey Chudnov wrote:
If we go down that road, it's not cabal-repl that would need to be extended. AFAIK, it's just a wrapper around GHCi that calls that latter with package info extracted from the cabal file and, optionally, the sandbox. So, really, it's GHCi that would need to be extended. I think, something as simple as adding an option to output JSON (or some other structured format with wide support in editors) instead of plain text would go a long way to allow editors to consume it's output: they could just pipe commands via stdin and read the JSON output via stdout, parse it and display the info in the buffer. Depending on how GHCi looks inside and how amenable it is to such modifications, it might actually be a doable summer project. Am I missing any technical gotchas?

Regarding the data format, ghc-mod uses the GHC API to get the de-sugared and type-checked representation of a module, e.g.

https://github.com/kazu-yamamoto/ghc-mod/blob/master/Language/Haskell/GhcMod/Gap.hs#L326-L337

Or, my tool ghc-imported-from needs to get the "guts" from the GHC API to get the global reader environment:

https://github.com/carlohamalainen/ghc-imported-from/blob/master/Language/Haskell/GhcImportedFrom.hs#L343-L373

So my feeling is that we'd need to pass Haskell values back and forth, not just JSON.

Personally I would love it if GHCi was extended in the way that you are suggesting, as it would make my tool much faster. Ditto for ghc-mod.

-- Carlo Hamalainen http://carlo-hamalainen.net

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe



_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe




_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe




--
JP Moresmau
http://jpmoresmau.blogspot.com/
_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Bardur Arantsson | 11 Mar 19:48 2014
Picon

Re: On Haskell IDEs

On 2014-03-11 09:20, JP Moresmau wrote:
> Unifying all the different tools underlying IDEs has been talked again
> before, because the pain is real. For example, see
> https://github.com/fpco/haskell-ide/wiki. I've written about the GHC API in
> that wiki, and some of the issues that it has for such tools.
> Currently EclipseFP uses buildwrapper, which is a backend I wrote because I
> was dissatisfied with scion. It integrates the GHC API and Cabal, so that
> it aims to give the same results that running cabal configure/build. It
> usually run as a shortlived executable but for performance reason it is
> possible to have a long lived session to avoid reloading the modules in GHC
> all the times.
> Yes, GHC depends on the Cabal API for some types, which causes untold woes
> when you use buildwrapper after upgrading Cabal, for example. There is a
> bug report to remove that dependency but I had no time yet to work on it,
> so far I've used dynamic-cabal in buildwrapper to allow a newer version of
> Cabal, at the price of more complexity. But a complete tool would need to
> integrate both, so it can't be ghci only, otherwise Cabal and GHC would
> become really stuck together.
> The GHC API uses some global settings, which means you sometimes have no
> choice (if stcouldatic flags change) than to respawn a new process. This
> could probably be improved since it's not really the way a well behaved
> Haskell program should act. Memory usage is also a concern.
> 
> So I think there should be two lines of work here: one would be to improve
> the GHC API and/or the ghci interface, to make it more IDE friendly, the
> other would be to unify all these tools like scion/buildwrapper/ghc-mod in
> one tool that would hopefully provide all services an IDE needs, and that
> would allow easy plugin of additional functionality (so for example HaRe
> could easily add its refactoring capabilities to it). I would happily
> switch from buildwrapper to another community supported tool if all the
> functionality I currently have is kept in one form or another.
> 

Since you've obviously been doing a lot of investigation here, I thought
I might ask:

What is GHC using Cabal for in *functional terms*, i.e. what actual
functionality is it using? Is it just parsing package information files?
What is it using that information for? Would it be possible to supply
that information via other mechanisms than GHC snooping in the files?
I'm thinking something along the lines of pkg-config here.

(I understand the build system is a different issue, let's just ignore
that completely.)

Regards,
Carter Schonwald | 11 Mar 20:49 2014
Picon

Re: On Haskell IDEs

literally just for sharing the code for that parsing afaik.
I think theres a ticket for doing this.


On Tue, Mar 11, 2014 at 2:48 PM, Bardur Arantsson <spam <at> scientician.net> wrote:
On 2014-03-11 09:20, JP Moresmau wrote:
> Unifying all the different tools underlying IDEs has been talked again
> before, because the pain is real. For example, see
> https://github.com/fpco/haskell-ide/wiki. I've written about the GHC API in
> that wiki, and some of the issues that it has for such tools.
> Currently EclipseFP uses buildwrapper, which is a backend I wrote because I
> was dissatisfied with scion. It integrates the GHC API and Cabal, so that
> it aims to give the same results that running cabal configure/build. It
> usually run as a shortlived executable but for performance reason it is
> possible to have a long lived session to avoid reloading the modules in GHC
> all the times.
> Yes, GHC depends on the Cabal API for some types, which causes untold woes
> when you use buildwrapper after upgrading Cabal, for example. There is a
> bug report to remove that dependency but I had no time yet to work on it,
> so far I've used dynamic-cabal in buildwrapper to allow a newer version of
> Cabal, at the price of more complexity. But a complete tool would need to
> integrate both, so it can't be ghci only, otherwise Cabal and GHC would
> become really stuck together.
> The GHC API uses some global settings, which means you sometimes have no
> choice (if stcouldatic flags change) than to respawn a new process. This
> could probably be improved since it's not really the way a well behaved
> Haskell program should act. Memory usage is also a concern.
>
> So I think there should be two lines of work here: one would be to improve
> the GHC API and/or the ghci interface, to make it more IDE friendly, the
> other would be to unify all these tools like scion/buildwrapper/ghc-mod in
> one tool that would hopefully provide all services an IDE needs, and that
> would allow easy plugin of additional functionality (so for example HaRe
> could easily add its refactoring capabilities to it). I would happily
> switch from buildwrapper to another community supported tool if all the
> functionality I currently have is kept in one form or another.
>

Since you've obviously been doing a lot of investigation here, I thought
I might ask:

What is GHC using Cabal for in *functional terms*, i.e. what actual
functionality is it using? Is it just parsing package information files?
What is it using that information for? Would it be possible to supply
that information via other mechanisms than GHC snooping in the files?
I'm thinking something along the lines of pkg-config here.

(I understand the build system is a different issue, let's just ignore
that completely.)

Regards,


_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe

_______________________________________________
Haskell-Cafe mailing list
Haskell-Cafe <at> haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe

Gmane