As a Tweag Open Source
fellow, I aimed to
improve and build on the Haskell IDE experience, mainly by contributing to the
haskell-language-server projects. My main goals were to polish up
the overall experience, and integrate
hiedb, a product of a Summer of Code
project last year, into
The product of this fellowship was a good selection of
haskell-language-server features that you can use right now, or will be able
to use very soon, including better search, richer information and more
efficient queries. Let’s go through these features.
hiedb: searching references and workspace symbols
hiedb is a tool to index and query
files that I have been working on for some time. It reads
.hie files and
extracts all sorts of useful information from them, storing it in a
database for fast and easy querying.
ghcide has many obvious (and non-obvious) benefits.
For example, we can finally add support for
find reference, as well as
allowing you to search across all the symbols defined in your project.
In addition to this, the
hiedb database serves as an effective way to
persist information across
ghcide runs, allowing greater responsiveness,
ease of use and flexibility to queries.
hiedb also works extremely well for
saving information that is not local to a particular file, like definitions,
documentation, types of exported symbols and so on.
Under this paradigm,
ghcide acts as an indexing service for
generating .hi and .hie files which are indexed and saved in the database,
available for all future queries, even across restarts. A local cache of
files/typechecked modules is maintained on top of this to answer queries for the
files the user is currently editing, while non-local information about other
files in the project is accessed through the database.
This work is being carried out in this
branch and should land in
Responsive IDEs using stale information
I discussed, in an earlier blog
could only process a single request at a time, and was cancelling old requests,
which lead to slow response times, and features like completion being almost
The solution mentioned in the blog post above has now been merged into mainline
ghcide, but with a few major changes. Pepe Iborra came up with an alternative
approach that allowed managing the Shake session in a more fine grained manner,
which let us eliminate needless restarts without the need for another queue.
Here are a few graphs that demonstrate the massive improvements in response times that are achieved by using stale information:
As also mentioned in the blog post, my
hiedb-4 branch of
ghcide can also pick on on
.hie files written by the previous run of
ghcide, to allow you to immediately
use your IDE even before the initial configuring and typechecking step has run.
Typechecking your entire project
With this PR,
typecheck your entire project on the initial run, and when you save a file, typecheck
all the files that (transitively) depend on that file to give you
up to date and accurate diagnostics for your entire project.
This behaviour is completely configurable, so if you don’t want to see
diagnostics for your entire project and only want to see them for the files you
have open, you can configure
ghcide to do so using your editor’s LSP configuration.
We could not do this earlier because
ghcide did not know what the module
graph of your project looked like. We have plumbed in this information to the
correct places now, so that
ghcide can perform these crucial functions.
Find all variables of a given type
Sometimes, you want to know all the places that could potentially be affected if
you change the definition of a type. Maybe you want to gauge how much a type is
used, to check how painful it would be to delete it. Well, now you can, using the
find references for a type, and your editor
will also highlight all the variables which mention it.
You can restrict the query to a particular depth. For example,
foo :: Int foo = 1 bar :: [Maybe Int] bar = Just foo
the type of
Int at a depth of 0. The type of
Int at a depth of 2.
Here, you can see that viewing references for
VFSHandle also highlights all
the symbols that include
VFSHandle in their type.
Use the IDE on all your dependencies!
It is always fun to zip around your code using the
goto definition and
find references features. But this comes to quick stop as soon as you try to go to
the definition of a function not defined in your project, but imported from an
external dependency. Don’t worry,
.hie files can come to the rescue!
In addition to all sorts of other useful information,
.hie files also contain
the original source of the Haskell file they were generated from. If
knows about the
.hie files for your dependencies, it can use those to show you
This is also available on the
First, you need to generate
.hie files for your dependencies. This can be
easily done by adding the following to your
package * ghc-options: -fwrite-ide-info -hiedir <some-directory>
Then you simply need to inform ghcide of the directory you told
ghc to put
hie files in, and you are ready to go! Thanks to all the useful
.hie files, many of the IDE features like types and
documentation on hover, go to definition, references and more will be available
on these files also, so your IDEing can continue on seamlessly.
Currently, it is not possible to navigate into boot libraries (the libraries that
ship with GHC) using this, as the standard distribution of GHC doesn’t ship with
.hie files for these libraries, and it is quite an involved procedure to compile
these libraries yourself.
Hopefully, future versions of GHC will ship with
.hie files so that we can
navigate into those too using
Scope aware local completions
Recently, Sandy Maguire wrote a plugin for
haskell-language-server that added
the ability to
plugin required the ability to get all variables in scope at a particular point
in the source. For this, we needed a data structure that could store this
information and allow efficient queries. We settled on the an
populated with scoping information obtained from the
.hie file, which allows
querying all the identifiers available at a particular point, along with their
Once we had this scoping information, it was very simple to use it to
ghcide completion subsystem to generate accurate completion
for local variables. This is now available in the branch of
ghcide used by
haskell-language-server, and will soon come to
A type safe interface for LSP
I have also been working on improvements to the
haskell-lsp library that
hie, providing a Haskell interface for
the Language Server Protocol that makes it possible to communicate with editors . We are
in the process of moving to a type safe encoding of the
LSP specification, which
should make the API much more usable and less prone to errors, as well as making
it much easier to detect any deviations from the specification.
This also leads to great improvements in the interface of the
library, so that the compiler can infer and check the shape of the data
you are sending matches the method type
The PR also brings a host of
other improvements and bug fixes for the
haskell-lsp library, simplifying and
cleaning up much of the code and making the interface much more consistent.
Go to instance definition and view all usages of an instance
Recently, one of my
MRs was merged into GHC, which adds information about typeclass evidence to
.hie files. This will allow for features like going to the definition of an
instance used at a particular point, or viewing all usages of a particular
instance across all your code. These features can be added to
GHC 9.0 lands.
I’ve implemented a proof of concept for this in the
Call Hierarchy graphs
The upcoming 3.16 version of the Language Server Protocol has added support for Call Hierarchies in the editor. This will allow language servers to expose the call graph of the project to user.
Fortunately for us,
hiedb has supported generating call graphs for a while,
as Graphviz graphs. It will be relatively straightforward to adapt this
functionality for LSP.
Types for all subexpressions in GHC
Currently, it is not possible to easier extract the type of arbitrary expressions from the GHC AST. Currently the most straightforward way to do this is to desugar the expression, and then extract the type from the desugared expression. Unfortunately, this approach doesn’t scale when you want to extract the types of all subexpressions in a particular program, since it means that you have to desugar an exponential amount of expressions!
I have been working on a custom compiler
pass building off
of work done by
Ben Gamari, which will annotate expressions with their types. This will allow
tooling such as
.hie files to include this information, and will allow
ghcide to tell you the type of any arbitrary expression in your program.
The open source fellowship was a great opportunity. It was very enlightening and helpful to interact with Tweagers. I really enjoyed the freedom allowed to choose where to focus my efforts myself. The funding allowed me to devote a significant amount of time to working on open-source and improve the Haskell developer experience.
IDEs are not easy to write and maintain, and they require a lot of effort to develop and extend. Funding is absolutely critical for this. Fortunately, this year we had two IDE related GSOC projects, along with this Tweag Fellowship, so were able to make great strides with the help of all other volunteers who work and contribute to these projects.