17 February 2021
Derivation outputs in a content-addressed world
This is another blog post on the upcoming content-addressed derivations for Nix. We’ve already explained the feature and some of its advantages, as well as the reasons why it isn’t easy to implement. Now we’re going to talk about about a concrete user-facing change that this…
10 February 2021
Announcing linear-base, a standard library for linearly typed Haskell programs.
5 February 2021
Refinement and dependent types fellowship
This post is about compiling refinement types into a dependently typed core language, my project as an Open Source Fellow for Tweag. The post assumes some passing familiarity with the concept of dependent types, and little else. By reading this post, you will learn what a…
29 January 2021
Intern opening to improve GHC performance
A call to action to improve GHC's performance via a paid internship
22 January 2021
Programming with contracts in Nickel
In a previous post, I gave a taste of Nickel, a configuration language we are developing at Tweag. One cool feature of Nickel is the ability to validate data and enforce program invariants using so-called contracts. In this post, I introduce the general concept of programming…
7 January 2021
Haskell dark arts, part I: importing hidden values
How to break encapsulation and import a hidden value in Haskell.
16 December 2020
Trustix aims at enhancing trust in software binaries and improve reproducibility tracking using distributed binary caches.
25 November 2020
A tale of Template Haskell and cross compilation
Why Template Haskell is hard for cross compilation and how Asterius does it
18 November 2020
Self-references in a content-addressed Nix
Why wasn't Nix built with a content-addressed store?
11 November 2020
Pure destination-passing style in Linear Haskell
Destinations give you control over your memory allocations, and linear types make them pure.
28 October 2020
Markov chain Monte Carlo Sampling (4)
In the final post of Tweag's four-part series, we discuss Replica Exchange, a powerful MCMC algorithm designed to improve sampling from multimodal distributions. An illustrative example and, as always, an interactive Python notebook with easy-to-modify code lead to an intuitive understanding and invite experimentation.
22 October 2020
Nickel: better configuration for less
We are making the Nickel repository public. Nickel is an experimental configuration language developed at Tweag. While this is not the time for the first release yet, it is an occasion to talk about this project. The goal of this post is to give a high-level overview of the…
9 October 2020
Edge computing with servant on Cloudflare
Just a few days ago, Cloudflare and Stack builders published a excellent blog post where they demonstrate how to deploy a Haskell program to Cloudflare Worker with Asterius. In this post we go one step further and show that you can deploy full-fledged servant applications just as…
7 October 2020
GHCIDE Fellowship summary
Summary of the Tweag open source fellowship work related to ghcide
30 September 2020
Fully statically linked Haskell binaries with Bazel
Bazel gains support for creating fully statically linked Haskell binaries with rules_haskell and rules_nixpkgs.
23 September 2020
Meet Lagoon, a new open source tool for centralizing and querying semi-structured datasets.
16 September 2020
Implicit Dependencies in Build Systems
In making a build system for your software, you codified the dependencies between its parts. But, did you account for implicit software dependencies, like system libraries and compiler toolchains? Implicit dependencies give rise to the biggest and most common problem with…
20 August 2020
How Nix grew a marketing team
What do we need to focus on to increase adoption of Nix?
12 August 2020
Developing Python with Poetry & Poetry2nix
Learn how to use Nix to create highly reproducible Python builds that are aware of native dependencies.
6 August 2020
Markov Chain Monte Carlo Sampling (3)
Learn about Hamiltonian Monte Carlo, and how to implement it from scratch.
31 July 2020
Nix Flakes (3): Managing NixOS systems
How to manage NixOS systems using Nix flakes.
28 July 2020
Fall 2020 Open Source Fellowships
The second call for projects for the Tweag Open Source Fellowship is open and brings financial support and mentorship for Open source contributors.
13 July 2020
Announcing qualified do
Announcement of the upcoming QualifiedDo language extension.
8 July 2020
Setting up Buildkite for Nix-based projects
How to setup a Buildkite-based CI for Nix projects with workers running on GCP.
29 June 2020
Testing splittable PRNGs
How we ensured that the implementation of random v1.2 produces higher quality random numbers than v1.1.
25 June 2020
Nix Flakes (2): Evaluation caching
How Nix flakes enable caching of evaluation results of Nix expressions.
19 June 2020
Linear types are merged in GHC
Looking back at the journey which brought us there, and forward to what still lies ahead.
18 June 2020
Nix and Software Heritage
How Nix is collaborating with Software Heritage for long-term software reproducibility.
11 June 2020
Benchmarks of inline-java and a discussion of its performance trade-offs.
5 June 2020
Tweag Open Source Fellows
Tweag introduces our first cohort of Open Source Fellows.
25 May 2020
Nix Flakes (1): Introduction
An introduction to Nix flakes and a tutorial on how to use them.
6 May 2020
A Bazel for Haskell tutorial
How to build a complete Haskell project with Bazel.
16 April 2020
The three kinds of Haskell exceptions
Imprecise, synchronous, asynchronous exceptions: what do they mean, and what to do with them.
9 April 2020
Shared Bazel cache
How to set up a remote Bazel cache, and why it is a good idea.
2 April 2020
Eager vs. Lazy Instantiation
This blog post describes the tradeoffs of the choice between eager and lazy instantiation of type variables in GHC.
12 March 2020
Inferred or Specified Types? Your Choice!
During my internship, I implemented the explicit specificity proposal to make GHC type signatures complete.
5 March 2020
Code is Engineering, Types are Science
Peirce divides reasoning into three complementary processes: deduction, abduction and induction. In this post, I will show how these logical processes relate to software.
26 February 2020
Probabilistic Programming with monad‑bayes (3)
In this blog post series, we're going to lead you through Bayesian modeling in Haskell with the monad-bayes library. In the third part of the series, we setup a simple Bayesian neural network.
19 February 2020
On linear types and exceptions
A primer on the interaction between linear types and exceptions in Haskell.
14 February 2020
Tweag's Open Source Fellowship
The first call for the new Tweag Open Source Fellowship is open. It offers financial support and mentorship to open source contributors.
6 February 2020
A safer inline-java
In this post about inline-java I aim to walk you through the upcoming safe interface of the library, which allows detecting memory management mistakes at compile time using linear types.
30 January 2020
Profiling Large Haskell Codebases
This post describes a profiling technique for Haskell codebases that yields faithful results and is well adapted to large repositories.
16 January 2020
A Tale of Two Functors
Haskell's Data and Control module hierarchies have always bugged me. Now, I understand that the intuition behind the Data/Control separation is rooted in a deep technical justification.
9 January 2020
Markov chain Monte Carlo Sampling (2)
In this second post of Tweag's four-part series, we discuss Gibbs sampling, an important MCMC-related algorithm which can be advantageous when sampling from multivariate distributions. Two different examples and, again, an interactive Python notebook illustrate use cases and the issue of heavily correlated samples.
19 December 2019
Art in browser with Asterius
Asterius compiles Haskell code into WebAssembly code to be executed in a browser or in Node.js. It has reached a new milestone by being able to compile the diagrams library and its dependencies.
28 November 2019
How to make your papers run
How to use the Makam metalanguage to implement an executable formal semantics for a simple language.
21 November 2019
Untrusted CI using Nix
Learn how Nix's new post-build hooks feature can automatically sign and upload artifacts to a binary cache in a trusted way.
8 November 2019
Probabilistic Programming with monad‑bayes (2)
Here's Part 2 in Tweag's Series about Bayesian modeling in Haskell with the monad-bayes library.
30 October 2019
Porcupine: Announcing First Release
We're happy to announce the first release of Porcupine, an open source framework to express portable and customizable data pipelines.
25 October 2019
Markov chain Monte Carlo Sampling (1)
In this first post of Tweag's four-part series on Markov chain Monte Carlo sampling algorithms, you will learn about why and when to use them and the theoretical underpinnings of this powerful class of sampling methods. We discuss the famous Metropolis-Hastings algorithm and give an intuition on the choice of its free parameters. Interactive Python notebooks invite you to play around with MCMC yourself and thus deepen your understanding of the Metropolis-Hastings algorithm.
11 October 2019
Ormolu: Announcing First Release
We're happy to announce the first release of Ormolu, a formatter for Haskell source code.
9 October 2019
Third-party Haskell libraries in Bazel
Bazel gets native support for third-party Haskell libraries and building Cabal packages since the 0.10 release of rules_haskell.
25 September 2019
Bazel's Persistent Worker Mode for GHC
I got the opportunity to work on Bazel's Persistent Worker Mode for GHC during my internship at Tweag. My goal was to improve the mode of communication between Bazel and the Haskell GHC compiler.
20 September 2019
Probabilistic Programming with monad‑bayes (1)
In this blog post series, we're going to lead you through Bayesian modeling in Haskell with the monad-bayes library. In the first part of the series, we introduce two fundamental concepts of `monad-bayes`: `sampling` and `scoring`.
12 September 2019
War Stories of Asterius
I got the opportunity to work on Asterius, a new Haskell to WebAssembly compiler, during my internship at Tweag. My task was to get anything numerics-related stabilized in its compiled code.
6 September 2019
Why Haskell is important
People keep asking me, “What's the point of Haskell, why do you folks even use it?”. How can you answer this?
1 August 2019
Code Line Patterns
We visualize large collections of Haskell and Python source codes as 2D maps using methods from Natural Language Processing (NLP) and dimensionality reduction and find a surprisingly rich structure for both languages. Clustering on the 2D maps allows us to identify common patterns in source code which give rise to these structures. Finally, we discuss this first analysis in the context of advanced machine learning-based tools performing automatic code refactoring and code completion.
17 July 2019
Revelations from repetition
Every day we write repetitive code. A lot of it is boilerplate that you write only to satisfy your compiler/interpreter. But how do languages differ in their boilerplate content? We explore these questions using data sets of Python and Haskell code.
27 June 2019
CPP considered harmful
Edsger Dijkstra took issue with the “unbridled” use of the programming construct in 1968. He noted that our already limited ability to reason about the dynamic behaviour of imperative programs was nigh impossible in the presence of -jumps to arbitrary labels. With , control flow…
27 May 2019
Ormolu: a new Haskell formatter
If we think of the average Haskeller, who do we see? A poor, broken person. Tired head falls into the open palms, sobbing, deprived of the basic human right for automated source code formatting. Is it at all conceivable that so many attempts were made and none quite succeeded…
9 May 2019
Tweag.io has a bit of a history with language interop. By this point, we created or collaborated with others in the community on HaskellR, inline-c, inline-java, and now inline-js. The original idea for this style of interop was realized in language-c-inline by Manuel Chakravarty…
10 April 2019
The Sneakernet: Towards A Much Faster Internet
Inspired by the Event Horizon Telescope images, we develop a quick exploratory study about future possibilities of this technology called the Sneakernet: Could massive data transfer give a new live to the homing pigeon industry? How about using transportation means that are optimized to carry incredible amounts of weight? Or transportation means that are designed to be fast as a bullet?
3 April 2019
Storing secrets in Terraform
I want to present you a Terraform plugin for securely managing secrets that was written for Digital Asset, who kindly allowed me to open source it. The general idea of this plugin is to protect secrets by making use of Terraform’s state. Terraform maintains the state of the world…
28 March 2019
Introducing lorri, your project's nix-env
Today we’re excited to announce lorri, a new tool for NixOS, Linux, and macOS that makes developing with Nix even nicer: When people try lorri, we often hear that it is more magical than they expected. What is lorri? lorri is a nix-shell replacement for project development…
20 March 2019
Capability is about free monads
There has been recent online discussions about impredicative encoding (aka final encoding) of free monads. Capabilities-as-type-classes, as embodied by the capability library, are essentially the same thing.
11 March 2019
Internships at Tweag
The time has come again to invite applications for Tweag I/O’s paid internship programme this summer. For the past few years Tweag has taken on a number of students interested in developing their experience working on projects of benefit to the wider programming community. This…
7 March 2019
Kubernetes, KubeNix and kind
In a recent project I was tasked with creating a local testing environment for a Kubernetes cluster the client was actively working on. The main requirements were: Cross platform: It should work on Linux and macOS. Ease of use: The setup should be easy to use. Suitable for CI…
28 February 2019
Declarative, Reproducible Jupyter Environments
Millions of Jupyter notebooks are spread over the internet - machine learning, astrophysics, biology, economy, you name it. What a great age for reproducible science! Or that's what you think until you try to actually run these notebooks. Then you realize that having understandable high-level code alone is not enough to reproduce something on a computer. JupyterWith is a solution to this problem.
20 February 2019
Have you ever needed to execute in another computer a binary that you have built, only to find that it doesn’t work because it depends on shared libraries that aren’t installed? Or even worse, you don’t have admin rights on the machine, so you cannot install the correct…
13 February 2019
The types got you
Haskell is in a way a unique language. It is, to be clear, a language people and businesses use for building “serious stuff”, yet it is also a language which remains a platform for experimentation with functional programming. This means that the teams which develop in Haskell…
6 February 2019
Mapping a Universe of Open Source Software
The repositories of distributions such as Debian and Nixpkgs are among the largest collections of open source (and some unfree) software. They are complex systems that connect and organize many interdependent packages. In this blog post I'll try to shed some light on them from the perspective of Nixpkgs, mostly with visualizations of its complete dependency graph.
23 January 2019
Harnessing the Power of Haskell in JupyterLab
Introduction Haskell and data science - on first sight a great match: native function composition, lazy evaluation, fast execution times, and lots of code checks. These sound like ingredients for scalable, production-ready data transformation pipelines. What is missing then? Why…
20 December 2018
TodoMVC in Haskell via WebAssembly
Getting your browser to print “Hello World!” is fun, and a milestone for compiler writers targeting the web. Soon expectations change. Can you call existing code? Can you write a moderately complex web app? How fast does it run? Previously, we showed how Haskell can call…
12 December 2018
DIY benchmark history
If you’re a conscientious developer like I am my boss is, you probably have a benchmark suite for the programs and libraries you develop. This allows you to see the impact of your changes on the performance of your applications. However, the temporal aspect of these benchmark…
4 October 2018
About a year ago, Michael Snoyman made a blog post about the ReaderT pattern. Ostensibly, it’s a blog post about preferring to encode state as -s when you can. But that’s not how we read it. What we saw first and foremost is a story about using extensional type classes describing…
12 September 2018
10 July 2018
Funflow Example: emulating Make
Funflow is a workflow management tool. It turns out that workflow management tools and build tools are closely related. So if you’re more familiar with the latter, this post might be of interest to you. We’ll be illustrating some of Funflow’s features by implementing Make on top…
21 June 2018
Streaming with linear types
In an earlier blog post, we discussed how streaming libraries help writing composable programs without lazy I/O. We showed, for a simple program, how using a streaming library helps reducing the amount of unaided bookkeeping that the programmer needs to do in order to make it…
29 May 2018
Compiling Haskell to WebAssembly
Asterius is an experimental WebAssembly backend for GHC. We aim to support most GHC features while making the emitted WebAssembly code as small and fast as possible. The project recently reached an exciting milestone: for the first time, simple Haskell functions can be compiled…
25 April 2018
Funflow is a system for building and running workflows. It’s a system we’ve been working on for the last few months with a small number of clients. In this blog post we’ll talk about what workflows are, why we built Funflow, and what we’d like to do with it in the future. What is…
17 April 2018
Stackage HEAD is now live!
We are happy to announce that Stackage HEAD is now functional. For those who had missed the original blog post, or already forgot it, let’s refresh the idea behind Stackage HEAD. Each Stackage Nightly is a build plan that includes a large set of real-world packages that are known…
29 March 2018
We're hiring! Software engineers
We’re hiring multiple software engineers to join our team. Our mission: deliver correct high-performance software faster. We have found functional programming in particular to be an enabling factor for us on some of the most ambitious projects we’ve faced for our clients…
15 March 2018
Nix + Bazel
This is the second post in our series about building polyglot projects using Bazel. You might find it useful to start with our first installment. A is an extraordinary thing. You specify a set of targets, you tell Make what the dependencies are between these targets, and Make…
8 March 2018
Implementing a safer sort with linear types
In the “all about reflection” post, we introduced the data type. Any sorting function should have the type . However, this doesn’t actually guarantee that is indeed a sorting function: it only says that is a sorted list, but not that it is actually a sort of . For example is…
28 February 2018
Bazel now supports building Haskell
Publishing code to the world is easy these days: take your code, tack some extra metadata onto it, call that a package and upload it to npmjs.com / crates.io / Hackage / etc. It’s also easy for other developers to include your code as part of their project: build tools are…
8 February 2018
Internships at Tweag
Tweag I/O is inviting applications from students for our paid internship programme this summer. If you’re a student excited about working with Haskell, Nix and similar things, then get in touch! In previous years, our interns have worked on a variety of projects across research…
5 February 2018
Free monads for cheap interpreters
The utility of free monads can show up in surprising places. One of the projects I’m working on is an AI, and part of the strategy that it uses for responding to user input is quite simple: it generates many possible responses, and then evaluates them. Most of the computations it…
21 December 2017
All about reflection
An important device in the tool belt I carry around everyday is type class reflection. I don’t reach for it often, but it can be very useful. Reflection is a little known device. And for some reason it is often spoken of with a hint of fear. In this post, I want to convince you…
29 November 2017
Making two garbage collectors be good neighbours
Foreign function interfaces (FFI) allow fast interop between languages. Unlike other approaches, like performing RPC calls between different components written in different languages, using the FFI allows for all manner of data to be shared between each language runtime, in the…
16 November 2017
Parallelising your array code
This is the fifth post in a series about array programming in Haskell — you might be interested in the first, second, third, and fourth, too. A recurring theme in array programming is performance. After all, many algorithms in numerical computing and data science are…
10 November 2017
Nix on the Windows Subsystem for Linux
Nix on Windows: does it run yet? That’s the question I wondered about while testing the latest NixOS release, version 17.09. To that end, I had the idea of running the Nix installation process from inside the Windows Subsystem for Linux (WSL) see if it worked. And it worked…
1 November 2017
The Exodus to Streamgard, an epic poem
If Haskell was a god, often would he be depicted with the ravens Modularity and Abstraction flying above him, hovering the world and reporting to him every detail of our whereabouts. Haskell would sit on the Throne of Purity and look upon the world with an eye full of wisdom. And…
27 October 2017
Using Stackage for GHC regression testing
A recent development in Haskell land is the formation of the GHC DevOps Group, which was the topic of last week’s blog post. The group is a community of parties committed to the future of GHC. Tweag I/O is one such party. We are helping the group achieve its goals with concrete…
19 October 2017
Announcing the GHC DevOps Group
As Haskell is increasingly used in production environments, the Haskell toolchain is of critical importance to a growing number of people and organisations. At the heart of this toolchain is GHC (The Glasgow Haskell Compiler). Conceived nearly thirty years ago as a research…
12 October 2017
Array fusion with vector
This is the fourth post in a series about array programming in Haskell — you might be interested in the first, second, and third, too. In the previous post of this series, we explored the basic, down-to-earth, index-based array interfaces that have their roots in the Haskell…
5 October 2017
How to compose streaming programs
In our first blog post about streaming, we discussed how streaming libraries strengthen types to catch more errors. However, when we strengthen types, we need to be careful to not hinder program composition. After all, precise type information can make it more onerous to convince…
27 September 2017
Immutability and unboxing in array programming
This is the third post in a series about array programming in Haskell — you might be interested in the first and second, too. In the previous post of this series, we discussed commonly used vector and matrix routines, which are available in highly-optimised implementations in…
22 September 2017
GHC compiler plugins in the wild: typing Java
Previously, we discussed how to use inline-java to call any Java function from Haskell. The reverse is also possible, though that will be a topic for a future post. In this post, we’ll peek underneath the hood to talk a little about how inline-java does its deed. You might find…
15 September 2017
Java from Haskell: a tutorial
Our introductory post for inline-java showed that we could call methods written in Java (or indeed any JVM function) from Haskell. Much in the style of other packages, it is moreover possible to do using java syntax, so examples from Java API documentation can be reused as-is. In…
6 September 2017
In this post I’ll show you how to see the performance of your software project improve over the entire lifetime of the project. That is, how to get something like the below picture. For that, you’ll need to systematically track and store benchmark results over time. We’ll use…
31 August 2017
Enter the matrix, Haskell style
This is the second post in a series about array programming in Haskell — you might be interested in the previous post, too. Matrices are the bread and butter of most scientific and numeric computing. It is not surprising then that there is a range of standard libraries and…
24 August 2017
Compact normal forms + linear types
We saw last time that with linear types, we could precisely capture the state of sockets in their types. In this post, I want to use the same idea of tracking states in types, but applied to a more unusual example from our paper: sending rich structured data types across the…
17 August 2017
Diversity through inline code
Haskell is an awesome language, but we need to remember that it is not very useful in isolation. In almost any realistic application, Haskell has to coexist with other languages, even if only to call existing C libraries or to make use of operating system services. In actual…
9 August 2017
Array programming in Haskell
Array programming in Haskell is attractive, but it is also somewhat confusing. Haskell’s functional syntax is close to the mathematical notation that is often used to develop and explain the algorithms on which array programs are based, which leads to great clarity of expression…
3 August 2017
Encode state transitions in types using linear types
At the time of our first post on linear types, we were fresh out of the design phase to extend GHC with linear types. We had a prototype implementation, but it was just a proof of concept: there was precious little you could do with it. A few months down the line, we are now in a…
1 August 2017
I am a functional programming evangelist
I have had a passion for programming ever since I wrote my first piece of code as a kid. I am not sure why. Maybe it is the absolute, god-like control over the behaviour of the machine, or losing myself in the details of the code. In any case, it is intriguing. When teenage play…
27 July 2017
Streaming programs without laziness
In school, we’re taught that I/O is a simple affair: read data in, write data out. Rinse, repeat. But then as practitioners we realize matters are often more complicated. For one, I/O is slow, so we probably want operations to overlap (i.e. be processed asynchronously…
10 July 2017
R and Haskell: best of both worlds with HaskellR
A guest post by Sigrid Keydana telling us the backstory behind the very cool trading app notebook she put together… Keras, ggplot2, Haskell and R all in one Jupyter notebook! Post originally appeared here. Earlier today, I presented at UseR! 2017 about HaskellR: a great piece of…
23 May 2017
Théophane is a summer intern here at Tweag I/O. Thanks also in part to the financial help from the Nix community, he’s spending six months on devising a type system for the Nix language. Nix, the purely functional package manager Nix is a cross platform tool for managing the…
18 May 2017
We're hiring! Technical evangelist
Our mission: deliver correct high-performance software faster. To do that, we recruited a team of experimentalists who use their formal training in a variety of theories and their experience to continuously improve our methodology and publish new tech. Theories that are…
13 March 2017
Linear types make performance more predictable
We’re extending GHC with linear types. Ever since Jean-Yves Girard discovered linear logic in 1986, researchers around the world have been going “wow! resource tracking, this must be useful for programming languages”. After all, any real computation on a real machine takes…
17 October 2016
A new ecosystem for Haskell: the JVM
20 June 2016
Haskell compute PaaS with sparkle
Maintaining a compute cluster for batch or stream data processing is hard work. Connecting it up to storage facilities and time sharing resources across multiple demands even more so. Fortunately cloud service providers these days typically upscale their offering to not just…
25 February 2016
Haskell meets large scale distributed analytics
Large scale distributed applications are complex: there are effects at scale that matter far more than when your application is basked in the warmth of a single machine. Messages between any two processes may or may not make it to their final destination. If reading from a memory…
9 December 2015
Stack + Nix = portable reproducible builds
Stack and Cabal are great tools for incremental development. But before even starting to think about what your first commit will be about on a freshly checked out project, what you have to do is get a working development environment, by which we mean: see if it builds! Stack…
8 September 2015
Programming R at native speed using Haskell
Most good data stories start with a interesting question. If the average request latency went down by a further 100ms, by how much could we expect user engagement to increase? How can we detect evidence of corruption of government officials given a list of all bids nationwide for…