28 November 2023
Source filtering with file sets
Introducing the lib.fileset Nix library for easy and safe source filtering
23 November 2023
Separating debug symbols from executables
Make your C/C++ builds faster and smaller with debug fission.
16 November 2023
tl;dr: We’re pleased to announce the beta release of Organist, a tool designed to ease the definition of reliable and low-friction development environments and workflows, building on the combined strengths of Nix and Nickel. A mess of cables and knobs I used to play piano as a…
9 November 2023
Integrating Testwell CTC++ with Bazel
A Bazel war story of how we integrated a proprietary code coverage tool that does not play according to Bazel's rules.
2 November 2023
Great Nickel configurations from little merges grow
Explore how to write modular and reusable configuration in Nickel using its merge system.
19 October 2023
JSON Schemas to Nickel contracts
Get better error reporting and LSP integration for your JSON Schemas in Nickel.
12 October 2023
Use monad-bayes and rhine in your interactive machine learning application
21 September 2023
Behind the scenes with FawltyDeps 0.13.0
FawltyDeps 0.13.0 introduces a brand new mapping strategy. In this post, we'll delve into the mechanics of how dependencies and imports are matched, as well as how you can leverage these new features to boost your Python dependency management workflow.
10 August 2023
Supercharging your Rust static executables with mimalloc
How to link against mimalloc when creating a static Linux executable for Rust, and why you should do the same.
20 July 2023
How to Prevent GHC from Inferring Types with Undesirable Constraints
A GHC pattern for improving the user experience of type-level programs
6 July 2023
A Tour Around Buck2, Meta's New Build System
An overview of Meta's new build system Buck2 and how it differs from Bazel
29 June 2023
Packaging Topiary in OPAM
Now that Topiary is out and supports OCaml, we need to package it. But how exactly does one package a Rust projet in OPAM, the OCaml Package Manager?
22 June 2023
Assumptions for Liquid Haskell in the large
A revamp of the mechanism for introducing assumptions
15 June 2023
Coverage-guided fuzzing of Haskell programs for cheap
A simple trick to fuzz Haskell programs with libFuzzer
8 June 2023
halide-haskell is a Haskell wrapper for Halide that allows you write and just-in-time compile high-performance array and image processing kernels.
1 June 2023
Testing Control-Flow Translations in GHC
Translations of control flow can be tested before the rest of the code generator is ready. The tests examine sequences of abstract state transformations and observations.
30 May 2023
Announcing Tf-Ncl 0.1
With Nickel releasing 1.0 I'm excited to announce the 0.1 release of Tf-Ncl, a tool for writing Terraform deployments with Nickel instead of HCL.
25 May 2023
PBT and the Ghost in the Machine
Commandeering techniques from richly typed, functional languages into Python for fun and profit. In this episode: Testing strategies.
17 May 2023
Announcing Nickel 1.0
Today, I am very excited to announce the 1.0 release of Nickel. A bit more than one year ago, we released the very first public version Nickel (0.1). Throughout various write-ups and public talks (1, 2, 3), we’ve been telling the story of our dissatisfaction with the state of…
11 May 2023
A journey through the auditing process of a smart contract
Embark with the High Assurance Software Group for a guided tour through the stages of a smart-contract audit, secret weapon included.
27 April 2023
Type-safe data processing pipelines
Using type-level dependency tags to verify a Haskell pipeline.
20 April 2023
Processing medical images at scale on the cloud
To allow innovation in medical imaging with AI, we need efficient and affordable ways to store and compute at scale.
13 April 2023
crem: compositional representable executable machines
How to build state machines in a composable way, controlling which transitions can be executed
4 April 2023
Python Monorepo; Part 1
How to build your Python monorepo from scratch: structure and tooling
23 March 2023
Linear Constraints: the problem with scopes
How linear constraints get rid of scope functions and why it matters.
14 March 2023
FawltyDeps is a new tool to help you identify undeclared and unused dependencies in your Python code, making your projects leaner and more reproducible.
9 March 2023
We're happy to announce the first release of Topiary, a formatter for many languages that leverages Tree-sitter.
2 March 2023
Chainsail goes open-source
Tweag releases the full source code of the Chainsail web service, for sampling multimodal distributions, first announced in August 2022. This blog post gives a tour of the Chainsail service architecture, links out to the relevant parts of the source code and proposes possible extensions to Chainsail for which the Tweag team would welcome contributions from the community.
23 February 2023
Taking the pulse of infrastructure management in 2023
We went to FOSDEM23 and CfgMgmtCamp23. Here are my impressions of the trends and evolutions in infrastructure and configuration management
16 February 2023
opam-nix: Nixify Your OCaml Projects
We present opam-nix, show how it helps to easily develop, build and test your OCaml projects with Nix
14 February 2023
smtlib-backends: faster SMT-LIB-based Haskell interface to SMT solvers
Announcement of smtlib-backends, a Haskell library providing a generic interface for interacting with SMT solvers using SMT-LIB
9 February 2023
Running a NixOS VM on macOS
Make your NixOS development on macOS easier with this simple trick...
31 January 2023
A Semester of HLS: An Internship Report
An experience report on improving HLS during my internship at Tweag.
26 January 2023
Linear Constraints: the problem with O(1) freeze
How a linear constraint arrow relates to freezing arrays in Haskell.
24 January 2023
Nix with; with Nickel
The Nix `with` keyword is challenging, but we transpiled it to Nickel anyway.
19 January 2023
Dial M for Monoid
Commandeering techniques from richly typed, functional languages into Python for fun and profit. In this episode: Typeclasses and continuation-passing style.
12 January 2023
Haskell at Work
Learning and adopting Haskell is not hard with the right resources at hand.
22 December 2022
Making GHC faster at emitting code
How we gained 2–3% improvements in compile times by making GHC better at printing code.
15 December 2022
A Bazel and Nix Migration
Getting an old Make based project building again after 10 years with the help of Bazel and Nix.
13 December 2022
Optimizing NixOS Search
An internship report on my experience working on improving search.nixos.org and my internship at Tweag
8 December 2022
Creating a Delta Lake in Haskell with Sparkle
A practical introduction to using Sparkle to write Haskell programs which interface with Delta Lake.
1 December 2022
Higher-orderness is first-order interaction
Revisit the fundamental concept of higher-order functions with a new perspective: interaction.
24 November 2022
Threads and messages with Rust and WebAssembly
How and why to share threads in WASM workers (and when not to)
22 November 2022
WebAssembly backend merged into GHC
Announcing the GHC WebAssembly backend – present and future
15 November 2022
Staged programming with typeclasses
Introduction to staged programming in Haskell with typeclasses
10 November 2022
Reinforcement Learning for Open Games
How to tap into the power of reinforcement learning while specifying and executing open games in Haskell.
8 November 2022
Functional programming from sets and functions
Understanding the basic ideas of functional programming by writing a program using only sets and functions
3 November 2022
Recompilation avoidance in rules_haskell
Bazel avoids superfluous recompilation for free, but we can make it even better.
1 November 2022
Hard user separation with NixOS
Setup two encrypted partitions with a shared Nix store
27 October 2022
A nub in the haystack
Exploration of a regression in GHC's bytecode generator performance
25 October 2022
Chainsail goes Bayesian statistics
Tweag intern Abdellatif summarizes his internship, in which he augmented Chainsail with a Bayesian Replica Exchange scheme to improve sampling of multimodal distributions.
20 October 2022
Converting a polyglot project build to Bazel: part 1
Painlessly switch your Haskell build from Cabal to Bazel
18 October 2022
My experience improving monad-bayes, the probabilistic programming language package, as a Tweag fellow.
14 October 2022
Testing stateful systems, part two: Linear Temporal Logic
On a previous post, we explained how to write tests for stateful systems using traces — sequences of stateful actions — that can be combined and modified to write complex test cases easily and transparently. This post elaborates on the combinators used to generate new traces from…
11 October 2022
Worry-free NixOS refactors
Comparing Nix input hashes can guarantee that some refactors are bug-free.
29 September 2022
Four months into The Nix Book
A comprehensive report on what happened in four months of dedicated work on Nix documentation, and what I learned from it.
22 September 2022
Building Nix flakes from Rust workspaces
Learn the options for building a Nix flake out of a Rust workspace, outputting a mix of native code and WebAssembly.
20 September 2022
Optimizing Nickel's Array Contracts
A short dive into Nickel's reference interpreter
13 September 2022
Construction and analysis of the build and runtime dependency graph of nixpkgs
A nixpkgs content database with graph building!
8 September 2022
Commandeering techniques from richly typed, functional languages into Python for fun and profit. In this episode: Gradual typing and algebraic data types.
25 August 2022
Introducing the Scalable Build Systems Team
Introducing the Scalable Build Systems Team
18 August 2022
Introduction to NixOS specialisations
How and when to use NixOS specialisations to manage different boot environments
11 August 2022
Soft k-means clustering with Chainsail
We recently released a beta version of a web service called Chainsail, that helps with sampling multimodal probability distributions (you can check out our Chainsail announcement blog post for a quick introduction to this tool). In parallel, this post aims to illustrate a use…
9 August 2022
Chainsail: sampling multimodal distributions made easy
Tweag announces Chainsail, a simple-to-use web service for better sampling of multimodal distributions with a scalable and auto-tuning Replica Exchange algorithm at its core.
28 July 2022
Profiling non-CPU time in Haskell
Limitations of the GHC profiler and announcement of the timestats library
14 July 2022
Nix – taming Unix with functional programming
Nix is all about applying functional programming to files and processes.
1 July 2022
Introducing Pirouette 2: formal methods for smart contracts
The new Pirouette 2 introduces formal method techniques to the verification of smart contracts; in this post we focus particularly in how incorrectness logic helps this goal.
23 June 2022
Incremental builds for Haskell with Bazel
Announcing the new haskell_module rule in rules_haskell
2 June 2022
Stack and Nix integration
Integrating Haskell Stack and Nix in a smooth yet non-invasive way, with an explanation, example and template.
26 May 2022
Reproducible probabilistic programming environments
How to get reproducible development environments for probabilistic programming packages such as PyMC3, Theano or TensorFlow using Nix.
12 May 2022
Comparing strict and lazy
Arnaud takes a step back from the slogans and examines the trade-offs between lazy and strict languages.
5 May 2022
A presentation of an alternative encoding for optics such as lenses and prisms using existential types.
28 April 2022
Union and intersection contracts
Why union and intersection contracts are hard in a lazy language, and how Nickel handles them.
21 April 2022
Announcement of a WebAuthn Haskell library, and a description of how the protocol works
31 March 2022
Running wasm modules as if they are native executables
Introduce the trick we use to run ghc test suite that needs to exec wasm code
25 March 2022
Our first audit of Minswap and the aftermath
What Happened On the morning of the 22nd of March, 2022, we were notified about the existence of an unknown critical vulnerability on one of Minswap’s contracts. Everyone here on the audit team was dismayed, since we had conducted an audit for Minswap not long before. We found…
17 March 2022
Porting libffi to pure WebAssembly
Cheng demonstrates why it's challenging to port libffi to wasm32, how he implemented it, and how it's useful for cross-compiling the GHC RTS
11 March 2022
First release of Nickel
I am excited to announce the first release of Nickel! In the original introductory blog post, I’ve written about why we, at Tweag, are developing yet another configuration language. Our goal is to empower people to write correct, programmable and maintainable configurations…
3 March 2022
TLDR; Benchmarking in Rust is now faster, easier, and runs in your browser. My name is David and for the past three months, Tweag has generously sponsored my work on criterion.rs. Criterion.rs is a widely used (with nearly 1.4 million downloads in the last 90 days) benchmarking…
17 February 2022
Writing a type-checking plugin
A tutorial on how to write a type-checking plugin for GHC
10 February 2022
Ormolu: formatting operator chains
Why formatting chains of infix operators is hard in Haskell. And how I addressed it for Ormolu.
3 February 2022
Trustix - Consensus and voting
We discuss existing consensus models and why no global consensus mode is well suited to Trustix, opting for a user-defined algorithm instead.
26 January 2022
A Case Study on Correctness and Safety Testing of Stateful Systems
How to automatically inject attacks and faults in safety tests using random traces.
19 January 2022
Why Liquid Haskell matters
On the relevance of Liquid Haskell in programming languages
14 January 2022
In this post we introduce Merkle trees, the data structures needed to make the Trustix log append-only and searchable.
5 January 2022
Scoped Effect Resources for Polysemy
How to transparently separate resource allocation from business logic for scoped effects in Polysemy
15 December 2021
21.11 Zero Hydra Failure Moscow report
Details about the 21.11 Nix Moscow Hackaton, and some suggestions for organizers of similar events.
9 December 2021
GHC's constraint solver
In part two of this series on type-checking plugins, we will see how GHC's constraint solver works.
2 December 2021
Implementing a content-addressed Nix
What had to change in the Nix internals to make it able to build content-addressed derivations?
25 November 2021
The Varieties of the Haskelling Experience
A collection of ways various Tweagers develop in Haskell
17 November 2021
Safe Sparkle: a resource-safe interface with linear types
On design choices to build a resource-safe interface for Sparkle using linear types
4 November 2021
Profiling Haskell with Cachegrind
Small tutorial on using valgrind to find hot spots in ghc and other Haskell programs
28 October 2021
Hacking on Ormolu: An internship report
An experience report on polishing Ormolu, the Haskell code formatter, during my internship at Tweag.
21 October 2021
Why write a type-checking plugin?
Why would you want to write a type-checking plugin? Many limitations of the GHC type-checker are exposed here.
13 October 2021
Denotational Homomorphic Testing
Property test a denotational model of a data structure.
30 September 2021
A higher-order integrator for Hamiltonian Monte Carlo
A discussion and benchmark of an alternative integrator for Hamiltonian Monte Carlo.
23 September 2021
Functional data pipelines with funflow2
Introducing a library for writing data pipelines which compose well and fail early
8 September 2021
Building a Go project using Bazel
Building a Go project using Bazel, generating build files with Gazelle
28 July 2021
Convert Cabal-based projects to Bazel automatically
Announcing the gazelle_cabal tool for automatic Bazel rules generation for Cabal projects
21 July 2021
Integrated shrinking in QCheck
An introduction to Integrated Shrinking and an implementation in QCheck (OCaml)
14 July 2021
Searching for COVID-19 treatments using metabolic networks
Apostolos Chalkis introduces dingo, a package for the analysis of metabolic networks, on which he worked as part of this Tweag fellowship.
1 July 2021
Building OCaml Projects with Bazel
Building OCaml projects with the new Bazel ruleset, OBazl
15 June 2021
AsciiDoc in Pandoc and Haskell
Announcing asciidoc-hs, a new AsciiDoc parser and converter in Haskell, adapted for use with Pandoc.
15 April 2021
Arrows, through a different lens
How to use lenses and other optics within an Arrow-based framework.
8 April 2021
Ad-hoc interpreters with capability
Define effect interpreters dynamically using reflection
18 March 2021
Types à la carte in Nickel
A dive into Nickel's gradual type system, which mixes dynamic and static typing
17 February 2021
Derivation outputs in a content-addressed world
This is another blog post on the upcoming content-addressed derivations for Nix. We’ve already explained the feature and some of its advantages, as well as the reasons why it isn’t easy to implement. Now we’re going to talk about about a concrete user-facing change that this…
10 February 2021
Announcing linear-base, a standard library for linearly typed Haskell programs.
5 February 2021
Refinement and dependent types fellowship
This post is about compiling refinement types into a dependently typed core language, my project as an Open Source Fellow for Tweag. The post assumes some passing familiarity with the concept of dependent types, and little else. By reading this post, you will learn what a…
29 January 2021
Intern opening to improve GHC performance
A call to action to improve GHC's performance via a paid internship
22 January 2021
Programming with contracts in Nickel
Presenting Nickel: better configuration for less Programming with contracts in Nickel Types à la carte in Nickel Great Nickel configurations from little merges grow In a previous post, I gave a taste of Nickel, a configuration language we are developing at Tweag. One cool feature…
7 January 2021
Haskell dark arts, part I: importing hidden values
How to break encapsulation and import a hidden value in Haskell.
16 December 2020
Trustix aims at enhancing trust in software binaries and improve reproducibility tracking using distributed binary caches.
25 November 2020
A tale of Template Haskell and cross compilation
Why Template Haskell is hard for cross compilation and how Asterius does it
18 November 2020
Self-references in a content-addressed Nix
Why wasn't Nix built with a content-addressed store?
11 November 2020
Pure destination-passing style in Linear Haskell
Destinations give you control over your memory allocations, and linear types make them pure.
28 October 2020
Markov chain Monte Carlo Sampling (4)
In the final post of Tweag's four-part series, we discuss Replica Exchange, a powerful MCMC algorithm designed to improve sampling from multimodal distributions. An illustrative example and, as always, an interactive Python notebook with easy-to-modify code lead to an intuitive understanding and invite experimentation.
22 October 2020
Nickel: better configuration for less
Presenting Nickel: better configuration for less Programming with contracts in Nickel Types à la carte in Nickel Great Nickel configurations from little merges grow We are making the Nickel repository public. Nickel is an experimental configuration language developed at Tweag…
9 October 2020
Edge computing with servant on Cloudflare
Just a few days ago, Cloudflare and Stack builders published a excellent blog post where they demonstrate how to deploy a Haskell program to Cloudflare Worker with Asterius. In this post we go one step further and show that you can deploy full-fledged servant applications just as…
7 October 2020
GHCIDE Fellowship summary
Summary of the Tweag open source fellowship work related to ghcide
30 September 2020
Fully statically linked Haskell binaries with Bazel
Bazel gains support for creating fully statically linked Haskell binaries with rules_haskell and rules_nixpkgs.
23 September 2020
Meet Lagoon, a new open source tool for centralizing and querying semi-structured datasets.
16 September 2020
Implicit Dependencies in Build Systems
In making a build system for your software, you codified the dependencies between its parts. But, did you account for implicit software dependencies, like system libraries and compiler toolchains? Implicit dependencies give rise to the biggest and most common problem with…
12 August 2020
Developing Python with Poetry & Poetry2nix
Learn how to use Nix to create highly reproducible Python builds that are aware of native dependencies.
6 August 2020
Markov Chain Monte Carlo Sampling (3)
Learn about Hamiltonian Monte Carlo, and how to implement it from scratch.
28 July 2020
Fall 2020 Open Source Fellowships
The second call for projects for the Tweag Open Source Fellowship is open and brings financial support and mentorship for Open source contributors.
8 July 2020
Setting up Buildkite for Nix-based projects
How to setup a Buildkite-based CI for Nix projects with workers running on GCP.
29 June 2020
Testing splittable PRNGs
How we ensured that the implementation of random v1.2 produces higher quality random numbers than v1.1.
25 June 2020
Nix Flakes (2): Evaluation caching
How Nix flakes enable caching of evaluation results of Nix expressions.
19 June 2020
Linear types are merged in GHC
Looking back at the journey which brought us there, and forward to what still lies ahead.
18 June 2020
Nix and Software Heritage
How Nix is collaborating with Software Heritage for long-term software reproducibility.
11 June 2020
Benchmarks of inline-java and a discussion of its performance trade-offs.
25 May 2020
Nix Flakes (1): Introduction
An introduction to Nix flakes and a tutorial on how to use them.
16 April 2020
The three kinds of Haskell exceptions
Imprecise, synchronous, asynchronous exceptions: what do they mean, and what to do with them.
2 April 2020
Eager vs. Lazy Instantiation
This blog post describes the tradeoffs of the choice between eager and lazy instantiation of type variables in GHC.
12 March 2020
Inferred or Specified Types? Your Choice!
During my internship, I implemented the explicit specificity proposal to make GHC type signatures complete.
5 March 2020
Code is Engineering, Types are Science
Peirce divides reasoning into three complementary processes: deduction, abduction and induction. In this post, I will show how these logical processes relate to software.
26 February 2020
Probabilistic Programming with monad‑bayes (3)
In this blog post series, we're going to lead you through Bayesian modeling in Haskell with the monad-bayes library. In the third part of the series, we setup a simple Bayesian neural network.
19 February 2020
On linear types and exceptions
A primer on the interaction between linear types and exceptions in Haskell.
14 February 2020
Tweag's Open Source Fellowship
The first call for the new Tweag Open Source Fellowship is open. It offers financial support and mentorship to open source contributors.
6 February 2020
A safer inline-java
In this post about inline-java I aim to walk you through the upcoming safe interface of the library, which allows detecting memory management mistakes at compile time using linear types.
30 January 2020
Profiling Large Haskell Codebases
This post describes a profiling technique for Haskell codebases that yields faithful results and is well adapted to large repositories.
16 January 2020
A Tale of Two Functors
Haskell's Data and Control module hierarchies have always bugged me. Now, I understand that the intuition behind the Data/Control separation is rooted in a deep technical justification.
9 January 2020
Markov chain Monte Carlo Sampling (2)
In this second post of Tweag's four-part series, we discuss Gibbs sampling, an important MCMC-related algorithm which can be advantageous when sampling from multivariate distributions. Two different examples and, again, an interactive Python notebook illustrate use cases and the issue of heavily correlated samples.
19 December 2019
Art in browser with Asterius
Asterius compiles Haskell code into WebAssembly code to be executed in a browser or in Node.js. It has reached a new milestone by being able to compile the diagrams library and its dependencies.
28 November 2019
How to make your papers run
How to use the Makam metalanguage to implement an executable formal semantics for a simple language.
21 November 2019
Untrusted CI using Nix
Learn how Nix's new post-build hooks feature can automatically sign and upload artifacts to a binary cache in a trusted way.
8 November 2019
Probabilistic Programming with monad‑bayes (2)
Here's Part 2 in Tweag's Series about Bayesian modeling in Haskell with the monad-bayes library.
30 October 2019
Porcupine: Announcing First Release
We're happy to announce the first release of Porcupine, an open source framework to express portable and customizable data pipelines.
25 October 2019
Markov chain Monte Carlo Sampling (1)
In this first post of Tweag's four-part series on Markov chain Monte Carlo sampling algorithms, you will learn about why and when to use them and the theoretical underpinnings of this powerful class of sampling methods. We discuss the famous Metropolis-Hastings algorithm and give an intuition on the choice of its free parameters. Interactive Python notebooks invite you to play around with MCMC yourself and thus deepen your understanding of the Metropolis-Hastings algorithm.
11 October 2019
Ormolu: Announcing First Release
We're happy to announce the first release of Ormolu, a formatter for Haskell source code.
9 October 2019
Third-party Haskell libraries in Bazel
Bazel gets native support for third-party Haskell libraries and building Cabal packages since the 0.10 release of rules_haskell.
25 September 2019
Bazel's Persistent Worker Mode for GHC
I got the opportunity to work on Bazel's Persistent Worker Mode for GHC during my internship at Tweag. My goal was to improve the mode of communication between Bazel and the Haskell GHC compiler.
20 September 2019
Probabilistic Programming with monad‑bayes (1)
In this blog post series, we're going to lead you through Bayesian modeling in Haskell with the monad-bayes library. In the first part of the series, we introduce two fundamental concepts of `monad-bayes`: `sampling` and `scoring`.
12 September 2019
War Stories of Asterius
I got the opportunity to work on Asterius, a new Haskell to WebAssembly compiler, during my internship at Tweag. My task was to get anything numerics-related stabilized in its compiled code.
6 September 2019
Why Haskell is important
People keep asking me, “What's the point of Haskell, why do you folks even use it?”. How can you answer this?
1 August 2019
Code Line Patterns
We visualize large collections of Haskell and Python source codes as 2D maps using methods from Natural Language Processing (NLP) and dimensionality reduction and find a surprisingly rich structure for both languages. Clustering on the 2D maps allows us to identify common patterns in source code which give rise to these structures. Finally, we discuss this first analysis in the context of advanced machine learning-based tools performing automatic code refactoring and code completion.
17 July 2019
Revelations from repetition
Every day we write repetitive code. A lot of it is boilerplate that you write only to satisfy your compiler/interpreter. But how do languages differ in their boilerplate content? We explore these questions using data sets of Python and Haskell code.
27 June 2019
CPP considered harmful
Edsger Dijkstra took issue with the “unbridled” use of the programming construct in 1968. He noted that our already limited ability to reason about the dynamic behaviour of imperative programs was nigh impossible in the presence of -jumps to arbitrary labels. With , control flow…
27 May 2019
Ormolu: a new Haskell formatter
If we think of the average Haskeller, who do we see? A poor, broken person. Tired head falls into the open palms, sobbing, deprived of the basic human right for automated source code formatting. Is it at all conceivable that so many attempts were made and none quite succeeded…
9 May 2019
Tweag.io has a bit of a history with language interop. By this point, we created or collaborated with others in the community on HaskellR, inline-c, inline-java, and now inline-js. The original idea for this style of interop was realized in language-c-inline by Manuel Chakravarty…
10 April 2019
The Sneakernet: Towards A Much Faster Internet
Inspired by the Event Horizon Telescope images, we develop a quick exploratory study about future possibilities of this technology called the Sneakernet: Could massive data transfer give a new live to the homing pigeon industry? How about using transportation means that are optimized to carry incredible amounts of weight? Or transportation means that are designed to be fast as a bullet?
3 April 2019
Storing secrets in Terraform
I want to present you a Terraform plugin for securely managing secrets that was written for Digital Asset, who kindly allowed me to open source it. The general idea of this plugin is to protect secrets by making use of Terraform’s state. Terraform maintains the state of the world…
28 March 2019
Introducing lorri, your project's nix-env
Today we’re excited to announce lorri, a new tool for NixOS, Linux, and macOS that makes developing with Nix even nicer: When people try lorri, we often hear that it is more magical than they expected. What is lorri? lorri is a nix-shell replacement for project development…
20 March 2019
Capability is about free monads
There has been recent online discussions about impredicative encoding (aka final encoding) of free monads. Capabilities-as-type-classes, as embodied by the capability library, are essentially the same thing.
11 March 2019
Internships at Tweag
The time has come again to invite applications for Tweag I/O’s paid internship programme this summer. For the past few years Tweag has taken on a number of students interested in developing their experience working on projects of benefit to the wider programming community. This…
7 March 2019
Kubernetes, KubeNix and kind
In a recent project I was tasked with creating a local testing environment for a Kubernetes cluster the client was actively working on. The main requirements were: Cross platform: It should work on Linux and macOS. Ease of use: The setup should be easy to use. Suitable for CI…
28 February 2019
Declarative, Reproducible Jupyter Environments
Millions of Jupyter notebooks are spread over the internet - machine learning, astrophysics, biology, economy, you name it. What a great age for reproducible science! Or that's what you think until you try to actually run these notebooks. Then you realize that having understandable high-level code alone is not enough to reproduce something on a computer. JupyterWith is a solution to this problem.
20 February 2019
Have you ever needed to execute in another computer a binary that you have built, only to find that it doesn’t work because it depends on shared libraries that aren’t installed? Or even worse, you don’t have admin rights on the machine, so you cannot install the correct…
13 February 2019
The types got you
Haskell is in a way a unique language. It is, to be clear, a language people and businesses use for building “serious stuff”, yet it is also a language which remains a platform for experimentation with functional programming. This means that the teams which develop in Haskell…
6 February 2019
Mapping a Universe of Open Source Software
The repositories of distributions such as Debian and Nixpkgs are among the largest collections of open source (and some unfree) software. They are complex systems that connect and organize many interdependent packages. In this blog post I'll try to shed some light on them from the perspective of Nixpkgs, mostly with visualizations of its complete dependency graph.
23 January 2019
Harnessing the Power of Haskell in JupyterLab
Introduction Haskell and data science - on first sight a great match: native function composition, lazy evaluation, fast execution times, and lots of code checks. These sound like ingredients for scalable, production-ready data transformation pipelines. What is missing then? Why…
20 December 2018
TodoMVC in Haskell via WebAssembly
Getting your browser to print “Hello World!” is fun, and a milestone for compiler writers targeting the web. Soon expectations change. Can you call existing code? Can you write a moderately complex web app? How fast does it run? Previously, we showed how Haskell can call…
12 December 2018
DIY benchmark history
If you’re a conscientious developer like I am my boss is, you probably have a benchmark suite for the programs and libraries you develop. This allows you to see the impact of your changes on the performance of your applications. However, the temporal aspect of these benchmark…
4 October 2018
About a year ago, Michael Snoyman made a blog post about the ReaderT pattern. Ostensibly, it’s a blog post about preferring to encode state as -s when you can. But that’s not how we read it. What we saw first and foremost is a story about using extensional type classes describing…
12 September 2018
10 July 2018
Funflow Example: emulating Make
Funflow is a workflow management tool. It turns out that workflow management tools and build tools are closely related. So if you’re more familiar with the latter, this post might be of interest to you. We’ll be illustrating some of Funflow’s features by implementing Make on top…
21 June 2018
Streaming with linear types
In an earlier blog post, we discussed how streaming libraries help writing composable programs without lazy I/O. We showed, for a simple program, how using a streaming library helps reducing the amount of unaided bookkeeping that the programmer needs to do in order to make it…
29 May 2018
Compiling Haskell to WebAssembly
Asterius is an experimental WebAssembly backend for GHC. We aim to support most GHC features while making the emitted WebAssembly code as small and fast as possible. The project recently reached an exciting milestone: for the first time, simple Haskell functions can be compiled…
25 April 2018
Funflow is a system for building and running workflows. It’s a system we’ve been working on for the last few months with a small number of clients. In this blog post we’ll talk about what workflows are, why we built Funflow, and what we’d like to do with it in the future. What is…
17 April 2018
Stackage HEAD is now live!
We are happy to announce that Stackage HEAD is now functional. For those who had missed the original blog post, or already forgot it, let’s refresh the idea behind Stackage HEAD. Each Stackage Nightly is a build plan that includes a large set of real-world packages that are known…
29 March 2018
We're hiring! Software engineers
We’re hiring multiple software engineers to join our team. Our mission: deliver correct high-performance software faster. We have found functional programming in particular to be an enabling factor for us on some of the most ambitious projects we’ve faced for our clients…
15 March 2018
Nix + Bazel
This is the second post in our series about building polyglot projects using Bazel. You might find it useful to start with our first installment. A is an extraordinary thing. You specify a set of targets, you tell Make what the dependencies are between these targets, and Make…
8 March 2018
Implementing a safer sort with linear types
In the “all about reflection” post, we introduced the data type. Any sorting function should have the type . However, this doesn’t actually guarantee that is indeed a sorting function: it only says that is a sorted list, but not that it is actually a sort of . For example is…
28 February 2018
Bazel now supports building Haskell
Publishing code to the world is easy these days: take your code, tack some extra metadata onto it, call that a package and upload it to npmjs.com / crates.io / Hackage / etc. It’s also easy for other developers to include your code as part of their project: build tools are…
8 February 2018
Internships at Tweag
Tweag I/O is inviting applications from students for our paid internship programme this summer. If you’re a student excited about working with Haskell, Nix and similar things, then get in touch! In previous years, our interns have worked on a variety of projects across research…
5 February 2018
Free monads for cheap interpreters
The utility of free monads can show up in surprising places. One of the projects I’m working on is an AI, and part of the strategy that it uses for responding to user input is quite simple: it generates many possible responses, and then evaluates them. Most of the computations it…
21 December 2017
All about reflection
An important device in the tool belt I carry around everyday is type class reflection. I don’t reach for it often, but it can be very useful. Reflection is a little known device. And for some reason it is often spoken of with a hint of fear. In this post, I want to convince you…
29 November 2017
Making two garbage collectors be good neighbours
Foreign function interfaces (FFI) allow fast interop between languages. Unlike other approaches, like performing RPC calls between different components written in different languages, using the FFI allows for all manner of data to be shared between each language runtime, in the…
16 November 2017
Parallelising your array code
This is the fifth post in a series about array programming in Haskell — you might be interested in the first, second, third, and fourth, too. A recurring theme in array programming is performance. After all, many algorithms in numerical computing and data science are…
10 November 2017
Nix on the Windows Subsystem for Linux
Nix on Windows: does it run yet? That’s the question I wondered about while testing the latest NixOS release, version 17.09. To that end, I had the idea of running the Nix installation process from inside the Windows Subsystem for Linux (WSL) see if it worked. And it worked…
1 November 2017
The Exodus to Streamgard, an epic poem
If Haskell was a god, often would he be depicted with the ravens Modularity and Abstraction flying above him, hovering the world and reporting to him every detail of our whereabouts. Haskell would sit on the Throne of Purity and look upon the world with an eye full of wisdom. And…
27 October 2017
Using Stackage for GHC regression testing
A recent development in Haskell land is the formation of the GHC DevOps Group, which was the topic of last week’s blog post. The group is a community of parties committed to the future of GHC. Tweag I/O is one such party. We are helping the group achieve its goals with concrete…
19 October 2017
Announcing the GHC DevOps Group
As Haskell is increasingly used in production environments, the Haskell toolchain is of critical importance to a growing number of people and organisations. At the heart of this toolchain is GHC (The Glasgow Haskell Compiler). Conceived nearly thirty years ago as a research…
12 October 2017
Array fusion with vector
This is the fourth post in a series about array programming in Haskell — you might be interested in the first, second, and third, too. In the previous post of this series, we explored the basic, down-to-earth, index-based array interfaces that have their roots in the Haskell…
5 October 2017
How to compose streaming programs
In our first blog post about streaming, we discussed how streaming libraries strengthen types to catch more errors. However, when we strengthen types, we need to be careful to not hinder program composition. After all, precise type information can make it more onerous to convince…
27 September 2017
Immutability and unboxing in array programming
This is the third post in a series about array programming in Haskell — you might be interested in the first and second, too. In the previous post of this series, we discussed commonly used vector and matrix routines, which are available in highly-optimised implementations in…
22 September 2017
GHC compiler plugins in the wild: typing Java
Previously, we discussed how to use inline-java to call any Java function from Haskell. The reverse is also possible, though that will be a topic for a future post. In this post, we’ll peek underneath the hood to talk a little about how inline-java does its deed. You might find…
15 September 2017
Java from Haskell: a tutorial
Our introductory post for inline-java showed that we could call methods written in Java (or indeed any JVM function) from Haskell. Much in the style of other packages, it is moreover possible to do using java syntax, so examples from Java API documentation can be reused as-is. In…
6 September 2017
In this post I’ll show you how to see the performance of your software project improve over the entire lifetime of the project. That is, how to get something like the below picture. For that, you’ll need to systematically track and store benchmark results over time. We’ll use…
31 August 2017
Enter the matrix, Haskell style
This is the second post in a series about array programming in Haskell — you might be interested in the previous post, too. Matrices are the bread and butter of most scientific and numeric computing. It is not surprising then that there is a range of standard libraries and…
24 August 2017
Compact normal forms + linear types
We saw last time that with linear types, we could precisely capture the state of sockets in their types. In this post, I want to use the same idea of tracking states in types, but applied to a more unusual example from our paper: sending rich structured data types across the…
17 August 2017
Diversity through inline code
Haskell is an awesome language, but we need to remember that it is not very useful in isolation. In almost any realistic application, Haskell has to coexist with other languages, even if only to call existing C libraries or to make use of operating system services. In actual…
9 August 2017
Array programming in Haskell
Array programming in Haskell is attractive, but it is also somewhat confusing. Haskell’s functional syntax is close to the mathematical notation that is often used to develop and explain the algorithms on which array programs are based, which leads to great clarity of expression…
3 August 2017
Encode state transitions in types using linear types
At the time of our first post on linear types, we were fresh out of the design phase to extend GHC with linear types. We had a prototype implementation, but it was just a proof of concept: there was precious little you could do with it. A few months down the line, we are now in a…
1 August 2017
I am a functional programming evangelist
I have had a passion for programming ever since I wrote my first piece of code as a kid. I am not sure why. Maybe it is the absolute, god-like control over the behaviour of the machine, or losing myself in the details of the code. In any case, it is intriguing. When teenage play…
27 July 2017
Streaming programs without laziness
In school, we’re taught that I/O is a simple affair: read data in, write data out. Rinse, repeat. But then as practitioners we realize matters are often more complicated. For one, I/O is slow, so we probably want operations to overlap (i.e. be processed asynchronously…
10 July 2017
R and Haskell: best of both worlds with HaskellR
A guest post by Sigrid Keydana telling us the backstory behind the very cool trading app notebook she put together… Keras, ggplot2, Haskell and R all in one Jupyter notebook! Post originally appeared here. Earlier today, I presented at UseR! 2017 about HaskellR: a great piece of…
23 May 2017
Théophane is a summer intern here at Tweag I/O. Thanks also in part to the financial help from the Nix community, he’s spending six months on devising a type system for the Nix language. Nix, the purely functional package manager Nix is a cross platform tool for managing the…
18 May 2017
We're hiring! Technical evangelist
Our mission: deliver correct high-performance software faster. To do that, we recruited a team of experimentalists who use their formal training in a variety of theories and their experience to continuously improve our methodology and publish new tech. Theories that are…
13 March 2017
Linear types make performance more predictable
We’re extending GHC with linear types. Ever since Jean-Yves Girard discovered linear logic in 1986, researchers around the world have been going “wow! resource tracking, this must be useful for programming languages”. After all, any real computation on a real machine takes…
17 October 2016
A new ecosystem for Haskell: the JVM
20 June 2016
Haskell compute PaaS with sparkle
Maintaining a compute cluster for batch or stream data processing is hard work. Connecting it up to storage facilities and time sharing resources across multiple demands even more so. Fortunately cloud service providers these days typically upscale their offering to not just…
25 February 2016
Haskell meets large scale distributed analytics
Large scale distributed applications are complex: there are effects at scale that matter far more than when your application is basked in the warmth of a single machine. Messages between any two processes may or may not make it to their final destination. If reading from a memory…
9 December 2015
Stack + Nix = portable reproducible builds
Stack and Cabal are great tools for incremental development. But before even starting to think about what your first commit will be about on a freshly checked out project, what you have to do is get a working development environment, by which we mean: see if it builds! Stack…
8 September 2015
Programming R at native speed using Haskell
Most good data stories start with a interesting question. If the average request latency went down by a further 100ms, by how much could we expect user engagement to increase? How can we detect evidence of corruption of government officials given a list of all bids nationwide for…