I recently open sourced my first large PureScript project. It’s a slack bot that allows searching for cards from the Epic Card Game. In this post, I’ll discuss the process of writing the application, what went well and what went poorly.

An Overview of the Slack Bot

I was excited about building this project in PureScript because it was complicated enough to be interesting but not so complicated as to be overwhelming. The key requirements for the bot were:

Handle incoming HTTP requests and validate a signature
Scrape the Epic Card Game site for card names and images (no API exists)
Index the card names for in-memory full-text search
Parse JSON requests and generate JSON responses

Notably, one feature that was not required was a database.

Web Framework

I chose HTTPure for my web framework. The library has a simple design, good general documentation and good examples of using middleware.

I used middleware for two primary reasons: validating slack signatures and running my application monad. The middleware design worked nicely for both of these use cases.

Routing was very straightforward with only two paths. The root path handles all commands and the /interactive path handles interactive input from the user.

Here’s a look at the router:

module Epicbot.Web.Router
  ( new
  ) where

import Epicbot.App (ResponseM)
import Epicbot.Web.Service.Command as CommandService
import Epicbot.Web.Service.Interactive as InteractiveService
import HTTPure as HTTPure

new :: HTTPure.Request -> ResponseM
new req = case req of
  { path: [] } ->
    CommandService.handle req

  { path: ["interactive"] } ->
    InteractiveService.handle req

  _  ->
    HTTPure.notFound

Web Scraping

To retrieve details about Epic Card Game’s cards, I used a combination of the PureScript Milkis library to make HTTP requests and JavaScript’s cheerio to extract data from the HTML responses.

Making HTTP requests with Milkis was a simple one-liner. When the application is in offline mode (for testing) I simply read a fixture from my filesystem rather than making an HTTP request.

module Epicbot.Scraper
  ( scrape
  ) where

import Prelude

import Effect.Aff (Aff)
import Epicbot.Card (Card)
import Epicbot.Http as Http
import Epicbot.Html.Parser as Parser
import Epicbot.OnlineStatus (OnlineStatus(..))
import Milkis as Milkis
import Node.Encoding (Encoding(UTF8))
import Node.FS.Aff as FS

testDocPath :: String
testDocPath = "./data/card-gallery.html"

prodUrl :: Milkis.URL
prodUrl = Milkis.URL "http://www.epiccardgame.com/card-gallery/"

getPage :: OnlineStatus -> Aff String
getPage Offline = FS.readTextFile UTF8 testDocPath
getPage Online  = Milkis.text =<< Http.get prodUrl

scrape :: OnlineStatus -> Aff (Array Card)
scrape onlineStatus = Parser.parseCards <$> getPage onlineStatus

My HTML parsing code was only 30 or so lines of JavaScript and some FFI in PureScript.

Full-Text Search

I decided to use JavaScript’s elasticlunr for full-text search. This is the totality of the JavaScript code for building the index, adding documents and searching:

const elasticlunr = require("elasticlunr");

exports._addDoc = function (doc, index) {
  index.addDoc(doc);

  return index;
};

exports._newDocIndex = elasticlunr(function () {
  this.addField("name");
  this.setRef("id");
});

exports._searchDoc = function (term, index) {
  return index.search(term, {});
};

On the PureScript side, I’m mostly using FFI and wrapping the JavaScript in a more idiomatic interface.

JSON Handling

I used Argonaut to handle JSON parsing and generation. Here’s an example of some custom JSON parsing and generation I’m doing to interact with the Slack API:

newtype Action = Action
  { name  :: Maybe String
  , text  :: Maybe String
  , type  :: Maybe String
  , value :: Maybe String
  }

derive newtype instance eqAction :: Eq Action

derive newtype instance ordAction :: Ord Action

derive newtype instance showAction :: Show Action

instance encodeJsonAction :: EncodeJson Action where
  encodeJson :: Action -> Json
  encodeJson (Action obj) = do
    "value" :=? obj.value
    ~>? "type" :=? obj.type
    ~>? "text" :=? obj.name
    ~>? "name" :=? obj.name
    ~>? jsonEmptyObject

instance decodeJsonAction :: DecodeJson Action where
  decodeJson :: Json -> Either String Action
  decodeJson json = do
    obj <- decodeJson json
    name <- obj .:? "name"
    text <- obj .:? "text"
    t <- obj .:? "type"
    value <- obj .:? "value"

    pure $ Action { name, text, type: t, value }

Yes, it’s probably wrong to have a bunch of Maybe Strings in a record. I’m still learning.

Application Monad

I decided to follow the ReaderT design pattern when architecting my application. Here’s the definition of my App type:

newtype App a = App (ReaderT RequestEnv Aff a)

It’s a simple newtype over a ReaderT. The RequestEnv type represents the application configuration (e.g. the full-text search index) as well as request-specific configuration (e.g. the unique request id). The base monad is Aff, PureScript’s asynchronous effect monad.

The Good Stuff

Other than the fact that I was actively learning the PureScript language while building the bot, most things went remarkably well. I’m very happy with the final application, though I might build some parts differently were I starting today.

HTTPure is an excellent web framework. It’s both simple and powerful and its middleware implementation seems unrivaled in the PureScript ecosystem.

The ability to FFI into the JavaScript ecosystem is also a huge boon. Both elasticlunr and cheerio made quick work of what could have been very challenging problems. Even though I was relying on “unsafe” JS code for these portions of the application, once the FFI was in place and the JS code was written, I’ve never had a runtime issue with these seams. In a more robust production system, I may choose to use Foreign at my FFI boundaries, but avoiding that here worked fine in practice.

Spago, PureScript’s package manager and build tool, is an absolutely delight to use. It has spoiled me for other ecosystem’s package managers.

The editor tooling in PureScript is also fantastic thanks to the PureScript Language Server. Again, this makes it hard for me to go back to other languages.

Last, but certainly not least, the PureScript language itself is a pleasure to work with. When I began learning it, I had no experience with Haskell nor any other pure functional language. I’ve felt incredibly productive in PureScript once over the initial learning curve. It feels both light-weight and powerful. While the community is small, the people are fantastic and the available libraries are top notch. Much to my surprise, it even has excellent documentation.

The Less Good Stuff

Because I’m still new to pure functional languages, there are places where I can acutely feel the boilerplate. Here’s the minimum amount of code required to build a custom application monad:

newtype App a = App (ReaderT RequestEnv Aff a)

derive newtype instance functorApp :: Functor App

derive newtype instance applyApp :: Apply App

derive newtype instance applicativeApp :: Applicative App

derive newtype instance bindApp :: Bind App

derive newtype instance monadApp :: Monad App

derive newtype instance monadEffectApp :: MonadEffect App

derive newtype instance monadAffApp :: MonadAff App

instance monadAskApp :: TypeEquals e RequestEnv => MonadAsk e App where
  ask = App $ asks from

The Slack API itself is quite finicky, often times having different behavior if a JSON value is included with a null value or excluded completely. This led to lots of verbose JSON generation. This is more of a comment on the Slack API than any particular PureScript feature. It’s also entirely possible that I could structure this code differently to make it less verbose.

PureScript is a small language and ecosystem and this led to some DIY that I wasn’t expecting. For example, I couldn’t find an existing library for parsing an application/x-www-form-urlencoded body. I ended up writing one myself and learning parser combinators in the process, which was great, but this was a piece of work I wasn’t expecting. I also had to write my own implementation for shuffling an Array (note to reader: my implementation is horrible).

Overall Impression

This project has only increased my enthusiasm around PureScript. It was a joy to use and I’m happy with the resulting application. The language feels flexible, light-weight and very well designed.

I’m also watching the PureScript Native project excitedly. The ability to target the go ecosystem as an alternative backend would be fantastic.

An Overview of the Slack Bot#

Web Framework#

Web Scraping#

Full-Text Search#

JSON Handling#

Application Monad#

The Good Stuff#

The Less Good Stuff#

Overall Impression#