Host Bitbucket add-ons on Github

This weekend, I came across an interesting article regarding building a static Bitbucket add-on comprising just HTML, CSS and JS. Generally, add-ons for Atlassian products are built using the connect framework which is based on REST APIs and web hooks. However, it’s also possible to build some add-ons that do not have any server component.

From the README file, it looks like the author  hosted the example code on firebase. However, another way to host such add-ons is on Github. Github has had static file hosting support for some time. I used this approach for hosting my version of this example. The biggest convenience is that there is no separate deployment step required. Just pushing the code to Github updates the add-on!

 

A Tour of Go – Exercise Solutions

I spent the past weekend learning Go. A Tour of Go is a great resource to quickly come up to speed with Go. It’s a very small language (the spec is only around 50 pages!) and is really easy to pick up if you are already familiar with languages like C, JavaScript, Java or C#. I really liked the exercises that were scattered around in the tour. The ones that I found interesting were:

I’ve put up my solutions to all the exercises in a Github repository – https://github.com/parambirs/atourofgo-exercises. I’d be interested in seeing alternate solutions and discussing different approaches to solving these exercises in Go!

How to write a Docker based Haskell web service — from scratch

This is a hands-on guide to running a Haskell based web service on docker. No previous knowledge of haskell or docker is required.

The complete source for this tutorial is available on github: https://github.com/parambirs/hello-scotty/tree/with-json-types

Over the last couple of weeks, I spent some of my free time trying to run a basic Haskell+Docker web service. As it is with every new language or platform, I had to set up and understand a lot of things to get my app up and running. I’m sharing my experience here, hoping that it will help others who might be inclined to get into the Haskell world.

We’ll go through the following 4 steps:

  1. Set up the development environment
  2. Build and package a simple app using cabal
  3. Run a basic Haskell web app using Scotty
  4. Dockerize your scotty-webapp

As of this writing, I have less than 100 hours of Haskell development experience. Therefore, this is a pretty basic guide and focuses on the development tools rather than the language. It was possible for a newbie like me to build and run a basic Haskell web app using docker without much difficulty. Hopefully it’ll encourage more people to experiment with Haskell for their side projects!

1. Set up the development environment

You may skip this step if you already have ghc and cabal installed on your system.

Install the Haskell Platform (GHC)

GHC is the Glasgow Haskell Compiler package which provides you the compiler (ghc) as well as a REPL (ghci). Cabal is Haskell’s package manager and works similar to npm (NodeJS), sbt (Scala) and maven (Java).

% brew update
% brew install ghc cabal-install

Haskell REPL (ghci)

Next, we’ll start the Haskell REPL which is called ghci and get comfortable with a bit of Haskell syntax.

% ghci

Let’s define two simple functions that convert temperature values between celsius and fahrenheit:

Prelude> let f2c f = (f — 32) * 5 / 9
Prelude> let c2f c = (c * 9/5) + 32

Now, we can call these functions to get converted temperatures:

Prelude> f2c 800
426.6666666666667
Prelude> c2f 100
212.0
Prelude> c2f 37
98.6
Prelude> <Ctrl+D> — To exit

Run Haskell Scripts (runhaskell)

We’ll move these function definitions to a haskell source file and define a main method that prints the converted temperature value.

Copy the following code into a file named c2f.hs:

c2f c = (c * 9/5) + 32
main = do
  print (c2f 37)

Note: We used let for defining functions inside ghci. When writing Haskell source files, you don’t require let.

Run the script from the command line using runhaskell:

% runhaskell c2f.hs
98.6

Build an executable

runhaskell makes the development cycle easier when you are writing a Haskell application. But you can also create executable files using the ghc compiler. Let’s convert our script to a binary and then run it:

% ghc c2f.hs
[1 of 1] Compiling Main ( c2f.hs, c2f.o )
Linking c2f …
% ls
c2f c2f.hi c2f.hs c2f.o
% ./c2f
98.6

Load your source files into ghci

You can load external source files into ghci by using the :load or :l command. Let’s start a ghci session and load ourMain.hs file so that we can use the c2f function inside the REPL:

% ghci
GHCi, version 7.10.2: http://www.haskell.org/ghc/
Prelude> :load Main.hs
[1 of 1] Compiling Main ( Main.hs, interpreted )
Ok, modules loaded: Main.
*Main> c2f 37
98.6

Now that our development environment is set up, we’ll move on to building a simple app using cabal which is Haskell’s package manager.

2. Build and package a simple app using cabal

Introducing Cabal

Cabal (Common Architecture for Building Applications and Libraries) is a system for building and packaging Haskell libraries and programs. Think of it as Haskell’s equivalent of Python’s pip, Node’s npm and Scala’s sbt.

Before we get started, make sure that you can cabal installed and on the system path:

% cabal — version
cabal-install version 1.22.6.0
using version 1.22.4.0 of the Cabal library

Create a cabal app

Cabal makes it easy to build, package and test your application. It’s also easy to add dependencies on other libraries that cabal will fetch for you.

It is possible that different Haskell apps have dependencies on different versions of the same library. To prevent conflicts in your global Haskell environment, it’s a good idea to create a sandbox for your app. Cabal will then install the dependent libraries inside the sandbox.

% mkdir cabal-example
% cd cabal-example
% cabal sandbox init
Writing a default package environment file to
/Users/psingh/tmp/haskell/eac-articles/cabal
example/cabal.sandbox.config
Creating a new sandbox at
/Users/psingh/tmp/haskell/eac-articles/cabal-example/.cabal-sandbox

Now, we can initialise our app. The easiest way to create a cabal app is using the cabal init command. Follow the prompts and cabal will generate the config file for you. Here’s an example interaction on my system:

% cabal init
Package name? [default: cabal-example]
Package version? [default: 0.1.0.0]
Please choose a license:
* 1) (none)
2) GPL-2
3) GPL-3
4) LGPL-2.1
5) LGPL-3
6) AGPL-3
7) BSD2
8) BSD3
9) MIT
10) ISC
11) MPL-2.0
12) Apache-2.0
13) PublicDomain
14) AllRightsReserved
15) Other (specify)
Your choice? [default: (none)] 9
Author name? [default: Parambir Singh]
Maintainer email? [default: user@domain.com]
Project homepage URL?
Project synopsis?
Project category:
* 1) (none)
2) Codec
3) Concurrency
4) Control
5) Data
6) Database
7) Development
8) Distribution
9) Game
10) Graphics
11) Language
12) Math
13) Network
14) Sound
15) System
16) Testing
17) Text
18) Web
19) Other (specify)
Your choice? [default: (none)]
What does the package build:
1) Library
2) Executable
Your choice? 2
What is the main module of the executable:
* 1) Main.hs (does not yet exist)
2) Main.lhs (does not yet exist)
3) Other (specify)
Your choice? [default: Main.hs (does not yet exist)] 1
What base language is the package written in:
* 1) Haskell2010
2) Haskell98
3) Other (specify)
Your choice? [default: Haskell2010] 1
Include documentation on what each field means (y/n)? [default: n] n
Source directory:
* 1) (none)
2) src
3) Other (specify)
Your choice? [default: (none)] 1
Guessing dependencies…
Generating LICENSE…
Generating Setup.hs…
Generating cabal-example.cabal…
Warning: no synopsis given. You should edit the .cabal file and add 
one.
You may want to edit the .cabal file and add a Description field.

You will now have a cabal-example.cabal file in your source directory. Remember that we selected Main.hs as our main module during the cabal init process. Add a Main.hs file now to the root of your source folder with the following content:

c2f c = (c * 9/5) + 32
main = do
  print (c2f 37)

Running your cabal app (cabal run)

To run your app, use the cabal run command:

% cabal run
Package has never been configured. Configuring with default flags. If this fails, please run configure manually.
Resolving dependencies…
Configuring cabal-example-0.1.0.0…
Preprocessing executable ‘cabal-example’ for cabal-example-0.1.0.0…
[1 of 1] Compiling Main ( Main.hs, dist/build/cabal-example/cabal-example-tmp/Main.o )
Linking dist/build/cabal-example/cabal-example …
Running cabal-example…
98.6

Other useful cabal commands

cabal clean

Removes the generated artifacts (executables etc.)

% cabal clean
cleaning…

cabal build

Compiles and links your source code to generate the executable.

% cabal build
Package has never been configured. Configuring with default flags. If this fails, please run configure manually.
Resolving dependencies…
Configuring cabal-example-0.1.0.0…
Building cabal-example-0.1.0.0…
Preprocessing executable ‘cabal-example’ for cabal-example-0.1.0.0…
[1 of 1] Compiling Main ( Main.hs, dist/build/cabal-example/cabal-example-tmp/Main.o )
Linking dist/build/cabal-example/cabal-example …
Run the generated executable:
% ./dist/build/cabal-example/cabal-example
98.6

cabal install

Installs the generated artifact into the cabal sandbox. This is somewhat similar to mvn install

% cabal install
Resolving dependencies…
Notice: installing into a sandbox located at
/Users/psingh/tmp/haskell/eac-articles/cabal-example/.cabal-sandbox
Configuring cabal-example-0.1.0.0…
Building cabal-example-0.1.0.0…
Installed cabal-example-0.1.0.0

cabal repl

Starts a ghci session for your project (i.e. with your modules loaded). You can easily test your functions here.

% cabal repl
Preprocessing executable ‘cabal-example’ for cabal-example-0.1.0.0…
GHCi, version 7.10.2: http://www.haskell.org/ghc/ 😕 for help
[1 of 1] Compiling Main ( Main.hs, interpreted )
Ok, modules loaded: Main.
*Main> c2f 37 — call the c2f function defined in our Main.hs file
98.6
*Main> main — yes, this is the main function we defined in Main.hs
98.6
*Main>

cabal update

Downloads the latest package list for cabal. It’s useful to run cabal update before running cabal init or cabal install to make sure that cabal has knowledge about the latest packages

% cabal update
Downloading the latest package list from hackage.haskell.org

Source

The complete source for this section is available on github: https://github.com/parambirs/hello-cabal

Next…

We’ll write a simple web app using the Scotty web framework for Haskell.

3. Run a basic Haskell web app using Scotty

Introducing Scotty

There are many web frameworks available for Haskell: Yesod, Snap, Scotty, etc. I chose Scotty over others as it seemed easier to get started with.

We’ll write a simple web service that responds to various HTTP request types (GET, PUT, POST, DELETE). We’ll see how to get request headers, path parameters and form fields and how to respond with plain-text, html or JSON response.

Initialise Cabal

Let’s initialise a cabal app for our web service. I’ve mostly chosen the default options. Two notable exceptions include:

  • license: 9) MIT
  • source directory: 2) server
% mkdir scotty-webapp-example
% cd scotty-webapp-example
% cabal sandbox init
Writing a default package environment file to
/Users/psingh/tmp/haskell/eac-articles/scotty-webapp-example/cabal.sandbox.config
Using an existing sandbox located at
/Users/psingh/tmp/haskell/eac-articles/scotty-webapp-example/.cabal-sandbox
% cabal init
Package name? [default: scotty-webapp-example]
Package version? [default: 0.1.0.0]
...
...

Write the server code

Since we told cabal earlier that our main module for the executable will be Main.hs, and that it will live inside the server folder, let’s add server/Main.hs file to our source.

As you can probaby figure out, this code will start a server on port 3000 and:

  • For a GET request on the ‘/’ path, the server will respond with an HTML response with the content “This was a GET request!”
  • For a DELETE request on the ‘/’ path, the server will respond with a ‘plain-text’ response with the content “This was a DELETE request!”

Add dependencies

Let’s add a dependency for scotty and http-types libraries in our cabal file.

This is how the build-depends segment of the scotty-webapp-example.cabal files looks right now:

build-depends: base >=4.8 && <4.9

Change it to:

build-depends: base >=4.8 && <4.9
             , scotty
             , http-types

Next, run cabal install to add the dependencies into your sandbox followed by cabal run to run the server.

% cabal install
Resolving dependencies…
Notice: installing into a sandbox located at
/Users/psingh/tmp/haskell/eac-articles/scotty-webapp-example/.cabal-sandbox
Configuring ansi-terminal-0.6.2.3…
Configuring appar-0.1.4…
…
…
% cabal run
Package has never been configured. Configuring with default flags. If this fails, please run configure manually.
Resolving dependencies…
Configuring scotty-webapp-example-0.1.0.0…
Preprocessing executable ‘scotty-webapp-example’ for
scotty-webapp-example-0.1.0.0…
[1 of 1] Compiling Main ( server/Main.hs, dist/build/scotty-webapp-example/scotty-webapp-example-tmp/Main.o )
Linking dist/build/scotty-webapp-example/scotty-webapp-example …
Running scotty-webapp-example…
Setting phasers to stun… (port 3000) (ctrl-c to quit)

The server will be running now on port 3000. Let’s verify that:

% http delete :3000
HTTP/1.1 200 OK
Content-Type: text/html; charset=utf-8
Date: Mon, 14 Sep 2015 05:44:57 GMT
Server: Warp/3.1.3
Transfer-Encoding: chunked
This was a DELETE request!

Great!

Handling more complex requests

Let’s add a few more handlers to handle different kinds of requests. Add the following to server/Main.hs:

Encode/Decode JSON

Most web services these days interact via JSON. Haskell provides a type safe way to encode/decode JSON strings using theAeson library.

Defining the model

Let’s create an Article data type that could represent a news article for example. An article consists of 3 fields: anInteger id, a Text title and a Text bodyText. By making Article an instance of FromJSON andToJSON typeclasses, we can use Aeson library for converting between JSON strings and Article objects. Add the following code to the file server/Article.hs:

We’ll need to add a couple of routes to our Scotty router function to handle encoding and decoding Article types:

We’ll also need to add a couple of dependencies to scotty-webapp-example.cabal file:

build-depends: base >=4.8 && <4.9
             , scotty
             , http-types
             , text
             , aeson

Test JSON encoding/decoding

Let’s fire up Scotty and see if it can handle JSON properly:

GET /article

% http get :3000/article
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Date: Mon, 14 Sep 2015 06:51:17 GMT
Server: Warp/3.1.3
Transfer-Encoding: chunked
{
  “bodyText”: “content”,
  “id”: 13,
  “title”: “caption”
}

POST /article

% http post :3000/article id:=23 title=”new caption” bodyText=”some content”
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Date: Mon, 14 Sep 2015 06:56:57 GMT
Server: Warp/3.1.3
Transfer-Encoding: chunked
{
  “bodyText”: “some content”,
  “id”: 23,
  “title”: “new caption”
}

Source

The complete source for this section is available on github: https://github.com/parambirs/hello-scotty/tree/with-json-types

Next…

We know how to build & run a basic web service in Haskell that can talk JSON. In the next section we’ll dockerize it!

4. Dockerize your scotty-webapp

This section assumes that you have docker installed on your system. Please go to https://docs.docker.com/installation/ if you need to install Docker on your machine.

Write a Dockerfile

We’ll base our docker image on the official haskell image. The image has GHC and Cabal available so we’ll copy our source files and use cabal install to build our executable.

Start Docker

I’m assuming you have docker installed and set up correctly. On OSX, you need to run the following two commands to get docker up and running:

% boot2docker up
Waiting for VM and Docker daemon to start…
……………..ooooooooo
Started.
Writing /Users/psingh/.boot2docker/certs/boot2docker-vm/ca.pem
Writing /Users/psingh/.boot2docker/certs/boot2docker-vm/cert.pem
Writing /Users/psingh/.boot2docker/certs/boot2docker-vm/key.pem
To connect the Docker client to the Docker daemon, please set:
  export DOCKER_HOST=tcp://192.168.59.103:2376
  export DOCKER_CERT_PATH=/Users/psingh/.boot2docker/certs/boot2docker-vm
  export DOCKER_TLS_VERIFY=1
Or run: `eval “$(boot2docker shellinit)”`
% `eval “$(boot2docker shellinit)”`
Writing /Users/psingh/.boot2docker/certs/boot2docker-vm/ca.pem
Writing /Users/psingh/.boot2docker/certs/boot2docker-vm/cert.pem
Writing /Users/psingh/.boot2docker/certs/boot2docker-vm/key.pem

Build Docker Image

% docker build -t scotty-webapp-docker .
Sending build context to Docker daemon 136.6 MB
Step 0 : FROM haskell:7.10
…
…
Step 9 : CMD scotty-webapp-example
 — -> Running in 96b20daed3cc
 — -> cc46fe7ce920
Removing intermediate container 96b20daed3cc
Successfully built cc46fe7ce920

Run the Docker Image

% docker run -i -t -p 3000:3000 scotty-webapp-docker
Setting phasers to stun… (port 3000) (ctrl-c to quit)

So, our web app is running inside the docker container!

Let’s test it now. On OSX, Docker runs inside a virtual machine. So, accessing localhost:3000 won’t work. Use the following command instead:

% open http://$(boot2docker ip):3000

We can also test our web app using the httpie client:

% http $(boot2docker ip):3000
HTTP/1.1 200 OK
Content-Type: text/plain; charset=utf-8
Date: Mon, 14 Sep 2015 23:37:46 GMT
Server: Warp/3.1.3
Transfer-Encoding: chunked
This was a GET request!
% http $(boot2docker ip):3000/article
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Date: Mon, 14 Sep 2015 23:38:27 GMT
Server: Warp/3.1.3
Transfer-Encoding: chunked
{
  “bodyText”: “content”,
  “id”: 13,
  “title”: “caption”
}

Source

The complete source for the final application is available on github: https://github.com/parambirs/hello-scotty/tree/with-json-types

Finally

I hope you found this tutorial useful. Please let me know if something didn’t work for you or if I missed documenting any step.

Downloading sources for maven and sbt project dependencies

My experience with Intellij for downloading sources for the dependencies hasn’t been good. It frequently fails to download the sources. I’ve tried a few (googled) suggestions but they haven’t helped. However, there’s an alternative way using the command line. The downside to this is that it will download the sources for all dependencies. But, I don’t really mind that till I start running out of space on my laptop!

Maven

$ mvn dependency:sources

sbt

$ sbt updateClassifiers

Failing fast in bash scripts

If you want your bash script to stop execution as soon as any of the commands returns a non-zero exit status, invoke bash with the -e option.

For example, the following script will continue to run, even though the sub-shell exited with the status 1.

#!/bin/bash 
( exit 1 ) # subshell that returns non-zero exit status
echo "This will be echoed"

However, if the script invokes bash with the -e option, the script ends as soon as the subshell returns 1.

#!/bin/bash -e
( exit 1 ) # subshell that returns non-zero exit status
echo "This will not be echoed"

Note: Although I’ve used the subshell as an example here, but the script will stop executing for any command failure e.g. wget, curl, etc.

Another way to enable bash scripts to stop execution on failure is by using the set -e option. This can be set at any point in the script.

#!/bin/bash
( exit 1 ) # subshell that returns non-zero exit status
echo "This will be echoed"
set -e # enable fail fast mode
( exit 1 ) # script will not execute beyond this
echo "This will NOT be echoed"

Lastly, the fail fast mode can be disabled by using set +e in the script.

#!/bin/bash
( exit 1 )
echo "This will be echoed"
set -e # enable fail fast mode
echo "Just echoing something"
set +e # disable fail fast mode
( exit 1 )
echo "This will STILL be echoed"