Shell Programming in Haskell: Converting S5 Slides to PDF

Shell Programming in Haskell: Converting S5 Slides to PDF

Recently, I gave an introduction to Python for Chris’ and Kelly’s GIS Workshop. It was a really great experience, and we had a lot of fun learning about Python and how to use it with ArcGIS.

I did my slides for it in Markdown, using S5. Others around the Scholars’ Lab have used Show-off to compose slide-shows in Markdown, but I wanted something a little simpler, and it had been a while since I’d looked at S5, so I used that instead.

Then Kelly asked me for a PDF version of the slideshow. Heh.

At first I thought I might have to covert it to Showoff or (worse yet) PowerPoint. But I Googled around and found that converting it wouldn’t be too difficult. The process itself would be simple, and a small shell script would make it even easier.

And then my infallible instinct to make any project ten times more interesting (i.e., complicated) kicked in.

I remembered that I’d just read Greg Weber’s post about Shelly, a library to make shell scripting a bit easier in Haskell. I’ve been seriously playing with Haskell for almost a year now, using it for most of my side-projects and for anything that no one else will have to maintain. The thought of using Haskell for shell scripting was intriguing, just because it would be another way for me to wrap my head around this very different computer language.

But I was skeptical. At first glance, Haskell doesn’t seem like a good candidate for shell programming. Typically, these scripts are quick, one-off programs, often written in anger, that need to be created quickly and nimbly (dare I say, agily?). However, Haskell is statically-typed, and its type system is not given to making quick changes. (Well, I’ve found that not to be quite accurate, but it is the perception.) Generally, I think that languages like Haskell are more suited to larger systems, because their power and concision really only become apparent when working with large bodies of code.

Whatever my reaction, though, a small script like this, with limited scope, seemed perfect.

The Process

The process I found to handle the conversion was fairly simple.

  1. Get a PNG screenshot of each slide using webkit2png;
  2. Concatenate all of the PNGs into a PDF using the ImageMagick tool convert;
  3. Clean up the PNGs.

With that laid out, let’s jump in.

Preface

First, some book-keeping: I have to let Haskell know that I’m going to use string literals in places that require Data.Text.Text instances:

{-# LANGUAGE OverloadedStrings #-}

Also, we have to import the Shelly module.

import           Shelly

And we need some other modules for working with characters, text, and other things.

import           Control.Monad (forM_)
import qualified Data.Char as C
import qualified Data.Text.Lazy as T
import           Filesystem.Path
import           Prelude hiding (FilePath)
import           System.Environment

Converting to PNGs

The first step is taking screenshots of each slide. To do that, I used the webkit2png script.

For most things, I’m using Python 2.7, but I haven’t bothered installing pyobjc for it. webkit2png uses pyobjc, though, so I have to run that program with Python 2.6, which is the default Python shipped with Mac OS 10.6. I only generate the full-sized screenshot, and I output it to a filename that includes the slide number. In Bash, that would look like this:

python2.6 $(which webkit2png) 
        --fullsize 
        --filename pythongis-000 

http://people.virginia.edu/~err8n/pythongis/#slide0

First, let’s create a generic function to run commands in Python 2.6. In Shelly, the convention is to add an underscore to functions that throw away their output:

python26_ script args = run_ "python2.6" (script:args)

This is kind of interesting because I wouldn’t abstract this out if I were writing this in Bash, Python, or Ruby. But adding this function felt quite natural in Haskell, which tends to encourage smaller, more generic, yet more focused, functions.

Now I’ll build on that to create a command to look for the program webkit2png, and if it finds it, pass it to Python 2.6:

webkit2png_ filename url = do
    script <- which "webkit2png"
    case script of
        Nothing      -> echo "ERROR: webkit2png not installed."
        Just script' -> do
            s <- toTextWarn script'
            python26_ s [ "--fullsize"
                        , "--filename", filename
                        , url
                        ]

This could be better. For one thing, this command could print an error message if webkit2png isn’t available. If that happens, it should probably also short-circuit the rest of the script. The way to do this in Haskell would be to return a Maybe value, which is what the which function above does. In this case, I know that the program is installed and on the PATH, so I’m being a little sloppy.

Converting to PDF

The next step is to concatenate all the PNGs into one PDF. I’m using the convert program from ImageMagick to do this. This takes a list of PNG files to convert, the name of the PDF file, and generates the output.

convert :: FilePath -> [FilePath] -> ShIO ()
convert pdf pngs = run_ "convert" =<< mapM toTextWarn (pngs ++ [pdf])

Working on Multiple Files

Right now, webkit2png_ (the function to download the slides as PNGs) operates on a single slide. But we’ll need to do this for every slide in the show. downloadSlides takes the number of slides and the base URL, and it calls webkit2png_ for each slide. It returns a list of file names for the downloaded PNGs.

downloadSlides :: Int -> String -> ShIO [FilePath]
downloadSlides slideCount baseUrl = do
    forM_ inputs $ (url, file) -> webkit2png_ file url
    return files'
    where
        baseUrl' = T.pack $ baseUrl ++ "#slide"
        range    = map (T.pack . show) [0..slideCount]
        urls     = map (T.append baseUrl') range
        files    = map (T.append "slide-") range
        files'   = map (fromText . flip T.append "-full.png") files
        inputs   = zip urls files

The only wrinkle here is that the file names that are passed to webkit2png aren’t the ones that are output. Instead, the program appends the size of the image (thumbnail, full, etc.) and the .png extension. Since I want to operate on those files later, I have to create both the file name prefix to pass to webkit2png and the full file name to process later. This is unfortunate and brittle, because if webkit2png ever changes how it names the output files, my script will break.

This is also shell-script sloppy in another way. I should really create a temporary directory and download the PNGs there. Maybe someday.

Putting it all Together and Getting the Inputs

All the pieces are in place. The only things left are to parse the command-line arguments, call downloadSlides and convert, and delete the downloaded PNGs.

The main function is the entry-point for the script. It picks three parameters from the command line and tries to make one a Int. If that can’t happen for any reason, it prints the usage message and exits. If the command-line is right, the script continues processing.

main :: IO ()
main = shelly $ verbosely $ do
    args <- liftIO $ getArgs
    case args of
        [slides, url, pdf] | all C.isNumber slides -> do
            pngs <- downloadSlides (read slides) url
            convert (fromText $ T.pack pdf) pngs
            echo . T.pack $ "Wrote PDF to " ++ pdf
            mapM_ rm_f pngs
        otherwise -> echo usage

This is the usage/help message.

usage :: T.Text
usage = "
    usage: s5topdf.lhs [slides] [url] [output] n
     n
      slides is the number of slides in the slideshow.n
      url    is the URL to access the slideshow at.n
      output is the filename of the PDF file to create.n"

Running

To run this script, pass it to runhaskell with the right command-line arguments. For example, here’s a small wrapper script.

Conclusion

Using Haskell for shell programming hasn’t been bad, but it’s not as fast as shell programming usually is, either. This is still more verbose than the bash, Python, or Ruby versions would be, and it took me (a little) longer to write. (Of course, I was unfamiliar with several of these libraries, and that slowed me down.)

However, I needed to do almost no debugging. Once I got the types to line up and runghc to stop complaining, it just worked. There were no bugs hiding in parts that hadn’t run yet. Based on experience with other languages, I’d expected to have to tweak the convert function (the second stage of processing) once I got the webkit2png part working (the first stage). But that wasn’t necessary. After I coaxed the complete script into printing the usage message, everything else worked flawlessly.

The bottom line: For very short one-off scripts, this seems like over-kill. For scripts that you expect to grow, Haskell plus Shelly might be more attractive.

Second Conclusion

One of the things that attracts me to Haskell is it’s history of using literate programming. In fact, I’m using it right now. This post was generated from the script itself. I’ve posted the raw version to a gist, so you can compare them.

Using literate Haskell was a success. I really liked being able to interleave extended commentary with the code and to have both be part of the final product. I think it changed the nature of both the script and the post. This might not work as well for larger projects with more lines of code and multiple modules, but for a small script, it was very comfortable. I can see doing this again for descriptions of small algorithms, projects, or demos.

Also, having this file double as a script and the post is kind of neat, at least for the moment.

-- vim: set filetype=lhaskell:

My interests include text processing, text mining, and natural language processing, as well as web-development and general programming. Studied medieval English literature and linguistics at UGA. Dissertated on lexicography. Now I program in Haskell and write when I'm not building cool stuff for the Scholars' Lab. Also, husband and parent. Do you notice that sleep…

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Archives