AWS Lambda Experiments

I recently discovered AWS Lambda – a way to run a small-ish piece of code in the cloud without having to provision any infrastructure. You give AWS your code (e.g., a Java class that implements the required Lambda interface) and they run it for you. You can run the function based on a periodic schedule, in response to various AWS events, or from a web page (via the Lambda JavaScript API).

I realized that some of the little web services I’ve built on Google App Engine (such as the UPenn Calendar Fixer-Upper, and the Architecture & Compilers Conference Map) could actually run nicely via AWS Lambda instead. The data for these web services changes only very rarely, but with my GAE setup I was recomputing everything from scratch on each request. For example, with the Arch Conference Map, every request for the page triggered several requests to wikicfp.com to scrape CFPs. But new CFPs are posted only a couple of times per year, so it’s a bit wasteful to recompute everything from scratch on each request. The Arch Conference Map page was also slow to load as a result, which was always annoying to me. GAE has various request caching features I could use, but this would add more complexity.

Instead, I’ve ported over the Arch Conference Map to AWS Lambda. I wrote a little Java function that scrapes CFPs from wikicfp.com, extracts their information, and builds a static HTML page with these results. The static HTML page then gets uploaded to Amazon S3, which can be configured to serve static web content. As a bonus, it’s easier for me to debug issues with the site because I can run the Lambda function on my local machine under a Java debugger to see how it’s behaving. My “web app” isn’t part of some big framework with lots of abstractions, it’s really just a plain Java function.

Serving static content from S3 is also pretty fast – in my informal testing the page loads several times faster now. Plus, the Lambda function only runs once per day, instead of on each request. If I could measure the carbon footprint of the app, I imagine it’d be a lot lower now.

I ported the Penn Calendar Fixer-Upper over to AWS Lambda as well, to generate a slightly nicer iCal feed than what the university provides. There are iCal feeds for the CIS and ESE departments now.

The only hiccup I’ve run into with Lambda is the limit on code size. The jar file I upload has to be under 50MB, which is actually something of a limitation since the jar needs to include all library dependencies outside of the JDK. While my code is only a couple of hundred lines, adding in the AWS Java SDK and some Apache code it’s quite easy to exceed 50MB. I had to use some maven magic to include only the imports I actually use. Even with that, my simple programs end up generating ~20MB jars.

My apps were operating entirely for free on GAE. In contrast, using AWS costs some money. Lambda is free for my low-rate usage, but there is a nominal cost (a few cents per month) for storing and serving the S3 content. However, for me the ease of working with and debugging Lambda functions makes it worthwhile.

 

 

schedule Gmail drafts for sending with GmailSendLater

I have often found myself writing emails late at night but, wanting to keep up appearances of propriety, preferring to have those emails sent at more respectable hours. I looked around on the internet and found some tools that would do this for Gmail. There are Chrome extensions with fancy UIs, like Boomerang and rightinbox, but they cost money and only seem to work in desktop browsers. I also came across GmailDelaySend, which uses the recent Google Apps Script API to accomplish much of the same functionality for free. I poked around GmailDelaySend’s code and decided that I’d prefer an alternative interface, where the time at which an email should be sent is placed in a custom label instead of in the message itself. Hence, my slavish GmailSendLater was born.

GmailSendLater is a Google Apps Script web app, which means that it installs some code that runs in the cloud and you can interact with it via a website. GmailSendLater’s web interface is here: if you haven’t installed the script this link will prompt you to install it; if you have installed it then you can use the web interface to view debug logs or manually trigger the script to run. There’s more information on the Github wiki.

Sending a draft email with GmailSendLater is a 3-step process:

1. You label the draft with a label like “send at 2pm” and then take a nice afternoon nap.
2. GmailSendLater runs (every 15 minutes by default), and replaces your label with a new label like GmailSendLater/sending_at Thursday, February 13, 2014 2:00:00 PM GMT-0900. This allows you to verify that GmailSendLater understands when to send the draft. If you remove this GmailSendLater/sending_at … label, the draft will not be sent.
3. GmailSendLater sends any drafts that should be sent at the current time.

Timezones

Figuring out the correct timezone in which to interpret sending times is a bit tricky. GmailSendLater uses the timezone of the draft email, as opposed to the timezone in which the script runs. Some informal testing has confirmed that, at least for me, the GmailSendLater script was running in US Central Time even though my email messages are created in my time zone, US Eastern Time. It also seems possible to get the time zone associated with the user’s Gmail account, which is potentially different still. I haven’t found a need for this third option but perhaps it will arise someday. The SugarJS library that GmailSendLater uses for parsing dates has a limited understanding of timezones, and one can use clunky dates like “30 August 14:30+13:00” to specify timezones by their UTC offset.

Google Apps Script isn’t quite enough

In an initial version of GmailSendLater, I used the Google Apps Script API to send email drafts. This mostly worked except that the GAS API doesn’t have a way to directly send a draft email, so (inspired by GmailDelaySend) I was (ab)using functionality for sending new messages and replying to/forwarding existing email threads. A copy of the draft would be created with the same content, and the copy would be sent.

This copy-and-send approach broke message threading. Most annoyingly, the original draft email could be moved to the Trash but not formally discarded. While the desktop browser version of Gmail is happy to suppress the presence of trashed draft emails within a thread, the Android app does not do this. So, when an email was sent, it would appear twice within a thread: once as a draft and then as a sent email. Ugh. These trashed drafts do not appear in the Drafts listing on the Android Gmail app, so finding them and discarding them is tedious.

Fortunately, I came across a StackOverflow post that showed how to use the Gmail API from Google Apps Script. This allows for first-class manipulation of draft emails, allowing them to be sent directly. This solution, which GmailSendLater adopts, appears to offer the same experience as pushing the send button in Gmail directly, preserving threading and avoiding the pollution of trashed draft messages.

Try it out

GmailSendLater is free and open-source. I’ve been using it for a few months now and have found it very helpful. Bug reports and feature requests are welcome!

Google Calendar Timezone Switcher

Scheduling meetings with folks in remote timezones is always a fraught process for me. There are just so many opportunities for me to have an off-by-one (or two, or three) error. Google Calendar helps a lot by showing an additional time zone on the calendar, but I am typically flipping back and forth between a handful of remote timezones. Changing the remote timezone displayed in Google Calendar involves a lot of friction: a roundtrip to the Settings menu. So typically I don’t bother changing the timezone, instead opting to recalculate time zones in my head which is kinda back to square one.

I figured there had to be a faster way, so I decided to celebrate Spring Break by hacking up a little UI modification that allows for faster switching among a handful of timezones. Leveraging moment.js for its wonderful timezone support and a bit of magic to get a dropdown box to appear, I put together a simple userscript/content script, available on github.

Here’s a screenshot of the original Google Calendar setup:

With the script installed, the sea label is now a dropdown box:
Here’s what the dropdown box looks like when open:

You can switch between timezones easily: as soon as a new timezone is selected, the times listed in the first column of the calendar are recomputed. moment.js does the actual time computation, taking daylight savings and lots of other complications into account.

To edit the timezone abbreviations displayed in the dropdown box (and the actual timezones they correspond to) you have to edit the script, which is a bit tedious. Details of this process are available on github. Caveat: I’ve only tested the code on Chrome 33.0 on Mac so far.

Alternative Heading Numbering in Word

Word offers many options for numbered headings, and working on a recent grant proposal forced me to delve a little more deeply into Word’s heading numbering mechanisms. My experience is working with Word 2011 for Mac, but the online documentation I’ve found for Word 2010 (for Windows) has described very similar behavior.

The format that I wanted was as follows, a pretty standard outline for an NSF grant:

Hack your Word documents with VBA

Tom Bergan, one of my grad school buddies, had the bright idea of writing a little lint script for our LaTeX papers. The script would use grep to find various inconsistencies in our papers, e.g., times when we said “non-determinism” instead of “nondeterminism”, didn’t follow “e.g.” with a comma, and other small, domain-specific semantic errors. It worked great and kept our writing more consistent than we ever could have done by hand.

Figures with captions in Microsoft Word

Figures are an important part of writing a technical document. Figures with captions are even better, because then you can refer to them from the text! Since I decided to write my dissertation in Word, I had to figure out how to do captioned figures. Word offers many ways to do this, and the most obvious ways don’t really work that well! Fortunately, I found a way that does work – using a somewhat-hidden feature called “frames.” For the record, my experience is with Word 2011 (on Mac).

Writing a Dissertation in Microsoft Word

A little while back I decided to write my dissertation in Microsoft Word instead of the venerable LaTeX. I’d used LaTeX exclusively for papers and was looking to try something new. I’d seen friends use Word for technical papers, and plenty of people on the Internet were using Word for their dissertations, so why not me?

UPenn Calendar Fixer-upper

Penn has a neat university-wide event calendar which is useful for keeping track of events in my own and other departments.  Even better, the Penn calendar has a fancy export web service that lets you export filtered information (such as events for a specific department, or for a specific date range) to a variety of different formats (such as XML or iCal).  I like the iCal format because I can import the iCal URL into Google Calendar and everything shows up in one place automatically. For reference, the (underdocumented) magic incantation to get an iCal feed of Electrical & Systems Engineering events is http://www.upenn.edu/calendar-export/?school=4&owner=115&showndays=200&type=ical2.

Running a shell script with a timeout

In my line of work, deadlock is always close at hand. Running multithreaded programs through prototype compilers, runtime systems, or simulators (and sometimes all three!), there is always the chance that a bug will trigger a deadlock in some layer of the system. There’s nothing like setting off a bunch of experiments, going to sleep, and waking up the next morning to find that the first experiment (or, inevitably, the experiment that ran right after I stopped checking on things) deadlocked and prevented everything else from running. After being burned by this particular variant of Murphy’s Law countless times, I learned to always run benchmarks with a hard timeout. Knowing that my experiments won’t hang forever is honestly right up there with fancy foam mattresses in terms of helping me sleep at night.

New Computer Architecture/Compilers Conference Map

The Computer Architecture/Compilers Conference Map is an indispensable tool in designing a research agenda that has high impact (on your/your adviser’s travel budget). It’s a Google Maps mash-up that plots upcoming conference deadlines on a world map, so you can decide where you’d like to travel and then go about doing some research to get you there. Ah, the scientific method at work!