New Computer Architecture/Compilers Conference Map

The Computer Architecture/Compilers Conference Map is an indispensable tool in designing a research agenda that has high impact (on your/your adviser’s travel budget). It’s a Google Maps mash-up that plots upcoming conference deadlines on a world map, so you can decide where you’d like to travel and then go about doing some research to get you there. Ah, the scientific method at work!

Update 18 May 2013: Due to moving on from UW I’ve moved the Architecture/Compilers Conference Map yet again, to a permanent, institution-agnostic home at http://archconfmap.com/. It’s previous location(s) should transparently redirect to its new home as well.  I’ve also ported the code from PHP to Java running on Google App Engine.  Source code is available on github.

Update 24 September, 2012: I’ve moved the Architecture/Compilers Conference Map to a new location, http://homes.cs.washington.edu/~devietti/archmap, to be compatible with home page upgrades here at UW. I’ve updated the links in this post, too.

Previous editions of this site used a manually-maintained database of conference information but this was growing tedious so I decided to help out the site creators (Karin Strauss and Luis Ceze) by automating this process. The general idea was to scrape WikiCFP for calls for papers of various conferences of interest, parse that information and then feed it into the mash-up. Pretty standard Web 2.0/SOA/AJAX/circa-2004 stuff, but this is 2011 so it’s even easier now!

Indeed, there were only 2 slightly tricky parts. The first was that WikiCFP doesn’t run a proper web service for obtaining conference information, so this has to be extracted in a somewhat precarious manner from their HTML pages. The second issue is that the same-origin policy prevents this work from happening entirely on the client-side. I would have preferred a solution involving only client-side javascript to keep the design simple and to make it easier for others to see all of the code. But this was not to be: a client script served up by http://www.cs.washington.edu cannot, try as it might, make requests to http://www.wikicfp.com. After some wheel-spinning, I decided to take the easy way out and do the cross-domain stuff on the server, via PHP.

When you hit the page, the server script sends a search query to wikicfp.com and parses the resulting HTML webpage (using the fantastic simplehtmldom library). This gives a list of upcoming conferences with their deadlines and locations. The server script packages all of this information into JSON format and sends it in a script to the client. The client then uses Google’s Geocoding Javascript API to figure out where the conference is and add a marker to the mash-up.

Unfortunately, the search results from wikicfp.com do not contain links to the actual conference websites, only to the WikiCFP call for papers pages. It’s nice to have direct links to the conference sites, but doing so would involve another HTML request for each conference displayed, and each request was taking nearly a second to complete (the initial search request also takes about a second). The simplest thing to code up is to perform these requests serially, but with 8 or more conferences to display on the page at once, that’s several seconds of valuable researcher time spent waiting to see where the international conferences are, instead of doing the research necessary to attend these conferences. I shudder to think at the lost productivity this would entail.

So, these requests must be done in parallel. The solution I chose was to send the page to the client with links only to the WikiCFP pages. Then, Javascript on the client makes asynchronous requests to the server to fetch the direct conference links, and the client script sneakily rewrites the links while the user is looking at the page. To resolve the direct conference links, the initial page includes the WikiCFP eventId for each conference.  There’s a “web service” (if 40 lines of PHP can qualify as such) on the server that, given such an eventid, fetches the appropriate call for papers page from wikicfp.com, extracts the link to the main conference site, and then returns that link bundled up in JSON format.

Some unscientific benchmarking on my home machine reveals that the initial page load takes about a second, and fetching each direct conference link takes 700-900ms, but the latency of these additional requests is overlapped. So, if you load the page and immediately scroll down and click on one of the conference links at the bottom (or in the map pop-ups) you’ll go to wikicfp.com. But if you can wait a second (literally) you’ll get the direct conference link. It is humanly possible to trigger this behavior (it took me about 10 tries), but not by any reasonable usage pattern. However, others looking to re-mash this mash-up be warned!

In the future I’d like to get the initial page load time down as well.  Since I’ve already used the parallelism silver bullet, I suspect it’s time for caching to make an appearance…

Advertisements

2 thoughts on “New Computer Architecture/Compilers Conference Map

  1. Great work! I’ve seen your page running and I’m wondering whether I can use your code by myself with all acknowledgements obviously. Or better, maybe you can allow to parameterize your service and thus I use my own list from wikicfp.

    • Hi Wendell,

      Thanks for your interest. I have made all the code available: there’s a link to the tarball beneath the map display. I don’t care about attribution, but the code is AGPL’d so if you make modifications then please redistribute them.

      I’d prefer you run your own service, as I may in the future modify the interface to this service in various incompatible ways, and I’m sure you’d like the freedom to do so as well!

      joe

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s