Linux is for Lovers

installing ubuntu karmic server on bochs

Posted by: Joe Devietti on: February 6, 2010

I recently decided to try using the x86 emulator bochs for some of my architecture research, as bochs seems to have a well-structured code base highly amenable to hacking.  I also considered using qemu, but qemu’s design was ultimately not a great fit for the do-1-insn-at-a-time model of an architecture simulator.  Instead, qemu is designed to run a bunch of instructions (really fast) instead of stopping precisely after each one.  For the record, I was using bochs from CVS as of 2 Feb 2010 (bochs 2.4.2 was the latest release at the time) and qemu 0.12.2.

Next came the supposedly easy part: create a disk image with Linux on it (I wanted to use Ubuntu Karmic Server) and start simulating.

It turns out that running the Ubuntu installer in bochs is a real pain.  Not only is it slow, but I encountered an assertion failure (in bochs) during the disk partition step of the installation.  It turned out that this was due to some funkiness in the bochs plugin system (which is based on dynamically-linked libraries); switching to static linking finally got me past the assertion failure.  After that adventure I had some kind of boot loader corruption that caused the virtual machine to fail to boot.  Don’t even ask about getting networking to work (it still doesn’t), or why the terminal occasionally looks like the picture below.

graphical corruption of the terminal in bochs

graphical corruption of the terminal in bochs.

An alternative approach is to use debootstrap to setup a chroot Linux installation, and then copy that installation onto the virtual hard disk, as outlined here.  However, this is a pretty involved procedure, debootstrap with Ubuntu didn’t install any kernels for me, and installing the boot loader also proved tricky.

Contrast these circuitous approaches with qemu, which is very fast (even without using KVM acceleration) and installed Ubuntu without a hitch on the first try.  With networking.  And a nice virtual monitor via VNC.  All straight out of the box.  qemu+kvm might even be fast enough to compile code inside the VM, which would ensure that the right headers, libraries, etc are used (otherwise you can always just mount the virtual drive as a regular filesystem and use cp from the host machine).  qemu is everything I could ask for — if only its codebase was more in tune with what I wanted to do!

The Best of Both Worlds

It took me a while to realize that I could have my bochs and qemu it, too, as both emulators can read and write a common virtual disk format: it’s called “raw” for qemu and “flat” for bochs.  Essentially, it just uses a single regular file as the virtual disk.  The only trick is that you have to create the virtual disk using bochs’ bximage program; if it’s created with qemu’s corresponding qemu-img program then bochs won’t recognize the virtual disk for some reason.

With this hybrid approach, I can boot the disk using qemu for installing the OS, interactively setting things up via the commandline, downloading packages from the internet and so on, and then reboot into bochs to run simulations.  It gives me the excellent speed and hardware emulation of qemu, with the more-natural-for-simulation interface of bochs.

Tags: , ,

thunderbird to (hosted) gmail

Posted by: Joe Devietti on: November 10, 2009

My school (UW) recently allowed students to switch their school email accounts over to one of 2 providers: Google (via gmail) or Microsoft (via Outlook Web Access).  Feeling the inexorable pull of the cloud, I decided to take the plunge and switch to gmail, but one thing held me back.  I have used Thunderbird for a few years now and have accumulated roughly 1GB of archived messages that are generally unimportant but occasionally vital.  I’ve also espoused Lifehacker’s Trusted Trio email organization system and so many of the emails in my archive are tagged for easier organization.

I’d like to have this email archive searchable from gmail.  And I would really like to have all those tags I’ve accumulated over the years show up as gmail “labels”.  Fortunately, this isn’t too difficult thanks to the following: Read the rest of this entry »

Tags: , ,

powerpoint+flash interaction

Posted by: Joe Devietti on: April 17, 2009

One of the cool things I learned about Powerpoint the other day is that it’s possible to create interactions between an embedded Flash movie and the presentation itself.  This allows you to coordinate animations across the movie and the slide, e.g. to have a mouse click sometimes advance the slide and sometimes trigger some action in the movie.

The reason all this works is because you can do function calls up from the Flash movie, through the ActiveX container and into VBA code attached to the Powerpoint presentation.   You can also call down from Powerpoint into Flash in a similar manner, though I haven’t experimented with this.

The Easy Part

Perhaps surprisingly, writing the code is the easy part.  The Visual Basic code attached to the Powerpoint slide looks like this:


Private Sub ShockwaveFlash1_FlashCall(ByVal request As String)
  'request looks like:
  '<invoke name="foo" returntype="xml"><arguments></arguments></invoke>;
  If InStr(request, "gotoNextSlide") > 0 Then
    ActivePresentation.SlideShowWindow.View.Next
  ElseIf InStr(request, "gotoPrevSlide") > 0 Then
    ActivePresentation.SlideShowWindow.View.Previous
  End If
End Sub

ShockwaveFlash1 is the name of the ActiveX control for the Flash movie. If you only have one of these on a slide, this will be its name. But if you have more than one, the names will be ShockwaveFlash2 and so on.

In ActionScript land, the code that calls out of the Flash movie looks like this:


import flash.external.ExternalInterface;

if ( ExternalInterface.available ) {
   ExternalInterface.call( "gotoNextSlide" );
}

The Hard Part

Unfortunately, this code won’t do anything out of the box.

This is because of the Flash player security model, with which I became overly familiar over the course of this project. There are several different security sandboxes in which a Flash movie can potentially be running.  In order for ExternalInterface function calls to work, the movie needs to be, sensibly, in the most trusted sandbox: the LOCAL_TRUSTED sandbox. Fortunately, a movie can check what sandbox it’s running in with the following code:

import flash.system.Security;
trace( Security.sandboxType );

If you run a Flash movie in the standalone Flash player, it runs with LOCAL_TRUSTED privileges. However, if you run that same movie via an ActiveX control in Powerpoint, the default security settings put it in the LOCAL_WITH_NETWORK sandbox, which is insufficiently privileged to allow Powerpoint-Flash interaction to work. Instead, things will fail silently.  And no amount of tinkering with Powerpoint’s security settings (trusting ActiveX content, macros, etc.) will fix things. After much anguish, I discovered that you have to talk directly to the Flash runtime to get a movie into the LOCAL_TRUSTED sandbox when running inside an ActiveX control.

There are several ways of telling the Flash runtime to put a particular movie in the LOCAL_TRUSTED sandbox, all of them documented at the Adobe link above. Probably the easiest way is via the Flash Security Settings Manager gui: it’s a Flash applet that runs in your browser to control your local Flash settings – spooky!  You can decide to trust particular .swf files, or directories, or even the Powerpoint executable itself which will automatically trust all movies that are included in Powerpoint slides.  I’ve blogged previously about how to do this, but there are a few tricky parts I haven’t covered before.

If you’re embedding Flash movies into Powerpoint (as opposed to just linking to the movies), then you have to trust the Powerpoint binary, as the Flash movies don’t have a real filesystem location – if you check where an embedded movie lives via root.loaderInfo.url you will get the path to the Powerpoint binary. Trusting the Powerpoint binary is a hack that allows the movie, indirectly, to be trusted as well.  Trusting Powerpoint will also enable non-embedded Flash movies to run in the LOCAL_TRUSTED sandbox.

If your Powerpoint slides just link to the Flash movies, then you can have finer-grained control over security, but of course you have to ensure the links are always valid or the movies won’t work.

Other security considerations

Getting the Flash player security settings right was definitely the trickiest part, because they’re a bit out of the way and not terribly popular topics of conversation according to Google.  But to make Powerpoint and Flash play nice together you also need to (clearly) enable macros in Powerpoint and allow ActiveX controls to run.  ActiveX controls don’t need any special treatment however: they can run in Safe Mode with privileges that prompt you before they are enabled with “minimal restrictions”.  I never saw a prompt, because I don’t think the Flash ActiveX control requires any special privileges.

Once you get the security settings resolved things work great. It is highly unlikely, however, that things will work out-of-the-box if you move the presentation to a new computer, due to the default security settings. But once you get things integrated, you get the power of Flash for doing complicated animations with the ease of Powerpoint for the simple stuff. Pretty much the best of both worlds!

Tags: ,

Sharing Amazon Elastic Block Store among multiple instances

Posted by: Joe Devietti on: April 11, 2009

I love Amazon’s Elastic Compute Cloud, and have been using it to run research experiments without having to worry about multiplexing computing resources among other members of my research group.  No running top after I login to make sure I’m not stepping on someone else’s experiments: I launch an instance and I get it all to myself.

Sharing storage across instances, however, is tricky.  For my purposes, having a read-only copy distributed among my instances is sufficient; of course adding read/write access makes things substantially trickier.  Yet even given that I was fine with read-only access, none of the solutions that immediately came to mind were satisfactory:

  1. Put everything in the root partition of my instance.  Every instance launched will have this same image, which is great.  The problem is that the root partition is just 10GB and doesn’t seem to be able to be increased.  Also, bundling up an instance is quite slow, making updates painful.
  2. You can get copious amounts of ephemeral storage via /mnt on each instance, but this doesn’t persist across the lifetime of an instance.  I could download a tarball from S3 and extract it to /mnt every time when I launch an instance, but this seems very hackish.
  3. You can get persistent network storage in the form of the Elastic Block Store, but this can only be attached to one instance at a time.  Doh!
  4. You can run NFS or GFS or whatever you want on your instances after you launch them, but this seems like a lot of work.  I’m supposed to be working on research, after all.

What I really wanted was the ability to mount an EBS volume read-only on multiple instances.  Since things are read-only, there won’t be any consistency issues but, still, Amazon doesn’t support this.  Until I discovered a hack to make it possible, using EBS snapshots.

The basic idea is to have a master EBS volume V that you want to replicate with read-only copies across a number of instances.  Upon bootup, each instance makes a snapshot of V and then its own personal volume Vp based on that snapshot.  Each instance can then attach the volume Vp and voila – we’ve got our data replicated across our instances.  No fancy network filesystem or S3 hacks necessary.

What makes all this go is that EBS snapshots are very fast (because they’re lazily constructed).  My master volume V is 10GB in size, and about 7GB full at the moment.  And this whole take-a-snapshot-and-mount-it routine takes less than 10 seconds.  After I’m done with an instance, I have it throw away the snapshot and volume Vp to save space.  But since snapshots are built on diffs, having a bunch of snapshots doesn’t take up much room in S3 (i.e. cost much money) anyway.  Ultimately, EBS is doing exactly what I would want to provide a high performance read-only version of the volume: lazy creation of snapshots makes replication fast, and each snapshot volume functions as a cache to increase read bandwidth.  And all this without any extra engineering on my part!

I put together some Python scripts (with the help of the excellent Boto library) to automate this read-only replication of an EBS volume.  All you have to do is edit some parameters in ec2lib.py and then link these scripts into your distro’s boot/shutdown routines; this code is designed to be run from the instance itself.  The code is available under the MIT license (like Boto itself).  The repository includes a copy of Boto 1.7a to keep things self-contained.

Lockfox version 0.1 released

Posted by: Joe Devietti on: March 20, 2009

The first beta version of the Lockfox Firefox extension has been posted to addons.mozilla.org!  I developed this with Rohit Chaudhri as our class project for Yoshi Kohno’s graduate course in computer security at the University of Washington.  I’ll use this blog to talk about the development of Lockfox and interact with its (eventual) users.  Right now it’s just an experimental addon (so it’s kinda hard to find) but hopefully it will soon pass AMO’s code review and be listed as a trusted, public addon!

How Lockfox works

Lockfox works in a manner similar to the SSH known_hosts database, but instead of remembering an association between public keys and domain names, Lockfox remembers an association between a password submitted on a web form and the domain to which that password was submitted. For added privacy, Lockfox stores only the SHA1 hash of the password, not the password itself. Lockfox monitors all password submissions and checks on each submission if a remembered (hashed) password is being submitted to a new, unknown domain. Lockfox prompts the user with a dialog to ensure that the user wants to submit their password to a new domain. If the user authorizes the submission, Lockfox forges a new trusted association between the password and the new domain. Otherwise, Lockfox redirects the user to the old, trusted domain.

For additional security, Lockfox also remembers any SSL certificate information available for the site to which a password is submitted. If this certificate information changes unexpectedly for some password submission, Lockfox requires the user to authorize the submission. The intuition here is exactly the same as that behind SSH’s known_hosts database: an unexpectedly changed SSL certificate likely indicates a man-in-the-middle attack.

Why Lockfox (hopefully) works

The intuition behind Lockfox is that a password is generally used only with a very limited set of domains – if a password gets sent to a different domain, it is likely that a phishing attack is taking place.  Lockfox has a number of advantages compared to existing anti-phishing techniques:

  • Lockfox works entirely locally, building up a custom set of associations for each user without any global coordination or public key infrastructure.  There’s no blacklist to be broadcast or constantly updated.
  • By remembering a set of trusted associations between passwords and web sites, Lockfox builds up a custom set of associations for each user.  This is, in a sense, a per-user whitelist: it has the advantages of a whitelist (shorter to specify, default action is to be secure) but without the disadvantage of maintaining a whitelist that works for all users.
  • Lockfox is able to detect and prevent phishing attacks at exactly the moment that they occur, which will hopefully result in an effective form of user education about how to detect and avoid phishing sites.

To be fair, there are some disadvantages to Lockfox’s approach, too:

  • If a user recycles a password between multiple sites, the user will receive a Lockfox warning for each new site.  This may annoy the user enough to make them disable Lockfox altogether.  To try to ameliorate this, Lockfox can optionally, at installation time, import passwords (and their corresponding web sites) that are remembered by Firefox as “trusted”.  So if you’ve built up a large set of remembered passwords with Firefox, Lockfox can automatically trust those sites that you, implicitly, already trust.
  • Lockfox works only for passwords, and not for other information a user may want protected, such as credit card numbers, social security numbers, etc.

Ultimately, Lockfox is just one more tool for keeping people safe online.  It will work best when combined with existing anti-phishing techniques (such as those already built-in to Firefox).

Are you using Lockfox?  Thanks for your bravery, and please post any feedback in the comments!

Tags:

DMP ASPLOS presentation README

Posted by: Joe Devietti on: March 15, 2009

Here are some instructions to help you run the presentation I gave on Deterministic Shared Memory Multiprocessing at ASPLOS 2009.  The presentation itself can be downloaded here.  This presentation uses Shockwave Flash movies embedded in Powerpoint, and interaction between the two, so there are a few steps you have to go through to get everything to work right.

Software you need installed:

  1. Microsoft Powerpoint, 1997 or newer.  I’ve only tested this on Powerpoint 2007 however (on Vista, but hopefully that doesn’t matter).
  2. Install the Adobe Flash 10 ActiveX control, if you want to play the Flash movies inside Powerpoint.

There are macros on the slides with Flash movies that synchronize the advancing of the slide with the animation being played by Flash.  Essentially, any slide with a Flash movie is “driven by” the movie – clicking on the movie will either advance the movie or the animated elements (e.g. bullet points appearing) on the Powerpoint slide, as appropriate.  The Flash movies need enhanced privileges to be able to call code outside their ActiveX container, and you need Powerpoint macros enabled so that the Flash movies have some code to call.

I just wanted to look at your slides…

If you just want to look over the slides and/or play the Flash movies in a standalone Flash player, you can ignore all the nonsense below.  This is also helpful if you don’t want to run Powerpoint macros or trust any Flash content to run with local privileges.

To get things to work with macros disabled and the Flash content remaining untrusted: just click on the movie as usual and, if nothing happens, that click was supposed to animate something on the slide instead.  You can just click on the slide (not the Flash movie! – clicking on the slide title text is safe) to animate the slide manually.  It requires moving the mouse around, so I didn’t want to do that for my actual presentation, but it works just as well for a less formal setting.

If you don’t want to install or run any ActiveX plugins in Powerpoint, you can also just play the Flash movies in a standalone Flash player.  Synchronizing this with the slides is kind of klunky though.

I want things to work seamlessly…

First off, I’m pretty sure this only works on Windows, due to the dependence on the Flash ActiveX Control.  I’ve tested it on Windows Vista, but XP should be okay too.  There are 4 things you need for this to work seamlessly.

  1. Powerpoint 1997 or newer, and the Flash Player 10 ActiveX Control (installation instructions for the latter here).
  2. Enable macros in Powerpoint.
  3. ActiveX controls can be set to run with their default privileges, i.e. you will get a prompt before they do anything scary, and they can also run in Safe Mode.
  4. The Flash movies need to be trusted so that they can interact with the Powerpoint presentation.  This is described below.

The Easy Way (slightly less secure)

Visit the Flash Security Settings Manager web page and specify that you trust the Powerpoint executable via the “Add location” dropdown.  With Office 2007 on Vista, my Powerpoint executable was at C:\Program Files\Microsoft Office\Office12\POWERPNT.EXE

Why is this way less secure?  By setting Powerpoint as a trusted source for Flash, any Flash content that gets embedded into Powerpoint will run with the highest privileges (which includes access to local files).  You may or may not feel comfortable with this.  The upside is that you can move the presentation to a new directory without breaking anything.

The Secure Way (slightly less easy)

Visit the Flash Security Settings Manager web page and specify that you trust only the directory where the Flash movies for the presentation are (e.g. C:\Users\Admin\Desktop\dmp-asplos2009-presentation\ if you downloaded the presentation and unzipped it onto your Desktop).  You do this via the “Add location” dropdown.

This is more secure because only the Flash content in the specified directory will be trusted, but if you move the movies or the presentation to a new directory, things will stop working (and silently!).

Eep – something doesn’t work…

There’s a test slide at the end of the presentation which has an embedded Flash movie in it.  The movie runs some basic sanity checks on your configuration and displays the results.  If everything appears ok, then the presentation itself should run fine.

Interacting with the movies

Clicking on a Flash movie will advance the animation one step forward.  Pressing the ‘c’ key makes the animation go one step backward.  That’s all there is to it!

Tags: , ,

Installing Adobe Flash Player 10 ActiveX Control

Posted by: Joe Devietti on: March 12, 2009

Here’s how I got the Adobe Flash Player 10 ActiveX Control to install under Windows Vista, so I could embed Flash movies in Powerpoint.

  1. Go to the Flash Player download site.  Download the “Adobe Flash Player 10 Update for Flash CS4 Professional” zipfile – 44MB.
  2. Unzip the archive somewhere, and go to the Players/Release/ directory.
  3. You’ll find an executable named Install Flash Player 10 ActiveX.exe, which will do just what the name says!

This is kind of convoluted; maybe people have found easier ways?

Tags: ,

Powerpoint + Flash = Awesome (eventually)

Posted by: Joe Devietti on: February 21, 2009

I recently attended ASPLOS 2009 in Washington, D.C., and put together some slides for the presentation I gave on Deterministic Shared Memory Multiprocessing. To appease my endless thirst for new technologies, I figured I’d try using Powerpoint with embedded Shockwave Flash movies for the presentation, instead of the traditional vanilla-Powerpoint. Why bother making my life so complicated? Well, there were 5 main reasons.

  1. I like complicated things.
  2. The animations I wanted to do were, in my opinion, just too complicated (see above point) to create in regular Powerpoint. I wanted to move shapes in very precise ways. Doing motion paths in Powerpoint is particularly tricky, because the motion path is based on the center-point of the shape, so it’s hard to judge what the motion will look like until you actually see it.
  3. Even more importantly, animations in Powerpoint are hard to change. It’s hard to keep things consistent, because changes don’t propagate. By doing things in Flash instead, I could write code to script my animations, so my toolkit was now limited only by the power of the abstractions I could build.
  4. The animation engine in Powerpoint is a bit sluggish – I observed a fair amount of graphical tearing when moving shapes around (especially large ones), even though I was running the presentation on a Core2 dual-core laptop, had the CPU running at full power with nothing in the background, had hardware acceleration enabled in Powerpoint, etc. Flash just brings a little extra crispness that’s hard to justify until you see it running at a beautiful 30 fps.  Worth all the pain I went through to make it work? Probably. Easy on the eyes? Definitely.
  5. Finally, Adobe has graciously open-sourced the toolchain needed to compile your own Flash movies from Actionscript. This, coupled with the excellent FlashDevelop Flash IDE, was all I needed to convince myself that this crazy adventure was going to be worth it. Convincing my adviser, however, was a different story… ;-)

So, the next series of posts will describe how I got Flash, Powerpoint, ActiveX and other buzzwords to play nicely together.  I hope this will prove useful to others who might be crazy enough to try this out – I can definitely see myself doing this again someday.

Tags: , ,

Search


  • Helen Neely: Thanks for sharing your code. I will check it out later today and play around with it. I just started working on SVN for the first time; and think Goo
  • Joe Devietti: Hi Rahul, Yes, Lockfox is open source: it's all written in Javascript, and licensed under the GPLv3. A few ways to peek at the code: 1. Create your
  • Rahul: Hey guys! Great job making lockfox. Is it open source by any chance? Where can i get the source code?

Categories

Archives

Blog at WordPress.com.

Albeo theme by Design Disease