New Serving Area in Floor Lounge

After renovations this summer left Burton Third with a bare floor lounge, Bombers built a new serving area for our favorite room on the floor. It was built in just a few days so that we could have it ready for when freshmen arrived for REX.

Impressively, the whole thing separates into 3 pieces for portability. We also added felt sliders so we can move it when not in use.

The serving area separates into three pieces for portability

While we are very happy with the newest addition to the floor lounge as is, numerous Bombers are already discussing future improvements to the serving area.

Bombers in Paris

Endless bottles of wine. Fresh cheese from the local fromager. French women. The scent of warm croissants in a narrow cobblestone alley. Beautiful French women. Meticulously prepared escargot. Artistic and esoteric films. And of course, French women. Such are the wonders of life around us that I and three other Bombers have seen as we spend the summer in enrapturing city of Paris.

Having studied various levels of French, MIT’s MISTI program provided us with the opportunity to spend the summer in Paris, France interning at various institutions around the city. It is a unique program, as MISTI coordinators help to find placements for MIT students that are unavailable for the typical college student. Of course, this is not a study abroad program; students spend 5 days a week working in labs and companies. Yet it is a small price to pay for the opportunity to spend a paid summer in la Ville Lumière.

Being Bombers, we like to stick together. As such, we were able to find an amazingly lovely apartment in the 13th arrondissement through AirBnb. It is fairly spacious and quite livable, and the rent split between 4 people isn’t bad at all. More interesting though is our neighborhood. We live in a neighborhood called La Butte aux Cailles (The Hill of Quails). Located outside the busy and crowded city center, La Butte aux Cailles forgoes tourists in favor of a much more authentic Parisian feel. The streets are narrow and cobbled, and awesome street art adorns the buildings. The entire neighborhood is very unpretentious, yet people are always out late walking the streets or drinking at cafés and bars. If you visit France, make sure to stop by for a more authentic view of Paris. Also, the restaurants serve amazing French cuisine without the exorbitant prices of the inner arrondissements.

How do we spend our weekdays?

As a mathematics and management science double major (Courses 18 and 15), I, Zac, am working at a company called Kyriba. They provide a cloud-based corporate treasury management application, allowing “CFOs and finance teams to optimize their corporate treasury functions and minimize risk…” Yeah, it is much cooler than it sounds. My work has consisted of IT research in which I research and present various IT solutions to the Head of R&D and Senior Developer in order to tackle problems as the company begins rapid scaling of both engineers and infrastructure. Additionally, I’m working with a lab at INRIA in an attempt to implement a real world deployment of their research project at Kyriba that automates and optimizes the deployment of cloud-based virtual machines.

Kristen, a Class of 2014 senior studying Biological Engineering, arguably has the most interesting work. Her work involves researching the role of the mGlu7 receptor in cocaine and opiate addiction. As she says, “the blocking of this receptor could lead to a novel method of blocking addictive effects of these drugs, and could be used to prevent withdrawal symptoms in users of such drugs.” So yes, she spends her days injecting mice with cocaine and opium as a mouse drug lord. Unfortunately, lab research positions don’t pay nearly as much as actual cocaine drug lords.

Camille is the only veteran francophile, having spent her past two summers in France through the MISTI program. As a linguistics major (Course 24, Class of 2014), she is working at Le Laboratoire de la Linguistique Formelle studying the effects of hand gestures on sentence comprehension and comparing these effects as seen in healthy young people, middle-aged citizens, and Parkinson’s patients. Much to my dismay, this does not mean she yells at people while flipping them off, and then seeing if they understand her better. She also runs other psycholinguistic experiments, and spent a week on an island off the Western coast of France for a seminar on experimental methods in linguistics.

Sunanda (Course 7, Class of 2014), our fourth and final Bomber, works in the Livet Lab at l’Institut de la Vision working on a technique for genetically encoding a fluorescent expression called “Brainbow”. According to her, “it allows for cells that come from the same progenitor to express the same fluorescent color, which can be useful in identifying cell types and migration. One application, for example, would be tagging a cancer cell – as it divides, all of its daughter cells will be easily identifiable for removal.” As such, she spends her days staring at MATLAB code and maliciously cutting open quail eggs.

What do we do in Paris while not working?

Explore, eat and drink, pretend as if I know what people are saying to me.
Of course everyone knows about the artifacts in the Louvre, the impressionism paintings in the Musée d’Orsay, and the splendid Eiffel Tower at night, so I won’t bother describing those. Much cooler is the Musée de l’Érotisme. It displays displays sexual and erotic artifacts from cultures around the world, ranging from ancient China and India to 19th century France. In addition, they have some more modern expositions. I know that you probably think “who the hell would want to visit a museum of erotica?”, but get over your awkward fears and insecurities and go visit. The exhibits were strange, intriguing, beautiful, and sometimes disturbing (Google “harukawa namio maxi cula” if you are curious), but (almost) never did the museum feel crude or dirty. It is really worth a visit, as you have probably never experienced anything like it.

As you know, food in Paris is magical. The absolute best meal I have had in Paris, and my life overall, was at Le Chateaubriand. It is a small and unpretentious restaurant and bar, and for 65 euros, I had the best 8 course meal of my life. But as college students, our typical meal costs a bit less. We spend many a night are spent eating bread, cheese, and wine along the Seine. Jussieu and Invalides are always filled with people along the Seine listening to music or dancing salsa. There are few things as wonderous as looking over the river in a wine induced haze with far off lights dancing in the Seine.

Other adventures include bike rides through the Bois de Boulogne, dancing to shitty French DJs at various clubs, drinking some delicious mojitos at 2 am, discussing Drake with a lovely Moroccan girl, and salsa lessons in a menage of French and Spanish. Possibly my favorite experience was during the Paris Cinema International Film Festival where I saw directors discuss and present their films, all the while surrounded by other lovers of French and foreign film.

As the summer winds to a close, we will unfortunately be splitting up for voyages to other parts of Europe. Kristen and Camille will be searching for One Direction in London as well as adventuring in Kiev, Istanbul, and Morocco. Sunanda is going to experience life amongst the sheep in Estonia with her boyfriend’s family, and I will be heading to Copenhagen with the goal of eating as much uncooked fish as possible. And as wonderful as this summer in Paris has been, I think we are all craving a little bit of ‘Murica.

Hacking the Dropbox Space Race

Five days ago, Dropbox launched its first collegiate Space Race, and gave us MIT folk a chance at eternal glory.  To nobody’s surprise, it was only a matter of hours before we had achieved a landslide victory.  With numbers equivalent to 75% of our student body quickly joining the fun, there was no way other universities could come close to our Dropbox dedication.  Students proudly wore their fresh-pressed, Career Fair-issued Dropbox shirts all across campus, and walked with a new swagger in their step.  MIT students aren’t ones to brag, but for once our unity had truly accomplished something great, while other campuses were probably off making another Gangnam Style parody.

But then disaster struck.  Just one day later, international universities with larger student bodies pulled ahead in the rankings and left us in the dust.  Dropbox’s founder and MIT Alum, Drew Houston, tried to lessen our emotional damage by creating a “United States Leaderboard” where we still held the #1 position, but the damage was already done. Our swagger was replaced with a limp, and some students even started creating an MIT Gangnam Style parody.

All hope was lost, but a number of student groups refused to give up.  With unparalleled determination, these students worked through the night as they slowly pieced together a solution.  Tonight, one of those solutions was put into action, and MIT once again became #1 in the world.  Here’s how it worked:

Post Mortem Note: The blog post below was written with a cheerful sense of optimism before we ran everything together.  Although everything below has now worked, the issues we hit during deployment are interesting and warrant another blog post.  Disclaimer: please don’t try this at home!  It’s left one of us without access to MIT’s network.

 

First things first, we needed a way to automatically generate lots of email addresses.  They didn’t actually need to be MIT email addresses for us to earn points, but what fun would it be if they weren’t?  We’re fortunate that MIT keeps a relatively open network, so students are able to create their own @mit.edu mailing lists without any approval.  The obvious solution was to create mailing lists directly in the terminal with blanche but unfortunately only administrators can create lists through the command line.

That meant we were stuck doing it the old-fashioned way, through a GUI web interface.  Most of MIT’s utility websites (checking registration, grades, billpay, etc) require special MIT certificates to access.  Usually when a web developer wants to secure their website, they will purchase an SSL Certificate from a Certificate Authority, and the common Certificate Authority signatures (GoDaddy, Verisign, etc) are built directly into the browser.  Then, the developer will create a login system to determine identity.  MIT does things completely differently.  MIT is it’s own Certificate Authority which isn’t included in browsers, so students must first download the MIT Certificate Authority.  Then, instead of using a login system for identity, students download an MIT X.509 Personal Certificate to prove their identity to MIT web services.

While it’s relatively easy to implement a web crawler that uses standard SSL, creating one that uses MIT X.509 certificates is notably more difficult.  We spent a couple of hours trying to figure it out until someone came up with a better solution.  In general, using certificates is the easiest way to access all MIT web services, so we all assumed it was the only way.  Then we remembered that some services, including mailing list administration, are also also available with our student login information through Touchstone. http://ist.mit.edu/touchstone-detail

This meant we could programmatically step through mailing list creation using standard SSL authentication.  We used a headless browser called Mechanize to automate the creation of mailing lists.  But before actually creating the lists, we needed to come up with names for them.

We decided that total, we’d like to register about 30,000 Dropbox accounts.  Our names needed to look realistic, but also be random enough that they couldn’t stop us with simple pattern recognition.  We decided to take the list of all currently registered email accounts at MIT and add some random characters to them.

The next issue was that we asked MIT’s network administrators how we could programmatically create lists, and they told us we couldn’t.  Making 30,000 mailing lists the next day probably wouldn’t look too good, so we came up with an alternative.  Instead of creating 30,000 mailing lists, we’d only create 1,000, then rename them after the Dropbox account is registered.  When the process was complete, we’d delete the 1,000 lists to effectively leave no trace.

Now that we had a way of getting our mailing lists, we needed a way to actually register those emails with Dropbox.  Submitting the registration form was easy, we just used Mechanize again to automate the browser interaction.  The next step was to automate clicking the verification link that Dropbox emails to new users.

We’re using mailing lists, and there isn’t any way to directly check a mailing list’s email.  Instead, we added the same Gmail account to each mailing list.  We also made sure that each mailing list was private, so no rogue MIT interns at Dropbox could tell what email account we put on the list, or which MIT account created the list.

We created a process that would poll the Gmail account every 30 seconds using Python’s imaplib.  Then, we used Mechanize again to click the link. At this step, Dropbox also requires that the new user re-enter their password.  We created each account with the same password, so we just enter that password with Mechanize and the account is verified.  This step brought up my favorite conversation through this whole process:

“Crap, they’ll be able to tell that we’re using the same password for each account, and stop us that way.”
“Wait no, they won’t be able to tell if they’re hashing their passwords properly.”

Unfortunately, just verifying an account isn’t enough to actually get us any points.  Dropbox requires that the new user install the Dropbox client before awarding any referral points.  This is where things started to get more tricky.

This has to completely automated, so we decided to use Linux, where we figured it would be easiest to automate installation.  Our first test was to uninstall Dropbox, then reinstall it with the new user.  The installation worked fine, but when we tried to get the referral points we were told, “This referral looks sketchy.”  Busted.  They must have put a flag somewhere on our computed to indicate that we had installed Dropbox before.

So we set up two fresh Linux boxes.  On the first, we installed Dropbox but never logged in with a user.  On the second, we installed Dropbox and logged in with a user. Then we diff’ed the two installations in hopes of finding the flag.

Meanwhile, someone else went to Google and discovered that all we needed to do was change our MAC Address. That was easier.

After doing some research on how we can spoof our MAC Address, we came across a utility that we could use while installing Dropbox from the command line. Great, now we we’re making some real progress.

We needed to run the Daemon without any environment variables to prevent a GUI from opening.  In the command line, Dropbox outputs a URL that we’ll use to link our registered account to this installation of the client.  We open that URL with Mechanize to link the account, and our referral is complete.

But we didn’t stop there.  A simple referral only gets us one point, for more points the new account actually needs to start “using” Dropbox.  For the purposes of getting referral points, this means completing Dropbox’s “Getting Started” wizard, and sharing a file with someone else. This involved need the csrf token from the page and watching the ajax requests that were made to complete the steps online. The token was then taken from the cookie of the Mechanize browser that was used to log the user in.

We used Mechanize against to walk through the online Getting Started wizard.  Then, we ‘touch’ed a file into a new folder within Dropbox, and used Mechanize to share the folder through Dropbox’s web interface.

Now we had all of the individual components running, but we still needed to figure out how they would work together.  The process of creating a single account actually takes a few minutes because of delays while we rename mailing lists and wait for emails from Dropbox.  We needed to have a number of queues, and processes adding and removing from the queues.  We couldn’t bare to run all this “space race” complication from a generically named server, so we decided to call ours “Houston.”  Pronounced like the city in Texas, not the street in New York, and certainly not Drew Houston’s last name.  Although, we do get a chuckle from the idea of someone calling Drew and saying “Houston, we have a problem.”

Our application effectively used 6 different queues and 4 different processes.

The 6 queues are:

  1. unused_usernames – Contains a list of all the usernames that we will register on Dropbox
  2. available_mailing_lists – Contains a list of the mailing list names that aren’t currently undergoing registration.  It starts with the 1,000 mailing lists initially created which are randomly named.
  3. awaiting_registration – Contains a list of functioning mailing lists that will be registered
  4. registered – Contains a list of functioning mailing lists that have been registered
  5. verified – Contains a list of functioning mailings that have been verified
  6. winners – Contains a list of all registered, verified, and installed mailing lists.

The four processes are:

  1. renamer
  2. registrar
  3. verifier
  4. installer

renamer takes a mailing list from available_mailing_lists, and renames it to one of the unused_usernames.  It then removes the entries from both available_mailing_lists and unused_usernames.  Now we have to wait a few minutes for the changes to take effect, then we add the new mailing list name to awaiting_registration.

registrar takes emails from the awaiting_registration list and uses Mechanize to walk through Dropbox registration.  Once complete, it moves the name from awaiting_registration to registered.

verifier constantly polls the Gmail account to look for verification emails.  When it sees one come in, it completes the verification with Mechanizer then finds the name within registered and moves it to verified.  Because we no longer need the email to function at this point, we re-add the name to available_mailing_lists to be renamed again.

installer takes names from verified and runs a Dropbox installation for that name.  It does this with fakemac on a Virtual Machine in AWS.  After installation, it walks through Getting Started and Shares a folder for additional referral points.  Finally, it moves the name from verified to winners.

a btb “So I got to South Station, pulled out my laptop, and…” comm.prod

Bomber Hack Project: CommRater

The CommRater site was originally Aviana’s idea which she came up with one of the many times we were procrastinating in the updown this past year. We talked about the project sometime before (or at the beginning of second semester) but development didn’t start until this summer. I really wanted to start during the year but got caught up with work and became too lazy to start a side project. In the end I think this was the best decision since it resulted in a much higher quality website being built by Max and I.

At my summer internship the project I’m working on involves using Django so we thought it would be a good idea to make this a Django project. Neither of us had ever used Django before but we decided that since I had to learn the framework for work anyway. Django has many built-in features that something more lightweight like Flask doesn’t offer out of the box which also made it a good choice. We began the project towards the end of June (or sometime thereabouts) and have been working on the site almost every night ever since. We committed most of our nights after work and a fair amount of our weekend time the past few weeks. The site is now live but is still very much in development right now.

The basis of the site is fairly simple. People have to write comm.prods (funny snippets at the end of emails to the floor) and there was no quantitative way to rate the quality of these comm.prods The site initially started as a simple voting system of up and down votes for each prod allowing the floor to collectively decide which prods were actually good or bad.

As time went on, however, Max and I decided that the site should be much more than a rating system and should really be a home online for Bombers. I’ll briefly outline the systems that we have in place currently as well as our vision for what the site is to become in the future. If you are (or were) a Bomber and you want to register email us at: comm.prod@mit.edu and we’ll hook it up.

Scraping:

The basis of the site is being able to take prods from the floor’s email and post them to our site. This functionality took a lot of trial and error due to dealing with the nightmare of email parsing. I must of dumped the database 100 times before we got the regexes and email thread splitting functionality working at the level that we wanted it.

The basic flow is as follows:

First we added a gmail to the floor mailing list so that we could have a reliable place to get the emails to the floor from. We then setup a MIT XVM machine running a cronjob to ping Gmail every minute and check for new mail. If mail is received it is parsed and posted to the website so we have (almost) live data.

The overall process is fairly simple however the details were hard to get right. The parser accounts for the hundred different ways that email clients decided to make threads (we need to chop off the old content to avoid duplicate prods) and all of the ways that people write ‘ a btb “content here” comm.prod’. It was a fun process and definitely helped me get more savvy with regexs and parsing. Another complication was that people send emails to the floor from multiple emails (MIT mail, Gmail, etc) so we setup a ‘half’ user profile for this case and give users the options to claim these account which merges the two accounts into one.

UserProfiles:

Each user has a profile page in which we display some basic stats of how your rank against the floor and against your class. We also offer the ability to filter the user’s prods with filters such as Recent, Best, or Worst. Each user also has a score which is, based on the number of upvotes for commprods and corrections (discussed below) that they submit.

Correction System:

The parsing problem is one that will never be perfectly solved. Users can write their email in such a way that it is incorrectly parsed leaving wrong content or missing content on the site. To help fix this I built a correction system that allows users to try and fix the parsing mishaps. The basic idea is that the problem can be fixed by crowd sourcing. Users can submit a correction which can then be upvoted and submitted (a correction needs 5 upvotes) or downvoted and deleted (5 downvotes). This is probably the least used feature on the site right now except by admin use but with V2.0 it is more accesible for users.

Display:

The display of commprods is largely influenced by Max’s experience working at Twitter (he copied some of the CSS directly :) ) giving each commprod a “tweet” look and feel. Each prod has a voting option, current score, author and content. In addition you can hover for a snippet of the original email content or click to get the permalink page to see the full content or submit a correction. Commprods containing media content (a url, image, or youtube video) are displayed inline so that the prod is fully shown during the render. The latest release will also allow users the favorite commprods so you can go back to ones that you like.

Trending Score and Data:

Another way to sort prods is by trending score. Each vote adds a fixed amount to the trending score and each minute (with the ping to Gmail) we subtract a fixed amount off of this. This allows us to sort on trending score and display recently voted on prods to users.

In addition to trending scores, each day we record a trenddata point for each user. We basically take a snapshot of their current total score and store them so that we (in a later version) we can show floor trend graphs of user performance. Currently we use the trend data to compute your 30 day trend score.

Future:

The latest release features a whole new set of commprods (thanks to Chris Post) so the site has data from 2005 to present. Max is working hard on a recommendation algorithm so users can be targeted with relevant prods (which is important as the dataset grows and spans many years). Some other new features that I have been working on include favoriting of prods and sorting by prods that contain media content as well as some internal admin tools and optimizations.

Ultimately our goal for the site is to be more than just a way to display and rate prods. We have a few people that want to help develop and we hope to continue working on the project during the school year. Some ideas that have bounced around for addition to the site are as follows:

Text alert system to notify the floor of happenings

Airport/Taxi management system

Floor *shopping* management system

Photo display for prospective MIT students to see what it’s like to be a Bomber

Donations page for alum to give back some love (by that I mean money :) )

Currently the site is up at: burtonthird.com

Sorry, but it’s not public. If you want access (and are a bomber) let us know at comm.prod@mit.edu

Our code is up at, check it out: https://github.com/cantbloom/comm.prod

Bomber Hack Project: Hungry4

Hungry4 was the first project that Max and I worked on this summer and was built during the Hack To the Future hackathon. This was my first hackathon as well as my first node.js project and was really fun to work on and spend a day building. Max and I had talked about the idea a few days before the hackathon and discussed some basic details of the implementation, however, we started and finished everything (except a few tweaks) over the 24 hour period during the hackathon.

The basic idea is that it is hard to find what you are hungry for. Hungry4 directs you to tasty food based on your location (and willingness to travel). Max wrote most of the frontend and recommendation algorithm while I worked on the backend and pulling the data.

The entire app is a single page which start here getting your basic location information:

Based on your search query we hijack the Foodspotting API and query their site for results for your location. We contacted Foodspotting multiple times asking for access to the API but they only replied with “It’s private, sorry” so we just emulated their own requests to get back the data. The call to Foodspotting provides the high quality photos for you to browse and metadata about the dish (name, restaurant, etc) and help us direct you to whatever you’re Hungry4.

The results let you upvote or downvote an item until you find something that you’re satisfied with. We rated each dish in 8 different dimensions [i.e. dinner, breakfast, lunch, hipster, spicy, etc] by dividing the number of Google search results for the dish’s name by the number of results for the name and each different dimension. Then, giving each user a starting standard user vector, your vote would change your vector directing you towards (or away) certain types of dishes. The entire site is stateless, we hold no user information and didn’t user a database of any kind. When the server is first started we cached results for San Francisco since that was our most common query and we would preload the next group of dishes to display to reduce loading times.

If you didn’t find anything you can search again and restart the process. Clicking the “Show more info” button would give you a Google Maps view of the restaurant as well as Yelp information and the option to Tweet or post to your Tumblr about your find.

Overall this was a really fun project to work on and gave me a better taste for web development (this was my second web project). It was also the beginning of the many hack nights that Max and I have done over the summer building the CommRater site.

The website is up at: http://hungry4.herokuapp.com/

And our code is public at: https://github.com/cantbloom/Hungry4

Bomber Hack Project: DTYD ’11

A large part of Bomber culture is the hacking culture. Whether you are exploring MIT buildings at night or building stuff in your dorm room it is certainly a Bomber tradition to hack on whatever you can in your spare time build something. That being said, for the next few posts I want to discuss a few cool hacks that I have been involved with during my time on the floor.

The first hack that I was a part of was the freshman hack for DTYD ’11. Each year before DTYD the freshman class has to promote the party in a unique way with their own hack. My class decided to build a giant replica bomber shirt for the ‘The Alchemist’ a.k.a. the math man shown below:

David and I decided to take charge and began a search across Cambridge for supplies. This involved buying out a local fabric store from all of the orange fabric that they had and several trips to Home Depot. David and I made some rough measurements (mostly by seeing how much taller or longer than our arms the dimensions had to be) and began the long process of learning how to sew and sewing the shirt together.

Most of the class helped with the building of the shirt — mostly building stencils and spray painting the designs on the shirt. After about a week of preparations we were ready to go and deploy. We had no idea if the shirt would actually fit we only went off our rough estimates.

No deploy is good without some hiccups. While we were deploying the MIT police came by and told us we had to stop. There were about 4 people on the deploy team so I had them keep working and tried to talk to the cops so that they could keep itl going up. Overall the cops were very nice and I showed them an email saying that we were *allowed* to put up the hack. Without further incident this was the final product with David and I:

Here is the back of the shirt from The Tech the next day:

The hack was vandalized a day or two later by unknown persons and the shirt was never recovered. The stencils lived on and helped make a mural of the plane in the 41 suite.

Renovations

Years ago, Burton Conner stayed open for the summer and Bombers would use the time to improve and renovate the floor.  Recently though, the dorm has been closed and the floor has become quite run down.  So in the final week of school, a bunch of us got together and did what we could to fix things up.

(Apologies for the bad pictures throughout this post, still getting used to the iPhone camera)

Most of our work was focused on the floor lounge: fixing holes in the walls,  cleaning behind the bar, and repainting.

The far wall, before anything.  Note some spackling from past holes never paints over, and the incomplete paint job at the top:

After fixing more holes:

And after being finished.  We also painted the radiator in the bottom right, it’s paint was chipping and was just generally gross:

Behind the bar, before anything.  It’s littered with wires that are way too long, tons of old electronics, and things we once thought would be good to keep.

And After:

I didn’t take a before picture of this wall before, but it was decorated with old street signs, and had a number of wires strewn along it.  You can get a decent look on the right hand side of this floor photo:

We took those down, organized the wiring a bit, and gave it fresh paint.  That last wire is a subwoofer wire that was missing when we fixed the others:

Lastly, we have a bunch of special lighting installed in the floor lounge.  The two lights high in the middle, the Coors sign on the left, and the line of orange light along the ceiling:

In the past, turning these on meant finding the individual plugs and an outlet to plug them into.  Now they’re plugged into one power strip with one switch:

a btb “wait, you mean Patricia doesn’t fix the holes in the walls” comm.prod