Memecached: Real-Time meme sharing with node.js, now.js, and MongoDB

Yesterday, I wrote a web-app named Memecached. It is a service which allows you to quickly generate a meme and publish it in real-time. It went viral for quite a few hours after I tweeted about it and posted it on Hacker News, with hundreds of memes being shared, sometimes a new meme every three seconds!

Memecached is extremely lightweight. The server is tiny, written entirely in exactly 50 lines of Javascript using node.js. It uses MongoDB as the data store and now.js for real-time, remote method invocation.

Awkward text blackened

Using it is fairly trivial – you open the page, and the latest N (25 by default) memes are streamed back and populated into the webpage. On the left is a collections of meme templates – you can click any one, enter the top and bottom text, and hit publish to see it reflected on each open client.

Some Code

On the server side, the now.js initialization didn’t work normally as sometimes the client – my chrome browser – would not perform a handshake (I’ve reported this in the now.js issue here: , so I had to fiddle around with the options till I found that xhr-polling seemed to work.

var everyone = require("now").initialize(server, { socketio: {'transports': ['xhr-polling'] }} );

Publishing a meme involves validating the meme document, inserting it into mongo, and the passing it to all clients with:

// publish meme
everyone.now.publish = function(meme) {
    if(meme.name && meme.text.line1 && meme.text.line2) {
        db.collection('memes', function(err, collection) {
            // add a date field and save
            meme.date = Date.now();
            collection.insert(meme, function(err) {
                if(!err)
                    everyone.now.receiveMeme(meme);
            });
        });
    }
};

Retrieving a recent memes’ list is a trivial matter of looking up the last 25 in the ‘memes’ collecion:

// retrieve the latest few memes of a name. If there is no name, retrieve a mixture
everyone.now.getRecent = function(memeName) {
    var client = this;
    console.log("retrieving");
    db.collection('memes', function(err, collection) {
        if(memeName == undefined) {
            collection.find( {}, { sort: [[ "date", "desc" ]], limit: 25 }).toArray( function(err, docs) {
                client.now.getContent(docs);
            });
        }
        else {
            collection.find( {"name": memeName}, { sort: [[ "date", "desc" ]], limit: 25 }).toArray( function(err, docs) {
                client.now.getContent(docs);
            });
        }
    });
};

How it works

All meme template images are stored on Dropbox.

  • When a new meme is published, the server takes a JSON object from the client, containing the meme name and text. No image whatsoever.
  • The received meme object is inserted into the mongo collection memes with a date timestamp added to it. The object is also sent to all connected clients immediately so that they may update their timelines. This also means that no further database queries are required to retrieve new memes.
  • When a client connects, the last few (25) memes are queried from the database and sent back. This can further be optimized by having an in-memory queue of the most recent memes, in addition to the database.
  • All meme generation is done entirely client-side, by drawing the text over the images using Canvas. The client doesn’t have to download images, and the server doesn’t have to handle them at all.

How to run

  1. First, run mongod to start the Mongo daemon.
  2. Next, run the server: node app.js
  3. Point your browser to http://localhost:8080.
  4. Bask in Memetic paradise.

Things to do

  • Upvotes/Downvotes: This should be a ten-minute job. Just add two new keys to the ‘schema’ of the doc.
  • Sharing: Memes are rendered in Canvas. Need to get the dataUrl() and allow the user to save the image to a file.
  • Line breaks: There need to be line breaks in a meme phrase. Right now, extra long sentences will make the font shrink too much to be visible.

6 thoughts on “Memecached: Real-Time meme sharing with node.js, now.js, and MongoDB

  1. Awesome! One question: Isn’t constantly sending updates to ALL clients a little expensive when there’s a really big number of clients connected?

  2. It is the least expensive of all operations, because only *one* meme is sent to all clients simultaneously when published, not the entire database.
    Then again, large numbers of clients are a problem for any service, the only way to fix that is to have more threads/machines and load balance. Not a problem here.

Leave a Comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 44 other followers

%d bloggers like this: