1. image: Download

    The output of an image filter I’ve been working on.
source: github.com/bobpoekert/spatchwork

    The output of an image filter I’ve been working on.

    source: github.com/bobpoekert/spatchwork

     
  2. A function to get all the info about available memory from Linux that you could ever want.

    >> memory_info()
    
    {'Active': 497659904,
    'Active(anon)': 142782464,
    'Active(file)': 354877440,
    'AnonHugePages': 0,
    'AnonPages': 427524096,
    'Bounce': 0,
    'Buffers': 180330496,
    'Cached': 629043200,
    'CommitLimit': 6396567552L,
    'Committed_AS': 2035286016,
    'DirectMap2M': 926941184,
    'DirectMap4k': 8380416,
    'Dirty': 45056,
    'HardwareCorrupted': 0,
    'HighFree': 2417508352L,
    'HighTotal': 3354845184L,
    'HugePages_Free': 0,
    'HugePages_Rsvd': 0,
    'HugePages_Surp': 0,
    'HugePages_Total': 0,
    'Hugepagesize': 2097152,
    'Inactive': 745345024,
    'Inactive(anon)': 319483904,
    'Inactive(file)': 425861120,
    'KernelStack': 2187264,
    'LowFree': 450445312,
    'LowTotal': 860950528,
    'Mapped': 49483776,
    'MemFree': 2867953664L,
    'MemTotal': 4215795712L,
    'Mlocked': 0,
    'NFS_Unstable': 0,
    'PageTables': 5345280,
    'SReclaimable': 56696832,
    'SUnreclaim': 14139392,
    'Shmem': 28622848,
    'Slab': 70836224,
    'SwapCached': 7176192,
    'SwapFree': 4247822336L,
    'SwapTotal': 4288671744L,
    'Unevictable': 0,
    'VmallocChunk': 112480256,
    'VmallocTotal': 125829120,
    'VmallocUsed': 12304384,
    'Writeback': 0,
    'WritebackTmp': 0}
    
     
  3. A simple mixin for making buttons in sass without having to manually specify all of the colors.

     
  4.  
  5. image: Download

    Check out this star animation I made. It’s 50 randomly-colored triangles spinning at different rates, so they go in and out of phase making a cool glitch effect.

It doesn’t work in IE (data urls), and it’s broken in Chrome unless you’re running the latest Canary build (see this bug).  Firefox and Safari work.

    Check out this star animation I made. It’s 50 randomly-colored triangles spinning at different rates, so they go in and out of phase making a cool glitch effect.

    It doesn’t work in IE (data urls), and it’s broken in Chrome unless you’re running the latest Canary build (see this bug). Firefox and Safari work.

     
  6. I pulled my python thread-pool implementation out into a stand-alone library. It’s basically the reverse of Twisted’s reactor.callInThread; you specify whether a function/method runs in the thread pool when you define the function, not when you call it.

    Ex:

     1 from thread_pool import in_thread_pool, in_ioloop, blocking
     2 
     3 @blocking
     4 def get_thing_from_database():
     5     # If this method is not called from the thread pool,
     6     # it will result in a warning.
     7     return db.get(thing)
     8 
     9 @in_thread_pool
    10 def blocking_method(callback):
    11     # Call some blocking api, like a database driver.
    12     # When called, it will always return immediately,
    13     # and do its work at some future time in a thread pool.
    14     # NOTE: Methods decorated with @in_thread_pool are expected not to have a return value.
    15     callback(get_thing_from_database())
    16 
    17 @in_ioloop
    18 def non_blocking_method(callback, data):
    19     # Call some non-blocking api, like AsyncHTTPClient.
    20     # Guarunteed to run in a tornado IOLoop.
    21 

    Check it out on GitHub.

     
  7. This is a class I end up defining on every async project I work on. It’s the cleanest way I know of to have a callback fire when a set of operations finish, regardless of the order that the operations happen in. It really should be in functools.

     
  8. A talk by Mike Krieger, one of the cofounders of Instagram, about how they scaled their backend.

    Slides

    Notes:

    • You don’t get to choose when scaling problems will come up. There will never be another engineer to fix it for you.
    • Postgres is awesome.
    • MongoDB is iffy.
    • Use schemas as a layer of indirection in front of your tables, and pre-shard your tables into small chunks. Then, when you need more database capacity, use streaming replication to move some of those tables to a new database instance. I’ve never heard of that technique before. It definitely sounds like an improvement over doing sharding only in application logic.
    • Redis lists are great for storing user feeds. Rather than computing what needs to be in a user’s feed when it’s requested, add new items to the feeds when the item is created.
    • Instead of doing an ltrim on feeds on every insert, do it randomly some percentage of the time.
    • Apparently they use gearman instead of an amqp server.
    • You never know what you’re going to need to measure, so measure everything
    • statsd is useful for monitoring

    Recorded 4/11/2012 at airbnb HQ.

     
  9. image: Download

    Experimenting with image shearing. Click through for the actual animation (tumblr won’t let me embed it).

    Experimenting with image shearing. Click through for the actual animation (tumblr won’t let me embed it).

     
  10. image: Download

    A graph of 1000 wikipedia topics by similarity. Edges represent articles in common. Ex: Gettysburg is a town in Pennsylvania and the site of a battle in the civil war, so Battles in the Civil War would be connected to Towns in Pennsylvania.
If you click here you’ll go to a tiled version of the image that you can reasonably open in a web browser. If you want the original, gigantic image, click here.

    A graph of 1000 wikipedia topics by similarity. Edges represent articles in common. Ex: Gettysburg is a town in Pennsylvania and the site of a battle in the civil war, so Battles in the Civil War would be connected to Towns in Pennsylvania.

    If you click here you’ll go to a tiled version of the image that you can reasonably open in a web browser. If you want the original, gigantic image, click here.