allskonar - All

zencoder ... transcoding in style

2010-05-05T22:28:00Z

I am not saying it isn’t fun hacking together a combination of ffmpeg, mencoder and dealing with all the dependencies and throwing down one ninja trick after another, just to transcode your video so it will display properly on the web, iphone or what ever device you want them on.

Using zencoder makes processing videos really satisfying. Just tell them where the source video is and tell them where you want them to dump the output, and as any modern web application it has implemented a web hook so if you want to stay tuned about the process specify a url where the status updates will be posted to. It looks like they only send you status when it is completed. Here is a quick example of how to get this working with ruby code.


require 'rubygems'
require 'rest_client'
require 'json'

HOST  = "https://app.zencoder.com/api/jobs" 

opts = { 
  "test" =>  1,  
  "input" => "s3://bucketname/filename",
  "output" => [
    {   
      "label" => "fromapi",
      "base_url" => "s3://bucketname/",
      "filename" => "newfilename",
      "width" => 500,
      "quality" => 3,
      "speed" => 1,
      "h264_profile" => "high",
      "audio_channels" => 2,
      "notifications" => [
        "http://example.com/status",
        "you@example.com" 
      ]
    }   
  ],  
  "api_key" => "youapikeyhere" 
}

headers = {:content_type => :json, :accept => :json}
RestClient.post HOST, JSON.generate(opts), headers

Then on the receiving end you get the status update


{
 "output": {"url":"s3://bucketname/filename",
                "state":"finished",
                "label":"fromapi",
                "id":38384},
 "job":{"test":true,
          "state":"finished",
          "id":38328}
}

This is pretty much all you need to start up your “next going to conquer the world and become a web celeb” venture, well this and a lot of free time. There are some acl settings you need to apply on your bucket something I have covered previously but using the firefox pluggin for S3 is good for that stuff as well

The things I really like about this approach is that it’s decoupled from the upload process. They will pick up your file so the only thing is that you have to either have it uploaded to S3 or simply have your somewhere accessibly to them. The benefits of having it all on S3 is that the access control is built in so you don’t have to expose your high quality videos to the public unless you want to.

The pricing seems reasonable, especially given that they charges for minutes of video transcoded instead of filesize, which is a really refreshing perspective. 6 cents a minute makes it $3.6 to transcode all of the stuff my flip mino HD can record.

Go check it out it’s really fun to play with and it was written up by techcrunch today

Pimping ruby Logger instance

2009-11-24T21:53:00Z


# pimping loggin a bit 
LOG = Logger.new('logs/application.log')
%w(debug info warn error fatal).each do |level|
  eval <<-end_eval
    def LOG.#{level}(message="", &blk)
      if message == "" and block_given?
        message = blk.call
      end 
      file,line_number,method_name = caller.first.split(":")
      method_name = "" unless method_name
      filename = File.basename(file)
      super(filename + ":" + line_number) { method_name + " " + message }
    end 
  end_eval
end

I had to brag, it’s rare that I dabble in magic code i.e. code that is too clever to be understood right away and impossible to troubleshoot unless you wrote it yourself (recently). I usually try to keep my code quite simple. Clever has a bad rap in coding and for good reason since even the cleverest person can easily forget the lightning of cleverness that struck when he or she wrote something just a few weeks earlier. Even though clever code is bad it’s extremely satisfying when you solve something cleverly.

I am writing an application that operates mostly in the background i.e. without user interacting with it directly which makes troubleshooting tricky since one can’t easily throw in a debug statement and step through the code execution. I ofcourse have decent test coverage helps but in production code seems determinate to make once life miserable. This is where good logging becomes vital in chasing down bugs or performance bottlenecks in production code.

Logger in the standard ruby library is extremely awesome I have found that I just wanted a little more out of it namely I wanted by default a single log file with the name script file name, line number and perhaps the method name where a particular message was sent to the logger. There didn’t seem to be an automatic way of doing this so I resulted to do a little overwriting (the magic). I accomplished this by defining a singleton method on my logger instance.

Ruby Enumerable#partition methdod

2009-08-30T12:04:00Z

I finally found an opportunity to use ruby partition method. It splits an array you have upon a criteria you supply in a block. Typically, in the old days, I would have done this by doing something like this:


big_array = Array.new(11) {|i| i}
number_lower_than_5 = []

big_array.each do |number|
  if number < 5 
    number_lower_than_5 << number
  end 
end

# >> number_lower_than_5
# => [0,1,2,3,4]

or more recently I would use select


big_array = Array.new(11) {|i| i}
number_lower_than_5 = []

number_lower_than_5 = 
big_array.select do |number|
  number < 5 
end

# >> number_lower_than_5
# => [0,1,2,3,4]

These are both fine methods but what happens if you need the other halve as well? The above approaches just collect you an array to work with and both lack the convenient array of the other stuff you sometimes care about. That’s where ruby’s partition comes in handy. It may not be that often that’s why I was so pumped up when I found an opportunity to use it.


big_array = Array.new(11) {|i| i}
number_lower_than_5, the_rest =
big_array.partition do |number|
  number < 5 
end

# >> number_lower_than_5
# => [0,1,2,3,4]
# >> the_rest
# => [5,6,7,8,9,10]

When needed the partition method can be very convenient

no man

2009-04-16T18:03:00Z

on a recent ubuntu server instance I had to issue


$ sudo apt-get install man

is this recession taking everything away from us?

Different ways to moniter file system events

2009-04-15T22:15:00Z

The apple way

In Leopard Apple introduced launchd which is their new system startup program. Launchd is a deamon which is a manager and scheduler. In addition to eliminating the need for crontab, which is traditionally how you would run scheduled jobs on unix systems, launchd offers you a whole slew of other events you can react to programatically. Events such as mounts, file creation and modifications. The example below demonstrates creating files in a watched folder. To do this you’ll need to create a property list file with a .plist suffix and place it in the Library/LaunchAgents/ folder for the user you can also have it work system wide for all users. Name the file com.foobar.watchmyfolder.plist re-login to your computer and you should be able to test this by placing a file in the folder $HOME/dropfolder and you should see that it in return touched a file did_it_work file in your home directory.



<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" 
"http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>com.foobar.watchmyfolder</string>
    <key>ProgramArguments</key>
    <array>
        <string>touch</string>
        <string>/Users/username/did_it_work</string>
    </array>
    <key>QueueDirectories</key>
    <array>
        <string>/Users/username/watchfolder</string>
    </array>
    <key>StartOnMount</key>
    <false/>
    <key>WatchPaths</key>
    <array/>
</dict>
</plist>

Its kind of a bummer that the format for the .plist files is xml since other much nicer formats could have been used such as yaml or something lightweight format. But to get around that bump I installed lingon which has really open up the possibilities of launchd, it doesn’t include all the options it does include enough to get your imagination going.

More information

The linux way

During my research on this one it seems like the inotify library is the best option which can be used directly with the inotify-tools and also through some apis (see a c example in a link below). I have scetched a bash example below which I don’t like so much since relies on a crazy while loop to do its thing which just doesn’t seem right to me … or rather I would much prefer someone hiding that mess from me. So my hope is that there is something useful out there that works with configuration files similar to the launchd on the Mac OS X.


#!/bin/sh

while inotifywait -e create ~/watchfolder
do
  touch ~/watchfolder.log
  `date` >> ~/watchfolder.log
done

More information

Rotten Shark Tasting Fest

2009-04-11T01:18:00Z

Ah these fellas thought they knew what disgusting meant … bad taste has now reached another level.

Working With Swedes

2009-03-11T00:56:00Z

For the last few weeks in new job I hear a lot of Swedish. And all I can contribute is BORK BORK BORK!

Serializing a Bunch of Files

2009-02-15T18:55:00Z

One thing I do love is to spend 15 minutes to solve a problem that would have taken me 20 minutes to do manually … heck that’s how I got into this programming thing. Bash scripting is becoming one of my favorite tools as it’s so simple once you wrap your head around it’s weirdness … which I am hoping to do early 2010 :)

So the problem I had was that I had a bunch of photos with some stupid names which were automatically generated by my stupid scanner software, sc30rr.jpg not joking it was total rubbish so after spending roughly 30 minutes scanning photographs I had this folder filled with files with that stupid name and I just wanted to make it name_of_event_001 and have the last bit be a serialized number.


for filename in *.jpg ; \
  do n=$(( $n+1 )); \
  mv $filename name_of_event_`printf "%03d" $n`.jpg; \
done

My hope is next time I need to do this I will not have to fudge around for 15 minutes and just either remember how I did it this time or alternatively look it up on my blog :)

love/hate OS statistics

2009-02-14T03:26:00Z

Lets define Friday night boredom :)

i love windows= 45k i hate windows= 79k factor: 45/79 = 0.57

i love mac= 228k i hate mac= 10k factor: 228/10 = 22.8

i love linux= 73k i hate linux= 7k factor: 73/7 = 10.4

A kewl bash Trick

2009-02-13T11:46:00Z

when ever I get an opportunity to swings this trick out I get very excited. Say you have a task you wanna do and it’s should be applied to a number of files with a common name or are all with the same extension, basically any situation where you would be doing something that seems too repetitive for you taste.

Too create our example we can actually use the same trick. Lets say you have a number of files with the same name but have different number at the back of it: test_1.txt, test_2.txt, test_3.txt. So instead of doing touch file.txt n number of times you can instead do the following.


$ touch test_{1,2,3}.txt

Now when I was trying to figure out where I learned this from I found the website where it came from so go there for more kewl tricks.

deleting everything but ...

2009-02-10T02:17:00Z

Now if you have a problem where you wanna delete a couple of folders in a directory it’s easy to do that in a one liner


$ rm -rf folder_1 folder_2

but what if you want to delete all but those specific directories? Today my college showed me this amazing trick for finding everything but that


$ find . -maxdepth 1 ! -name folder_1 ! -name folder_2

which lists all the other folders in that particular directory and then that can easily be piped to rm.


$ find . -maxdepth 1 ! -name folder_1 ! -name folder_2 | xargs rm -rf

My Recent last.fm Tracks

2009-01-16T03:01:00Z

The art of loving music is learning to never be ashamed of your taste, learn to embrace and defend it to death. No matter how lame or kewl the stuff you have in your ear buds you have to pretend that you don’t care what others think of you … yes even while you get caught listening to Supertramp.

Now you can follow my latest tracks on the sidebar here to the right =>

Right underneath my adorable daughter Freyja which has been keeping my away from coding for the most part :) recently. Can’t wait to teach her to code!

So to back up my claim on not being ashamed of my music taste and attempt at getting familiar with actionscript I wrote a little package that will grap your rss feed from last.fm and dump it on your blog. This is the the kewl thing about actionscript it can snag stuff hosted else where and publish it on your website provided that you allow it in your crossdomain.xml file.

So here are some things I thought about actionscript until my college Adam showed me otherwise.

You don’t have to have the Flash application to author actionscript.
You don’t need flex in order to not use the Flash App.
And compiling it is easy using the straight up mxmlc from the Flex SDK just do


 $ mxmlc MyAwesomeScript.as

and that will spit out the MyAwesomeScript.swf.

Now let’s take a look at my snippet


package {
    import flash.display.Sprite;
    import flash.events.*;
    import flash.net.*;
    import flash.external.ExternalInterface;

    public class RecentTracks extends Sprite {
        private var recentlyPlayed:XML;
        private var rssUrl:String = "http://ws.audioscrobbler.com/2.0/user/bakdyr/recenttracks.rss";

        public function RecentTracks() {
            var loader:URLLoader = new URLLoader();
                loader.addEventListener(Event.COMPLETE, completeHandler);
            var request:URLRequest = new URLRequest(rssUrl);
                loader.load(request);
        }

        private function completeHandler(event:Event):void {
            var outHtml:String;
            var loader:URLLoader = URLLoader(event.target);
            recentlyPlayed = new XML(loader.data);
            outHtml = "<ul>\n";                           
            for each (var title:XML in recentlyPlayed..item..title) {
               var artist:String = title.toString().split(" – ")[0];
               var song:String = title.toString().split(" – ")[1];
               outHtml += "<li>" + song + "<br /><em>" + artist + "</em></li>\n";                           
            }                                          
            outHtml += "</ul>\n";                           
            ExternalInterface.call('Snitchmedia.recentTracks', outHtml);
        }
    }
}

Very simple little script that dials up the rss feed to last.fm and once it’s done it will call a javascript function that will handle outputting it on the page. Now this is very basic but I think an elegant solution to handle this sort of thing as I did not need to do any heroic mephisto, wordpress (or what ever the blog engine flavor of the month is) hacking and still able to add functionality to my blog. Obviously there is a lot of polishing to be done and I think I might make this a bit generic so it can be used to call in all sorts of third party fun stuff.

Now for completeness here is the javascript bit using jQuery.


var Snitchmedia = {};
Snitchmedia.recentTracks = function(playlist) {
  $('#recent_tracks').html(playlist);
};

Happy Holidays

2008-12-29T02:40:00Z

Seriously never seen a happier dog than Scout!

Why Grubsnitch is closed

2008-12-18T14:24:00Z

So I like everyone else have been scrambling to save some $$$ and one expense that was hard to justify was to keep an Amazon EC2 image running for My blog, and other pet projects that didn’t get any traffic to speak one of which was Grubsnitch.com. But now that I have settled on just using my age old Dreamhost account to store these things until that becomes unreasonable. I finally managed to get this mephisto instance running on there thanks to this page and my next projecdt will be to get grubsnitch.com running there as well.

Obsession with curl

2008-12-18T14:12:00Z

Ok, this is hardly worth putting down but it will work for me as a reminder. I have been loving the curl utility allot lately. I had a situation where I needed a to hit a page rather frequently and I really wanted to just stick it in a bash loop and just have it run. The problem is that the page was behind user/password credentials. But as it turns out curl can have it’s own cookie jar so it can sustain a session between requests. Here is how.


  curl -d "user[login]=username&user[password]=password" \\
  -c domain.cookie http://example.com/user/login

Also another trick I have been addicted to is curling a domain and only getting the response headers


curl -I http://domain.example.com

Who said I don’t like browsers?