allskonar

I am not saying it isn’t fun hacking together a combination of ffmpeg, mencoder and dealing with all the dependencies and throwing down one ninja trick after another, just to transcode your video so it will display properly on the web, iphone or what ever device you want them on.

Using zencoder makes processing videos really satisfying. Just tell them where the source video is and tell them where you want them to dump the output, and as any modern web application it has implemented a web hook so if you want to stay tuned about the process specify a url where the status updates will be posted to. It looks like they only send you status when it is completed. Here is a quick example of how to get this working with ruby code.

require 'rubygems'
require 'rest_client'
require 'json'

HOST  = "https://app.zencoder.com/api/jobs" 

opts = { 
  "test" =>  1,  
  "input" => "s3://bucketname/filename",
  "output" => [
    {   
      "label" => "fromapi",
      "base_url" => "s3://bucketname/",
      "filename" => "newfilename",
      "width" => 500,
      "quality" => 3,
      "speed" => 1,
      "h264_profile" => "high",
      "audio_channels" => 2,
      "notifications" => [
        "http://example.com/status",
        "you@example.com" 
      ]
    }   
  ],  
  "api_key" => "youapikeyhere" 
}

headers = {:content_type => :json, :accept => :json}
RestClient.post HOST, JSON.generate(opts), headers 

{
 "output": {"url":"s3://bucketname/filename",
                "state":"finished",
                "label":"fromapi",
                "id":38384},
 "job":{"test":true,
          "state":"finished",
          "id":38328}
}

This is pretty much all you need to start up your “next going to conquer the world and become a web celeb” venture, well this and a lot of free time. There are some acl settings you need to apply on your bucket something I have covered previously but using the firefox pluggin for S3 is good for that stuff as well

The things I really like about this approach is that it’s decoupled from the upload process. They will pick up your file so the only thing is that you have to either have it uploaded to S3 or simply have your somewhere accessibly to them. The benefits of having it all on S3 is that the access control is built in so you don’t have to expose your high quality videos to the public unless you want to.

The pricing seems reasonable, especially given that they charges for minutes of video transcoded instead of filesize, which is a really refreshing perspective. 6 cents a minute makes it $3.6 to transcode all of the stuff my flip mino HD can record.

Go check it out it’s really fun to play with and it was written up by techcrunch today

I finally found an opportunity to use ruby partition method. It splits an array you have upon a criteria you supply in a block. Typically, in the old days, I would have done this by doing something like this:

big_array = Array.new(11) {|i| i}
number_lower_than_5 = []

big_array.each do |number|
  if number < 5 
    number_lower_than_5 << number
  end 
end

# >> number_lower_than_5
# => [0,1,2,3,4]

or more recently I would use select

big_array = Array.new(11) {|i| i}
number_lower_than_5 = []

number_lower_than_5 = 
big_array.select do |number|
  number < 5 
end

# >> number_lower_than_5
# => [0,1,2,3,4]

These are both fine methods but what happens if you need the other halve as well? The above approaches just collect you an array to work with and both lack the convenient array of the other stuff you sometimes care about. That’s where ruby’s partition comes in handy. It may not be that often that’s why I was so pumped up when I found an opportunity to use it.

big_array = Array.new(11) {|i| i}
number_lower_than_5, the_rest =
big_array.partition do |number|
  number < 5 
end

# >> number_lower_than_5
# => [0,1,2,3,4]
# >> the_rest
# => [5,6,7,8,9,10]

When needed the partition method can be very convenient

The apple way

In Leopard Apple introduced launchd which is their new system startup program. Launchd is a deamon which is a manager and scheduler. In addition to eliminating the need for crontab, which is traditionally how you would run scheduled jobs on unix systems, launchd offers you a whole slew of other events you can react to programatically. Events such as mounts, file creation and modifications. The example below demonstrates creating files in a watched folder. To do this you’ll need to create a property list file with a .plist suffix and place it in the Library/LaunchAgents/ folder for the user you can also have it work system wide for all users. Name the file com.foobar.watchmyfolder.plist re-login to your computer and you should be able to test this by placing a file in the folder $HOME/dropfolder and you should see that it in return touched a file did_it_work file in your home directory.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" 
"http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>com.foobar.watchmyfolder</string>
    <key>ProgramArguments</key>
    <array>
        <string>touch</string>
        <string>/Users/username/did_it_work</string>
    </array>
    <key>QueueDirectories</key>
    <array>
        <string>/Users/username/watchfolder</string>
    </array>
    <key>StartOnMount</key>
    <false/>
    <key>WatchPaths</key>
    <array/>
</dict>
</plist>

Its kind of a bummer that the format for the .plist files is xml since other much nicer formats could have been used such as yaml or something lightweight format. But to get around that bump I installed lingon which has really open up the possibilities of launchd, it doesn’t include all the options it does include enough to get your imagination going.

• Getting Started with launchd
• Lingon

The linux way

During my research on this one it seems like the inotify library is the best option which can be used directly with the inotify-tools and also through some apis (see a c example in a link below). I have scetched a bash example below which I don’t like so much since relies on a crazy while loop to do its thing which just doesn’t seem right to me … or rather I would much prefer someone hiding that mess from me. So my hope is that there is something useful out there that works with configuration files similar to the launchd on the Mac OS X.

#!/bin/sh

while inotifywait -e create ~/watchfolder
do
  touch ~/watchfolder.log
  `date` >> ~/watchfolder.log
done

• inotify-tools
• Moniter file system activity with inotify
• Manpage for inotifywait

Now if you have a problem where you wanna delete a couple of folders in a directory it’s easy to do that in a one liner

$ rm -rf folder_1 folder_2 

but what if you want to delete all but those specific directories? Today my college showed me this amazing trick for finding everything but that

$ find . -maxdepth 1 ! -name folder_1 ! -name folder_2 

which lists all the other folders in that particular directory and then that can easily be piped to rm.

$ find . -maxdepth 1 ! -name folder_1 ! -name folder_2 | xargs rm -rf

So I like everyone else have been scrambling to save some $$$ and one expense that was hard to justify was to keep an Amazon EC2 image running for My blog, and other pet projects that didn’t get any traffic to speak one of which was Grubsnitch.com. But now that I have settled on just using my age old Dreamhost account to store these things until that becomes unreasonable. I finally managed to get this mephisto instance running on there thanks to this page and my next projecdt will be to get grubsnitch.com running there as well.

Ok, this is hardly worth putting down but it will work for me as a reminder. I have been loving the curl utility allot lately. I had a situation where I needed a to hit a page rather frequently and I really wanted to just stick it in a bash loop and just have it run. The problem is that the page was behind user/password credentials. But as it turns out curl can have it’s own cookie jar so it can sustain a session between requests. Here is how.

  curl -d "user[login]=username&user[password]=password" \\
  -c domain.cookie http://example.com/user/login

Also another trick I have been addicted to is curling a domain and only getting the response headers

curl -I http://domain.example.com

Who said I don’t like browsers?

Here is an excellent case of a big problem … with a incredible simple solution. I upgraded my slicehost ubunto vpn from dapper to hardy. A risky business with nothing real in jeopardy except for the pride of a 380 days up time on the server. I shut down everything running on the server and upgrade to hardy. All is fine except that I realize that when I am starting all the services that small things tab completion doesn’t work among other annoyances and doing work on a server without that is practically unbearable. So I figured I would make some time to fix it… the best place to go on these issues is slicehost … THE ROCK. Tony on the slicehost campfire chat immediately identified the problem and it was fixed within 5 minutes.

*Tony | try just running “bash”

And as he said that I was like shit and ran echo $SHELL only to realize that I was in “sh” but now everything is fixed and dandy and I have my tab completion back along with other normal bashy things … thank god. Did I mention that slicehost rocks!

Using AWS::S3

Now setting a object to :public_read is trivial if it’s done when you store it to S3. That doesn’t seem to be the case if you want to change on an object that already has been stored privately. After banging my head against it a bit I finally “got it” and I figured I would share it to either show off or try to be helpful.

dial into s3sh session see url above for details on how to store keys in environment variables etc.

once in there get a bucket object

• my_bucket = Bucket.find(‘foo’)

now get the s3obj that you want to become public

• s3obj = my_bucket[‘object_name’]
• policy = s3obj.acl
• grant = ACL::Grant.new
• grant.permission = ‘READ’
• grantee = ACL::Grantee.new
• grantee.group = ‘AllUsers’
• grant.grantee = grantee
• policy.grants << grant
• s3obj.acl(policy)

The access control on the buckets must be very rich since it’s this tricky make things public. I must be overlooking something.

A while back I decided to get fancy and start hosting this blog on ec2 cloud just to sort of get the hang of working on that kind of a system. This of course involved moving database over and the works. Everything seemed to be going fine until I started posting… things were just busted bad. I kept getting

ActiveRecord::StatementInvalid (Mysql::Error: Duplicate entry ‘0’ for key 1

errors and for the longest time I just ignored it since I was able to post by using a some of my masterpiece drafts that I wasn’t going to use any how… but then I ran out of those and I felt an urge to post but that meant I needed to tackle the problem and as it turned out it was a simple fix after I had done the research. The primary key fields had lost the AUTO_INCREMENT magic during my db move and the solution was simply to go in and ad them where a appropriate.

ALTER TABLE `contents` CHANGE `id` `id` INT NOT NULL AUTO_INCREMENT;

It’s a continues struggle to become efficient in using VIM, especially if one ventures outside of what one regularly needs to do. It’s well worth it though since it might be the last text editor that you’ll ever need to master.

I was faced with having to produce pdf with selected code snippets.

Here is the fastest approach… be warned this example is for macs but I am sure it can work in other Li/Unix environments as well.

Install CUPS PDF and setup a virtual printer as is described on that page. Make sure that it’s the default printer after you have set it up.

The command to print in vim is :ha which is a short for :hardcopy this will spool the whole file to the print que for that driver and ultimately drop a pdf in cups-pdf folder on your desktop. To print out a selection simply highlight some text before issuing :ha command. I needed the line-numers of the code samples and this can be achieved by setting printoptions.

set printoptions=number:y

Now to be a complete terminal snob you should check out pdftk