allskonar

# pimping loggin a bit 
LOG = Logger.new('logs/application.log')
%w(debug info warn error fatal).each do |level|
  eval <<-end_eval
    def LOG.#{level}(message="", &blk)
      if message == "" and block_given?
        message = blk.call
      end 
      file,line_number,method_name = caller.first.split(":")
      method_name = "" unless method_name
      filename = File.basename(file)
      super(filename + ":" + line_number) { method_name + " " + message }
    end 
  end_eval
end

I had to brag, it’s rare that I dabble in magic code i.e. code that is too clever to be understood right away and impossible to troubleshoot unless you wrote it yourself (recently). I usually try to keep my code quite simple. Clever has a bad rap in coding and for good reason since even the cleverest person can easily forget the lightning of cleverness that struck when he or she wrote something just a few weeks earlier. Even though clever code is bad it’s extremely satisfying when you solve something cleverly.

I am writing an application that operates mostly in the background i.e. without user interacting with it directly which makes troubleshooting tricky since one can’t easily throw in a debug statement and step through the code execution. I ofcourse have decent test coverage helps but in production code seems determinate to make once life miserable. This is where good logging becomes vital in chasing down bugs or performance bottlenecks in production code.

Logger in the standard ruby library is extremely awesome I have found that I just wanted a little more out of it namely I wanted by default a single log file with the name script file name, line number and perhaps the method name where a particular message was sent to the logger. There didn’t seem to be an automatic way of doing this so I resulted to do a little overwriting (the magic). I accomplished this by defining a singleton method on my logger instance.

I finally found an opportunity to use ruby partition method. It splits an array you have upon a criteria you supply in a block. Typically, in the old days, I would have done this by doing something like this:

big_array = Array.new(11) {|i| i}
number_lower_than_5 = []

big_array.each do |number|
  if number < 5 
    number_lower_than_5 << number
  end 
end

# >> number_lower_than_5
# => [0,1,2,3,4]

or more recently I would use select

big_array = Array.new(11) {|i| i}
number_lower_than_5 = []

number_lower_than_5 = 
big_array.select do |number|
  number < 5 
end

# >> number_lower_than_5
# => [0,1,2,3,4]

These are both fine methods but what happens if you need the other halve as well? The above approaches just collect you an array to work with and both lack the convenient array of the other stuff you sometimes care about. That’s where ruby’s partition comes in handy. It may not be that often that’s why I was so pumped up when I found an opportunity to use it.

big_array = Array.new(11) {|i| i}
number_lower_than_5, the_rest =
big_array.partition do |number|
  number < 5 
end

# >> number_lower_than_5
# => [0,1,2,3,4]
# >> the_rest
# => [5,6,7,8,9,10]

When needed the partition method can be very convenient

The apple way

In Leopard Apple introduced launchd which is their new system startup program. Launchd is a deamon which is a manager and scheduler. In addition to eliminating the need for crontab, which is traditionally how you would run scheduled jobs on unix systems, launchd offers you a whole slew of other events you can react to programatically. Events such as mounts, file creation and modifications. The example below demonstrates creating files in a watched folder. To do this you’ll need to create a property list file with a .plist suffix and place it in the Library/LaunchAgents/ folder for the user you can also have it work system wide for all users. Name the file com.foobar.watchmyfolder.plist re-login to your computer and you should be able to test this by placing a file in the folder $HOME/dropfolder and you should see that it in return touched a file did_it_work file in your home directory.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" 
"http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>com.foobar.watchmyfolder</string>
    <key>ProgramArguments</key>
    <array>
        <string>touch</string>
        <string>/Users/username/did_it_work</string>
    </array>
    <key>QueueDirectories</key>
    <array>
        <string>/Users/username/watchfolder</string>
    </array>
    <key>StartOnMount</key>
    <false/>
    <key>WatchPaths</key>
    <array/>
</dict>
</plist>

Its kind of a bummer that the format for the .plist files is xml since other much nicer formats could have been used such as yaml or something lightweight format. But to get around that bump I installed lingon which has really open up the possibilities of launchd, it doesn’t include all the options it does include enough to get your imagination going.

• Getting Started with launchd
• Lingon

The linux way

During my research on this one it seems like the inotify library is the best option which can be used directly with the inotify-tools and also through some apis (see a c example in a link below). I have scetched a bash example below which I don’t like so much since relies on a crazy while loop to do its thing which just doesn’t seem right to me … or rather I would much prefer someone hiding that mess from me. So my hope is that there is something useful out there that works with configuration files similar to the launchd on the Mac OS X.

#!/bin/sh

while inotifywait -e create ~/watchfolder
do
  touch ~/watchfolder.log
  `date` >> ~/watchfolder.log
done

• inotify-tools
• Moniter file system activity with inotify
• Manpage for inotifywait

One thing I do love is to spend 15 minutes to solve a problem that would have taken me 20 minutes to do manually … heck that’s how I got into this programming thing. Bash scripting is becoming one of my favorite tools as it’s so simple once you wrap your head around it’s weirdness … which I am hoping to do early 2010 :)

So the problem I had was that I had a bunch of photos with some stupid names which were automatically generated by my stupid scanner software, sc30rr.jpg not joking it was total rubbish so after spending roughly 30 minutes scanning photographs I had this folder filled with files with that stupid name and I just wanted to make it name_of_event_001 and have the last bit be a serialized number.

for filename in *.jpg ; \
  do n=$(( $n+1 )); \
  mv $filename name_of_event_`printf "%03d" $n`.jpg; \
done

My hope is next time I need to do this I will not have to fudge around for 15 minutes and just either remember how I did it this time or alternatively look it up on my blog :)

when ever I get an opportunity to swings this trick out I get very excited. Say you have a task you wanna do and it’s should be applied to a number of files with a common name or are all with the same extension, basically any situation where you would be doing something that seems too repetitive for you taste.

Too create our example we can actually use the same trick. Lets say you have a number of files with the same name but have different number at the back of it: test_1.txt, test_2.txt, test_3.txt. So instead of doing touch file.txt n number of times you can instead do the following.

$ touch test_{1,2,3}.txt 

Now when I was trying to figure out where I learned this from I found the website where it came from so go there for more kewl tricks.

Now if you have a problem where you wanna delete a couple of folders in a directory it’s easy to do that in a one liner

$ rm -rf folder_1 folder_2 

but what if you want to delete all but those specific directories? Today my college showed me this amazing trick for finding everything but that

$ find . -maxdepth 1 ! -name folder_1 ! -name folder_2 

which lists all the other folders in that particular directory and then that can easily be piped to rm.

$ find . -maxdepth 1 ! -name folder_1 ! -name folder_2 | xargs rm -rf

The art of loving music is learning to never be ashamed of your taste, learn to embrace and defend it to death. No matter how lame or kewl the stuff you have in your ear buds you have to pretend that you don’t care what others think of you … yes even while you get caught listening to Supertramp.

Now you can follow my latest tracks on the sidebar here to the right =>

Right underneath my adorable daughter Freyja which has been keeping my away from coding for the most part :) recently. Can’t wait to teach her to code!

So to back up my claim on not being ashamed of my music taste and attempt at getting familiar with actionscript I wrote a little package that will grap your rss feed from last.fm and dump it on your blog. This is the the kewl thing about actionscript it can snag stuff hosted else where and publish it on your website provided that you allow it in your crossdomain.xml file.

So here are some things I thought about actionscript until my college Adam showed me otherwise.

• You don’t have to have the Flash application to author actionscript.
• You don’t need flex in order to not use the Flash App.
• And compiling it is easy using the straight up mxmlc from the Flex SDK just do

 $ mxmlc MyAwesomeScript.as

and that will spit out the MyAwesomeScript.swf.

Now let’s take a look at my snippet

package {
    import flash.display.Sprite;
    import flash.events.*;
    import flash.net.*;
    import flash.external.ExternalInterface;

    public class RecentTracks extends Sprite {
        private var recentlyPlayed:XML;
        private var rssUrl:String = "http://ws.audioscrobbler.com/2.0/user/bakdyr/recenttracks.rss";

        public function RecentTracks() {
            var loader:URLLoader = new URLLoader();
                loader.addEventListener(Event.COMPLETE, completeHandler);
            var request:URLRequest = new URLRequest(rssUrl);
                loader.load(request);
        }

        private function completeHandler(event:Event):void {
            var outHtml:String;
            var loader:URLLoader = URLLoader(event.target);
            recentlyPlayed = new XML(loader.data);
            outHtml = "<ul>\n";                           
            for each (var title:XML in recentlyPlayed..item..title) {
               var artist:String = title.toString().split(" – ")[0];
               var song:String = title.toString().split(" – ")[1];
               outHtml += "<li>" + song + "<br /><em>" + artist + "</em></li>\n";                           
            }                                          
            outHtml += "</ul>\n";                           
            ExternalInterface.call('Snitchmedia.recentTracks', outHtml);
        }
    }
}

Very simple little script that dials up the rss feed to last.fm and once it’s done it will call a javascript function that will handle outputting it on the page. Now this is very basic but I think an elegant solution to handle this sort of thing as I did not need to do any heroic mephisto, wordpress (or what ever the blog engine flavor of the month is) hacking and still able to add functionality to my blog. Obviously there is a lot of polishing to be done and I think I might make this a bit generic so it can be used to call in all sorts of third party fun stuff.

Now for completeness here is the javascript bit using jQuery.

var Snitchmedia = {};
Snitchmedia.recentTracks = function(playlist) {
  $('#recent_tracks').html(playlist);
};

Ok, this is hardly worth putting down but it will work for me as a reminder. I have been loving the curl utility allot lately. I had a situation where I needed a to hit a page rather frequently and I really wanted to just stick it in a bash loop and just have it run. The problem is that the page was behind user/password credentials. But as it turns out curl can have it’s own cookie jar so it can sustain a session between requests. Here is how.

  curl -d "user[login]=username&user[password]=password" \\
  -c domain.cookie http://example.com/user/login

Also another trick I have been addicted to is curling a domain and only getting the response headers

curl -I http://domain.example.com

Who said I don’t like browsers?

When I write web apps I try my best to keep my urls clean and legible. One thing that has always troubled me is how to handle label creation of named things web users key into the app. For example if a user enters a name of a restaurant Alicanté Café the uri safe for of that (if there is no intervention) ‘Alicant%C3%A9%20Caf%C3%A9’, which is total gobblygoo and one might as well show a uid or something of that nature since at least that doesn’t physically hurt the eye. As someone who has worked on a lot of Restaurant related apps this has always been something I wanted there to be an easy solution to. This weekend I decided it was worth trying once again to see what would work best. And after reading the “Internationalization in Ruby” in The Ruby Way. Here is the super simple trick

require 'iconv'
strange_name = "Alicanté Café" 
converter = Iconv.new('ASCII//TRANSLIT', 'UTF-8')
converter.iconv(strange_name)
"Alicant'e Caf'e" 

It leaves some odd ’ and “sfor some reason but they will get stripped away with all the other potential weird stuff out (”,’,etc). Still this discovery makes me stop complaining about pretentious foreign sounding restaurant names.

Using AWS::S3

Now setting a object to :public_read is trivial if it’s done when you store it to S3. That doesn’t seem to be the case if you want to change on an object that already has been stored privately. After banging my head against it a bit I finally “got it” and I figured I would share it to either show off or try to be helpful.

dial into s3sh session see url above for details on how to store keys in environment variables etc.

once in there get a bucket object

• my_bucket = Bucket.find(‘foo’)

now get the s3obj that you want to become public

• s3obj = my_bucket[‘object_name’]
• policy = s3obj.acl
• grant = ACL::Grant.new
• grant.permission = ‘READ’
• grantee = ACL::Grantee.new
• grantee.group = ‘AllUsers’
• grant.grantee = grantee
• policy.grants << grant
• s3obj.acl(policy)

The access control on the buckets must be very rich since it’s this tricky make things public. I must be overlooking something.

A while back I decided to get fancy and start hosting this blog on ec2 cloud just to sort of get the hang of working on that kind of a system. This of course involved moving database over and the works. Everything seemed to be going fine until I started posting… things were just busted bad. I kept getting

ActiveRecord::StatementInvalid (Mysql::Error: Duplicate entry ‘0’ for key 1

errors and for the longest time I just ignored it since I was able to post by using a some of my masterpiece drafts that I wasn’t going to use any how… but then I ran out of those and I felt an urge to post but that meant I needed to tackle the problem and as it turned out it was a simple fix after I had done the research. The primary key fields had lost the AUTO_INCREMENT magic during my db move and the solution was simply to go in and ad them where a appropriate.

ALTER TABLE `contents` CHANGE `id` `id` INT NOT NULL AUTO_INCREMENT;

It’s a continues struggle to become efficient in using VIM, especially if one ventures outside of what one regularly needs to do. It’s well worth it though since it might be the last text editor that you’ll ever need to master.

I was faced with having to produce pdf with selected code snippets.

Here is the fastest approach… be warned this example is for macs but I am sure it can work in other Li/Unix environments as well.

Install CUPS PDF and setup a virtual printer as is described on that page. Make sure that it’s the default printer after you have set it up.

The command to print in vim is :ha which is a short for :hardcopy this will spool the whole file to the print que for that driver and ultimately drop a pdf in cups-pdf folder on your desktop. To print out a selection simply highlight some text before issuing :ha command. I needed the line-numers of the code samples and this can be achieved by setting printoptions.

set printoptions=number:y

Now to be a complete terminal snob you should check out pdftk

Just this week, motionbox the company I work for released it’s first open source code the Motionbox EventHandler spearheaded by Topping Bowers with contributions from the rest of us in the group. The javascript library implements event delegation techniques and is built on the prototype library. There are other such libraries for yui which this was in part inspired by.

One of the kewlest thing about the library is the ability to subscribe dom elements to events before they even show up in the dom. Don’t take my word for it it was written up in ajaxian. There you have it: Topper, me and my colleges are almost famous now. If you are using prototype go and get the source and try it out it’s totally brilliant.

I have written about event handling before: The evolution of body onload Unobtrusive Javascript which might give some context on the topic of event handling in javascript.

It’s not uncommon that databases are filled with duplicate data sets. And it’s even more common that you want to clear it out so you don’t have to bothered with duplicates. This is obviously the sort of thing you want to fix at the entrance, on the way in to the database and not collect duplicates, but this solution works well if that is too late.

To get things started let’s write up a file that we can play with. I am using MySQL and I don’t know the SQL standard well enough to say but I am sure this example can be modified to work on other database servers. I generally just keep a file open and then retrieve it on the mysql console. Download the example file.

mysql>  \. dup_tutorial.sql

CREATE TABLE peoples (
  id int(11) NOT NULL auto_increment,
  f_name varchar(128) NOT NULL default '',
  l_name varchar(128) NOT NULL default '',
  age text NOT NULL,
  updated_on timestamp NOT NULL default
       CURRENT_TIMESTAMP on update CURRENT_TIMESTAMP,
  PRIMARY KEY  (id)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;

INSERT INTO peoples (f_name, l_name, age) 
   VALUES ('baldur', 'gudbjornsson', 28);
INSERT INTO peoples (f_name, l_name, age) 
   VALUES ('baldur', 'gudbjornsson', 28);
INSERT INTO peoples (f_name, l_name, age) 
   VALUES ('baldur', 'gudbjornsson', 28);
INSERT INTO peoples (f_name, l_name, age) 
   VALUES ('baldur', 'gudbjornsson', 31);

In this sample we have 4 rows and but there are only 2 rows unique if updated_on and id fields are not taken into the account. Now the way to query this to only receive the unique set group by can be used or SELECT DISTINCT.

SELECT *
FROM peoples 
GROUP BY f_name, l_name, age

And here is the trick to collect the others to dispose of them. First we’ll join the peoples table with a unique rows from it self in a sub query to retrieve the ones that don’t match.

SELECT o_peoples.id 
FROM peoples o_peoples
LEFT OUTER JOIN 
  (SELECT id 
   FROM peoples 
   GROUP BY f_name, l_name, age) u_peoples 
ON o_peoples.id = u_peoples.id
WHERE u_peoples.id IS NULL;

Then we’ll delete those extra rows from the database. Do this at your own risk and obviously back up your database five times and have a quick recovery plan it things go south.

DELETE o_peoples 
FROM peoples o_peoples
LEFT OUTER JOIN 
  (SELECT Id FROM peoples 
   GROUP BY f_name, l_name, age) u_peoples
ON o_peoples.id = u_peoples.id
WHERE u_peoples.id IS NULL;

Once you get into the habit of using jslint it’s hard not to, it’s just so satisfying when it passes and all the lint has been cleared away. The first few times I used I went to the web application and pasted the file in there and looked at the results, fixed what was wrong and pasted it back into the original file. This is especially inconvenient if you use vim or emacs since pasting from an external environment has always been a bit of a challenge. But fear not, Douglas Crowford, best known for his work on the json spec, has an extended version that works with rhino the java javascript interpreter. The advantage of using jslint through rhino is that you can inspect your javascript files from the terminal. Installing it on a mac is trivial just follow Peters Michaux Install instructions

Here is the short version:

$> curl -O ftp://ftp.mozilla.org/pub/mozilla.org/js/rhino1_6R7.zip
$> unzip rhino1_6R7.zip
$> cp rhino1_6R7/js.jar /Library/java/Extensions/

Download the jslint file

$> curl -O http://www.jslint.com/rhino/jslint.js

And then you should be able to:

$> java org.mozilla.javascript.tools.shell.Main jslint.js your_program.js