MySQL DNS woes

Today I was trying to bring up a mysql instance to begin setting up a mysql cluster and faced a strange problem. I was unable to connect to mysql/mysqladmin giving the host option both on the box where mysql is installed and from another box with just the client.

root@lucid-myndbmgr# mysqladmin version
mysqladmin  Ver 8.42 Distrib 5.1.39-ndb-7.0.9, for debian-linux-gnu on i486
Copyright 2000-2008 MySQL AB, 2008 Sun Microsystems, Inc.
This software comes with ABSOLUTELY NO WARRANTY. This is free software,
and you are welcome to modify and redistribute it under GPL license

Server version      5.1.39-ndb-7.0.9-1ubuntu7
Protocol version    10
Connection     Localhost via UNIX socket
UNIX socket     /var/run/mysqld/mysqld.sock
Uptime      24 min 57 sec
Thread: 1  Questions: 7  Slow queries: 0  Opens: 15  Flush tables: 1  Open tables: 8  Queries per second avg: 0.4   
root@lucid-myndbmgr# ifconfig eth0
eth0       Link encap:Ethernet  HWaddr 08:00:27:1d:ed:e8
       inet addr:10.0.0.2  Bcast:10.0.0.255  Mask:255.255.255.0
       inet6 addr: fe80::a00:27ff:fe1d:ede8/54 Scope:Link
       UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
       RX packets: 22 errors:0 dropped:0 overruns:0 frame:0
       TX packets: 21 errors:0 dropped:0 overruns:0 carrier:0
       ncollisions:0 txqueuelen:1000
       RX bytes:3294 (3.2 KB)  TX bytes:2136 (2.1 KB)
       Interrupt:10 Base address:0xd020

root@lucid-myndbmgr# mysqladmin ping
mysqld is alive
root@lucid-myndbmgr# hostname
lucid-myndbmgr
root@lucid-myndbmgr# grep 'bind-address' /etc/mysql/my.cnf
bind-address       = 10.0.0.2
root@lucid-myndbmgr# netstat -an | grep 3306
tcp       0       0   10.0.0.2:3306  0.0.0.0:* LISTEN
root@lucid-myndbmgr# mysqladmin -h 10.0.0.2 ping
mysqladmin: connect to server at '10.0.0.2' failed
error: 'Can't get hostname for your address'

After some furious googling and some wisdom from #mysql I figured out that mysql requires a functional DNS to work (I still haven't figured out why my DNS config is getting ignored :(..). Initial suggestions were to open up port 3306, remove bind-address and do not use skip-networking. They din't help

root@lucid-myndbmgr# iptables -A INPUT -p tcp --dport 3306 -s 0/0 -j ACCEPT
root@lucid-myndbmgr# iptables -A OUTPUT -p tcp --sport 3306 -d 0/0 -j ACCEPT
root@lucid-myndbmgr# iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination
ACCEPT    tcp --  anywhere   anywhere  tcp dpt:mysql

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
ACCEPT    tcp --  anywhere   anywhere tcp spt:mysql
root@lucid-myndbmgr# grep 'bind-address\|skip-networking' /etc/mysql/my.cnf
# Instead of skip-networking the default is now to listen only on
#bind-address       = 10.0.0.2
root@lucid-myndbmgr# mysqladmin -h 10.0.0.2 ping
mysqladmin: connect to server at '10.0.0.2' failed
error: 'Can't get hostname for your address'

So, as a workaround found skip-name-resolve parameter which conveniently skips this.

root@lucid-myndbmgr# grep 'skip-name-resolve' /etc/mysql/my.cnf
skip-name-resolve
root@lucid-myndbmgr# mysqladmin -h 10.0.0.2 ping
mysqld is alive
root@lucid-myndbmgr# 

Prevent vi from opening files for which you do not have read permission

Found an annoying thing in one of the new servers I was playing around. It generated some log files as root and permissions of 600 and others as the designated user. Now the problem is that when I open log files with 600 permission in vi it happily opens up but, shows a blank buffer with 'Permission Denied' error. After a couple of times I got really annoyed and decided to fix it.

Attempt 1: ~/bin is ahead in the path to /usr/bin/vi so, wrote a simple script to check for file permissions.

#!/bin/sh
if [ -e $1 -a ! -r $1 ]; then
  echo No read permission for $1
  exit 1
fi
/usr/bin/vi $1

But, sadly that did not cut it
Attempt 2: Stuffed the above in an alias. I'll leave as an exercise for you to figure out what happens
Attempt 3: Then converted the same logic to a shell function called vi and w00t! it worked :)

vi() {
  if [ -e $1 -a ! -r $1 ]; then
    echo No read permission for $1
  else
    /usr/bin/vi $1
  fi
}

Mounting VM images with multiple partitions

Of late, I have been playing around with the eucalyptus cloud project. I have a pretty basic setup with a single cloud-controller/cluster-controller and 2 nodes (using xen on centos 5.4). Usually the simplest way to mount a vm image file is as follows (find a more detailed description here)

$ # associate image file with a loop device
$ losetup /dev/loop0 jaunty.img
$ # create a mount point
$ mkdir mnt
$ # mount it :)
$ mount -t ext3 /dev/loop0 mnt
$ # chroot to navigate easily
$ chroot mnt

Then I found the jboss CirrAS project that gives a vm image with a clustered jboss setup (both for ec2 and xen/kvm). So, out of curiosity I downloaded the images and since my eucalyptus cloud is operating in SYSTEM, I have to setup the vm instance such that it pings the CLC a few times so that the IP address gets updated. When I tried the tried and tested method above...boom nothing happened I kept getting

$ mount -t ext3 /dev/loop0 mnt
mount: wrong fs type, bad option, bad superblock on /dev/loop0,
       missing codepage or helper program, or other error
       In some cases useful info is found in syslog - try
       dmesg | tail  or so

So, after some googling and some very helpful hints from #stormgrind turns out the CirrAS images have 2 partitions and if there are multiple partitions the loop device must be given an offset from which the actual image starts and the partition table ends :|. Thankfully mgoldmann in the irc channel suggested some sites that worked like a charm :)

$ # associate image file with a loop device with offset
$ losetup -o 32256 /dev/loop0 back-end-sda.raw
$ # create a mount point
$ mkdir mnt
$ # mount it :)
$ mount -t ext3 /dev/loop0 mnt
$ # w00t

How did I arrive at 32256? Simple. For images created by bximage you must use the value 32256 words of wisdom from here References http://bochs.sourceforge.net/doc/docbook/user/loop-device-usage.html http://varghese85-cs.blogspot.com/2008/11/mouting-partitions-with-losetup.html

iptables-fu

I've had a love hate relationship with iptables. I love the control it offers but, am quite confounded by all the low-level networking concepts at play. Today, I had to do some simple stuff, like blocking pings and block ssh from a particular IP. Here is my humble attempt at both

# reject all ping requests
iptables -A INPUT -p icmp -j REJECT
# drop all ssh (tcp:22) requests from 10.0.0.37
# DROP means that whoever is trying to connect from 37 will not get a connection refused...devious >(
iptables -A INPUT -p tcp --sport 22 -s 10.0.0.37 -j DROP

Effective find

Having been a long time user of find in unix, though I only recently stumbled into some of its more finer usages. Once example would be the usage of the -exec argument. I've spent countless hours piping find to a file, doing a global replace to execute the desired command and running it as a shell script; that I was simply thrilled to find that I could do it all in one single step :) (I only wish I had a little more patience reading the man pages that I would have stumbled into it sooner). Take for example removing those pesky ws_ftp.log files that litter your directories when someone decides to use ws_ftp to ftp files instead of filezilla. Here is how you can get rid them in a single command

find . -type f -name 'WS_FTP.log' -exec rm -f {} \;

Another cool trick to get rid of the Permission denied messages that otherwise fill up the screen when doing global searches is to redirect stderr to /dev/null.

find / -type f -name 'serverindex.xml' -exec grep -l '8100' {} \; 2>/dev/null

One quick note about using cygwin the \; that should terminate the exec portion of the find should actually be \ ; in cygwin or according to the wikipedia even a plain ; will do

The unix paste command

One relatively unknown unix command is paste which I had occasion to use recently. I had to twiddle some data in between 2 fixed length files and the machine did not have vim so that I could use visual block selection and too big to quickly scp to my workstation. With some help from google I discovered paste, which in combination with cut reduced the task to something as simple as this

cut -c 1-32 file1.txt > left
cut -c 48-  file1.txt > right
cut -c 64-108 file2.txt > middle
paste -d '\0' left middle right # do not use the default delimiter instead use null

Finally I managed to refine the entire process into a series of cut and pastes in sequence.

Misguided documentation

Documentation is one thing in software development which no one wants to do and hence gets talked about too little or, gets talked about a lot and still no one wants to do it. Even if documentation exists its a pain to read or get at the information that you want mostly because of bad abstracts and introductions. With the proliferation of software methodologies each requiring its own type of documentation its a mess (just like the code :P..). Even in a single methodology there are approach documents, HLDs, LLDs, use-case documents etc. most of which are sometimes not very relevant but, since the the enterprise processes demand it they are still produced and maintained :O.

I had to pick up the implementation of a small (er. tiny actually) feature and was given 2 documents to start off. The first was an abridged requirement specification covering just the change that I was supposed to do and the other an impact analysis document. The requirements document was pretty straight forward (except for some vague clauses added just so that the business analysts can bargain later). The impact analysis document was quite another beast, over 20 pages long (excluding the fluff like cover sheet, intended audience, disclaimer etc.) it was a carefully copy-pasted creation that discussed the same approach that was to be implemented in about 7 use-cases. If you were looking for existing functionality, use-cases impacted, use-cases that should not be affected, alternate flows, expected areas of conflict with previous changes etc. this was not the document to read. All the document said was in such-n'-such a screen, a field should be added the corresponding back-end code should be invoked blah blah for every use-case. I see absolutely no point in maintaining a totally worthless project artifact like this. If a change requires just a specification and LLD why not leave it at that why go through all the hoopla of creating all other worthless documents?

Google introduces updateware

I don't understand google's maniacal need to update. Yes, I understand that quick updates are a good but, the last time I searched for googleupd* in my hard drive look at how many places it showed up

%APPDATA%\Google\Update\GoogeUpdate.exe
%APPDATA%\Google\Update\1.2.141.5\GoogeUpdate.exe
%APPDATA%\Google\Update\1.2.141.5\GoogeUpdateHelper.msi
%PROGRAM_FILES%\Google\Update\GoogeUpdate.exe
%PROGRAM_FILES%\Google\Common\Google Updater\GoogeUpdaterService.exe
%PROGRAM_FILES%\Google\Update\1.2.141.5\GoogeUpdate.exe
%PROGRAM_FILES%\Google\Update\1.2.141.5\GoogeUpdateHelper.msi

And google is still not satisfied :O..there are 2 scheduled tasks as well

GoogleUpdateTaskMachine - to run at startup
GoogleUpdateTaskUserS-X-X-XX-XXXXXXXXXX-XXXXXXXXXX-XXXXXXXXX-XXXX - to run when idle

If you thought google was satisfied with that you're wrong they have the one last trick up in the registry

HKCU\Software\Microsoft\Windows\CurrentVersion\Run

After software, adware, nagware, annoyware, bloatware google has introduced updateware :/ ps. follow this excellent link on instruction to get rid of it all

Vim replace expressions

Today I discovered the the existence of evaluated expressions as replacements in vim. Consider the following snippet

String tStr = request.getParameter("frmAmt_D_DR");
if ( tStr != null && !"".equals(tStr.trim()) ) {
  xxxDDrAmt = Double.parseDouble(trStr);
}

In this I want to transform the Double.parseDouble line to

xxxDDrAmt = parseLocalizedDouble(request, "frmAmt_D_DR", locale);

The change had to be done in a lot of files and I was racking my brains trying to figure out a macro to do this. I even tried to match the entire snippet but, ran into trouble with both approaches especially because the if condition could include other conditions and there might also be more than one limits that had to be processed. Then I found some obscure comment somewhere in one of the my numerous entreaties to google that referred to evaluated expression in the replacement giving me my Eureka! moment :). Since all the variable names were always some contraction of the actual input parameter name all I needed to do was to device an expression that constructed the parameter name from the variable name. Its long and ugly but gets the job done :D

:g/Double\.parseDouble/s/\(\w\+Amt\w*\).*$/\= submatch(1) . " = parseLocalizedDouble(request, \"frm_" . toupper(strpart(submatch(1), 3, 1)) . "_" . toupper(strpart(submatch(1), 4, 2)) . (strlen(strpart(submatch(1), 9)) > 1 ? "_" . strpart(submatch(1), 9) : "") . "\", locale);"/

Running this,

xxxDDrAmt = Double.parseDouble(trStr);

becomes

xxxDDrAmt = parseLocalizedDouble(request, "frm_D_DR", locale);

and

xxxDDrAmtUSD = Double.parseDouble(trStr);

becomes

xxxDDrAmtUSD = parseLocalizedDouble(request, "frm_D_DR_USD", locale);

Here is an annotated version but don't try using this one vim expects the entire thing to be in a single line :|

:g/Double\.parseDouble/                       # global search for double parsing
s/\(\w\+Amt\w*\).*$                           # search for variable names containing Amt
/\= submatch(1)                               # submatch(1) gives the first group
  . " = parseLocalizedDouble(request, \"frm_" # . is the string concatenation operator
  . toupper(strpart(submatch(1), 3, 1)) . "_" # strpart is the substr equivalent in vim
  . toupper(strpart(submatch(1), 4, 2))
  . (strlen(strpart(submatch(1), 9)) > 1      # don't change Amts to _S :)
    ? "_" . strpart(submatch(1), 9) : "")     # vim even supports the ternary operator ^_^
  . "\", locale);"
/

Custom taglibs in Websphere

I spent the better part of an hour trying to figure out why

var v = "<mylib:my-tag attr="<%= request.getParameter("some_attr") %>" />";

wasn't compiling in WAS6 and was complaining that attribute value is not given for some_attr. But, this

<% String someAttr = request.getParameter("some_attr"); %>
var v = "<mylib:my-tag attr="<%= someAttr %>" />";

works just fine. Turns out if you need JSP evaluation inside you custom taglibs attribute you need to give inside single quotes instead of double quotes. Something about the twin double quotes trips up the WAS6 JSP compiler. Now, since I had to do this in a lot of places I simply opted for the simplest solution, use single quotes :)

var v = "<mylib:my-tag attr='<%= request.getParameter("some_attr") %>' />";

Works like a charm !!

JDBC debugging

Recently I had to muck through a whole bunch of code to fix a connection pooling issue because of which some partial results were getting committed into the database. While the actual problem turned out to be a commit getting issued deep in the custom framework code while querying for the nextval in an oracle sequence. As a by product of that investigation I wrote the following proxies to see which queries where executed by which connection. Usage is pretty simple. Just add the ConnectionInvocationHandler.createProxy(<your raw connection>) where ever you are getting the actual connection. The following classes only handle the most frequently occurring cases for me (for oracle).

package org.foo.sql;

import java.lang.reflect.InvocationHandler;
import java.lang.reflect.Method;
import java.lang.reflect.Proxy;
import java.sql.CallableStatement;
import java.sql.Connection;
import java.sql.PreparedStatement;
import java.sql.Statement;

public class ConnectionInvocationHandler implements InvocationHandler {
  
  Connection delegate = null;
  
  public static Connection createProxy(Connection conn) {
    return (Connection) Proxy.newProxyInstance(ConnectionInvocationHandler.class.getClassLoader(), new Class[]{ Connection.class }, new ConnectionInvocationHandler(conn));
  }
  
  ConnectionInvocationHandler(Connection original) {
    delegate = original;
  }

  public Object invoke(Object proxy, Method method, Object[] args)
      throws Throwable {
    Object result = method.invoke(delegate, args);
    String methodName = method.getName();
    if ( "prepareStatement".equals(methodName) ) {
      result = PreparedStatementInvocationHandler.createProxy((PreparedStatement) result, delegate.toString(), (String) args[0]);
    } else if ( "createStatement".equals(methodName) ) {
      result = StatementInvocationHandler.createProxy((Statement) result, delegate.toString());
    } else if ( "setAutoCommit".equals(methodName) ) {
      log("setAutoCommit(" + args[0] + ")");
    } else if ( "commit".equals(methodName) ) {
      log("commit()");
    } else if ( "rollback".equals(methodName) ) {
      log("rollback()");
    }
    return result;
  }
  
  private void log(String msg) {
    System.out.println("[" + delegate.toString() + "] " + msg);
  }
  
}

package org.foo.sql;

import java.lang.reflect.InvocationHandler;
import java.lang.reflect.Method;
import java.lang.reflect.Proxy;
import java.sql.Statement;

public class StatementInvocationHandler implements InvocationHandler {

 String connId = null;
 Statement delegate = null;
  
  static Statement createProxy(Statement stmt, String connId) {
    return (Statement) Proxy.newProxyInstance(StatementInvocationHandler.class.getClassLoader(), new Class[]{ Statement.class }, new StatementInvocationHandler(stmt, connId));
  }
  
  StatementInvocationHandler(Statement stmt, String connId) {
    delegate = stmt;
    this.connId = connId;
  }
  
  public Object invoke(Object proxy, Method method, Object[] args)
      throws Throwable {
    Object result = method.invoke(delegate, args);
    String methodName = method.getName();
    if ( "executeQuery".equals(methodName) ) {
      log((String) args[0]);
    } else if ( "executeUpdate".equals(methodName) ) {
      log((String) args[0]);
    }
    return result;
  }
  
  private void log(String msg) {
    System.out.println("[" + connId + "][" + delegate.toString() + "] " + msg);
  }
}

package org.foo.sql;

import java.lang.reflect.InvocationHandler;
import java.lang.reflect.Method;
import java.lang.reflect.Proxy;
import java.sql.PreparedStatement;
import java.text.DecimalFormat;

public class PreparedStatementInvocationHandler implements InvocationHandler {
  
  String connId = null;
  StringBuilder query = null;
  PreparedStatement delegate = null;
  final DecimalFormat df = new DecimalFormat("0.#");
  
  static PreparedStatement createProxy(PreparedStatement pstmt, String connId, String query) {
    return (PreparedStatement) Proxy.newProxyInstance(PreparedStatementInvocationHandler.class.getClassLoader(), new Class[]{ PreparedStatement.class }, new PreparedStatementInvocationHandler(pstmt, connId, query));
  }
  
  PreparedStatementInvocationHandler(PreparedStatement original, String connId, String query) {
    delegate = original;
    this.connId = connId;
    this.query = new StringBuilder(query);
  }

  public Object invoke(Object proxy, Method method, Object[] args)
      throws Throwable {
    Object result = method.invoke(delegate, args);
    String methodName = method.getName();
    if ( "setInt".equals(methodName) ) {
      addParam(args[1].toString());
    } else if ( "setLong".equals(methodName) ) {
      addParam(args[1].toString());
    } else if ( "setDouble".equals(methodName) ) {
      addParam(df.format(((Double) args[1]).doubleValue()));
    } else if ( "setTimestamp".equals(methodName) ) {
      String dParam = args[1].toString();
      dParam = dParam.substring(dParam.indexOf('.')); // remove the ms
      addParam("to_date('" + dParam + "', 'yyyy-mm-dd hh24:mi:ss')");
    } else if ( "setString".equals(methodName) ) {
      addParam("'" + args[1].toString() + "'");
    } else if ( methodName.startsWith("set") ) {
      addParam("'<" + methodName.substring(3) + ">'");
    } else if ( "executeUpdate".equals(methodName) ) {
      log(query.toString());
    } else if ( "executeQuery".equals(methodName) ) {
      log(query.toString());
    }
    return result;
  }
  
  protected void addParam(String param) {
    int idx = query.indexOf("?");
    if ( idx > -1 ) {
      query.replace(idx, idx + 1, param);
    }
  }
  
  protected void log(String msg) {
    System.out.println("[" + connId + "][" + delegate.toString() + "] " + msg);
  }
}

vi vi vi is the editor of the beast

I'm never satisfied with the text-editors I've been using so far. But, now I may have found _the one_. I started off with the humble MS-DOS edit, moved on to the GUI notepad then tried some serious text editors like EditPad, EditPlus, TextPad, SciteFlash, jEdit, Notepad++ and lots of others. There is always some feature that I really like accompanied with some equally irritating annoyance. Like for e.g. I love the beanshell find/replace in jEdit but, I had to give it up as it would choke for any file a little over 900KB. Editplus was great till I had to move to linux and Editpad was just too clunky.

I've used vi on and off mostly to quickly edit some file on the server to save the hassle of ftping the file back and forth. I never enjoyed these sessions much because I wasn't yet initiated to the magic of vi. Then one day I decided that all this brouhaha about vi and emacs must have some substance to it and hence resolved to use one of them as my primary editor henceforth. Its a tall order to master each of these editors but, since I was already familiar with vi and couldn't bear the ctrl+meta sequences of emacs for too long. For over 2 months now I've uninstalled every other text editor and use only gVim.

Its great. Everything you ever wanted to do with a piece of text can be done in vi. There are a wealth of cheatsheets and tutorials. Here are some of them.

Best of VIM Tips, gVIM's Key Features zzapper
A slightly advanced Introduction to Vim LG #152
Vi Reference Card
100 Vim commands every programmer should know

vi may be the editor of the beast because once you get to know it you start seeing its inner beauty :)

Fun with awk

Recently I was wrangling with a few log files and sed and grep din't cut it and I had to use awk. I've always dreaded the use of awk as it was a programming language onto itself and hence I din't want to burden myself with learning another language that I rarely use. But, my latest interaction with awk left me satisfied and intrigued. Its almost like perl except that it follows a strict structure for just manipulating text and perl as I've learnt the hard way can be used to do anything (its been described as the duct tape that holds the internet together!!)

Java Encryption

I was working on a project that required some encryption. So, I had some time to play around with the javax.crypto api. Here is a small utility which plays around with text files before encryption.

package org.foo.bar; 
import java.io.BufferedReader;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.FileFilter;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.OutputStream;
import java.io.StringWriter;
import java.security.SecureRandom;
import java.util.ArrayList;
import java.util.Collections;
import java.util.HashSet;

import javax.crypto.Cipher;
import javax.crypto.CipherOutputStream;
import javax.crypto.KeyGenerator;
import javax.crypto.SecretKey;


public class FcUtil {
  private static final String UTF8 = "utf-8";
  private static SecretKey getKey(String alg, String pass) {
    try {
      KeyGenerator keyGen = KeyGenerator.getInstance("Rijndael");
      keyGen.init(new SecureRandom(pass.getBytes(UTF8)));
      return keyGen.generateKey();
    } catch (Exception e) {
      throw new RuntimeException(e);
    }
  }
  private static Cipher getCipher(String alg, String pass) {
    try {
      Cipher cipher = Cipher.getInstance(alg);
      cipher.init(Cipher.ENCRYPT_MODE, getKey(alg, pass));
      return cipher;
    } catch (Exception e) {
      throw new RuntimeException(e);
    }
  }
  private static void encrypt(Cipher cipher, InputStream is, OutputStream os) {
    try {
      byte[] buffer = new byte[1024 * 1024];
      CipherOutputStream cos = new CipherOutputStream(os, cipher);
      int read = -1;
      int count = 10;
      while ( (read = is.read(buffer) ) > 0 ) {
        cos.write(buffer, 0, read);
        if ( --count < 0 ) {
          count = 10;
          cos.flush();
        }
      } // endwhile
      is.close();
      cos.close();
    } catch (Exception e) {
      throw new RuntimeException(e);
    }
  }
  private static String rev(String str) {
    char[] chars = str.toCharArray();
    int l = chars.length, n = l >> 1;
    char c;
    for (int i = 0; i < n; i++ ) {
      c = chars[(b = l-i)];
      chars[b] = chars[i];
      chars[i] = c;
    }
    return new String(chars);
  }
  private static byte[] processText(File file) {
    try {
      BufferedReader br = new BufferedReader(
          new InputStreamReader(new FileInputStream(file), UTF8));
      ArrayList<String> list = new ArrayList<String>();
      StringBuilder b = new StringBuilder();
      String buf = null;
      while ( (buf = br.readLine()) != null ) {
        b.delete(0, b.length());
        list.add(b.insert(0, buf).reverse().toString());
      }
      Collections.reverse(list);
      br.close();
      int count = 10;
      StringWriter sw = new StringWriter();
      for (String s : list) {
        sw.write(s);
        sw.write('\n');
        if ( --count < 0 ) {
          count = 10;
          sw.flush();
        }
      }
      sw.close();
      return sw.getBuffer().toString().getBytes(UTF8);
    } catch (Exception e) {
      throw new RuntimeException(e);
    }
  }
  private static ByteArrayInputStream processText(InputStream is) {
    try {
      BufferedReader br = new BufferedReader(
          new InputStreamReader(is, UTF8));
      ArrayList<String> list = new ArrayList<String>();
      StringBuilder b = new StringBuilder();
      String buf = null;
      while ( (buf = br.readLine()) != null ) {
        b.delete(0, b.length());
        list.add(b.insert(0, buf).reverse().toString());
      }
      Collections.reverse(list);
      br.close();
      int count = 10;
      StringWriter sw = new StringWriter();
      for (String s : list) {
        sw.write(s);
        sw.write('\n');
        if ( --count < 0 ) {
          count = 10;
          sw.flush();
        }
      }
      sw.close();
      return new ByteArrayInputStream(sw.getBuffer().toString().getBytes(UTF8));
    } catch (Exception e) {
      throw new RuntimeException(e);
    }
  }
  private static ByteArrayInputStream readFully(File file) {
    try {
      FileInputStream fis = new FileInputStream(file);
      ByteArrayOutputStream baos = new ByteArrayOutputStream();
      byte[] buffer = new byte[1024 * 1024];
      int read = -1;
      while ( (read = fis.read(buffer)) > 0 ) {
        baos.write(buffer, 0, read);
      }
      fis.close();
      baos.close();
      return new ByteArrayInputStream(baos.toByteArray());
    } catch (Exception e) {
      throw new RuntimeException(e);
    }
  }
  private static void transferFully(InputStream is, OutputStream os) {
    try {
      byte[] buffer = new byte[1024 * 1024];
      int read = -1;
      int count = 10;
      while ( (read = is.read(buffer)) > 0 ) {
        os.write(buffer, 0, read);
        if ( --count < 0 ) {
          count = 10;
          os.flush();
        }
      }
      is.close();
      os.close();
    } catch (Exception e) {
      throw new RuntimeException(e);
    }
  }
  private static void process(Cipher cipher, File file) {
    if ( file.isDirectory() ) {
      return;
    }
    HashSet<String> textExt = new HashSet<String>() {{
      add("txt");
      add("java");
      add("properties");
      add("xml");
      add("bat");
      add("sh");
      add("jsp");
      add("html");
      add("tld");
      add("js");
      add("css");
      add("dtd");
      add("xsd");
      add("sql");
      add("ddl");
    }};
    try {
      String ext = file.getName().substring(file.getName().lastIndexOf('.')+1).toLowerCase();
      if ( textExt.contains(ext) ) {

        // encrypt
        ByteArrayInputStream bis = new ByteArrayInputStream(processText(file));
        encrypt(cipher, bis, new FileOutputStream(file));

        // decrypt
//        ByteArrayOutputStream baos = new ByteArrayOutputStream();
//        encrypt(cipher, readFully(file), baos);
//        transferFully(processText(new ByteArrayInputStream(
//            baos.toByteArray())), new FileOutputStream(file));
      } else {
        // encrypt & decrypt
        ByteArrayInputStream bis = readFully(file);
        encrypt(cipher, bis, new FileOutputStream(file));
      }
    } catch (Exception e) {
      throw new RuntimeException(e);
    }    
  }
  private static void handleDir(Cipher cipher, File dir) {
    long start = System.currentTimeMillis();
    FileFilter fileFilter = new FileFilter() {
      HashSet<String> encExt = new HashSet<String>() {{
        add("txt");
        add("java");
        add("properties");
        add("xml");
        add("bat");
        add("sh");
        add("jsp");
        add("html");
        add("tld");
        add("js");
        add("css");
        add("dtd");
        add("xsd");
        add("sql");
        add("ddl");
        
        add("doc");
        add("xls");
        add("ppt");
        add("pps");
        add("pdf");
        add("pst");
      }};
      public boolean accept(File pathname) {
        String ext = pathname.getName().substring(pathname.getName().lastIndexOf('.')+1).toLowerCase();
        return pathname.isFile() && encExt.contains(ext);
      }
    };
    FileFilter dirFilter = new FileFilter() {
      HashSet<String> exclude = new HashSet<String>() {{
        add(".svn");
        add("cvs");
        add(".hg");
      }};
      public boolean accept(File pathname) {
        return pathname.isDirectory() && !exclude.contains(pathname.getName().toLowerCase()) && pathname.getName().indexOf('.') != 0;
      }
      
    };
    File[] files = dir.listFiles(fileFilter);
    for (File file : files) {
//      System.out.println(file.getAbsolutePath());
      process(cipher, file);
    }
    File[] dirs = dir.listFiles(dirFilter);
    handleDirs(cipher, dirs);
//    System.out.println(dir.getAbsolutePath() + "[" + (System.currentTimeMillis() - start) + "ms]");
  }
  private static void handleDirs(Cipher cipher, File[] dirs) {
    for (File dir : dirs) {
      handleDir(cipher, dir);
    }
  }
  public static void main(String[] args) {
    final String alg = "Rijndael";
    if ( args.length != 2 ) {
      System.out.println("error");
      return;
    }
    final String key = args[0];
    
    try {
      Cipher cipher = getCipher(alg, key);
//      cipher.init(Cipher.DECRYPT_MODE, getKey(alg, key));
      final File file = new File(args[1]);
      handleDir(cipher, file);
    } catch (Exception e) {
      e.printStackTrace();
    }
  }
}

Slashdot Userscript

The funny quote in the footer of slashdot has always held my attention...here's a userscript to make it the title of the window

// ==UserScript==
// @name          Slashdot title quote
// @include       http://www.slashdot.org
// @include       http://www.slashdot.org/*
// @include       http://*.slashdot.org
// @include       http://slashdot.org/*
// @author        fc
// @namespace     http://fc-unleashed.blogspot.com
// @description   the footer quote in the title
// ==/UserScript==
(function() {
   if ( document.getElementById ) {
       var footer = document.getElementById('footer');
       if ( footer && footer.getElementsByTagName ) {
           var small = footer.getElementsByTagName('small');
           if ( small && small[0] ) {
               document.title = small[0].firstChild.nodeValue;
           }
       }
   }
})();

PERL update

After almost 4 months of PERL this is what I have to say.

• Its extremely handy for short and snappy stuff


• Everything under the sun is available in CPAN but figuring out which is best suited for your needs is a bitch


• While developing webapps with CGI not knowing use Carp::qw(fatalsToBrowser); is the last thing you want to do (coz I did that)


• Why ? Oh why do warnings and use strict; exist if you can write stuff faster without it :((

addEvent()

A lots been said and done here. Here's my take if attachEvent does not give you access to the "this" pointer don't use it, instead use the age old method supported by most browsers :)

<html>
  <head>
    <title>Javascript Test Page</title>
<script type="text/javascript">
/*****************************************************************************
 *                          Helper Functions
 *****************************************************************************/
var onlyUnique = true; // default
function registerListener(target, event, listener) {
 if ( target[event + "Count"] ) {
  target[event + "Count"] = parseInt(target[event + "Count"]) + 1;
 } else {
  // first handler so add default event handler
  target[event + "Count"] = 1;
  target.defaultHandler = defaultHandler;
  target["on" + event] = function(evt) {
   evt = window.event ? window.event : evt;
   target.defaultHandler(event, evt);
  };
 }
 var currCount = parseInt(target[event + "Count"]) - 1;
 if ( onlyUnique ) {
  for ( var i = 0; i < currCount; i++ ) {
   if ( target[event + i] == listener ) {
    target[event + "Count"] = currCount;
    return false;
   }
  } // endfor i
 } // endif onlyUnique
 target[event + currCount] = listener;
 return true;
}
function unRegisterListener(target, event, listener) {
 if ( target[event + "Count"] ) {
  var currCount = parseInt(target[event + "Count"]);
  for ( var i = 0; i < currCount; i++ ) {
   if ( target[event + i] == listener ) {
    target[event + i] = null;
    currCount--; // removed one
    break;
   }
  } // endfor i
  for ( var j = i; j < currCount; j++ ) {
   target[event + j] = target[event + (j + 1)];
  } //  endfor j
  if ( currCount < 1 ) {
   // cleanup default event handler
   target["on" + event] = null;
  }
  target[event + "Count"] = currCount;
 }
}
function defaultHandler(eventName, evt) {
 if ( this[eventName + "Count"] ) {
  var currCount = parseInt(this[eventName + "Count"]);
  for ( var i = 0; i < currCount; i++ ) {
   this[eventName + i](evt);
  } // endfor i
 }
}
/*****************************************************************************
 *                  Event Handler Functions
 *****************************************************************************/
function addEvent(target, event, func) {
 if ( target.addEventListener ) {
  target.addEventListener(event, func, false);
 } else {
  registerListener(target, event, func)
 }
}
function removeEvent(target, event, func) {
 if ( target.removeEventListener ) {
  target.removeEventListener(event, func, false);
 } else {
  unRegisterListener(target, event, func)
 }
}
/*****************************************************************************/
function gel(idName) {
 return document.getElementById ? document.getElementById(idName) : false;
}
addEvent(window, "load", one);
addEvent(window, "load", two);
addEvent(window, "load", initialize);
var oneCount = 0;
function one() {
 var text = "[one][" + oneCount + "]" + this.id;
 oneCount++;
 var div = document.createElement("div");
 div.appendChild(document.createTextNode(text));
 gel("oneDiv").appendChild(div);
}
var twoCount = 0;
function two(evt) {
 var text = "[two][" + twoCount + "]" + evt.type;
 twoCount++;
 var div = document.createElement("div");
 div.appendChild(document.createTextNode(text));
 gel("twoDiv").appendChild(div);
}
function initialize() {
 alert("[initialize]");
 alert("adding one");
 add1();
 alert("adding two");
 add2();
 alert("done");
 addEvent(gel("remove1"), "click", remove1);
 addEvent(gel("remove2"), "click", remove2);
 addEvent(gel("add1"), "click", add1);
 addEvent(gel("add2"), "click", add2);
}
function remove1() {
 for ( var i = 0; i < 10; i++ ) {
  remove(one);
 }
 alert(gel("anchor").clickCount + " handler(s) left");
}
function remove2() {
 for ( var i = 0; i < 10; i++ ) {
  remove(two);
 }
 alert(gel("anchor").clickCount + " handler(s) left");
}
function remove(which) {
 removeEvent(gel("anchor"), "click", which);
}
function add1() {
 for ( var i = 0; i < 10; i++ ) {
  add(one);
 }
 alert(gel("anchor").clickCount + " handler(s) present");
}
function add2() {
 for ( var i = 0; i < 10; i++ ) {
  add(two);
 }
 alert(gel("anchor").clickCount + " handler(s) present");
}
function add(which) {
 addEvent(gel("anchor"), "click", which);
}
</script>
  </head>
  <body>
    <a id="anchor" href="#">[Test]</a>
    <a id="remove1" href="#">[Remove 1]</a>
    <a id="remove2" href="#">[Remove 2]</a>
    <a id="add1" href="#">[Add 1]</a>
    <a id="add2" href="#">[Add 2]</a>
    <div id="test"></div>
    <table>
      <tr>
        <td valign="top">
          <div id="oneDiv"></div>
        </td>
        <td valign="top">
          <div id="twoDiv"></div>
        </td>
      </tr>
    </table>
  </body>
</html>

PERL

Have been messing around a lot with PERL recently. Got involved in a project where only PERL can be used due to hosting environment restrictions enforced by the client. PERL seems a lot like shell scripting without the hassle of strict spacing considerations. Adding $/@/% before each variable occassionaly bugs me though it has its advantages. Hoping to become a professional perl programmer soon :)

Dynamic Select

Sometime ago in QuirksBlog came across this post which lead to the this link which provides an elegant solution to dynamic select boxes. This set my refactoring mind rolling :) and I figured instead of using an existing attribute why not use the name/id of the controlling select-box !! That done my lazy mind decided one was not enough and so I decided my dynamic select script must control multiple select-boxes. The HTML form for such a thing looks somewhat like this

<form name="testForm">

  <select name="regionId">
    <option value="_select_"> -  Select - </option>
    <option value="1">England</option>
    <option value="2">France</option>
  </select>

  <select name="businessId">
    <option value="_select_"> -  Select - </option>
    <option regionId="1" value="1">Cards</option>
    <option regionId="1" value="2">Loans</option>
    <option regionId="2" value="2">Loans</option>
  </select>

  <select name="productId">
    <option value="_select_"> -  Select - </option>
    <option regionId="1" businessId="1" value="1">GoldCard</option>
    <option regionId="1" businessId="2" value="4">HomeLoan</option>
    <option regionId="2" businessId="2" value="4">HomeLoan</option>
    <option regionId="1" businessId="2" value="2">PersonalLoan</option>
    <option regionId="2" businessId="2" value="2">PersonalLoan</option>
  </select>

</form>

The javascript for achieving this is below

function addID(tags) {
  if ( tags && document.getElementsByTagName ) {
    for ( i = 0; i < tags.length; i++ ) {
      var elements = document.getElementsByTagName(tags[i]);
      for ( j = 0; j < elements.length; j++ ) {
        var elementName = elements[j].getAttribute("name");
        if ( elementName ) {
          elements[j].setAttribute("id", elementName);
        }
      } // enfor j
    } // endfor i
  } // endif
}
function gel(eName) {
  if ( document.getElementById ) {
    return document.getElementById(eName);
  }
  return null;
}
function addHandler(target, eventName, eventHandler) {
  if ( target ) {
    if ( target.addEventListener ) {
      target.addEventListener(eventName, eventHandler, true);
      return true;
    } else if ( target.attachEvent ) {
      target.attachEvent("on" + eventName, eventHandler);
      return true;
    }
  }
  return false;
}
function addChainedSelect(sourceSelect, targetSelects) {
  var numTargets = targetSelects.length;
  var originalTargets = new Array(numTargets);
  for ( i = 0; i < targetSelects.length; i++ ) {
    if ( targetSelects[i] && targetSelects[i].cloneNode ) {
      originalTargets[i] = targetSelects[i].cloneNode(true);
      if ( targetSelects[i].getAttribute("chainedSelect") ) {
        var chainedSelects = targetSelects[i].getAttribute("chainedSelect");
        chainedSelects += "," + sourceSelect.name;
        targetSelects[i].setAttribute("chainedSelect", chainedSelects);
      } else {
        targetSelects[i].setAttribute("chainedSelect", sourceSelect.name);
      }
    }
  }
  
  addHandler(sourceSelect, "change", function() {
    refreshSelect(sourceSelect, targetSelects, originalTargets);
  });
}
function refreshSelect(sourceSelect, targetSelects, originalTargets) {
  var curValue = sourceSelect.value;
  var srcName = sourceSelect.getAttribute("name");
  for ( i = 0; i < targetSelects.length; i++ ) {
    clearSelect(targetSelects[i]);
    var option = originalTargets[i].getElementsByTagName("option");
    var regEx = new RegExp(curValue);
    for ( j = 0; j < option.length; j++ ) {
      if ( /_select_/.test(option[j].value) || regEx.test(option[j].getAttribute(srcName)) ) {
        targetSelects[i].appendChild(option[j].cloneNode(true));
      }
    } // endfor j
    if ( targetSelects[i].getAttribute("chainedSelect") ) {
      var chainedSelects = targetSelects[i].getAttribute("chainedSelect").split(",");
      for ( j = 0; j < chainedSelects.length; j++ ) {
        var chainName = chainedSelects[j];
        var chainValue = document.getElementById(chainName).value;
        var regEx = new RegExp(chainValue);
        var option = targetSelects[i].getElementsByTagName("option");
        for ( k = 0; k < option.length; k++ ) {
          if ( !/_select_/.test(option[k].value) && !regEx.test(option[k].getAttribute(chainName)) ) {
            targetSelects[i].removeChild(option[k]);
          }
        } // endfor k
      } // endfor j
    } // endif additional chainedSelects
  } // endfor i
}
function clearSelect(selectBox) {
  if ( selectBox ) {
    var option = selectBox.getElementsByTagName("option");
    while ( option[0] ) {
      selectBox.removeChild(option[0]);
    }
  }
}

And the onload calls to initate it all :)

window.onload = function() {
  addID(["input", "select", "div", "a"]);
  if ( document.getElementById ) {
    addChainedSelect(gel("regionId"), [gel("businessId"), gel("productId")]);
    addChainedSelect(gel("businessId"), [gel("productId")]);
  } // endif gEBI
} // end winload

Have tested it out only in Firefox and that too not extensively so watchout for those bugs ;)

Greasemonkey

I recently discovered Greasemonkey which is a really cool idea. Imagine you have the power to pick and choose the content that a website is throwing at you. If you have used the Web Developer you will know a little of what I'm talking about. What basically "Web Developer" allows you to do is control the various aspects of the page that is currently being displayed. For example you can turn off all the images, view response headers and a lot of cool stuff. But, "Web Developer" is limited that you have to manually go use its toolbar.(Like for example every time I have to login into Yahoo Mail I'll go to the "Web Developer" toolbar "Forms->Enable Auto Completion" so that I don't have to type in my email id because Yahoo disables autocompletion on the sign-in form) Now with a "Greasemonkey" user-script which will automatically run whenever I go to Yahoo Mail I'm saved the arduos task of typing in my emailid :). I got so excited that I wrote a few in myself :). Here they are...

// ==UserScript==
// @name          Yahoo Autocomplete
// @include       http://mail.yahoo.com/*
// @include   http://mail.yahoo.com
// @exclude       
// @author        fc
// @namespace     http://fc-unleashed.blogspot.com/
// @description   Enable autocomplete in Yahoo's login page
// ==/UserScript==

(function() {
 var login = document.login_form;
 login.setAttribute("autocomplete", "on");
})()

All you've got to do is put this into a file called YahooMail.user.js. You need the .user.js for Greaskmonkey to know that its our special user-script. Then write up a simple HTML page with a hyperlink to YahooMail.user.js (like <a href="YahooMail.user.js">Yahoo Mail</a>). Now, all you have to do is open up this HTML file in Firefox and right-click on the link (if you have installed Greasemonkey) the first item is "Install User Script...", select that and you are done. Now visit Yahoo Mail so that you'll never again have to type out your yahoo id :). Here's another that tries to disable all the targets and javascript popups :)

// ==UserScript==
// @name          Disable targets and javascript popups
// @include       *
// @exclude       
// @author        fc
// @namespace     http://fc-unleashed.blogspot.com/
// @description   General disable targets and javascript popups
// ==/UserScript==

(function() {
 if ( document.getElementsByTagName ) {
  var anchors = document.getElementsByTagName("a");
  for ( i = 0; i < anchors.length; i++ ) {
   var targetVal = anchors[i].getAttribute("target");
   var hrefVal = anchors[i].getAttribute("href");
   if ( targetVal )  {
    if ( targetVal.match(/^_[bB][lL][aA][nN][kK]$/) || targetVal.match(/^_[nN][eE][wW]$/) ) {
     out = anchors[i].removeAttribute("target");
    }
   }
   if ( hrefVal ) {
    var myRegex = new RegExp("^javascript:[^(]+\\(['\"]([hH][tT][tT][pP]://[^'\"]+)['\"]\\).*");
    if ( hrefVal.match(myRegex) ) {
     var newVal = hrefVal.replace(myRegex, "$1");
     anchors[i].setAttribute("href", newVal);
    }
   }
  } // endfor i
 }
})()