PDFs in the linux commandline

I was recently faced with the task of scanning a lot of documents and wanted to preserve them as multi-page pdfs. Converting the raw jpegs to pdfs proved to be quite easy using ImageMagick's convert that I got from http://stackoverflow.com/questions/8955425/how-can-i-convert-a-series-of-images-to-a-pdf-from-the-command-line-on-linux. But, the file sizes of the pdfs was really large when using convert to create multi-page documents. A little bit of googling and I found pdfunite which was perfect and I was able to create reasonably sized multi-page pdfs from a set of single page pdfs :)

Naruto downloader from managreader.net

After reading Naruto upto chapter 520 (courtesy naruto with elisp) I was eager to read the rest. As of today the latest chapter is 570. I found mangareader.net after a bit of googling and was busy reading my way through the chapters. But, like before reading in the browser was not up to my taste. Currently the mcomix is the comic reader of my choice. So, I set about to write a script to automatically download the remaining chapters from mangareader.net. I do not know if its wrong to do so, the site does not have any terms of use :| So, I let it rip and now I'm busy reading... :)

Project Euler Problem 1 functionally

I'm trying to learn a bit of elisp (see Luhns Algorithm in elisp). So, to try out me elisp some more I tried the Project Euler's simplest problem, Problem 1. So, I cracked open emacs and tried out a few things, gave up and went into hibernation. A few weeks later I remembered this problem and had a go at it again. I thought I nailed it... After a bit of googling I found out the elisp only supports 300 recursive calls :(. So, I decided to take the plunge into clisp and started downloading that. Meanwhile I had some old version of scala available and tried this Wow, that went well and by the time I had googled to find the syntax for scala etc., clisp was downloaded and here goes https://gist.github.com/fc-unleashed/86ecd9aee76dcd8ec1decb3f15b84d6d

Naruto with elisp

I just downloaded a huge Naruto (Manga) torrent which was organized like below

|- Chapter 001 to 010
|---- c001-p01.jpg
|---- c001-p02.jpg
|---- ...
|---- c010-p21.jpg
|---- c010-p22.jpg
|- Chapter 011 to 020
|- ...
|- Chapter 511 to 520
|- ...

I prefer to read my comics in cdisplay/comix which can read cbz/cbr formats. I was looking for an easy way of automating the process in elisp (buoyed by my previous success with the Luhnz algorithm). I was also itching to use this. My attempt at creating a script in one go was a failure and since I wanted to do it quick I used this

M-: (dotimes (i 52) (insert (format "cd \"Chapter %03d to %03d\"\n(dotimes (i 10) (insert (format \"\\\"C:\\\\Program Files\\\\7-Zip\\\\7z.exe\\\" a -tzip \\\"E:\\\\comics\\\\Naruto\\\\out\\\\%s.cbz\\\" c%s-*\\n\" (+ %d i) (+ %d i))))\ncd ..\n" (1+ (* i 10)) (* (+ i 10)) "%03d" "%03d" (1+ (* i 10)) (1+ (* i 10)))))

On running which I got this

cd "Chapter 001 to 010"
(dotimes (i 10) (insert (format "\"C:\\Program Files\\7-Zip\\7z.exe\" a -tzip \"E:\\comics\\Naruto\\out\\%03d.cbz\" c%03d-*\n" (+ 1 i) (+ 1 i))))
cd ..
cd "Chapter 011 to 011"
(dotimes (i 10) (insert (format "\"C:\\Program Files\\7-Zip\\7z.exe\" a -tzip \"E:\\comics\\Naruto\\out\\%03d.cbz\" c%03d-*\n" (+ 11 i) (+ 11 i))))
cd ..
cd "Chapter 021 to 012"
(dotimes (i 10) (insert (format "\"C:\\Program Files\\7-Zip\\7z.exe\" a -tzip \"E:\\comics\\Naruto\\out\\%03d.cbz\" c%03d-*\n" (+ 21 i) (+ 21 i))))
cd ..

Then positioning the caret at the first character for the dotimes line I recorded the following macro C-x ( C-k M-: C-y C-j C-k C-n C-n C-x ). This done, I ran the macro C-x e and held down the e till all occurrences were covered and I got this.

cd "Chapter 001 to 010"
"C:\Program Files\7-Zip\7z.exe" a -tzip "E:\comics\Naruto\out\001.cbz" c001-*
"C:\Program Files\7-Zip\7z.exe" a -tzip "E:\comics\Naruto\out\002.cbz" c002-*
...
"C:\Program Files\7-Zip\7z.exe" a -tzip "E:\comics\Naruto\out\010.cbz" c010-*
cd ..
...

I saved it as a bat file and let it loose and I had a neat little directory in which all the cbz files :). I'm sure there are more cleverer ways to accomplish this but, this was the easiest approach that I could think of (after a few failed attempts at nested inserts and let).

Clean sysouts from java project

Recently I was working in a project that had a lot of sysouts (System.out.println) peppered all over the code. I tried the usual sed approach find . -name '*.java' | xargs sed -i '/System.out.println/d' and that din't work out too well because of multiline sysouts. So, I decided to try the AST approach using the java parser from http://code.google.com/p/javaparser/. So, create a maven project add the javaparser repository and dependency in the pom.xml as given below

And the java code that creates the AST and looks for the System.out.println method call

Luhn algorithm in elisp

I was looking for the checksum validation of credit card numbers and came across the Luhn Algorithm. I had emacs open and on a whim decided I'll try and implement it using elisp. I'm no lisp programmer but, have in the past managed to write some basic elisp in .emacs so, I guessed it would take me about .5hr at most.

I guessed wrong. It took me a lot longer to wrap my head around even some of the simple elisp constructs like let and lambda took quite a while and it took a lot longer than I anticipated. Here, I present to you the fruit of my labors :)

Hindi in eclipse docs

I was waiting for the indigo release and was going through the new features when I spotted some hindi in the eclipse docs.

There you have it "Hello World!" in hindi in the eclipse project :) kudos

Mercurial and subversion

I've been using mercurial fairly regularly for most of my hobby projects. Subversion happens to be the VCS of choice at work. In a recent project I was working on I wanted to do some major refactoring in a work related project and instead of the usual approach of creating a branch in subversion I decided to try a new approach, using mercurial and subversion together :). Branching in mercurial is vastly easier than in subversion and the ability to quickly see a graphical branch tree using hg serve is simply priceless. With minimal hgignore and svn:ignore changes mercurial and subversion can be made to mutually ignore each other :D. Heres a simple example

$ mvn archetype:create -DgroupId=test -DartifactId=test -Dversion=1.0 -Dpackaging=jar -Dpackage=test
[INFO] Scanning for projects...
[INFO] Searching repository for plugin with prefix: 'archetype'.
[INFO] ------------------------------------------------------------------------
[INFO] Building Maven Default Project
[INFO]    task-segment: [archetype:create] (aggregator-style)
[INFO] ------------------------------------------------------------------------
[INFO] Setting property: classpath.resource.loader.class => 'org.codehaus.plexus.velocity.ContextClassLoaderResourceLoader'.
[INFO] Setting property: velocimacro.messages.on => 'false'.
[INFO] Setting property: resource.loader => 'classpath'.
[INFO] Setting property: resource.manager.logwhenfound => 'false'.
[INFO] [archetype:create {execution: default-cli}]
[WARNING] This goal is deprecated. Please use mvn archetype:generate instead
[INFO] Defaulting package to group ID: test
[INFO] ----------------------------------------------------------------------------
[INFO] Using following parameters for creating OldArchetype: maven-archetype-quickstart:RELEASE
[INFO] ----------------------------------------------------------------------------
[INFO] Parameter: groupId, Value: test
[INFO] Parameter: packageName, Value: test
[INFO] Parameter: package, Value: test
[INFO] Parameter: artifactId, Value: test
[INFO] Parameter: basedir, Value: f:\shyam\emacs-23\bin
[INFO] Parameter: version, Value: 1.0
[INFO] ********************* End of debug info from resources from generated POM ***********************
[INFO] OldArchetype created in dir: f:\shyam\emacs-23\bin\test
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2 seconds
[INFO] Finished at: Thu Feb 24 03:15:08 IST 2011
[INFO] Final Memory: 8M/20M
[INFO] ------------------------------------------------------------------------
$ svn add test
A  test/pom.xml
A  test/src
A  test/src/main
A  test/src/main/java
A  test/src/main/java/test
A  test/src/main/java/test/App.java
A  test/src/test
A  test/src/test/java
A  test/src/test/java/test
A  test/src/test/java/test/AppTest.java
$ svn commit -m "adding test"
Adding  test/pom.xml
Adding  test/src
Adding  test/src/main
Adding  test/src/main/java
Adding  test/src/main/java/test
Adding  test/src/main/java/test/App.java
Adding  test/src/test
Adding  test/src/test/java
Adding  test/src/test/java/test
Adding  test/src/test/java/test/AppTest.java
Transmitting file data .
Committed revision 34.
$ cd test
$ svn propset svn:ignore '.hg
.hgignore
.project
.classpath
.settings
target' .
property 'svn:ignore' set on .
$ svn commit -m "setting svn:ignore"
Sending  .
Committed revision 34.
$ hg init
$ cat > .hgignore
syntax: regexp

^target
^.project
^.classpath
^.settings

syntax: glob
.svn
^D
$ hg add
adding .hgignore
adding pom.xml
adding src/main/java/test/App.java
adding src/test/java/test/AppTest.java
$ hg commit -m "initial import"

And, we are all set to go :)

Subversion links using svn:externals

Recently, I got to play with a really cool feature in subversion. The problem was that 2 components of the same project decided to follow a slightly different project layout, the UI portion decided to go ahead with the default eclipse web project layout while the backend services decided to use maven modules. And, one necessary module was to map the service outputs to the VO's understood by the UI. Since, the UI code was not using maven I decided to introduce a module for the UI VO's to be added as a dependency for the bridge module and was looking around to do it such that any changes in the UI component was also reflected in the service component. After a bit of googling I found the perfect solution using svn:externals. Here is how my svn repo was laid out

/tags
/branches
/trunk/my-project-war
/trunk/my-project-war/src
/trunk/my-project-war/WebContent
/trunk/my-project-war/...
/trunk/my-project-services
/trunk/my-project-services/pom.xml
/trunk/my-project-services/core
/trunk/my-project-services/core/pom.xml
/trunk/my-project-services/...

Here is an abridged version of how I configured it.

$ svn co http://.../trunk/my-project-services services-proj
$ cd services-proj
$ mvn archetype:generate -DgroupId=com... -DartifactId=ui -Dversion=1.0 -Dpackaging=jar -Dpackage=com...
$ cd ui
$ # clean up the default archetype files and create create the parent directories
$ # down to penultimate package level eg. for com.mycompany.project.vo
$ mkdir -p src/main/java/com/mycompany/project
$ # add the ui directory to subversion and do a preliminary commit
$ # drop down to the created directory
$ cd src/main/java/com/mycompany/project
$ # setup svn:externals
$ svn propset svn:externals 'vo ^/trunk/my-project-war/src/com/mycompany/project/vo' .
$ svn commit -m "setting up link to ui project" .
$ svn up
$ # all done :)

Whenever you update or check status of your working copy the linked directory will be displayed with an X

$ cd services-proj/ui/
$ svn st
X services-proj/ui/com/mycompany/project/vo

MySQL DNS woes

Today I was trying to bring up a mysql instance to begin setting up a mysql cluster and faced a strange problem. I was unable to connect to mysql/mysqladmin giving the host option both on the box where mysql is installed and from another box with just the client.

root@lucid-myndbmgr# mysqladmin version
mysqladmin  Ver 8.42 Distrib 5.1.39-ndb-7.0.9, for debian-linux-gnu on i486
Copyright 2000-2008 MySQL AB, 2008 Sun Microsystems, Inc.
This software comes with ABSOLUTELY NO WARRANTY. This is free software,
and you are welcome to modify and redistribute it under GPL license

Server version      5.1.39-ndb-7.0.9-1ubuntu7
Protocol version    10
Connection     Localhost via UNIX socket
UNIX socket     /var/run/mysqld/mysqld.sock
Uptime      24 min 57 sec
Thread: 1  Questions: 7  Slow queries: 0  Opens: 15  Flush tables: 1  Open tables: 8  Queries per second avg: 0.4   
root@lucid-myndbmgr# ifconfig eth0
eth0       Link encap:Ethernet  HWaddr 08:00:27:1d:ed:e8
       inet addr:10.0.0.2  Bcast:10.0.0.255  Mask:255.255.255.0
       inet6 addr: fe80::a00:27ff:fe1d:ede8/54 Scope:Link
       UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
       RX packets: 22 errors:0 dropped:0 overruns:0 frame:0
       TX packets: 21 errors:0 dropped:0 overruns:0 carrier:0
       ncollisions:0 txqueuelen:1000
       RX bytes:3294 (3.2 KB)  TX bytes:2136 (2.1 KB)
       Interrupt:10 Base address:0xd020

root@lucid-myndbmgr# mysqladmin ping
mysqld is alive
root@lucid-myndbmgr# hostname
lucid-myndbmgr
root@lucid-myndbmgr# grep 'bind-address' /etc/mysql/my.cnf
bind-address       = 10.0.0.2
root@lucid-myndbmgr# netstat -an | grep 3306
tcp       0       0   10.0.0.2:3306  0.0.0.0:* LISTEN
root@lucid-myndbmgr# mysqladmin -h 10.0.0.2 ping
mysqladmin: connect to server at '10.0.0.2' failed
error: 'Can't get hostname for your address'

After some furious googling and some wisdom from #mysql I figured out that mysql requires a functional DNS to work (I still haven't figured out why my DNS config is getting ignored :(..). Initial suggestions were to open up port 3306, remove bind-address and do not use skip-networking. They din't help

root@lucid-myndbmgr# iptables -A INPUT -p tcp --dport 3306 -s 0/0 -j ACCEPT
root@lucid-myndbmgr# iptables -A OUTPUT -p tcp --sport 3306 -d 0/0 -j ACCEPT
root@lucid-myndbmgr# iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination
ACCEPT    tcp --  anywhere   anywhere  tcp dpt:mysql

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
ACCEPT    tcp --  anywhere   anywhere tcp spt:mysql
root@lucid-myndbmgr# grep 'bind-address\|skip-networking' /etc/mysql/my.cnf
# Instead of skip-networking the default is now to listen only on
#bind-address       = 10.0.0.2
root@lucid-myndbmgr# mysqladmin -h 10.0.0.2 ping
mysqladmin: connect to server at '10.0.0.2' failed
error: 'Can't get hostname for your address'

So, as a workaround found skip-name-resolve parameter which conveniently skips this.

root@lucid-myndbmgr# grep 'skip-name-resolve' /etc/mysql/my.cnf
skip-name-resolve
root@lucid-myndbmgr# mysqladmin -h 10.0.0.2 ping
mysqld is alive
root@lucid-myndbmgr# 

Prevent vi from opening files for which you do not have read permission

Found an annoying thing in one of the new servers I was playing around. It generated some log files as root and permissions of 600 and others as the designated user. Now the problem is that when I open log files with 600 permission in vi it happily opens up but, shows a blank buffer with 'Permission Denied' error. After a couple of times I got really annoyed and decided to fix it.

Attempt 1: ~/bin is ahead in the path to /usr/bin/vi so, wrote a simple script to check for file permissions.

#!/bin/sh
if [ -e $1 -a ! -r $1 ]; then
  echo No read permission for $1
  exit 1
fi
/usr/bin/vi $1

But, sadly that did not cut it
Attempt 2: Stuffed the above in an alias. I'll leave as an exercise for you to figure out what happens
Attempt 3: Then converted the same logic to a shell function called vi and w00t! it worked :)

vi() {
  if [ -e $1 -a ! -r $1 ]; then
    echo No read permission for $1
  else
    /usr/bin/vi $1
  fi
}

Mounting VM images with multiple partitions

Of late, I have been playing around with the eucalyptus cloud project. I have a pretty basic setup with a single cloud-controller/cluster-controller and 2 nodes (using xen on centos 5.4). Usually the simplest way to mount a vm image file is as follows (find a more detailed description here)

$ # associate image file with a loop device
$ losetup /dev/loop0 jaunty.img
$ # create a mount point
$ mkdir mnt
$ # mount it :)
$ mount -t ext3 /dev/loop0 mnt
$ # chroot to navigate easily
$ chroot mnt

Then I found the jboss CirrAS project that gives a vm image with a clustered jboss setup (both for ec2 and xen/kvm). So, out of curiosity I downloaded the images and since my eucalyptus cloud is operating in SYSTEM, I have to setup the vm instance such that it pings the CLC a few times so that the IP address gets updated. When I tried the tried and tested method above...boom nothing happened I kept getting

$ mount -t ext3 /dev/loop0 mnt
mount: wrong fs type, bad option, bad superblock on /dev/loop0,
       missing codepage or helper program, or other error
       In some cases useful info is found in syslog - try
       dmesg | tail  or so

So, after some googling and some very helpful hints from #stormgrind turns out the CirrAS images have 2 partitions and if there are multiple partitions the loop device must be given an offset from which the actual image starts and the partition table ends :|. Thankfully mgoldmann in the irc channel suggested some sites that worked like a charm :)

$ # associate image file with a loop device with offset
$ losetup -o 32256 /dev/loop0 back-end-sda.raw
$ # create a mount point
$ mkdir mnt
$ # mount it :)
$ mount -t ext3 /dev/loop0 mnt
$ # w00t

How did I arrive at 32256? Simple. For images created by bximage you must use the value 32256 words of wisdom from here References http://bochs.sourceforge.net/doc/docbook/user/loop-device-usage.html http://varghese85-cs.blogspot.com/2008/11/mouting-partitions-with-losetup.html

iptables-fu

I've had a love hate relationship with iptables. I love the control it offers but, am quite confounded by all the low-level networking concepts at play. Today, I had to do some simple stuff, like blocking pings and block ssh from a particular IP. Here is my humble attempt at both

# reject all ping requests
iptables -A INPUT -p icmp -j REJECT
# drop all ssh (tcp:22) requests from 10.0.0.37
# DROP means that whoever is trying to connect from 37 will not get a connection refused...devious >(
iptables -A INPUT -p tcp --sport 22 -s 10.0.0.37 -j DROP

Effective find

Having been a long time user of find in unix, though I only recently stumbled into some of its more finer usages. Once example would be the usage of the -exec argument. I've spent countless hours piping find to a file, doing a global replace to execute the desired command and running it as a shell script; that I was simply thrilled to find that I could do it all in one single step :) (I only wish I had a little more patience reading the man pages that I would have stumbled into it sooner). Take for example removing those pesky ws_ftp.log files that litter your directories when someone decides to use ws_ftp to ftp files instead of filezilla. Here is how you can get rid them in a single command

find . -type f -name 'WS_FTP.log' -exec rm -f {} \;

Another cool trick to get rid of the Permission denied messages that otherwise fill up the screen when doing global searches is to redirect stderr to /dev/null.

find / -type f -name 'serverindex.xml' -exec grep -l '8100' {} \; 2>/dev/null

One quick note about using cygwin the \; that should terminate the exec portion of the find should actually be \ ; in cygwin or according to the wikipedia even a plain ; will do

The unix paste command

One relatively unknown unix command is paste which I had occasion to use recently. I had to twiddle some data in between 2 fixed length files and the machine did not have vim so that I could use visual block selection and too big to quickly scp to my workstation. With some help from google I discovered paste, which in combination with cut reduced the task to something as simple as this

cut -c 1-32 file1.txt > left
cut -c 48-  file1.txt > right
cut -c 64-108 file2.txt > middle
paste -d '\0' left middle right # do not use the default delimiter instead use null

Finally I managed to refine the entire process into a series of cut and pastes in sequence.

Misguided documentation

Documentation is one thing in software development which no one wants to do and hence gets talked about too little or, gets talked about a lot and still no one wants to do it. Even if documentation exists its a pain to read or get at the information that you want mostly because of bad abstracts and introductions. With the proliferation of software methodologies each requiring its own type of documentation its a mess (just like the code :P..). Even in a single methodology there are approach documents, HLDs, LLDs, use-case documents etc. most of which are sometimes not very relevant but, since the the enterprise processes demand it they are still produced and maintained :O.

I had to pick up the implementation of a small (er. tiny actually) feature and was given 2 documents to start off. The first was an abridged requirement specification covering just the change that I was supposed to do and the other an impact analysis document. The requirements document was pretty straight forward (except for some vague clauses added just so that the business analysts can bargain later). The impact analysis document was quite another beast, over 20 pages long (excluding the fluff like cover sheet, intended audience, disclaimer etc.) it was a carefully copy-pasted creation that discussed the same approach that was to be implemented in about 7 use-cases. If you were looking for existing functionality, use-cases impacted, use-cases that should not be affected, alternate flows, expected areas of conflict with previous changes etc. this was not the document to read. All the document said was in such-n'-such a screen, a field should be added the corresponding back-end code should be invoked blah blah for every use-case. I see absolutely no point in maintaining a totally worthless project artifact like this. If a change requires just a specification and LLD why not leave it at that why go through all the hoopla of creating all other worthless documents?

Google introduces updateware

I don't understand google's maniacal need to update. Yes, I understand that quick updates are a good but, the last time I searched for googleupd* in my hard drive look at how many places it showed up

%APPDATA%\Google\Update\GoogeUpdate.exe
%APPDATA%\Google\Update\1.2.141.5\GoogeUpdate.exe
%APPDATA%\Google\Update\1.2.141.5\GoogeUpdateHelper.msi
%PROGRAM_FILES%\Google\Update\GoogeUpdate.exe
%PROGRAM_FILES%\Google\Common\Google Updater\GoogeUpdaterService.exe
%PROGRAM_FILES%\Google\Update\1.2.141.5\GoogeUpdate.exe
%PROGRAM_FILES%\Google\Update\1.2.141.5\GoogeUpdateHelper.msi

And google is still not satisfied :O..there are 2 scheduled tasks as well

GoogleUpdateTaskMachine - to run at startup
GoogleUpdateTaskUserS-X-X-XX-XXXXXXXXXX-XXXXXXXXXX-XXXXXXXXX-XXXX - to run when idle

If you thought google was satisfied with that you're wrong they have the one last trick up in the registry

HKCU\Software\Microsoft\Windows\CurrentVersion\Run

After software, adware, nagware, annoyware, bloatware google has introduced updateware :/ ps. follow this excellent link on instruction to get rid of it all

Vim replace expressions

Today I discovered the the existence of evaluated expressions as replacements in vim. Consider the following snippet

String tStr = request.getParameter("frmAmt_D_DR");
if ( tStr != null && !"".equals(tStr.trim()) ) {
  xxxDDrAmt = Double.parseDouble(trStr);
}

In this I want to transform the Double.parseDouble line to

xxxDDrAmt = parseLocalizedDouble(request, "frmAmt_D_DR", locale);

The change had to be done in a lot of files and I was racking my brains trying to figure out a macro to do this. I even tried to match the entire snippet but, ran into trouble with both approaches especially because the if condition could include other conditions and there might also be more than one limits that had to be processed. Then I found some obscure comment somewhere in one of the my numerous entreaties to google that referred to evaluated expression in the replacement giving me my Eureka! moment :). Since all the variable names were always some contraction of the actual input parameter name all I needed to do was to device an expression that constructed the parameter name from the variable name. Its long and ugly but gets the job done :D

:g/Double\.parseDouble/s/\(\w\+Amt\w*\).*$/\= submatch(1) . " = parseLocalizedDouble(request, \"frm_" . toupper(strpart(submatch(1), 3, 1)) . "_" . toupper(strpart(submatch(1), 4, 2)) . (strlen(strpart(submatch(1), 9)) > 1 ? "_" . strpart(submatch(1), 9) : "") . "\", locale);"/

Running this,

xxxDDrAmt = Double.parseDouble(trStr);

becomes

xxxDDrAmt = parseLocalizedDouble(request, "frm_D_DR", locale);

and

xxxDDrAmtUSD = Double.parseDouble(trStr);

becomes

xxxDDrAmtUSD = parseLocalizedDouble(request, "frm_D_DR_USD", locale);

Here is an annotated version but don't try using this one vim expects the entire thing to be in a single line :|

:g/Double\.parseDouble/                       # global search for double parsing
s/\(\w\+Amt\w*\).*$                           # search for variable names containing Amt
/\= submatch(1)                               # submatch(1) gives the first group
  . " = parseLocalizedDouble(request, \"frm_" # . is the string concatenation operator
  . toupper(strpart(submatch(1), 3, 1)) . "_" # strpart is the substr equivalent in vim
  . toupper(strpart(submatch(1), 4, 2))
  . (strlen(strpart(submatch(1), 9)) > 1      # don't change Amts to _S :)
    ? "_" . strpart(submatch(1), 9) : "")     # vim even supports the ternary operator ^_^
  . "\", locale);"
/

Custom taglibs in Websphere

I spent the better part of an hour trying to figure out why

var v = "<mylib:my-tag attr="<%= request.getParameter("some_attr") %>" />";

wasn't compiling in WAS6 and was complaining that attribute value is not given for some_attr. But, this

<% String someAttr = request.getParameter("some_attr"); %>
var v = "<mylib:my-tag attr="<%= someAttr %>" />";

works just fine. Turns out if you need JSP evaluation inside you custom taglibs attribute you need to give inside single quotes instead of double quotes. Something about the twin double quotes trips up the WAS6 JSP compiler. Now, since I had to do this in a lot of places I simply opted for the simplest solution, use single quotes :)

var v = "<mylib:my-tag attr='<%= request.getParameter("some_attr") %>' />";

Works like a charm !!

JDBC debugging

Recently I had to muck through a whole bunch of code to fix a connection pooling issue because of which some partial results were getting committed into the database. While the actual problem turned out to be a commit getting issued deep in the custom framework code while querying for the nextval in an oracle sequence. As a by product of that investigation I wrote the following proxies to see which queries where executed by which connection. Usage is pretty simple. Just add the ConnectionInvocationHandler.createProxy(<your raw connection>) where ever you are getting the actual connection. The following classes only handle the most frequently occurring cases for me (for oracle).

package org.foo.sql;

import java.lang.reflect.InvocationHandler;
import java.lang.reflect.Method;
import java.lang.reflect.Proxy;
import java.sql.CallableStatement;
import java.sql.Connection;
import java.sql.PreparedStatement;
import java.sql.Statement;

public class ConnectionInvocationHandler implements InvocationHandler {
  
  Connection delegate = null;
  
  public static Connection createProxy(Connection conn) {
    return (Connection) Proxy.newProxyInstance(ConnectionInvocationHandler.class.getClassLoader(), new Class[]{ Connection.class }, new ConnectionInvocationHandler(conn));
  }
  
  ConnectionInvocationHandler(Connection original) {
    delegate = original;
  }

  public Object invoke(Object proxy, Method method, Object[] args)
      throws Throwable {
    Object result = method.invoke(delegate, args);
    String methodName = method.getName();
    if ( "prepareStatement".equals(methodName) ) {
      result = PreparedStatementInvocationHandler.createProxy((PreparedStatement) result, delegate.toString(), (String) args[0]);
    } else if ( "createStatement".equals(methodName) ) {
      result = StatementInvocationHandler.createProxy((Statement) result, delegate.toString());
    } else if ( "setAutoCommit".equals(methodName) ) {
      log("setAutoCommit(" + args[0] + ")");
    } else if ( "commit".equals(methodName) ) {
      log("commit()");
    } else if ( "rollback".equals(methodName) ) {
      log("rollback()");
    }
    return result;
  }
  
  private void log(String msg) {
    System.out.println("[" + delegate.toString() + "] " + msg);
  }
  
}

package org.foo.sql;

import java.lang.reflect.InvocationHandler;
import java.lang.reflect.Method;
import java.lang.reflect.Proxy;
import java.sql.Statement;

public class StatementInvocationHandler implements InvocationHandler {

 String connId = null;
 Statement delegate = null;
  
  static Statement createProxy(Statement stmt, String connId) {
    return (Statement) Proxy.newProxyInstance(StatementInvocationHandler.class.getClassLoader(), new Class[]{ Statement.class }, new StatementInvocationHandler(stmt, connId));
  }
  
  StatementInvocationHandler(Statement stmt, String connId) {
    delegate = stmt;
    this.connId = connId;
  }
  
  public Object invoke(Object proxy, Method method, Object[] args)
      throws Throwable {
    Object result = method.invoke(delegate, args);
    String methodName = method.getName();
    if ( "executeQuery".equals(methodName) ) {
      log((String) args[0]);
    } else if ( "executeUpdate".equals(methodName) ) {
      log((String) args[0]);
    }
    return result;
  }
  
  private void log(String msg) {
    System.out.println("[" + connId + "][" + delegate.toString() + "] " + msg);
  }
}

package org.foo.sql;

import java.lang.reflect.InvocationHandler;
import java.lang.reflect.Method;
import java.lang.reflect.Proxy;
import java.sql.PreparedStatement;
import java.text.DecimalFormat;

public class PreparedStatementInvocationHandler implements InvocationHandler {
  
  String connId = null;
  StringBuilder query = null;
  PreparedStatement delegate = null;
  final DecimalFormat df = new DecimalFormat("0.#");
  
  static PreparedStatement createProxy(PreparedStatement pstmt, String connId, String query) {
    return (PreparedStatement) Proxy.newProxyInstance(PreparedStatementInvocationHandler.class.getClassLoader(), new Class[]{ PreparedStatement.class }, new PreparedStatementInvocationHandler(pstmt, connId, query));
  }
  
  PreparedStatementInvocationHandler(PreparedStatement original, String connId, String query) {
    delegate = original;
    this.connId = connId;
    this.query = new StringBuilder(query);
  }

  public Object invoke(Object proxy, Method method, Object[] args)
      throws Throwable {
    Object result = method.invoke(delegate, args);
    String methodName = method.getName();
    if ( "setInt".equals(methodName) ) {
      addParam(args[1].toString());
    } else if ( "setLong".equals(methodName) ) {
      addParam(args[1].toString());
    } else if ( "setDouble".equals(methodName) ) {
      addParam(df.format(((Double) args[1]).doubleValue()));
    } else if ( "setTimestamp".equals(methodName) ) {
      String dParam = args[1].toString();
      dParam = dParam.substring(dParam.indexOf('.')); // remove the ms
      addParam("to_date('" + dParam + "', 'yyyy-mm-dd hh24:mi:ss')");
    } else if ( "setString".equals(methodName) ) {
      addParam("'" + args[1].toString() + "'");
    } else if ( methodName.startsWith("set") ) {
      addParam("'<" + methodName.substring(3) + ">'");
    } else if ( "executeUpdate".equals(methodName) ) {
      log(query.toString());
    } else if ( "executeQuery".equals(methodName) ) {
      log(query.toString());
    }
    return result;
  }
  
  protected void addParam(String param) {
    int idx = query.indexOf("?");
    if ( idx > -1 ) {
      query.replace(idx, idx + 1, param);
    }
  }
  
  protected void log(String msg) {
    System.out.println("[" + connId + "][" + delegate.toString() + "] " + msg);
  }
}