Alex's Unix hint's

The missing option for "wc" - linelength

When you import fixed column sized textfiles into a database you need to verify the file has no additional linebreaks, or to put it simple - you need to make sure all lines are complete and have the same length.
I usually use this:

perl -n -e 'print length . "\n"' file

This prints out each line's length, one number per line, so you can pipe this through sort and uniq (-c) to immediately see if something's wrong.
But how to find out which of my 1 million lines is too short?
You could pipe the output of above through "cat -n" to preceed each line with its linenumber and then grep for the deviating linelength. This would give you the linenumber, if you want to see the line you could then "cat -n file|grep linenumber" or use this

perl -n -e 'print length . ":$_"' file

and grep for your linelength. This command prepends the length and a ':' to each line.
If you just want to look at line X you can use:

sed -n Xp file

Note that there is no space between the linenumber and "p". The last line would be '$' and the selection can be inverted by a '!' sign. So to print the whole file except for the last line do:

sed -n '$!p' file


If your file is tab separated, you can use this to get the columns per line:

perl -n -e '@t=split("\t");print $#t . "\n"' file

Remove old files

If I want to automatically remove old files, I usually use something like this:

find /my/dir -type d -name log.\* -mtime +7 -print0|xargs -0 -r rm -Rf

Nowadays you usually have newsyslog for taiming the logfiles, but this can be used for various purposes.

Sorting files by version number

To sort files by version numbers I once devised this contraption:

echo -e "foobar-2.0.0.tar.gz\nfoobar-1.4.1.tar.gz\nfoobar-1.3.21.tar.gz\nfoobar-1.3.9.tar.gz" | \
perl -ne 'push @t,$_;END{print sort{@a=split/(\D)/,$a;@ab=(@a);@b=split/(\D)/,$b;
@bb=(@b);my$t=0;while(shift(@a)eq shift(@b)){$t++;}if(($ab[$t]=~/\d+/)&&($bb[$t]=~/\d+/)){
$ab[$t]<=>$bb[$t];}else{$ab[$t]cmp$bb[$t];};}@t}'

gives:

foobar-1.3.9.tar.gz
foobar-1.3.21.tar.gz
foobar-1.4.1.tar.gz
foobar-2.0.0.tar.gz

I know it's not pretty but this is not as trivial as it sounds even some humans get confused..

Missing "gdk-pixbuf.loaders" file?

/usr/X11R6/bin/gdk-pixbuf-query-loaders >gdk-pixbuf.loaders

should take care of that. Typical locations are:
• /usr/X11R6/etc/gtk-2.0/gdk-pixbuf.loaders
• /usr/X11R6/etc/gtk/gdk-pixbuf.loaders

Verifying file integrity with public key crypto using openssl

1. Create a new private key:

   openssl genrsa -des3 -out mykey.pem 2048
   

2. Create the public key:

   openssl rsa -in mykey.pem -pubout >mykey.pub
   

3. Create signature of your file "foo.sh"

   openssl dgst -sha1 -sign mykey.pem -out foo.sh.sha1 foo.sh
   

4. Verify signature for your file "foo.sh"

   openssl dgst -sha1 -verify mykey.pub -signature foo.sh.sha1 foo.sh
   

   should give:

   Verified OK
   

Speedy Shell scripts

The key to make shell scripts run quickly is primarily runnnig as few programs as possible. In order to do so it's important to understand what your shell is capable of. Your shell can, for example do string operations and thereby alleviate the need to use dirname, basename, grep, cut or tr.
For example:

$(basename $foo) can be replaced by ${foo##*/}

and

$(dirname $foo) can be replaced by ${foo%/*}

If find that this page gives a very thorough and comprehensive description of what bash (and to large extends also sh and ksh) can do for you (check the "More obscure but useful string operations" section).
Another rarely but valuable trick is to use the shells internal test instead of the regular "[". So instead of:

 if [ "$foo" == "bar" ]

simply write

 if [[ "$foo" == "bar" ]]

and save another fork().
Ofcourse you can combine both tricks to check for example if $foo contains a ":":

 if [[ "${foo}" != "${foo#*:}" ]]
 then	echo "no : found."
 fi