Peeter Joot's (OLD) Blog.

Math, physics, perl, and programming obscurity.

Archive for February, 2010

Shell tips and tricks. The most basic stuff.

Posted by peeterjoot on February 28, 2010

A while back I assembled the following shell tips and tricks notes for an ad-hoc ‘lunch and learn’ session at work. For some reason (probably for colour) I had made these notes in microsoft word instead of plain text. That made them of limited use for reference, not being cut and pastable (since word mucks up the quote characters). Despite a few things that are work centric (references to clearcase and our source code repository directory structure), there’s enough here that is generally applicable that the converted-to-text version makes sense to have available as a blog post.


# a=foo
# b=goo
# echo $a $b

foo goo
# p=/view/joes_view/vbs/engn/sqo
# diff $p/sqlofmga.C .

You will have many predefined variables when you login. Examples could include

$HOME                            home dir.
$EDITOR                          preferred editor.
$VISUAL                          preferred editor.
$REPLYTO                         where mail should be addressed from.
$PS1                             What you want your shell prompt to look like.
$TMPDIR                          Good to set to avoid getting hit as badly when /tmp fills up.
$CDPATH                          good for build tree paths.

CDPATH Example:


one can run: ‘cd sqo’ and go right to that component dir.

$CDPATH                            good for build tree paths.  Example: CDPATH=.:..:/home/hotelz/peeterj:/vbs/engn:/vbs/test/fvt/standalone:/vbs/common:/vbs/common/osse/core ; one can run: 'cd sqo' and go right to that component dir.
$1                                 First argument passed to a shell script (or shell function).
$*                                 All arguments that were passed to a shell script


All files starting with an 'a', and ending with a 'b'
# ls a*b

All files of the form 'a'{char}'b'

# ls a?b


Three different kinds. This is one of the most important things to know for any "shell programming".

Single quotes

Variables and wildcards are NOT expanded when contained in a set of single quotes. Example:

# a=foo
# b='goo boo'
# echo '$a $b'

$a $b

Double quotes

Variables and wildcards (*, ?, $...) are expanded (called globbing and/or interpolation sometimes depending on the context).

# a=foo
# b='goo boo'
# echo "$a $b"
foo goo boo

You don't have to double quote something for this sort of wildcard, and variable expansion, so you could write:

# echo $a $b

and the result will be the same:

foo goo boo

There is a difference though, namely, echo will treat this as three arguments, because the command is expanded before the final execution. This can be important when you want something with spaces to be treated as a single argument. Example:

# gcc x.c | grep ': error'

Back Quotes

Expression is executed as a command.

# cleartool checkin -c 'Please, let this compile and link this time.' `lsco`

Execution of a command in another one can also be done with a variable syntax (sometimes useful for nesting stuff). These would produce the same output:

# echo It once was: `date`
# echo It once was: $(date)

It once was: Mon Jun 18 16:20:28 EDT 2007

The alternate syntax can be useful if you wanted to run a command inside of a command.

Other Special Shell Characters

~              your home dir.
;              command separator
\              backslash (escape).  When you want to use a special character as is, you either have to single quote it, or use an escape character to let the shell know what you want.

Redirect input and output

|                            pipe input from another command
<                            redirect input
>                            redirect output
2>&1                         redirect stderr to stdin
echo hi > hi.out
cat < hi.out
cat /tmp/something | grep ': error' | sort
something_that_may_fail >/tmp/blah 2>&1
something_that_may_fail >/tmp/blah 2>/tmp/blah.err

The for loop.

If you have the quotes and variables mastered, this is probably the next most useful construct for ad-hoc command line stuff. We use computers for repetitive stuff, but it's amazing how little people sometimes take advantage of this.

By example:

# for i in `grep : /tmp/something` ; do echo $i ; done

Here, i is the variable you name, and you can reference it in the loop as $i.

# for i in `cat my_list_of_files` ; do cleartool checkout -nc $i ; done

If the command you want to run is something that accepts multiple arguments then you may even need a for loop. The second example above could be written:

# cleartool checkout -nc `cat my_list_of_files`

It's good to know both ways of doing this, since the backquote method can sometimes hit shell "environment length" limits that force you to split up what has to be done or to do parts individually.

Some common useful programs for command line use.

grep search for an expression or expressions

# gcc x.c | grep ': error'
# grep -n mystring `lsbranch -file` > o ; vim -q o
# grep -e something -e somethingelse
# grep -v '^ *$'                            # all non blank lines.

tr translate characters (one to another, to uppercase, ...)

# echo $PATH | tr : '\n'              # print out path elements on separate lines.
# tr '[a-z]' '[A-Z]'                  # upper case something.

cut Extract parts of a line

# cut -f1 -d:            # extract everything in the line(s) up until the first colon char.
# cut -f3-4              # extract from positions 3 and 4.

sort sort stuff

# sort                 # plain old sort.              
# sort -u              # unique lines only
# sort -n              # numeric sort
# sort -t :            # sort using alternate field separator (default is whitespace).

xargs Run a command on all the items in standard input.

# find . -type f -maxdepth 2 | xargs ls -l

sed search and replace program

# sed 's/string1/string2/g'
# sed 's/#.*//'                                          # remove script "comments"
# sed 's!//.*!!'                                           # remove C++ comments.
# sed 's/[ \t]*$//'                            # strip trailing whitespace
# sed 's/\(.*\):\(.*\)/\2:\1/'              # swap stuff separated by colons.

perl Any of the above and pretty much anything else you can think of.

Explaining perl is a whole different game, and if you don't know it, it probably won't look much different than ASCII barf (much like some of the sed commands above).

Some examples (things done above with a mix of other commands) :

# g++ x.c | perl -ne 'print if /: error/'
# perl -pe 's/string1/string2/g'
# perl -e ' $p = $ENV{PATH}; $p =~ s/:/\n/g ; print "$p\n" '
# perl -pe '$_ = uc($_)'

What's notable here is not the perl itself, but the fact that to run some of these commands required passing a pile of shell special characters. In order to pass these all to perl unaltered, it was required to use single quotes, and not double quotes.

Common to grep, sed, and perl is a concept called a regular expression (or regex). This is an extremely valuable thing to get to know well if you do any programming, since there's often a lot of text manipulation required as a programmer. Going into detail on this topic will require it's own time.

Shell Aliases

These are one liner "shortcuts". ksh/bash example:

alias la='ls -a'

Shell Functions

Multiline shortcuts. ksh/bash example:

function foo
   echo boo
   echo foo

This is similar to putting the commands in their own file and running that as a script, and can be used as helper functions in other scripts or as more complex "alias"es.

The example above could be written as:

alias foo='echo boo ; echo foo'

But functions also allow you do pass arguments. Example:

function debugattach {
    $1 ~/sqllib/adm/db2sysc --pid $(db2pd -edus -dbp $2 | perl -ne 'next unless s/db2sysc PID: // ; print')
alias ddda='debugattach ddd'
alias gdba='debugattach gdb'

calling this with ddda 0 will attach the ddd debugger to the db2sysc process db2pd reports to be node 0.

Except for the perl fragment, which is basically a combined 'grep' and 'sed', this example uses many of the things that have been explained above (variables, embedded command, single quotes to avoid expansion and for grouping arguments).

Posted in C/C++ development and debugging. | Tagged: , , , , | 1 Comment »

Refocusing my blog. No more personal entropy.

Posted by peeterjoot on February 23, 2010

The two or three people that read this blog may have noticed that it has degenerated into a mess of bad poetry and strange bits of writing, recipes, and even a flammable posts commenting on how graphic porn has gotten since last purchasing a such a mag over 15 years ago (that post I deleted since the masturbatory references so close to all the math and physics was just too strange).

The reasons for this sudden deviation from my old weird blogging norm of perl, math, and physics, is that after 11 years, my marriage suddenly and unexpectedly dissolved. Things are all very confused right now, but I’ll eventually get my life under control again.

I’m going to restore this blog to its old norm of obscurity, and when or if I do feel inclined to post personal entropy I’ll do it on the blogger blog I’d created once and never really used (since it didn’t handle math typesetting in a natural way like on wordpress).

Posted in Incoherent ramblings | Leave a Comment »

prerequisites for Feynman’s thesis: A new approach on Quantum Theory.

Posted by peeterjoot on February 21, 2010

I thought I was doing pretty well with a read of this book, having found typos on page 15, where the index variables got mixed up, leading to something that was obviously wrong. However, after not too much further I get bogged down when he starts referring to use of contact transformations. These are used to develop a systematic way to arrive at the quantum mechanical equations that correspond to a classical action principle (perhaps without even a Hamiltonian). So, I’ve got to shelf this book for a while and go study some more of Goldstein, who appears to cover this topic under the guise of Canonical transformations. I’d seen that chapter and skipped it, not having any idea where it was really headed or why I’d want to study it (like a lot of Goldstein, which at least in the first edition is all too terse). Now I know some of the point of it all, and I expect this will make the study of it go more smoothly.

I also have to say that Feynman is pretty causal with his algebra. I’ve blundered around with trying to reproduce his equation (28), which “This can be expressed in other ways” produces. I got (27) after some messy manipulations, but a hint about how to get to (28) would be nice. Trying again with fresh eyes may help. I’d say that’s a problem for another day.

Posted in Incoherent ramblings | Leave a Comment »

1D forced harmonic oscillator. Quick solution of non-homogeneous problem.

Posted by peeterjoot on February 19, 2010

[Click here for a PDF of this post with nicer formatting]Note that this PDF file is formatted in a wide-for-screen layout that is probably not good for printing.


In [1] equation (25) we have a forced harmonic oscillator equation

\begin{aligned}m \ddot{x} + m \omega^2 x = \gamma(t).\end{aligned} \hspace{\stretch{1}}(1.1)

The solution of this equation is provided, but for fun lets derive it.



\begin{aligned}\omega u = \dot{x},\end{aligned} \hspace{\stretch{1}}(2.2)

we can rewrite the second order equation as a first order linear system

\begin{aligned}\dot{u} + \omega x &= \gamma(t)/m \omega \\ \dot{x} - \omega u &= 0,\end{aligned} \hspace{\stretch{1}}(2.3)

Or, with X = (u, x), in matrix form

\begin{aligned}\dot{X} + \omega \begin{bmatrix}0 & 1 \\ -1 & 0\end{bmatrix}X&=\begin{bmatrix}\gamma(t)/m \omega \\ 0\end{bmatrix}.\end{aligned} \hspace{\stretch{1}}(2.5)

The two by two matrix has the same properties as the complex imaginary, squaring to the identity matrix, so the equation to solve is now of the form

\begin{aligned}\dot{X} + \omega i X &= \Gamma.\end{aligned} \hspace{\stretch{1}}(2.6)

The homogeneous part of the solution is just the matrix

\begin{aligned}X &= e^{-i \omega t} A \\ &= \left( \cos(\omega t) \begin{bmatrix}1 & 0 \\ 0 & 1\end{bmatrix}- \sin(\omega t)\begin{bmatrix}0 & 1 \\ -1 & 0\end{bmatrix}\right) A,\end{aligned}

where A is a two by one column matrix of constants. Assuming for the specific solution X = e^{-i \omega t} A(t), and substuiting we have

\begin{aligned}e^{-i \omega t} \dot{A} = \Gamma(t).\end{aligned} \hspace{\stretch{1}}(2.7)

This integrates directly, fixing the unknown column vector function A(t)

\begin{aligned}A(t) = A(0) + \int_0^t e^{i \omega \tau} \Gamma(\tau).\end{aligned} \hspace{\stretch{1}}(2.8)

Thus the non-homogeneous solution takes the form

\begin{aligned}X = e^{-i \omega t} A(0) + \int_0^t e^{i \omega (\tau - t)} \Gamma(\tau).\end{aligned} \hspace{\stretch{1}}(2.9)

Note that A(0) = (\dot{x}_0/\omega, x_0). Multiplying this out, and discarding all but the second row of the matrix product gives x(t), and Feynman’s equation (26) follows directly.


[1] L.M. Brown, G.D. Carson, L.F. Locke, W.W. Spirduso, S.J. Silverman, D. Holtom, E. Fisher, J.E. Mauch, J.W. Birch, K.L. Turabian, et al. Feynman’s thesis: A New approach to quantum theory. Houghton Mifflin, 1954.

Posted in Math and Physics Learning. | Tagged: | Leave a Comment »

Lasagne. Feb 7th attempt.

Posted by peeterjoot on February 8, 2010

About 16 lasagna noodles for one big pan (spinach noodles are nice). Whole box if making 1.5 pans (ie. one for freezing).

1 package (500g / 1lb) lean ground beef.
1 jar spaghetti sauce.
1 container of cottage cheese (don’t have to bother straining).
Bag of shredded mozzarella on hand.
Parmesan to grate on top when done.
A bunch of mushrooms (~12).
1 zucchini.
Some spinach.
1 pepper (yellow makes a nice colour contrast in the sauce)

Dice up the shrooms, and fry em. Add a bit of white wine to help them get soggy, and fry till the wine evaporates.

Dice up the pepper. Small enough that the kids and wife won’t try to pick em out.

Cut zuke into smallish pieces, and fry em up too (separate from the mushrooms since they cook at separate speeds).

Fry up the ground beef. If frozen make sure to take it out of the freezer in the morning (if cooking mid afternoon). Breaking it up right at the beginning is helpful so it doesn’t clump.

Now’s a good time to get the noodles cooking.

Combine the meat, the peppers, the mushrooms, and the sauce in a big pan, and mix em and cook em a bit all together.

Michelle says I should have fried the spinach a bit to make it floppy. Try that next time. That would also make it easier to add it to the layers, because it makes the layers all wobbly when putting it on raw.

Bottom layer
– noodles on the bottom, slightly overlapping
– cottage cheese, then mozzarella, then sauce, then some spinach.

2nd layer
– same thing

top layer

– noodles, mozzarella, then grate on the parmesan. Cover with foil and cook at 375 (having preheated oven), then 5-10 more minutes uncovered. I’m not sure if the foil makes a difference and I’m going to try next time without it.

Posted in reciepes | 1 Comment »