Wildcards
All files starting with an 'a', and ending with a 'b'
# ls a*b
All files of the form 'a'{char}'b'
# ls a?b
Quotes
Three different kinds. This is one of the most important things to know for any "shell programming".
Single quotes
Variables and wildcards are NOT expanded when contained in a set of single quotes. Example:
# a=foo
# b='goo boo'
# echo '$a $b'
$a $b
Double quotes
Variables and wildcards (*, ?, $...) are expanded (called globbing and/or interpolation sometimes depending on the context).
# a=foo
# b='goo boo'
# echo "$a $b"
foo goo boo
You don't have to double quote something for this sort of wildcard, and variable expansion, so you could write:
# echo $a $b
and the result will be the same:
foo goo boo
There is a difference though, namely, echo will treat this as three arguments, because the command is expanded before the final execution. This can be important when you want something with spaces to be treated as a single argument. Example:
# gcc x.c | grep ': error'
Back Quotes
Expression is executed as a command.
# cleartool checkin -c 'Please, let this compile and link this time.' `lsco`
Execution of a command in another one can also be done with a variable syntax (sometimes useful for nesting stuff). These would produce the same output:
# echo It once was: `date`
# echo It once was: $(date)
It once was: Mon Jun 18 16:20:28 EDT 2007
The alternate syntax can be useful if you wanted to run a command inside of a command.
Other Special Shell Characters
~ your home dir.
; command separator
\ backslash (escape). When you want to use a special character as is, you either have to single quote it, or use an escape character to let the shell know what you want.
Redirect input and output
| pipe input from another command
< redirect input
> redirect output
2>&1 redirect stderr to stdin
echo hi > hi.out
cat < hi.out
cat /tmp/something | grep ': error' | sort
something_that_may_fail >/tmp/blah 2>&1
something_that_may_fail >/tmp/blah 2>/tmp/blah.err
The for loop.
If you have the quotes and variables mastered, this is probably the next most useful construct for ad-hoc command line stuff. We use computers for repetitive stuff, but it's amazing how little people sometimes take advantage of this.
By example:
# for i in `grep : /tmp/something` ; do echo $i ; done
Here, i is the variable you name, and you can reference it in the loop as $i.
# for i in `cat my_list_of_files` ; do cleartool checkout -nc $i ; done
If the command you want to run is something that accepts multiple arguments then you may even need a for loop. The second example above could be written:
# cleartool checkout -nc `cat my_list_of_files`
It's good to know both ways of doing this, since the backquote method can sometimes hit shell "environment length" limits that force you to split up what has to be done or to do parts individually.
Some common useful programs for command line use.
grep search for an expression or expressions
# gcc x.c | grep ': error'
# grep -n mystring `lsbranch -file` > o ; vim -q o
# grep -e something -e somethingelse
# grep -v '^ *$' # all non blank lines.
tr translate characters (one to another, to uppercase, ...)
# echo $PATH | tr : '\n' # print out path elements on separate lines.
# tr '[a-z]' '[A-Z]' # upper case something.
cut Extract parts of a line
# cut -f1 -d: # extract everything in the line(s) up until the first colon char.
# cut -f3-4 # extract from positions 3 and 4.
sort sort stuff
# sort # plain old sort.
# sort -u # unique lines only
# sort -n # numeric sort
# sort -t : # sort using alternate field separator (default is whitespace).
xargs Run a command on all the items in standard input.
# find . -type f -maxdepth 2 | xargs ls -l
sed search and replace program
# sed 's/string1/string2/g'
# sed 's/#.*//' # remove script "comments"
# sed 's!//.*!!' # remove C++ comments.
# sed 's/[ \t]*$//' # strip trailing whitespace
# sed 's/\(.*\):\(.*\)/\2:\1/' # swap stuff separated by colons.
perl Any of the above and pretty much anything else you can think of.
Explaining perl is a whole different game, and if you don't know it, it probably won't look much different than ASCII barf (much like some of the sed commands above).
Some examples (things done above with a mix of other commands) :
# g++ x.c | perl -ne 'print if /: error/'
# perl -pe 's/string1/string2/g'
# perl -e ' $p = $ENV{PATH}; $p =~ s/:/\n/g ; print "$p\n" '
# perl -pe '$_ = uc($_)'
What's notable here is not the perl itself, but the fact that to run some of these commands required passing a pile of shell special characters. In order to pass these all to perl unaltered, it was required to use single quotes, and not double quotes.
Common to grep, sed, and perl is a concept called a regular expression (or regex). This is an extremely valuable thing to get to know well if you do any programming, since there's often a lot of text manipulation required as a programmer. Going into detail on this topic will require it's own time.
Shell Aliases
These are one liner "shortcuts". ksh/bash example:
alias la='ls -a'
Shell Functions
Multiline shortcuts. ksh/bash example:
function foo
{
echo boo
echo foo
}
This is similar to putting the commands in their own file and running that as a script, and can be used as helper functions in other scripts or as more complex "alias"es.
The example above could be written as:
alias foo='echo boo ; echo foo'
But functions also allow you do pass arguments. Example:
function debugattach {
$1 ~/sqllib/adm/db2sysc --pid $(db2pd -edus -dbp $2 | perl -ne 'next unless s/db2sysc PID: // ; print')
}
alias ddda='debugattach ddd'
alias gdba='debugattach gdb'
calling this with ddda 0 will attach the ddd debugger to the db2sysc process db2pd reports to be node 0.
Except for the perl fragment, which is basically a combined 'grep' and 'sed', this example uses many of the things that have been explained above (variables, embedded command, single quotes to avoid expansion and for grouping arguments).
Peeter Joot's (OLD) Blog. 