Peeter Joot's (OLD) Blog.

Math, physics, perl, and programming obscurity.

xargs. The command line loop without a loop.

Posted by peeterjoot on September 1, 2009

Here’s the task. Check out a bunch of files from version control to work on them, perhaps driven by error or warning messages. Example:

/vbs/engn/include/psm_pvmdebug.h:95: warning #858: type qualifier on return type is meaningless
/vbs/engn/include/sqlbgbc.h:647: warning #858: type qualifier on return type is meaningless
/vbs/engn/include/sqlbgbc.h:667: warning #858: type qualifier on return type is meaningless
/vbs/engn/include/sqlbWLPages.h:28: warning #418: declaration requires a typedef name
/vbs/engn/include/sqldstat.h:1406: warning #68: integer conversion resulted in a change of sign
/vbs/engn/include/sqlmemgr.h:259: warning #68: integer conversion resulted in a change of sign

If you are using ksh, and want to checkout all these files (I’m using clearcase right now, but the translation to any other version control system should be similar), you can do something like:

for i in `cat ./myCompilerOutput | cut -f1 -d: | sort -u` ; do cleartool checkout -nc $i ; done

The effect of this loop, with the compiler output as above, is as if you typed the following on the command line:

cleartool checkout -nc /vbs/engn/include/psm_pvmdebug.h
cleartool checkout -nc /vbs/engn/include/sqlbgbc.h
cleartool checkout -nc /vbs/engn/include/sqlbgbc.h
cleartool checkout -nc /vbs/engn/include/sqlbWLPages.h
cleartool checkout -nc /vbs/engn/include/sqldstat.h
cleartool checkout -nc /vbs/engn/include/sqlmemgr.h

but with a lot less typing or cut and pasting.

If you haven’t seen it before, xargs is a sometimes handy alternative to loops of this sort, and I can do the same thing with:

cut -f1 -d: ./myCompilerOutput | sort -u | xargs cleartool checkout -nc

The cut -f1 stuff says grab all the text from the file ./myCompilerOutput up to the first colon character. The sort -u say to do a unique sort (removing all the duplicate lines), and the last bit, the “xargs” command, says, run the command, ‘cleartool checkout -nc’ on all the names passed from standard input. This will have exactly the same effect as the throw away command line for loop above.

Note that there are other ways of doing this too. I could have done just

cleartool checkout -nc `cat ./myCompilerOutput | cut -f1 -d: | sort -u`

This would be as if you constructed the following big ass checkout command:

cleartool checkout -nc /vbs/engn/include/psm_pvmdebug.h /vbs/engn/include/sqlbgbc.h /vbs/engn/include/sqlbWLPages.h /vbs/engn/include/sqldstat.h /vbs/engn/include/sqlmemgr.h

If you ever try to modify 2000 files at a time, you’ll probably find that the shell may object to that sort of long command. Then you’ll have to either use a loop or a helper tool like xargs;)

Advertisements

2 Responses to “xargs. The command line loop without a loop.”

  1. xargs is indeed very useful to avoid cumbersome loops. Perhaps you know this already, but there is a way to use xargs even when the command to call only takes one parameter, and that is by using the -i switch, that introduces the use of a custom placeholder:

    ps aux | cut -d' ' -f2 | xargs -iXXX echo XXX

    as opposed to

    ps aux | cut -d' ' -f2 | xargs echo

    • peeterjoot said

      I didn’t know that, but would I remember it next time? … probably not.

      for more complex loops I’m likely more inclined to use perl or the ‘for i in …’ bash loop. I tend to use xargs mostly for simpler commands.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

 
%d bloggers like this: