xargs. The command line loop without a loop.
Posted by peeterjoot on September 1, 2009
Here’s the task. Check out a bunch of files from version control to work on them, perhaps driven by error or warning messages. Example:
/vbs/engn/include/psm_pvmdebug.h:95: warning #858: type qualifier on return type is meaningless /vbs/engn/include/sqlbgbc.h:647: warning #858: type qualifier on return type is meaningless /vbs/engn/include/sqlbgbc.h:667: warning #858: type qualifier on return type is meaningless /vbs/engn/include/sqlbWLPages.h:28: warning #418: declaration requires a typedef name /vbs/engn/include/sqldstat.h:1406: warning #68: integer conversion resulted in a change of sign /vbs/engn/include/sqlmemgr.h:259: warning #68: integer conversion resulted in a change of sign
If you are using ksh, and want to checkout all these files (I’m using clearcase right now, but the translation to any other version control system should be similar), you can do something like:
for i in `cat ./myCompilerOutput | cut -f1 -d: | sort -u` ; do cleartool checkout -nc $i ; done
The effect of this loop, with the compiler output as above, is as if you typed the following on the command line:
cleartool checkout -nc /vbs/engn/include/psm_pvmdebug.h cleartool checkout -nc /vbs/engn/include/sqlbgbc.h cleartool checkout -nc /vbs/engn/include/sqlbgbc.h cleartool checkout -nc /vbs/engn/include/sqlbWLPages.h cleartool checkout -nc /vbs/engn/include/sqldstat.h cleartool checkout -nc /vbs/engn/include/sqlmemgr.h
but with a lot less typing or cut and pasting.
If you haven’t seen it before, xargs is a sometimes handy alternative to loops of this sort, and I can do the same thing with:
cut -f1 -d: ./myCompilerOutput | sort -u | xargs cleartool checkout -nc
The cut -f1 stuff says grab all the text from the file ./myCompilerOutput up to the first colon character. The sort -u say to do a unique sort (removing all the duplicate lines), and the last bit, the “xargs” command, says, run the command, ‘cleartool checkout -nc’ on all the names passed from standard input. This will have exactly the same effect as the throw away command line for loop above.
Note that there are other ways of doing this too. I could have done just
cleartool checkout -nc `cat ./myCompilerOutput | cut -f1 -d: | sort -u`
This would be as if you constructed the following big ass checkout command:
cleartool checkout -nc /vbs/engn/include/psm_pvmdebug.h /vbs/engn/include/sqlbgbc.h /vbs/engn/include/sqlbWLPages.h /vbs/engn/include/sqldstat.h /vbs/engn/include/sqlmemgr.h
If you ever try to modify 2000 files at a time, you’ll probably find that the shell may object to that sort of long command. Then you’ll have to either use a loop or a helper tool like xargs;)