“fun” bug in 64-bit perl 5.10.0 bigint sprintf

Here’s a rather unexpected bug with perl sprintf

#! /usr/bin/perl

use strict;
use warnings;
use bigint ;

my $a = hex( "0x0A0000001D05A820" ) ;
printf( "0x%016X\n", $a ) ;
printf( "%d\n", $a ) ;
printf( "$a\n" ) ;

The %X printf produces a value where the least significant 0x20 is lost:

$ ./bigint
0x0A0000001D05A800
720575940866189312
720575940866189344

Observe that the loss occurs in the printf and not the hex() call, since 720575940866189344 == 0x0A0000001D05A820.

This bug appears to be fixed in some version of perl <= 5.16.2. Oh, the joys of using ancient operating system versions so that we can support customers on many of the ancient deployments that they seem to like to run on.

AFS uid lookup

Despite AFS having been out of service for a number of years, our lab still has some dependencies on it. One of these is that we use AFS uids to keep local uids in sync (since there’s a policy that our local uids have to match our AFS uids even if we aren’t using the AFS filesystem actively in any way).

A side effect of this is that you’ll sometimes see uids on NFS mounted storage that aren’t in the local /etc/passwd file. Here’s an example of how to look these beasties up:

$ /@sys/supp/bin/afs/pts examine pjoot
Name: pjoot, id: 6415, owner: system:administrators, creator: admroot,
  membership: 6, flags: S----, group quota: 17.

$ /@sys/supp/bin/afs/pts examine 6415
Name: pjoot, id: 6415, owner: system:administrators, creator: admroot,
  membership: 6, flags: S----, group quota: 17.

vi (vim) trick of the day: replace commas on current line with newlines

:, ! tr , '\n'

search within search in vim

I was asked how to search within a function for something and report an error if not found (as opposed to continuing past that function for the pattern of interest).

I’d never done this. Something I have done a bazillion times is search and replace within a function. Suppose one has the following:

void foo( int c )
{
   for ( int i = 0 ; i < c ; i++ )
   {
      bar( i ) ;
   }
}

Then positioning yourself on the first line of the function, you can do something like:

:,/^}/ s/bar/Blah/

to replace bar’s with Blah’s on all lines in the range (current line), to curly brace anchored on the beginning of the line (although this requires a coding convention where that is done).

How would you search only within the current function? I tried:

:,/^}/ /bar/

but this appears to take me past my current function, to the next /bar/ match that is NOT in the specified range.

I admit this is something that would have been handy a few times. In particular, we’ve got one source file that’s now 45000 lines long. I’ve gotten messed up a few times by searching for something, and ending up in a completely different function.

Here’s what I came up with:

,/^}/ s/bar//c

Use a prompted search and replace using the /c modifier, and ‘q’uit the replace once a match is found. If no match is found in the range, one gets an error.

This seems like a somewhat abusive use of /c, but works. I’d be curious if there are any other ways?

Use of __LINE__ in perl.

I’d wondered a couple times how to do this, and had littered scripts occasionally with various manually created unique text markers for debugging purposes. Here’s an easier way:

print " Line: ", __LINE__, "\n";

(works for __FILE__ too).

perl implicit matching operator.

Perl can regularly surprise you with many ways of doing the same thing. I’d seen the following fragment of a script and thought “how can that work … there’s no match operator”.

if ( $CC =~ "xlC|xlc" )
{
  # stuff.
}

What I’d expected is a match // operator, or one that used explicit delimiters such as m@@, as in one of:

if ( $CC =~ /xlC|xlc/ )
{
  # stuff.
}

# or

if ( $CC =~ m@xlC|xlc@ )
{
  # stuff.
}

I’d had the urge to “correct” this incorrect if condition, but a small experiment confirms that this code actually worked as intended:

$CC = "/blah/xlC" ;
#$CC = "/blah/gcc" ;

if ( $CC =~ "xlC|xlc" )
{
print "m\n" ;
}

No explicit match delimited expression is required. It appears that the context of the binding operator is enough to convert a subsequent string into a regular expression. I think I still like doing it explicitly better, perhaps just since that’s how I’ve seen it done the most.

comparing some times for perl vs sort command line hacking

I had a 2M line file that contained among other things function identifier strings such as:

SAL_MANAGEMENT_PORT_HANDLE::SAL_ManagementGetServerRole
SAL_MANAGEMENT_PORT_HANDLE::SAL_ManagementHandleClose
SAL_MANAGEMENT_PORT_HANDLE::SAL_ManagementHandleOpen

I wanted to extract just these and sort them by name for something else. I’d first tried this in vim, but it was taking too long. Eventually I control-C’ed it and realized I had to be a bit smarter about it. I figured something like perl would do the trick, and I was able to extract those strings easily with:

cat flw.* | perl -p -e 's/.*?(\S+::\S+).*/$1/;'

(ie: grab just the not-space::not-space text and spit it out). passing this to ‘sort -u’ was also taking quite a while. Here’s a slightly smarter way to do it, still also a one-liner:

cat flw.* | perl -n -e 's/.*?(\S+::\S+).*/$h{$1}=1/e; END{ foreach (sort keys %h) { print "$_\n" ; } } '

All the duplicates are automatically discarded by inserting the matched value into a hash instead of just spitting it out. Then a simple loop over the hash keys gives the result directly. For the data in question, this ended up reducing the time required for the whole operation to just 12.5seconds (eventually I ran the original ‘perl -… | sort -u’ in the background and found it would have taken 1.6 minutes). It took far less time to tweak the command line than the original command would have taken, and provides a nice example where an evaluated expression in the regex match can be handy.

Of course, I then lost my time savings by writing up these notes for posterity;)

Windows powershell. aliases.

 

We’ve been instructed here at work that we must now install Windows XP service pack III, since service pack II is going into a non-update mode.  Four reboots later and much time elapsed, I’ve got the service pack and various other recent windows updates that I’d neglected installed.  One of these is the Windows Powershell.  Well, as a command line kind of guy, I couldn’t turn that down.  Has windows gotten its act together and finally produced a usable shell that one wouldn’t have to run something like the slow cygnus bash to avoid cmd.exe?

Here I’ll make some notes about first impressions.  One of these is that we have aliases, some of which appear to be to help a Unix person feel at home:

PS C:\Documents and Settings\Administrator> alias ls

CommandType     Name                                                Definition
———–     —-                                                ———-
Alias           ls                                                  Get-ChildItem

Interestingly, the old style DOS dir command is also an alias:

 

PS C:\Documents and Settings\Administrator> alias dir

CommandType     Name                                                Definition
———–     —-                                                ———-
Alias           dir                                                 Get-ChildItem

What is this Get-ChildItem.  It appears to be what the powershell help calls a command-let, and it’s behaviour can be viewed with:

 

get-help Get-ChildItem

 

or

 

get-help Get-ChildItem –detailed

 

or

 

get-help Get-ChildItem –full

 

This help text shows that this command-let does a lot more than dir or ls.  For example, it has the capability to query the registry, and do recursion, as well as having exclusion and other flags.  For example, if I don’t want to see the *.txt files in my current dir, I can run:

PS C:\Documents and Settings\Administrator> Get-ChildItem -exclude *.txt

    Directory: Microsoft.PowerShell.Core\FileSystem::C:\Documents and Settings\Administrator

Mode                LastWriteTime     Length Name
—-                ————-     —— —-
d—-        11/24/2008  10:34 AM            (null)
d—-        11/24/2008  10:34 AM            .jnlp-applet
d—-        11/24/2008  10:34 AM            Bluetooth Software
d—-         4/29/2010  11:16 AM            Desktop
d-r–         4/29/2010   1:00 PM            Favorites
d—-        11/24/2008  10:35 AM            IBM
d—-        11/24/2008  10:33 AM            My Documents
d—-         2/10/2010  11:11 AM            SametimeMeetings
d—-        11/24/2008  10:33 AM            SametimeTranscripts
d-r–         9/17/2009  12:39 AM            Start Menu
d—-          2/9/2010  12:18 PM            Tracing
-a—         2/28/2010   9:50 AM      11332 gsview32.ini
-a—         7/30/2009   2:59 PM      46520 install.xml
-a—         3/22/2010   6:15 PM      17437 _viminfo

This little beastie of a command appears to be something of a hybrid of ls, and find.

 

How do we create an alias?

get-help alias | more

This help shows that we can do some interesting queries, like seeing what aliases all map to the Get-ChildItem above.  Example:

 

PS C:\Documents and Settings\Administrator> get-item -path alias:* | where-object {$_.Definition -eq "Get-Childitem"}

CommandType     Name                                                Definition
———–     —-                                                ———-
Alias           gci                                                 Get-ChildItem
Alias           ls                                                  Get-ChildItem
Alias           dir                                                 Get-ChildItem

and also provides an example of creating one:

 

PS C:\Documents and Settings\Administrator> new-item -path alias:np -value c:\windows\notepad.exe

CommandType     Name                                                Definition
———–     —-                                                ———-
Alias           np                                                  c:\windows\notepad.exe

The syntax isn’t exactly obvious seeming like the Unix alias builtin, but it appears usable.  Now that we have an alias, putting it somewhere for use in our shell start up is the next logical step.  This appears to do the job

 

PS C:\Documents and Settings\Administrator> export-alias -name np,note blah
PS C:\Documents and Settings\Administrator> cat blah
# Alias File
# Exported by : Peeter
# Date/Time : Thursday, April 29, 2010 2:36:40 PM
# Machine : JOOT
"np","c:\windows\notepad.exe","","None"
"note","c:\windows\notepad.exe","","None"

 

with import-alias blah, to get them back later when we start a new shell again. 

Cheating header file dependencies with touch and checkin -ptime.

In our build (DB2) we have some header files, that if changed will result in thousands of source files being recompiled. Sometimes you make a change to a header that you know doesn’t really require rebuilding everything. One way of cheating the build dependency system is to judiciously use the touch command to force the date backwards, as in

touch -t197301010000 /vbs/engn/include/sqleSmartArrayDefines.h
cleartool checkin -nc -ptime /vbs/engn/include/sqleSmartArrayDefines.h

An ls -l command after this will show the file time ‘1973-01-01’ (somewhat arbitrarily picked). Thirty five years or so should be early enough to not have make notice it;)

Since we use clearcase for our version control, a checkin of the file into your working branch will change the timestamp unless the -ptime option is used. Care has to be required if you are going to cheat, but I’ve seen people turning off all dependency checking in their build to try to avoid full builds after touching headers, so selecting cheating in a controlled fashion if you are going to do it is a much better option.

Another handy way to do this sort of cheating is ‘touch –reference’. This is a gnu-touch specific (ie: works on Linux) option, that allows you to specify the timestamp using a file instead of a hardcoded date, so you can match the file modification time to some previous point.

Example:

touch --reference sqle_ca_smart_array_cmd.h@@/main/temp_peeterj_db2_galileo_defect_saDeleteDebug/2 sqle_ca_smart_array_cmd.h

In this example, the checked out version (from clearcase) was touched to the predecessor version timestamp after making a change to just a comment that I didn’t want to rebuild everything for.

stripping color control characters from lssam output.

There’s probably a billion ways to do this, but here’s one that appears to work.  If you have TSA command output that has been collected by a somebody who did not use the –nocolor option.

perl -p -e 's/\x1b\[\d+m//g;' < lssamBEFORE.out

The \x1b is the ESC character.  This says to remove that ESC followed by a ‘[‘ character, then 1 or more digits and the character ‘m’, and do it for all lines.