Storing the Header

To make CAR files portable across different systems and architectures, you need to take care when writing data to files. Conventionally, if we were to write this structure out to a file, we might use a line of code that looks like this:

   fwrite( header, sizeof( HEADER ), 1, outfile );

This writes a binary image of the header data directly out to the file, so that can be easily read in using an equivalent fread() statement.

There are two potential problems that crop up when writing data out this way. The first relates to the packing of structures. Different C compilers will pack structure elements in different fashions. For example, we have a single char element as the second element of the header array shown above. Since the next element is a long integer, and MS-DOS compiler might put in three bytes of padding so that the long element is aligned on a four-byte boundary. Generally, this is done to generate faster, more efficient code. (Many CPUs tend to work better with data aligned on certain boundaries.) A different compiler on a different machine might not insert any padding, or might use an eight-byte boundary.

When structures are packed differently, we can no longer count on portability between binary files generated using fwrite() calls such as the one shown above. However, it would seem that this would be a relatively easy problem to overcome. Instead of writing out the structure as a single entity, we could just store it one element at a time, which would guarantee that no packing bytes were inadvertently added to our output.

This solution runs afoul of our second portability problem. Unfortunately, we cannot be sure that different computers will store identical data elements using the same structure. For example, a long integer with the hex value 0x12345678L would be stored in the following manner on an Intel 8086 machine:

The same long integer stored on machine based on the Motorola 68000 architecture would have the bytes stored in exactly the reverse order! These differences result from decisions the hardware designers made long ago, for better or worse, and we all have to live with the consequences. In this case, the consequence is a problem with binary file interchange.

The solution is to take control of the binary file format at the lowest level. Instead of trying to write out short and long integers in one fell swoop, we write them out a byte at a time, using the ordering that we select. This way we should be able to store and retrieve data items so that our CAR files can be ported across various systems without worrying about incompatibilities.

When reading and writing the headers, you would first pack and unpack the short and long integer data in the header file into a character array, using a pair of utility routines. We arbitrarily pack the data with the least significant bytes first, although it could just as easily be done in the other order. The routines that do the packing follow:

   void PackUnsignedData( number_of_bytes, number, buffer )
   int number_of_bytes;
   unsigned long number;
   unsigned char *buffer;
   {

    while ( number_of_bytes-- > 0 ) {
     *buffer++ = ( unsigned char ) number & Oxff;
     number >>= 8;
    }
   }

   unsigned long UnpackUnsignedData( number_of_bytes, buffer
   int number_of_bytes;
   unsigned char *buffer;
   {
    unsigned long result;
    int shift_count;

    result = 0;
    shift_count = 0;
    while ( number_of_bytes-- > 0 ) {
     result |= (unsigned long) *buffer++ << shift_count;
     shift_count += 8;
    }
    return( result );
   }

Given these packing and unpacking routines, reading and storing the header files is simple. The process is accomplished for the file I/O using an intermediate character array. The actual header data is packed and unpacked to and from the array.

void WriteFileHeader()
{
  unsigned char header_data[ 17 ];
  int i;

  for ( i = 0 ; ; ) {
   putc( Header.file_name[ i ], OutputCarFile );
   if ( Header.file_name[ i++ ] == '\O' )
    break;
  }

  Header.header_crc = CalculateBlockCRC32( i, CRC_MASK,
                          Header.file_name );
  PackUnsignedData( 1, (long)
            Header.compression_method, header_data + 0 );
  PackUnsignedData( 4, Header.original_size, header_data + 1 );
  PackUnsignedData( 4, Header.compressed_size, header_data + 5 );
  PackUnsignedData( 4, Header.original_crc, header_data + 9 );
  Header.header_crc = CalculateBlockCRC32( 13, Header.header_crc,
       header_data );
  Header.header_crc ^= CRC_MASK;
  PackUnsignedData( 4, Header.header_crc, header_data + 13 );
  fwrite( header_data, 1, 17, OutputCarFile );
}

The routine to write the file header out to the CAR file is somewhat simpler than the routine to read the same data, since it doesn’t have to check for some of the possible error conditions. The first part of a header consists of the name of the compressed file, stored with a null terminator character. No special care needs to be taken when writing out the file name, since eight-bit ASCII characters are portable across all of the systems toward which CARMAN is targeted.

The remaining elements of the header are packed, one by one, into a character array using the PackUnsignedData() routine. Once they have all been properly packed, they can be written out with a call to fwrite(), with everything being in a known state.