Performance of unpack() I wrote a PHP class in Zephir to increase performance, but it is still very slow and memory consuming. To compare, I wrote several variants of the same code, but unpack() is always nearly as slow at storing the data in a csv file.

Is there any way to optimize this code? About 30% of the whole program time is spent in unpack.

class Hdb implements \Iterator, \Countable {
    /**
     * File handle.
     * @var resource
     */
    protected fh;

    protected line_count = 0;
    protected position = 0;

    protected row = [];

    public function __construct(string file) -&gt; void {
        let this-&gt;line_count = floor((filesize(file)) / 12);
        let this-&gt;fh = fopen(file, "r");
    }

    public function readLine() {
        var timestamp;
        var value;
        var status;

        let timestamp = unpack("L", fread(this-&gt;fh, 4));
        let value     = unpack("f", fread(this-&gt;fh, 4));
        let status    = unpack("I", fread(this-&gt;fh, 4));

        let this-&gt;position++;

        return [
            "timestamp": timestamp[1],
            "value":     value[1],
            "status":    status[1]
        ];
    }

    // ... The standard Iterator stuff.


Gist
I think your performance issue is caused by going to file multiple times. The more you read in at once the less time you'll spend asking the operating system to provide you with a few more bytes for unpack to operate on.

Similar to the above answer:

$data = fread(fh,12);
$a = unpack("L",substr($data,0,4));
$b = unpack("f",substr($data,4,4));
$c = unpack('I',substr($data,8,4));


If you want to reorganize your code you could load the entire file and then work on the data within it.

$packed = file_get_contents("somefile.dat");
$len  = strlen($packed);

$offset = 0;
while ( $offset &lt; $len )
{
  $a[] = unpack("L",$packed,$offset,4);
  $b[] = unpack("f",$packed,$offset+4,4);
  $c[] = unpack("I",$packed,$offset+8,4);
  $offset += 12;
}


Of course, you may have to reorganize various sections of code to support working on larger chunks of the input data at a time.

Note: It appears that unpacked is being unfairly blamed for being slow due to the filesystem access embedded in the function call. You might also want to do a few tests to clarify what is taking time in the small chunk method and see how much it speeds up based on increasing the read size used (if you have the time and inclination).

EDIT: Adding the following after comments.

You are right. I did some tests and while reading more data at once does save time it isn't as significant as reducing calls to unpack.

Running unpack on binary file (12 byte reads, 3 unpacks per read)...
... fread time: 3.5979931354523
... unpack time: 12.179003238678
... total measured time: 15.77699637413
... records: 265533

Running unpack on binary file (preload file, 3 unpacks per 12 bytes)...
... fread time: 0.016999959945679
... unpack time: 12.446995973587
... total measured time: 12.463995933533
... records: 265533

Running unpack on binary file (preload file, 12 bytes per unpack)...
... fread time: 0.01200008392334
... unpack time: 4.1770012378693
... total measured time: 4.1890013217926
... records: 265533

Running unpack on binary file (preload, 24 bytes per unpack)...
... fread time: 0.010999917984009
... unpack time: 2.189001083374
... total measured time: 2.200001001358
... records: 265534 (* padding added at end of input)


Run using PHP with microtime(true) capture before and after relevant actions.

EDIT: With a bit more fooling around I can now give you an "answer" regarding how to increase performance.

Split your data internally in order to get as much work as possible out of a single call to unpack.

if ( 1 )
{
  echo "&lt;br&gt;\n";
  echo "Running unpack on binary file (preload, split into 3 streams, unpack each stream)...&lt;br&gt;\n";
  ob_flush();
  flush();

  $records = 0;
  $fread_time = 0.0;
  $unpack_time = 0.0;
  $measured = 0.0;

  $start = microtime(true);
  $data  = file_get_contents($bin_file);
  $size  = filesize($bin_file);
  $stop  = microtime(true);
  $fread_time = $stop - $start;

  $Ldata = '';
  $fdata = '';
  $Idata = '';
  $offset = 0;
  $start = microtime(true);
  while ($offset &lt; $size) 
  {
    $Ldata .= substr($data,$offset,4);
    $offset += 4;
    $fdata .= substr($data,$offset,4);
    $offset += 4;
    $Idata .= substr($data,$offset,4);
    $offset += 4;

    $records++;
  }
  $stop = microtime(true);
  $split_time = $stop - $start;

  $start = microtime(true);
  $a = unpack('L*',$Ldata);
  $b = unpack('f*',$fdata);
  $c = unpack('I*',$Idata);
  $stop = microtime(true);
  $unpack_time += ($stop - $start);
  $measured_time = $fread_time + $split_time + $unpack_time;

  echo "... fread time: $fread_time&lt;br&gt;\n";
  echo "... split_time: $split_time&lt;br&gt;\n";
  echo "... unpack time: $unpack_time&lt;br&gt;\n";
  echo "... total measured time: $measured_time&lt;br&gt;\n";
  echo "... records: $records&lt;br&gt;\n";
  ob_flush();
  flush();
}


This provides a much better time.

Running unpack on binary file (preload, split into 3 streams, unpack each stream)...
... fread time: 0.013000011444092
... split_time: 0.29400014877319
... unpack time: 0.54299998283386
... total measured time: 0.85000014305115
... records: 265533


Hopefully this is a bit more useful.