Monday 25 February 2013

Progress tracker in PHP


I wanted to test a very slow service today and realized it's probably gonna take hours. Then the obvious reaction to add some timers - also I suspected quite memory consumption as well so checking that also wouldn't hurt.

When I was writing those few line I realized it could be a tiny class so it's at least reusable. Let's see how it looks.
First, I'm investigating the performance and actual states - so it has provide me reports. And knowing that there will be more type of measurement facilities let's make it an interface, so we can add it to all the trackers:

interface iProgressReporter {

  public function report();

}


Then I was thinking - all performance checker has a common point: memory consumption. No matter how you measure a process the way it collects memory is quite common:

class ProgressMemoryTracker implements iProgressReporter {

  protected  $memoryInitial;

  protected  $memoryInitialReal;

  public function __construct() {
    $this->memoryInitial = memory_get_usage(FALSE);
    $this->memoryInitialReal = memory_get_usage(TRUE);
  }

}


We save obviously the initial memory footprint. One thing we definitely want to measure is how much is the process memory allocation at any given point:

  public function getConsumption($real = FALSE) {
    return $real ?
      memory_get_usage(TRUE) - $this->memoryInitialReal :
      memory_get_usage(FALSE) - $this->memoryInitial;
  }


Also we can gather the memory limit for PHP - and from that we can calculate how much is left - approximately:

  public function getAvailableMemory() {
    $totalAvailable = ProgressMemoryTracker::returnBytes(ini_get('memory_limit'));
    return $totalAvailable - memory_get_usage(TRUE);
  }


Here I'm using a handy function I've found on PHP.net to get the byte value of memory_limit:

  protected static function returnBytes($iniValue) {
    $iniValue = trim($iniValue);
    $last = strtolower($iniValue[strlen($iniValue) - 1]);

    switch($last) {
      case 'g':
        $iniValue *= 1024;
      case 'm':
        $iniValue *= 1024;
      case 'k':
        $iniValue *= 1024;
    }

    return $iniValue;
  }


Since ProgressMemoryTracker is implementing iProgressReporter we have to define it here. We just have to provide the details we can get by the class functions:

  public function report() {
    return 'Memory consumption: ' . $this->getConsumption() . ' (total: ' . memory_get_usage(TRUE) . ')' . ' Available memory: ' . $this->getAvailableMemory();
  }


We're using composition to add the memory tracker to any other progress trackers, so let's make it an abstract base class:

abstract class ProgressBasicTracker implements iProgressReporter {

  protected $memoryTracker;

  public function __construct() {
    $this->memoryTracker = new ProgressMemoryTracker();
  }

  abstract function report();

}


Now we can create our first real useful tracker - a single time based tracker. You start it and then you can query the actual state anytime:

class ProgressSingleTracker extends ProgressBasicTracker {

  protected  $startTime;

  public function __construct() {
    parent::__construct();

    $this->startTime = time();
  }

}


Not much addition, only the initial timestamp. One thing we have to define is the abstract interface method:

  public function report() {
    $time_elapsed = time() - $this->startTime;
    return 'Time elapsed: ' . $time_elapsed . ' sec (' . ($time_elapsed / 60) . ' min) ' . $this->memoryTracker->report();
  }


See how we attach the memory report too. All right, let's see an example how to use it:

$progressTracker = new ProgressSingleTracker();

for ($i = 0; $i < 10; $i++) {
  // Do some heavy calculations.
  sleep(0.1);
}

echo $progressTracker->report();


To use the benefits of an iterated process let's make a new subclass that can keep track of loops and potentially provide some estimations. The extra data it requires is the number of iterations and the current state of it:

class ProgressBatchTracker extends ProgressSingleTracker {

  protected  $itemsTotalCount;

  protected  $itemsFinishedCount = 0;

  public function __construct($itemCount) {
    parent::__construct();

    $this->itemsTotalCount = $itemCount;
  }

}


To keep track of the state we need a method where we can update the state:

  public function step() {
    $this->itemsFinishedCount++;
  }


And let's not forget about the report:

  public function report() {
    $time_elapsed = time() - $this->startTime;
    $time_left = ($time_elapsed / $this->itemsFinishedCount) * ($this->itemsTotalCount - $this->itemsFinishedCount);

    return 'Processed items: ' . $this->itemsFinishedCount . ' / ' . $this->itemsTotalCount .
      ' Time elapsed: ' . $time_elapsed . ' sec (' . ($time_elapsed / 60) . ' min) ' .
      'Time left: ' . $time_left . ' sec (' . ($time_left / 60) . ' min) ' .
      $this->memoryTracker->report();
  }


To make it really convenient we can even return the report in the step() call:

  public function step() {
    $this->itemsFinishedCount++;
    return $this->report();
  }


And a sample use case will look like this:

$vector = range(0, 30);
$progressTracker = new ProgressBatchTracker(count($vector));

foreach ($vector as $item) {
  sleep(0.1);
  echo $progressTracker->step() . "\n";
}


You can check out the full code at GitHub.

---

Clearly it's a dumb little library but at least you can toss it in the code, measure, and improve if you want. Actually, please feel free to send me push requests on GitHub, if you wanna use it and have ideas how to improve it.

Peter

No comments:

Post a Comment

Note: only a member of this blog may post a comment.