Skip to content

Commit

Permalink
Added second level ID callback implementation
Browse files Browse the repository at this point in the history
  • Loading branch information
vgarvardt committed Oct 24, 2017
1 parent 54955b7 commit 0a31f0d
Show file tree
Hide file tree
Showing 3 changed files with 372 additions and 2 deletions.
56 changes: 54 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ dashboards to track activity and problems.
* `noop` for environments that do not require any stats gathering
* Fixed metric sections count for all metrics to allow easy monitoring/alerting setup in `grafana`
* Easy to build HTTP requests metrics - timing and count
* Generalise or modify HTTP Requests metric - e.g. skip ID part

## Installation

Expand Down Expand Up @@ -90,6 +91,57 @@ $ordersInTheLast24Hours = OrdersService::count(60 * 60 * 24);
$statsClient->trackState($section, $operation, $ordersInTheLast24Hours);
```

## TODO
### Generalise resources by type and stripping resource ID

* [ ] Generalise or modify HTTP Requests metric - e.g. skip ID part
In some cases you do not need to collect metrics for all unique requests, but a single metric for requests of the similar type,
e.g. access time to concrete users pages does not matter a lot, but average access time is important.
`hellofresh/stats-php` allows HTTP Request metric modification and supports ID filtering out of the box, so
you can get generic metric `get.users.-id-` instead thousands of metrics like `get.users.1`, `get.users.13`,
`get.users.42` etc. that may make your `graphite` suffer from overloading.

To use metric generalisation by second level path ID, you can pass
`HelloFresh\Stats\HTTPMetricAlterCallback\HasIDAtSecondLevel` instance to
`HelloFresh\Stats\Client::setHTTPMetricAlterCallback()`. Also there is a builder method
`HelloFresh\Stats\HTTPMetricAlterCallback\HasIDAtSecondLevel::createFromStringMap()`
that builds a callback instance from string map, so you can get these values from config.
It accepts a list of sections with test callback in the following format: `<section>:<test-callback-name>`.
You can use either double colon or new line character as section-callback pairs separator, so all of the following
forms are correct:

* `<section-0>:<test-callback-name-0>:<section-1>:<test-callback-name-1>:<section-2>:<test-callback-name-2>`
* `<section-0>:<test-callback-name-0>\n<section-1>:<test-callback-name-1>\n<section-2>:<test-callback-name-2>`
* `<section-0>:<test-callback-name-0>:<section-1>:<test-callback-name-1>\n<section-2>:<test-callback-name-2>`

Currently the following test callbacks are implemented:

* `true` - second path level is always treated as ID,
e.g. `/users/13` -> `users.-id-`, `/users/search` -> `users.-id-`, `/users` -> `users.-id-`
* `numeric` - only numeric second path level is interpreted as ID,
e.g. `/users/13` -> `users.-id-`, `/users/search` -> `users.search`
* `not_empty` - only not empty second path level is interpreted as ID,
e.g. `/users/13` -> `users.-id-`, `/users` -> `users.-`

You can register your own test callback functions using the
`HelloFresh\Stats\HTTPMetricAlterCallback\HasIDAtSecondLevel::registerSectionTest()` instance method
or the second parameter of builder method - builder method validates test callback functions against the registered list.

```php
<?php

use HelloFresh\Stats\Factory;
use HelloFresh\Stats\HTTPMetricAlterCallback\HasIDAtSecondLevel;

$statsClient = Factory::build(getenv('STATS_DSN'), $logger);
// STATS_IDS=users:numeric:search:not_empty
$callback = HasIDAtSecondLevel::createFromStringMap(getenv('STATS_IDS'));
$statsClient->setHTTPMetricAlterCallback($callback);

$timer = $statsClient->buildTimer()->start();

// GET /users/42 -> get.users.-id-
// GET /users/edit -> get.users.edit
// POST /users -> post.users.-
// GET /search -> get.search.-
// GET /search/friday%20beer -> get.search.-id-
$statsClient->trackRequest($request, $imer, true);
```
128 changes: 128 additions & 0 deletions src/HTTPMetricAlterCallback/HasIDAtSecondLevel.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,128 @@
<?php
namespace HelloFresh\Stats\HTTPMetricAlterCallback;

use HelloFresh\Stats\Bucket;
use HelloFresh\Stats\Bucket\MetricOperation;
use HelloFresh\Stats\HTTPMetricAlterCallback;
use Psr\Http\Message\RequestInterface;

/**
* HTTPMetricAlterCallback implementation for filtering IDs on the second level of HTTP path,
* e.g. to build for all requests like "GET /users/1", "GET /users/2", "GET /users/3" metric
* like "get.users.-id-". See usage examples in README for the library.
*/
class HasIDAtSecondLevel implements HTTPMetricAlterCallback
{
const SECTION_TEST_TRUE = 'true';
const SECTION_TEST_IS_NUMERIC = 'numeric';
const SECTION_TEST_IS_NOT_EMPTY = 'not_empty';
const SECTIONS_DELIMITER = ':';

/** @var array */
protected $map = [];

/** @var callable[] */
protected $sectionsTest = [];

/**
* HasIDAtSecondLevel constructor.
*
* @param array $map sections test map with key as the first section of request path
* and value as section test callback.
*/
public function __construct(array $map)
{
$this->map = $map;

$this->registerSectionTest(static::SECTION_TEST_TRUE, function ($pathSection) {
return true;
})->registerSectionTest(static::SECTION_TEST_IS_NUMERIC, function ($pathSection) {
return is_numeric($pathSection);
})->registerSectionTest(static::SECTION_TEST_IS_NOT_EMPTY, function ($pathSection) {
return $pathSection != Bucket::METRIC_EMPTY_PLACEHOLDER;
});
}

/**
* Creates HasIDAtSecondLevel instance by building sections test map from string value.
* Main use-case for this builder method is for settings loaded from config file or environment variable.
*
* @param string $map
* @param array $registerSectionTests section tests that must be registered for a new instance
* @return self
*/
public static function createFromStringMap($map, array $registerSectionTests = [])
{
$parts = [];
foreach (explode("\n", $map) as $line) {
$line = trim($line);
if ($line !== '') {
foreach (explode(static::SECTIONS_DELIMITER, $line) as $part) {
$part = trim($part);
if ($part !== '') {
$parts[] = $part;
}
}
}
}

if (count($parts) % 2 !== 0) {
throw new \InvalidArgumentException('Invalid sections format');
}

$instance = new static([]);
foreach ($registerSectionTests as $name => $callback) {
$instance->registerSectionTest($name, $callback);
}

$arrayMap = [];
for ($i = 0; $i < count($parts); $i += 2) {
$pathSection = $parts[$i];
$sectionTestName = $parts[$i + 1];
if (!isset($instance->sectionsTest[$sectionTestName])) {
throw new \InvalidArgumentException('Unknown section test callback name: ' . $sectionTestName);
}
$arrayMap[$pathSection] = $sectionTestName;
}

$instance->map = $arrayMap;

return $instance;
}

/**
* @param string $name
* @param callable $callback section test callback that accepts string test section as parameter and returns bool
* if given parameter passes the test.
*
* @return $this
*/
public function registerSectionTest($name, callable $callback)
{
$this->sectionsTest[$name] = $callback;

return $this;
}

/**
* @inheritdoc
*/
public function __invoke(MetricOperation $metricParts, RequestInterface $request)
{
$firstFragment = '/';
foreach (explode('/', $request->getUri()->getPath()) as $fragment) {
if ($fragment !== '') {
$firstFragment = $fragment;
break;
}
}

if (isset($this->map[$firstFragment]) && isset($this->sectionsTest[$this->map[$firstFragment]])) {
if (call_user_func($this->sectionsTest[$this->map[$firstFragment]], $metricParts[2])) {
$metricParts[2] = Bucket::METRIC_ID_PLACEHOLDER;
}
}

return $metricParts;
}
}
190 changes: 190 additions & 0 deletions tests/HTTPMetricAlterCallback/HasIDAtSecondLevelTest.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,190 @@
<?php
namespace HelloFresh\Stats\HTTPMetricAlterCallback;

use HelloFresh\Stats\Bucket;
use HelloFresh\Stats\Bucket\MetricOperation;
use PHPUnit\Framework\TestCase;

class HasIDAtSecondLevelTest extends TestCase
{
/**
* @dataProvider defaultSectionTestsProvider
*
* @param MetricOperation $inputMetric
* @param MetricOperation $result
* @param array $map
*/
public function testDefaultSectionTests(MetricOperation $inputMetric, MetricOperation $result, array $map)
{
$uri = $this->getMockBuilder('\Psr\Http\Message\UriInterface')->getMock();

$uri->expects($this->atLeastOnce())
->method('getPath')
->will($this->returnValue(sprintf('/%s/%s', $inputMetric[1], $inputMetric[2])));

/** @var \PHPUnit_Framework_MockObject_MockObject|\Psr\Http\Message\RequestInterface $request */
$request = $this->getMockBuilder('\Psr\Http\Message\RequestInterface')->getMock();

$request->expects($this->atLeastOnce())
->method('getUri')
->will($this->returnValue($uri));

$callback = new HasIDAtSecondLevel($map);
$this->assertEquals($result->toArray(), $callback($inputMetric, $request)->toArray());
}

public function testRegisterSectionTest()
{
$inputMetric = new MetricOperation(['get', 'users', 'edit']);

$uri = $this->getMockBuilder('\Psr\Http\Message\UriInterface')->getMock();

$uri->expects($this->atLeastOnce())
->method('getPath')
->will($this->returnValue(sprintf('/%s/%s', $inputMetric[1], $inputMetric[2])));

/** @var \PHPUnit_Framework_MockObject_MockObject|\Psr\Http\Message\RequestInterface $request */
$request = $this->getMockBuilder('\Psr\Http\Message\RequestInterface')->getMock();

$request->expects($this->atLeastOnce())
->method('getUri')
->will($this->returnValue($uri));

$callback = new HasIDAtSecondLevel(['users' => 'edit']);
$this->assertEquals($inputMetric->toArray(), $callback($inputMetric, $request)->toArray());

$callback->registerSectionTest('edit', function ($pathSection) {
return $pathSection == 'edit';
});
$this->assertEquals(
(new MetricOperation(['get', 'users', Bucket::METRIC_ID_PLACEHOLDER]))->toArray(),
$callback($inputMetric, $request)->toArray()
);
}

/**
* @dataProvider createFromStringMapProvider
* @expectedException \InvalidArgumentException
* @expectedExceptionMessage Invalid sections format
*
* @param string $map
*/
public function testCreateFromStringMap_InvalidFormat($map)
{
HasIDAtSecondLevel::createFromStringMap($map);
}

/**
* @expectedException \InvalidArgumentException
* @expectedExceptionMessage Unknown section test callback name: foo
*/
public function testCreateFromStringMap_UnknownSectionTest()
{
HasIDAtSecondLevel::createFromStringMap('users:foo');
}

public function testCreateFromStringMap()
{
$inputMetric1 = new MetricOperation(['get', 'users', '1']);
$inputMetric2 = new MetricOperation(['get', 'users', 'foo']);

$uri = $this->getMockBuilder('\Psr\Http\Message\UriInterface')->getMock();

$uri->expects($this->at(0))
->method('getPath')
->will($this->returnValue(sprintf('/%s/%s', $inputMetric1[1], $inputMetric1[2])));
$uri->expects($this->at(1))
->method('getPath')
->will($this->returnValue(sprintf('/%s/%s', $inputMetric2[1], $inputMetric2[2])));

/** @var \PHPUnit_Framework_MockObject_MockObject|\Psr\Http\Message\RequestInterface $request */
$request = $this->getMockBuilder('\Psr\Http\Message\RequestInterface')->getMock();

$request->expects($this->atLeastOnce())
->method('getUri')
->will($this->returnValue($uri));

$callback = HasIDAtSecondLevel::createFromStringMap('users:foo', ['foo' => function ($pathSection) {
return $pathSection == 'foo';
}]);

$this->assertEquals(
$inputMetric1->toArray(),
$callback($inputMetric1, $request)->toArray()
);
$this->assertEquals(
(new MetricOperation(['get', 'users', Bucket::METRIC_ID_PLACEHOLDER]))->toArray(),
$callback($inputMetric2, $request)->toArray()
);
}

public function defaultSectionTestsProvider()
{
return [
// GET /users/1
[
new MetricOperation(['get', 'users', '1']),
new MetricOperation(['get', 'users', Bucket::METRIC_ID_PLACEHOLDER]),
['users' => HasIDAtSecondLevel::SECTION_TEST_TRUE],
],
[
new MetricOperation(['get', 'users', '1']),
new MetricOperation(['get', 'users', Bucket::METRIC_ID_PLACEHOLDER]),
['users' => HasIDAtSecondLevel::SECTION_TEST_IS_NUMERIC],
],
[
new MetricOperation(['get', 'users', '1']),
new MetricOperation(['get', 'users', Bucket::METRIC_ID_PLACEHOLDER]),
['users' => HasIDAtSecondLevel::SECTION_TEST_IS_NOT_EMPTY],
],
// GET /users/edit
[
new MetricOperation(['get', 'users', 'edit']),
new MetricOperation(['get', 'users', Bucket::METRIC_ID_PLACEHOLDER]),
['users' => HasIDAtSecondLevel::SECTION_TEST_TRUE],
],
[
new MetricOperation(['get', 'users', 'edit']),
new MetricOperation(['get', 'users', 'edit']),
['users' => HasIDAtSecondLevel::SECTION_TEST_IS_NUMERIC],
],
[
new MetricOperation(['get', 'users', 'edit']),
new MetricOperation(['get', 'users', Bucket::METRIC_ID_PLACEHOLDER]),
['users' => HasIDAtSecondLevel::SECTION_TEST_IS_NOT_EMPTY],
],
// GET /users
[
new MetricOperation(['get', 'users']),
new MetricOperation(['get', 'users', Bucket::METRIC_ID_PLACEHOLDER]),
['users' => HasIDAtSecondLevel::SECTION_TEST_TRUE],
],
[
new MetricOperation(['get', 'users']),
new MetricOperation(['get', 'users', Bucket::METRIC_EMPTY_PLACEHOLDER]),
['users' => HasIDAtSecondLevel::SECTION_TEST_IS_NUMERIC],
],
[
new MetricOperation(['get', 'users']),
new MetricOperation(['get', 'users', Bucket::METRIC_EMPTY_PLACEHOLDER]),
['users' => HasIDAtSecondLevel::SECTION_TEST_IS_NOT_EMPTY],
],
// does not match
[
new MetricOperation(['get', 'clients']),
new MetricOperation(['get', 'clients', Bucket::METRIC_EMPTY_PLACEHOLDER]),
['users' => HasIDAtSecondLevel::SECTION_TEST_TRUE],
],
];
}

public function createFromStringMapProvider()
{
return [
['foo'],
['foo:bar:baz'],
["foo\n"],
["foo:bar\nbaz"],
];
}
}

0 comments on commit 0a31f0d

Please sign in to comment.