Iterables

The Iterable package provides many list methods to process large streams of items. Most of the methods are lazy if the underlying datastream supports it. While all of the data in Catmandu are native Perl hashes and arrays it can be impratical to load a result set of thousands of records into memory. Most Catmandu packages such as Importers, Exporters, and Stores provide therefor an Iterable implementation.

Using a Mock importer we can generate some Perl hashes on-the-fly and show the functionality provided by Iterable:

use Catmandu::Importer::Mock;
my $it = Catmandu::Importer::Mock->new(size => 10);

With each you can loop over all the items in an iterator:

$it->each(sub {
     printf "My n is %d\n" , shift->{n};
});

Using any, many, all you can test for the existence of items in an Iterator:

my $answer = $it->any(sub { shift->{n} > 4});
printf "Iterator contains n > 4 = %s\n" , $answer ? 'TRUE' : 'FALSE';

my $answer = $it->many(sub { shift->{n} > 8});
printf "Iterator contains n > 8 = %s\n" , $answer ? 'TRUE' : 'FALSE';

my $answer = $it->all(sub { shift->{n} =~ /^\d+$/});
printf "Iterator contains only digits = %s\n" , $answer ? 'TRUE' : 'FALSE';

map and reduce are functions that evaluate a function on all the items in an iterator to procude a new iterator or a summary of the results:

# $it contains: [ { n => 1 } , { n => 2 } , ... ];
my $ret = $it->map(sub {
   my $hash = shift;
   { n => $hash->{n} * 2 }
});

# $ret contains : [ { n => 2 } , { n => 4 } , ... ];

my $result = $it->reduce(0,sub {
   my $prev = shift;
   my $this = shift->{n} * 2;
   $prev + $this;
});
printf "SUM [ Iterator * 2] = %d\n" , $result;

The Iterable package provides many more functions such as: to_array, count, each, first, slice, take, group, tap, detect, select, reject, any, many, all, map, reduce and invoke.