Geek asks and answers

Perl Stream Performance with a Common Variable

I am working on a project implemented in Perl and thought that it would be an idea to use threads to distribute work, because tasks can be performed independently of each other and only read from shared data in memory. However, performance is nowhere near as I expect. So after some research, I can only conclude that threads in Perl basically suck, but I keep wondering how performance goes down as soon as I implement one common variable.

For example, this small program has nothing in common and consumes 75% of the CPU (as expected):

use threads;

sub fib {
  my ( $n ) = @_;
  if ( $n < 2 ) {
     return $n;
  } else {
     return fib( $n - 1 ) + fib( $n - 2 );
  }
}

my $thr1 = threads->create( 'fib', 35 );
my $thr2 = threads->create( 'fib', 35 );
my $thr3 = threads->create( 'fib', 35 );

$thr1->join;
$thr2->join;
$thr3->join;

And as soon as I present the general variable $a, CPU utilization is somewhere between 40% and 50%:

use threads;
use threads::shared;

my $a : shared;
$a = 1000;

sub fib {
  my ( $n ) = @_;
  if ( $n < 2 ) {
    return $n;
  } else {
    return $a + fib( $n - 1 ) + fib( $n - 2 ); # <-- $a was added here
  }
}

my $thr1 = threads->create( 'fib', 35 );
my $thr2 = threads->create( 'fib', 35 );
my $thr3 = threads->create( 'fib', 35 );

$thr1->join;
$thr2->join;
$thr3->join;

, $a , . , .

Perl 5.10.1 Cygwin Windows XP. , , Windows, () Perl.

+3
2

. , - - .

+3

Perl , , . , , , .

use MCE::Hobo;    # use threads okay or parallel module of your choice
use MCE::Shared;

# The module option constructs the object under the shared-manager.
# There no trace of data inside the main process. The construction
# returns a shared reference containing an id and class name.

my $data = MCE::Shared->share( { module => 'My::Data' } );
my $b;

sub fib {
  my ( $n ) = @_;
  if ( $n < 2 ) {
    return $n;
  } else {
    return $b + fib( $n - 1 ) + fib( $n - 2 );
  }
}

my @thrs;

push @thrs, MCE::Hobo->create( sub { $b = $data->get_keys(1000), fib(35) } );
push @thrs, MCE::Hobo->create( sub { $b = $data->get_keys(2000), fib(35) } );
push @thrs, MCE::Hobo->create( sub { $b = $data->get_keys(3000), fib(35) } );

$_->join() for @thrs;

exit;

# Populate $self with data. When shared, the data resides under the
# shared-manager thread (via threads->create) or process (via fork).

package My::Data;

sub new {
  my $class = shift;
  my %self;

  %self = map { $_ => $_ } 1000 .. 5000;

  bless \%self, $class;
}

# Add any getter methods to suit the application. Supporting multiple
# keys helps reduce the number of trips via IPC. Serialization is
# handled automatically if getter method were to return a hash ref.
# MCE::Shared will use Serial::{Encode,Decode} if available - faster.

sub get_keys {
  my $self = shift;
  if ( wantarray ) {
    return map { $_ => $self->{$_} } @_;
  } else {
    return $self->{$_[0]};
  }
}

1;
0

More articles:


All Articles