2010Q1 Grant Proposal: perl core memory improvements

perl core memory improvements

Name:
Jim Cromie
Email:
hidden email
Amount Requested:
How much is your project worth? $3000

Synopsis

Memory allocation enhancements in core (sv.c).

Perl's variable namespace model is very flexible, users can:


 - create vars, in any package, or in my scope, by naming them;
 - give them complex values: my $foo = [ 1, { a => 2}, 3 ];
 - share/assign/shallow-copy them: $main::bar = $foo;
 - crosslink or self ref them: $a[2] = [$a[2], $a[1]];
 - other hairy stuff

This user data is all built on-demand from an inventory of sv-parts which is kept on the interpreter's freelists (sv_root, PL_body_roots). These are refilled periodically by S_more_bodies, which gets-an-arena, slices it into sv-parts, and threads them onto the freelist.

This can result in user data spread across memory like a spiderweb in a corner; its hard to clean the corner without destroying the web. IOW, it makes memory reclaim "hard", and probably ineffective. As a result I think, perl core has never really seen the need/benefit to bother reclaiming arenas.

One important workload however could benefit; Storable::freeze() uses a ptr-table to track SVs that it has %seen, but its PTEs hang off the interpreter until process termination. For a long-running process, this is clearly suboptimal.

Tags:

Post a comment

If you have an OpenId URL, you can enter it here to post comments on this site.

Category

This page contains a single entry from the blog posted on February 06, 2010.

Many more entries can be found on the main index page or by looking through the archives.

Powered by
Socialtext