perl core memory improvements
Name:
Jim Cromie
Email:
hidden email
Amount Requested:
How much is your project worth? $3000
Synopsis
Memory allocation enhancements in core (sv.c).
Perl's variable namespace model is very flexible, users can:
- create vars, in any package, or in my scope, by naming them;
- give them complex values: my $foo = [ 1, { a => 2}, 3 ];
- share/assign/shallow-copy them: $main::bar = $foo;
- crosslink or self ref them: $a[2] = [$a[2], $a[1]];
- other hairy stuff
This user data is all built on-demand from an inventory of sv-parts which is kept on the interpreter's freelists (sv_root, PL_body_roots). These are refilled periodically by S_more_bodies, which gets-an-arena, slices it into sv-parts, and threads them onto the freelist.
This can result in user data spread across memory like a spiderweb in a corner; its hard to clean the corner without destroying the web. IOW, it makes memory reclaim "hard", and probably ineffective. As a result I think, perl core has never really seen the need/benefit to bother reclaiming arenas.
One important workload however could benefit; Storable::freeze() uses a ptr-table to track SVs that it has %seen, but its PTEs hang off the interpreter until process termination. For a long-running process, this is clearly suboptimal.