The patch looks ok to me, except that this bit appears unnecessary:
+ /* Only store pair if there's a chance we'll look at it later */ + if (m && (!allocated_here || num)) + mapping_insert(m, from, to);
The inserts that matter take place just after allocation in the copy_*_recursively functions. (The corresponding mapping_insert in the old code was apparently just as unnecessary.)