Would it be OK to merge the split string code to pike 8.1?
I have done the merge to a private branch, and now wonder if it would be OK to push it upstream.
It splits strings into two parts: The pike_string structure and the actual string.
There are at least two advantages that are available right now:
1: "static" strings (C-strings and static C-data) can be converted to pike strings without allocating any memory for the string data. This actually saves measurable amount of string bytes.
Also, these strings are faster to create since there is no need to allocate storage for the actual string data (useful as an example in GTK2, there is an actual measurable speedup in the init of that function. Well. microseconds, but it's faster. :))
2: It actually improves the speed of some string operations, probably because the string metadata is closer in memory (more likely to be cached).
The disadvantages are, at least:
1: one more pointer in the string struct. This is, however, balanced for everythinge except short strings by the fact that malloc is not used to allocate them, so there is one less size_t in them (malloc overhead)
2: string data is individually mallocd (see 1), so here we get the overhead back, except for short strings. :)
So, basically: 8 bytes more overhead for strings.
As for the speed of string creation/free:ing:
Without patch:
| String Creation . . . . . . . . . . . . . . . . . . 2207k/s | String Creation (existing) . . . . . . . . . . . . 8335k/s
With patch:
| String Creation . . . . . . . . . . . . . . . . . . 3069k/s | String Creation (existing) . . . . . . . . . . . . 8771k/s
Seen this way the split-method is faster for basic string operations. The numbers, however, vary so much that it's basically a wash.
Another advantage with this method is that it is reasonably trivial to add new string types. As an example that I am seriously considering: A string that is a tail port of another string.
The tail-string would be used for this somewhat unfortunate but rather common coding pattern:
sscanf( str, "%2c%s", part, str );
and, in a similar manner:
len = file->write(str); str = str[written..]
These are both O(n^2), but if tail-of string are added they could both be about O(n), only reallocate the data if more than 50% is wasted.
Unfortunately the same can not be done for random substrings, since strings are guaranteed to be 0-terminated. But that is probably not much of an issue in practice.
Also, when gc:ing code could be added to "trim" all tail-of strings.
There is at least one slightly incompatible change to the C-level API:
push_constant_text and the related macros (create a static pike_string the first time it's called, and then use that from now on) now require actual static lifetime of the argument passed to it.
This is already the case for all code in the pike tree, but external modules might break this assumption.