With more and more interfaces using utf-8, would it be a good idea to add two weak mappings with references to strings before and after utf-8 conversions?
How speedy is the conversion today (in MB/sec)?
One possible optimization for repeated calls to string_to_utf8() is to add a flag in the string table for strings consisting of 7-bit contents only since those can be returned unmodified next time.
I get 30 MB/s, so it's not really an issue.
I like your optimization though, since it takes no additional memory and is very cheap and simple to implement.
Good! I thought it would be higher, though. Maybe I'll Shark it myself to see where the bottleneck is.
It should be noted that my suggestion does not save memory because there is already a pre-flight phase where the code estimates the output string size and implements an early exit for 7-bit content. However, the size calculation reads every byte in the string data and probably kicks out other data from the L1/L2 caches so the flag may still be worthwhile.
Turns out that string_to_utf8(), as well as several other functions such as string replace, are using the accessors generic_extract() and index_shared_string() in stralloc.c which in my case (ppc970/gcc 3.3) aren't inlined.
Currently the inner loop will look at string shift and compute addresses in a switch statement for every character. Add to this the overhead for making the function call and it's easy to see the potential for optimization. At least on ppc970 I also get costly loop misalignments but that can be solved by having the configure script dynamically choose optimizer flags instead of using a ppc750 baseline.
Doing some brute-force hacking I got speedups of 2-2.5x (reaching about 170 MB/sec for 7-bit input), but I'm trying to see whether inlining, alignment changes or compiler version/flags are responsible for that.
pike-devel@lists.lysator.liu.se
-
Jonas Walld�n ＠ Pike developers forum -
Martin Nilsson (Opera) ＠ Pike (-) developers forum