why is pike not using utf-8 internally?

...

explaining that Pike doesn't advocate using UTF8 as an internal format, and why

that is a good question. why is that?

UTF8 is an encoding format; a character (byte) in an UTF8 bytestring doesn't represent a character in the string.

The answer is more or less the same as why you don't use gzipped strings as an internal format: an indexed character in your data doesn't represent a character in the text.

Both gzip and UTF8 are good transfer encoding formats (and orthogonal), but neither are very useful in string manipulation. (Ok, gzip is worse, I'll give you that. But you get my point.)

Nothing stops you from using neither gzip nor UTF-8-encoded strings in Pike though, if you find it useful.