On Fri, Nov 05, 2004 at 01:40:38PM +0100, Henrik Grubbström (Lysator) @ Pike (-) developers forum wrote:
It doesn't? How does it handle the case where you have inserted data with one of the *16 functions, and want to extract it with one of the non *16 functions then?
When I insert the data with *16() function, it will be converted to UTF-8 before storing to file, then, when I extract with non *16() function, no conversion nor checks will be done at all. If I don't use *16() on insertion, then no conversion nor checks done (see below).
My question concerned the other way around. ie inserting invalid UTF-8, and attempting to extract it with one of the *16 functions.
As far as I can see, it does perform conversions internally, and they will most likely fail if the inserted data isn't properly encoded.
It does only if you use *16() functions, or if (not implemented yet) database file is not in UTF-8. From the docs:
"SQLite is not particular about the text it receives and is more than happy to process text strings that are not normalized or even well-formed ^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^ UTF-8 or UTF-16. Thus, programmers who want to store IS08859 data can do so using the UTF-8 interfaces. As long as no attempts are made to use a UTF-16 collating sequence or SQL function, the byte sequence of the text will not be modified in any way."
Yes, that's a natural consequence of using UTF-8 as the native storage format. Note that it doesn't mention what happens if you have stored iso8859 data with the UTF-8 interface, and attempt to retreive it with the UTF-16 interface.
Regards, /Al