When someone writes UTF-8 and UTF-16 strings to the same file in binary format without converting between them…

A while ago, I had to fix some stuff in an application that would write – using a binary mechanism – UTF-8 and UTF-16 strings (part of it XML in various flavours)  to the same byte stream without converting between the two encodings.

Some links that helped me investigate what was wrong, choose what encoding to use for storage and fix it:

• unicode – How to Convert Ansi to UTF 8 with TXMLDocument in Delphi – Stack Overflow
• delphi – What should I use? UTF8 or UTF16? – Stack Overflow
• IXMLDocument.SaveToStream does not always use UTF-16 encoding | Marc Durdin’s Blog
• delphi – Length() vs Sizeof() on Unicode strings – Stack Overflow
• TEncoding.UTF8.GetBytes: utf 8 – String to byte array in UTF-8? – Stack Overflow
• SysUtils.ByteLength() is not the nicest function, so use only for a quick-fix Delphi Unicode String Length in Bytes – Stack Overflow
• AnsiString, UnicodeString et al: String Types (Delphi) – RAD Studio
• Question about type identity – delphi
• On the type compatibility in Delphi | The Programming Works
• Joe White’s Blog » Blog Archive » Grammar details of Delphi’s “type type” feature
• Type Compatibility and Identity (Delphi) – RAD Studio
• Delphi “type types”: similar types but not the same type identity, some examples.
• Converting in various Delphi versions (pre-Unicode and post-Unicode versions): Delphi: Encoding Strings as Python do – Stack Overflow
• UTF-8 Conversion Routines – RAD Studio

–jeroen