File Encoding

• NPL text and code encoding is changed from system default to UTF8. This is a major change, especially for file related strings. Use "script/ide/Encoding.lua" whenever you want to display any file related strings, or use GUI strings as input to file IOs. In the future, we must be aware of the NPL file encoding, MCML encoding, GUI encoding, jabber message, REST API, and all system IO string encodings. UTF8 is the recommented choice for NPL file, MCML file, GUI, Jabber and REST. however, all strings used by system calls like IO functions are only supported in native system encoding. For an example, please see FileExplorerCtrl.lua and PersonalWorldPage.lua. PersonalWorldPage.lua demostrates conversion between IO, MCML, and GUI.
  • Rule: whenever you get a string in the GUI, and want to use it as input to file IO functions, you need to convert it from UTF8 to default encoding; whenever you want to display a filename via a GUI object, you need to convert it from default encoding to UTF8.
• It is highly recommended that all model asset files and world files use only standard english words.
• Visual studio and VSS(source control server) both recognize file encoding when checking in/out files. They will automatically omit BOM (such as utf8 signature when comparing files). Ultraedit also use some magic code to decide the best encoding by examing a large segment of code if BOM is not at the beginning of the file.
• NPL.load is able to load source files with or without UFT8 signature. However, the visual studio NPL compiler will fail if there is a BOM (0xEFBBBF) at the beginning of the file. So it is highly recommended that one saves lua file as UTF8 without signature. If you are not sure about which encoding to use, just never include localized strings in source files( just read those strings from other places (such as text or db files) whose encoding you know).
• currently all world files and model assets path strings use system default encoding, this makes saving and reading on the native computer easier. And it is generally not a problem to copy and paste files from one machine to another. However, if the default encoding on the other machine is different from the development machine's, it will display bizzar characters, but data itself should be fine. So if one wants to display strings based on file name, make sure they are explicitly saved with a known encoding (such as UTF8), other than the default.