Problem with strings åäö

jaglitegrann

Hey, I have a problem with showing the letters ÅÄÖ åäö. It is usually not a problem but in this case it is.
A normal print like this:

print("åäö")
result: åäö


Work without problems. But the Strings i try to write comes from facebooks and they look like this when i print them:

print(myString)
result: hall�


I also try to write from ascii like this and get the same problem:

local to_ascii = string.byte("ö")
local from_ascii = string.char(to_ascii)

print(from_ascii);
result: �


And this work whitout any problem whit other characters like this:

local to_ascii = string.byte("a")
local from_ascii = string.char(to_ascii)

print(from_ascii);
result: a



And now I wonder if someone else have been stuck with this problem and know a way around it?

I hope you understand my question

OZApps

Lua 5.2 has better unicode support than the currently used 5.1.4

Now about your question

The size of a character depends on the codepage in use. I recollect this from early 2000's when we were working with an Arabc/English interface and Outlook and Visual Basic which would change the characters typed in as native characters to the latin code page characters, but would render them quite properly.

Try this out

function dumpDetails(theStr) print("String :", theStr) print("Size :", #theStr) print("Details : ") for i = 1, #theStr do print(i, string.byte(theStr:sub(i,i))) end end   dumpDetails("a") -- Normal English dumpDetails("Ä") -- Extended English Script dumpDetails("你") -- Chinese Simplified   dumpDetails("د") -- Arabic dumpDetails("ह") -- Hindi dumpDetails("ภ") -- Thai

You would get the idea how the characters/unicode page changes or works, you might need to check the size of the characters to work with them, you cannot simply use them with a single size of 1 byte per character.

Hope that gives you some insight into this unicode issue.

jaglitegrann

Thank you for your answer!

It make things much clearer for me. Still not sure how to fix the problem in a good way.
But I think I find a way that works anyway

:-bd

krisis

@jaglitegrann - fyi, I saw similar issues with json decoding (see thread http://www.giderosmobile.com/forum/discussion/3038/problem-with-json-decode-and-char-encoding#Item_5). Not sure if it's related to what you're seeing - i.e. if you're decoding json from Facebook.

ar2rsawseen

@krisis it seems that it is not really related, but just a bug in the json library. I forgot to update the thread, but I tested it with native json and it works properly, I just did not have time to build it for all platforms and update in the Labs.
Have repairs in my home and now living at relatives, thus don't have access to all my devices, including Mac. But I can build for Windows, Android and probably iOS if you want

jaglitegrann

Okey, thanks for great answers. That could be my problem, as I read json from facebook as said.

That's great news @ar2rsawseen, and that would be really great, just don't feel any stress about it, cause I can manage with my quickfix for now

nelinho

I was facing same issue, actually you can use Json library from ar2rsawseen that i have forked on github.

url: https://github.com/Nelinho/Json

last update:
Included ability to perform operation with diacritics and special characters, eg: À È Ì Ò Ù à è ì ò ù Â Ê Î Ô Û â ê î ô û
Normally come from facebook's API response of spanish and brasilians names with accented characters in unicode.

romka

You can use this library: https://github.com/romka/lua-transliterator to transliterate strings.