Quick Links: Download Gideros Studio | Gideros Documentation | Gideros Development Center | Gideros community chat
Attempt on Localization - Page 4 - Gideros Forum

Attempt on Localization

124

Comments

  • keszeghkeszegh Member
    @MoKaLux
    i did not read ealier comments, so maybe what i say is not new.
    gideros can handle json files pretty well and there are converters from csv to json, google them. so these together should do the job for you. don't give up on your time invested into gideros yet, i'm almost sure your life with libgdx won't be easier.
  • hgy29hgy29 Maintainer
    @MoKaLux, seriously ? CSV is most straightforward file format I know, it could be read with just a few lines of lua code without using an external lib. It makes me feel that either you don't know what a CSV is or you don't have basic lua knowledge. That said without wanting to offend of course.
  • MoKaLuxMoKaLux Member
    @keszegh thank you for the support. Gideros is quite fun to use, the community is great. I will come back to gideros for another project.

    For the project I am on I need some basic things done that is I have a .csv file with headers and I want to retrieve the value of one cell based on its row and column.

    JAVA code:
    tblarab = loadTable("_arab.csv", "header");
    trow = tblarab.getRow(10); // 10th row is an example
    String mystring = trow.getString("myheader");

    voilà!
  • MoKaLuxMoKaLux Member
    edited August 2
    hello @hgy29, yep you've got a point, I am new to lua! As you say csv is the most basic file format but lua isn't straight forward. I have a header, I have a row, what value is in that cell? 25 lines of lua code to get it? I don't understand! I want to have some fun but I'm stuck.
  • hgy29hgy29 Maintainer
    let me craft you a sample code, I will post it here in a few hours
  • keszeghkeszegh Member
    @MokaLux, if you are that new to lua than you probably have many other things missing from your game anyway, by the time you have some practice, reaading a csv will be a breeze (especially that it means only adding one of the lua csv reader implementations and then a simple read call).
  • MoKaLuxMoKaLux Member
    ok thank you very much for this. I'm having a quick try using libgdx to see! It is a simple project, nothing complicated, just to get some values from different rows and headers. Take your time!
  • SinisterSoftSinisterSoft Maintainer
    @MoKaLux Make sire you save your csv as a UTF-8 csv and not an MS-DOS csv - otherwise any special characters that are not in Ascii will go.

    Download this and copy it into your source directory, include it in your project, right-click it and exclude from execution.

    https://github.com/geoffleyland/lua-csv/blob/master/lua/csv.lua

    This is a snip from my code. The csv file has columns that have the headers (in row 1), the first being my language 'en'. In the rows below that column I have the original text in my game - one per row. The other columns have 'fr', 'it', etc... and below each of the column headings I have the text for each language. If I add a language I simply add another column and the translated text below it.
    function arabicProcessing(text)
    	local Achar={0x002,0x024,0x062,0x084,0x0C4,0x104,0x144,0x184,0x1C2,0x1E2,0x202,0x222,
                                   0x244,0x284,0x2C4,0x304,0x344,0x384,0x3C4,0x404,0x440,0x440,0x440,0x440,0x440,0x440,
                                   0x444,0x484,0x4C4,0x504,0x544,0x584,0x5C4,0x602,0x622,0x644}
    	local last=0	
    	local rtext=""
    	for p, cur in utf8.codes(text) do 
    		if ((last>=0x627) and (last<=0x64A)) then
    			local A=Achar[last-0x626]
    			local unibase=0xFE8D+(A>>4)
    			local of=0 --0=iso,1=fin,2=ini,3=med
    			if ((cur>=0x627) and (cur<=0x64A)) then
    				of=2+state
    			else
    				of=state
    			end
    			state=1
    			if (of>=(A&0xF)) then of=0 end
    			of=of+unibase
    			rtext=rtext..utf8.char(of)
    		else
    			if last>0 then
    				rtext=rtext..utf8.char(last)
    			end
    		state=0
    		end
    		last=cur
    	end
    	if last>0 then
    		rtext=rtext..utf8.char(last)
    	end
    	text=""
    	for p, cur in utf8.codes(rtext) do
    		if cur==0x28 or cur==0x29 then cur=cur~1 end
    		text=utf8.char(cur)..text
    	end
    	return text
    end
     
    local osLanguage=lower(sub(application:getLanguage(),1,2))
    --osLanguage="fr"
    local translated={}
    if osLanguage~="en" then
    	message("Language_"..osLanguage)
    	if osLanguage=="pt" then osLanguage="pb" end
    	local f=io.open("languages.csv","r")
    	if f then
    		local data=f:read("*a")
    		f:close()
    		local csv=require("csv")
    		local f=csv.openstring(data,{header=true})
    		if osLanguage=="ar" then osRightToLeft=true end
    		for fields in f:lines() do
    			local t=fields[osLanguage]
    			if t then
    				if t~="" then
    					if osRightToLeft then
    						t=arabicProcessing(t)
    					else
    						t=gsub(t,"ę","e")	-- replace letters our font doesn't have
    						t=gsub(t,"Ő","Õ")
    						t=gsub(t,"ő","õ")
    						t=gsub(t,"Ű","Û")
    						t=gsub(t,"ű","û")
    						t=gsub(t,"ý","y")
    						t=gsub(t,"’","'")
    						t=gsub(t,"“","'")
    						t=gsub(t,"”","'")
    						t=gsub(t,"…","...")
    					end
    					translated[fields["en"]]=t
    				end
    				local n={"fr","it","se","hu","es","pb","ru","cs","ja"}
    				for loop=1,#n do
    					if osLanguage==n[loop] then
    						narrow=0.8
    						break
    					end
    				end
    				if osLanguage=="ru" then narrow=0.7 end
    			else
    				print("*** language not found!")
    				break
    			end
    		end
    	else
    		message("no languages file")
    	end
    end
     
    local function translate(t)
    	return translated[t] or t
    end
    In my code I then have things like this:
    local t=TextField.new(smallFont,translate("Hello World!"))
    So I always have the correct characters available I use this for setting up the fonts:
    local chrs=""
    fonts={{file="retro.ttf",sizeMult=1.0},{file="cyrillic.ttf",sizeMult=1.0},{file="japanese.ttf",sizeMult=0.8},{file="arabic.ttf",sizeMult=1.0}}
    smallFont=TTFont.new(fonts,33,chrs,3)
    Basically that will generate cached font data depending on which characters are used in the game - as they are used.

    Armed with this information you should be ok.
  • MoKaLuxMoKaLux Member
    wouaw! I would need some time to test! Ok, let's do it (libgdx can wait for godot :-)

    Thank you all.

    Likes: SinisterSoft

    +1 -1 (+1 / -0 ) Share on Facebook
  • hgy29hgy29 Maintainer
    As promised, a little sample code to show how it could be done without even using an external csv lib:
    local function loadCSV(file)
    	local function csvsplit(line)
    		local t,w,l2={},nil,nil
    		while #line>0 do
    			w,l2=line:match("\"([^\"]*)\"[,\r\n]?(.*)") -- Check for quoted string
    			if not w then w,line=line:match("([^,]*)[,\r\n]?(.*)") -- Non quoted 
    			else line=l2 end
    			if not w then break end --Nothing or issue, break
    			t[#t+1]=w
    		end
    		return t
    	end
    	local header,csv=nil,{}
    	for line in io.lines(file) do
    		f=csvsplit(line)
    		if not headers then -- Assume first line is headers
    			headers={} for n,v in ipairs(f) do headers[v]=n end
    		else
    			csv[#csv+1]=f
    		end
    	end
    	csv.getField=function (self,row,field) return self[row][headers[field]] end
    	return csv
    end
     
    csv=loadCSV("135-bis-bt.csv")
    print(csv:getField(4,"Value "))
    print(csv:getField(7,"Time"))
    +1 -1 (+3 / -0 ) Share on Facebook
  • SinisterSoftSinisterSoft Maintainer
    @MoKaLux I was stuck with the Arabic text so @Hgy29 sent me the arabicProcessing function in my example.
  • SinisterSoftSinisterSoft Maintainer
    edited August 2
    You should also comment out or ignore the 'message' function in my code, that just sends out a print to any debug systems I use - like the console, flurry, facebook logs, and firebase.

    The narrow variable I use to set an XScale for the text - to make it narrower for those languages.

    I also set the textfield spacing to -1 for if the osRightToLeft has been set.
  • totebototebo Member
    A lot of activity in this thread! @MoKaLux, let us all know if you get it to work. I haven't used Arabic, but CSV files are easy and pretty much required for localisation .
    My Gideros games: www.totebo.com
  • MoKaLuxMoKaLux Member
    edited August 2
    I have got module csv not found error
    no field package.preload['csv']

    I did as you explained, included the code for csv.lua in my project then right clicked exclude from executing.
    When I check the path for csv.lua it is in my project classes/csv.lua (so it is included).

    I right clicked my lua file arabic_text.lua and checked csv.lua in file dependencies but still the error.

    Can you help some more?
  • MoKaLuxMoKaLux Member
    ok I need to check the other answers as I was busy on the first one. Let me try again please.
  • hgy29hgy29 Maintainer
    MoKaLux said:


    When I check the path for csv.lua it is in my project classes/csv.lua (so it is included).

    If it is in classes/csv.lua, then require should be 'classes/csv' or 'classes.csv' instead of just 'csv'

    Likes: MoKaLux

    +1 -1 (+1 / -0 ) Share on Facebook
  • MoKaLuxMoKaLux Member
    edited August 2
    thank you all + thank you @hgy29 your solution "worked" but the letters are not attached.

    Now I need to mix your csv solution to your arabicProcessing code and get those letters linked together!

    Thank you very much for your kind cooperation. I will post my progress and hopefully my success.

    I am enjoying gideros a lot!
  • MoKaLuxMoKaLux Member
    and by the way we should post the csv code in some more accessible place as well. That's a little gem!
  • antixantix Member
    @MoKaLux what were you having trouble with? As far as I am aware a CSV file is just a text file with comma separated values on each line of the file. To read and parse a csv file you can do something like this...
    -- split text into substrings using separator char
    function string:split(sep)
      local sep, fields = sep or ",", {}
      local pattern = string.format("([^%s]+)", sep)
      self:gsub(pattern, function(c) fields[#fields + 1] = c end)
      return fields
    end   
     
    -- read a text file line by line and print each line
    local line
    local file = io.open('contacts.csv', 'rb')
    while true do
      local line = file:read("*l") -- read next line from file
      if line == nil then break end -- exit if no more lines
     
      local fields = line:split() -- split line into strings (comma separated)
     
      for i, v in ipairs(fields) do -- print all values in line
        print(i, v)
      end
    end
    file:close()
    I tried LibGDX some time ago before I found Gideros. Looking at LibGDX now it seems kind of primative IMHO ;)
  • MoKaLuxMoKaLux Member
    hi antix, yes you are all familiar with gideros and lua, but I come from java where it is a little bit more to the point regarding csv files.

    Gideros and the gideros community are really GREAT that's why I am interested in lua but I prefer java code to be honest. Thank you for your post. Peace. I'll get back soon to post here my progress.
  • SinisterSoftSinisterSoft Maintainer
    @antix csv files can get (slightly) complicated - sometimes the file has a BOM header to say it's UTF-8 (that's a relatively recent addiction thanks to Microsoft and it's UTF8 CVS export from Office 365). Also some of the data may use quotes and that has to be accounted for.
  • MoKaLuxMoKaLux Member
    edited August 2
    hello everybody, it is now working as "expected".

    So the steps are:
    1- a .csv file with "headers" UTF8 ex: "txtCode","txtAr","txtFr","gfx1","gfx2","txtEn"

    2 - some data in the next rows ex: "","هٰذا كُرْسِىٌ.","Ceci est une chaise.","gchair","ghandy","This is a chair." *

    * please note I have inserted arabic words with vowels on purpose to test but it does not work with vowels so your words should be without vowels.

    3 - use the loadCSV.lua (it's a gem please see post above) from @hgy29

    4 - use arabicProcessing.lua from the same nice author @hgy29

    5 - use some appropriate fonts ex: DejaVuSans.ttf, DroidNaskh-Regular.ttf, ...

    Et voilà!

    Thank you all for this, I wouldn't have figure it out by myself, I don't like lua but I love Gideros!

    One downside: the arabicProcessing doesn't take care of arabic vowels I don't know if that is possible like in java :-) that would be great.

    Once again thank you ALL for the help.
    +1 -1 (+2 / -0 ) Share on Facebook
  • hgy29hgy29 Maintainer
    edited August 2
    lua can be a little difficult to grasp at first when coming from Java or C/C++, but you'll eventually become used to the freedom lua brings: no static typing, inject variable or code into whatever class/instance you want (tables in lua), override 'system' code, etc. It reminds me of BASIC somehow, but it is way more powerful.

    But yeah, I had hard time with it the first time I tried lua, I had to write a wireshark dissector at that time...

    My favorite languages are Java (for large codebases) and Lua (for rapid prototyping or small projects), although I am comfortable with most languages: C/C++, PHP, assembly, VHDL, Pascal, Javascript,... I didn't like python, I am scared by perl, and hadn't a chance to touch ruby yet :)

    Likes: Apollo14

    +1 -1 (+1 / -0 ) Share on Facebook
  • SinisterSoftSinisterSoft Maintainer
    @MoKaLux Glad you managed to sort it in the end. If you figure out what's wrong with the vowels can you post the update here?
  • MoKaLuxMoKaLux Member
    ok, but for that I need to search if someone did it in pure lua and then see if I can manage to transpose it to gideros. I may have a look at corona forum as well but I promise I won't use corona (I don't like it). Gideros rulez! It's no problem in java :-)
  • MoKaLuxMoKaLux Member
    @hgy29 yes that's what I call a professional coder and you showed your talent in this post (as in many other posts as well). I started lua with love2D, it's great but no easy export to android! Gideros rulez! Godot is very powerful too I did some apps with it. I had a lot of fun, very easy, very fun, ... Why changed? the apk size it produces are a bit too big for me (40mb) for a basic project. Gideros is around 8mb for a basic project (I mean once installed on the device). Hope to make great apps using gideros in the near future with lua, oh lua!
  • keszeghkeszegh Member
    @MokaLux, threatening us to move to another sdk was a good trick to get many responses, i shall try this trick too when i'm stuck.
    joke aside i'm glad you found the solution with the help of the forum members.

    about apk size, i think if you exclude some plugins then this can be decreased further. maybe you already do this.
  • Apollo14Apollo14 Member
    edited August 2
    MoKaLux said:

    I give up! going back to java :-(

    libgdx here I come!

    why not C++ & Cocos2d-x?
    it would be way better choice than libgdx
    > Newcomers roadmap: from where to start learning Gideros
    "What one programmer can do in one month, two programmers can do in two months." - Fred Brooks
  • MoKaLuxMoKaLux Member
    @keszegh yes that worked quite well! but I was not joking, I have been on a quest for a month testing any kind of engine I could find (@Apollo14 no I didn't try cocos2d-x, I don't know c/c++).

    The apk size was between 40mb(godot, embarcadero) up to 100mb (haxe flixel) for a basic project so that was a no go further for me. So now I really want to get going coding and having fun.

    With gideros getting the arabic part working would be awesome and that would mean the end of my quest. So now I need to do a little more searching how to do that in lua (or back to java?)
  • SinisterSoftSinisterSoft Maintainer
    I think you need to look at how utf8 characters are made up - so that the text is converted properly.
Sign In or Register to comment.