Quick Links: Gideros Home | Download Gideros | Developer Guide
TTS PROJECT UPDATE: Text To Speech plug-in for Android now in the testing stage
  • simwhisimwhi +1 -1
    Member
    Hi there,

    We are looking for someone to develop and document a small plug-in project for our games. The project would be to implement Text To Speech (TTS) plug-in for Android.

    If you are interested, please send me a private message so that we can discuss the details further.

    Regards
  • simwhisimwhi +1 -1
    Member
    Hi Gideros maintainers and plug-in authors,

    Bump.

    We are looking for a developer to help us create a text-to-speech (TTS) plug-in for Android. We are prepared to pay the going rate for this project.

    On completion of this project we will make this plug-in available to the whole community so that other would-be plug-in developers can benefit.

    We think that a TTS plug-in would be of use in many different kinds of games and applications.

    If you are interested, please get in touch with us via private message, so that we can discuss the details in more depth. Alternatively, could anyone suggest where we could find a developer that would be able to complete such as project?

    Many thanks,

    Simon


  • hgy29hgy29 +1 -1
    Maintainer
    Hi Simon,

    If you find someone who is able to do the android part (java), then I'd be happy to help/do the gideros integration.
  • simwhisimwhi +1 -1
    Member
    @hgy29
    I already have a working example java project in AS that implements basic TTS with Android, but I don't know how to integrate it with Gideros bindings etc.

    I'll let you know if we get any offers.

  • hgy29hgy29 +1 -1
    Maintainer
    Allright, so if you do have the java project and are happy with what it can do, then just give me the API the gideros plugin should expose to lua and I'll write the plugin.
  • simwhisimwhi +1 -1
    Member
    @hgy29

    That sounds great!!

    If you would like to discuss payment, please send me a private message.

    In the meantime, I'll take another look at my Android project to make sure that I have covered the relevant TTS Android APIs.

    Many thanks.
  • simwhisimwhi +1 -1
    Member
    @hgy29 Now that I have got some other tasks out of the way, I will now start on updating my TTS Android project as well as create a Lua API spec.

  • simwhisimwhi +1 -1 (+1 / -0 )
    Member
    @hgy29 Here's a link to my TTS Android project https://www.dropbox.com/s/lwxxyzjhxlotkgv/TextToSpeechExample.zip?dl=0

    Just rebuild the project and everything should work.

    I have also attached a simple TTS class that wraps the Lua API function calls. I have kept it fairly simple as I would like to add other features in the future.

    Please let me know if you require any further information. I'm really looking forward to getting this plugin working. We will do the same for iOS after.

    Many thank in advance.
    TTS-interface.lua
    2K

    Likes: hgy29

  • hgy29hgy29 +1 -1 (+3 / -0 )
    Maintainer
    This should do for now, I started writing the plugin.

    Likes: pie, NatWobble, simwhi

  • simwhisimwhi +1 -1 (+1 / -0 )
    Member
    @hgy29 WOW!! That's great news! I'll do all the beta testing for... and when I understand how it works I'll write some documentation too...

    Likes: pie

  • hgy29hgy29 +1 -1 (+4 / -0 )
    Maintainer
    Hi @simwhi,

    Here is my first attempt. Requires Gideros 2017.4.1 or better...

    Demo code:
    require "tts"
     
    local tts=TTS.new("UK",1,1)
     
    tts:addEventListener(Event.TTS_ERROR,function (e)
    print("TTS Error:"..e.error)
    end)
     
    tts:addEventListener(Event.TTS_INIT_COMPLETE,function ()
    print("TTS Initied")
    tts:speak("Hello world","hello")
    end)
     
    tts:addEventListener(Event.TTS_UTTERANCE_COMPLETE,function (e)
    print("Utterance complete:"..e.utteranceId)
    end)

    tts.zip
    264K
  • hgy29hgy29 +1 -1
    Maintainer
    And here is the improved, debugged and polished version.
    Sample code:
    require "tts"
     
    local tts=TTS.new("EN-uk")
     
    tts:addEventListener(Event.TTS_ERROR,function (e)
    print("TTS Error:"..e.error)
    end)
     
    tts:addEventListener(Event.TTS_INIT_COMPLETE,function ()
    print("TTS Initied")
    --tts:setLanguage("FR")
    tts:speak("Hello world!","hello")
    --Timer.delayedCall(200,function() tts:stop() end)
    end)
     
    tts:addEventListener(Event.TTS_UTTERANCE_COMPLETE,function (e)
    print("Utterance:"..e.utteranceId.." state:"..e.state)
    end)
     
    --tts.speak("Hello world","hello")
    --tts.stop()


    tts.zip
    266K
  • simwhisimwhi +1 -1
    Member
    @hgy29 Great job!!
  • simwhisimwhi +1 -1
    Member
    @hgy29

    All looks good so far. The only thing I can't do is to change between EN-US and EN-UK (Locale.US and Locale.UK).

  • hgy29hgy29 +1 -1
    Maintainer
    I checked and it works: try using EN-us and EN-uk instead, and be sure you have UK and US voices samples installed on your phone.
  • simwhisimwhi +1 -1
    Member
    @hgy29 I tried those combinations, but they have no effect. I know that both en-us and en-uk are installed as the original Android Studio TTS template app works fine. Other languages seem to work fine.
  • hgy29hgy29 +1 -1
    Maintainer
    Thats weird because I tried that case specifically and did recognize the accent change between uk and us in the spoken sentence..
  • simwhisimwhi +1 -1
    Member
    @hgy29 That is strange. I have a Samsung A9 Pro (Android 6) and a Huawei (Android 5)

    I have Google TTS installed on both.
  • simwhisimwhi +1 -1
    Member
    Hi @hgy29

    The locales "en-us" and "en-uk" should in theory work on my devices, but they don't for some strange reason.

    I decided to explicitly set Locale.US and Locale.UK for these cases and everything now works fine. Here is the code:
    	public boolean setLanguage(String language) {
    mTtsLanguage = language;
     
    if (isLoaded) {
     
    Locale rl = new Locale(language);
    Log.i("Language", rl.toString());
    if (mTts.isLanguageAvailable(rl) >= 0) {
    String lang = rl.toString();
    Log.i("Language available ", lang);
    if (lang.equals("en-us")) {
    rl = Locale.US;
    } else if (lang.equals("en-uk")) {
    rl = Locale.UK;
    }
    mTts.setLanguage(rl);
    }
    else {
    if (android.os.Build.VERSION.SDK_INT >= 21)
    mTtsLanguage = mTts.getVoice().getLocale().getLanguage();
    else
    mTtsLanguage = mTts.getLanguage().getLanguage();
    eventTtsError(ttsIndex, "Language '" + language + "' is not supported, using '" + mTtsLanguage + "'");
    }
    }
    return true;
    }
  • simwhisimwhi +1 -1 (+1 / -0 )
    Member
    @hgy29

    I created a Gideros project to test setSpeed(), setLanguage(), setPitch() and stop() functions continuously. All work fine.

    The stop() function was called using a delayed call. The current speech stopped as expected and the next sentence started (as expected).

    I also tested Locale.US and Locale.UK as described in the previous post.

    Here's the test code:
    require "tts"
     
    local tts=TTS.new("en-uk",1,1)
    local count = 1
    local text = "This is a fairly long sentence that will test the new TTS engine for Android devices."
     
    tts:addEventListener(Event.TTS_ERROR,function (e)
    print("TTS Error:"..e.error)
    end)
     
    tts:addEventListener(Event.TTS_INIT_COMPLETE,function ()
    print("TTS Initialised")
    local sentence = "Test "..count..text
    tts:speak(sentence,tostring(count))
    count = count + 1
    end)
     
    tts:addEventListener(Event.TTS_UTTERANCE_COMPLETE,function (e)
    print("Utterance:"..e.utteranceId.." state:"..e.state)
     
    if e.state == "start" then
    print("started")
    elseif e.state == "done" then
    local sentence = "test "..count .. text
     
    if count == 6 then
    tts:shutdown()
    tts = nil
    elseif count == 5 then
    tts:setSpeed(math.random(30)/10)
    tts:speak(sentence,tostring(count))
    Timer.delayedCall(3000,function() tts:stop() end)
    elseif (count % 2) == 0 then
    tts:setLanguage("en-us")
    tts:setSpeed(math.random(30)/10)
    tts:speak(sentence,tostring(count))
    else
    tts:setLanguage("en-uk")
    tts:setPitch(math.random(30)/10)
    tts:speak(sentence,tostring(count))
    end
    count = count + 1
    end
     
    end)

    Likes: hgy29

  • hgy29hgy29 +1 -1 (+1 / -0 )
    Maintainer
    We need to address this UK/US thing in a more elegant way IMHO. It would be interesting to know what is the difference between Locale.US and Locale.new("EN-us") and the same for UK.
    Anyway I am happy you like it, I will make it available as part of gideros distribution if you agree.

    Likes: antix

  • simwhisimwhi +1 -1
    Member
    @hgy29 I agree, but I don't know what it is, or how to fix it.

    I also agree to making it available to the community, too.
  • pie +1 -1 (+1 / -0 )
    Member
    Thank you! :)
    it's a great addition to gideros! :)>-

    Likes: simwhi

  • NatWobbleNatWobble +1 -1
    Member
    @hgy29 Thanks, this is great.

    I'll polish up my prototype iOS TTS plugin, make it consistent with the Android version, and make it available too.
  • simwhisimwhi +1 -1
    Member
    @hgy29

    I managed to find a solution that is consistent. Here are some test cases:

    Locale rl = new Locale("en", "GB");
    Locale rl = new Locale("en", "US");
    Locale rl = new Locale("en", "AU");
    Locale rl = new Locale("fr", "fr");

  • simwhisimwhi +1 -1
    Member
    @ghy29

    I updated this method:
    	public boolean setLanguage(String language) {
    mTtsLanguage = language;
     
    //Log.i("Language", mTtsLanguage);
    if (isLoaded) {
    String lang = language.substring(0, 2);
    String country = language.substring(3);
    Log.i("Language", lang);
    Log.i("Country", country);
     
    Locale rl = new Locale(lang, country);
    if (mTts.isLanguageAvailable(rl) >= 0) {
    mTts.setLanguage(rl);
    }
    else {
    if (android.os.Build.VERSION.SDK_INT >= 21)
    mTtsLanguage = mTts.getVoice().getLocale().getLanguage();
    else
    mTtsLanguage = mTts.getLanguage().getLanguage();
    eventTtsError(ttsIndex, "Language '" + language + "' is not supported, using '" + mTtsLanguage + "'");
    }
    }
    return true;
    }
  • hgy29hgy29 +1 -1
    Maintainer
    It looks like en_UK is not a valid locale code: http://stackoverflow.com/questions/7296262/is-en-uk-an-illegal-locale
    en_GB is the correct one for UK
  • simwhisimwhi +1 -1
    Member
    @hgy29 Yes, indeed. Locale.UK is valid, but en-uk isn't. I still couldn't get en-gb to work with language parameter though.

  • simwhisimwhi +1 -1
    Member
    Here's a more polished version:
    	public boolean setLanguage(String language) {
    mTtsLanguage = language;
     
    //Log.i("Language", mTtsLanguage);
    if (isLoaded) {
    String lang;
    String country;
    Locale rl = Locale.US;
     
    if (mTtsLanguage.length() == 5) {
    lang = mTtsLanguage.substring(0, 2);
    country = mTtsLanguage.substring(3);
    Log.i("Language", lang);
    Log.i("Country", country);
     
    rl = new Locale(lang, country);
    }
    else if (mTtsLanguage.length() == 2) {
    rl = new Locale(mTtsLanguage);
    }
     
    if (mTts.isLanguageAvailable(rl) >= 0) {
    mTts.setLanguage(rl);
    }
    else {
    if (android.os.Build.VERSION.SDK_INT >= 21)
    mTtsLanguage = mTts.getVoice().getLocale().getLanguage();
    else
    mTtsLanguage = mTts.getLanguage().getLanguage();
    eventTtsError(ttsIndex, "Language '" + language + "' is not supported, using '" + mTtsLanguage + "'");
    }
    }
    return true;
    }


    The following Lua test code uses both 2 and 5 character codes ("fr", "en-gb" and "en-us":
    require "tts"
     
    local tts=TTS.new("fr",1,1)
    local count = 1
    local text = "This is a fairly long sentence that will test the new TTS engine for Android devices."
     
    tts:addEventListener(Event.TTS_ERROR,function (e)
    print("TTS Error:"..e.error)
    end)
     
    tts:addEventListener(Event.TTS_INIT_COMPLETE,function ()
    print("TTS Initialised")
    local sentence = "Test "..count..text
    tts:speak(sentence,tostring(count))
    --count = count + 1
    end)
     
    tts:addEventListener(Event.TTS_UTTERANCE_COMPLETE,function (e)
    print("Utterance:"..e.utteranceId.." state:"..e.state.. " Count: "..count)
     
    if e.state == "start" then
    print("started")
    elseif e.state == "done" then
    count = count + 1
    local sentence = "test "..count .. text
     
    function stopSpeech()
    -- local delay = math.random(2000) + 1000
    -- Timer.delayedCall(delay,function()
    -- tts:stop()
    -- end
    -- )
    end
     
    if count > 10 then
    tts:shutdown()
    tts = nil
    elseif (count % 2) == 0 then
    stopSpeech()
    tts:setLanguage("en-gb")
    --tts:setSpeed(math.random(30)/10)
    tts:speak(sentence,tostring(count))
    else
    stopSpeech()
    tts:setLanguage("en-us")
    --tts:setPitch(math.random(30)/10)
    tts:speak(sentence,tostring(count))
    end
     
    end
     
    end)
  • hgy29hgy29 +1 -1
    Maintainer
    @simwhi, thanks for your testing and your solution :)
    @NatWobble, yes you you have the iOS part then we can make a dual platform plugin
  • NatWobbleNatWobble +1 -1
    Member
    Here's where I am up to with the iOS plugin (plugin attached).

    Everything seems to be working consistently with the Android version except for the fact that the iOS plugin initialises instantly, so there isn't a TTS_INIT_COMPLETE event.

    I really don't know much about C++/Objective C so if anyone notices anything glaring in the code, please let me know. Specifically, I'm not really sure how to shut it down properly. Currently the shutdown event only stops the current utterance and repeatedly refreshing the player will make it crash. Any ideas would be greatly appreciated.

    The plugin requires AVSpeechSynthesis.

    I exposed a few more of the ios features:
    setVolume,
    getVolume
    setVoiceIdentifier
    getVoicesInstalled
    getPitch
    getSpeed

    Here's the sample lua code:
    require "tts"
     
    local platform =application:getDeviceInfo()
     
    print(platform)
     
    local tts=TTS.new("fr",1,1)
    local count = 1
    local text = "This is a fairly long sentence that will test the new TTS engine for Android devices."
     
    tts:addEventListener(Event.TTS_ERROR,function (e)
    print("TTS Error:"..e.error)
    end)
     
    local function stopSpeech()
    tts:stop()
     
    local function resumeSpeech()
    tts:speak("It was stopped and has been resumed")
    end
    local timerB=Timer.new(500,1)
    timerB:addEventListener(Event.TIMER,resumeSpeech)
    timerB:start()
    end
    local timer=Timer.new(9000,1)
    timer:addEventListener(Event.TIMER,stopSpeech)
     
     
    if platform =="Android" then
    tts:addEventListener(Event.TTS_INIT_COMPLETE,function ()
    print("TTS Initialised")
    local sentence = "Test "..count..text
    tts:speak(sentence,tostring(count))
    timer:start()
    end)
    else
    local sentence = "Test "..count..text
    tts:speak(sentence,tostring(count))
    timer:start()
     
    end
     
    tts:addEventListener(Event.TTS_UTTERANCE_COMPLETE,function (e)
     
    print("Utterance:"..e.utteranceId.." state:"..e.state.. " Count: "..count)
     
    if e.state == "start" then
    print("started")
    elseif e.state == "done" then
    count = count + 1
    local sentence = "test "..count .. text
     
    if count == 2 then
    tts:setLanguage("en-gb")
    tts:speak(sentence.. " in British English",tostring(count))
    elseif count == 3 then
    tts:setSpeed(0.4)
    tts:setPitch(2)
    tts:speak("The pitch and speed have been changed",tostring(count))
    elseif count == 4 and platform == "iOS" then
    tts:setSpeed(1)
    tts:setPitch(1)
    tts:setVolume(0.2)
    print("volume: "..tts:getVolume())
    tts:speak("This is quiet","3")
    tts:setVolume(1)
    elseif count ==4 then
    count=count+1
    tts:shutdown()
    elseif count == 5 and platform == "iOS" then
    tts:setVoiceIdentifier("com.apple.ttsbundle.siri_female_en-GB_premium")
    tts:speak("A new voice has been selected by the identifier, but it needs to be installed on the device.",tostring(count))
    print("pitch: "..tts:getPitch()..", speed: "..tts:getSpeed())
    local voices = tts:getVoicesInstalled()
    for i=1,#voices do
    print(voices[i]["language"]..", "..voices[i]["identifier"]..", "..voices[i]["quality"]..", "..voices[i]["name"])
    end
    elseif count > 5 then
    tts:shutdown()
    end
     
    end
     
    end)


    I'd suggest that we change it so that Event.TTS_UTTERANCE_COMPLETE does't fire at both the start and the end of the utterance and add a new event for the start (Event.TTS_UTTERANCE_STARTED).
    TextToSpeechAndroidAndIOS.zip
    5K
  • simwhisimwhi +1 -1
    Member
    @hgy29 Would it be possible to post the NDK project here (zip file) so I can use it to recreate the plugin on my PC. I'd like to start implementing some new features too, now that we also have the iOS version.

  • simwhisimwhi +1 -1 (+1 / -0 )
    Member
    @SinisterSoft Sure. I don't really mind where it is located. I'd like to be able to document and update it for all to use.

    Likes: pie

  • hgy29hgy29 +1 -1 (+2 / -0 )
    Maintainer
    Hi @simwhi, I followed @SinisterSoft advice and put it directly in gideros repo here: https://github.com/gideros/gideros/tree/master/plugins/tts
  • simwhisimwhi +1 -1
    Member
    Hi @hgy29 I understand the need to add this to the repo, but how do I compile the tts plugin without having to download the complete Gideros repo? What is the project structure on a local PC just for the tts plugin?

  • hgy29hgy29 +1 -1
    Maintainer
    Ah sorry, I thought it was easier to build the plugin with gideros sources alongside. I will try to add an autonomous Makefile to help building tts plugin alone, and you should be able to download only tts folder from github thanks to DownGit: https://minhaskamal.github.io/DownGit/#/home?url=https://github.com/gideros/gideros/tree/master/plugins/tts
  • simwhisimwhi +1 -1
    Member
    @hgy29 Really appreciate it.
  • hgy29hgy29 +1 -1
    Maintainer
    Done, I added a Makefile into tts folder.

    To build it, just grab the entire tts folder, modify the first two lines of the Makefile so that they point to the Sdk folder of your gideros installation and to the ndk-build script from android NDK installation.

    Then navigate to the tts folder from a command line and launch make.
    The resulting plugin files should end up in Build/tts subfolder.
  • simwhisimwhi +1 -1
    Member
    @hgy29

    I downloaded the android-ndk-r14b SDK and updated the Makefile as to your instructions in your previous post. Gideros 2017.04.1 is installed in the default location so I did not change it.

    Here are the first 2 lines:

    GIDEROS_SDK="/c/Program Files (x86)/Gideros/Sdk"
    NDKBUILD=cmd //c /c/dev/android-ndk-r14b/ndk-build.cmd


    I also updated the PATH system environment variable in include the new ndk path.

    Forgive my ignorance, but how to I execute the Makefile?

  • hgy29hgy29 +1 -1
    Maintainer
    I was unsure wether you were using Windows or Mac. On Mac make tool is preinstalled, but on Windows you need to install a Unix like shell with a few build tools.

    I use MSYS2 for this purpose, and I recommand to use it, you can get it here https://msys2.github.io/

    Once installed, open a MSYS2 terminal (msys2_shell.cmd) then finish your MSYS2 setup with the following commands:
    pacman –Sy pacman
    pacman –Syu
    pacman –Su
    pacman –S tar wget
    pacman –S zip unzip
    pacman –S git
    pacman –S base-devel

    after those, navigate to your tts folder (cd /drive/xxx/tts) then just type 'make'
  • simwhisimwhi +1 -1 (+1 / -0 )
    Member
    @hgy29 I followed your instructions in your previous post, and all seemed to work fine. I'll need to do some dev work on tts and test it on a device to be sure!!

    Thanks again for all your help. I wouldn't have been able to do this without your guidance.

    Simon
    PS- Does this make me an official maintainer? LOL

    Likes: hgy29

  • hgy29hgy29 +1 -1
    Maintainer
    @simwhi, I am glad it worked for you indeed!
    With this simple setup you could even start building some gideros parts from source!
    If you want to contribute to the source code, it would be best to start by forking gideros project on github, learn to use git (I use gitbash to deal with git) to checkout your repository and commit back your changes, then ask for pull requests to gideros official repository once your happy with your changes and wish to share them.
  • hgy29hgy29 +1 -1
    Maintainer
    Oh, and I wanted to explain a few things about the plugin structure:
    - in root directory you have the .gplugin file, which tells gideros what should be done to integrate your plugin into final project when it is ticked in plugin selection pane.
    - in source/common, the .cpp file is the plugin entry point from gideros point of view, its purpose is to register the plugin into lua and translate lua calls into C style calls defined in the .h file sitting alongside
    - in source/Android, you will find the android implementation of the C calls, which again translate C calls to Java/JNI
    When adding iOS support, we would add a source/iOS folder with a .m file containing the iOS implementation of the same C calls.
  • NatWobbleNatWobble +1 -1 (+2 / -0 )
    Member
    Here is a stable version of the TTS plugin for iOS, fixing some major bugs following real world testing.

    I have tried and tried to split the code into 3 files to fit the new Gideros plugin structure but have really struggled as my Objective C skills are a bit weak. The templates I followed use a single .mm file structure. So unless I figure it out or someone can help, in the meantime just adding this file to the plugin folder in XCode will work. It is dependent on the AVFoundation framework, which is already exported by Gideros.
    TextToSpeechPluginIOS.zip
    5K
  • hgy29hgy29 +1 -1 (+2 / -0 )
    Maintainer
    Thanks @NatWobble, I'll be adding this to gideros repo right now
  • Hi guys! I see that "text-to-speech" functionality is available.

    I wanted to ask - is it possible to implement "speech-to-text" in my project?
    (user says a word, app checks if it matches text)
  • Hi @Appollo14 Unfortunately not. I think it would be nice to have though.
  • It is possible to create "Speech-to-text" plugin, right? I would have to find Java Developer, who will provide api from Java side?

    I see there's some API on Android: https://developer.android.com/reference/android/speech/SpeechRecognizer.html
  • simwhisimwhi +1 -1 (+1 / -0 )
    Member
    @Apollo14 Yes, it's definitely possible. I wrote the java code for tts with a lot of help from @hgy29 with getting the plugin working with Gideros.

    I have already written a demo Android app for speech to text, but I haven't got it working with Gideros yet.

    Likes: antix

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Login with Facebook Sign In with OpenID

In this Discussion

Top Posters