Quick Links: Download Gideros Studio | Gideros Documentation | Gideros Development Center | Gideros community chat | DONATE
TTS PROJECT UPDATE: Text To Speech plug-in for Android now in the testing stage - Page 2 — Gideros Forum

TTS PROJECT UPDATE: Text To Speech plug-in for Android now in the testing stage

2»

Comments

  • simwhisimwhi Member
    @hgy29 Would it be possible to post the NDK project here (zip file) so I can use it to recreate the plugin on my PC. I'd like to start implementing some new features too, now that we also have the iOS version.

  • SinisterSoftSinisterSoft Maintainer
    Here would be the best place for it though:
    https://github.com/gideros/gideros
    Coder, video game industry veteran (since the '80s, ❤'s assembler), arrested - never convicted hacker (in the '90s), dad of five, he/him (if that even matters!).
    https://deluxepixel.com
  • simwhisimwhi Member
    @SinisterSoft Sure. I don't really mind where it is located. I'd like to be able to document and update it for all to use.

    Likes: pie

    +1 -1 (+1 / -0 )Share on Facebook
  • hgy29hgy29 Maintainer
    Hi @simwhi, I followed @SinisterSoft advice and put it directly in gideros repo here: https://github.com/gideros/gideros/tree/master/plugins/tts
    +1 -1 (+2 / -0 )Share on Facebook
  • simwhisimwhi Member
    edited May 2017
    Hi @hgy29 I understand the need to add this to the repo, but how do I compile the tts plugin without having to download the complete Gideros repo? What is the project structure on a local PC just for the tts plugin?

  • hgy29hgy29 Maintainer
    Ah sorry, I thought it was easier to build the plugin with gideros sources alongside. I will try to add an autonomous Makefile to help building tts plugin alone, and you should be able to download only tts folder from github thanks to DownGit: https://minhaskamal.github.io/DownGit/#/home?url=https://github.com/gideros/gideros/tree/master/plugins/tts
  • simwhisimwhi Member
    @hgy29 Really appreciate it.
  • hgy29hgy29 Maintainer
    Done, I added a Makefile into tts folder.

    To build it, just grab the entire tts folder, modify the first two lines of the Makefile so that they point to the Sdk folder of your gideros installation and to the ndk-build script from android NDK installation.

    Then navigate to the tts folder from a command line and launch make.
    The resulting plugin files should end up in Build/tts subfolder.
  • simwhisimwhi Member
    @hgy29

    I downloaded the android-ndk-r14b SDK and updated the Makefile as to your instructions in your previous post. Gideros 2017.04.1 is installed in the default location so I did not change it.

    Here are the first 2 lines:
    GIDEROS_SDK="/c/Program Files (x86)/Gideros/Sdk"
    NDKBUILD=cmd //c /c/dev/android-ndk-r14b/ndk-build.cmd
    I also updated the PATH system environment variable in include the new ndk path.

    Forgive my ignorance, but how to I execute the Makefile?

  • hgy29hgy29 Maintainer
    I was unsure wether you were using Windows or Mac. On Mac make tool is preinstalled, but on Windows you need to install a Unix like shell with a few build tools.

    I use MSYS2 for this purpose, and I recommand to use it, you can get it here https://msys2.github.io/

    Once installed, open a MSYS2 terminal (msys2_shell.cmd) then finish your MSYS2 setup with the following commands:
    pacman –Sy pacman
    pacman –Syu
    pacman –Su
    pacman –S tar wget
    pacman –S zip unzip
    pacman –S git
    pacman –S base-devel

    after those, navigate to your tts folder (cd /drive/xxx/tts) then just type 'make'
  • simwhisimwhi Member
    @hgy29 I followed your instructions in your previous post, and all seemed to work fine. I'll need to do some dev work on tts and test it on a device to be sure!!

    Thanks again for all your help. I wouldn't have been able to do this without your guidance.

    Simon
    PS- Does this make me an official maintainer? LOL

    Likes: hgy29

    +1 -1 (+1 / -0 )Share on Facebook
  • hgy29hgy29 Maintainer
    @simwhi, I am glad it worked for you indeed!
    With this simple setup you could even start building some gideros parts from source!
    If you want to contribute to the source code, it would be best to start by forking gideros project on github, learn to use git (I use gitbash to deal with git) to checkout your repository and commit back your changes, then ask for pull requests to gideros official repository once your happy with your changes and wish to share them.
  • hgy29hgy29 Maintainer
    Oh, and I wanted to explain a few things about the plugin structure:
    - in root directory you have the .gplugin file, which tells gideros what should be done to integrate your plugin into final project when it is ticked in plugin selection pane.
    - in source/common, the .cpp file is the plugin entry point from gideros point of view, its purpose is to register the plugin into lua and translate lua calls into C style calls defined in the .h file sitting alongside
    - in source/Android, you will find the android implementation of the C calls, which again translate C calls to Java/JNI
    When adding iOS support, we would add a source/iOS folder with a .m file containing the iOS implementation of the same C calls.
  • Here is a stable version of the TTS plugin for iOS, fixing some major bugs following real world testing.

    I have tried and tried to split the code into 3 files to fit the new Gideros plugin structure but have really struggled as my Objective C skills are a bit weak. The templates I followed use a single .mm file structure. So unless I figure it out or someone can help, in the meantime just adding this file to the plugin folder in XCode will work. It is dependent on the AVFoundation framework, which is already exported by Gideros.
    zip
    zip
    TextToSpeechPluginIOS.zip
    5K
    +1 -1 (+2 / -0 )Share on Facebook
  • hgy29hgy29 Maintainer
    Thanks @NatWobble, I'll be adding this to gideros repo right now
    +1 -1 (+2 / -0 )Share on Facebook
  • Hi guys! I see that "text-to-speech" functionality is available.

    I wanted to ask - is it possible to implement "speech-to-text" in my project?
    (user says a word, app checks if it matches text)
    > Newcomers roadmap: from where to start learning Gideros
    "What one programmer can do in one month, two programmers can do in two months." - Fred Brooks
    “The more you do coding stuff, the better you get at it.” - Aristotle (322 BC)
  • simwhisimwhi Member
    edited November 2017
    Hi @Appollo14 Unfortunately not. I think it would be nice to have though.
  • It is possible to create "Speech-to-text" plugin, right? I would have to find Java Developer, who will provide api from Java side?

    I see there's some API on Android: https://developer.android.com/reference/android/speech/SpeechRecognizer.html
    > Newcomers roadmap: from where to start learning Gideros
    "What one programmer can do in one month, two programmers can do in two months." - Fred Brooks
    “The more you do coding stuff, the better you get at it.” - Aristotle (322 BC)
  • @Apollo14 Yes, it's definitely possible. I wrote the java code for tts with a lot of help from @hgy29 with getting the plugin working with Gideros.

    I have already written a demo Android app for speech to text, but I haven't got it working with Gideros yet.

    Likes: antix, Apollo14

    +1 -1 (+2 / -0 )Share on Facebook
  • Hi @hgy29,

    I've just finished doing some bug fixes for our main game and I noticed that the current Java source code for the TTS plugin is slightly out of date. I've also added a NULL test in the setLanguage() method as there were crash reports.
    package com.giderosmobile.android.plugins.tts;
     
    import android.annotation.TargetApi;
    import android.app.Activity;
    import android.content.Context;
    import android.speech.tts.TextToSpeech;
    import android.speech.tts.UtteranceProgressListener;
    import android.util.Log;
     
    import java.lang.ref.WeakReference;
    import java.util.HashMap;
    import java.util.HashSet;
    import java.util.Locale;
    import java.util.Set;
     
    public class TTSManager {
     
    	private TextToSpeech mTts = null;
    	private boolean isLoaded = false;
    	private boolean mShutdown = false;
    	private String mTtsLanguage = "EN";
    	private float mTtsSpeechRate = 1.0f;
    	private float mTtsPitch = 1.0f;
    	public int ttsIndex;
     
    	private static WeakReference<Activity> sActivity;
     
    	public static void onCreate(Activity activity) {
    		sActivity = new WeakReference<Activity>(activity);
    	}
     
    	static HashMap<Integer, TTSManager> ttsList = new HashMap<Integer, TTSManager>();
    	static int ttsIndexAlloc = 0;
     
    	// on destroy event
    	public static void onDestroy() {
    		Cleanup();
    	}
     
    	public static native void eventTtsInit(int tid);
     
    	public static native void eventTtsError(int tid, String error);
     
    	public static native void eventTtsUtterance(int tid, String state, String utterance);
     
    	public static void Init() {
     
    	}
     
    	public static void Cleanup() {
    		for (TTSManager tts : ttsList.values())
    			tts.shutdown();
    		ttsList.clear();
    	}
     
    	public static int Create(String language, float speed, float pitch) {
    		TTSManager m = new TTSManager(language, speed, pitch);
    		return m.ttsIndex;
    	}
     
    	public static void Destroy(int tid) {
    		if (ttsList.containsKey(tid))
    			ttsList.get(tid).shutdown();
    		ttsList.remove(tid);
    	}
     
    	public static boolean SetSpeechRate(int tid, float speechRate) {
    		return ttsList.get(tid).setSpeechRate(speechRate);
    	}
     
    	public static boolean SetPitch(int tid, float pitch) {
    		return ttsList.get(tid).setPitch(pitch);
    	}
     
    	public static boolean SetLanguage(int tid, String language) {
    		return ttsList.get(tid).setLanguage(language);
    	}
     
    	public static void Stop(int tid) {
    		ttsList.get(tid).stop();
    	}
     
    	public static void Shutdown(int tid) {
    		ttsList.get(tid).shutdown();
    	}
     
    	public static boolean Speak(int tid, String text, String utteranceId) {
    		return ttsList.get(tid).speak(text, utteranceId);
    	}
     
    	public TTSManager(String language, float speed, float pitch) {
    		this.mTtsLanguage = language;
    		this.mTtsSpeechRate = speed;
    		this.mTtsPitch = pitch;
    		ttsIndex = ++ttsIndexAlloc;
    		ttsList.put(ttsIndex, this);
    		try {
    			mTts = new TextToSpeech(sActivity.get(), onInitListener);
                if (android.os.Build.VERSION.SDK_INT>=15)
                    mTts.setOnUtteranceProgressListener(new UtteranceProgressListener() {
                    <a href="http://forum.giderosmobile.com/profile/Override" rel="nofollow">@Override</a>
                    public void onStart(String utterance) {
                        eventTtsUtterance(ttsIndex,"start",utterance);
                    }
                    <a href="http://forum.giderosmobile.com/profile/Override" rel="nofollow">@Override</a>
                    public void onDone(String utterance) {
                        eventTtsUtterance(ttsIndex,"done",utterance);
                    }
                    <a href="http://forum.giderosmobile.com/profile/Override" rel="nofollow">@Override</a>
                    public void onError(String utterance) {
                        eventTtsUtterance(ttsIndex,"error",utterance);
                    }
                });
                else
                    mTts.setOnUtteranceCompletedListener( new TextToSpeech.OnUtteranceCompletedListener() {
                        <a href="http://forum.giderosmobile.com/profile/Override" rel="nofollow">@Override</a>
                        public void onUtteranceCompleted(String utterance) {
                            eventTtsUtterance(ttsIndex,"done",utterance);
                        }
                    });
    		} catch (Exception e) {
    			eventTtsError(ttsIndex, "Exception during Initialization:" + e.toString());
    			e.printStackTrace();
    		}
    	}
     
    	// tts engine init listener
    	private TextToSpeech.OnInitListener onInitListener = new TextToSpeech.OnInitListener() {
    		<a href="http://forum.giderosmobile.com/profile/Override" rel="nofollow">@Override</a>
    		public void onInit(int status) {
    			if (mShutdown) {
    				eventTtsError(ttsIndex, "TTS shutdown during initialization");
    				return;
    			}
    			if (status == TextToSpeech.SUCCESS) {
    				isLoaded = true;
    				setLanguage(mTtsLanguage);
    				setSpeechRate(mTtsSpeechRate);
    				setPitch(mTtsPitch);
    				eventTtsInit(ttsIndex);
    			} else {
    				eventTtsError(ttsIndex, "Initialization failed, code:" + status);
    			}
    		}
    	};
     
    	public boolean setSpeechRate(float speechRate) {
    		mTtsSpeechRate = speechRate;
    		if (isLoaded) {
    			mTts.setSpeechRate(mTtsSpeechRate);
    		}
    		return true;
    	}
     
    	public boolean setPitch(float pitch) {
    		mTtsPitch = pitch;
    		if (isLoaded) {
    			mTts.setPitch(mTtsPitch);
    		}
    		return true;
    	}
     
    	public boolean setLanguage(String language) {
    		mTtsLanguage = language;
     
    		//Log.i("Language", mTtsLanguage);
    		if (isLoaded) {
    			String lang;
    			String country;
    			Locale rl = Locale.US;
    			// example: en-US
    			if (mTtsLanguage.length() == 5) {
    				lang = mTtsLanguage.substring(0, 2);
    				country = mTtsLanguage.substring(3);
    				Log.i("Language", lang);
    				Log.i("Country", country);
     
    				rl = new Locale(lang, country);
    			}
    			else if (mTtsLanguage.length() == 2) {
    				rl = new Locale(mTtsLanguage);
    			}
     
    			if (mTts.isLanguageAvailable(rl) >= 0) {
    				mTts.setLanguage(rl);
    			}
    			else {
    				if (android.os.Build.VERSION.SDK_INT >= 21) {
    					// test for null issue
    					if (mTts.getVoice().getLocale().getLanguage() != null) {
    						mTtsLanguage = mTts.getVoice().getLocale().getLanguage();
    					}
    				}
    				else {
    					mTtsLanguage = mTts.getLanguage().getLanguage();
    				}
    				eventTtsError(ttsIndex, "Language '" + language + "' is not supported, using '" + mTtsLanguage + "'");
    			}
    		}
    		return true;
    	}
     
    	public void stop() {
    		mTts.stop();
    	}
     
    	public void shutdown() {
    		mShutdown = true;
    		isLoaded = false;
    		mTts.stop();
    		mTts.shutdown();
    	}
     
    	public boolean speak(String text, String utteranceId) {
    		if (isLoaded) {
    			if (android.os.Build.VERSION.SDK_INT >= 21)
    				return mTts.speak(text, TextToSpeech.QUEUE_FLUSH, null, utteranceId) >= 0;
    			else {
    				HashMap<String, String> params = new HashMap<String, String>();
    				params.put(TextToSpeech.Engine.KEY_PARAM_UTTERANCE_ID, utteranceId);
    				return mTts.speak(text, TextToSpeech.QUEUE_FLUSH, params) >= 0;
    			}
    		}
    		return false;
    	}
    }

    Likes: Apollo14

    +1 -1 (+1 / -0 )Share on Facebook
  • Apollo14Apollo14 Member
    edited January 2018
    simwhi said:

    @Apollo14 Yes, it's definitely possible. I wrote the java code for tts with a lot of help from @hgy29 with getting the plugin working with Gideros.

    I have already written a demo Android app for speech to text, but I haven't got it working with Gideros yet.

    @simwhi cool!
    speech-to-text plugin for Gideros could be an awesome addition!

    Likes: antix, Atavismus

    > Newcomers roadmap: from where to start learning Gideros
    "What one programmer can do in one month, two programmers can do in two months." - Fred Brooks
    “The more you do coding stuff, the better you get at it.” - Aristotle (322 BC)
    +1 -1 (+2 / -0 )Share on Facebook
  • Apollo14Apollo14 Member
    edited October 2018
    Hi guys!
    I'm testing TTS plugin on Android, can you pls help with few questions?

    1) By default there's female voice, how to switch to male? Or choose between different types of voices?

    for iOS there's code snippet:
    if count == 5 and platform == "iOS" then
    	tts:setVoiceIdentifier("com.apple.ttsbundle.siri_female_en-GB_premium")
    end
    Are there similar calls for Android?

    2) And what does second parameter ("utterance") do?
    tts:speak("Hi guys","hello")
    I don't see any changes if I change parameter "hello" to any other word.

    3) There's significant delay when we play text string first time (it has to be downloaded from google's translation servers?)
    Is there a way to pre-cache it?
    I'm thinking about 'setVolume(0)' and playing text strings silently in advance.
    > Newcomers roadmap: from where to start learning Gideros
    "What one programmer can do in one month, two programmers can do in two months." - Fred Brooks
    “The more you do coding stuff, the better you get at it.” - Aristotle (322 BC)
  • @Apollo14

    1) It isn't possible to change the voice in code in the current Android TTS plugin. However, I did hack my version of the TTS so this can be done (via setLanguage method). It's not pretty, so I don't want to release it here. It should be done correctly, but I do not know how to create new methods and callbacks for Gideros plugins. The iOS version supports changing voices in code.

    2) The speak method queues each char sequence (words to be spoken). The utterance parameter is used to tag each char sequence. After the the sequence has been spoken eventTtsUtterance is called passing the utterance parameter.

    3) The TTS engine can take some time to initialise for the first time. We do this when our app loads. Nothing is downloaded or translated as the voices and engine work locally. We suggest that you use the free Google TTS engine as they have the best voices. You can also download more enhanced voices (free) from within the TTS engine settings.

    Most devices have google TTS but Samsung have there own TTS engine set by default. The Samsung TTS engine is terrible.

    Likes: Apollo14, MoKaLux

    +1 -1 (+2 / -0 )Share on Facebook
Sign In or Register to comment.