niedev / RTranslator

RTranslator is the world's first open source real-time translation app.

Connect to someone who has the app, connect Bluetooth headphones, put the phone in your pocket and you can have a conversation as if the other person spoke your language.

Conversation mode

The Conversation mode is the main feature of RTranslator. In this mode, you can connect with another phone that uses this app. If the user accepts your connection request:

• When you talk, your phone (or the Bluetooth headset, if connected) will capture the audio.

• The audio captured will be converted into text and sent to the interlocutor's phone.

• The interlocutors' phone will translate the text received into his language.

• The interlocutors' phone will convert the translated text into audio and will reproduce it from its speaker (or by the Bluetooth headset of the interlocutor, if connected to his phone).

All this in both directions.

Each user can have more than one interlocutor so that you can translate conversations between more than two people and in any combination.

WalkieTalkie mode

If conversation mode is useful for having a long conversation with someone, this mode instead is designed for quick conversations, such as asking for information on the street or talking to a shop assistant.

This mode only translates conversations between two people, it doesn't work with Bluetooth headsets, and you have to talk in turns. It's not a real simultaneous translation, but it can work with only one phone.

In this mode, the smartphone microphone will listen in two languages (selectable in the same screen of the walkie talkie mode) simultaneously.
The app will detect in which language the interlocutor is speaking, translate the audio into the other language, convert the text into audio, and then reproduce it from the phone speaker. When the TTS has finished, it will automatically resume listening.

General

Both translation and speech recognition use Google's APIs to ensure the best possible quality.

Also, RTranslator works even in the background, with the phone on standby or when using other apps (only when you use Conversation or WalkieTalkie modes).

Download and initial configuration

To use speech recognition and translation (without it, the app can do nothing), you need to create a Google Cloud Platform account and create and get a file to associate the account with the APIs for the payment based on the latter's use. If you create the account for the first time, activating the free trial, you will have $ 300 credit to use for 3 months in Google Cloud Platform.

To create the account and get the file follow this tutorial from a computer or using the "desktop site" option on the phone, this is because the Google Cloud Platform mobile version site does not have some options necessary to complete the tutorial.

First, download the latest version of the app apk file from https://github.com/niedev/RTranslator/releases/ and install it.

Then follow these passages:

1. Go to console.cloud.google.com and sign in with your Google account or create one.

2. Activate the free trial and enter the required data (select the private account type if you do not have a company), including the credit card details (only for when the free trial will be sold out), to see how to disconnect the card read the point 13.

3. Create a new project from the top left (there is already a default project called "My First Project" but that will give problems for the operations we will perform) by clicking on "My First Project" and then on "NEW PROJECT", the name doesn't matter.

4. From the left pop-up bar, select "API and services", then "Dashboard".

5. Click on "ENABLE API AND SERVICES" and enable "Cloud Speech-to-Text API" and "Cloud Translation API".

6. Also from the left pop-up bar, select "APIs & services" again, then "Credentials".

7. Click on "Create credentials", then on "Service account key", fill out the form by creating a new service account (if it is not already present), choose at will the name, the id and the description of the account and click on "Create", then, clicking on the Role, select "Owner" then press "Create". At this point under "Service Accounts" click on the one just created (the email start with the name you choose before), click on "KEYS" at the top, then on "ADD KEY", "Create new key", select "JSON" and press "Create".

8. At this point, the key will be generated and downloaded automatically.

9. If you used a computer to do these operations, move the key file to your phone in the download folder.

10. Open the RTranslator app and after the initial configuration, click on the three dots at the top right and then on "APIs Management", then click on the button to attach the APIs Key and select the above file from the list.

11. Excellent, from now on, you can use RTranslator freely.

Account management

12. You must keep the key file safe because if someone came into its possession, he could use Google's API at your expense; keys can be deleted, created, limited, etc. always from console.cloud.google.com. If you lose the key, therefore, it will have to be deleted via the site mentioned above. At that point, the key file will no longer be valid, and to use RTranslator you will have to repeat the tutorial from step 6.

13. To disconnect the credit card from the account, from the pop-up bar on the left select "Billing", then "Account management", at the top click on "CLOSE BILLING ACCOUNT" and confirm; the API keys from now until you reopen the account (click on "REOPEN BILLING ACCOUNT" instead of the button to close it) will not work, and money will not be deducted from the credit card.

14. In general, you can manage everything from the console.cloud.google.com site. Visit the website to learn more.

N.B. The cost of the API is around 2-3 dollars per hour. Make sure you don't forget the application in the background when it is in WalkieTalkie or Conversation mode (just exit from the selected mode by pressing back or the exit button instead of pressing the home button ). Moreover, the microphone data will be sent to Google servers to be processed in order to obtain the final result (voice recognition or translation). The data mentioned above will not be saved or used for other purposes by google unless you activate the logging in the cloud platform console. For more information about it, read the Privacy section below.

APIs prices

The voice recognition rounds the cost of each request to the nearest multiple of 15 seconds. The average total cost (both translator and voice recognition), without taking into account the rounding, is 2.5 dollars per hour of usage, but both have a free usage margin per month (500,000 characters for translation and 60 minutes for speech recognition). Also for WalkieTalkie must add the language detection costs to the translation. To learn more:

cloud.google.com/speech-to-text/pricing

cloud.google.com/translate/pricing

The translator uses API Translation v2; voice recognition uses standard models, data logging is disabled by default. If you want to reduce voice recognition costs, you can activate data logging. To learn more, visit   cloud.google.com/speech-to-text/docs/enable-data-logging.

Supported languages

The languages supported (excluding variants) are as follows:

Bengali, Czech, Chinese, Korean, Danish, Finnish, French, Japanese, Greek, Hindi, Indonesian, English, Italian, Khmer, Nepalese, Dutch, Polish, Portuguese, Romanian, Russian, Sinhalese, Slovak, Spanish, Sundanese, Swedish, German, Thai, Turkish, Ukrainian, Hungarian, Vietnamese.

Privacy

Privacy is a fundamental right. That's why RTranslator does not collect any personal data (I don't even have a server). For more information, read the privacy policy.

Concerning audio data and transcripts sent to Google for translation and speech recognition, they are sent only when speaking in WalkieTalkie mode or Conversation mode and the microphone becomes clearer, so only when necessary. Also, the data are used by Google only to carry out these operations and in no other way.

On the other hand, if data logging for Google Cloud Speech is activated, the audio data sent will be used by Google only to improve its products and services. For more information, read https://cloud.google.com/speech-to-text/docs/data-logging#data-security for Google Cloud Speech and https://cloud.google.com/translate/data-usage for Google Cloud Translation API.

Libraries

RTranslator uses three open-source libraries, one for communication between devices, another for selecting and cropping the profile image from the gallery and another for the cost chart.

The three libraries are BluetoothCommunicator, GalleryImageSelector and GraphView respectively. See their GitHub pages and sample apps for more details.

Bugs and problems

• For some languages, the TTS does not work. Reinstall the text-to-speech engine to resolve.
• On some devices, there are device search problems.
• Sometimes at the first start, the app notifies that Bluetooth LE is not supported. If your device supports Bluetooth LE, the next time you start the app, the message should no longer appear.

If you have found any other bug please report it by writing an email to [email protected]

Enjoy your simultaneous translator.