As part of its successful efforts to build major mobile mojo, Facebook is now poised to add “speech-to-speech” translation to its repertoire. Having dazzled investors and advertisers, alike, with stunning growth in mobile users and accompanying advertising dollars (as reported by Opus Research’s Greg Sterling here), its acquisition of Mobile Technologies LLC shows that it is ready to bring development in-house for speech recognition and machine translation technologies to support real-time conversations that overcome language barriers.

Among providers of speech-to-speech translation, Mobile Technologies has been a singularity of sorts. Dozens of apps in the iTunes Store or Google Store provide real-time, translation services. They range from electronic “phrase books” designed to help travelers hail a cab, find the nearest train station, order a meal or make friendly conversation to more robust speech-to-speech apps that let you utter a phrase, make sure it is accurately rendered, request translation and then show it or “play” it for your foreign-language-speaking companion.

Some of the apps are remarkably accurate, though quality varies by the chosen language pair. But all have a major shortcoming when one is traveling abroad. With the exception of Mobile Technologies’ Jibbigo, they require a wireless data connection (and the associated international roaming fees) to reach sufficient computing power in “the cloud” to transcribe spoken words, translate those words to another language and then use text-to-speech rendering software in the destination language. Jibbigo, by contrast, is a relatively large, downloadable app that performs speech recognition, machine translation and text-to-speech rendering on the device. It needs no data plan (except for the initial download).

As further background, understand that speech-to-speech apps are sold in language pairs. A “free” versions of Jibbigo, available for iOS and Android devices, is the first step to adding one or more language pairs as part of an “in-app” purchase. The Top 10 in-app purchases indicate the popularity of particular pairings. In the iTunes store, they are Spanish-English ($4.99), European Bundle ($9.99), French-English ($4.99), Italian-English ($2.99), World Bundle ($24.99), Chinese-English ($4.99), Japanese-English ($4.99), German-English ($4.99), Asian Bundle ($9.99) and Korean-English ($4.99).

Mobile Technologies also has a line of products that support real-time transcription/translation of popular lectures that are streamed from the halls of colleges and universities around the world. Thinking about how this may work with Skype video or Facetime among Facebooks millions of subscribers gets very interesting and I’m sure that the product managers, like Tom Stocky who announced the acquisition on his Facebook page have a lot of great ideas in mind for overcoming language barriers on the huge social network.

Yet there are some caveats to keep in mind. As users of Google Translate know, the core  technologies are not perfect. Speech-to-Speech Translation relies on fairly accurate recognizers to generate transcripts, robust text-to-text translation to render accurate written output and text-to-speech rendering to provide human-like output. While computing power has advanced, language databases are huge and the use of “neural networks” to tackle these sorts of tasks are on the horizon,  there have not been any major “breakthroughs” in machine translation for a number of years.

Solutions providers still rely on innovative implementation of statistical models and the application of semantic rules to take input in one language and echo back in a different language. The results, more often than not, are totally serviceable, but there are occasional mis-renderings which can lead to frustration or simple laughter. Facebook with Mobile Technologies and Google with its home-grown technologies will be vying to push the envelope of multi-language social networking. Facebook now has the edge in mobile.

Categories: Articles

Tags: Machine Translation, Mobile Experience, social networks, speech processing, Facebook