Published on 9|9|2019 by David Radosta & Ola Baniukiewicz
In business, English is a universal means of communication. Virtually every major corporation has a foreign-language website and employees trained to service English-speaking clients. But what if clients decide to communicate in their native language? Artificial intelligence comes to help.
Imagine a situation: You manage a large customer service centre in an international company. One day, your employees' inboxes are flooded with emails from around the world in different languages. From the excessive use of exclamation marks and Caps Lock by customers, you conclude that the situation is very serious. Employees cannot keep up with translators translating and replying individual messages. There is a crisis – no-one quite knows what is happening and employees will not really learn an additional couple languages in the matter of minutes. Enter a new team member: an algorithm based on artificial intelligence.
In the coming years, artificial intelligence will not only perform monotonous duties, but also facilitate management of the customer's query, where clients are using a natural language.
To shed some light on the workings of a cross-lingual AI solution, let us take a look at some key concepts.
NLU (Natural Language Understanding) focuses on interpreting the meaning of words. This discipline produces solutions that enable computers to understand human language. To have AI ready to process multiple languages, you first need an appropriate lexical basis. That means creating the so-called semantic spaces – spaces of word meanings and phrases that are processed within a given language.
Until now, each language required its own semantic space. That allows for processing only that one specific language. At the heart of the new approach is to use cross-lingual machine learning to create one common space for all languages so that they can be processed simultaneously and kept in a single dataset.
In our solutions in the field of NLP (Natural Language Processing), we transfer resources from all languages to one set, explains Tomáš Brychcín, the scientist responsible for developing NLP activities at SentiSquare. – This method allows us to easily map the meanings of individual words and phrases in languages other than English. The crucial advantage of our approach is the ease of extending the language set. For example, if we have a semantic space consisting of 3 languages, we can simply add new ones in an automated way. Consequently, our algorithms (semantic spaces) learn that phrases "flight delay", "zpoždění letu" and "Flugverspätung" mean something very similar.
As a result, AI gains the ability to learn, interpret and organize messages in every available language in the world.
The greatest advantage of using common semantic space is that companies can use artificial intelligence effectively even if they have a poor database of words and phrases in a given language.
How to deal with data sparsity?
“It is best to use an example” says Brychcín. “Let us take a closer look at airlines, because they serve customers in many languages. Statistically, 50% of all conversations are conducted in English and the rest in 50 other languages. According to available data, the company has a rich database of words in English, but poor in other languages. However, using a common semantic space, artificial intelligence can effectively use the remaining 50 languages. How? The answer is simple. This is because AI is able to interpret the meaning of words in all languages based on English meanings.”
This approach has implications beyond the aviation industry. It can be used in many sectors including banking, telecommunications, call centers, health and public institutions.
Solutions in multilingual language processing will likely soon revolutionize activities in many areas of modern business. We are talking about applications using automatic machine translation, international customer service departments and electronic information databases.
SentiSquare belongs to the still-narrow group of companies developing novel solutions in the NLP area. The Czechs are at the stage of implementing a commercial cross-lingual approach at one of the clients. The company has so far tested its technology using a semantic space covering 6 languages belonging to different language families – as the company representatives say – the effect exceeded their expectations.
NLU (Natural Language Understanding) – computers' understanding of natural language
NLP (Natural Language Processing) – language processing
Semantic space – a set of meanings of words and phrases that are processed within a given language
SentiSquare is a technology company that deals with customer-generated text analysis. As one of the few companies uses artificial intelligence based on principles of distribution semantics, which provides many competitive advantages, such as language independence. The company was founded in 2014 as a spin-off by team of researchers at the Faculty of Applied Sciences of the University of West Bohemia in Pilsen. SentiSquare currently supplies technology to the contact centers of large companies such as T-Mobile, E.ON or Albert.
+420 603 400 124