NEWS

Meta has introduced SeamlessM4T, the first universal multimodal and multilingual translation model with artificial intelligence that allows users to communicate across language barriers. SeamlessM4T is currently available to researchers and developers under the appropriate license. The metadata of SeamlessAlign, the largest open-source multimodal translation dataset with 270,000 hours of learned speech, has also been published.

Creating a universal language translator is no easy task, and current speech-to-text and text-to-speech systems cover only a small fraction of the world’s languages. Building on the achievements of researchers around the world who have been striving for many years to create a universal translator, SeamlessM4T uses a unified systems approach to increase the efficiency and quality of translations by reducing errors and delays compared to segmented model methods. It leverages learnings from these efforts to provide multilingual and multimodal translation through a unified model built from disparate oral data sources, yielding state-of-the-art results. SeamlessM4T supports:

• Speech recognition in almost 100 languages;
• Speech-to-text conversion for nearly 100 input and output languages;
• Speech-to-text conversion supporting nearly 100 input languages and 36 output languages;
• Text-to-speech translation for nearly 100 languages;
• Text-to-speech, supporting nearly 100 input languages and 35 output languages.

SeamlessM4T marks a new milestone in the development of artificial intelligence technology that bridges language gaps and facilitates connection and communication between people speaking different languages.

Companies using the language model of the GT-3.5 Turbo family can now adapt it to their needs by entering their own selected data. As a result, this customised model can match or even exceed the ability of the GPT-4 to perform highly specialised tasks.

Since the release of the TURBO family of AI models in the beginning of the year, developers and companies have been pushing to have the ability to customise the model so that it can more fully meet user requirements. For example, to create a unique chatbot to help clients by communicating with them in a certain language or in a certain way. Until now, only the GPT-3 like dаvіnсі-002 and bаbbаgе-002 models offered similar capabilities, writes Thе Vеrgе.

The advantages of this customisation include increased manageability of the model (the model follows the inputs much better); reliable formatting of the output ( when responses have to be provided in a certain format); customisation of the tone, including imitation of the brand voice, for example for advertisement or internal communications. For IT companies, the GPT-3.5 Turbo model offers the possibility for Al to create a routine voiceover or to format and finalise different fragments of a voiceover.

The GPT-3.5 Turbo model family can process up to 4,000 characters simultaneously, which according to OpenAl is twice as many as previous models. Tests show that after fine-tuning using the company’s data, the beacons became up to 90% faster.