TrustLLM
The European answer to ChatGPT
AI applications need a lot of computing power. With its booster module, the Jülich supercomputer JUWELS offers Germany's most powerful platform for AI. Picture: Forschungszentrum Jülich
A large-scale language model is being developed in Europe that is intended to be more reliable, more open, more transparent and more energy-efficient than ChatGPT. The key to this is Europe's largest computer, which is currently being built in Jülich.
Ever since the American company OpenAI made the ChatGPT language engine available in November 2022, the topic has gone through the roof. Can HR professionals now outsource their references, lawyers their briefs and programmers their code to language machines? Reactions range from panic to euphoria.
There is no doubt that (artificial) intelligence is behind these large language models (LLM). However, this does not mean that they also respond intelligently. If you ask the Bing search engine chatbot what LLMs are, it answers: "Large Language Models are powerful models designed to understand and generate human language. They can analyze text, generate coherent responses and perform language-related tasks." The key word is "coherent". This means that the content is plausible, the grammar is correct, but the facts don't have to be right.
Language models work with artificial neural networks that are inspired by the way the brain works. The software is fed a huge amount of text like a virtual Nuremberg funnel. During training, the machine learns from the mountain of information by searching for patterns and outputting new content based on them. The brain imitations do not reveal whether the whole thing is correct. They only say with what probability one word follows another in a sentence. The result is therefore not free of misjudgments, manipulation possibilities, breaches of data protection, copyright or personal rights. When asked about the reliability of ChatGPT, Bing's chatbot replies: "We recommend using ChatGPT as a source of inspiration and feedback - but not as a source of information."
Anyone using the free version of ChatGPT agrees that the data may be used to further develop the model. Not everyone wants that. "Everyone hopes that such language models will increase productivity. But it would be bitter if Europe only gained these advantages by giving up data sovereignty," says Stefan Kesselheim from the Supercomputing Centre at Forschungszentrum Jülich. The market is currently dominated by American companies. "There is no doubt that Google and OpenAi in particular have a technological edge. They are the most advanced in making these language models usable," says Kesselheim.
Stefan Kesselheim, Head of Applied AI Lab, Forschungszentrum Jülich. Picture: FZJ
There are downsides to this, because while the language machines are chatting away, their producers are cautious about which data they train with which software. This is a fact that is causing concern among politicians and scientists in Europe. The European answer is called TrustLLM: "The idea is that we create a public good, i.e., language models that can be used commercially, so that Europe benefits economically," says Kesselheim. The EU is providing around 7 million euros for the development. Research institutions and companies from Germany, Sweden, Iceland, Denmark, Norway and the Netherlands are working on the three-year project.
TrustLLM aims to be better than the competition in every respect: More reliable, more open, ethically correct and, last but not least, more economical in terms of energy consumption. "It's about creating alternatives that are better suited to European values," says Kesselheim, citing the diversity of languages in Europe as an example. "The model should also work for languages that not many people speak, for example, Icelandic.” The “Trust” in the project name is part of the program: “Users should trust the product and the production process,” says Kesselheim. The aim is to have as few black boxes as possible and the option of an opt-out function: “This means that data records are not used if the owner prohibits them from being used to train a model.” On the technological side, the aim will be to train the machine to avoid “hallucinations”, i.e., factual errors, which the model sells as the truth.
The heart of the training center, the "machine hall", is currently being built in Jülich. "We are currently building Europe's largest computer, perhaps even the world's largest AI computer," says Kesselheim. The Jülich supercomputer JUPITER is set to be the first supercomputer in Europe to break the one trillion operations per second mark and thus enable breakthroughs in the use of AI.
Specifically, this involves around 6,000 interconnected, extremely powerful individual computers. They have a computing capacity roughly equivalent to that of ten million modern laptops. And they need a lot of power for this: "Training the language machine once would cost a seven-figure sum," estimates Kesselheim.
Software and hardware are therefore trimmed for efficiency, because the faster the training time is up, the less the electricity meter will read at the end. It's not easy, Kesselheim admits: "Especially on such large machines, it's a challenge to get them running so that they are fast and competitive."
To begin with, TrustLLM will concentrate on the Germanic language family. In addition to English and German, this includes Swedish, Danish and Icelandic, for example. However, the project is designed to be open so that other researchers can adopt and further develop language models for other European languages.
![[Translate to Englisch:] Bild: Adrian Yass, KIT](/assets/helmholtz_gemeinschaft/_processed_/0/d/csm_2023_084_KIT_am_Europaeischen_Quantenzentrum_beteiligt_72dpi_627f347d4e.jpg)
Readers comments