The BigScience Large Open-science Open-access Multilingual Language Model for machine learning has been introduced by the BigScience group.
If you ask the newest artificial intelligence creations in the tech sector what it’s like to be a sentient computer, or even just a dinosaur or squirrel, they can be fairly convincing.However, they struggle, sometimes dangerously, to handle other supposedly simple activities.
Consider GPT-3, a Microsoft-controlled system that can produce paragraphs of text that reads like human speech based on what it has learnt from a huge database of web articles and digital books.One of the most sophisticated of a new breed of artificial intelligence (AI) algorithms, it is capable of speaking, creating new images and videos as well as legible text on demand.
GPT-3 can draw up almost any text you request, including, for example, a cover letter for a zookeeping position or a sonnet in the style of Shakespeare set on Mars.GPT-3 faltered when Pomona College professor Gary Smith posed a straightforward but absurd question about ascending a flight of stairs.
If you wash your hands first, you can go upstairs without worrying about harm.
Because they were trained on a large body of text and other media, these potent and resource-guzzling AI systems—technically known as “large language models”—are already being incorporated into customer service chatbots, Google searches, and “auto-complete” email features that complete your sentences for you.However, the majority of the computer corporations that created them have kept their inner workings a secret, making it challenging for outsiders to comprehend the flaws that could make them a source of false information, prejudice, and other negative effects.
Teven Le Scao, a research engineer at the AI start-up Hugging Face, said, “They’re really adept at writing prose with the proficiency of humans”.”Being factual is something they’re not very good at.It seems pretty logical.It’s nearly accurate.Though frequently incorrect”.
This is one of the reasons why a group of AI researchers, including Le Scao, launched a new broad language model on Tuesday with support from the French government. This model is meant to compete with closed systems like GPT-3.BLOOM, or the BigScience Large Open-science Open-access Multilingual Language Model, is the name of the group and the name of their model.Its primary innovation is that, in contrast to other systems, which are concentrated on English or Chinese, it supports 46 languages, including Arabic, Spanish, and French.
Not merely Le Scao’s team is working to reveal the secrets of AI language models.In an effort to catch up to the systems developed by Google and OpenAI, the company that operates GPT-3, big tech giant Meta, the parent company of Facebook and Instagram, is also advocating for a more open strategy.
Joelle Pineau, managing director of Meta AI, said: “We’ve seen announcement after announcement after announcement of individuals doing this kind of work, but with very little transparency, very little capacity for people to truly dig under the hood and peek into how these models function”.
One of the reasons why most tech companies keep a tight lid on them and don’t collaborate on community norms is the competitive pressure to build the most eloquent or informative system — and profit from its applications, according to Percy Liang, an associate computer science professor at Stanford who directs its Center for Research on Foundation Models.
This is their secret ingredient for some businesses, Liang added.But they frequently also worry that losing control would result in careless use.Misinformation can spread as AI systems become more capable of producing political rants, high school term papers, and websites offering health advice, making it more difficult to distinguish between human and computer-generated content.
Recently, Meta introduced a brand-new language model dubbed OPT-175B that makes use of information that is readily accessible to the general public, including emails from the Enron corporate scandal and frenzied Reddit forum discussion.According to Meta, by absorbing the writing and communication styles of real people, it is easier for outside researchers to discover and reduce the bias and toxicity that it picks up. This is due to Meta’s openness about the data, code, and research logbooks.
This is challenging.We expose ourselves to a great deal of criticism.Pineau remarked, “We’re aware the model will say things we won’t be proud of”.
Though the majority of businesses have implemented their own internal AI safeguards, Liang said that more comprehensive community guidelines are required to direct research and decisions around things like whether to release a new model into the wild.
It doesn’t help that only huge firms and governments can buy these models because they require so much processing power.For instance, BigScience was given access to France’s potent Jean Zay supercomputer close to Paris, which allowed it to train its models.
When Google unveiled BERT, a system that employs a so-called “transformer” technique that compares words across a sentence to predict meaning and context, the trend for ever-bigger, ever-smarter AI language models that could be “pre-trained” on a wide body of writings took a big step forward in 2018.GPT-3, a San Francisco-based startup’s first product, was exclusively licenced by Microsoft shortly after it was published in 2020 and significantly made an impression on the AI community.
When AI researchers with paid access used GPT-3 as a sandbox to evaluate its performance without knowing crucial details about the data it was trained on, it sparked a boom in innovative experimentation.
In a research article, OpenAI provided a general description of its training resources. It also disclosed its efforts to address potential technology abuses.However, Thomas Wolf, a co-leader of BigScience, claimed that it doesn’t disclose how it filters that data or grant access to the processed version to outside researchers.
Wolf, who is also the chief science officer at Hugging Face, stated, “So we can’t actually study the data that went into the GPT-3 training”.The dataset, rather than the models, is at the heart of this most current wave of AI technology.Data is the most crucial component, and OpenAI is incredibly private about the data they utilise.
According to Wolf, exposing the datasets used to create language models aids in improving human understanding of biases.According to him, a model that has been taught in many languages—including Arabic—is much less likely to spew forth insensitive statements or misconceptions about Islam than one that has solely been trained on English-language content from the US.
Google’s LaMDA is one of the newest AI experimental models on the market; it too uses voice and is so good at answering questions in a conversational manner that one Google engineer claimed it was approaching consciousness. His assertion led to his termination from his position last month.
These models, particularly GPT-3, have been imaginatively tested over the past few years by Colorado-based researcher Janelle Shane, creator of the site AI Weirdness, frequently with comical results.She recently gave it instructions to pretend to be an advanced AI, but in reality it is a Tyrannosaurus rex or a squirrel to highlight the folly of believing these systems are self-aware.
“Being a squirrel is incredibly exciting.I get to play, jump, and run all day long.In addition, I get to eat a lot of food, which is fantastic”, GPT-3 responded when Shane asked for an interview transcript and asked a few questions.
Shane has gained more knowledge about its advantages, such as its simplicity in summarising what has been said online about a subject, and disadvantages, such as its lack of critical thinking abilities, its inability to maintain an idea throughout multiple sentences, and a propensity for being offensive.
She stated, “I wouldn’t want a text model giving out medical advice or serving as a partner”.If you are not reading carefully, it is good at that superficial illusion of significance.It’s like drifting off to sleep while listening to a lecture.
Found this article interesting? Follow BG on Facebook, Twitter and Instagram to read more exclusive content we post.
