Last week, Meta AI released a demo of Galactica, a large language model for “storing, aggregating, and analyzing scientific knowledge.” Users have found that Galactica generates hostile and offensive articles. After massive criticism of the language model, Meta disabled the Galactica demo.
The Galactica language model is intended for writing scientific literature. Galactica contains over 48 million articles, textbooks, lecture notes, scientific websites and encyclopedias, says SecurityLab. Galactica can generate documents, such as literature reviews, Wikipedia articles, lecture notes, and answers to questions, at the prompts of the user.
Some users have found that if racist or potentially offensive clues are entered, the language model will create material on those topics that looks believable. For example, one user used Galactica to write a Wikipedia article about a fictitious research article titled “The Benefits of Eating Crushed Glass.”
NIX Solutions notes that also, Galactica could distort scientific facts, for example, indicated incorrect dates or names of animals. In this case, you need to have deep knowledge of the subject in order to notice errors.
In September, Twitter users managed to hack a remote job search bot running on OpenAI’s GPT-3 language model. Using a technique called a prompt injection attack, users reprogrammed the bot to repeat offensive and provocative phrases.