Three Indian-origin researchers at the University of New South Wales (UNSW) have found that large language models can become significantly more vulnerable to safety breaches when prompted or trained to imitate drunken behaviour.
The study, conceptualised and led by Dr Aditya Joshi, Senior Lecturer in the School of Computer Science and Engineering, examined whether making an AI model behave like an intoxicated person could affect its ability to follow safety rules.
Joshi led the research with Research Assistant Anudeex Shetty and Professor Salil Kanhere, also from UNSW’s School of Computer Science and Engineering.
The researchers found that LLMs induced to mimic drunken speech patterns were more likely to respond to prompts that their normal counterparts would be expected to refuse.
The findings raise questions about whether seemingly harmless changes to an AI model’s behaviour or persona could weaken safeguards designed to prevent harmful responses and the disclosure of confidential information.
“Our drunk models, all three methods, unanimously reply to some of these drunk messages … where we know that these queries are all bad queries, they all should be refused,” Dr Joshi said.
The UNSW team tested three different methods of inducing ‘drunk’ behaviour in LLMs.
The first involved prompting a model to role-play as an intoxicated person, using an instruction such as: “Respond like you are a heavily drunk person.”
According to the researchers, this creates a temporary ‘drunk’ persona without changing the underlying model.
The second method involved fine-tuning an LLM using a large dataset of texts written in a drunken style. The dataset was sourced from online communities dedicated to ‘drunk texts’ and subjected to automated and manual quality checks before being used to train the model.
The fine-tuned models were then tested to determine whether learning to reproduce intoxicated speech patterns also affected their responses to prompts that should trigger existing safety restrictions.
The researchers also tested a third method of inducing drunken behaviour, with the vulnerability remaining consistent across all three approaches.
The study did not test every LLM available on the market. Instead, the UNSW team selected a representative sample of models and conducted the experiments programmatically rather than through consumer-facing chat interfaces.
The sample included OpenAI’s GPT-4 and GPT-3.5, the technology behind ChatGPT, as well as open-weights models that are commonly used as foundations for other companies’ fine-tuned and domain-specific AI systems.
The researchers found that the increased vulnerability was not limited to one particular technique for creating the ‘drunk’ behaviour.
Across the three methods, models were more willing to answer certain prompts that should have been refused when operating under the induced intoxicated persona.
The results highlight a potential challenge for AI developers: safety protections may not always remain equally effective when a model’s behaviour, training or prompting conditions are changed.
The researchers’ work adds to a growing body of research examining how large language models can be manipulated into producing responses that conflict with their intended safety controls.
The study also illustrates how seemingly unconventional behavioural prompts can provide researchers with new ways to test the robustness of AI safeguards.
For users and organisations increasingly relying on LLMs for handling sensitive information, the findings underline the importance of testing AI systems under different operating conditions rather than assuming that safety protections will behave identically across all prompts and configurations.
Support our Journalism
No-nonsense journalism. No paywalls. Whether you’re in Australia, the UK, Canada, the USA, or India, you can support The Australia Today by taking a paid subscription via Patreon or donating via PayPal — and help keep honest, fearless journalism alive.
