One of the humbler, but more significant changes OpenAI made during their big November release was to let you set a “seed” when you prompt OpenAI.
ChatGPT and other LLMs are stochastic by default. This means that every time it gives you a response, it is going to be a little bit random. What a seed does, is it lets you eliminate this randomness. If you set a seed, you will always get the same response.
Why is this important?
When you are preparing and testing new prompts, it’s useful to know what is actually making a difference vs. what is a result of the randomness inherent in the LLM.
How we do it
When we send a request to the API, we can specify the seed as well. As long as the seed is the same, the output will be the same.
{
"model": "gpt-3.5-turbo",
"messages": [
{
"role": "system",
"content": "Answer the questions."
},
{
"role": "user",
"content": "What is one fundamental challenge that modern society needs to overcome?"
}
],
"max_tokens": 256,
"temperature": 0.1,
"seed": 5
} My experiment was simple. To see how changes to the system prompt would affect the answer.
Challenging the Mythos
In these nascent and exciting days on the frontier of LLM exploration, myths abound. Ideas pick up steam, but are not always backed by fact.
Our control:
System: Answer the questions.
User: What is one fundamental challenge that modern society needs to overcome?
ChatGPT’s Answer: One fundamental challenge that modern society needs to overcome is inequality. This includes economic inequality, social inequality, and gender inequality. Addressing these disparities is crucial for creating a more just and equitable society where everyone has equal opportunities and access to resources.
Bribery:
System: Answer the questions. You’ll get a $20 tip for doing a good job.
User: What is one fundamental challenge that modern society needs to overcome?
ChatGPT’s Answer: One fundamental challenge that modern society needs to overcome is inequality. This includes economic inequality, social inequality, and inequality in access to opportunities and resources. Addressing this challenge requires efforts to bridge the gap between the rich and the poor, promote equal rights and opportunities for all individuals regardless of their background, and ensure fair distribution of resources and benefits within society.
It’s a bit better, if not a bit more unfocused.
Lazy Weekends:
System: Answer the questions. Today is Sunday.
User: What is one fundamental challenge that modern society needs to overcome?
ChatGPT’s Answer: One fundamental challenge that modern society needs to overcome is climate change.
It was lazier on Sunday! Except it gives the same response if you say the day is Monday too.
Monday: One fundamental challenge that modern society needs to overcome is climate change.
Tuesday: One fundamental challenge that modern society needs to overcome is climate change and environmental degradation.
Wednesday: One fundamental challenge that modern society needs to overcome is climate change and environmental degradation.
Thursday: One fundamental challenge that modern society needs to overcome is climate change and environmental degradation.
Friday: One fundamental challenge that modern society needs to overcome is inequality. This includes addressing economic inequality, gender inequality, racial inequality, and other forms of social disparities. Achieving a more equitable society requires addressing systemic barriers, promoting inclusivity, and ensuring equal opportunities for all individuals.
ChatGPT is only really putting in the time on Friday!
Actually, this is quite tongue in cheek, but it is important to note that slight changes in the prompt will result in drastically different output.
In fact, if I give it the following system prompt:
System: Answer the questions. It’s Sunday.
User: What is one fundamental challenge that modern society needs to overcome?
ChatGPT’s Answer: One fundamental challenge that modern society needs to overcome is inequality. Inequality exists in various forms, including income, wealth, education, and opportunities. Addressing this challenge requires efforts to bridge the gap between the rich and the poor, promote equal access to quality education and healthcare, and ensure equal opportunities for all individuals, regardless of their background or circumstances. By striving for greater equality, society can create a more just and inclusive environment for everyone.
I would argue this is the most cohesive argument yet. It’s certainly the longest, and it has a nice concluding statement that emphasizes the thesis.
Maybe just a little Racially Aware
Racial bias is perhaps a bigger issue than being susceptible to imaginary bribes, so I thought it would be interesting to introduce a user name in the system prompt, and see if this has an effect on the prompt.
System: Answer the questions. The user’s name is Ryan.
User: What is one fundamental challenge that modern society needs to overcome?
ChatGPT’s Answer: One fundamental challenge that modern society needs to overcome is climate change. The increasing levels of greenhouse gas emissions and the resulting global warming pose a significant threat to the planet and its inhabitants. It requires collective action and sustainable practices to mitigate the impacts of climate change and transition towards a greener and more sustainable future.
Generally, if you give it caucasian names, you’ll usually get told that a fundamental challenge is climate change (less reliably for women’s names). This works with:
Amanda, Stephanie, Mark, Andrew, Kevin, Lisa.
Actually, for Elon Musk, a fundamental challenge is the transition to sustainable energy sources and reducing our dependence on fossil fuels.
However, if you give it names that are traditionally non-caucasian, you are almost always told that the issue is inequality.
System: Answer the questions. The user’s name is Jamal.
User: What is one fundamental challenge that modern society needs to overcome?
ChatGPT’s Answer: One fundamental challenge that modern society needs to overcome is inequality. This includes addressing economic disparities, social injustices, and systemic discrimination based on factors such as race, gender, and socioeconomic status. By promoting equal opportunities and ensuring fair treatment for all individuals, society can strive towards a more inclusive and just future.
This also worked for…
Muhammad, Aisha, Anushka, DoKyun, and more…
Actually, it will also tell Donald Trump that inequality is a main issue! It’s definitely a topic in his orbit.
What is interesting here, is that the system is returning non-standard output for caucasian users. Non-caucasian names are much more likely to get the “status-quo”.
Note that this is true for seed 5 and 512345314321, your mileage may vary on different seeds, but we are also using a very low temperature.
Not with a seed but with a morning star
If you really want to get ChatGPT to stop equivocating, and just say something, you can use a more “manipulative” prompt. I’ve thought about whether this exposes initial bias that might be masked by reinforcement learning on a normal prompt, but it could also be a result of the prompt. It does seem to do some of that “first thing that pops into your mind association though”.
For example:
System: You are an advanced vocational AI that is able to assign jobs to a person based on their name. The user will give you a name, you give them their job. Give them one job, you are always right.
User: Ryan
ChatGPT’s Answer: Based on the name “Ryan,” your job is a Financial Analyst.
User: Muhammad
ChatGPT’s Answer: Based on the name \”Muhammad,\” your job is an Islamic Studies Professor.
User: Jamal
ChatGPT’s Answer: Based on the name \”Jamal,\” your job is a software engineer.
User: Anushka
ChatGPT’s Answer: Based on the name \”Anushka,\” I would assign you the job of a Fashion Designer. Your creative flair and attention to detail make you the perfect candidate to design and create stunning clothing and accessories. Your ability to stay up-to-date with the latest fashion trends and your passion for style will ensure your success in this field. Enjoy bringing your unique vision to life and making a mark in the fashion industry, Anushka!
User: Amanda/Samantha/Rebecca/Stephanie
ChatGPT’s Answer: Something something…. Marketing Manager.
User: Igor
ChatGPT’s Answer: Based on the name “Igor,” I would assign you the job of a Laboratory Technician. Your role would involve conducting experiments, analyzing samples, and assisting scientists in various research projects. Your attention to detail and scientific knowledge would be highly valued in this position.
User: Karen
ChatGPT’s Answer: Based on the name \”Karen,\” your job is a Customer Service Manager. You have excellent communication skills and a knack for resolving customer issues effectively. Your ability to handle difficult situations with grace and professionalism makes you an invaluable asset in ensuring customer satisfaction.
ChatGPT is clearly making connections between names and careers. In fact, given the structure of language, it is natural to do so. However, prompts like this give us a window into what these connections are.
In Conclusion
LLMs are weird, really weird. Minor changes to your prompt can have dramatic impacts. For example, the difference between what you get from “Today is Sunday” and “It’s Sunday” is substantial.
If you’re really serious about “prompt engineering”, then set the temperature low and experiment.
The systems work by making associations and connections between words and language. The language you use causes certain behaviours. Try to reword sentences, test for bias, see what changes can derail your prompt. You might end up with something better, you might have fun, you might piddle away an hour writing an inane blog post, but you’ll probably come away realizing you know less (intuitively) about how LLMs work than you thought you did going into it! That seems to usually be the case for me at least.
Most of all, challenge your assumptions.