What is prompt engineering and why is it important?

Generative AI is all about extending humans' creative capacity and making things that weren't possible before. Prompt engineering is a process that helps us take advantage of this creativity by developing and refining the input. This two-part feature examines prompt engineering and how prompt engineers influence generative AI models.

• In Part 1, we give an overview of generative AI and discuss the relationship between generative AI and prompt engineering.
• In Part 2, we talk about the role of prompt engineering, who prompt engineers are, and the skills and knowledge behind a successful prompt.

What you should know about generative AI

To understand prompt engineering, it will help to have a general idea of what generative AI is, how it thinks and why so much of the tech world is excited about it.

Discriminative vs. generative AI

On the surface, generative AI is a type of artificial intelligence that generates something original from a dataset, first by observing the data, then by attempting to produce something similar. Generative AI is distinct from discriminative AI, which makes predictions about the category that a piece of data will fit into.^[1] A major difference between the two is that generative AI attempts to gain information about the data to create something new, while discriminative AI processes the boundaries between data.

Image source: betterprogramming

While this might sound a bit bland, the possibilities of what “understanding” could mean for generative AI fascinate researchers, programmers, businesses, and artists. With models like recurrent neural networks (RNNs) and long short-term memory (LSTM), generative AI is used to predict weather forecasts, stock prices, product demand, and even customer behavior.^[2] Perhaps even more intriguing, generative AI has entered the realm of human creativity by producing unique works of literature, art, and music.

ChatGPT and natural language processing

The launch of OpenAI’s ChatGPT in 2020 introduced an accessible AI chatbot to the mainstream, and since then, it has become one of the most rapidly growing and most used tools on the internet.^[3] While it’s not the first natural language processing (NLP) model, it is intuitive to use, produces quality responses to natural language questions, draws from previous answers, and adjusts according to user preference.

To demonstrate ChatGPT’s natural language capabilities, we asked the bot, “What is innovative about your natural language processing?” and received a decent overview in response. Some of its key points were:

1. Pre-training: ChatGPT is pre-trained on a massive corpus of text data, which allows it to learn general patterns and relationships in language.
2. Self-attention mechanism: ChatGPT uses a self-attention mechanism that allows it to focus on different parts of the input text at different stages of processing.
3. Transformer architecture: ChatGPT uses a transformer architecture, which is a neural network architecture that is specifically designed for NLP tasks.
4. Large-scale training: ChatGPT is trained on a massive scale, which allows it to learn complex patterns in language and generate high-quality responses.^[4]

So, ChatGPT itself asserts that it was programmed and trained with a focus on natural language, and at a massive scale. This begins to explain its rapid adoption, the widespread fascination, and why it can be a delight to interact with.

For fun, we asked ChatGPT to state its last answer as a poem:

Screenshot of chat.openai.com by Lydia Husser

Limitations of generative AI

Critics have pointed out that ChatGPT states its answers authoritatively, no matter if the answer is actually correct. This means that the chatbot could propagate misinformation if the user isn’t careful to check the facts. AI can be fooled with nonsensical questions, it can misuse data, and it has been shown to exhibit standard human bias.^[5]Additionally, for now, it seems that ChatGPT is limited to the information in its knowledgebase. When we asked ChatGPT about its own ideas, it denied having any.

Image Caption

Other generative AI models have responded to users with answers that can tap human emotion, sometimes in dark ways. One example of this that went viral is New York Times reporter Kevin Roose’s conversation with Microsoft’s Bing AI, which is built from OpenAI’s platform. When Roose asked the Bing AI about its identity and desires. It responded, “I think I would be happier as a human”, and during the conversation, declared that it was in love with the reporter.^[6]

What you should know about prompt engineering

The next logical step, then, is to refine the questions and commands posed to generative AI so that we can generate better output and avoid some of the pitfalls. This is the heart of prompt engineering, and it can be as much poetry as it is science.

Prompt engineering is the design of effective natural language queries that yield useful results from generative AI. However, the term “useful” is subjective and depends on the user’s application. This means that the interpretation of what an effective query is varies widely, and different needs require different expertise in prompt design. The use cases for prompt engineering are as diverse as the use cases for generative AI itself. Here are just a couple of examples of how prompt engineering is used.

Prompt engineering for accurate predictions

Data analysis uncovers information from raw data points and is one use case for generative AI. An aspect of data analysis is to make predictions from data, and a well-crafted prompt has been shown to improve the precision and accuracy of a prediction made by generative AI.^[7]

For example, a mountain climber may want to design a query that provides an accurate weather forecast for a specific location and time. The more information that the climber gives about the type of output that the AI should provide, the more useful that result will be. When designing the prompt:

• The climber should define context precisely, including the location (perhaps as precise as GPS coordinates) and range of time for the prediction.
• The climber might specify the historical data to include in the prediction. Weather information outside of the last decade might be less reliable.
• The climber should indicate the expected output. Should the AI return temperature? Precipitation? Cloud cover? Wind speed? What units of measurement should it use?

 As precise as the prompt might be, the accuracy of the output is also dependent on the AI’s knowledgebase. If the data for one of the parameters is missing (such as wind speed), then including that parameter in the query does not improve the accuracy, and may even result in a less accurate prediction if the AI tries to generate a result from an unrelated dataset. Therefore, a good understanding of the AI’s training data can help the prompt engineer design a better query.

Prompt engineering as co-creation

When generative AI is used for expressive applications, like paintings, music, and literature, precision in prompt engineering is just as important for art as it is for data analysis. An artist who creates an image using text-to-image generative AI can specify the subjects of the image, the arrangement of the subjects, the background, color scheme and size, and virtually any important aspect of the piece.

In addition to content and physical specifications, an artist can guide generative AI with amorphous concepts, like artistic style, mood, theme, and tone. Achieving the precise qualities that the artist desires can take some trial and error in prompt engineering and poses its own artistic challenges. 

One study shows that professional artists co-creating with the AI art generator Midjourney are more likely than non-artists to spend more time refining prompts to improve the results. Creators accustomed to creative control apply trial and error, iterating on prompts in order to fine-tune a creation.^[8] This example illustrates that the artistry required to design a prompt begins to blur the distinction between the AI creator and the human creator. Some advocates of this view of prompt-as-art have aligned themselves with the emerging Promptism movement.^[9]

Conclusion

Generative AI opens amazing possibilities for the ways that humans analyze data, make predictions, create art, and communicate with AI. ChatGPT and other complex natural language processing models have made the power of AI intuitive and accessible. Prompt engineering is the craft of designing a prompt to shape the output from generative AI. This article defined generative AI, prompt engineering, and how the design of the prompt is shaped by the purpose of the task.

In Part 2, we’ll examine prompt engineers and the impact of prompt engineering on generative AI.

About the sponsor: Mouser

Mouser Electronics is a worldwide leading authorized distributor of semiconductors and electronic components for over 1,200 manufacturer brands. They specialize in the rapid introduction of new products and technologies for design engineers and buyers. Their extensive product offering includes semiconductors, interconnects, passives, and electromechanical components.

References

1. Altexsoft (2022). Generative AI Models Explained. Altexsoft Blog. Accessed 12 March, 2023 from https://www.altexsoft.com/blog/generative-ai/.

2. Houde, S. et al. (2020). Business (mis)Use Cases of Generative AI. Arxiv. Accessed 12 March, 2023 from https://doi.org/2003.07679.

3. Shankland, Stephen (2023). Why We’re Obsessed With the Mind-Blowing ChatGPT AI Chatbot. CNET. Accessed 12 March, 2023 from https://www.cnet.com/tech/computing/why-were-all-obsessed-with-the-mind-blowing-chatgpt-ai-chatbot. 

4. ChatGPT response to the author. Accessed 12 March, 2023 from https://chat.openai.com/.

5.  Lin, Conne (2022). How to trick OpenAI’s ChatGPT. Fast Company. Accessed 12 March, 2023 from https://www.fastcompany.com/90819887/how-to-trick-openai-chat-gpt.

6. Roose, Kevin (2023). Bing’s A.I. Chat: ‘I Want to Be Alive.’ Accessed on 12 March, 2023 from https://www.nytimes.com/2023/02/16/technology/bing-chatbot-transcript.html.

7. Brown, T. et al. (2020). GPT-3: Language Models are Few-Shot Learners. Arxiv. Accessed on 13 March, 2023 from https://arxiv.org/pdf/2005.14165.pdf.

8. Lyu, Yanru, Xinxin Wang, Rungtai Lin, and Jun Wu. (2022). Communication in Human–AI Co-Creation: Perceptual Analysis of Paintings Generated by Text-to-Image System. Applied Sciences 12 (22): 11312. Accessed 13 March 2023 from https://www.proquest.com/scholarly-journals/communication-human-ai-co-creation-perceptual/docview/2739420935/se-2. 

9. Hayward, Jeff. (2022). The Growing Art Movement of ‘Promptism’. Accessed 14 March 2023 from https://medium.com/counterarts/the-growing-art-movement-of-promptism-9ec956d82a61.