AI prompt engineering as a product manager

I have been using tools like ChatGPT more and more in my work. A key part of learning how to use these tools well is learning how to write good prompts – also known as prompt engineering. This post covers why I think prompt engineering is an important long-term skill for everyone to learn and some of the techniques that work for me in my product management work.

*ChatGPT’s cartoon representation of me giving clear instructions to an AI so that it knows what I want (aka prompt engineering)*

Prompt engineering as a long-term skill to build

When I first heard people talking about prompting being an important skill to learn, I was skeptical. I remember how being good at writing Google search queries used to be an important skill. And I remember how Google got better at giving good responses to even the laziest and most general user queries.

However, finding 10 blue links on the internet that relate to a thing is fundamentally different to the types of jobs we expect LLMs to perform. Traditional Google search helps you find the information you need to do the job. LLMs go further – finding, synthesising and summarising information. LLMs are rapidly proving themselves capable of doing tasks and jobs that felt like science fiction only a few years ago.

Someone told me a couple of years ago that AI is like having unlimited interns available to do your bidding. As the models have become more advanced, it is perhaps more like having a thousand experts. I have spent much of the past 15 years managing product and project work with teams of interns, experts and everyone in-between. Clearly communicating the work you want to get done is a timeless management skill. Having your intent understood is fundamental to a job being done well. Until AI tools can read our minds, I believe this will remain a key skill for working with LLMs, like it is now when working with people.

Engineering?

On a lighter note, I do find it kind of surprising that the industry name for writing effective natural language prompts has the word “engineering” in it. If you fancy a chuckle, I suggest Googling prompt engineering memes. One of my favourites is below.

Health warning

Even with the best prompts, AI tools as of 2025 can make mistakes, provide inconsistent results and not follow instructions. This is due to the probabilistic nature of LLM models.

I approach AI generated content with cautious optimism. Here are some personal ground rules:

I never use the outputs from an AI tool without checking them over myself first. More often than not I don’t copy and paste out of AI, preferring instead to pick out the best ideas manually and integrate them with my work. I think by writing. So if I give away too much of the writing, I give away too much of the thinking too. ChatGPT is an excellent thought partner, sounding board, idea generator and critic. But I retain agency over what I produce.

Before sharing AI generated content with people, I almost always edit it into my own style. When someone sends me something that looks like it has been written by ChatGPT, I have found myself instinctively less trusting of it. I know I am not alone in this thinking. Increasingly, something not looking like it has been written by AI is key to getting humans to pay attention to it.

When I share unedited AI generated content with people, I clearly label it as AI generated. It is important that readers of the content understand that it was made by AI, so that they can have a sense of its limitations. This is particularly important when sharing content that I lack domain expertise in, as my ability to detect inaccuracies is lower.

Writing good prompts

1. Role

Telling an AI tool to play a specific role primes it to use styles and domain knowledge associated with that role. LLMs are trained on almost everything humans have ever written. Specifying a role encourages it to come at the problem from a more specific angle.

When I am at work, I often tell ChatGPT to “act like an expert product manager with domain experience in climate policy and NLP”. This works because product management is what I do, and climate policy and natural language processing are the domains most relevant to my product.

2. Context

When LLMs answer questions, they do so based on what they already know about the world from their training, and the context provided in the prompt.

As a product manager, a big part of my role is making sure teams understand the context. A team that understands the product vision, strategy and problems to solve will deliver much more effective solutions than a team that is just given tasks to do.

The same goes for LLMs. When I ask an LLM to help me generate some product ideas, it does so in a very general way. When I give it my product strategy as context, it generates ideas that are more aligned with my strategy. This context is remembered in the chats that follow.

It is important to be aware of how much an AI can remember in a conversation before it forgets earlier parts. This is known as the context window. Every bit of information in a chat adds up: user prompts, AI responses, the content of links and attachments in chat. More content leads to more storage being used. For API users it also leads to more cost. Most AI models have a limit to the amount of storage space available for context. As this limit is hit, old context is lost. This is less of an issue now as models have quite large context windows. ChatGPT4o has a context window of 128k tokens (~100k words, >40x the length of this post). I don’t hit that limit very often, but in very long chats where I am bouncing a lot of ideas around with the AI tool, I might reshare my original context to reduce the risk that important details are overlooked.

3. Formatting the output

When responding, AI tools will format the response in the way it thinks we are most likely to understand it. But people are different. We all like to receive information in different ways. And different organisations have different standards and conventions for how certain documents are written.

So when prompting, it often makes sense to define the format of the response. For product comparisons or decisions between things, I like to see information in a table with columns as options, rows as criteria, and scores to indicate how aligned options are with different criteria. For drafting requirements docs or summarising meetings, we have a standard format we try to use in my organisation, so I tend to use those.

Another technique I like to use is “Explain this like I’m an 11 year old”. Or if I’m feeling particularly out of my depth “Explain this to me like I’m a 5 year old”. As a generalist, I often have to learn new technologies and domains fast, and this can help when I am struggling with more complex content.

4. Formatting the prompt

Would you rather read a block of text? Or something that was neatly formatted with headings, bullet points and lists? Documents are easier for humans to read when they are formatted. The same is true for AI.

I do a lot of my writing in Obsidian – a markdown based knowledge management tool. Markdown is a lightweight markup language for creating formatted text using a plain-text editor. In English, that means it is a simple way to style text using symbols, like # for headings or – for bullet points.

Markdown is easy for humans to read. Copying and pasting into ChatGPT from Obsidian preserves the markdown syntax. That makes it easy for AI to read too.

I use ## (heading 2) tags to explain the structure of my prompt: role, task, output format and context are my go-to headings.

I use ### (heading 3) tags to break down each of those further if needed. This is very useful when I share my product strategy as context.

5. Examples

If I ask you to recommend a movie or series for me, what would you recommend? What about if I tell you I like Star Wars? What about if I tell you that I like Star Wars, West Wing and Game of Thrones? The more examples you have of things I like, the more likely you are to recommend something that I also like.

The same goes for writing product requirements. Writing good requirements is often enough for engineers to understand what they need to build. Giving some specific examples of those requirements in action (i.e. test cases) often helps clarify things and remove room for ambiguity.

Now I am going to try to impress the data scientists at work by using some complex words for things. Zero shot prompting is where you provide no examples. One shot prompting is where you provide one. Few shot prompting is where you provide multiple. A paper written in 2020 titled “Language Models are Few-Shot Learners” found that language models get more accurate the more examples you give them.

Image copied from Language Models are Few-Shot Learners

That was five years ago. A lot has happened in the past 5 years in the world of generative AI, and powerful AI models are getting better at providing accurate answers without any examples. But if I was hiring a kitchen designer, I would start by showing them examples of other kitchens that I like. So intuitively, giving an AI some examples of what good (or bad) looks like feels like it can only help. The main risk with giving examples is prematurely constraining the AI’s thinking. So for idea generation with AI, it makes sense to generate at least some ideas in a zero shot way.

6. Breaking down work

If I am building a product, I don’t do it all in one go. I start with the most important problems I am trying to solve, test the riskiest assumptions and prioritise the most high impact features. I break work down into stages: discovery, ideation, prototyping, building, testing and learning. I tackle things step by step. The same is true for most other types of problems people solve. And the same is true for working with AI.

Prompting AI tools to think step-by-step has demonstrably increased ChatGPT’s accuracy at solving complex logical problems. Showing how it got to a conclusion also has the added benefit of making results easier for humans to interpret. I am not sure how important it is to use phrases like “think step-by-step” in prompts anymore, as I think newer models might be doing it by default.

Another way to break down work is to ask an AI to start by explaining how to go about solving a problem. And then ask it to follow those instructions in sequential prompts. Breaking work up into separate prompts reduces cognitive load by allowing the AI to focus on doing one task at a time and doing it well. It also makes it easier for me, the prompter, to intervene at each step and make sure that it is done to the right standard before moving on to the next one.

Most of my experimentation with AI tools has been within the official ChatGPT app. I have only spent a little bit of time using the ChatGPT API and trying to build my own workflows, connecting together different prompts with the output from one an input into another. This is something I am playing around with at the moment, but I have not had any real success with it yet to support me in my product management work. If you have, and you fancy getting in touch for a chat, then reach out.

7. Prompting for prompting

This is one of my favourite prompts:

Act like an expert prompt engineer. Assist me to turn this prompt into a better prompt using <MODEL NAME>. <INSERT ORIGINAL PROMPT>

Prompts like these have helped me learn about ways to improve my prompt. Starting conversations like this can help set the AI tool up to give a better answer the first time around. I am sure that genuine experts out there will be able to write better prompts than AIs are capable of. But for someone like me, this is an effective and low effort way to quickly make prompts better. I also know that there are lots of other people asking questions like these. All that evaluation data means AI tools are likely to give relatively good answers to this type of question.

8. Critical friend

One of the best uses of AI tools is as a thought partner and critical friend. I like to use it to generate and explore ideas. But sometimes I get the feeling it is being a little bit too nice. The pessimistic Brit within me can get a bit tired with the overly positive tone of AI tooling. OpenAI recently acknowledged their April update made this problem worse in ChatGPT.

The CPO of Anthropic (maker of Claude) likes to tell Claude “Be brutal, Claude, roast me.” He notes it is hard to get Claude to be too brutal, but this gives a marked improvement over prompts like “What could be better with this work?”. I can confirm from trying it out on this blog post that “Be brutal. Roast me.” is a very effective prompt!

My response to the above roasting:

Disagree. Plenty of structure in this bad boy. Look at those headings and numbered subheadings!
Your rules about blog post length do not concern me. I prefer long-form content, and I have had one or two compliments about my blog posts over the years
I think you might have been overfitted on American training data 😉 Here is what your buddy Gemini told me: “British humility, while not a universal trait, often manifests as self-deprecation and a preference for understatement”
I am not trying to be a thought leader or entertain thought leaders. I write primarily because writing helps me understand my own thoughts. Writing publicly forces me to raise the bar, and hopefully entertains a few people in the process

Example prompt I used recently to think through product ideas

Here is a prompt I used recently that uses some of the techniques listed above. I followed it up with more specific questions about different parts of the AI response. The end product was an opportunity solution tree in Whimsical (like Miro, except cheaper). I used this mainly to explore ideas, and didn’t end up copying and pasting anything more than a few words here and there into the final document I shared with the team.

## Role:

Act as a senior product manager at Climate Policy Radar, with domain expertise in climate policy and NLP. Based on the product strategy below, complete the following task:

## Task:

Create an Opportunity Solution Tree using the methodology from Teresa Torres described at: https://www.producttalk.org/2023/12/opportunity-solution-trees/?srsltid=AfmBOop4oYGkXYEUVhzoDj_2EvQ7eZ_Vr4W53ZwRwj6Vdyem1yu8Jxw9

The top level outcome is “Users can find and select the the result(s) most relevant to their search intent”

## Output format:

List the opportunities in a list and the solutions as sub items. Follow the style guide for writing opportunities and solutions described above

## Product strategy:

<INSERT PRODUCT STRATEGY HERE>

## Context:
<INSERT KNOWN PROBLEMS USERS ARE HAVING FINDING INFORMATION>