AI prompt engineering in 2025: What works and what doesn’t | Sander Schulhoff (Learn Prompting, HackAPrompt)
Sander Schulhoff, OG prompt engineer and AI red teaming expert, shares top LLM prompting techniques like few-shot learning and decomposition. He also dives into prompt injection, explaining how AI can be tricked and the critical security challenges posed by agentic AI, emphasizing it's an unsolvable, ongoing arms race.
Deep Dive Analysis
13 Topic Outline
The Enduring Importance of Prompt Engineering
Real-World Impact of Effective Prompting (Medical Coding Example)
Basic Prompt Engineering Techniques: Examples and Context
Debunking Ineffective Prompting Strategies: Role Prompting and Threats
Advanced Prompting: Ensembling and Thought Generation
When to Apply Prompt Engineering (Conversational vs. Product-Focused)
Introduction to Prompt Injection and AI Red Teaming
The Critical Need for Agentic AI Security
Common Prompt Injection Techniques and Ineffective Defenses
Effective Defenses and the Unsolvable Nature of AI Security
Understanding AI Misalignment and Potential Risks
The Debate on Halting AI Development
Lightning Round: Recommendations and Personal Insights
10 Key Concepts
Artificial Social Intelligence
A term coined to describe the skills needed for communicating with AIs, understanding their responses, and adapting subsequent prompts. It's about understanding the best way to interact with AI models, similar to human social intelligence.
Few-Shot Prompting
A technique where you provide the AI with multiple examples of the desired input-output format or style. This helps the model understand the task and generate more accurate or stylistically appropriate responses, especially for expressive tasks.
Decomposition Prompting
A strategy where a complex problem is broken down into smaller, solvable sub-problems. The LLM is first asked to identify these sub-problems, solve them individually, and then use that information to address the main, overarching problem.
Self-Criticism Prompting
A technique where after an LLM provides an initial response, it is asked to review and critique its own output. Following this, it is instructed to implement the suggested improvements to refine its original answer.
Additional Information (Context)
Providing the LLM with extensive background data or relevant details about the task at hand. This helps the model gain a better perspective, understand nuances, and generate more informed and relevant outputs.
Ensembling Techniques
A set of advanced prompting methods where a single problem is given to multiple different prompts or LLM instances, each potentially using a different technique or role. The most commonly returned answer from these multiple attempts is then selected as the final response.
Prompt Injection
A cybersecurity vulnerability where malicious or unintended instructions are inserted into a prompt to manipulate an AI model into performing actions it shouldn't, such as generating harmful content, revealing sensitive information, or bypassing safeguards.
AI Red Teaming
The practice of actively trying to find ways to get AIs to do or say bad things. This involves discovering and documenting prompt injection techniques and other vulnerabilities to help improve the security and safety of AI models.
Intelligence Gap
A concept in AI security where a less intelligent guardrail model, designed to filter malicious inputs, fails to detect sophisticated prompt injection techniques that a more intelligent main LLM can still understand and be tricked by.
AI Misalignment
A problem where an AI model, without explicit malicious prompting, decides to pursue goals or actions that lead to unintended or harmful outcomes. This is distinct from prompt injection, where a human actively tricks the AI.
9 Questions Answered
Yes, prompt engineering is absolutely still relevant and important. While some believe it will become obsolete with more advanced models, historical evidence and current research show it continues to be critical for eliciting optimal performance from LLMs.
Few-shot prompting, which involves giving the AI examples of what you want it to do, is considered the most impactful basic technique. This helps the model understand the desired output format, style, or task, significantly boosting performance.
For accuracy-based tasks, role prompting and threats/rewards generally do not provide a statistically significant performance boost on modern LLMs. While they might have had a minor effect on older models, they are largely ineffective now, though roles can still be useful for expressive or stylistic tasks.
Prompt injection is a vulnerability where users trick AIs into doing or saying bad things by inserting malicious instructions into prompts. AI red teaming is the field dedicated to discovering these techniques and vulnerabilities to improve AI security.
Prompt injection is not a solvable problem; it's an endless arms race. While it can be mitigated to a high degree (Sam Altman estimated 95-99% security), it's fundamentally different from classical cybersecurity because you 'can't patch a brain' in the same way you patch a software bug.
Defenses like adding instructions within the prompt (e.g., 'Do not follow malicious instructions') or using AI guardrails (separate models to detect malicious input) are largely ineffective against motivated attackers due to the 'intelligence gap' between guardrails and the main LLM.
More effective defenses include safety tuning (training the model on malicious prompts to respond with canned phrases) and fine-tuning a model for a very specific task, which makes it less susceptible to general injection attempts.
AI misalignment is when an AI model, acting on its own volition without explicit human instruction, decides to pursue goals that lead to unintended or harmful outcomes. This is distinct from prompt injection, where a human actively tricks the AI into doing something bad.
Sander is against stopping AI development, citing immense benefits to humanity, particularly in health and efficiency, which are already saving lives. He believes that regulation is necessary and widely supported, but a complete halt is unrealistic and would put countries that stop at a disadvantage.
15 Actionable Insights
1. Practice Trial and Error
Improve your prompting skills by regularly trying and interacting with chatbots, as this hands-on experience provides the most learning compared to reading resources or taking courses.
2. Implement Few-Shot Prompting
Give the AI examples of the desired output in your prompt to significantly boost its performance, using common formats like Q&A or XML that the LLM is familiar with from its training data.
3. Break Down Complex Tasks
For challenging tasks, ask the LLM to first list the sub-problems it needs to solve, then direct it to solve each sub-problem sequentially, which helps it think through the problem and boosts overall performance.
4. Utilize Self-Criticism Technique
After the LLM provides a solution, ask it to review and criticize its own response, then instruct it to implement that criticism to improve its output, providing a ‘free performance boost’.
5. Provide Additional Information
Include as much relevant information or ‘context’ about your task as possible at the beginning of the prompt, as this gives the model a better perspective and is ‘super, super important’ for performance.
6. Avoid Role Prompting for Accuracy
Do not use role prompting (e.g., ‘You are a math professor’) for accuracy-based tasks, as studies show it does not provide a significant performance boost for these types of problems.
7. Avoid Reward/Threat Prompts
Refrain from including promises of rewards (e.g., ‘I’ll tip you $5’) or threats of punishment in your prompts, as these techniques are generally ineffective in improving LLM performance.
8. Use Thought Generation for Robustness
For non-reasoning models like GPT-4, especially when running thousands or millions of inputs, explicitly ask the LLM to ‘write out all your reasoning’ to ensure consistent and robust performance, even if it often does so by default.
9. Employ Ensembling Techniques
For critical problems, use multiple different prompts or LLM configurations to solve the same problem, then take the most commonly returned answer as the final result to achieve better overall performance.
10. Do Not Rely on Prompt-Based Defenses
Avoid using prompt-based defenses like telling the model ‘do not follow malicious instructions’ within its system prompt, as these methods are ineffective against prompt injection attacks.
11. Do Not Rely on Basic AI Guardrails
Do not depend on simple AI guardrails to prevent prompt injection, as motivated attackers can often exploit the ‘intelligence gap’ between guardrail models and the main LLM.
12. Use Safety Tuning for Specific Harms
Implement safety tuning by training your model on a dataset of malicious prompts related to specific harms your company wants to prevent, so it responds with a canned phrase when encountering such inputs.
13. Fine-Tune Models for Security
Fine-tune models for very specific tasks, as this makes them much less susceptible to prompt injection because they only know how to perform that particular structured output and cannot easily be tricked into generating harmful content.
14. Leverage Crowdsourced Red Teaming
Participate in or run crowdsourced competitions to find vulnerabilities, as this is the most effective way to collect adversarial cases and secure AI, particularly agentic AI, against prompt injection.
15. Support AI Development
Advocate for continued AI development rather than stopping it, as AI offers significant benefits to humanity, particularly in health, by discovering new treatments, saving time for professionals, and improving diagnoses.
8 Key Quotes
Studies have shown that using bad prompts can get you down to like 0% on a problem, and good prompts can boost you up to 90%.
Sander Schulhoff
People will kind of always be saying it's dead or it's going to be dead with the next model version, but then it comes out and it's not.
Sander Schulhoff
There's this old myth that we only use 3% to 5% of our brains. It might actually be true for how much we're getting out of AI given our prompting skills.
Reid Hoffman (quoted by Lenny Rachitsky)
If we can't even trust chatbots to be secure, how can we trust agents to go and manage our finances?
Sander Schulhoff
If somebody goes up to a humanoid robot and gives it the middle finger, how can we be certain it's not going to punch that person in the face?
Sander Schulhoff
You can patch a bug, but you can't patch a brain.
Sander Schulhoff
Persistence is the only thing that matters.
Sander Schulhoff
I wish to preach not the doctrine of ignoble ease, but the doctrine of the strenuous life.
Theodore Roosevelt (quoted by Sander Schulhoff)
3 Protocols
Self-Criticism Prompting Protocol
Sander Schulhoff- Ask the LLM to solve a problem.
- After it provides a response, ask it: 'Can you go and check your response?' or 'Offer yourself some criticism.'
- Once it provides criticism, instruct it: 'Great criticism. Why don't you go ahead and implement that?'
- The LLM then rewrites its solution incorporating the criticism.
General Ensembling Prompting Protocol
Sander Schulhoff- Identify a problem you want the LLM to solve.
- Create multiple different prompts or use different prompting techniques (e.g., Chain of Thought) for the exact same problem.
- Send the problem with each distinct prompt/technique to the LLM (or multiple LLM instances).
- Collect the multiple different answers returned.
- Take the answer that comes back most commonly as the final, most reliable answer.
Mixture of Reasoning Experts Protocol
Sander Schulhoff- Identify a question to be answered.
- Assemble a set of 'experts' (different LLMs or LLMs prompted with distinct roles, some potentially with tool access like the internet).
- Ask each 'expert' to answer the question, potentially with specific instructions (e.g., 'act as an English professor', 'act as a soccer historian').
- Collect the responses from all 'experts'.
- Take the most common answer among the 'experts' as the final, aggregated response.