What is AI's Perfect Question? Prompting Q&A

Starting to prompt, I have so many ideas and questions I'd throw them together.  Here’s the question spoken back to me:

“Writing a historical accuracy fact checking article, outline why the 1970's economic downspin with high inflation was ended by technological advances in the 1980's which spurred on the economy. Compare and contrast this to the historical assumption that Reagan's tax cut spurred the 1980s economic recovery and rapid growth. Conclude with a statement of which is validated by the history experts as the reason for the US economic growth in the 1980s, and why the assumption of tax cuts as the cause is correct or incorrect.”

And as you listen to this spoken out loud, it’s obvious how much of an overload it is to AI, much less another human being.

For AI, asking the right question is an art form. The early days of ChatGPT left many grappling with how to effectively communicate with this new technology.

The challenge wasn't in AI's capabilities, but in our human ability to frame questions that get excellent answers.

This podcast explores AI's "perfect question." Drawing from insights shared by industry leaders and experience, we go through a five-step process designed to craft questions that elicit meaningful responses from AI with context, specificity, and follow-up.

You are invited to reconsider how to approach AI, learning not just how to ask better questions, but how to engage in a more productive dialogue with these powerful tools. It's a masterclass in communication to enhance not only our use of AI but also our critical thinking skills in the digital.

Prompting Q&A Pod: What is AI’s Perfect Question?

Can someone explain AI prompting to me like I’m a 5-year-old?

Instead of wondering how the AI black box works, let's start with your intent, the context of your question, and how to frame it.

Then, follow up by challenging your response and continuously checking facts.

The concept of "deliberate practice," as defined by Anders Ericsson and collaborators, challenges the idea that some achieve expertise through sheer practice.

By stressing the quality and structure of practice rather than just the quantity:

1. Individualized Training: Deliberate practice involves training activities designed by a coach or teacher to improve performance through trial, error, and adapting, unlike general practice, which may be less targeted or structured.

2. Focused Effort: Deliberate practice requires focused activity, not just running through the paces like it's tedious work.

3. Feedback and Adjustment: Get expert feedback and make adjustments to improve. Don't do this as a task; do things with a specific goal.

Let’s walk through 5 steps to improve your prompts - the questions you ask - and get better content that you can trust. No hallucinating, all facts, by following these steps.

1. Identify Role and Setting

Imagine you're chatting with a person who holds the answers to almost all questions. But throwing everything at the person makes them choose one of the many pieces.

They need the proper questions asked in a series or a simple fashion.

Asking questions is not easy for most people, listen to the CEO of Perplexity:

Lex Fridman Podcast: Arvind Srinivas is CEO of Perplexity

Prompts result in predictions because that's what GenAI does - provides predictions based on the directions and the questions you ask.

A great source of free prompts is at Claude

Here’s one about crafting questions for an interview:

It may have a data trove of information, but without some specific guidance, direction, and context, it will make things up because you didn't ask a good question and didn't follow up.

Let’s edit my first question following this example, by focusing on the role first, then the actual question narrowed down to one specific approach:

ROLE - Who is asking the question, and in what context?

“You are an economic researcher focusing on historical facts about the 1970s and 1980s. In your research you've found that the 1970s into the early 1980s was a period of economic downturn, and something changed in the 1980s. “

Then move into the question you want to ask, and keep it focused. Beware of compound questions - asking 2-3 questions like I did - and break those questions down.

Ask those in the follow up, and start like this:

What was the US economic recovery in the early 1980s most likely caused by?

Imagine being asked the question that started this article; like most of us, you’d find one or two parts of the question to answer, and avoid the rest.

AI does the same thing, but we call it “hallucinating”, when in this case it was just a poorly framed question.

If you are frustrated with bland answers, it's not the AI; it’s the question you're asking.

2. Create a Good Question

Questions are the compass, guiding the AI through its neural networks and fetching the answers we seek. If your compass is off, you might go south when the answer you seek points to the north.

Questions shape reality. They define what we seek and what we find.

A vague question like, "Write me a good outreach email," would lead ChatGPT into guesswork, and it might return with something equally mediocre.

Here’s an example:

The final email that ChatGPT created was OK, but I had to rewrite it.

Taking the next step and writing,

“Create a short marketing email to contact a possible partner. Write in a conversational, personable way. Use natural language and phrases that a person would use in everyday conversation about COMPANY that offers PRODUCT and get them to reply.”

That prompt, with a little context and specificity, would get you closer:

·         Be specific.

·         Please keep it simple.

·         Improve the first response (see the previous pod about Engage, Refine, and Iterate).

NOTE: The first two sentences are why anyone reads an email, or an article, or anything. AI is bad at writing intro sentences.

I always write the intro sentences and the conclusion after I finish the article. ChatGPT may be good, but it’s only going to take what other people have done.

Make sure to write it yourself, at least the intro and conclusion, to make it less AI and more you.

The Power of Questions:

Before you start your prompt, explore the importance of questions.

• Asking good questions is core to communication, learning, and science.

• The type of questions you ask determines your response – let's explore the best types.

• How you frame a question leads to the kind of response you get and the quality.

• The psychology (and bias) of cultural, social, and individual biases impact the response to your questions.

Questions guide GenAI into crafting its response. Encourage curiosity by moving beyond the surface of the question.

Need to improve at questions? Start with a simple prompt like:

I'm researching XXX. Suggest three prompts I can use. Each prompt should provide a different perspective on the research, including the context.

Asking AI for prompts isn’t a bad idea, though you’ll still want to tweak them a bit to get better responses.

We call AI a “co-pilot” for a reason; it only knows what has previously been created. To make it more, find out the questions it would like, then customize it to what you want.

If you don’t know what you want, start asking that, otherwise you’ll get poor content that people will say, that’s AI!

Understanding the types of questions to ask can help you do these prompts yourself, though you can make a question and follow-up response with GenAI to create your prompts.

Example: 3 Good Types of Questions About AI

1. Open-ended: Questions that don't have a simple 'yes' or 'no.'

2. Probing: Going deeper into a topic, often a follow-up to an open-ended question.

3. Specific/Closed-ended questions: These questions require precise, factual answers and are helpful when you need concrete information

For example, "What is the boiling point of water at sea level? Remember, when asking questions, consider:

·         The intended outcome – are you looking to validate a point of view or find original answers?

·         Who is the audience for this question, and who is asking the question? For example, asking about AI will vary significantly between beginners trying to understand the subject and a trained data science engineer.

·         The context matters and is most often overlooked, like:

• Temporal:  when something happens,  like historical or chronological circumstances.

• Spatial: The physical location where something occurs.

• Social: The cultural and societal links influence how something is understood.

• Situational: The conditions relevant to the event or concept.

• Cultural: The beliefs, values, and practices describing a society or group.

• Psychological: The mental and emotional state of the individuals involved in a situation may affect how they perceive or interpret it.

Open-Ended Question Examples:

Example: "What’s the impact of the project?"

Solution: Provide context and clarity. Ask,

"What are the economic impacts of the renewable energy project in rural communities?"

Follow-up Open-Ended Question (aka probing):

"Can you elaborate on the key figures or moments that were pivotal in its development since 2010?"

Open-ended questions allow for various responses and encourage detailed information or opinions.

Probing Question Examples:

Simple: "Do you think the Turing test was significant in the history of artificial intelligence?"

Follow-up Probing Question: "What other milestones or breakthroughs do you believe were equally or more important in shaping the field of AI?"

Probing questions dig deeper to explore the nuances of the topic, encouraging specific information or clarifying the initial response.

3. Ask Follow-Up Questions: The 5 Whys

One of the best ways to find follow-up questions is to write or speak the ideas. Brainstorm a bit and break it down into steps:

1.       The overview/general question to the topic;

2.       The next step is to look at what you wrote. Even prompt AI to break down your long thoughts into questions.

3.       Edit out the repeat questions; we often ask a different version of the same question.

In considering what to ask as follow-up questions, begin with where you come from:

• Challenge assumptions – your own when asking questions and GenAI's answers.

• Get a broad perspective by asking for different opinions in follow-up questions.

• Know your own biases; personal beliefs influence answers.

• Avoid confirmation bias (asking answers to confirm what you believe).

• Make the question clear and concise.

• Test questions out and adapt them if the response you get could be better.

Be sure to consider how you ask the question:

• Do your words trigger an emotional response?

• Did you ask 'why' and 'how' questions? ("Why did this happen?" asks for an opinion that GenAI doesn't have, vs. "How did this happen?" which focuses on facts.)

The 5 Whys technique gets a series of questions and the root question.

When you ask good questions, people open up and give great answers like this:

A. Method: Continuously ask "Why?" to determine the root cause or reasons for the first response.

Example:

• Initial question: "Why did the project fail?"

• Follow-Up: "Why was the project scope underestimated?"

• Next Follow-Up: "Why were the resources insufficient for the project scope?"

• Continue: Each response should lead to another "Why?" until you reach the primary cause.

B.      Ask to clear up general answers or missing pieces to the first response.

Method: Seek clarification or ask for elaboration to gain a more precise or more detailed understanding of the initial response.

Example:

• Initial question: "How does machine learning improve customer service?"

• Follow-Up: "Can you clarify how machine learning identifies customer preferences?"

• Next Follow-Up: "Could you explain the types of algorithms used for sentiment analysis in customer feedback?"

C. Scenario-Based Follow-Up

Method: Present a scenario to explore the initial response suggestions.

Example:

• Initial question: "What are the benefits of  AI in healthcare?"

• Follow-Up: "In a scenario where a hospital uses AI for diagnostic purposes, what challenges might they face?"

• Next Follow-Up: "How could these challenges be mitigated in a real-world hospital setting?"

Guide the conversation deeper, finding insights and a better understanding of the topic.

Multiple Perspectives

• Always ask the model to provide multiple perspectives.

• Use GenAI to both support and attack your perspective.

Lessons Learned from Prompts:

• Ambiguity: Vague prompts lead to general answers. Specify what you need.

• Assumptions: Assuming AI understands from a previous question is tricky. Open a new session to explore a new topic. Always provide relevant context.

• Check sources: GenAI is notorious for sharing inaccurate or nonexistent sources. It's not lying; it just doesn't have the data. Remember the law firm that didn't do this and lost its employees? Check your sources!

Mistakes:

• Leading the Model: Ask leading questions only if you have the skills to do it and are prepared to provide more input. For example, "Isn't it true that..." might not always give you the most objective answer. You get what you ask for.

• Assuming Infinite Knowledge: The model doesn't "know" everything, only the data it was trained on.

• Being Too Brief: While brevity is often good, too simple or too much jargon can make it less accurate.

4.  Review for Accuracy

Trusting an AI response is correct? That's not done, especially with experience. You want to know the correct answer. Here are some ways to do it:

WinstonAI is an excellent Accuracy Checker for AI - found this entire article was AI, which was correct.

1. Cross-reference with reliable sources:

2. Compare the AI's response with information from reputable websites, academic journals, or expert-written books. It does take time, but if your subject matters, do it.

3. Generate responses from different AI platforms on the same topic. Comparing these outputs can give you a broader perspective.

4. If recent, make sure the information is up to date.

5. AI models have knowledge cutoffs. One way is to use Perplexity.ai to check answers and sources compared to the answers you've gotten.

Perplexity also offers resources; check the date and time to ensure your answers are recent.

5. Use the Response

Now that you've tested all the ideas start putting the information out there.

1.       Run it by trusted experts if you know them or by a social community to gain feedback. Ask for it, and don't act like you are right; be looking for the right answer.

• Begin with a clear intent for what you want to know.

• If unhappy feedback with an answer, rephrase or specify further.

• For controversial or subjective topics, get more perspectives to get a rounded view.

• Remember, the model is as good as the question. A straightforward question is more likely to give a complete answer.

Resources

Based on the search results, here are the top AI accuracy checking tools:

1. Winston AI: With a 99.98% accuracy rate, Winston AI is the most accurate AI detector, detecting AI-generated content, including ChatGPT-4, Google Gemini, Claude, LLAMA, and more.

2. Originality.ai: Originality.ai is a two-in-one solution featuring an AI detector and plagiarism checker.

3. Scribbr’s AI Detector

4. QuillBot’s AI Detector

Action Items for Writing Prompts

1. Create clear and specific prompts.

2. Experiment to get varied responses; often, asking the same question twice will get you different answers. Compare answers between different LLMs; it's incredible how similar they are since they are primarily trained on the same data.

3. Understand the limits of GenAI's knowledge and understanding.

4. Remember the importance of open-ended questions for thorough answers.

Asking the right questions is a skill you can learn with GenAI.

Your part is learning to ask good questions and keep those prompts in a particular file as you grow, learn, and share with others.  

The roadmap to asking good questions lies in strategy:

1. Begin with a broad, simple question, narrow it down, like zooming in on a digital map.

2. Chisel away based on the response and

3. Ask for real-world examples when things seem abstract.

GenAI’s power lies not in its ability to think for itself but in offering perspectives based on training data. It's like having the voices of a million authors, scientists, and thinkers ready to present their take on your question.

Please don't ask what it thinks because it doesn't think; it predicts what you want!

Best Practices

• Be specific. Vague questions get general answers.

• Break down complex questions into simpler ones. Use follow-up questions to explore the complex issues; I'll often use 4 to 5 more questions.

• Ask for details and adjust the tone of the response to make it more or less formal.

• Opinions are a weakness of GenAI. Unless the question requires opinions, ask for the various sides of the opinions. Do not write an opinion piece because you'll get someone else's opinion, which is usually a weak article.

Best Examples of Questions:

• Open & Clear: "Explain the theory of relativity in simple terms a 10-year-old would understand."

• Be Specific & Contextual: "What were the primary causes of World War I from a European geopolitical perspective?"

• Sequential: Start with a general question and dive deeper based on the response. For example:

  1. "What is quantum mechanics?"

  2. "How does the double-slit experiment demonstrate quantum phenomena?"

  3. "Can you explain the concept of superposition in that context?”

 What to Ask and What Not to Ask

• Do ask: Specific factual questions, requests for summaries, language translations, etc.

• Don't ask: Personal advice, subjective questions, or illegal/contentious topics.

Keep it in context

• Frame your question in a clear context. For example, instead of "What's AI?" say "What does AI stand for, including specific examples of the different AI available?"

• Avoid using slang or informal phrases unless you're inquiring about them.

Start Here with Your Questions

By now, you might think, "Okay, so I get it—ask the right questions. But how exactly?"

Here's the actionable bit, the 'do-it-now' steps to ensure your next interaction with GenAI is a good one:

1. Draft Your Questions: Consider what you want to ask before posing it.

Start Broad: Lay the foundation. If you're curious about historical economic US recovery in the 1980s, start with " What was the US economic recovery in the early 1980s most likely caused by?"

2. Zoom In: Based on the first answer, go deeper.

Like "What's the significance of the technology in this recovery?"

3. Seek Multiple Views: Ask to provide varied views.

4. Feedback Loop: Not satisfied? Reframe, rephrase and ask again

5. Ask for Examples: After a theoretical or general answer, ask for real-world examples or case studies.

GenAI is a mirror that reflects the quality of your questions. It's not just about asking; it's about mastering the art of cross-questioning.

Notice the main difference between good questions and bad ones.

The good ones are led by. “Who,” “what,” “when,” “where”, and “how,” while “why” asks for an opinion.

Remember, you can ask GenAI to create prompts. There are tools for developing good prompts and many free sources of prompts.

You don't have to create the first prompt yourself, but it is not too difficult as you learn to do it.

Use sources like Claude's prompts to begin; while you can buy prompts, they are generally dated after a while.

Trust the ones from LLMs who are bringing you the information they’ve gathered.

Use this approach not only with GenAI but also in your business and life. Often, we get better answers when we learn the power of questions.

Because you get what you ask for, in business, life, and especially AI.