How To Work With Your AI: How To Ask AI Questions and Make Sense of What You Get
Part of a series | AI Insights
The first article in this three-part series, What AI Is and What It Does, introduced the technology and provided essential definitions. This article offers guidance and practical examples of how to prompt AI and evaluate the results.
Key takeaways
Artificial intelligence (AI) and machine learning often operate invisibly to enhance task efficiency.
Large Language Models (LLMs) are pattern detection and prediction machines that generate responses based on statistical probabilities.
Predictions provide helpful information for decision-making, but user context and risk assessment are essential for effectiveness.
AI responses differ from traditional search; they generate unique answers each time based on input prompts rather than retrieving matched data.
Clear goals lead to better results when interacting with AI; asking clarifying questions can help refine prompts.
Effective prompts may require iteration and experimentation to achieve desired outcomes.
Some of the AI or machine learning we use is invisible. It's built into the tools to perform specific tasks or functions more effectively. But working with AI sometimes involves asking questions, creating prompts, and interacting directly with AI through large language models (LLMs). LLMs process huge amounts of information to generate answers to questions, organize information, create and edit text and images, and perform tasks.
When interacting with LLMs it's important to remember that AI systems are fundamentally pattern-detection and prediction machines. Understanding some things about prediction will help you use LLMs more effectively and get better responses.
Using predictions
Predictions are statistical probabilities that calculate likelihood or chance. When you flip a coin, there's a 50% chance that it could be heads or tails. That's because there are two possible outcomes, each with equal likelihood. When there are many possible outcomes and different likelihoods, math gets more complex. But computers are great at complex math, which is partly why prediction makes sense as part of the design.
A familiar place we encounter predictions is weather information. AI systems can evaluate weather patterns and data about conditions like air pressure, humidity, or wind to predict if it will rain and what the temperature is likely to be. Usually, that's enough to know what coat to wear. Weather predictions are not always accurate in the moment, but they give us enough information to be comfortable most of the time.
Assessing risk is also an important part of using predictions effectively. If there is a 40% chance of rain, we might not bring an umbrella. But if there's a 40% chance the plane will crash, we may decide not to go. The more that's at stake, the more accurate predictions we want, or at least additional information so we can assess the situation more thoroughly.
Predictions offer helpful information to help us make decisions. But we must also bring our own knowledge, context, and risk assessment to the decision.
Predictions are different than search
We're used to searching where we ask a question, then the search engine finds information that matches our request and delivers it. When we ask the same question multiple times, we should get the same results each time.
AI prediction works differently. When we create a prompt for an LLM, it translates the prompt into a form it can use, which are numerical "tokens." The LLM then processes the tokens through neural networks to detect how those tokens relate to the patterns and data it already has. It then predicts the tokens to use for the response and assembles them in sequence, piece by piece. Each piece is a new prediction. The tokens are translated into words and that forms the response.
It can seem like magic, but it’s sophisticated computer processing, complex math, and mind-blowing amounts of data. But it's not finding and retrieving information that matches the question.
Instead, with AI responses, we get a newly created response that's built word by word. That means if we ask AI the same question multiple times, we may not get the same results. It's like tossing out a ball but never quite knowing what will come back.
AI doesn't play fetch. This takes some getting used to.
Learn more about how ADP can help deliver better outcomes in the AI era
Getting answers or asking better questions; why not both?
The first question to ask when using AI is, “What are you trying to do?”
The clearer we are on what we want, the better the results. Sometimes, AI gives us answers. Sometimes those answers are exactly what we wanted. But it's also important to make sure the AI response is correct.
Often, AI can help us ask better questions, clarify our thinking, and help identify what information is missing or would be nice to have.
The best human question will get you the best AI answer.
Sandra Villanueva, Director, Product Management, ADP Assist
The important thing to remember is that AI can do much more than just answer a question or retrieve information.
How to talk to your AI
The ideal way to work with LLMs is to craft the right instructions or request. This often takes a few tries. Here are some suggestions for creating effective prompts.
1. Determine your initial request
It's always fine to start with whatever question or task you have. Just don't stop there.
A great place to begin before creating prompts is with your goal. If you're not sure what your goal is yet, start with these questions.
What do I want to know?
What do I want to happen?
What problem am I trying to solve?
What am I trying to do?
If you're not sure, ask the LLM to tell you what's important or the key factors to help understand your topic. You can also ask for source material or even the most overlooked factor in learning about the topic. Approach the topic with curiosity; consider what you don't know and ask.
Once you have a sense of what you're dealing with and what you want, you'll be in a much clearer place to ask the right questions.
For example, if you want to ask how much family leave you can take for the birth of a baby, it's important to know what other information matters. You can start broadly by asking: how long can an employee take family medical leave for the birth of a child?
Pretty soon, you will probably need to say whether you are the one giving birth because your medical condition could be an important factor, especially if there are any complications. You will also need to specify which state you live in because state family leave laws can differ from federal law and differ from each other.
Whenever you are researching compliance issues, it's important to double-check any answer you get against a reliable source. If you plan to act on your research, check with legal first. It's almost always faster and less expensive to prevent problems based on accurate legal information than to try to clean up a mistake based on incorrect information or misunderstandings.
There's often some background research you need to understand the issue or problem better before you can get to the answers you're looking for. This usually leads to more questions.
2. Refine your prompts
Don't be afraid to try things and evaluate the results. If you don't get what you thought you wanted, you may have to spend some time thinking, learning, and trying more things.
Sometimes it helps to compare the LLM results with regular search results to identify what you might be missing or to learn more about the topic so you can ask more specific questions.
Continuing with the family leave example above, let's say you've figured out that you need to state that you are in California, that your partner is pregnant, and that there are no complications at this point.
You still may or may not get a straight answer on how much leave is available. There are three types of leave that California employees can apply for: pregnancy disability leave up to 4 months, California Family Rights Act leave up to 12 weeks, and Family Medical Leave, also up to 12 weeks. But that doesn't mean you can take 4 months plus 12 weeks, then 12 more weeks.
You also must figure out whether your employer is subject to each law (usually based on how many employees work there), what qualifies for each kind of leave, and what kinds of requests and paperwork are required to take it. You may need to come up with a series of additional questions about when you want to take the leave, whether it's all at once or at separate times, and whether state or federal leave run at the same time or one after the other.
Additional questions could include asking what if the baby is premature, what if delivery is by C-section, or whether leave is extended when there are post-partum complications. Family medical leave questions are complex and evolve as circumstances change. At this point, consider asking your HR representative. They may ask an employment lawyer because these issues are very fact and situation-specific.
One thing about LLMs is that they don't care if you ask silly questions. Keep playing with it until you are comfortable and have the information you need.
LLMs can also organize information, follow specific instructions, and handle multipart questions or requests. Explore what's possible and what works best for what you want to do.
3. Understand and assess the results
This is where it's good to remember that traditional search gives you a response based on what has happened and what is known. But predictive AI will give you a response based on patterns and possibilities. While search is backward-looking, prediction is forward-looking and creates something new. That means an AI response will give you relevant information, but it may not always be factually correct.
Sometimes, we get a plausible response that looks right. And sometimes that's good enough. It's like a weather prediction; we look at it to know what coat to wear, not because we can rely on it to tell us whether or not it will be raining at 2 p.m. a week from now. It can give us a prediction, but a lot can change between now and then.
It's important to be realistic about what we know and don't know. The biggest potential problems arise when we don't know what we don't know, because AI will give us answers that can seem definitive and plausible, no matter if we left out essential facts or questions.
As we assess results and expand or refine our questions, this is where we:
Consider how accurate a response we need for the purpose.
Think about what can or is likely to change over time and how the response might change. That helps us understand the shelf life of the response we get today.
Ask what information might be missing and whether we have everything we need.
Then we want to make sure we understand what the AI has provided so we can keep refining the prompts to get there and that we have enough information to be able to validate the response.
"A wrong answer isn't a failure; it's a signal that you're not done yet."
- Sandra Villanueva, Director, Product Management, ADP Assist
4. Validate the response
Even when your answer seems right, it's a good idea to validate the content before relying on it or sending it out. Here are some questions to ask and ways to double-check the content before you use an LLM response.
Check the source links and references to confirm whether they match the response in substance.
If you find something that doesn't make sense, ask the LLM for an explanation with reference material.
Try limiting the LLM to using source material that you know is reliable or trustworthy and have it respond based on those sources.
Ask the LLM to give you a confidence score which calculates how accurate its response is.
Ask someone who knows the material to review it.
If you have ever asked several different LLMs the same thing and received very different results, it's because different tools are created with different purposes, using different data, by different people.
One of the best predictors of whether you have the right response is whether you are using the right tool designed by people who are trying to do the same thing you want to do.
Many of us are used to computers giving us facts and answers. But AI gives us opinions based on statistical likelihoods calculated from the information it has. AI doesn't "know" anything.
We can do a lot of interesting things with AI, but it's important to understand its limits. And with any HR or compliance issue, be sure to consult with humans who do know things, are familiar with the situation and legal requirements, and have the experience and wisdom to manage it.
Learn how ADP can help organizations manage workforce complexity, maintain compliance and deliver better outcomes in the AI era.
