Skip forward into hiring notes
RosterAcorn

AI screening agents: 4 vendor figures to check

Before testing an AI screening agent, weigh its credit allowance, elimination rate, auto-advance threshold, and claimed time savings against your own results. Those figures can help you plan a trial. But the vendor chose them, and they usually describe its best examples. A small staffing team with a handful of open roles cannot assume it will see the same results.

This post takes the main figures from Workable's support guide for its Workable Agent and Workable MCP Server and shows how to weigh each one before you decide whether to trial an agent in your own system. Workable is one vendor, so the advice applies to any product you are considering. Each section quotes the guide where that helps, explains what the figure can and cannot tell you, and suggests a way to check it on your own roles.

What does the 3,000 AI credit allowance actually buy you?

Every Workable ATS account comes with 3,000 complimentary AI credits. Workable says that is enough to test the agent across about three roles before you commit to a larger investment ( Workable support guide). For a small team, that sentence is the most useful number in the guide, because it shows the scale the vendor expects a trial to cover.

The figure does not say how fast the credits disappear. The rate depends on how many roles you open, how many applicants each role attracts, and which agent features you switch on. A role with two hundred applicants can use credits very differently from a role with twenty. Ask the vendor in writing how credits are counted, and ask for a usage report partway through so the allowance does not run out without warning.

Choose your three trial roles on purpose. Pick one high-volume role where screening takes the most time, one specialist role where the must-haves are precise, and one role that has been hard to fill. If the agent works on all three, you have evidence across different kinds of hiring. If you spend the credits on three easy roles, you learn very little, and the allowance will have gone on the least useful work.

Keep a log as you go. Record each role you run through the agent, the date, the knockout questions you used, and how many applicants it removed. That log becomes your own evidence base, and it is what lets you compare roles fairly later.

Finally, decide what a credit is worth to you in hours. If a trial role takes your team a long stretch to screen by hand, ask whether the agent can compress that work inside the same allowance. The answer tells you more than the credit count does. For the next cost comparison, see Workable credit bundles.

How much of the applicant pool can the agent eliminate on its own?

In one Workable customer story, custom knockout questions let the agent automatically eliminate 20% of unqualified applicants, and the customer filled three roles quickly ( Workable support guide). Read that as one customer, one set of questions, and one set of roles. The 20% shows what a carefully built question set can do. It is not a forecast for your pipeline.

The number depends on the knockout questions, and you are the one who writes them. Questions that are too strict remove people who would have done the job well, often because they ask for an exact years-of-experience figure or a credential that is really a preference. Questions that are too loose remove almost nobody and leave the recruiter with the same pile as before. A useful check is to have the hiring manager read the questions and ask whether a strong applicant could answer no to any of them without good reason.

Once the trial runs, measure elimination for each role separately. A rate that looks healthy overall can hide one role where the agent removes nearly everyone and another where it removes almost no one. Then sample the rejected applicants. Read a random set from each role and ask whether a recruiter would have kept them. Keep a count of the sample, so that when the trial ends you can say how many rejections you reviewed and how many you would have overturned.

Be careful with the word "unqualified." The agent can only judge against the criteria you give it. If the criteria are vague, the label on the rejected pile will be vague too, and a person will need to decide what it means.

How much autonomy should you give each agent?

Decide up front which agents are active on each role and how much autonomy each one gets. The guide lists sourcing, screening, and engagement as the agents you choose from. Its example is auto-advancing applicants who meet 100% of your must-haves ( Workable support guide). That example shows how much one setting matters, because it moves applicants forward without a person looking at them first.

A 100% threshold sounds safe, but its safety depends on how the must-haves were written. If a must-have is really a preference, such as a tool an applicant could learn in a month, the agent will block people who would succeed. If a must-have is loose, the agent will advance people who match on paper and struggle in the interview. The threshold can only be as sound as the list it tests, so review that list line by line before you turn on auto-advance.

A sensible sequence for a small team starts with human review on every decision. Let the agent recommend, have a recruiter confirm, and compare the recommendations with what the recruiter would have done anyway. Only when the recommendations keep matching the recruiter's judgment should the agent move applicants forward on its own, and then only on roles where you have seen the results.

Write each agent's permissions on one page, with the role names beside them, so the whole team knows what is switched on. Review that page whenever a role changes. A role that starts as high-volume can become a specialist search, and settings that suited the first version may no longer fit the second.

If the engagement agent is active on a role, read its messages as carefully as you would read a note from your own recruiter.

Are the time savings in vendor stories realistic for your team?

Only partly, until you measure your own baseline. Workable's customer stories describe each recruiter reclaiming 10 hours a week, filling a critical role in a week, and processing hundreds of applications in under two days ( Workable support guide). The guide sums up the pattern as automation handling volume so recruiters get their time back for work that needs a person. That is a fair description of what good screening automation should do. The figures, though, come from stories the vendor selected to tell.

Your own result depends on your volume, your role mix, and how long screening took before the trial. A team that screens fifteen applicants a week may not reclaim 10 hours from an agent, however good the agent is. Before the trial, record how many hours each recruiter spends on screening over a few typical weeks. Then record the same thing during the trial. The difference is your number, and it is the only one to bring to your manager or budget holder.

Count the oversight too. Reviewing recommendations, checking knockout questions, and sampling rejections all take time, and that time belongs in the trial's time total. If oversight eats a large share of the time the agent frees up, you may still gain, but you should know that before you sign a larger contract.

Watch for the quieter kind of saving. A recruiter who no longer spends mornings sorting resumes may still spend the afternoon chasing hiring managers for feedback. Ask the vendor whether the agent can help with that. Note where the hours go after the trial, so you can see whether the time you freed reached the work you wanted it to reach.

Can the promise of applicant quality and compliance be checked?

Yes, but only through your own testing. The guide says the Workable Agent and Workable MCP Server help recruiting teams do more with fewer people without sacrificing applicant quality or compliance. That is a claim, not a result. The same guide describes very different users, including a 300-plus-location healthcare network, a UK nonprofit care provider, and a 30-person remote startup. That range shows the product is used in many settings. It does not show that the product meets the rules that apply to your hiring, because those depend on where you hire and what you do with the results.

Reach claims need the same care. LinkedIn's recruiter page says that by running dozens of searches across LinkedIn, Hiring Assistant surfaces qualified people you may not typically reach. Reaching people outside your usual pool helps only if they clear your must-haves. Track applicants from wider searches separately, so you can see whether they advance at the same rate as the people you already knew how to find.

Treat quality and compliance as things to test, not accept. For quality, ask hiring managers to rate the shortlist the agent produces against the one they would have built themselves. If you can arrange it, have them rate the two without knowing which is which. For compliance, ask your legal or HR adviser which rules apply to the locations and roles in your trial, and which records you must keep about screening decisions. Then ask the vendor for written documentation on what data the agent uses, how it reaches a recommendation, and how you can export records of its decisions.

Keep the review in your own hands. Even if the vendor provides an audit trail, someone on your team should be able to explain a rejection to an applicant who asks. If nobody on the team can do that during the trial, the agent is running ahead of your ability to supervise it.

How do you tell applicants that an agent is part of the process?

Say plainly when an automated tool is part of screening, and say who reviews the results. Keep the wording short and free of jargon, and give a clear route to a person if an applicant wants to ask a question or contest a decision. Your legal adviser can confirm what, if anything, you must disclose where you hire, so check that before you publish any notice.

Clear language tends to land better than a vague line buried in a privacy notice. A short sentence in the application flow, saying that an automated tool helps review applications and a recruiter makes the final call, sets expectations before anyone is rejected. It also gives a recruiter a fair starting point if the applicant later asks how the decision was made.

Make the human route easy to find. An applicant who was filtered out should be able to reply to a real person and ask for a second look. Log those requests during the trial. The number of requests, and how many lead to a changed decision, is useful evidence for the trial review and an early warning if the knockout questions are too strict.

Be consistent across roles. If the notice appears on one job posting but not another, an applicant can reasonably ask why. Use one wording for the whole trial, and put it on every role the agent screens.

How do you measure whether the trial worked?

Set your measures before the trial starts, and compare them with your own baseline rather than a vendor's figure. Indeed's recruiting guide asks the underlying question directly: "how do I measure the success of my recruiting efforts?" ( Indeed recruiting guide). Writing your answer down first keeps you from choosing, after the fact, the measure that happens to look best.

Useful measures for a screening trial are in the table below. Record each one the same way before the trial and during it.

MeasureHow to record itWhat it tells you
Time from application to first human contactAverage for each role, before and during the trialWhether the agent speeds up the first step
Share of applicants the agent removedCount for each role during the trialHow strict the knockout questions are in practice
Share of removals a recruiter overturnedResult of the sampled rejectionsWhether the criteria match recruiter judgment
Hiring manager satisfaction with shortlistsShort rating after each shortlist, before and duringWhether quality held up
Time to fill each role Days open, for similar roles, before and duringWhether roles close faster or slower

Decide what counts as a good result before you see the data. A simple rule could be that the agent must cut screening time on each trial role without raising the number of overturned rejections or lowering hiring manager satisfaction. If it fails on one role, you have learned something useful about that role, and you can still keep the agent where it passed.

Avoid judging on a single good week. Roles vary, and a hiring surge can make any tool look better or worse than it really is. Review the measures at the end of each trial role, then compare across roles at the end of the trial.

Share the results with the people who will live with the decision. Hiring managers see the shortlists, and recruiters see the rejections, so each group should see the measures that touch its work. A recruiter who sees only a single total will not know which roles to trust.

What should you decide before the trial starts?

Decide three things before you turn anything on: which roles the trial covers, which agents are active and how much autonomy each has, and who signs off on the results. Those decisions do more to shape a useful trial than any single vendor figure.

Write them on one page and share it with every recruiter and hiring manager who touches the trial roles. Include the three roles, the knockout questions and must-haves the agent will use, the auto-advance threshold if you set one, the sampling plan for rejections, the measures you chose, and the date you will review the results. With that page in place, the trial tests your plan instead of arguing about the vendor's numbers.

When the trial ends, you will have your own figures to set beside the vendor's. Use them to decide whether to buy, and do not let the credits, the 20%, or the 100% threshold make that decision for you. A figure from someone else's account is a reason to test. Your own results are the reason to buy or walk away.

Using AI software to screen job applicants? AI Hiring Paperwork Pack is a workbook filled from your answers that shows which rules reach each tool and the wording applicants should get.

Sources