Skip forward into hiring notes
RosterAcorn

Statistical pay review: 3 triggers for hiring

A statistical pay review is worth considering when comparable hires have unexplained pay gaps, job comparisons are unclear, or patterns appear across offers.

This post covers three triggers that should prompt a review, a practical way to run a first pass, and the point where you should bring in outside help. It is written for recruiters and hiring managers who must explain pay differences between comparable hires. It is not legal advice.

Nothing here requires a data team. A firm with a few dozen offers a year can do most of the work with a spreadsheet, a set of job descriptions, and a willingness to write things down. The harder part is discipline: deciding what counts as comparable before you look at who is paid more, and accepting an answer when the numbers are too thin to give one.

What is a statistical pay review?

A statistical pay review checks whether a pay difference between groups of comparable hires could plausibly be chance, or whether it points to a pattern that needs an explanation. The U.S. Equal Employment Opportunity Commission (EEOC) describes this kind of analysis in its guidance on compensation discrimination.

That is different from simply comparing averages. The guidance calls a plain comparison of raw numbers or percentages descriptive statistics. A descriptive figure shows the size of a gap, but it cannot show whether the gap is larger than chance would produce. Two averages can differ widely in a small group and still be consistent with random variation, while the same margin in a large group may be far less likely to be random.

For a small firm, the practical question is narrow. You can see a difference in your records. You want to know whether it is likely a real pattern, or whether it is the ordinary scatter you would expect from a handful of offers made by different people at different times. A review gives you a disciplined way to ask that question, write down the answer, and decide what to do next. It does not decide whether a pay decision was fair. That judgment needs context a review cannot capture on its own.

First trigger: an unexplained pay gap

The first trigger is two comparable hires paid differently with no documented reason for the gap. Comparable means the jobs involve substantially the same duties, skills, and level of responsibility. It does not mean the titles match, or that the two people were hired in the same month. When you pull the offer records for a role and see one hire at a clearly higher figure than a peer doing the same work, pause.

One hire may bring several years of directly relevant experience, or may have been hired into a location where local wages run higher. Some gaps reflect a market shift between two offers, which is a legitimate reason if it was recorded at the time. A reason you recall six months later is weaker than a note from the day, because memory tends to rebuild the story around the number you ended up paying.

So the trigger is not the gap itself. It is a gap with no record behind it. Pull the two offer files, confirm the job descriptions really match, and ask whether the difference has a stated, job-related basis. If the reason is in writing, include it in the review. If not, note that before running the review described below.

A quick test helps: can you write a sentence explaining the gap using documented, job-related factors? If you cannot write that sentence, the gap is not yet explained, whatever you believe about why it exists.

Second trigger: unclear job comparisons

The second trigger is when job titles or departments differ, and you cannot yet say whether two hires are comparable at all. The EEOC's guidance notes that differences in job titles, departments, or other organizational units "may reflect meaningful differences in job content or other factors that preclude direct pay comparisons between employees." A different title can mark a real difference in the work, or it can be a label that hides identical work. You have to find out which before you compare numbers.

Small firms run into this often. A "coordinator" in operations may do the same work as an "associate" in customer success. A "lead" title may go to someone who manages nobody, while a person with a plainer title runs a team of four. Titles also drift. Someone hired as an analyst two years ago may now do the work of a manager without any change in name. The title on the offer letter is where the question starts, not where it ends.

Write down four things for each role: the main duties, the skills the role needs, the level of decision-making, and who the person reports to. Put those lists side by side. If they match closely, the two hires belong in the same comparison group even though the titles differ. If they do not, the pay difference may be explained by the job itself, and the review can stop there with a clear note of why.

If you choose the comparison group after seeing which people are paid more, the review will confirm whatever you chose to see. Record the group definition in writing so you can show later that it was not built around the result.

Third trigger: a pattern across offers

The third trigger is a pattern that appears only when you look across several hires over a period, such as a hiring year or every offer made for one role family. A single gap can be noise. A group of comparable hires where offers to one set of people run consistently lower than offers to another, for the same work, is a different kind of signal. This is where a statistical test earns its place, because it can help distinguish a consistent pattern from the random variation you would expect across a handful of decisions.

The EEOC's guidance says statistical analyses are less reliable when they cover a small number of people. That caveat matters most for small firms. If you made four offers for a role family in a year, a statistical test can return a result that looks precise and means very little. In that case the honest answer is that the data cannot support a conclusion yet.

When the data is too thin, you have two sensible options. You can keep consistent records and rerun the review when the group is larger, which may take a while depending on your hiring volume. Or you can ask someone with statistical training whether the group is big enough to test at all. Either option beats a confident-looking number built on four data points.

Keep in mind what a test shows and what it does not. A test that finds a difference unlikely to be chance tells you that something systematic is probably happening. It does not tell you the cause. The cause could be a negotiation habit, a referral channel, a timing effect, or a decision-maker who sets offers differently. Finding the cause is the next job, and it usually means looking at individual offers, not just the summary.

How do I run a first pass?

Run a first pass in four steps, and stop as soon as the answer is clear. First, define the comparison group by job content, using the approach from the second trigger. Second, list each person's documented pay factors: relevant experience, required certifications, location, and the date of the offer. Third, compare pay within the group and note the raw gap. Fourth, if the gap remains after the documented factors are taken into account, run a statistical test on whether it is larger than chance would produce.

The EEOC's guidance gives a simple comparison a limited role. It advises using a simple tool only as an initial check for whether a statistically significant pattern exists, and turning to more sophisticated, resource-intensive techniques to test any explanation offered for that pattern. The same logic applies to your firm. A first pass answers whether something is worth a closer look.

For the statistical step, a regression that includes the documented factors is a common tool. Its output shows how much of the gap remains after those factors are accounted for, and how confident you can be that the remaining gap is not chance. Many people run this in a spreadsheet, and that can be fine, but only if you can check the result by hand on a small subset and explain each step to someone who was not in the room. Write down which test you ran, what data you used, and why you chose it.

When you read the output, watch for three things. First, look at the size of the remaining gap, not only whether it passes a test. A large gap in a small group can be unreliable, and a small gap in a large group can be statistically clear but too small to matter. Second, check whether a few unusual offers drive the result. Third, check your factors. A factor that is itself a product of the offer, such as a negotiated starting level, should not be used to explain that offer.

If the gap disappears once the documented factors are included, record that and move on. If it does not, you have a finding that needs a closer look, and the next sections explain what to do with it.

What should I do when a gap survives the check?

When a gap survives the check, decide whether a job-related explanation exists that you failed to record, or whether the pay itself needs to change. These are different outcomes, and you should not move to the second until the first has been tested honestly. Start by asking the people involved. The hiring manager who made the offer may remember a factor that never reached the file, such as a scarce skill the role needed at that moment or a commitment made during negotiation. Ask neutrally, and write down the answer as they give it.

If a real, job-related reason emerges, document it now, with the date you learned it and who provided it, and make clear that it was not recorded at the time. Do not rewrite the original offer note to match the new explanation. Add the new note alongside it.

If no reason emerges, the decision is about pay. Options include adjusting the lower offer at the next review cycle, setting a pay band for the role so future offers start from the same place, or both. Adjusting pay for current staff is a larger decision, and it is worth taking advice before you act.

A review that finds a gap and leads to a documented fix is a success, even though it is uncomfortable. A review that finds a gap and leads to nothing is the outcome to avoid, because the file would then show that the firm saw the problem and chose not to act. The point of the exercise is to act on what the data shows, not to produce a clean report.

When should a small firm bring in outside help?

A small firm should bring in outside help when the comparison group is too small for a reliable test, or when the explanations for a gap start to multiply. The EEOC's guidance also lists conditions that make a pay question especially demanding, including when the number of similarly situated individuals exceeds 25, or when several explanations and defenses are expected.

Those conditions were set out for agency investigators, so do not treat them as a rulebook for your firm. They are still a useful signal of complexity. If you do have a larger group, or the explanations for a gap are complicated, a specialist will make the work faster and more reliable.

Outside help can take several forms. An employment lawyer can advise you on what the findings mean for your obligations where you operate, and whether anything needs to change before you write up conclusions. A statistician or specialist consultant can tell you whether a test is appropriate. Whichever you choose, make sure the person has no stake in the individual decisions under review, because a reviewer who made the offers is reviewing their own work.

If a review finds a gap you cannot explain with documented, job-related factors, talk to an employment lawyer before you change anyone's pay or write conclusions for leadership. Changing pay quietly, without understanding why it differed, can create new problems. A written conclusion that has not been checked can also be hard to walk back later.

What should I write down?

Write down the reason for each offer at the time you make it, in a sentence or two, and keep that note with the offer. A useful record includes the job content used for comparison, the documented factors that differ between hires, the date of the offer, and who approved it. A hiring manager who writes "offer set near the top of the band because the new hire holds the required license" makes the next review far easier than one who leaves the reason to memory.

Keep the review file separate from the offer files. It should contain the comparison group and how you defined it, the method and tool you used, the results including the uncertain ones, and the decisions you made afterward. If a test on a small group gave an unclear answer, write that down too. An honest note saying the data could not support a conclusion is worth more than a clean-looking result nobody can reproduce.

Set a rhythm so reviews happen on a schedule rather than in a panic. One workable pattern is to check comparable hires once a year, and to run an extra check whenever one of the three triggers above appears. The schedule matters less than the habit. A firm that reviews consistently learns what normal variation looks like, and a gap outside that range stands out sooner.

Finally, share the process with the people who make offers. A recruiter who knows the review will ask for a reason is more likely to record one at the time. That small change helps capture the explanation while it is still accurate.

Related guides: Daylight saving day pay: 6 timesheet review steps · ADA staffing responsibilities: 9 topics to review · Check pay equity before approving an offer: 5 checks.

Sources