# How to A/B Test Cold Emails With Small Lists

> A practical, step-by-step way to test subject lines, openers and offers when you only send a few hundred emails a week, so you learn something real instead of chasing noise.

You changed your subject line, replies went up for a week, then dropped again. Was the new version better, or was it a lucky week? Without a proper test you can't tell, and you end up rewriting emails based on gut feeling.

This guide shows you how to run a simple A/B test on cold emails that a solo freelancer or small agency can do today, even without thousands of contacts.

## Decide what you are actually trying to improve

Pick one metric before you write anything. In cold email there are three that matter:

- **Positive replies**: people who say yes, ask a question or want a call. This is the one that pays your bills.
- **Total replies**: includes "not interested" and "remove me". Useful for spotting whether the message gets read at all.
- **Opens**: the weakest signal. Privacy features in mail apps can trigger opens automatically, so treat open rates as a rough hint, not proof.

If you can, judge tests on positive replies. Use opens only to compare subject lines, and even then, confirm with replies.

## Test one thing at a time, in this order

When you change the subject, the opener and the offer at once, a winner tells you nothing about why it won. Change one element per test. A sensible order, from biggest likely impact to smallest:

1. **The offer.** What you are proposing and how big the ask is. "Want a free audit?" versus "Worth a 10 minute call?" is a different deal, not just different wording.
2. **The first line.** The sentence shown in the inbox preview. A personal observation about their business versus a generic greeting.
3. **The subject line.** Short and plain versus specific. Our list of [cold email subject line ideas](https://getclient.pro/resources/cold-email-subject-line-ideas) gives you pairs to start with.
4. **The call to action.** A question they can answer in one word versus a calendar link.
5. **Length and format.** Three sentences versus six, plain text versus a lightly formatted layout.

The offer and the opener usually deserve your first tests because a better subject line can't save an email nobody wants to answer.

## Build two fair groups

A test is only fair if both groups are alike. Practical rules:

- **Same type of lead.** Don't send version A to dentists and version B to plumbers. Pick one niche and one region per test.
- **Random split.** Sort your list by something unrelated, like alphabetical business name, and alternate: row 1 gets A, row 2 gets B. Don't put the "best" leads in one group.
- **Same sending setup.** Same mailbox or mix of mailboxes, same days, same time window. Sending A on Monday and B on Friday tests the day, not the email.
- **Verify first.** Bounces skew results and hurt your sender reputation. Clean the list before splitting.

## Work out how many emails you need

This is where most small tests go wrong. If you send 40 emails per version and get 2 replies versus 1, that difference is noise.

A rough rule of thumb: aim for at least 100 to 200 sends per version, and don't call a winner until each version has collected a decent number of replies, ideally 10 or more. If your reply counts are tiny, the honest answer is "no clear winner yet", so keep sending.

If your volume is low, there are two ways around it:

- **Test bigger differences.** Two very different offers produce gaps you can actually see. Swapping one word won't.
- **Run the test longer.** Keep both versions going across two or three weeks of sending until you have enough data.

## Run the test step by step

Here is a version you can set up this afternoon.

### 1. Write a hypothesis

One sentence: "Opening with a comment about their Google reviews will get more positive replies than opening with my agency intro." This keeps you honest about what you are testing.

### 2. Write both versions

Keep everything identical except the element you test. Use the same merge tags in both, for example {{params.firstname}} and {{params.city}}, so personalisation doesn't become a hidden variable. If you draft with AI, read our notes on [AI-written cold emails](https://getclient.pro/blog/ai-written-cold-emails-pros-and-cons) so both versions sound like a person.

### 3. Set up two campaigns

Create campaign A and campaign B with the same follow-up sequence, same delays and same send-day limits.

### 4. Send in parallel

Launch both on the same day from the same accounts. Keep daily volume modest and steady; a sudden jump can hurt deliverability for both groups. Our guide on how to [protect your domain reputation](https://getclient.pro/blog/protect-your-domain-reputation) covers safe volumes and warm-up.

### 5. Wait for follow-ups to finish

Don't judge after day two. Let the whole sequence run, plus a few days for late replies, before you compare.

### 6. Tag replies by hand

Go through replies and mark each one positive, neutral or negative. Count positive replies per version. This takes ten minutes and is the most reliable number you have.

## Read the results without fooling yourself

| Situation | What to do |
|---|---|
| One version clearly ahead on positive replies, with solid counts | Make it your new control and test the next element |
| Close numbers or small counts | Call it a tie, keep the simpler version, test a bigger change |
| More opens but fewer replies | The subject attracts curiosity but the email doesn't deliver; fix the body |
| More replies but many are "remove me" | The message is reaching people but annoying them; rethink the offer or targeting |

Write down every test: date, niche, what changed, sends, replies and the decision. After a few months this log is worth more than any template.

## Keep your tests compliant and clean

Testing means sending more email, so the basics matter more. Include a working unsubscribe option in every version, honour opt-outs quickly and keep complaint rates low. Read up on [Gmail and Yahoo bulk sender rules](https://getclient.pro/blog/gmail-yahoo-bulk-sender-rules) and check the laws that apply where you and your recipients are based. This isn't legal advice, but a test that lands you in spam teaches you nothing.

If you want to try this with a fresh list, GetClient gives new accounts 100 free credits with no card needed; see [plans and credits](https://getclient.pro/pricing) for the rest.

## Frequently asked questions

### Can I A/B test cold emails with only one Gmail account?

Yes. Alternate leads between version A and version B and send both from the same account on the same days. Keep total volume within safe daily limits so neither version suffers from deliverability issues.

### Should I test follow-up emails too?

Only after your first email is in good shape. Once you have a winning opener, you can test follow-ups, for example a short bump versus a new angle, keeping the first email identical in both groups.

### How long should a cold email A/B test run?

Long enough for the full sequence to finish and for each version to collect a meaningful number of replies. For many small senders that means two to three weeks rather than a few days.

### Is it okay to test more than two versions at once?

You can, but each extra version splits your list further and makes results slower to trust. With small volumes, stick to two versions and run tests one after another.

### What if both versions get almost no replies?

That usually points to a bigger problem than wording: the wrong audience, a weak offer, or emails landing in spam. Check deliverability and targeting before running more copy tests.

Source: https://getclient.pro/blog/ab-test-cold-emails