Self-Consistency | Artizan Software

The Idea

When you ask AI a hard question, it might reason its way to the right answer — or it might take a wrong turn somewhere. The thing is, wrong turns are usually random. Ask it again, and it'll probably take a different path.

Self-Consistency exploits this. Ask the same question five or ten times, let AI reason through it independently each time, then see which final answer comes up most often. Random mistakes cancel out. The correct answer tends to survive.

Building Blocks

This composition runs multiple instances of:

Think Step by Step

Each attempt uses step-by-step reasoning (Chain-of-Thought). Self-Consistency adds the layer of doing it multiple times and voting on the result.

See It in Action

A baker makes 45 muffins. Sells 12 in the morning, gets a return of 3, then sells 8 more in the afternoon. A new batch of 20 comes out of the oven. How many muffins are there now?

Ask AI to reason through it — five separate times

Same question, five independent attempts

Each time, AI thinks step by step but takes a slightly different path through the problem.

↓

Collect the reasoning paths

Path 1

45 − 12 = 33, then 33 + 3 = 36, then 36 − 8 = 28, then 28 + 20 = 48

Path 2

Sold total: 12 + 8 = 20. Returns: 3. Net sold: 17. 45 − 17 + 20 = 48

Path 3

45 − 12 = 33, 33 + 3 = 36, 36 − 8 = 27 (arithmetic slip), 27 + 20 = 47

Path 4

Start: 45. Changes: −12, +3, −8, +20. Total change: +3. 45 + 3 = 48

Path 5

45 − 12 + 3 − 8 + 20 = 48

↓

Count the votes

4 votes

1 vote

Answer: 48 muffins (4 out of 5 agree)

Path 3 made an arithmetic mistake (36 − 8 = 27 instead of 28). But it didn't matter — the other four paths got it right, and the majority vote caught the error automatically.

Why This Works

Hard problems can be solved multiple ways. When AI reasons through a problem, it might add numbers in a different order, group things differently, or take any of several valid paths to the answer. Most paths lead to the right answer. The occasional wrong path makes a random error.

Since errors are random — one attempt might slip on subtraction, another might misread a step — they don't all make the same mistake. The correct answer keeps showing up, while wrong answers scatter. Majority voting filters out the noise.

It's the same principle behind asking five friends for directions. If four say turn left and one says turn right, you turn left.

The Composition

Run the same question through step-by-step reasoning multiple times. Take a majority vote on the final answers. Random errors get outvoted. The correct answer survives.

The Trade-offs

More Attempts = Better Accuracy

Five attempts gives a solid boost. Ten is better. But the gains shrink after that — going from 10 to 20 helps much less than going from 1 to 5.

Cost Scales Linearly

Five attempts costs five times as much. Use this when accuracy matters enough to justify the extra cost, not for casual questions AI already handles well.

How to Apply This

Ask AI to solve the problem step by step — then do it again several more times (most tools let you regenerate responses)
Look at just the final answers from each attempt, not the reasoning
The answer that appears most often is your best bet
If no answer dominates (say, 2-2-1 split), try a few more attempts or reconsider the question

When to Use This

• Math word problems where AI sometimes gets the arithmetic wrong
• Logic puzzles with a single correct answer
• Factual questions where you want higher confidence
• Any time you've noticed AI gives different answers to the same question
• When the stakes are high enough to justify asking multiple times

When Not to Use This

• Creative writing or open-ended tasks — there's no single "correct" answer to vote on
• Questions AI already answers consistently — if every attempt gives the same answer, voting adds cost with no benefit
• Systematic errors — if AI has a fundamental misunderstanding (not a random slip), all attempts will make the same mistake and voting won't help
• Speed-critical situations — multiple attempts take more time

How It Relates

Self-Consistency builds directly on Think Step by Step. Where that technique asks AI to reason once, Self-Consistency asks it to reason many times and uses the crowd wisdom of its own attempts.

It pairs well with Show by Example — you can give AI examples of step-by-step reasoning and then run multiple attempts with voting for even better results. It's also a foundation for more advanced compositions like Active Prompting, which uses Self-Consistency to find the questions AI struggles with most.