Duplication explained

This page covers what fallow dupes reports, how to read the output, and when to act.

Clone types

Fallow classifies clones using the standard taxonomy from clone detection research.

As you move from strict to semantic, recall increases (more clones found) but precision decreases (more potential false positives).

Detection modes in detail

● Clone group (196 lines, 2 instances) [semantic match]
  src/lib/dutch-holidays.ts:193-388
  src/lib/dutch-holidays.ts:389-584
  Renamed: holidays2024→holidays2025, year2024→year2025

Key metrics

Duplication percentage

Fraction of total source tokens that appear in at least one clone group. Computed over the full analyzed file set, not just the groups shown by --top.

Token count and line count

Each clone group reports both token count and line count.

• Tokens are language-aware units (keywords, identifiers, operators, literals). This is what the detection engine matches on.
• Lines are the source lines spanned by the clone. Useful for estimating refactoring effort.

Larger clones have higher refactoring value. A 200-line clone group is worth extracting; a 6-line one probably isn't.

Instance count

The number of locations where the same code appears. A clone group with 5 instances means the same block was copied to 5 places. Fixing a bug in the logic requires updating all 5.

Clone groups and families

Clone groups

A clone group is a single duplicated code block found at 2+ locations. Each location is an instance.

● Clone group 1 (8 lines, 2 instances)
  src/validators/userValidator.ts:12-19
  src/validators/orderValidator.ts:8-15

This means one block of 8 lines appears in both files. Here's what it looks like:

const errors: string[] = [];
for (const [key, rule] of Object.entries(rules)) {
  const value = data[key];
  if (rule.required && (value === undefined || value === null)) {
    errors.push(`${key} is required`);
  }
}
return errors;

One clone group = one duplicated block. But the same two files may share more than one block:

● Clone group 2 (6 lines, 2 instances)
  src/validators/userValidator.ts:24-29
  src/validators/orderValidator.ts:20-25

● Clone group 3 (5 lines, 2 instances)
  src/validators/userValidator.ts:35-39
  src/validators/orderValidator.ts:31-35

Three separate clone groups, all between the same two files. That pattern is a clone family.

Clone families

A clone family is when multiple clone groups involve the same files. Those files share multiple distinct clones and were probably copy-pasted from each other.

Family: src/validators/userValidator.ts ↔ src/validators/orderValidator.ts
  3 clone groups, 19 duplicated lines
  → Extract shared validation module

This changes the refactoring approach. Individual clone groups suggest extracting a function. A clone family suggests the files themselves need to be merged or restructured.

When duplication is acceptable

Not all duplication should be eliminated. Context matters.

• Test files: Test cases often repeat setup code intentionally. Abstracting test setup makes tests harder to read and debug. Use --production to exclude test/story/dev files entirely, or use dupes.ignore patterns for more granular control.
• Generated code: Codegen output (GraphQL, Prisma, OpenAPI) is inherently duplicative. Exclude with dupes.ignore.
• Small clones (< 10 lines): Very small clones are often idiomatic patterns (error handling, guard clauses) rather than meaningful duplication.
• Cross-language ports: If you maintain both .ts and .js versions intentionally, use --skip-local to focus on cross-directory duplicates instead.

Interpreting --threshold results

The --threshold flag sets a maximum allowed duplication percentage. If the project exceeds the threshold, fallow exits with code 1.

Duplication: 19.4% (27,255 duplicated lines across 398 files, exceeds 15% threshold)
Found 1,184 clone groups, 2,959 instances (0.23s) ✗

Start with a threshold above your current level, then ratchet it down over time. Combine with --baseline for even more gradual adoption.

How suffix-array detection works

Fallow concatenates all token streams into one sequence, builds a suffix array with LCP array, and scans for repeated subsequences above the minimum length. This runs in O(n log n) time, no pairwise file comparison needed.

Limitations

• Type-3 clones (gapped clones with inserted/deleted lines) are not detected by default. Add --near to find function-scoped near-miss clones with small structural edits. Near-miss detection uses semantic normalization and requires at least 80% shingle similarity between every pair in a group.
• Type-4 clones (semantically equivalent but syntactically different) are not detected. Two functions that do the same thing but are written differently will not be flagged.
• Cross-file-type detection only works between .ts and .js files (via --cross-language). Other language pairs are not supported.
• Minimum size thresholds mean very small clones (below --min-tokens / --min-lines) are invisible. This is intentional: small clones are usually idiomatic patterns.
• Semantic mode false positives: Normalizing identifiers and literals can match code that is structurally similar but semantically unrelated. Review semantic-mode results before acting.

JSON _meta object

When --explain is passed (or via MCP), the JSON output includes a _meta object:

{
  "schema_version": 9,
  "_meta": {
    "docs": "https://docs.fallow.tools/explanations/duplication",
    "metrics": {
      "duplication_percentage": {
        "name": "Duplication percentage",
        "description": "Fraction of total source tokens in clone groups",
        "range": "[0, 100]",
        "interpretation": "lower is better; <5% low, 5–15% moderate, >15% high"
      },
      "clone_group": {
        "name": "Clone group",
        "description": "Set of 2+ code fragments with identical normalized token sequences"
      },
      "clone_family": {
        "name": "Clone family",
        "description": "Multiple clone groups sharing the same file set, indicating systematic duplication"
      }
    }
  }
}

AI agents and CI systems can use this to interpret results without consulting external documentation.