Code Dedup

Comprehensive code analysis for duplicate detection, dead code elimination, and structure optimization

AI-friendly code analysis platform that detects duplicate code, identifies dead code, and suggests structural improvements. Powered by MinHash LSH for O(n) performance.

Installation

npm install

Quick Start

# Analyze a project
node src/cli.js analyze ./src

# Check for duplicates
node src/cli.js check-dup ./src

# Check for dead code
node src/cli.js check-dead ./src

# Check structure issues
node src/cli.js check-struct ./src

Features

• O(n) Duplicate Detection - MinHash LSH algorithm for fast approximate matching
• Dead Code Detection - AST-based static analysis to find unused code
• Structure Analysis - Complexity, function length, and nesting metrics
• AI-Friendly API - Simple function calls returning structured JSON
• Multiple Formats - JSON, console, and HTML output

Usage

import { analyzeCode, checkDuplicates, checkDeadCode, checkStructure } from './skills/code-dedup/src/index.js';

// Comprehensive analysis
const result = await analyzeCode('./src', {
  analyses: ['dedup', 'deadCode', 'structure']
});

console.log(result.summary);
// { status: 'success', filesAnalyzed: 42, issuesFound: 15 }

// Individual checks
const duplicates = await checkDuplicates('./src', { minSimilarity: 0.85 });
const deadCode = await checkDeadCode('./src', { ignoreExports: true });
const structure = await checkStructure('./src', { maxComplexity: 10 });

CLI Options

# Custom similarity threshold
node src/cli.js analyze ./src --min-similarity 0.9

# Specific analyses
node src/cli.js analyze ./src --analyses dedup,deadCode

# JSON output
node src/cli.js analyze ./src --format json --output report.json

# Custom config
node src/cli.js analyze ./src --max-complexity 8 --max-function-length 30

Configuration

Duplicate Detection

• minLines: Minimum lines (default: 5)
• minSimilarity: Similarity 0-1 (default: 0.85)
• ignoreWhitespace: Ignore whitespace (default: true)
• ignoreComments: Ignore comments (default: true)

Dead Code Detection

• ignoreExports: Ignore exports (default: true)
• ignoreTestFiles: Ignore tests (default: true)
• whitelistPatterns: Symbol patterns to ignore

Structure Analysis

• maxComplexity: Max cyclomatic complexity (default: 10)
• maxFunctionLength: Max lines per function (default: 50)
• maxParameters: Max parameters (default: 5)

Output Format

{
  "summary": {
    "status": "success",
    "filesAnalyzed": 42,
    "issuesFound": 15,
    "durationMs": 234
  },
  "results": {
    "duplicates": [...],
    "deadCode": [...],
    "structure": {...}
  },
  "recommendations": [...]
}

Performance

• Small (<100 files): <2s
• Medium (100-1K files): <10s
• Large (>1K files): <30s

Supported Languages

• JavaScript (ES6+)
• TypeScript
• JSX/TSX
• Python (experimental)

Documentation

See SKILL.md for complete documentation.

Integration

CI/CD Pipeline

- name: Check code quality
  run: node src/cli.js analyze ./src --format json --output report.json

Pre-commit Hook

{
  "husky": {
    "hooks": {
      "pre-commit": "node src/cli.js analyze ./src"
    }
  }
}

License

MIT - see LICENSE.txt

Part of the Agent Skills Ecosystem

Visit skills.sh to discover more skills for AI agents.