ast-grep: Structural Code Search and Automated Refactoring for Large Codebases

Text-based code search — grep, ripgrep, regex in your IDE — works fine for finding simple patterns. It breaks down when you need to find and rewrite code based on structure rather than literal text. A function called fetchUser might appear as fetchUser(), await fetchUser(), const user = fetchUser(id), or spread across multiple lines. Text search forces you to write a regex that handles every variation, or miss cases that don’t match.

ast-grep takes a different approach. It searches using AST (Abstract Syntax Tree) patterns — templates that match code structure regardless of formatting, variable names, or call variations. You write a pattern that looks like the code you want to find, using $VAR as a wildcard for any expression, and ast-grep finds every structurally matching instance across your codebase.

The tool is written in Rust, handles millions of lines of code in seconds, and supports JavaScript, TypeScript, Python, Rust, Go, Java, C, C++, and more. For large-scale refactors — API migrations, deprecation cleanups, security pattern enforcement — it covers cases that regex can’t.

Installation

# macOS
brew install ast-grep

# Cargo (cross-platform)
cargo install ast-grep

# npm (useful for JavaScript/TypeScript projects)
npm install -g @ast-grep/cli

# Verify
sg --version

Basic Search: Finding Code by Structure

The simplest use is interactive search with the -p (pattern) flag:

# Find all console.log calls in JavaScript files
sg -p 'console.log($$$)' -l js

# Find all async functions
sg -p 'async function $NAME($$$) { $$$ }' -l ts

# Find React useState calls
sg -p 'const [$STATE, $SETTER] = useState($$$)' -l tsx

The $NAME syntax matches a single identifier. $$$ matches zero or more nodes (like ...args in JavaScript). $$ matches exactly one node.

This already beats grep for several common cases. Finding console.log(something) with grep requires knowing it could be console.log("thing"), console.log(var), console.log(obj.prop) — you’d write .+ and get false positives. With ast-grep, console.log($$$) matches any call to console.log with any arguments, structurally.

Rewriting Code: The --rewrite Flag

Search becomes refactoring when you add --rewrite:

# Replace console.log with logger.debug
sg -p 'console.log($$$ARGS)' --rewrite 'logger.debug($$$ARGS)' -l ts

# Replace old API with new API
sg -p 'React.createElement($TYPE, $PROPS)' \
   --rewrite 'createElement($TYPE, $PROPS)' -l tsx

# Migrate from moment.js to date-fns
sg -p 'moment($DATE).format($FORMAT)' \
   --rewrite 'format(parseISO($DATE), $FORMAT)' -l js

The captured metavariables ($ARGS, $TYPE, etc.) carry the matched content forward into the replacement. This is what makes ast-grep useful for real migrations: you’re not just finding text and replacing it with different text, you’re rearranging code structure while preserving the content.

To apply rewrites in place, add -U (update):

sg -p 'console.log($$$ARGS)' --rewrite 'logger.debug($$$ARGS)' -l ts -U

Without -U, ast-grep prints a diff of what would change. Use that for preview before committing to the rewrite.

Writing Rule Files for Complex Patterns

For anything beyond simple one-liner rewrites, ast-grep uses YAML rule files. Rules can match on multiple conditions, enforce constraints on what a matched variable contains, and trigger on specific node types.

Basic Rule Structure

# rules/no-raw-console.yml
id: no-raw-console
language: TypeScript
rule:
  pattern: console.log($$$)
message: Use logger.debug() instead of console.log()
severity: warning
fix: logger.debug($$$)

Run with:

sg scan --rule rules/no-raw-console.yml src/

Matching Specific Node Types

Sometimes you want to match foo() only when called as a statement, not when used as an expression argument. The kind field restricts matching to a specific AST node type:

id: no-synchronous-fs
language: JavaScript
rule:
  pattern: fs.readFileSync($$$)
  kind: call_expression
message: Use fs.readFile() or fs.promises.readFile() instead
severity: error

You can discover node type names with sg --ast:

# Show the AST for a snippet
echo 'fs.readFileSync("path")' | sg --ast -l js

This outputs the full AST tree, showing node types at each level.

Compound Rules: AND, OR, NOT

Rules can combine multiple conditions:

id: deprecated-api-usage
language: TypeScript
rule:
  all:
    - pattern: $OBJ.oldMethod($$$)
    - not:
        pattern: $OBJ.oldMethod($$$, { deprecated: false })
message: oldMethod() is deprecated, use newMethod() instead

The all combinator requires all child rules to match. any requires at least one. not inverts. This handles the common migration case where an old API has both a deprecated and a non-deprecated call signature.

Matching Across Multiple Languages

For a TypeScript monorepo with both .ts and .tsx files:

id: react-class-component
language:
  - TypeScript
  - TSX
rule:
  pattern: class $NAME extends React.Component { $$$ }
message: Convert class components to functional components

Practical Migration Examples

Python: Migrate from os.path to pathlib

# rules/pathlib-migration.yml
id: os-path-join
language: Python
rule:
  pattern: os.path.join($$$PARTS)
fix: Path($$$PARTS)  # Simplified — manual review needed for chaining
message: Replace os.path.join() with pathlib.Path
severity: suggestion

For Python migrations, ast-grep pairs well with a manual review step for complex cases:

# Find all instances, review the diff
sg scan --rule rules/pathlib-migration.yml . --json | jq '.[] | .message, .file'

# Apply simple cases
sg -p 'os.path.exists($PATH)' --rewrite '$PATH.exists()' -l python -U

TypeScript: Enforce Error Handling Patterns

id: unhandled-promise
language: TypeScript
rule:
  any:
    - pattern: $PROMISE.then($CALLBACK)
    - pattern: $PROMISE.catch($CALLBACK)
  not:
    inside:
      pattern: try { $$$ } catch($E) { $$$ }
message: Promise chains should be inside try/catch or have explicit error handling
severity: warning

JavaScript: Remove Legacy Polyfills

If you’ve dropped IE11 support and have conditional polyfill loading:

sg -p 'if (!Array.prototype.includes) { $$$ }' \
   --rewrite '' -l js -U

sg -p 'if (!window.Promise) { $$$ }' \
   --rewrite '' -l js -U

Go: Migrate Error Wrapping

id: fmt-errorf-migration
language: Go
rule:
  pattern: fmt.Errorf("$MSG: %v", $ERR)
fix: fmt.Errorf("$MSG: %w", $ERR)
message: Use %w instead of %v for error wrapping (enables errors.Is/As)
severity: warning

Integrating into CI

ast-grep can run as a linter in CI, blocking PRs that introduce deprecated patterns or security anti-patterns. With a rules directory:

# .ast-grep/sgconfig.yml
ruleDirs:
  - rules
testConfigs:
  - testDir: rule_tests

# CI step — fails on any error-severity rule violation
sg scan --config .ast-grep/sgconfig.yml src/ --error-on-severity=error

GitHub Actions Example

name: ast-grep
on: [pull_request]

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - name: Install ast-grep
        run: npm install -g @ast-grep/cli
      
      - name: Run ast-grep rules
        run: sg scan --config .ast-grep/sgconfig.yml src/ --error-on-severity=error

Pre-commit Hook

# .pre-commit-config.yaml
repos:
  - repo: local
    hooks:
      - id: ast-grep
        name: ast-grep structural lint
        entry: sg scan --config .ast-grep/sgconfig.yml
        language: system
        types: [file]
        pass_filenames: false

Testing Rules

ast-grep has a built-in test framework for rules. Write test cases alongside your rules:

# rule_tests/no-raw-console.yml
id: no-raw-console
valid:
  - logger.debug("hello")
  - logger.info("world")
invalid:
  - console.log("debug message")
  - console.log(user.id, user.name)

sg test  # Runs all rule tests

This is important for non-trivial rules: structural patterns can have surprising edge cases, and test cases catch them before the rule hits production code.

ast-grep vs. Alternatives

The practical recommendation: use ast-grep when your migration is clear-cut (find this pattern, replace with that pattern) and you want something that runs locally, integrates into CI, and covers multiple languages. Use jscodeshift when you need programmatic logic in your transformation (complex conditional rewrites, import reorganisation). Use Semgrep when you’re primarily building a security rule library.

Workflow for a Large-Scale Migration

When you’re doing a significant migration across a large codebase, a structured approach helps:

1. Write and test the pattern. Run sg -p 'your-pattern' --l lang and review matches. Adjust until false positives are minimal.

2. Diff before applying. sg -p 'pattern' --rewrite 'replacement' -l lang without -U to preview changes. Pipe to a file if the output is large.

3. Apply incrementally. For a codebase with thousands of matches, apply directory by directory so you can review and commit in chunks.

4. Add a lint rule immediately. Once you’ve migrated, add an error-severity rule to prevent the old pattern from reappearing. The rule paying off the first time someone writes the old API accidentally makes it worth writing.

5. Leave notes in the diff. The PR description should explain what the migration was and link to the rule that enforces it going forward.

ast-grep won’t write your migration logic for you, but it eliminates the mechanical work that makes large-scale refactors take weeks instead of hours. The combination of fast structural search, rewrite-in-place, and CI enforcement covers the complete migration workflow without requiring a separate toolchain for each language.