Testing Parsers

Write comprehensive tests for your Tree-sitter parser

Testing Parsers

For each rule you add to your grammar, you should create tests that describe how syntax trees should look when parsing that rule. These tests serve as documentation and verify that your parser works correctly.

Test Format

Tests are written in specially-formatted text files in the test/corpus/ directory.

Basic Structure

Create test files like test/corpus/statements.txt:

test/corpus/statements.txt

==================
Return statements
==================

func x() int {
  return 1;
}

---

(source_file
  (function_definition
    (identifier)
    (parameter_list)
    (primitive_type)
    (block
      (return_statement (number)))))

1. Test name

   Written between two lines of = (equal signs)

2. Input source code

   The code to parse

3. Separator

   A line of three or more - (dashes)

4. Expected output

   The expected syntax tree as an S-expression

The S-expression only shows named nodes. Anonymous nodes (strings and regexes from the grammar) are not shown.

With Field Names

Include field names for better documentation:

==================
Return statements
==================

func x() int {
  return 1;
}

---

(source_file
  (function_definition
    name: (identifier)
    parameters: (parameter_list)
    result: (primitive_type)
    body: (block
      (return_statement (number)))))

Field names make tests more readable and document the structure of your syntax tree.

Running Tests

Run All Tests

tree-sitter test

This runs all tests in your test/corpus/ folder.

Run Specific Tests

Use the -i flag to filter by name:

tree-sitter test -i 'Return statements'

Update Tests

After modifying the grammar, update all expected outputs:

tree-sitter test -u

Review the changes carefully before committing. The -u flag overwrites all expected outputs with current parser output.

Debug Tests

Show detailed parsing information:

tree-sitter test -d
tree-sitter test --debug

Test Attributes

Annotate tests with attributes below the test name:

:skip

Temporarily disable a test:

==================
Future feature
:skip
==================

future_syntax;

---

:error

Assert that parsing produces an error:

==================
Invalid syntax
:error
==================

int main ( {}

---

With :error, you can omit the expected output section. The test passes if the parse tree contains an ERROR node.

:fail-fast

Stop testing if this test fails:

==================
Critical test
:fail-fast
==================

int main() {}

---

(source_file ...)

:platform(PLATFORM)

Run only on specific platforms:

==================
Windows-specific behavior
:platform(windows)
==================

C:\\path\\to\\file

---

(source_file ...)

Linux

:platform(linux)

:platform(macos)

:platform(windows)

Multiple platforms:

==================
Unix-like systems
:platform(linux)
:platform(macos)
==================

:language(LANG)

Use a different parser from a multi-parser repository:

==================
TypeScript test
:language(typescript)
==================

console.log('Hello');

---

(program ...)

==================
TSX test
:language(tsx)
==================

const x = <div />;

---

(program ...)

The default parser is the first entry in the grammars field in tree-sitter.json.

:cst

Show the complete concrete syntax tree:

==================
Full CST output
:cst
==================

int main() {}

---

(source_file
  (function_definition
    type: (primitive_type)
    name: (identifier)
    parameters: (parameter_list
      "("
      ")")
    body: (compound_statement
      "{"
      "}")))  

CST output includes anonymous nodes (punctuation, keywords) which are normally hidden.

Custom Separators

If your language uses === or ---, add a suffix:

==================|||
Basic module
==================|||

---- MODULE Test ----
increment(n) == n + 1
====

---|||

(source_file
  (module (identifier)
    (operator (identifier)
      (parameter_list (identifier))
      (plus (identifier_ref) (number)))))

The suffix ||| can be any identical string on all separator lines.

Organizing Tests

By Feature

Organize test files by language feature:

test/corpus/
  expressions.txt
  statements.txt
  declarations.txt
  types.txt
  comments.txt

Comprehensive Coverage

Be comprehensive. Test all permutations of each language construct to increase coverage and help readers understand edge cases.

For a for loop, test:

==================
Basic for loop
==================

for i := 0; i < 10; i++ {
  print(i);
}

---

(source_file (for_statement ...))

==================
For loop without initialization
==================

for ; i < 10; i++ {
  print(i);
}

---

(source_file (for_statement ...))

==================
For loop without increment
==================

for i := 0; i < 10; {
  print(i);
}

---

(source_file (for_statement ...))

==================
Infinite for loop
==================

for {
  print("forever");
}

---

(source_file (for_statement ...))

Example Test File

Here’s a complete example:

test/corpus/expressions.txt

==================
Binary expressions
==================

a + b
a + b * c
(a + b) * c

---

(source_file
  (expression_statement
    (binary_expression
      left: (identifier)
      operator: (operator)
      right: (identifier)))
  (expression_statement
    (binary_expression
      left: (identifier)
      operator: (operator)
      right: (binary_expression
        left: (identifier)
        operator: (operator)
        right: (identifier))))
  (expression_statement
    (binary_expression
      left: (parenthesized_expression
        (binary_expression
          left: (identifier)
          operator: (operator)
          right: (identifier)))
      operator: (operator)
      right: (identifier))))

==================
Unary expressions
==================

-x
!flag
~bits

---

(source_file
  (expression_statement
    (unary_expression
      operator: (operator)
      operand: (identifier)))
  (expression_statement
    (unary_expression
      operator: (operator)
      operand: (identifier)))
  (expression_statement
    (unary_expression
      operator: (operator)
      operand: (identifier))))

==================
Function calls
==================

foo()
bar(a, b, c)
nested(foo(), bar())

---

(source_file
  (expression_statement
    (call_expression
      function: (identifier)
      arguments: (argument_list)))
  (expression_statement
    (call_expression
      function: (identifier)
      arguments: (argument_list
        (identifier)
        (identifier)
        (identifier))))
  (expression_statement
    (call_expression
      function: (identifier)
      arguments: (argument_list
        (call_expression
          function: (identifier)
          arguments: (argument_list))
        (call_expression
          function: (identifier)
          arguments: (argument_list))))))

==================
Invalid expression
:error
==================

a + + b

---

Test-Driven Development

1. Write the test first

   Before implementing a grammar rule, write a test showing the expected behavior.

2. Run the test

   The test will fail because the grammar doesn’t support the feature yet.

3. Implement the rule

   Add or modify grammar rules to make the test pass.

4. Generate and test

   Run tree-sitter generate && tree-sitter test to verify.

5. Refine

   Adjust the grammar or test as needed until it works correctly.

Automatic Compilation

The first time you run tree-sitter test after generating your parser, it takes extra time to compile your C code into a dynamically-loadable library.

Tree-sitter automatically recompiles when:

• You run tree-sitter generate
• You modify src/scanner.c (external scanner)

Compiled parsers are cached, so subsequent test runs are much faster.

Best Practices

1. Test Early and Often

Add tests as you develop each rule. Don’t wait until the grammar is complete.

2. Test Edge Cases

Include tests for:

• Empty structures
• Nested constructs
• Ambiguous syntax
• Error cases
• Platform-specific behavior

3. Use Descriptive Names

Good:
==================
If statement with else clause
==================

Bad:
==================
Test 1
==================

4. Group Related Tests

Keep similar tests in the same file:

test/corpus/
  conditionals.txt    # if, else, switch
  loops.txt          # for, while, do-while
  functions.txt      # function declarations and calls

5. Document Expected Behavior

Use tests to document how your parser handles specific cases:

==================
Operator precedence: multiplication before addition
==================

a + b * c  // Should parse as: a + (b * c)

---

(source_file
  (expression_statement
    (binary_expression
      left: (identifier)
      right: (binary_expression  # * binds tighter
        left: (identifier)
        right: (identifier)))))

Debugging Failed Tests

View Parse Tree

When a test fails, examine the actual parse tree:

echo 'your code here' | tree-sitter parse

Compare Output

The test output shows:

Expected:
  (source_file (function_definition ...))

Actual:
  (source_file (ERROR (identifier) ...))

Use Debug Mode

tree-sitter test -d -i 'your test name'

Shows:

• Lexing output
• Parse states
• Conflict resolution
• Error recovery

Next Steps

Now that you know how to test your parser:

• Learn about Publishing your parser
• Explore existing parsers for test examples
• Read about CI/CD workflows for automated testing