All Projects → neogeny → Tatsu

neogeny / Tatsu

Licence: other
竜 TatSu generates Python parsers from grammars in a variation of EBNF

Programming Languages

python
139335 projects - #7 most used programming language
python3
1442 projects
python2
120 projects
grammar
57 projects

Projects that are alternatives of or similar to Tatsu

Participle
A parser library for Go
Stars: ✭ 2,302 (+1062.63%)
Mutual labels:  ast, parser-library, parser
Babylon
PSA: moved into babel/babel as @babel/parser -->
Stars: ✭ 1,692 (+754.55%)
Mutual labels:  ast, parser
Csly
a C# embeddable lexer and parser generator (.Net core)
Stars: ✭ 129 (-34.85%)
Mutual labels:  parser-generator, parser
Json To Ast
JSON AST parser
Stars: ✭ 161 (-18.69%)
Mutual labels:  ast, parser
Java Petitparser
Dynamic parser combinators in Java.
Stars: ✭ 118 (-40.4%)
Mutual labels:  parser-library, parser
Phplrt
PHP Language Recognition Tool
Stars: ✭ 127 (-35.86%)
Mutual labels:  ast, parser
Lioness
The Lioness Programming Language
Stars: ✭ 155 (-21.72%)
Mutual labels:  ast, parser
Graphql Go Tools
Tools to write high performance GraphQL applications using Go/Golang.
Stars: ✭ 96 (-51.52%)
Mutual labels:  ast, parser
Pegparser
💡 Build your own programming language! A C++17 PEG parser generator supporting parser combination, memoization, left-recursion and context-dependent grammars.
Stars: ✭ 164 (-17.17%)
Mutual labels:  parser-generator, parser
Query Translator
Query Translator is a search query translator with AST representation
Stars: ✭ 165 (-16.67%)
Mutual labels:  ast, parser
Libpypa
libpypa is a Python parser implemented in pure C++
Stars: ✭ 172 (-13.13%)
Mutual labels:  ast, parser-library
Flora Sql Parser
Parse SQL (select) statements into abstract syntax tree (AST) and convert ASTs back to SQL.
Stars: ✭ 186 (-6.06%)
Mutual labels:  ast, parser
Cppcmb
A generic C++17 parser-combinator library with a natural grammar notation.
Stars: ✭ 108 (-45.45%)
Mutual labels:  parser-library, parser
Php Zephir Parser
The Zephir Parser delivered as a C extension for the PHP language.
Stars: ✭ 129 (-34.85%)
Mutual labels:  parser-generator, parser
Elm Markdown
Pure Elm markdown parsing and rendering
Stars: ✭ 96 (-51.52%)
Mutual labels:  ast, parser
Pygdbmi
A library to parse gdb mi output and interact with gdb subprocesses
Stars: ✭ 139 (-29.8%)
Mutual labels:  parser-library, parser
Mediawiki
MediaWiki API wrapper in python http://pymediawiki.readthedocs.io/en/latest/
Stars: ✭ 89 (-55.05%)
Mutual labels:  parser-library, parser
Libdparse
Library for lexing and parsing D source code
Stars: ✭ 91 (-54.04%)
Mutual labels:  ast, parser
Npeg
PEGs for Nim, another take
Stars: ✭ 163 (-17.68%)
Mutual labels:  parser-generator, parser
Snapdragon
snapdragon is an extremely pluggable, powerful and easy-to-use parser-renderer factory.
Stars: ✭ 180 (-9.09%)
Mutual labels:  ast, parser

license pyversions fury circleci docs

At least for the people who send me mail about a new language that they're designing, the general advice is: do it to learn about how to write a compiler. Don't have any expectations that anyone will use it, unless you hook up with some sort of organization in a position to push it hard. It's a lottery, and some can buy a lot of the tickets. There are plenty of beautiful languages (more beautiful than C) that didn't catch on. But someone does win the lottery, and doing a language at least teaches you something.

Dennis Ritchie (1941-2011) Creator of the C programming language and of Unix

TatSu

def WARNING():
    """
    |TatSu|>=5.0.0 requires Python>=3.8

    Python 3.8 introduced new language features that allow writing better programs
    more clearly, and all code compatible with Python 3.7 should run fine on 3.8
    with minor, or no changes.

    Python has adopted an anual release schedule (PEP-602).

    Python 3.9 is due to be released on June 2020
    Python 3.7 will have bugfix releases only until mid 2020
    Python 3.6 had its last bugfix release on December 2019
    Python 3.5 entered "security fixes only" mode since August 2018
    Python 2.7 reached its end of life on January 2020

    There are compelling reasons to upgrade 3.x projects to Python 3.8
    """
    pass

TatSu is a tool that takes grammars in a variation of EBNF as input, and outputs memoizing (Packrat) PEG parsers in Python.

TatSu can compile a grammar stored in a string into a tatsu.grammars.Grammar object that can be used to parse any given input, much like the re module does with regular expressions, or it can generate a Python module that implements the parser.

TatSu supports left-recursive rules in PEG grammars using the algorithm by Laurent and Mens. The generated AST has the expected left associativity.

Installation

$ pip install TatSu

Using the Tool

TatSu can be used as a library, much like Python's re, by embedding grammars as strings and generating grammar models instead of generating Python code.

  • tatsu.compile(grammar, name=None, **kwargs)

    Compiles the grammar and generates a model that can subsequently be used for parsing input with.

  • tatsu.parse(grammar, input, **kwargs)

    Compiles the grammar and parses the given input producing an AST as result. The result is equivalent to calling:

    model = compile(grammar)
    ast = model.parse(input)
    

    Compiled grammars are cached for efficiency.

  • tatsu.to_python_sourcecode(grammar, name=None, filename=None, **kwargs)

    Compiles the grammar to the Python sourcecode that implements the parser.

This is an example of how to use 竜 TatSu as a library:

GRAMMAR = '''
    @@grammar::CALC


    start = expression $ ;


    expression
        =
        | expression '+' term
        | expression '-' term
        | term
        ;


    term
        =
        | term '*' factor
        | term '/' factor
        | factor
        ;


    factor
        =
        | '(' expression ')'
        | number
        ;


    number = /\d+/ ;
'''


if __name__ == '__main__':
    import pprint
    import json
    from tatsu import parse
    from tatsu.util import asjson

    ast = parse(GRAMMAR, '3 + 5 * ( 10 - 20 )')
    print('# PPRINT')
    pprint.pprint(ast, indent=2, width=20)
    print()

    print('# JSON')
    print(json.dumps(asjson(ast), indent=2))
    print()

TatSu will use the first rule defined in the grammar as the start rule.

This is the output:

# PPRINT
[ '3',
  '+',
  [ '5',
    '*',
    [ '10',
      '-',
      '20']]]

# JSON
[
  "3",
  "+",
  [
    "5",
    "*",
    [
      "10",
      "-",
      "20"
    ]
  ]
]

Documentation

For a detailed explanation of what 竜 TatSu is capable of, please see the documentation.

Questions?

Please use the [tatsu] tag on StackOverflow for general Q&A, and limit Github issues to bugs, enhancement proposals, and feature requests.

Changes

See the CHANGELOG for details.

License

You may use 竜 TatSu under the terms of the BSD-style license described in the enclosed LICENSE.txt file. If your project requires different licensing please email.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].