GitPlanet

Cheap and reliable Node.js hosting starts at $3/month, and $1/month static HTML hosting

Created with love in Canada, visit hostnodejs.com today

Feel like to post an Ad? Learn Details
All Projects → WolframResearch → codeparser

WolframResearch / codeparser

Licence: MIT license
Parse Wolfram Language source code as abstract syntax trees (ASTs) or concrete syntax trees (CSTs)

Programming Languages

Mathematica
289 projects
C++
36643 projects - #6 most used programming language
CMake
9771 projects

Labels

parsingpratt-parserastcstwolfram-languageabstract-syntax-treewolfram-mathematicaastsprattconcrete-syntax-treessyntax-treescodeparsercstscodetools

Projects that are alternatives of or similar to codeparser

kataw
An 100% spec compliant ES2022 JavaScript toolchain
Stars: ✭ 303 (+260.71%)
Mutual labels:  parsing, ast, cst
Meriyah
A 100% compliant, self-hosted javascript parser - https://meriyah.github.io/meriyah
Stars: ✭ 690 (+721.43%)
Mutual labels:  parsing, ast
Estree
The ESTree Spec
Stars: ✭ 3,867 (+4503.57%)
Mutual labels:  parsing, ast
tdop.github.io
Reprinting Vaughan Pratt's Paper on Top Down Operator Precedence Parsing
Stars: ✭ 99 (+17.86%)
Mutual labels:  parsing, pratt-parser
node-typescript-parser
Parser for typescript (and javascript) files, that compiles those files and generates a human understandable AST.
Stars: ✭ 121 (+44.05%)
Mutual labels:  parsing, ast
hxjsonast
Parse JSON into position-aware AST with Haxe!
Stars: ✭ 28 (-66.67%)
Mutual labels:  parsing, ast
Uaiso
A multi-language parsing infrastructure with an unified AST
Stars: ✭ 86 (+2.38%)
Mutual labels:  parsing, ast
tree-hugger
A light-weight, extendable, high level, universal code parser built on top of tree-sitter
Stars: ✭ 96 (+14.29%)
Mutual labels:  parsing, ast
Graphql Go Tools
Tools to write high performance GraphQL applications using Go/Golang.
Stars: ✭ 96 (+14.29%)
Mutual labels:  parsing, ast
Yacep
yet another csharp expression parser
Stars: ✭ 107 (+27.38%)
Mutual labels:  parsing, ast
Down
Blazing fast Markdown / CommonMark rendering in Swift, built upon cmark.
Stars: ✭ 1,895 (+2155.95%)
Mutual labels:  parsing, ast
kolasu
Kotlin Language Support – AST Library
Stars: ✭ 45 (-46.43%)
Mutual labels:  parsing, ast
awesome-ruby-ast
A list of awesome tools and libraries which deals with ASTs in Ruby
Stars: ✭ 24 (-71.43%)
Mutual labels:  ast, abstract-syntax-tree
inmemantlr
ANTLR as a libray for JVM based languages
Stars: ✭ 87 (+3.57%)
Mutual labels:  parsing, ast
cppcombinator
parser combinator and AST generator in c++17
Stars: ✭ 20 (-76.19%)
Mutual labels:  parsing, ast
Esprima
ECMAScript parsing infrastructure for multipurpose analysis
Stars: ✭ 6,391 (+7508.33%)
Mutual labels:  parsing, ast
Unified
☔️ interface for parsing, inspecting, transforming, and serializing content through syntax trees
Stars: ✭ 3,036 (+3514.29%)
Mutual labels:  ast, cst
markright
A customizable markdown parser in Elixir: pure pattern matching.
Stars: ✭ 14 (-83.33%)
Mutual labels:  parsing, ast
Libdparse
Library for lexing and parsing D source code
Stars: ✭ 91 (+8.33%)
Mutual labels:  parsing, ast
Escaya
An blazing fast 100% spec compliant, incremental javascript parser written in Typescript
Stars: ✭ 217 (+158.33%)
Mutual labels:  parsing, ast
View All Similar Projects ➔

CodeParser

CodeParser is a package for parsing Wolfram Language source code as abstract syntax trees (ASTs) or concrete syntax trees (CSTs). CodeParser is useful for inspecting code, formatting code, and instrumenting code (for e.g., coverage reporting or profiling), and much more!

CodeParser has many key features:

  • Understands practically entire Wolfram Language syntax.
  • Fast native library implementation.
  • Tested with combination of suite of hand-written tests and fuzz testing.
  • Gracious error handling and recovery
Needs["CodeParser`"]

CodeParse["1+1"]

Out[2]= ContainerNode[String, {CallNode[LeafNode[Symbol, "Plus", <||>], {LeafNode[Integer, "1", <|Source -> {{1, 1}, {1, 2}}|>], LeafNode[Integer, "1", <|Source -> {{1, 3}, {1, 4}}|>]}, <|Source -> {{1, 1}, {1, 4}}|>]}, <||>]

"CodeParser and CodeInspector" on community.wolfram.com

Parsing the Wolfram Language from WTC 2019: Watch Video (youtube)

Parsing the Wolfram Language from WTC 2019: Watch Video (wolfram.com)

Parsing the Wolfram Language from WTC 2019: Download Presentation

Setup

CodeParser is included in Mathematica 12.2 and above.

For older versions, install CodeParser paclet from the public paclet server:

PacletInstall["CodeParser"]

Build and install the CodeParser paclet locally

Using CodeParser

After CodeParser is installed, it can be used.

Needs["CodeParser`"]

CodeParse["1+1"]

Out[2]= ContainerNode[String, {CallNode[LeafNode[Symbol, "Plus", <||>], {LeafNode[Integer, "1", <|Source -> {{1, 1}, {1, 2}}|>], LeafNode[Integer, "1", <|Source -> {{1, 3}, {1, 4}}|>]}, <|Source -> {{1, 1}, {1, 4}}|>]}, <||>]

The input to CodeParse may be a string, a File, or a list of bytes.

Command-line tool (Optional)

An optional codeparser command-line tool is also built and can be used.

cmake -DBUILD_EXE=ON ..
cmake --build . --target codeparser-exe

$cpp/src/exe/codeparser
>>> 1+1
InfixNode[Plus, {LeafNode[Integer, "1", <|Source->{{1, 2}, {1, 2}}|>], LeafNode[Integer, 1, <|Source->{{1, 3}, {1, 4}}|>]}, <|Source->{{1, 1}, {1, 4}}|>]

>>>

Troubleshooting

Make sure that the CodeParser can be found on your system:

Needs["CodeParser`"]

and try a basic example:

CodeParse["1+1"]

You may get LibraryFunction messages:

In[1]:= Needs["CodeParser`"]

In[2]:= CodeParse["1+1"]

During evaluation of In[2]:= LibraryFunction::version: The version number 7 of the library is not consistent with the current or any previous WolframLibraryVersion.

During evaluation of In[2]:= LibraryFunction::initerr: A nonzero error code 7 was returned during the initialization of the library /Users/user/Library/Mathematica/Paclets/Repository/CodeParser-1.6/LibraryResources/MacOSX-x86-64/CodeParser.dylib.

During evaluation of In[2]:= LibraryFunction::libload: The function ConcreteParseBytes_Listable_LibraryLink was not loaded from the file /Users/user/Library/Mathematica/Paclets/Repository/CodeParser-1.6/LibraryResources/MacOSX-x86-64/CodeParser.dylib.

Out[2]= $Failed

This means that CodeParser was built with a newer version of Wolfram System than your system supports.

To fix this, build CodeParser from source with the version of Wolfram System that you will use.

Benchmarks

Some of the benchmarks test large data files. Those files are tracked in this repository to ensure that benchmarks are always run against identical input. Git LFS is used to ensure that a basic checkout of this repository remains small, which is important in CI/CD builds.

To run the benchmarks, first ensure that the large benchmark files have been checked out locally using:

$ git lfs pull --exclude="" --include="*"

This will override the default settings in .lfsconfig.

File Overview

  • Tests/files/large/ contains files managed by Git LFS. The files in this directory are used by the benchmarks. These files should never be modified, to ensure that benchmark comparisions between different revisions of this repository can be meaningfully compared.
Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].