All Projects → ChimeHQ → SwiftTreeSitter

ChimeHQ / SwiftTreeSitter

Licence: BSD-3-Clause license
Swift wrappers for the tree-sitter incremental parsing system

Programming Languages

swift
15916 projects
objective c
16641 projects - #2 most used programming language

Projects that are alternatives of or similar to SwiftTreeSitter

tree-hugger
A light-weight, extendable, high level, universal code parser built on top of tree-sitter
Stars: ✭ 96 (-17.24%)
Mutual labels:  tree-sitter, parsing
ltreesitter
Standalone tree sitter bindings for the Lua language
Stars: ✭ 62 (-46.55%)
Mutual labels:  tree-sitter, parsing
Tree Sitter
An incremental parsing system for programming tools
Stars: ✭ 7,083 (+6006.03%)
Mutual labels:  tree-sitter, parsing
tree-sitter-html
HTML grammar for Tree-sitter
Stars: ✭ 56 (-51.72%)
Mutual labels:  tree-sitter
zee
A modern text editor for the terminal written in Rust
Stars: ✭ 1,120 (+865.52%)
Mutual labels:  tree-sitter
text2tab
TAB-delimited text parser for ABAP
Stars: ✭ 16 (-86.21%)
Mutual labels:  parsing
angel.co-companies-list-scraping
No description or website provided.
Stars: ✭ 54 (-53.45%)
Mutual labels:  parsing
cppcombinator
parser combinator and AST generator in c++17
Stars: ✭ 20 (-82.76%)
Mutual labels:  parsing
tree-sitter-typescript
TypeScript grammar for tree-sitter
Stars: ✭ 154 (+32.76%)
Mutual labels:  tree-sitter
StatementParser
Idea behind the StatementParser is, that it would be nice to be able to process financial data from different kind of statements in automatized way. This is often pretty hard as brokers are giving these data only in form of xls/xlst/pdf or other format which is not directly processable and here comes StatmentParser.
Stars: ✭ 21 (-81.9%)
Mutual labels:  parsing
libvcs
⚙️ Lite, typed, pythonic utilities for git, svn, mercurial, etc.
Stars: ✭ 43 (-62.93%)
Mutual labels:  parsing
CoreJSON
Core Foundation, libyajl based JSON support.
Stars: ✭ 48 (-58.62%)
Mutual labels:  parsing
tree-sitter-legesher-python
✨ Legesher's Python grammar for Tree-Sitter 🌳
Stars: ✭ 40 (-65.52%)
Mutual labels:  tree-sitter
Deep-NLP-Resources
Curated list of all NLP Resources
Stars: ✭ 65 (-43.97%)
Mutual labels:  parsing
markright
A customizable markdown parser in Elixir: pure pattern matching.
Stars: ✭ 14 (-87.93%)
Mutual labels:  parsing
Plotty
C language compiler from scratch for a custom architecture, with virtual machine and all
Stars: ✭ 33 (-71.55%)
Mutual labels:  parsing
tree-sitter-elisp
tree-sitter grammar for emacs lisp
Stars: ✭ 20 (-82.76%)
Mutual labels:  tree-sitter
sledgehammer
🔨 📶 WiFi-Jammer/DoS toolset
Stars: ✭ 34 (-70.69%)
Mutual labels:  parsing
slash-command
Simple slash command parsing.
Stars: ✭ 15 (-87.07%)
Mutual labels:  parsing
kaliningraph
🕸️ Graphs, finite fields and discrete dynamical systems in Kotlin
Stars: ✭ 62 (-46.55%)
Mutual labels:  parsing

Build Status License Platforms

SwiftTreeSitter

Swift wrappers for the tree-sitter incremental parsing system.

SwiftTreeSitter is fairly low-level. If you are looking a higher-level system for syntax highlighting and other syntactic operations, you might want to have a look at Neon.

Integration

Swift Package Manager

dependencies: [
    .package(url: "https://github.com/ChimeHQ/SwiftTreeSitter")
]

Usage

Basic parsing:

// import the tree-sitter bindings
import SwiftTreeSitter

// import the tree-sitter swift parser (confusing naming, I know)
import TreeSitterSwift

// create a language
let language = Language(language: tree_sitter_swift())

// create a parser
let parser = Parser()
try parser.setLanguage(language)

let source = """
func hello() {
    print("hello from tree-sitter")
}
"""

let tree = parser.parse(source)

print("tree: ", tree)

Tree-sitter operates on byte indices and line/character-offset pairs (called a Point). It is, unfortunately, your responsibility to map your text storage and edits into these types. For the most part,

Processing edits:

// tree-sitter operates on byte indices and line/character-offset pairs (called a Point). It is, unfortunately,
// your responsibility to map your text storage and edits into these types

let edit = InputEdit(startByte: editStartByteOffset,
                                oldEndByte: preEditEndByteOffset,
                                newEndByte: postEditEndByteOffset,
                                startPoint: editStartPoint,
                                oldEndPoint: preEditEndPoint,
                                newEndPoint: postEditEndPoint)

// apply the edit first
existingTree.edit(edit)

// then, re-parse the text to build a new tree
let newTree = parser.parse(existingTree, string: fullText)

// you can now compute a diff to determine what has changed
let changedRanges = existingTree.changedRanges(newTree)

Using queries:

import SwiftTreeSitter
import TreeSitterSwift

let language = Language(language: tree_sitter_swift())

// find the SPM-packaged queries
let url = Bundle.main
              .resourceURL
              .appendingPathComponent("TreeSitterSwift_TreeSitterSwift.bundle")
              .appendingPathComponent("queries/highlights.scm")

// this can be very expensive, depending on the language grammar/queries
let query = try language.query(contentsOf: url!)

let tree = parseText() // <- omitting for clarity

let queryCursor = query.execute(node: tree.rootNode!, in: tree)

// the performance of nextMatch is highly dependent on the nature of the queries,
// language grammar, and size of input
while let match = queryCursor.nextMatch() {
    print("match: ", match)
}

Language Parsers

Tree-sitter language parsers are separate projects, and you'll probably need at least one. They can also be built with SPM, though they are more complex. If you would like SPM support for parser that doesn't have it yet, let me know and I'll help!

Parsers available via SPM: (* not merged into official repo yet)

While SPM is nice, it isn't a requirement. You can use git submodules. You can even build them yourself. In fact, I've struggled with this so much that I began adapting the runtime's Makefile for the parsers themselves. This is a work-in-progress. But, if the parser you'd like to use doesn't have a Makefile, let me know and I'll help get it set up.

Predicate/Directive Support

QueryMatch provides an API for getting at query predicates and directives. You are free to use/evaluate them yourself. However, there is also a ResolvingQueryCursor, which wraps a standard QueryCursor, but allows for resolution of predicates. It also provides some facilities for preloading all QueryMatch objects from the underlying QueryCursor, which can help with performance in some situations.

The following predicates are parsed and transformed into structured Predicate cases. All others are turned into the generic case.

  • eq?: fully supported
  • not-eq?: fully supported
  • match?: fully supported
  • not-match?: fully supported
  • any-of?: fully supported
  • not-any-of?: fully supported
  • is-not?: parsed, but not implemented

Please open up an issue if you need additional support here.

let resolvingCursor = ResolvingQueryCursor(cursor: queryCursor)

// this function takes an NSRange and Range<Point>, and returns
// the contents in your source text
let provider: TextProvider = { range, pointRange in ... }

resolvingCursor.prepare(with: provider)

// ResolvingQueryCursor conforms to Sequence
for match in resolvingCursor {
    print("match: ", match)
}

Suggestions or Feedback

We'd love to hear from you! Get in touch via twitter, an issue, or a pull request.

Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].