All Projects → Flinesoft → Anylint

Flinesoft / Anylint

Licence: mit
Lint anything by combining the power of Swift & regular expressions.

Programming Languages

swift
15916 projects

Labels

Projects that are alternatives of or similar to Anylint

Fsq
A tool for querying the file system with a SQL-like language.
Stars: ✭ 60 (-40%)
Mutual labels:  regex
Regex101 Osx
Regex101 packaged as an offline Mac OSX application
Stars: ✭ 72 (-28%)
Mutual labels:  regex
Djurl
Simple yet helpful library for writing Django urls by an easy, short and intuitive way.
Stars: ✭ 85 (-15%)
Mutual labels:  regex
Mog
A different take on the UNIX tool cat
Stars: ✭ 62 (-38%)
Mutual labels:  regex
Certstreamcatcher
This tool is based on regex with effective standards for detecting phishing sites in real time using certstream and can also detect punycode (IDNA) attacks.
Stars: ✭ 68 (-32%)
Mutual labels:  regex
Machine Learning
My Attempt(s) In The World Of ML/DL....
Stars: ✭ 78 (-22%)
Mutual labels:  regex
Regex In Python
A comprehensive guide for learning regular expressions using Python
Stars: ✭ 58 (-42%)
Mutual labels:  regex
Regex
Regular expression engine in Python using Thompson's algorithm.
Stars: ✭ 91 (-9%)
Mutual labels:  regex
Entityframework Reverse Poco Generator Ui
A simple UI to allow you to easily select which tables you want the EntityFramework Reverse POCO Code First Generator to use.
Stars: ✭ 69 (-31%)
Mutual labels:  regex
Lit Element Router
A LitElement Router (1278 bytes gzip)
Stars: ✭ 85 (-15%)
Mutual labels:  regex
Is Glob
If you use globs, this will make your code faster. Returns `true` if the given string looks like a glob pattern or an extglob pattern. This makes it easy to create code that only uses external modules like node-glob when necessary, resulting in much faster code execution and initialization time, and a better user experience. 55+ million downloads.
Stars: ✭ 63 (-37%)
Mutual labels:  regex
Hyperscan Java
Match tens of thousands of regular expressions within milliseconds - Java bindings for Intel's hyperscan 5
Stars: ✭ 66 (-34%)
Mutual labels:  regex
Gitmad
Monitor, Alert, and Discover sensitive info and data leakage on Github.
Stars: ✭ 81 (-19%)
Mutual labels:  regex
Zile
Extract API keys from file or url using by magic of python and regex.
Stars: ✭ 61 (-39%)
Mutual labels:  regex
Globbing
Introduction to "globbing" or glob matching, a programming concept that allows "filepath expansion" and matching using wildcards.
Stars: ✭ 86 (-14%)
Mutual labels:  regex
Applied Text Mining In Python
Repo for Applied Text Mining in Python (coursera) by University of Michigan
Stars: ✭ 59 (-41%)
Mutual labels:  regex
Rare
Fast, realtime regex-extraction, and aggregation into common formats such as histograms, numerical summaries, tables, and more!
Stars: ✭ 76 (-24%)
Mutual labels:  regex
To Regex Range
Pass two numbers, get a regex-compatible source string for matching ranges. Fast compiler, optimized regex, and validated against more than 2.78 million test assertions. Useful for creating regular expressions to validate numbers, ranges, years, etc.
Stars: ✭ 97 (-3%)
Mutual labels:  regex
Youtube Regex
Best YouTube Video ID regex. Online: https://regex101.com/r/rN1qR5/2 and http://regexr.com/3anm9
Stars: ✭ 87 (-13%)
Mutual labels:  regex
Rust Onig
Rust bindings for the Oniguruma regex library
Stars: ✭ 81 (-19%)
Mutual labels:  regex

CI Code Quality Coverage Version: 0.8.2 License: MIT
PayPal: Donate GitHub: Become a sponsor Patreon: Become a patron

InstallationGetting StartedConfigurationXcode Build ScriptDonationIssuesRegex Cheat SheetLicense

AnyLint

Lint any project in any language using Swift and regular expressions. With built-in support for matching and non-matching examples validation & autocorrect replacement. Replaces SwiftLint custom rules & works for other languages as well! 🎉

Installation

Via Homebrew:

To install AnyLint the first time, run these commands:

brew tap Flinesoft/AnyLint https://github.com/Flinesoft/AnyLint.git
brew install anylint

To update it to the latest version, run this instead:

brew upgrade anylint

Via Mint:

To install AnyLint or update to the latest version, run this command:

mint install Flinesoft/AnyLint

Getting Started

To initialize AnyLint in a project, run:

anylint --init blank

This will create the Swift script file lint.swift with something like the following contents:

#!/usr/local/bin/swift-sh
import AnyLint // @Flinesoft

Lint.logSummaryAndExit(arguments: CommandLine.arguments) {
    // MARK: - Variables
    let readmeFile: Regex = #"README\.md"#

    // MARK: - Checks
    // MARK: Readme
    try Lint.checkFilePaths(
        checkInfo: "Readme: Each project should have a README.md file explaining the project.",
        regex: readmeFile,
        matchingExamples: ["README.md"],
        nonMatchingExamples: ["README.markdown", "Readme.md", "ReadMe.md"],
        violateIfNoMatchesFound: true
    )

    // MARK: ReadmeTypoLicense
    try Lint.checkFileContents(
        checkInfo: "ReadmeTypoLicense: Misspelled word 'license'.",
        regex: #"([\s#]L|l)isence([\s\.,:;])"#,
        matchingExamples: [" license:", "## Lisence\n"],
        nonMatchingExamples: [" license:", "## License\n"],
        includeFilters: [readmeFile],
        autoCorrectReplacement: "$1icense$2",
        autoCorrectExamples: [
            ["before": " lisence:", "after": " license:"],
            ["before": "## Lisence\n", "after": "## License\n"],
        ]
    )
}

The most important thing to note is that the first three lines are required for AnyLint to work properly.

All the other code can be adjusted and that's actually where you configure your lint checks (a few examples are provided by default in the blank template). Note that the first two lines declare the file to be a Swift script using swift-sh. Thus, you can run any Swift code and even import Swift packages (see the swift-sh docs) if you need to. The third line makes sure that all violations found in the process of running the code in the completion block are reported properly and exits the script with the proper exit code at the end.

Having this configuration file, you can now run anylint to run your lint checks. By default, if any check fails, the entire command fails and reports the violation reason. To learn more about how to configure your own checks, see the Configuration section below.

If you want to create and run multiple configuration files or if you want a different name or location for the default config file, you can pass the --path option, which can be used multiple times as well like this:

Initializes the configuration files at the given locations:

anylint --init blank --path Sources/lint.swift --path Tests/lint.swift

Runs the lint checks for both configuration files:

anylint --path Sources/lint.swift --path Tests/lint.swift

There are also several flags you can pass to anylint:

  1. -s / --strict: Fails on warnings as well. (By default, the command only fails on errors.)
  2. -x / --xcode: Prints warnings & errors in a format to be reported right within Xcodes left sidebar.
  3. -l / --validate: Runs only validations for matchingExamples, nonMatchingExamples and autoCorrectExamples.
  4. -v / --version: Prints the current tool version. (Does not run any lint checks.)
  5. -d / --debug: Logs much more detailed information about what AnyLint is doing for debugging purposes.

Configuration

AnyLint provides three different kinds of lint checks:

  1. checkFileContents: Matches the contents of a text file to a given regex.
  2. checkFilePaths: Matches the file paths of the current directory to a given regex.
  3. customCheck: Allows to write custom Swift code to do other kinds of checks.

Several examples of lint checks can be found in the lint.swift file of this very project.

Basic Types

Independent from the method used, there are a few types specified in the AnyLint package you should know of.

Regex

Many parameters in the above mentioned lint check methods are of Regex type. A Regex can be initialized in several ways:

  1. Using a String:
let regex = Regex(#"(foo|bar)[0-9]+"#) // => /(foo|bar)[0-9]+/
let regexWithOptions = Regex(#"(foo|bar)[0-9]+"#, options: [.ignoreCase, .dotMatchesLineSeparators, .anchorsMatchLines]) // => /(foo|bar)[0-9]+/im
  1. Using a String Literal:
let regex: Regex = #"(foo|bar)[0-9]+"#  // => /(foo|bar)[0-9]+/
let regexWithOptions: Regex = #"(foo|bar)[0-9]+\im"#  // => /(foo|bar)[0-9]+/im
  1. Using a Dictionary Literal: (use for named capture groups)
let regex: Regex = ["key": #"foo|bar"#, "num": "[0-9]+"] // => /(?<key>foo|bar)(?<num>[0-9]+)/
let regexWithOptions: Regex = ["key": #"foo|bar"#, "num": "[0-9]+", #"\"#: "im"] // => /(?<key>foo|bar)(?<num>[0-9]+)/im

Note that we recommend using raw strings (#"foo"# instead of "foo") for all regexes to get rid of double escaping backslashes (e.g. \\s becomes \s). This also allows for testing regexes in online regex editors like Rubular first and then copy & pasting from them without any additional escaping (except for { & }, replace with \{ & \}).

Regex Options

Specifying Regex options in literals is done via the \ separator as shown in the examples above. The available options are:

  1. i for .ignoreCase: Any specified characters will both match uppercase and lowercase variants.
  2. m for .dotMatchesLineSeparators: All appearances of . in regexes will also match newlines (which are not matched against by default).

The .anchorsMatchLines option is always activated on literal usage as we strongly recommend it. It ensures that ^ can be used to match the start of a line and $ for the end of a line. By default they would match the start & end of the entire string. If that's actually what you want, you can still use \A and \z for that. This makes the default literal Regex behavior more in line with sites like Rubular.

CheckInfo

A CheckInfo contains the basic information about a lint check. It consists of:

  1. id: The identifier of your lint check. For example: EmptyTodo
  2. hint: The hint explaining the cause of the violation or the steps to fix it.
  3. severity: The severity of violations. One of error, warning, info. Default: error

While there is an initializer available, we recommend using a String Literal instead like so:

// accepted structure: <id>(@<severity>): <hint>
let checkInfo: CheckInfo = "ReadmePath: The README file should be named exactly `README.md`."
let checkInfoCustomSeverity: CheckInfo = "[email protected]: The README file should be named exactly `README.md`."

AutoCorrection

An AutoCorrection contains an example before and after string to validate that a given autocorrection rule behaves correctly.

It can be initialized in two ways, either with the default initializer:

let example: AutoCorrection = AutoCorrection(before: "Lisence", after: "License")

Or using a Dictionary literal:

let example: AutoCorrection = ["before": "Lisence", "after": "License"]

Check File Contents

AnyLint has rich support for checking the contents of a file using a regex. The design follows the approach "make simple things simple and hard things possible". Thus, let's explain the checkFileContents method with a simple and a complex example.

In its simplest form, the method just requires a checkInfo and a regex:

// MARK: EmptyTodo
try Lint.checkFileContents(
    checkInfo: "EmptyTodo: TODO comments should not be empty.",
    regex: #"// TODO: *\n"#
)

But we strongly recommend to always provide also:

  1. matchingExamples: Array of strings expected to match the given string for regex validation.
  2. nonMatchingExamples: Array of strings not matching the given string for regex validation.
  3. includeFilters: Array of Regex objects to include to the file paths to check.

The first two will be used on each run of AnyLint to check if the provided regex actually works as expected. If any of the matchingExamples doesn't match or if any of the nonMatchingExamples does match, the entire AnyLint command will fail early. This a built-in validation step to help preventing a lot of issues and increasing your confidence on the lint checks.

The third one is recommended because it increases the performance of the linter. Only files at paths matching at least one of the provided regexes will be checked. If not provided, all files within the current directory will be read recursively for each check, which is inefficient.

Here's the recommended minimum example:

// MARK: - Variables
let swiftSourceFiles: Regex = #"Sources/.*\.swift"#
let swiftTestFiles: Regex = #"Tests/.*\.swift"#

// MARK: - Checks
// MARK: empty_todo
try Lint.checkFileContents(
    checkInfo: "EmptyTodo: TODO comments should not be empty.",
    regex: #"// TODO: *\n"#,
    matchingExamples: ["// TODO:\n"],
    nonMatchingExamples: ["// TODO: not yet implemented\n"],
    includeFilters: [swiftSourceFiles, swiftTestFiles]
)

There's 3 more parameters you can optionally set if needed:

  1. excludeFilters: Array of Regex objects to exclude from the file paths to check.
  2. autoCorrectReplacement: Replacement string which can reference any capture groups in the regex.
  3. autoCorrectExamples: Example structs with before and after for autocorrection validation.
  4. repeatIfAutoCorrected: Repeat check if at least one auto-correction was applied in last run. Defaults to false.

The excludeFilters can be used alternatively to the includeFilters or alongside them. If used alongside, exclusion will take precedence over inclusion.

If autoCorrectReplacement is provided, AnyLint will automatically replace matches of regex with the given replacement string. Capture groups are supported, both in numbered style (([a-z]+)(\d+) => $1$2) and named group style ((?<alpha>[a-z])(?<num>\d+) => $alpha$num). When provided, we strongly recommend to also provide autoCorrectExamples for validation. Like for matchingExamples / nonMatchingExamples the entire command will fail early if one of the examples doesn't correct from the before string to the expected after string.

Caution: When using the autoCorrectReplacement parameter, be sure to double-check that your regex doesn't match too much content. Additionally, we strongly recommend to commit your changes regularly to have some backup.

Here's a full example using all parameters at once:

// MARK: - Variables
let swiftSourceFiles: Regex = #"Sources/.*\.swift"#
let swiftTestFiles: Regex = #"Tests/.*\.swift"#

// MARK: - Checks
// MARK: empty_method_body
try Lint.checkFileContents(
    checkInfo: "EmptyMethodBody: Don't use whitespaces for the body of empty methods.",
    regex: [
      "declaration": #"func [^\(\s]+\([^{]*\)"#,
      "spacing": #"\s*"#,
      "body": #"\{\s+\}"#
    ],
    matchingExamples: [
        "func foo2bar()  { }",
        "func foo2bar(x: Int, y: Int)  { }",
        "func foo2bar(\n    x: Int,\n    y: Int\n) {\n    \n}",
    ],
    nonMatchingExamples: [
      "func foo2bar() {}",
      "func foo2bar(x: Int, y: Int) {}"
    ],
    includeFilters: [swiftSourceFiles],
    excludeFilters: [swiftTestFiles],
    autoCorrectReplacement: "$declaration {}",
    autoCorrectExamples: [
        ["before": "func foo2bar()  { }", "after": "func foo2bar() {}"],
        ["before": "func foo2bar(x: Int, y: Int)  { }", "after": "func foo2bar(x: Int, y: Int) {}"],
        ["before": "func foo2bar()\n{\n    \n}", "after": "func foo2bar() {}"],
    ]
)

Note that when autoCorrectReplacement produces a replacement string that exactly matches the matched string of regex, then no violation will be reported. This enables us to provide more generic regex patterns that also match the correct string without actually reporting a violation for the correct one. For example, using the regex if\s*\(([^)]+)\)\s*\{ to check whitespaces around braces after if statement would report a violation for all of the following examples:

if(x == 5) { /* some code */ }
if (x == 5){ /* some code */ }
if(x == 5){ /* some code */ }
if (x == 5) { /* some code */ }

The problem is that the last example actually is our expected formatting and should not violate. By providing an autoCorrectReplacement of if ($1) {, we can fix that as the replacement would be equal to the matched string, so no violation would be reported for the last example and all the others would be auto-corrected – just what we want. 🎉

(The alternative would be to split the check to two separate ones, one fore checking the prefix and one the suffix whitespacing – not so beautiful as this blows up our lint.swift configuration file very quickly.)

Skip file content checks

While the includeFilters and excludeFilters arguments in the config file can be used to skip checks on specified files, sometimes it's necessary to make exceptions and specify that within the files themselves. For example this can become handy when there's a check which works 99% of the time, but there might be the 1% of cases where the check is reporting false positives.

For such cases, there are 2 ways to skip checks within the files themselves:

  1. AnyLint.skipHere: <CheckInfo.ID>: Will skip the specified check(s) on the same line and the next line.
var x: Int = 5 // AnyLint.skipHere: MinVarNameLength

// or

// AnyLint.skipHere: MinVarNameLength
var x: Int = 5
  1. AnyLint.skipInFile: <All or CheckInfo.ID>: Will skip All or specificed check(s) in the entire file.
// AnyLint.skipInFile: MinVarNameLength

var x: Int = 5
var y: Int = 5

or

// AnyLint.skipInFile: All

var x: Int = 5
var y: Int = 5

It is also possible to skip multiple checks at once in a line like so:

// AnyLint.skipHere: MinVarNameLength, LineLength, ColonWhitespaces

Check File Paths

The checkFilePaths method has all the same parameters like the checkFileContents method, so please read the above section to learn more about them. There's only one difference and one additional parameter:

  1. autoCorrectReplacement: Here, this will safely move the file using the path replacement.
  2. violateIfNoMatchesFound: Will report a violation if no matches are found if true. Default: false

As this method is about file paths and not file contents, the autoCorrectReplacement actually also fixes the paths, which corresponds to moving files from the before state to the after state. Note that moving/renaming files here is done safely, which means that if a file already exists at the resulting path, the command will fail.

By default, checkFilePaths will fail if the given regex matches a file. If you want to check for the existence of a file though, you can set violateIfNoMatchesFound to true instead, then the method will fail if it does not match any file.

Custom Checks

AnyLint allows you to do any kind of lint checks (thus its name) as it gives you the full power of the Swift programming language and it's packages ecosystem. The customCheck method needs to be used to profit from this flexibility. And it's actually the simplest of the three methods, consisting of only two parameters:

  1. checkInfo: Provides some general information on the lint check.
  2. customClosure: Your custom logic which produces an array of Violation objects.

Note that the Violation type just holds some additional information on the file, matched string, location in the file and applied autocorrection and that all these fields are optional. It is a simple struct used by the AnyLint reporter for more detailed output, no logic attached. The only required field is the CheckInfo object which caused the violation.

If you want to use regexes in your custom code, you can learn more about how you can match strings with a Regex object on the HandySwift docs (the project, the class was taken from) or read the code documentation comments.

When using the customCheck, you might want to also include some Swift packages for easier file handling or running shell commands. You can do so by adding them at the top of the file like so:

#!/usr/local/bin/swift-sh
import AnyLint // @Flinesoft
import ShellOut // @JohnSundell

Lint.logSummaryAndExit(arguments: CommandLine.arguments) {
    // MARK: - Variables
    let projectName: String = "AnyLint"

    // MARK: - Checks
    // MARK: LinuxMainUpToDate
    try Lint.customCheck(checkInfo: "LinuxMainUpToDate: The tests in Tests/LinuxMain.swift should be up-to-date.") { checkInfo in
        var violations: [Violation] = []

        let linuxMainFilePath = "Tests/LinuxMain.swift"
        let linuxMainContentsBeforeRegeneration = try! String(contentsOfFile: linuxMainFilePath)

        let sourceryDirPath = ".sourcery"
        try! shellOut(to: "sourcery", arguments: ["--sources", "Tests/\(projectName)Tests", "--templates", "\(sourceryDirPath)/LinuxMain.stencil", "--output", sourceryDirPath])

        let generatedLinuxMainFilePath = "\(sourceryDirPath)/LinuxMain.generated.swift"
        let linuxMainContentsAfterRegeneration = try! String(contentsOfFile: generatedLinuxMainFilePath)

        // move generated file to LinuxMain path to update its contents
        try! shellOut(to: "mv", arguments: [generatedLinuxMainFilePath, linuxMainFilePath])

        if linuxMainContentsBeforeRegeneration != linuxMainContentsAfterRegeneration {
            violations.append(
                Violation(
                    checkInfo: checkInfo,
                    filePath: linuxMainFilePath,
                    appliedAutoCorrection: AutoCorrection(
                        before: linuxMainContentsBeforeRegeneration,
                        after: linuxMainContentsAfterRegeneration
                    )
                )
            )
        }

        return violations
    }
}

Xcode Build Script

If you are using AnyLint for a project in Xcode, you can configure a build script to run it on each build. In order to do this select your target, choose the Build Phases tab and click the + button on the top left corner of that pane. Select New Run Script Phase and copy the following into the text box below the Shell: /bin/sh of your new run script phase:

if which anylint > /dev/null; then
    anylint -x
else
    echo "warning: AnyLint not installed, download it from https://github.com/Flinesoft/AnyLint"
fi

Next, make sure the AnyLint script runs before the steps Compiling Sources by moving it per drag & drop, for example right after Dependencies. You probably also want to rename it to somethng like AnyLint.

Note: There's a known bug when the build script is used in non-macOS platforms targets.

Regex Cheat Sheet

Refer to the Regex quick reference on rubular.com which all apply for Swift as well:

In Swift, there are some differences to regexes in Ruby (which rubular.com is based on) – take care when copying regexes:

  1. In Ruby, forward slashes (/) must be escaped (\/), that's not necessary in Swift.
  2. In Swift, curly braces ({ & }) must be escaped (\{ & \}), that's not necessary in Ruby.

Here are some advanced Regex features you might want to use or learn more about:

  1. Back references can be used within regexes to match previous capture groups.

    For example, you can make sure that the PR number and link match in PR: [#100](https://github.com/Flinesoft/AnyLint/pull/100) by using a capture group ((\d+)) and a back reference (\1) like in: \[#(\d+)\]\(https://[^)]+/pull/\1\).

    Learn more

  2. Negative & positive lookaheads & lookbehinds allow you to specify patterns with some limitations that will be excluded from the matched range. They are specified with (?=PATTERN) (positive lookahead), (?!PATTERN) (negative lookahead), (?<=PATTERN) (positive lookbehind) or (?<!PATTERN) (negative lookbehind).

    For example, you could use the regex - (?!None\.).* to match any entry in a CHANGELOG.md file except empty ones called None..

    Learn more

  3. Specifically you can use a lookbehind to make sure that the reported line of a regex spanning multiple lines only reports on the exact line where the developer needs to make a change, instead of one line before. That works because the pattern matched by a lookbehind is not considered part of the matching range.

    For example, consider a regex violating if there's an empty line after an opening curly brace like so: {\n\s*\n\s*\S. This would match the lines of func do() {\n\n return 5}, but what you actually want is it to start matching on the empty newline like so: (?<={\n)\s*\n\s*\S.

    See also #3

Donation

AnyLint was brought to you by Cihat Gündüz in his free time. If you want to thank me and support the development of this project, please make a small donation on PayPal. In case you also like my other open source contributions and articles, please consider motivating me by becoming a sponsor on GitHub or a patron on Patreon.

Thank you very much for any donation, it really helps out a lot! 💯

Contributing

Contributions are welcome. Feel free to open an issue on GitHub with your ideas or implement an idea yourself and post a pull request. If you want to contribute code, please try to follow the same syntax and semantic in your commit messages (see rationale here). Also, please make sure to add an entry to the CHANGELOG.md file which explains your change.

License

This library is released under the MIT License. See LICENSE for details.

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].