GitPlanet

Cheap and reliable Node.js hosting starts at $3/month, and $1/month static HTML hosting

Created with love in Canada, visit hostnodejs.com today

Feel like to post an Ad? Learn Details
All Projects → ahmet2mir → wildq

ahmet2mir / wildq

Licence: Apache-2.0 license
Command-line TOML/JSON/INI/YAML/XML/HCL processor using jq c bindings

Programming Languages

python
139335 projects - #7 most used programming language
Makefile
30231 projects
shell
77523 projects

Labels

cliproductivityyamltomljsonterminaltoolxmljqhcl

Projects that are alternatives of or similar to wildq

parse it
A python library for parsing multiple types of config files, envvars & command line arguments that takes the headache out of setting app configurations.
Stars: ✭ 86 (+290.91%)
Mutual labels:  yaml, toml, hcl
transfer
Converts from one encoding to another. Supported formats HCL ⇄ JSON ⇄ YAML⇄TOML⇄XML⇄plist⇄pickle⇄properties ...
Stars: ✭ 70 (+218.18%)
Mutual labels:  yaml, toml, hcl
Resticprofile
Configuration profiles for restic backup
Stars: ✭ 48 (+118.18%)
Mutual labels:  yaml, toml, hcl
Config
📝 Go config manage(load,get,set). support JSON, YAML, TOML, INI, HCL, ENV and Flags. Multi file load, data override merge, parse ENV var. Go应用配置加载管理，支持多种格式，多文件加载，远程文件加载，支持数据合并，解析环境变量名
Stars: ✭ 225 (+922.73%)
Mutual labels:  yaml, toml, hcl
Config Lite
A super simple & flexible & useful config module.
Stars: ✭ 78 (+254.55%)
Mutual labels:  yaml, toml
Re Txt
converts text-formats from one to another, it is very useful if you want to re-format a json file to yaml, toml to yaml, csv to yaml, ... etc
Stars: ✭ 59 (+168.18%)
Mutual labels:  yaml, toml
Yq
Command-line YAML, XML, TOML processor - jq wrapper for YAML/XML/TOML documents
Stars: ✭ 1,688 (+7572.73%)
Mutual labels:  yaml, jq
Oq
A performant, and portable jq wrapper to facilitate the consumption and output of formats other than JSON; using jq filters to transform the data.
Stars: ✭ 132 (+500%)
Mutual labels:  yaml, jq
Mconfig
MCONFIG is a lightweight Golang library for integrating configs files like (json, yml, toml) and environment variables into one config struct.
Stars: ✭ 28 (+27.27%)
Mutual labels:  yaml, toml
Datafiles
A file-based ORM for Python dataclasses.
Stars: ✭ 113 (+413.64%)
Mutual labels:  yaml, toml
Rq
Record Query - A tool for doing record analysis and transformation
Stars: ✭ 1,808 (+8118.18%)
Mutual labels:  yaml, toml
Tfk8s
A tool for converting Kubernetes YAML manifests to Terraform HCL
Stars: ✭ 167 (+659.09%)
Mutual labels:  yaml, hcl
Ansible Config encoder filters
Ansible role used to deliver the Config Encoder Filters.
Stars: ✭ 48 (+118.18%)
Mutual labels:  yaml, toml
Night Config
Powerful java configuration library for toml, yaml, hocon, json and in-memory configurations
Stars: ✭ 93 (+322.73%)
Mutual labels:  yaml, toml
Configr
Implements the JSON, INI, YAML and TOML parser, for R setting and writing of configuration file.
Stars: ✭ 38 (+72.73%)
Mutual labels:  yaml, toml
Config Lint
Command line tool to validate configuration files
Stars: ✭ 118 (+436.36%)
Mutual labels:  yaml, hcl
Simple Settings
A simple way to manage your project settings.
Stars: ✭ 165 (+650%)
Mutual labels:  yaml, toml
Python Benedict
dict subclass with keylist/keypath support, I/O shortcuts (base64, csv, json, pickle, plist, query-string, toml, xml, yaml) and many utilities. 📘
Stars: ✭ 204 (+827.27%)
Mutual labels:  yaml, toml
Dasel
Query, update and convert data structures from the command line. Comparable to jq/yq but supports JSON, TOML, YAML, XML and CSV with zero runtime dependencies.
Stars: ✭ 759 (+3350%)
Mutual labels:  yaml, toml
Hugo Elasticsearch
Generate Elasticsearch indexes for Hugo static sites by parsing front matter
Stars: ✭ 19 (-13.64%)
Mutual labels:  yaml, toml
View All Similar Projects ➔

wildq - Command-line TOML/JSON/INI/YAML/XML/HCL processor using jq c bindings

GitHub image Build Status

Purpose of this package is to provide a simple wrapper arround jq for different formats. I'm tired of searching a package doing yaml jq, toml jq, ini jq etc. mainly used for scripting.

This script uses:

  • @mwilliamson Python bindings on top of @stedolan famous jq lib
  • swiss knife for coloration pygments
  • binary built with pyinstaller
  • easy CLI with click
  • for supported types sources, check table Supported file types

Installation

You could install via pipx, pip, brew, deb, rpm or fallback to binary version.

Pipx

pipx install and run python applications in isolated environments

pipx install wildq

Pip

pip install wildq

MacOS

brew install ahmet2mir/tap/wildq
brew install ahmet2mir/tap/wq

Debian (no gpg signature) >= 10

VERSION=$(curl -s "https://api.github.com/repos/ahmet2mir/wildq/releases/latest" | grep '"tag_name":' | sed -E 's/.*"v([^"]+)".*/\1/')
curl -sL https://github.com/ahmet2mir/wildq/releases/download/v${VERSION}/wildq_${VERSION}-1_amd64.deb -o wildq_${VERSION}-1_amd64.deb
sudo dpkg -i wildq_${VERSION}-1_amd64.deb

wq --help
wildq --help

Centos (no gpg singature) >= 7

VERSION=$(curl -s "https://api.github.com/repos/ahmet2mir/wildq/releases/latest" | grep '"tag_name":' | sed -E 's/.*"v([^"]+)".*/\1/')
curl -sL https://github.com/ahmet2mir/wildq/releases/download/v${VERSION}/wildq-${VERSION}-1.x86_64.rpm -o wildq-${VERSION}-1.x86_64.rpm
sudo yum install -y ./wildq-${VERSION}-1.x86_64.rpm

wq --help
wildq --help

Gentoo

@zmedico did a portage on app-misc

emerge -av app-misc/wildq

GNU/Linux Binary

Compiled using glibc 2.17, it should work on lot of stable/LTS distros.

mkdir -p ~/bin/
VERSION=$(curl -s "https://api.github.com/repos/ahmet2mir/wildq/releases/latest" | grep '"tag_name":' | sed -E 's/.*"v([^"]+)".*/\1/')
curl -sL https://github.com/ahmet2mir/wildq/releases/download/v${VERSION}/wildq-${VERSION}-linux-x86_64.tar.gz -o wildq-${VERSION}-linux-x86_64.tar.gz
tar xvfz wildq-${VERSION}-linux-x86_64.tar.gz -C ~/bin
export PATH="~/bin:$PATH"

wq --help
wildq --help

Windows

Wildq use jq.py and it's not yet available on windows platforms. I tried to compile it without windows machine and I failed, and I don't had the time to try to understand how Windows / C binding / Python works. If anybody would contribute, there is an open issue (jq and onigurama are 'compilable' on Windows so I think that someone confortable with that OS could make it);

Supported file types

type color ordering output source
hcl json no json pyhcl by @virtuald
ini yes no yes ConfigParser
json yes yes yes json
toml yes no yes toml by @uiri
xml yes no yes xmldict by @martinblech
yaml yes yes yes pyyaml

Usage

$ wildq --help
Usage: wildq [OPTIONS] JQ_FILTER [FILE]

Options:
  -c, --compact-output            compact instead of pretty-printed output
  -r, --raw                       output raw strings, not content texts
  -C, --color-output              colorize content (default), mutally
                                  exclusive with --monochrome-output

  -M, --monochrome-output         monochrome (don't colorize content), mutally
                                  exclusive with --color-output

  --hcl                           Combine --input hcl --output json, mutally
                                  exclusive with other Combined options

  --ini                           Combine --input ini --output json, mutally
                                  exclusive with other Combined options

  --json                          Combine --input json --output json, mutally
                                  exclusive with other Combined options

  --toml                          Combine --input toml --output json, mutally
                                  exclusive with other Combined options

  --xml                           Combine --input xml --output json, mutally
                                  exclusive with other Combined options

  --yaml                          Combine --input yaml --output json, mutally
                                  exclusive with other Combined options

  -i, --input [hcl|ini|json|toml|xml|yaml]
                                  Define the content type of file, mutally
                                  exclusive with Combined option

  -o, --output [hcl|ini|json|toml|xml|yaml]
                                  Define the content type of printed output,
                                  mutally exclusive with Combined option
                                  (default input format)

  --version                       Show the version and exit.
  --help                          Show this message and exit.

For backward compatibility in previous version only --[yaml|json|toml|ini|xml|hcl] was possible with default to json output. We still keep Monochrome, raw and json output with thoses options. Output was similar to jq -MCr (no color, no compact and no quote on single value)

But now, by default it's colorized, not raw and if you specify input using -i or --input output will be the same format.

There is also a shorter command wq comming with the package.

Like jq cli, wildq supports both of stdin and file to the function

See examples to get some example.

Content of examples/json.json

{
    "general": {
        "user": "admin"
    },
    "keys": [
        {"key": "value1"},
        {"key": "value2"},
        "alone"
    ]
}

cat examples/json.json | wildq -i json ".keys[]"
{
    "key": "value1"
}
{
    "key": "value2"
}
alone

or

wildq -i json ".keys[]" examples/json.json
{
    "key": "value1"
}
{
    "key": "value2"
}
alone

or

wq -i json ".keys[]" examples/json.json
{
    "key": "value1"
}
{
    "key": "value2"
}
alone

For TOML

cat examples/toml.toml | wildq -i toml ".keys[]"
{
    "key": "value1"
}
{
    "key": "value2"
}
alone

For INI (no array)

cat examples/ini.ini | wildq -i ini ".keys"
{
    "key1": "value1",
    "key2": "value2"
}

For XML

cat examples/xml.xml | wildq -i xml "."
{
    "root": {
        "general": {
            "user": "admin"
        },
        "keys": {
            "element": [
                {
                    "key": "value1"
                },
                {
                    "key": "value2"
                },
                "alone"
            ]
        }
    }
}

For YAML

cat examples/yaml.yaml  | wildq -i yaml ".keys[]"
{
    "key1": "value1"
}
{
    "key2": "value2"
}
alone

For HCL

cat examples/hcl.hcl  | wildq -i hcl ".keys[]"
{
    "key": "value1"
}
{
    "key": "value2"
}

Tips and tricks

Loop on keys in bash without creating a subshell

wildq -i toml "keys[]" examples/toml.toml | while read -r key 
do
    echo "Getting key ${key}"
done

You could also found some examples on some blogs/websites:

TODO

  • add tests...
  • add more control over filters and files
  • use click for the CLI
  • support different output
  • detect automagically filetype
  • support all jq types
  • ordering

Contributing

Merge requests are welcome :)

License

Licensed under the terms of the Apache License, Version 2.0.

Repository URL

https://github.com/ahmet2mir/wildq

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].