Abstract JSON
Abstract JSON is a small golang package provides a parser for JSON with support of JSONPath, in case when you are not sure in its structure.
Method Unmarshal will scan all the byte slice to create a root node of JSON structure, with all its behaviors.
Method Marshal will serialize current Node object to JSON structure.
Each Node has its own type and calculated value, which will be calculated on demand.
Calculated value saves in atomic.Value, so it's thread safe.
Method JSONPath will returns slice of found elements in current JSON data, by JSONPath request.
Compare with other solutions
Check the cburgmer/json-path-comparison project.
Usage
package main
import (
"fmt"
"github.com/spyzhov/ajson"
)
func main() {
json := []byte(`...`)
root, _ := ajson.Unmarshal(json)
nodes, _ := root.JSONPath("$..price")
for _, node := range nodes {
node.SetNumeric(node.MustNumeric() * 1.25)
node.Parent().AppendObject("currency", ajson.StringNode("", "EUR"))
}
result, _ := ajson.Marshal(root)
fmt.Printf("%s", result)
}Console application
You can download ajson cli from the release page, or install from the source:
go get github.com/spyzhov/ajson/cmd/[email protected]Usage:
Usage: ajson "jsonpath" ["input"]
Read JSON and evaluate it with JSONPath.
Argument:
jsonpath Valid JSONPath or evaluate string (Examples: "$..[?(@.price)]", "$..price", "avg($..price)")
input Path to the JSON file. Leave it blank to use STDIN.
Examples:
ajson "avg($..registered.age)" "https://randomuser.me/api/?results=5000"
ajson "$.results.*.name" "https://randomuser.me/api/?results=10"
curl -s "https://randomuser.me/api/?results=10" | ajson "$..coordinates"
ajson "$" example.json
echo "3" | ajson "2 * pi * $"JSONPath
Current package supports JSONPath selection described at http://goessner.net/articles/JsonPath/.
JSONPath expressions always refer to a JSON structure in the same way as XPath expression are used in combination with an XML document. Since a JSON structure is usually anonymous and doesn't necessarily have a "root member object" JSONPath assumes the abstract name $ assigned to the outer level object.
JSONPath expressions can use the dot–notation
$.store.book[0].title
or the bracket–notation
$['store']['book'][0]['title']
for input paths. Internal or output paths will always be converted to the more general bracket–notation.
JSONPath allows the wildcard symbol * for member names and array indices.
It borrows the descendant operator .. from E4X and the array slice syntax proposal [start:end:step] from ECMASCRIPT 4.
Expressions of the underlying scripting language (<expr>) can be used as an alternative to explicit names or indices as in
$.store.book[(@.length-1)].title
using the symbol @ for the current object. Filter expressions are supported via the syntax ?(<boolean expr>) as in
$.store.book[?(@.price < 10)].title
Here is a complete overview and a side by side comparison of the JSONPath syntax elements with its XPath counterparts.
| JSONPath | Description |
|---|---|
$ |
the root object/element |
@ |
the current object/element |
. or [] |
child operator |
.. |
recursive descent. JSONPath borrows this syntax from E4X. |
* |
wildcard. All objects/elements regardless their names. |
[] |
subscript operator. XPath uses it to iterate over element collections and for predicates. In Javascript and JSON it is the native array operator. |
[,] |
Union operator in XPath results in a combination of node sets. JSONPath allows alternate names or array indices as a set. |
[start:end:step] |
array slice operator borrowed from ES4. |
?() |
applies a filter (script) expression. |
() |
script expression, using the underlying script engine. |
Script engine
Predefined constant
Package has several predefined constants.
e math.E float64
pi math.Pi float64
phi math.Phi float64
sqrt2 math.Sqrt2 float64
sqrte math.SqrtE float64
sqrtpi math.SqrtPi float64
sqrtphi math.SqrtPhi float64
ln2 math.Ln2 float64
log2e math.Log2E float64
ln10 math.Ln10 float64
log10e math.Log10E float64
true true bool
false false bool
null nil interface{}
You are free to add new one with function AddConstant:
AddConstant("c", NumericNode("speed of light in vacuum", 299_792_458))Examples
Using `true` in path
package main
import (
"fmt"
"github.com/spyzhov/ajson"
)
func main() {
json := []byte(`{"foo": [true, null, false, 1, "bar", true, 1e3], "bar": [true, "baz", false]}`)
result, _ := ajson.JSONPath(json, `$..[?(@ == true)]`)
fmt.Printf("Count of `true` values: %d", len(result))
}Output:
Count of `true` values: 3
Using `null` in eval
package main
import (
"fmt"
"github.com/spyzhov/ajson"
)
func main() {
json := []byte(`{"foo": [true, null, false, 1, "bar", true, 1e3], "bar": [true, "baz", false]}`)
result, _ := ajson.JSONPath(json, `$..[?(@ == true)]`)
fmt.Printf("Count of `true` values: %d", len(result))
}Output:
Count of `true` values: 3
Supported operations
Package has several predefined operators.
Precedence Operator
6 **
5 * / % << >> & &^
4 + - | ^
3 == != < <= > >= =~
2 &&
1 ||
** power integers, floats
+ sum integers, floats, strings
- difference integers, floats
* product integers, floats
/ quotient integers, floats
% remainder integers
& bitwise AND integers
| bitwise OR integers
^ bitwise XOR integers
&^ bit clear (AND NOT) integers
<< left shift integer << unsigned integer
>> right shift integer >> unsigned integer
== equals any
!= not equals any
< less any
<= less or equals any
> larger any
>= larger or equals any
=~ equals regex string strings
You are free to add new one with function AddOperation:
AddOperation("<>", 3, false, func(left *ajson.Node, right *ajson.Node) (node *ajson.Node, err error) {
result, err := left.Eq(right)
if err != nil {
return nil, err
}
return BoolNode("neq", !result), nil
})Examples
Using `regex` operator
package main
import (
"fmt"
"github.com/spyzhov/ajson"
)
func main() {
json := []byte(`[{"name":"Foo","mail":"[email protected]"},{"name":"bar","mail":"[email protected]"}]`)
result, err := ajson.JSONPath(json, `$.[?(@.mail =~ '.+@example\\.com')]`)
if err != nil {
panic(err)
}
fmt.Printf("JSON: %s", result[0].Source())
// Output:
// JSON: {"name":"Foo","mail":"[email protected]"}
}Output:
JSON: {"name":"Foo","mail":"[email protected]"}
Supported functions
Package has several predefined functions.
abs math.Abs integers, floats
acos math.Acos integers, floats
acosh math.Acosh integers, floats
asin math.Asin integers, floats
asinh math.Asinh integers, floats
atan math.Atan integers, floats
atanh math.Atanh integers, floats
avg Average array of integers or floats
b64decode b64 Decoding string
b64encode b64 Encoding string
b64encoden b64 Encoding (no padding) string
cbrt math.Cbrt integers, floats
ceil math.Ceil integers, floats
cos math.Cos integers, floats
cosh math.Cosh integers, floats
erf math.Erf integers, floats
erfc math.Erfc integers, floats
erfcinv math.Erfcinv integers, floats
erfinv math.Erfinv integers, floats
exp math.Exp integers, floats
exp2 math.Exp2 integers, floats
expm1 math.Expm1 integers, floats
factorial N! unsigned integer
floor math.Floor integers, floats
gamma math.Gamma integers, floats
j0 math.J0 integers, floats
j1 math.J1 integers, floats
length len array
log math.Log integers, floats
log10 math.Log10 integers, floats
log1p math.Log1p integers, floats
log2 math.Log2 integers, floats
logb math.Logb integers, floats
not not any
pow10 math.Pow10 integer
rand N*rand.Float64 float
randint rand.Intn integer
round math.Round integers, floats
roundtoeven math.RoundToEven integers, floats
sin math.Sin integers, floats
sinh math.Sinh integers, floats
sum Sum array of integers or floats
sqrt math.Sqrt integers, floats
tan math.Tan integers, floats
tanh math.Tanh integers, floats
trunc math.Trunc integers, floats
y0 math.Y0 integers, floats
y1 math.Y1 integers, floats
You are free to add new one with function AddFunction:
AddFunction("trim", func(node *ajson.Node) (result *Node, err error) {
if node.IsString() {
return StringNode("trim", strings.TrimSpace(node.MustString())), nil
}
return
})Examples
Using `avg` for array
package main
import (
"fmt"
"github.com/spyzhov/ajson"
)
func main() {
json := []byte(`{"prices": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]}`)
root, err := ajson.Unmarshal(json)
if err != nil {
panic(err)
}
result, err := ajson.Eval(root, `avg($.prices)`)
if err != nil {
panic(err)
}
fmt.Printf("Avg price: %0.1f", result.MustNumeric())
// Output:
// Avg price: 5.5
}Output:
Avg price: 5.5
Examples
Calculating AVG(price) when object is heterogeneous.
{
"store": {
"book": [
{
"category": "reference",
"author": "Nigel Rees",
"title": "Sayings of the Century",
"price": 8.95
},
{
"category": "fiction",
"author": "Evelyn Waugh",
"title": "Sword of Honour",
"price": 12.99
},
{
"category": "fiction",
"author": "Herman Melville",
"title": "Moby Dick",
"isbn": "0-553-21311-3",
"price": 8.99
},
{
"category": "fiction",
"author": "J. R. R. Tolkien",
"title": "The Lord of the Rings",
"isbn": "0-395-19395-8",
"price": 22.99
}
],
"bicycle": {
"color": "red",
"price": 19.95
},
"tools": null
}
}Unmarshal
package main
import (
"fmt"
"github.com/spyzhov/ajson"
)
func main() {
data := []byte(`{"store": {"book": [
{"category": "reference", "author": "Nigel Rees", "title": "Sayings of the Century", "price": 8.95},
{"category": "fiction", "author": "Evelyn Waugh", "title": "Sword of Honour", "price": 12.99},
{"category": "fiction", "author": "Herman Melville", "title": "Moby Dick", "isbn": "0-553-21311-3", "price": 8.99},
{"category": "fiction", "author": "J. R. R. Tolkien", "title": "The Lord of the Rings", "isbn": "0-395-19395-8", "price": 22.99}],
"bicycle": {"color": "red", "price": 19.95}, "tools": null}}`)
root, err := ajson.Unmarshal(data)
if err != nil {
panic(err)
}
store := root.MustKey("store").MustObject()
var prices float64
size := 0
for _, objects := range store {
if objects.IsArray() && objects.Size() > 0 {
size += objects.Size()
for _, object := range objects.MustArray() {
prices += object.MustKey("price").MustNumeric()
}
} else if objects.IsObject() && objects.HasKey("price") {
size++
prices += objects.MustKey("price").MustNumeric()
}
}
if size > 0 {
fmt.Println("AVG price:", prices/float64(size))
} else {
fmt.Println("AVG price:", 0)
}
}JSONPath:
package main
import (
"fmt"
"github.com/spyzhov/ajson"
)
func main() {
data := []byte(`{"store": {"book": [
{"category": "reference", "author": "Nigel Rees", "title": "Sayings of the Century", "price": 8.95},
{"category": "fiction", "author": "Evelyn Waugh", "title": "Sword of Honour", "price": 12.99},
{"category": "fiction", "author": "Herman Melville", "title": "Moby Dick", "isbn": "0-553-21311-3", "price": 8.99},
{"category": "fiction", "author": "J. R. R. Tolkien", "title": "The Lord of the Rings", "isbn": "0-395-19395-8", "price": 22.99}],
"bicycle": {"color": "red", "price": 19.95}, "tools": null}}`)
nodes, err := ajson.JSONPath(data, "$..price")
if err != nil {
panic(err)
}
var prices float64
size := len(nodes)
for _, node := range nodes {
prices += node.MustNumeric()
}
if size > 0 {
fmt.Println("AVG price:", prices/float64(size))
} else {
fmt.Println("AVG price:", 0)
}
}Eval
package main
import (
"fmt"
"github.com/spyzhov/ajson"
)
func main() {
json := []byte(`{"store": {"book": [
{"category": "reference", "author": "Nigel Rees", "title": "Sayings of the Century", "price": 8.95},
{"category": "fiction", "author": "Evelyn Waugh", "title": "Sword of Honour", "price": 12.99},
{"category": "fiction", "author": "Herman Melville", "title": "Moby Dick", "isbn": "0-553-21311-3", "price": 8.99},
{"category": "fiction", "author": "J. R. R. Tolkien", "title": "The Lord of the Rings", "isbn": "0-395-19395-8", "price": 22.99}],
"bicycle": {"color": "red", "price": 19.95}, "tools": null}}`)
root, err := ajson.Unmarshal(json)
if err != nil {
panic(err)
}
result, err := ajson.Eval(root, "avg($..price)")
if err != nil {
panic(err)
}
fmt.Println("AVG price:", result.MustNumeric())
}Marshal
package main
import (
"fmt"
"github.com/spyzhov/ajson"
)
func main() {
json := []byte(`{"store": {"book": [
{"category": "reference", "author": "Nigel Rees", "title": "Sayings of the Century", "price": 8.95},
{"category": "fiction", "author": "Evelyn Waugh", "title": "Sword of Honour", "price": 12.99},
{"category": "fiction", "author": "Herman Melville", "title": "Moby Dick", "isbn": "0-553-21311-3", "price": 8.99},
{"category": "fiction", "author": "J. R. R. Tolkien", "title": "The Lord of the Rings", "isbn": "0-395-19395-8", "price": 22.99}],
"bicycle": {"color": "red", "price": 19.95}, "tools": null}}`)
root := ajson.Must(ajson.Unmarshal(json))
result := ajson.Must(ajson.Eval(root, "avg($..price)"))
err := root.AppendObject("price(avg)", result)
if err != nil {
panic(err)
}
marshalled, err := ajson.Marshal(root)
if err != nil {
panic(err)
}
fmt.Printf("%s", marshalled)
}Benchmarks
Current package is comparable with encoding/json package.
Test data:
{ "store": {
"book": [
{ "category": "reference",
"author": "Nigel Rees",
"title": "Sayings of the Century",
"price": 8.95
},
{ "category": "fiction",
"author": "Evelyn Waugh",
"title": "Sword of Honour",
"price": 12.99
},
{ "category": "fiction",
"author": "Herman Melville",
"title": "Moby Dick",
"isbn": "0-553-21311-3",
"price": 8.99
},
{ "category": "fiction",
"author": "J. R. R. Tolkien",
"title": "The Lord of the Rings",
"isbn": "0-395-19395-8",
"price": 22.99
}
],
"bicycle": {
"color": "red",
"price": 19.95
}
}
}JSONPath: $.store..price
$ go test -bench=. -cpu=1 -benchmem
goos: linux
goarch: amd64
pkg: github.com/spyzhov/ajson
BenchmarkUnmarshal_AJSON 138032 8762 ns/op 5344 B/op 95 allocs/op
BenchmarkUnmarshal_JSON 117423 10502 ns/op 968 B/op 31 allocs/op
BenchmarkJSONPath_all_prices 80908 14394 ns/op 7128 B/op 153 allocs/op
License
MIT licensed. See the LICENSE file for details.