bqv
The simplest tool to manage views of BigQuery.
How to install
go install github.com/k-kawa/bqvOr you can use bqv as a Docker image which is available at kkawa/bqv
How to use
Make a directory based on the names of the dataset and the view that you want to create.
$ mkdir -p your_dataset/your_viewMake a query.sql file in it, which is going to be used to create the view.
$ cat <<EOF > your_dataset/your_view/query.sql
SELECT 1 AS one
EOF(Optional) You can also make a meta.json file in it to describe the meta data of the view.
The supported option is description, schema and labels the value of which is to be the description of it for now.
(We want to suport more. see the issues
$ cat <<EOF > your_dataset/your_view/meta.json
{
"description": "this is my awesome view!",
"schema": [
{"name": "my_column_name_1", "description": "your column description 1!!"},
{"name": "my_column_name_2", "description": "your column description 2!!"},
{"name": "my_column_name_3", "description": "your column description 3!!"}
],
labels: {
"key": "value",
"key": "value"
}
}
EOFList the view names which are going to be managed with bqv list command.
$ bqv list
your_dataset.your_viewMake the view into the GCP project named your_project with bqv apply command.
$ bqv apply --projecID=your_project
INFO[0001] Creating view(your_dataset.your_view)
bra bra bra ....Destroy all the created views in the GCP project with bqv destroy command.
$ bqv destroy --projectID=your_project
INFO[0001] Deleting view your_dataset.your_viewWith parameter file
You can use the query.sql file as a template and replace the placeholders in it when you run bqv apply.
# Create another directory
$ mkdir -p your_dataset/your_new_view
# Create the new query.sql following the Golang's template syntax.
$ cat <<EOF > your_dataset/your_new_view/query.sql
SELECT "{{.data}}" AS data
EOF
# Prepare a JSON file the keys and values in which are going to fill the query.sql
$ cat <<EOF > parameters.json
{"data": "data"}
EOF
# Run bqv apply with the parameters.json
$ bqv apply --paramFile=parameters.json