Engarde - Stay on guard with Envoy Access Logs
Parse default envoy access logs like a champ with engarde and jq
Motivation
Envoy access log messages are packed with a lot of useful information but in an unstructured log format. Without prior context, or even with context it can get cumbersome to visually inspect these log messages to extract useful information.
I was inspired by this tweet from Megan O'Keefe on twitter to create this tool for better readability of the envoy/istio-proxy access logs.
In addition, a special shout out to Richard Li from ambassador.io for this excellent article that provides more details on each of the subcomponents of the log message.
Installing
Homebrew
brew tap nitishm/homebrew-engarde
brew install engardeScoop
scoop bucket add engarde https://github.com/nitishm/scoop-bucket-engarde
scoop install engardeSource
go get github.com/nitishm/engardeThis should install the compiled binary to your $GOBIN (or $GOPATH/bin).
Otherwise, clone the repository and build the binary manually using,
This package uses gomodules. Ensure that GO111MODULE=on is set if building outside $GOPATH in go version 1.11+
git clone https://github.com/nitishm/engarde.git
cd engarde/
go build -o engarde .
mv engarde /usr/local/bin/
export PATH=$PATH:/usr/local/bin/Example (default format only)
Prerequisites
jq must be in your $PATH. If you do not have jq installed please download the binary here
Envoy
echo '[2016-04-15T20:17:00.310Z] "POST /api/v1/locations HTTP/2" 204 - 154 0 226 100 "10.0.35.28" "nsq2http" "cc21d9b0-cf5c-432b-8c7e-98aeb7988cd2" "locations" "tcp://10.0.2.1:80"' | engarde | jq2019/09/03 22:31:33 Reading input from STDIN. Use the pipe "|" operator to redirect traffic to engarde
{
"authority": "locations",
"bytes_received": "154",
"bytes_sent": "0",
"duration": "226",
"method": "POST",
"protocol": "HTTP/2",
"request_id": "cc21d9b0-cf5c-432b-8c7e-98aeb7988cd2",
"response_flags": "-",
"status_code": "204",
"timestamp": "2016-04-15T20:17:00.310Z",
"upstream_service": "tcp://10.0.2.1:80",
"upstream_service_time": "100",
"uri_path": "/api/v1/locations",
"user_agent": "nsq2http",
"original_message": "[2016-04-15T20:17:00.310Z] \"POST /api/v1/locations HTTP/2\" 204 - 154 0 226 100 \"10.0.35.28\" \"nsq2http\" \"cc21d9b0-cf5c-432b-8c7e-98aeb7988cd2\" \"locations\" \"tcp://10.0.2.1:80\""
}Istio Proxy
kubectl logs -f foo-app-1 -c istio-proxy | engarde --use-istio | jq{
"authority": "hello-world",
"bytes_received": "148",
"bytes_sent": "171",
"duration": "4",
"method": "GET",
"protocol": "HTTP/1.1",
"request_id": "c0ce81db-4f5a-9134-8a5c-f8c076c91652",
"response_flags": "-",
"status_code": "200",
"timestamp": "2019-09-03T05:37:41.341Z",
"upstream_service": "192.168.89.50:9001",
"upstream_service_time": "3",
"upstream_cluster": "outbound|80||hello-world.default.svc.cluster.local",
"upstream_local": "-",
"downstream_local": "10.97.86.53:80",
"downstream_remote": "192.168.167.113:39953",
"uri_path": "/index",
"user_agent": "-",
"mixer_status": "-",
"original_message": "[2019-09-03T05:37:41.341Z] \"GET /index HTTP/1.1\" 200 - \"-\" 148 171 4 3 \"-\" \"-\" \"c0ce81db-4f5a-9134-8a5c-f8c076c91652\" \"hello-world\" \"192.168.89.50:9001\" outbound|80||hello-world.default.svc.cluster.local - 10.97.86.53:80 192.168.167.113:39953 -"
}Projects using Engarde
A UI powered by Engarde and Envoy docs for viewing envoy access logs on the browser : greghanson/engarde-viewer