Shell Scripting jq yq Intermediate May 2026

Shell Scripting — JSON & YAML Processing

Master jq for JSON parsing, filtering, and construction. Learn yq for YAML manipulation. Build safe JSON payloads for APIs, parse Kubernetes and Docker output, and transform structured data in pipelines.

Modern infrastructure lives in JSON and YAML — Kubernetes manifests, Docker configs, API responses, cloud provider outputs. jq is the essential tool for JSON and yq for YAML. Both let you read, filter, transform, and write structured data directly from bash without temp files or Python scripts.

1 jq — read and filter JSON

BASH

# ── Basic extraction ──────────────────────────────────────
echo '{"name":"prod-01","cpu":85,"status":"up"}' | jq '.name'     # "prod-01"
echo '{"name":"prod-01","cpu":85}'             | jq -r '.name'   # prod-01 (no quotes)

# ── Nested fields ─────────────────────────────────────────
echo '{"data":{"host":"db-01","port":3306}}' \
  | jq -r '.data.host'                    # db-01
echo '{"tags":["web","prod"]}' \
  | jq -r '.tags[0]'                       # web

# ── Array iteration ───────────────────────────────────────
echo '[{"name":"web-01"},{"name":"web-02"}]' \
  | jq -r '.[].name'                       # web-01 then web-02

# ── Filter and select ─────────────────────────────────────
echo '[{"name":"web-01","up":true},{"name":"web-02","up":false}]' \
  | jq -r '.[] | select(.up==false) | .name'  # web-02

# ── Multiple fields ───────────────────────────────────────
echo '{"host":"db-01","port":3306,"db":"myapp"}' \
  | jq -r '[.host,.port,.db] | @csv'     # "db-01",3306,"myapp"
echo '{"host":"db-01","port":3306}' \
  | jq -r '"\(.host):\(.port)"'           # db-01:3306

# ── Assign to variables ───────────────────────────────────
json=$(curl -sf https://api.example.com/server)
host=$(echo "${json}" | jq -r '.host')
port=$(echo "${json}" | jq -r '.port')
echo "Connecting to ${host}:${port}"

2 jq — build and transform JSON

BASH

# ── Build JSON safely (no string concatenation) ───────────
alert=$(jq -n \
  --arg host "$(hostname)" \
  --arg msg  "Disk 92% full" \
  --argjson pct 92 \
  '{"host":$host,"message":$msg,"percent":$pct}')

echo "${alert}"
# {"host":"prod-01","message":"Disk 92% full","percent":92}

# ── Modify existing JSON ──────────────────────────────────
echo '{"status":"pending","retries":0}' \
  | jq '.status="running" | .retries+=1'

# ── Merge objects ─────────────────────────────────────────
echo '{"a":1,"b":2}' | jq '. + {"c":3}'   # {"a":1,"b":2,"c":3}

# ── Map over array ────────────────────────────────────────
echo '[{"name":"WEB-01"},{"name":"WEB-02"}]' \
  | jq '[.[] | .name |= ascii_downcase]'

# ── Real-world: parse kubectl output ─────────────────────
kubectl get pods -o json \
  | jq -r '.items[] | "\(.metadata.name) \(.status.phase)"'

# ── Parse docker inspect ─────────────────────────────────
docker inspect mycontainer \
  | jq -r '.[0].NetworkSettings.Networks[].IPAddress'

3 yq — read and write YAML

BASH

# Install yq (Mike Farah's Go version — most common)
wget -qO /usr/local/bin/yq \
  https://github.com/mikefarah/yq/releases/latest/download/yq_linux_amd64
chmod +x /usr/local/bin/yq

# ── Read YAML values ──────────────────────────────────────
yq '.database.host' config.yaml
yq '.database.port' config.yaml
yq '.servers[0].name' inventory.yaml
yq '.servers[].name' inventory.yaml    # all server names

# ── Update YAML in place ──────────────────────────────────
yq '.image.tag = "v2.5.0"' -i deployment.yaml
yq '.replicas = 4' -i deployment.yaml

# ── Set image tag in Kubernetes deployment ────────────────
NEW_TAG="v2.5.0"
yq ".spec.template.spec.containers[0].image = \"myapp:${NEW_TAG}\"" \
  -i k8s/deployment.yaml

# ── Convert YAML to JSON ──────────────────────────────────
yq -o=json config.yaml | jq '.database'

# ── Read multiple docs (--- separated) ───────────────────
yq 'select(.kind=="Deployment") | .metadata.name' k8s-bundle.yaml

# ── Validate YAML syntax ─────────────────────────────────
yq '.' config.yaml >/dev/null 2>&1 \
  && echo "Valid YAML" || echo "Invalid YAML"

bash — jq and yq in action

vriddh@prod-01:~/scripts$kubectl get pods -o json | jq -r '.items[] | "\(.metadata.name) \(.status.phase)"'

myapp-web-7d9f4-xk2p9 Running

myapp-web-7d9f4-bm3r1 Running

myapp-worker-5c8b2-qw4s Pending

vriddh@prod-01:~/scripts$yq '.spec.template.spec.containers[0].image = "myapp:v2.5.0"' -i deployment.yaml && yq '.spec.template.spec.containers[0].image' deployment.yaml

myapp:v2.5.0

█

⚠ Never parse JSON with grep/sed/awk — JSON field order is not guaranteed, values can span multiple lines, and strings can contain anything. Always use jq for JSON and yq for YAML. A one-liner like grep -o '"host":"[^"]*"' will break on the first edge case.

✔ jq and yq essentials — Use jq -r (raw) when assigning to bash variables. Use jq -n --arg key value to build JSON safely. Use jq '.[] | select(.field=="value")' to filter arrays. Use yq -i to update YAML files in place. Install yq from the official GitHub releases — the Python yq from pip has different syntax.