I am writing a Bash script to create a csv from a sample inconsistent json data but running into a bunch of issues.
My json data looks like this :
json_data='[ {"ResourceARN": "arn","Tags": [ {"Key": "STAGE","Value": "st" } ] }, {"ResourceARN": "arn","Tags": [ {"Key": "STAGE","Value": "st" } ] }, {"ResourceARN": "arn","Tags": [ {"Key": "aws:cloudformation:stack-name","Value": "aud" }, {"Key": "CostCenter","Value": "Development" }, {"Key": "aws:cloudformation:stack-id","Value": "arn" }, {"Key": "STAGE","Value": "ec" }, {"Key": "Environment","Value": "development" }, {"Key": "Region","Value": "us-west-2" }, {"Key": "Service","Value": "MP" }, {"Key": "aws:cloudformation:logical-id","Value": "ApiGatewayRestApi" }, {"Key": "Team","Value": "VNP" } ] } and so on]'I am trying to create a csv with the following column names : ARN, Stage, CostCenter, Service, Domain, Team and the values should be what is found in the key value pairs of the Tags array and whatever key value pairs do not exist then just leave the column as blank.
My code looks like this :
# Define the columnscolumns=("ARN" "STAGE" "CostCenter" "Service" "Domain" "Team")# Create CSV headerheader=$(IFS=','; echo "${columns[*]}")# Function to get value for a specific key in the Tags arrayget_value() { key="$1" tags="$2" value=$(echo "$tags" | jq -r --arg key "$key" '.[] | select(.Key == $key) | .Value') if [ "$value" == "null" ]; then echo "" else echo "$value" fi}# Create the CSV datacsv_data=""for item in "${json_data[@]}"; do resource_arn=$(echo "$item" | jq -r '.ResourceARN') row="$resource_arn" for col in "${columns[@]}"; do value=$(get_value "$col" "$(echo "$item" | jq -c '.Tags')") row="$row,$value" done csv_data="$csv_data$row\n"done# Output the CSV to a fileecho -e "$header\n$csv_data" > output.csvecho "CSV data has been written to output.csv"But I keep running into
jq: error (at <stdin>:57): Cannot index array with string "ResourceARN"jq: error (at <stdin>:57): Cannot index array with string "Tags"jq: error (at <stdin>:57): Cannot index array with string "Tags"jq: error (at <stdin>:57): Cannot index array with string "Tags"jq: error (at <stdin>:57): Cannot index array with string "Tags"jq: error (at <stdin>:57): Cannot index array with string "Tags"jq: error (at <stdin>:57): Cannot index array with string "Tags"Basically the final CSV should look like
ARN STAGE CostCenter Service Domain Teamarn st. "" "" "" ""arn st "" "" "" "" arn ec Development MP. "" VNP....What am I doing wrong or is there a simple way of doing it? I would imagine jq would simply leave the values blank and output the ones having values?