0Hello,
I have a json stream coming into my aws firehose that can contain either metrics or report key-value pairs based on the data source. They look like this -
metrics stream
{ "metric": { "k1" : "v1", "k2" : "v2", ... } }
report stream
{ "report": { "k1" : "v1", "k2" : "v2", ... } }
I am trying to use JQ expressions to check if the json has("metric")/has("report") and then derive the values from the blob. or eg,
if has("metric") then "metric" else "report" end
This seemed valid syntax from the online JQ Playground(ref). However, it fails when it is defined as a Dynamic partitioning Key in the firehose. I want to read different keys from the json to define the output of the stream. For eg, both Keys - K1, K2 are important to be read and they are dependent on being able to parse if the parent key is metric/report.
In this stream, metirc/report form the 1st level partition key, followed by the K1 value. I am using below JQ syntax to derive K1,
if has("metric") then .metric.k1 else .report.k1 end
Error message -
JQ Subprocess failed due to: jq: error: syntax error, unexpected if(Unix shell quoting issues?) at , line 2: {dataType:ifhas("metric") then "metric" else "report" end} jq: 1 compile error
Error code: DynamicPartitioning.MetadataExtractionFailed
What's the mistake with JQ expression here. It seems pretty normal here but I am unable to get the right expression to verify the metric or report key.