Quantcast
Channel: Active questions tagged jq - Stack Overflow
Viewing all articles
Browse latest Browse all 608

Curl list of links and save files according to URL hierarchy

$
0
0

From the first API call, a JSON response is returned which includes a bunch of links. I want to curl each link, and download the file to a location based on the last 3 path segments.

For example, if the URL is https://data.ninjakiwi.com/btd6/races/The_Olympics_lzlqex8k/metadata, then I want to download the file to races/The_Olympics_lzlqex8k/metadata.json. Apparently cut cannot handle indexing from the end, so I've hardcoded the number of slashes (including 2 in https://) to skip. I came up with an example command. Is there a more elegant way to write this? xargs has showed me this is probably possible without a loop and less subshell usage.

for url in $(curl https://data.ninjakiwi.com/btd6/races | jq -r '.body [] | .leaderboard, .metadata'); do         echo $url; curl --create-dirs -o $(echo $url | cut -d/ -f5-).json $url; done

Viewing all articles
Browse latest Browse all 608

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>