How can I use a bash script to extract the data from this klipfolio table into a csv file?
https://app.klipfolio.com/published/6b16ab677623c60708ba3ef462e6ad8e/football-victoria-referee-appointments
All columns and rows are required, including the name of the Referee, AR1 & AR2 (which are visible in the page source). Example of required output (top 3 rows only):
DATE,TIME,COMPETITION,DIVISION,HOME TEAM,AWAY TEAM,VENUE,REFEREE,AR1,AR2,REFEREE NAME,AR1 NAME,AR2 NAMEFriday 2 May,18:15,Veto Sports State League (Womens),Women 1 South-East Reserves,Croydon City Soccer Club (SLW1),Ashburton United SC (SLW1),Dorset Recreation Reserve,FV Appointed,,,Kosta Kelly,,Friday 2 May,18:30,Veto Sports State League (Womens),Women 3 South,Chelsea FC (SLW3),Monash University SC - Women's Thirds (SLW3),Edithvale Recreation Reserve,FV Appointed,,,Sarah Rogers,,Friday 2 May,18:30,Veto Sports State League (Womens),Women 4 East,Mooroolbark SC (SLW4),Eastern Lions SC (SLW4),Esther Park,FV Appointed,,,Pearl Peckrose,,My attempts thus far have yielded the first 100 unique values for each column. Starting to feel like I may be barking up the wrong tree?
#!/bin/bashFileBase="${HOME}/tmp/RefAppts_tmp"RefApptUrl="https://app.klipfolio.com/published/6b16ab677623c60708ba3ef462e6ad8e/football-victoria-referee-appointments"curl ${RefApptUrl} -o $FileBase.1#Extact the line with the data stringcat $FileBase.1 | grep dashboardSchemaString | head -1 > $FileBase.2#Extract the data string from within the double quotescut -d"\"" -f2 $FileBase.2 > $FileBase.3#Make readable json by unescaping the data stringecho -e $(cat $FileBase.3) > $FileBase.4#Extract a subset of the data which looks interesting. jq .[].klips[].components[].components $FileBase.4 > $FileBase.5#Extract Venuesjq '.[] | select(.displayName=="FV Venue")' $FileBase.5 > $FileBase.6#Extract TIMEsjq '.[] | select(.displayName=="TIME")' $FileBase.5 > $FileBase.7#And so on for each column