Quantcast
Channel: Active questions tagged jq - Stack Overflow
Viewing all articles
Browse latest Browse all 524

Pipe commands not getting killed after head exits

$
0
0

So, I'm not sure if the commands are actually important, but for background this is the command I was running:

aws s3 ls s3://REDACTED/ | jq -nR '[inputs | split(" +"; null).[3]] | reverse.[] | "bkt --ttl '1h' -- aws s3 cp s3://REDACTED/\(.) - | tac"' -r | bash | head

Basically, I wanted to get the last 10 lines from all the lines of all the objects (by name in lexicographic order). To do this, I list the bucket, then reverse it, then construct some commands to cat them in reverse order, then run them, then limit the output with head. For those who don't know, bkt is a wrapper that adds caching to any command...

When I ran that I was a bit surprised that it didn't quit after the first 10 lines, in-fact it seems like it is still running all the aws cp commands...

My question here is why didn't the bash command in the pipe abort? it should have gotten a SIGPIPE after head exits right?

EDIT:This is a less complicated command that does the same thing:

seq 100000000000 | sed -E $'s/(.*)/sh -c \'echo \\1; sleep .1\'/' | sh | head

EDIT:Note: As @Fravadona brings up, please use caution when piping commands into a shell interpreter. This is not an example of a robust solution to any specific problem


Viewing all articles
Browse latest Browse all 524

Trending Articles