Code to fetch BitBucket repos This actually works nicely (we have > 2k repos) but I would really like it to start producing results right away as soon as it has any. Mostly so it seems faster than it is (by being more responsive).

   bbrepos(){
        #clone[1] is ssh using a filter made the escaping really ugly. 
        #If it becomes necessary we can use a jq arg
        # get max number of repos, ceil(repos / 50), then create a page 
        #sequence and curl them all
        local url=https://api.bitbucket.org/2.0/repositories/twengg
        local project=${1:+"q=project.key=\"$1\""}
        curl -snL  "$url?pagelen=0&amp;page=1" | jq '"$(((\(.size) / 50) + 1 ))"' \
            | xargs bash -c 'eval "echo $1"' _ | xargs seq 1 | xargs -L 1 printf "?pagelen=50&amp;page=%s&amp;${1:-$project}\n" \
            | xargs -I {} -L 1 -P 20 -I {} bash -c 'curl -snL  "$1/$2" | jq -er .values[].links.clone[1].href' _ $url {} \ #clone[1] is ssh
            | sort -u
    }


Sample workflow:

bbsearch something | xargs -P 20 git clone


Alternatively, it might also be handy to have these other workflows:

bbsearch something other thing | xargs -P 20 git clone

cat things-to-find.txt | bbsearch | xargs -P 20 git clone


Sample payload (it's normally huge, so I stripped it down to what is needed):

{
  "pagelen": 1,
  "size": 3054,
  "values": [
    {
      "links": {
        "clone": [
          {
            "href": "https://chb0bitbucket@bitbucket.org/twengg/development-process.git",
            "name": "https"
          },
          {
            "href": "git@bitbucket.org:twengg/development-process.git",
            "name": "ssh"
          }
        ],
        "self": {
          "href": "https://api.bitbucket.org/2.0/repositories/twengg/development-process"
        }
      }
    }
  ],
  "page": 1,
  "next": "https://api.bitbucket.org/2.0/repositories/twengg?page=2"
}
eval is evil

Always look for ways to avoid eval.

Relevant snippet from the posted code:


curl -snL  "$url?pagelen=0&amp;page=1" \
    | jq '"$(((\(.size) / 50) + 1 ))"' \
    | xargs bash -c 'eval "echo $1"' _



The code uses jq to create a formatted expression to evaluate by Bash,
with .size injected in the middle of the expression.

A better way to achieve the same thing would be to make jq simply output the variable itself, and then write the expression in Bash:

curl -snL  "$url?pagelen=0&amp;page=1" \
    | jq -r .size \
    | xargs bash -c 'echo $((($1 / 50) + 1))' _


Unnecessary pipeline

It's strange to use xargs to process one line of input,
and it's strange to spawn another Bash just to evaluate an expression.
There's really no need to force a chain of operations in a single pipeline.

I'd rewrite the above snippet like this:

repos_count=$(curl -snL "$url?pagelen=0&amp;page=1" | jq -r .size)
((pages_count = (repos_count / 50) + 1))


Store positional arguments in variables with descriptive names

It's a mental burden to have to remember what $1 and $2 mean.
I'd store them in local variables with descriptive names at the beginning of the function.

Avoid seq

Is it obsolete, and not available in all systems.
Use a Bash counting loop instead:

for ((page = 1; page &lt;= pages_count; page++)); do
    ...
done \
| xargs -L 1 ...


Break lines in long pipelines

It's easier to read code when there's one statement per line.
This is especially true when a line is so long you have to scroll to the right to see it.
So instead of this:


| xargs bash -c 'eval "echo $1"' _ | xargs seq 1 | xargs -L 1 printf "?pagelen=50&amp;page=%s&amp;${1:-$project}\n" \
| xargs -I {} -L 1 -P 20 -I {} bash -c 'curl -snL  "$1/$2" | jq -er .values[].links.clone[1].href' _ $url {} \ #clone[1] is ssh
| sort -u



I recommend to write like this:

| xargs bash -c 'eval "echo $1"' _ \
| xargs seq 1 \
| xargs -L 1 printf "?pagelen=50&amp;page=%s&amp;${1:-$project}\n" \
| xargs -I {} -L 1 -P 20 -I {} bash -c 'curl -snL  "$1/$2" \
| jq -er .values[].links.clone[1].href' _ $url {} \
| sort -u


Always double-quote variables used as command line arguments

In | jq -er .values[].links.clone[1].href' _ $url {}
you forgot to double-quote $url.