Bitbuckets Pipelines create a step with parallel steps - yaml

Question

I want to have one step, that will setup everything and then in parallel run other steps. Currently I have something like this:
image: python:3.9.16-alpine
pipelines:
default:
- step:
runs-on:
- self.hosted
- regressiontests
name: First Step
clone:
enabled: false
caches:
- pip
script:
- apk add git
- apk add openssh-client
- git clone myrepository.git
- pip install -r myrepository/requirements.txt
- echo $ENV_FILE | base64 -d -i > myrepository/.env
artifacts:
- myrepository/**
- step:
runs-on:
- self.hosted
- regressiontests
name: Second Step
clone:
enabled: false
caches:
- pip
script:
- cd myrepository
- pip install -r requirements.txt
parallel:
- step:
name: Step 2.1
script:
- python fancy command 1
- step:
name: Step 2.2
script:
- python fancy command 2
- step:
name: Step 2.3
script:
- python fancy command 3
- step:
name: Step 2.4
script:
- python fancy command 4
But the only steps that I see is First Step and Second Step none of parallel steps is executed in pipelines

Answer 1

That's not the correct syntax for parallel steps. Checkout https://support.atlassian.com/bitbucket-cloud/docs/set-up-or-run-parallel-steps/
image: python:3.9.16-alpine
pipelines:
default:
- step:
name: First Step
script: []
- step:
name: Second Step
script: []
- parallel:
- step:
name: Step 3.1
script: []
- step:
name: Step 3.2
script: []
# ...
This what I think you are trying to achieve
BUT
Each step script happens in a new pristine docker container, so any setup must happen in the same script where the software will be used.
Therefore I am afraid your whole effort to speed up your steps setup is futile.
Instead, you'd like to tune your caches. For python you may want to cache both ~/.cache/pip and a virtualenv so that pip install -r ... instructions are sped up.
Plus, I have a feeling that bitbucket artifacts are quite slow so I'd expect disabling the repository clone in every step to be actually slower. I'd use a shallow clone instead with
clone:
depth: 1