GitHub Actions step executed while it shouldn't, based on a conditional - bash

Question

I have the following steps' sequence in a GitHub Actions job (first one is used more or less for debugging purposes
env:
FAIL_OUTCOME: 'fail'
- name: debug
shell: bash
run: |
echo "evaluation-1 result is ${{ steps.evaluation_1.outputs.evaluation-1-outcome }}
echo "evaluation-2 result is ${{ steps.evaluation_2.outputs.evaluation-2-outcome }}
echo $FAIL_OUTCOME
- name: send slack failure
if: ${{ steps.evaluation_1.outputs.evaluation-1-outcome }} == $FAIL_OUTCOME || ${{ steps.evaluation_2.outputs.evaluation-2-outcome }} == $FAIL_OUTCOME
uses: rtCamp/action-slack-notify#v2
env:
...
- name: send slack success
if: ${{ steps.evaluation_1.outputs.evaluation-1-outcome }} != $FAIL_OUTCOME && ${{ steps.evaluation_2.outputs.evaluation-2-outcome}} != $FAIL_OUTCOME
uses: rtCamp/action-slack-notify#v2
env:
...
Here is the outcome of the debug action:
echo "evaluation-1 result is
echo "evaluation-2 result is fail
where it seems that first outcome is not set.
However, what puzzles me is that success action is also executed, i.e.
${{ steps.evaluation_1.outputs.evaluation-1-outcome }} != $FAIL_OUTCOME && ${{ steps.evaluation_2.outputs.evaluation-2-outcome}} != $FAIL_OUTCOME
becomes true. How is it possible?
To provide for more context, the outputs' assignment in previous steps are as follows:
echo "::set-output name=evaluation-2-outcome::$FAIL_OUTCOME"

Answer 1

The straight answer to your question is that you are misusing the $FAIL_OUTCOME value in if statements.
There you need to use:
if: steps.evaluation_1.outputs.evaluation-1-outcome == env.FAIL_OUTCOME || steps.evaluation_2.outputs.evaluation-2-outcome == env.FAIL_OUTCOME
...
if: steps.evaluation_1.outputs.evaluation-1-outcome != env.FAIL_OUTCOME && steps.evaluation_2.outputs.evaluation-2-outcome != env.FAIL_OUTCOME
or:
if: ${{ steps.evaluation_1.outputs.evaluation-1-outcome == env.FAIL_OUTCOME || steps.evaluation_2.outputs.evaluation-2-outcome == env.FAIL_OUTCOME }}
...
if: ${{ steps.evaluation_1.outputs.evaluation-1-outcome != env.FAIL_OUTCOME && steps.evaluation_2.outputs.evaluation-2-outcome != env.FAIL_OUTCOME }}
Env variables can be accessed as $FAIL_OUTCOME only in bash scopes - everywhere else, you need to explicitly use env. prefix.
However, I would recommend doing it properly and not fighting with env variables combined with set-output madness - which has a lot of gotchas and it's hard to debug and maintain.
Slack success and failures can be easily handled by checking the output of the whole job:
- name: send slack failure
if: failure()
- name: send slack success
if: success()
If you want to communicate about failure of some steps only:
notify_failure:
if: always() && !cancelled() && needs.check_failure_step.result != 'success'
needs: check_failure_step
And instead of using echo set-output:: just exit 1 to fail a certain job.
Combining that with needs and jobs output values and if: always() you can achieve anything you want.
It gives you also a huge advantage of seeing which job has failed straight on workflow run summary without looking to its "output" or "debug" logs.