Update variable in bash function and use it in prompt (PS1) - bash

Question

What I want to do is to call a function in PS1 to update a variable inside the function. Then I like to use that variable to add another line to PS1. Like bellow:
my_func(){
var="_RED_"
echo "hello in red"
}
PS1="\[\033]0;\w\007\]"
PS1+='$(my_func)'
if [ $var = "_RED_" ]; then # here I want to use that var
PS1+="\[$(tput setaf 124)\] red"
fi
The reason for doing this is to bring non-printable characters \[ and \] out of the function to prevent from overlapping long lines that is caused by \[ \]

Answer 1

You can absolutely update global variables inside a shell function -- all assignments inside functions modify global variables unless local or declare variables were used to create a new scope.
The problem here, however, is that you aren't running your function in the same shell process as the code that later tries to read var (though whether it is in fact "later" or not is a separate issue)! When you use command substitution -- the $() in $(my_func) -- you're creating a new forked-off subprocess to run that function. When that subprocess exits, all changes to variable values it's made are lost with it.
However, you can work around that by not using command substitution at all. Consider the below, which uses the PROMPT_COMMAND hook to assign PS1:
# code to run before each time a prompt is printed
PROMPT_COMMAND='build_ps1_func'
# rewrite your function to write to a named variable, not stdout
my_func(){
local outvar=$1; shift # named variable to write stdout to
var="_RED_" # hardcoded global to update, per question
printf -v "$outvar" '%s' "hello in red"
}
build_ps1_func() {
local your_str # define your_str as a local variable
my_func your_str # call my_func, telling it to write output to your_str
PS1="\[\033]0;\w\007\]"
PS1+="$your_str"
if [ $var = "_RED_" ]; then # using that variable here
PS1+="\[$(tput setaf 124)\] red"
fi
}

Answer 2

What I want to do is to call a function in PS1 to update a variable inside the function. Then I like to use that variable to add another line to PS1.
From your sample code, I suppose you mean that you want to perform a command expansion involving a shell function in the process of assigning a value to PS1. [update:] Since you've tagged the question [bash], we'll presume that you are specifically interested in the behavior of GNU Bash, which differs from that of a fully-conforming POSIX shell in this area. It is important in that case to recognize that that the command expansion will be performed once, at the time the value of PS1 is set / modified, not each time PS1 is displayed. Your wording and specific syntax make me suspect that you have a different expectation.
Consider this part of your code:
PS1="\[\033]0;\w\007\]"
PS1+='$(my_func)'
Because it appears in single quotes, the $(my_func) is not subject to command expansion or any other expansion at the time that it is appended to your prompt string. Although the single quotes are removed before the value is appended to PS1, that does not mean it will be subject to expansion later. Unlike a fully-conforming POSIX shell, however, Bash will perform command substitution on the prompt string before printing it.
Now, because the function body is a curly-braced compound command, if it is executed then it will indeed set the value of var in the current shell, and that effect will be visible after the function returns. However,
(update, per #CharlesDuffy:) the command in a command substitution is run in a subshell. Although a variable update in that subshell would persist past the function return, its scope is limited to the subshell, which exits almost immediately thereafter.
Again, nothing in the code you presented results in your function ever being called, and
even if it were called, and it did set var in the current shell, you test for a value of $var different from the one your function would set.
In contrast, consider this other fragment of your code:
PS1+="\[$(tput setaf 124)\] red"
Because the whole string is double-quoted in this case, the contents are unequivocally subject to command expansion. If this were executed, the $(tput setaf 124) would be replaced with the output from running tput setaf 124. That would happen at the time that PS1's value is modified, not every time its value is displayed.
Although you can generate a prompt that contains ANSI escape sequences, you cannot do it quite the way you're trying to do. Inasmuch as your specific needs are unclear, I hesitate to suggest a particular alternative.