bash—Better way to store variable between runs? - bash

Question

I've made a bash script which I run every hour with crontab, and I need to store one variable so that I can access it the next time I run it. The script changes the variable every time it runs, so I can't hardcode it in. Right now I am writing it to a txt file and then reading it back. Is there a better way to do it than this? And the way I am reading the txt file is something I found on here, I don't understand it, and it's kinda clunky. Is there not a built in command for this? Anyway, here's the applicable code, with some of the variables changed to make it easier to read.
while read x; do
var=$x
done < var.txt
# Do some stuff, change var to a new value
echo $var > var.txt
The variable is only a single integer, so the text file feels overkill.

Answer 1

There's no need to use var; x will be in scope for the current shell. Alternately,
read var < var.txt
# do stuff with var
echo $var > var.txt
I recommend using a simple text file to store the variable. However, there is the (highly questionable) option of a self-modifying script. FOR ENTERTAINMENT PURPOSES ONLY!
#!/bin/bash
read val < <( tail -n 1 "$0" )
(( val++ ))
echo "$val"
tmp=$(mktemp /tmp/XXXXXXX)
sed '$s/.*/'$val'/' "$0" > "$tmp"
mv "$tmp" "$0"
exit
0
The key is to have the next-to-last line be the exit command, so nothing after it will execute. The last line is the variable value you want to persist. When the script runs, it reads from its own last line. Before it exits, it uses sed to write a copy of itself toa temp file, with the last line modified with the current value of the persistent value. Then we overwrite the current script with the temp file (assuming we will have permission to do so).
But seriously? Don't do this.

Answer 2

I know this is an old question. But, I still decide to post my solution here in the hope that it might be helpful to others who come here in search of a way to serialize env vars between sessions.
The simple way is just write "var_name=var_value" into a file, say "./environ". And then "source ./envrion" in following sessions. For example:
echo "var1=$var1" > ./environ
A more comprehensive (and elegant?) way which persist all attributes of variables is to make use of "declare -p":
declare -p var1 var2 > ./environ
# NOTE: no '$' before var1, var2
Later on, after "source ./envrion" you can get var1 var2 with all attributes restored in addition to its value. This means it can handle arrays, integers etc.
One caveat for the "declare -p xx", though: if you wrap the "source ./environ" into a function, then all sourced variables are visible within the function only because "declare" by default declares variables as local ones. To circumvent this, you may either "source" out of any function (or in your "main" function) or modify the ./environ to add "-g" after declare (which makes corresponding variable global). For instance:
sed -i 's/^declare\( -g\)*/declare -g/' ./environ
# "\( -g\)?" ensure no duplication of "-g"

Answer 3

1- You can simplify your script, as you only have one variable
var=`cat var.txt`
# Do some stuff, change var to a new value
echo $var > var.txt
2- You can store your variable in the environment:
export var
# Do some stuff, change var to a new value
But you'll need to prompt it . script.ksh (dot at the beggining). But it shouldn't have 'exit' in it and i'm not sure this would work in cron...

Answer 4

Depending on your use case this might be overkill, but if you need to store and keep track of multiple variables (or from multiple scripts) then consider using sqlite which has a command line interface (sqlite3), and which is usually preinstalled ootb on linux/macos systems.
DB='storage.db'
KEY1='eurusd'
VAL1=1.19011
KEY2='gbpeur'
VAL2=1.16829
# create table if not present (ONLY NEEDS TO BE RUN ONCE)
QUERY_CREATE="CREATE TABLE IF NOT EXISTS records (id INTEGER PRIMARY KEY, name TEXT NOT NULL, value NUMERIC NOT NULL);"
sqlite3 "$DB" "$QUERY_CREATE"
# write a key-value pair to database (creates a new row each time)
QUERY_INSERT="INSERT INTO records(name, value) VALUES ('${KEY1}', '${VAL1}');"
sqlite3 "$DB" "$QUERY_INSERT"
# write a key-value pair to database (REPLACE previous value!)
# using 42 as a hard-coded row ID
QUERY_REPLACE="REPLACE INTO records(id, name, value) VALUES (42, '${KEY2}', '${VAL2}');"
sqlite3 "$DB" "$QUERY_REPLACE"
# read value from database
QUERY_SELECT1="SELECT value FROM records WHERE name='${KEY1}';"
QUERY_SELECT2="SELECT value FROM records WHERE name='${KEY2}';"
echo "***** $KEY1 *****"
# store db value in a variable
db_value1=$(sqlite3 "$DB" "$QUERY_SELECT1")
echo $db_value1
## OUTPUT: 1.19011
echo "***** $KEY2 *****"
db_value2=$(sqlite3 "$DB" "$QUERY_SELECT2")
echo $db_value2
## OUTPUT: 1.16829
NOTE: If you do not explicitly pass the row ID then a new row will be added on each script invocation. To always update into the same row use REPLACE INTO with an explicit ID (e.g. 42 as can be seen in the REPLACE INTO... statement). Run the script multiple times to see how the output differs for KEY1 and KEY2.
NOTE2: In this example the values are numeric, if you need to store strings then in CREATE TABLE instead of NUMERIC use TEXT.
And if you want an open-source GUI for visualising the database then DB Browser for SQLite is available for mac/linux/windows (there are dozens more).

Answer 5

To store multiple variables between runs, a solution I considered is to save them under the format my_var=my_value in a separated file.
Then, I include two function to set and retrieve the variables
In the file storing the variables and their values:
Let's call this file context.dat
# Here I store the variables and their values
my_var_x=1
my_var_y=boo
my_var_z=0
In the actual script:
Let's call the file multiple_run.sh
context=./context.dat
function update_variables(){
# update the variable context
source $context
}
function set_variable(){
# store variable
variable=$1 #variable to be set
value=$2 # value to give to the value
# modify the file storing the value
sed -i 's/'${variable}'.*/'${variable}'='${value}'/' $context
}
##################
# Test code
echo var_x
update_variables
echo var_x
# do something
set_variable var_x 2
echo $var_x
This is one approach among other. With such method, you need to create the storing file before and create each line for each variable. Besides, the context.dat is a priori accessible by any other script.

Answer 6

Just discovered this great simple project (a rewritten fork). A simple yet powerful key/value pair store for bash. Looks perfect. Behind the scenes each database is a directory, each key is a file, and the values are in the file.
https://github.com/imyller/kv-sh
Tiny key-value database
Configurable database directory (default: ~/.kv-sh)
Used by importing functions via $ . ./kv-sh
Full database dump/restore
Support for secondary read-only defaults database
. ./kv-sh # import kv-sh functions (use default database directory; see
configuration environment variables for available options)
kvset <key> <value> # assign value to key
kvget <key> # get value of key
kvdel <key> # delete key
kvexists <key> # check if key exists
kvkeys {-l|-d|-a} # list all keys (-l local only, -d default only, -a all (default))
kvlist {-a} # list all key/value pairs (-a all keys, including default)
kvdump {-a} # database dump (-a all keys, including default)
kvimport # database import (overwrite)
kvrestore # database restore (clear and restore)
kvclear # clear database
Defaults database
kv-sh supports secondary read-only defaults database. If enabled, keys-value pairs from default value database are returned if local value is not specified.
Enable defaults database by setting DB_DEFAULTS_DIR:
DB_DIR="/tmp/.kv" DB_DEFAULTS_DIR="/tmp/.kv-default" . ./kv-sh

Answer 7

I ended up doing the following. Would prefer the variables in one file, but this bloats the code slightly. How does this read thing work? You can store multiple variables in a seperate file, say variables.txt, and then have your main program in say main.sh. It might be better to write seperate scripts for loading and saving variables though.
For varibles.txt:
A=0
B=0
C=0
For main.sh:
#!/bin/bash
#reload variables
A=`cat ./variables.txt|grep "A="|cut -d"=" -f2`
B=`cat ./variables.txt|grep "B="|cut -d"=" -f2`
C=`cat ./variables.txt|grep "C="|cut -d"=" -f2`
#print variables
printf "$A\n"
printf "$B\n"
printf "$C\n"
#update variables
A=$((($A+1)))
B=$((($B+2)))
C=$((($C+3)))
#save variables to file
#for A
#remove entry for A
cat ./variables.txt|grep -v "A=">>./tmp.txt
#save entry for A
printf "A=$A\n">>./tmp.txt
#move tmp.txt to variables.txt
mv ./tmp.txt ./variables.txt
#for B
#remove entry for B
cat ./variables.txt|grep -v "B=">>./tmp.txt
#save entry for B
printf "B=$B\n">>./tmp.txt
#move tmp.txt to variables.txt
mv ./tmp.txt ./variables.txt
#for C
#remove entry for C
cat ./variables.txt|grep -v "C=">>./tmp.txt
#save entry for C
printf "C=$C\n">>./tmp.txt
#move tmp.txt to variables.txt
mv ./tmp.txt ./variables.txt