I've been trying to run the following function in a .sh file.
function run_test()
{
local executable=$1
BINNAME=$(basename ${executable})
local LOGDIR=$2
# Get the list of timestamps from the log file names in log directory 'LOGDIR'
LIST_OF_TS=`ls $LOGDIR/*.gz | awk -F "/" '{print $NF}' | awk -F'_' '{print $2}' | awk -F'.'
echo "Adding run of $executable to $RUNDIR/$BINNAME.dat"
for i in $LIST_OF_TS; do
gunzip -c $LOGDIR/*$i*.gz | sort -n
done | $executable "$2" > $RUNDIR/$BINNAME.dat
echo "Finished $executable"
}
This gives the error
for i in $LIST_OF_TS;
do
gunzip -c $LOGDIR/*$i*.gz | sort -n;
done
11726 Segmentation fault (core dumped) | $executable "$2" > $RUNDIR/$BINNAME.dat
When I remove the "$2", this works fine. My goal is to pass in $2 (or $LOGDIR) as an argument to $executable. I have no idea why this segfaults. Any advice?
Related
I have a simple script that checks the process id of some process using ps. When I run it directly on command line, it works fine but does not when I run it in a script. What am I doing wrong?
This works fine:
ps auwx | grep elasticsearch | grep -v grep | grep user | awk '{print $2}' | tail -1
In script, it does not:
#!/bin/bash
#Setting ES Heap to 50GB
ES_HEAP_SIZE="50g"
#Finding dump file to be deleted
FILE_ID=$(ps auwx | grep elasticsearch | grep -v grep | grep user | awk '{print $2}' | tail -1)
FILE_NAME="java_pid$FILE_ID.hprof"
echo "Elasticsearch pid: $FILE_ID"
echo "Dump file name if it exists: $FILE_NAME. Checking now."
if [ -s $FILE_NAME ]
then
rm $FILE_NAME
kill -9 $FILE_ID
#Starting elasticsearch daemon
/data/elasticsearch-1.4.4/bin/elasticsearch -d
else
echo "All good. Dump file $FILE_NAME does not exist."
fi
Personal pet peeve: why do you have six executions in your pipeline when two will handle everything you need?
ps auwx | awk '/elasticsearch/ && /user/ { x=$2 } END{ print x; }'
As an aside, you wanted PID? Because it looks like you're reading PPID.
Hope that helps.
I have a shell script which captures the Process ID, CPU and Memory of the JVM every nth second and writes the output to a file. Below is my code:
JVM="aaa001_bcdefx01"
systime=$(date +"%m-%d-%y-%T")
for i in {1..10}
do
PID=`ps -auxwww | grep "jdk" | grep $JVM | grep -v grep | cut -c -30 | awk '{print $2}'`
MEM=`ps -auxwww | grep "jdk" | grep $JVM | grep -v grep | cut -c -30 | awk '{print $4 }'`
CPU=`ps -auxwww | grep "jdk" | grep $JVM | grep -v grep | cut -c -30 | awk '{print $3 }'`
printf '%-5s %-20s %-20s %-20s %-20s \n' "$systime $JVM $PID $CPU $MEM " >> $LOGFILE
sleep 5
done
This run perfectly fine when i have only one JVM in that server. How can i execute the same script in parallel and fetch the details if i have multiple JVM for a server.
I looked for some solutions and found & to be used in the script but couldn't understand how this can be implemented to my above script. Let's say I have 5 JVMs. How can i run the script and fetch the stats in parallel for all the below JVMs in parallel. Kindly guide. Any help would be appreciated.
JVM="aaa001_bcdefx01"
JVM="aaa002_bcdefx01"
JVM="aaa003_bcdefx01"
JVM="aaa004_bcdefx01"
JVM="aaa005_bcdefx01"
GNU Parallel is made for this kind of stuff
doit() {
JVM="$1"
systime=$(date +"%m-%d-%y-%T")
for i in {1..10}
do
PID=`ps -auxwww | grep "jdk" | grep $JVM | grep -v grep | cut -c -30 | awk '{print $2}'`
MEM=`ps -auxwww | grep "jdk" | grep $JVM | grep -v grep | cut -c -30 | awk '{print $4 }'`
CPU=`ps -auxwww | grep "jdk" | grep $JVM | grep -v grep | cut -c -30 | awk '{print $3 }'`
printf '%-5s %-20s %-20s %-20s %-20s \n' "$systime $JVM $PID $CPU $MEM "
sleep 5
done
}
export -f doit
parallel -j0 --linebuffer --tag doit ::: aaa00{1..5}_bcdefx01 >> $LOGFILE
The function is basically your code. The change is that it takes the JVM as argument and it prints to stdout (standard output). GNU Parallel calls the function with the arguments aaa00N_bcdefx01 where N = 1..5, and saves the output to $LOGFILE. It uses --linebuffer to pass the output as soon as there is a full line, and thus guarantees that you will not get half-a-line from one process mixed with a line from another process. --tag prepends the line with the JVM.
How about using subshell?
Each JVM shell script should go inside '(' and ')'. Put '&' at the end so that it executes in the background.
An example is given here.
#!/bin/bash
echo > testfile.txt
echo "execute subshell 1"
(
#JVM 1 should go here
sleep 10
echo "subshell 1" >> testfile
)&
echo "execute subshell 2"
(
#JVM 2 should go here
sleep 10
echo "subshell 2" >> testfile
)&
echo "execute subshell 3"
(
#JVM 3 should go here
sleep 10
echo "subhsell 3" >> testfile
)&
Here each subshell writes data to testfile.txt after waiting for 10 seconds.
I have created a little shellscript that is capable of receiving a list of values such as "MY_VAR_NAME=var_value MY_VAR_NAME2=value2 ...", separated by spaces only. There should be also the possibility to use values such as MY_VAR_NAME='' or MY_VAR_NAME= (nothing).
These values are then used to change the value inside a environment variables file, for example, MY_VAR_NAME=var_value would make the script change the MY_VAR_NAME value inside the .env file to var_value, without changing anything else about the file.
The env file has the following configuration:
NODE_ENV=development
APP_PATH=/media
BASE_URL=http://localhost:3000
ASSETS_PATH=http://localhost:3000
USE_CDN=false
APP_PORT=3000
WEBPACK_PORT=8080
IS_CONNECTED_TO_BACKEND=false
SHOULD_BUILD=false
USE_REDUX_TOOL=false
USE_LOG_OUTPUT_AS_JSON=false
ACCESS_KEY_ID=
SECRET_ACCESS_KEY=
BUCKET_NAME=
BASE_PATH=
MIX_PANEL_KEY=
RDSTATION_KEY=
RESOURCE_KEY=
SHOULD_ENABLE_INTERCOM=false
SHOULD_ENABLE_GTM=false
SHOULD_ENABLE_UTA=false
SHOULD_ENABLE_WOOTRIC=false
I have debugged my script, and found out that this is the point where sometimes it has a problem
cat .envtemp | awk -v var_value="$VAR_VALUE" \
-v var_name="$VAR_NAME" \
-F '=' '$0 !~ var_name {print $0} $0 ~ var_name {print $1"="var_value}' | tee .envtemp
This piece of code sometimes outputs to .envtemp the proper result, while sometimes it just outputs nothing, making .envtemp empty
The complete code i am using is the following:
function change_value(){
VAR_NAME=$1
VAR_VALUE=$2
cat .envtemp | awk -v var_value="$VAR_VALUE" \
-v var_name="$VAR_NAME" \
-F '=' '$0 !~ var_name {print $0} $0 ~ var_name {print $1"="var_value}' | tee .envtemp
ls -l -a .env*
}
function manage_env(){
for VAR in $#
do
var_name=`echo $VAR | awk -F '=' '{print $1}'`
var_value=`echo $VAR | awk -F '=' '{print $2}'`
change_value $var_name $var_value
done
}
function main(){
manage_env $#
cat .envtemp > .env
exit 0
}
main $#
Here is an example script for recreating the error. It does not happen every time, and when it happens, it is not always with the same input.
#!/bin/bash
ENV_MANAGER_INPUT="NODE_ENV=production BASE_URL=http://qa.arquivei.com.br ASSETS_PATH=https://d4m6agb781hapn.cloudfront.net USE_CDN=true WEBPACK_PORT= IS_CONNECTED_TO_BACKEND=true ACCESS_KEY_ID= SECRET_ACCESS_KEY= BUCKET_NAME=frontend-assets-dev BASE_PATH=qa"
cp .env.dist .env
#Removes comment lines. The script needs a .envtemp file.
cat .env.dist | grep -v '#' | grep -v '^$' > .envtemp
./jenkins_env_manager.sh ${ENV_MANAGER_INPUT}
Have you tried use two files:
mv .envtemp .envtemp.tmp
cat .envtemp.tmp | awk ... | tee .envtemp
I have a shell script and I need help to make it efficient. I am using temp files to store and read the data, but I need to read the data in memory.
It collects metrics from the Postgres database using a command and fetches the metrics. My current script fetches the metrics to a temp file, then reads from it.
I want to stop using temp files and use memory instead.
The script works, I just need help to automate more and get rid of reading data from temp files.
List item
INPUT=`mktemp`
#/usr/pgsql-9.5/bin/pgbench -c1 -j1 -t 1000 -S man > $INPUT
TESTTIME=15 #seconds
echo "Waiting $TESTTIME seconds..."
/usr/pgsql-9.5/bin/pgbench -c1 -j1 -T $TESTTIME -r man > $INPUT
OLDIFS=$IFS
IFS=" "
[ ! -f $INPUT ] && { echo "$INPUT file not found"; exit 99; }
tps=`cat $INPUT |awk '/^tps/ {print $3}' |awk -F'.' '{print $1}' |head -n1`
update_l=`cat $INPUT |awk '/UPDATE/ {print $1}' |tail -n1`
select_l=`cat $INPUT |awk '/SELECT/ {print $1}' |tail -n1`
insert_l=`cat $INPUT |awk '/INSERT/ {print $1}' |tail -n1`
echo ${PLOTTER_PREFIX}.tps $tps kv
echo ${PLOTTER_PREFIX}.update_latency $update_l kv
echo ${PLOTTER_PREFIX}.select_latency $select_l kv
echo ${PLOTTER_PREFIX}.insert_latency $insert_l kv
#{ while read line; do
# # statsite_buildData ${PLOTTER_PREFIX}.latency average ${latency average} kv
# echo ${PLOTTER_PREFIX}.${line} kv
# done } < $INPUT
statsite_sendData
#echo $Test
IFS=$OLDIFS
rm -f $INPUT
You can capture the output of the command to a variable, like so:
output=$(/usr/pgsql-9.5/bin/pgbench -c1 -j1 -T $TESTTIME -r man)
Then just use echo instead of cat and substitute $INPUT with the variable name.
tps=`echo "$output" | awk '/^tps/ {print $3}' | awk -F'.' '{print $1}' |head -n1`
update_l=`echo "$output" | awk '/UPDATE/ {print $1}' | tail -n1`
...
I would also suggest using $() instead of surrounding commands with backticks. So the above would become:
tps=$(echo "$output" | awk '/^tps/ {print $3}' | awk -F'.' '{print $1}' |head -n1)
update_l=$(echo "$output" | awk '/UPDATE/ {print $1}' | tail -n1)
...
My bash script is:
output=$(curl -s http://www.espncricinfo.com/england-v-south-africa-2012/engine/current/match/534225.html | sed -nr 's/.*<title>(.*?)<\/title>.*/\1/p')
score=echo"$output" | awk '{print $1}'
echo $score
The above script prints just a newline in my console whereas my required output is
$ curl -s http://www.espncricinfo.com/england-v-south-africa-2012/engine/current/match/534225.html | sed -nr 's/.*<title>(.*
?)<\/title>.*/\1/p' | awk '{print $1}'
SA
So, why am I not getting the output from my bash script whereas it works fine in terminal am I using echo"$output" in the wrong way.
#!/bin/bash
output=$(curl -s http://www.espncricinfo.com/england-v-south-africa-2012/engine/current/match/534225.html | sed -nr 's/.*<title>(.*?)<\/title>.*/\1/p')
score=$( echo "$output" | awk '{ print $1 }' )
echo "$score"
Score variable was probably empty, since your syntax was wrong.