So I've searched quite a bit for this and I'm pretty new to Shell,
I want to iterate over Resultset rows in SHELL script and for each row I want to execute some code using each column of the current row.
Lets assume the resultset look like this.
Query Priority Size
---------------------------------------------------
this is a sentence to execute high 124400
this is another example low 15000000
...
So how do I manage to iterate over this Resultset and storing each column into his own variable? Here is an exemple for the first line:
var1="this is a sentence to execute"
var2="high"
var3=124400
#repeat process for next line
Here is what you could do:
Extract your data into a delimited file (say, csv), if possible, avoid having headers. If you do have headers, make sure to exclude them while reading.
Read data from the file into an array or a set of variables
like so:
# using an array - works great if we have a variable number of columns
while IFS=, read -r -a row; do
# ${row[0]} => column1, ${row[1]} => column2 and so on
# process data
done < input.dat
# using variables - works great if we have a fixed set of variables
while IFS=, read -r column1 column2 column3; do
# process data
done < input.dat
See also: How do I split a string on a delimiter in Bash?
Related
I am inserting SQL query result into a csv file. At the end of file two rows are getting added as shown below. Can anyone please tell me how to delete these two rows (bank row and '168 rows selected').
Also, it will be great if these rows don't get inserted into csv file in the first place while inserting sql query result into csv file.
pvm:0a12dd82,pvm:0a12dd84,TechnicalErrorProcess,21-JUN-19 07.01.58.560000 AM,pi_halted,38930,1
pvm:0a12dd77,pvm:0a12dd79,TechnicalErrorProcess,20-JUN-19 12.36.27.384000 PM,pi_halted,1572846,1
pvm:0a12dd6t,pvm:0a12dd6v,TechnicalErrorProcess,20-JUN-19 12.05.22.145000 PM,pi_halted,38929,1
pvm:0a12dd4h,pvm:0a12dd4l,TechnicalErrorProcess,17-JUN-19 07.11.43.522000 AM,pi_halted,9973686,1
168 rows selected.
For MSSQL, before select query append the following,
set nocount on; select ...
I'm not sure if that will work for other databases.
Filter the output of the command above to exclude to two last lines.
I see two ways for doing it with a bash command:
head --lines=-2
Or
sed -e '/rows selected/d' -e '/^ *$/d'
(Indeed, this was a placeholder).
You can specify a negative number for the -n parameter using the head command:
-n, --lines=[-]K
print the first K lines instead of the first 10; with the leading '-', print all but the last K lines of each file
so:
head -n -2 input-file.txt > output-file.txt
I have a table with column called "names". Can i store each name into different variable?
Right now I am using a for loop and storing the results into single variable.
for j in `psql database_name -U admin -c "select names from table;"`
; do
done
When i do echo$j it prints below result
tom
Harry
steve
How do i store each values(names) in different variables?
Creating one variable per value is impractical. Think about it: How would you access these variables? What you want is an array.
You can define the array as follows:
mapfile -t names < <(psql database_name -U admin -c 'select names from table;')
This will add every line printed by psql as an entry of the array names. To access the entries use ${names[0]}, ${names[1]}, and so on. The size of the array is ${#names[#]}.
Since the question is called »Convert for loop results into array in bash« here's another way to build up the array. However, I wouldn't use a for ... in $(...) loop here as names with spaces will be split and symbols like * will expand.
names=()
for name in $(psql database_name -U admin -c 'select names from table;'); do
names+=("$name")
done
If you want the loop to run some additional commands use the first approach in this answer and loop over the array – this is safer:
mapfile ... # see above
for name in "${names[#]}"; do
# do something with $name
done
I need to make a program that takes as arguments a number of files that contain lines like this: num1:num2.
I need to store the left column of numbers in one array and the right column and then do some things to them. I need some help on the first part.
The number of files passed as arguments is variable. Also I don't know the name of the files neither how many lines they have. I just know that I will get at least 1 file with 1 line.
I am trying to make a loop for each argument file and then read each file line, break down each line with some string manipulation and then store the results in the 2 arrays. However I haven't succeeded. I know that I also have other kinds of mistakes but I can fix those.
When I try to run the program using:
sh <my_program_name>.sh <argument1_filename>
I just get no results on the terminal, blank screen like it is calculating something in an endless loop.
#!/bin/bash
length=0
b=1
c=1
d=0
args=$#
j=0
temp=0
temp2=0
temp3=0
temp4=0
for temp in "$#"
do
while read line
do
stringtmp=line
tmp=`expr index "$stringtmp" :`
let tmp=tmp-1
stringtmp2='expr substr $stringtmp $1 $tmp'
lengh=`expr index "$stringtmp" \n`
let tmp=tmp+2
let lengh=lengh-1
stringtmp3='expr substr $stringtmp $tmp $lengh'
array1[$length]=stringtmp2
array2[$length]=stringtmp3
let length=length+1
done
...
done
Your while loop is waiting for input from stdin. If you want to loop through contents of temp, you could use:
while read line; do
...
done<$temp
I have a folder where there are 50 excel sheets in CSV format. I have to populate a particular value say "XYZ" in the column I of all the sheets in that folder.
I am new to unix and have looked for a couple of pages Here and Here . Can anyone please provide me the sample script to begin with?
For example :
Let's say column C in this case:
A B C
ASFD 2535
BDFG 64486
DFGC 336846
I want to update column C to value "XYZ".
Thanks.
I would export those files into csv format
- with semikolon as field separator
- eventually by leaving out column descriptions (otherwise see comment below)
Then the following combination of SHELL and SED script could more or less do already the trick
#! /bin/sh
for i in *.csv
do
sed -i -e "s/$/;XZY/" $i
done
-i means to edit the file in place, here you could append the value to all lines
-e specifies the regular expresssion for substitution
You might want to use a similar script like this, to rename "XYZ" to "C" only in the 1st line if the csv files should contain also column descriptions.
I have a bash script that take advantage of a local toolbox to perform an operation
my question is fairly simple
I have multiple files that are the same quantities but different time steps i would like to first untar them all, and then use the toolbox to perform some manipulation but i am not sure if i am on the right track.
=============================================
The file is as follows
INPUTS
fname = a very large number or files with same name but numbering
e.g wnd20121.grb
wnd20122.grb
.......
wnd2012100.grb
COMMANDS
> cdo -f nc copy fname ofile(s)
(If this is the ofile(s)=output file how can i store it for sequent use ? Take the ofile (output file) from the command and use it / save it as input to the next, producing a new subsequent numbered output set of ofile(s)2)
>cdo merge ofile(s) ofile2
(then automatically take the ofile(s)2 and input them to the next command and so on, producing always an array of new output files with specific set name I set but different numbering for distinguishing them)
>cdo sellon ofile(s)2 ofile(s)3
------------------------------------
To make my question clearer, I would like to know the way in which I can instruct basically through a bash script the terminal to "grab" multiple files that are usually the same name but have a different numbering to make the separate their recorded time
e.g. file1 file2 ...file n
and then get multiple outputs , with every output corresponding to the number of the file it converted.
e.g. output1 output2 ...outputn
How can I set these parameters so the moment they are generated they are stored for subsequent use in the script, in later commands?
Your question isn't clear, but perhaps the following will help; it demonstrates how to use arrays as argument lists and how to parse command output into an array, line by line:
#!/usr/bin/env bash
# Create the array of input files using pathname expansion.
inFiles=(wnd*.grb)
# Pass the input-files array to another command and read its output
# - line by line - into a new array, `outFiles`.
# The example command here simply prepends 'out' to each element of the
# input-files array and outputs each (modified) element on its own line.
# Note: The assumption is that the filenames have no embedded newlines
# (which is usually true).
IFS=$'\n' read -r -d '' -a outFiles < \
<(printf "%s\n" "${inFiles[#]}" | sed s'/^/out-/')
# Note: If you use bash 4, you could use `readarray -t outFiles < <(...)` instead.
# Output the resulting array.
# This also demonstrates how to use an array as an argument list
# to pass to _any_ command.
printf "%s\n" "${outFiles[#]}"