I want to change in several txt-files a variable. But this variable shouldn't be equal in every txt-file.
I tried this, but it doesn't work.
for file in ./*.txt ;do
for value in $(seq 1 5); do
sed -i 's/x_max=.*/x_max='$value'/ ' $file
done
done
So every x_max has got the value:5
This should do the trick. Replace each file only once, with a different value each time.
value=1
for file in *.txt; do
sed -i 's/x_max=.*/x_max='$value'/' $file
value=$((value + 1))
done
That should do it - only iterate once and raise the counter by 1 after each run:
counter=1
for file in ./*.txt ;do
sed -i 's/x_max=.*/x_max='$counter'/ ' $file
(( counter++ ))
done
Related
I have a list of files named "LL1.txt" to "LL1180.txt" but I am only interested in files from 1 to 50 to paste them in one file.
I tried using:
seq 1 50
for n in $(seq $1 $2); do
f="LL$n.txt"
if [ -e $f ]; then
paste "$f" > LL$1_$2.txt
fi
done
but it did not work.
for n in `seq $start $stop` ; do
if [ -e "$LL$n.txt" ] ; then
cat LL$n.txt >> output_file
fi
done
or if you enjoy harder way:
cat > output_file <<< `cat LL{1..50}.txt`
You need to give all the filenames as arguments to paste so it will combine them.
paste FILE{1..50}.txt > LL1_50.txt
Note that you can't use variables in the braced range. See Using a variable in brace expansion range fed to a for loop if you need workarounds.
im new in bash,
Im trying to edit line with sed with for loop from other file.
Please tell me what i doing wrong in my small code?
Do i missing another loop?
#!/bin/bash
# read and taking the line needed:
for j in `cat /tmp/check.txt`; do
# replacing the old value with and value:
sed -i "s+/tmp/old_name/+/${j}/+gi" file_destantion.txt$$
#giving numbers to the logs for checking
Num=j +1
# moving the changed file to .log number ( as for see that it is changed):
mv file_destantion.txt$$ file_destantion.txt$$.log$Num
#create ne source file to do the next value from /tmp/check:
cp -rp file_destantion.txt file_destantion.txt$$
done
On /tmp/check i have the info that i want to enter on each loop turn.
in /tmp/check:
/tmp/check70
/tmp/check70_1
/tmp/_check7007
In the end this is what i want it to be like:
.log1 > will contain /tmp/check70
.log2 > will contain /tmp/check70_1
.log3 will contain /tmp/check7007
I have found this solution worked for me.
#!/bin/bash
count=0
grep -v '^ *#' < /tmp/check | while IFS= read -r line ;do
cp -rp file_destantion.txt file_destantion.txt$$
sed -i "s+/tmp/old_name/+${line}/+gi" file_destantion.txt$$
(( count++ ))
mv file_destantion.txt$$ "file_destantion.txt$$.log${count}"
cp -rp file_destantion.txt file_destantion.txt$$
done
thank you very much #Cyrus for your guiding.
I have the current script which combines all the CSV files in a folder into a single CSV file and it works great. I need to add functionality to add the filename of the original csv's as a header row for each data block so I know which section is which.
Can someone assist as this is not by strong point and I am over my head
#!/bin/bash
OutFileName="./Data/all/all.csv" # Fix the output name
i=0 # Reset a counter
for filename in ./Data/all/*.csv; do
if [ "$filename" != "$OutFileName" ] ; # Avoid recursion
then
if [[ $i -eq 0 ]] ; then
head -1 $filename > $OutFileName # Copy header if it is the first file
fi
tail -n +2 $filename >> $OutFileName # Append from the 2nd line each file
i=$(( $i + 1 )) # Increase the counter
fi
done
I will be automating this and using and run shell script in apple automator.
Thank you got any help.
This is one of the files that are imported and output example
Example of current input file
Once combined I need the filename where the "headers are"
When you want to generate something like ...
Header1,Header2,Header3
file1.csv
a,b,c
x,y,z
file2.csv
1,2,3
9,9,9
file3.csv
...
... then you just have to insert an echo "$filename" >> "$OutFileName" in front of the tail command. Here is an updated version of your script with some minor improvements.
#!/bin/bash
out="./Data/all/all.csv"
i=0
rm -f "$out"
for file in ./Data/all/*.csv; do
(( i++ == 0)) && head -1 "$file"
echo "$file"
tail -n +2 "$file"
done > "$out"
There is no concept of "header line" other than the first line of the CSV file. What you can do is add a new column.
I've switched to Awk because it simplifies the script considerably. Your original would be literally a one-liner.
awk -F , 'NR==1 { OFS=FS; $(NF+1) = "Filename" }
FNR>1{ $(NF+1) = FILENAME }1' all/*.csv >all.csv
Not saving the output in the same directory as the inputs removes the pesky corner case handling.
I am trying to replace the duplicate UUIDs from multiple files in a directory. Even the same file can have duplicate UUIDs.
I am using Unix utilities to solve this.
Till now I have used grep, cut, sort and uniq to find all the duplicate UUIDs across the folder and store it in a file (say duplicate_uuids)
Then I tried sed to replace the UUIDs by looping through the file.
filename="$1"
re="*.java"
while read line; do
uuid=$(uuidgen)
sed -i'.original' -e "s/$line/$uuid/g" *.java
done < "$filename"
As you would expect, I ended up replacing all the duplicate UUIDs with new UUID but still, it is duplicated throughout the file!
Is there any sed trick that can work for me?
There are a bunch of ways this can likely be done. Taking a multi-command approach using a function might give you greater flexibility if you want to customize things later, for example:
#!/bin/bash
checkdupes() {
files="$*"
for f in $files; do
filename="$f"
printf "Searching File: %s\n" "${filename}"
while read -r line; do
arr=( $(grep -n "${line}" "${filename}" | awk 'BEGIN { FS = ":" } ; {print $1" "}') )
for i in "${arr[#]:1}"; do
sed -i '' ''"${i}"'s/'"${line}"'/'"$(uuidgen)"'/g' "${filename}"
printf "Replaced UUID [%s] at line %s, first found on line %s\n" "${line}" "${i}" "${arr[0]}"
done
done< <( sort "${filename}" | uniq -d )
done
}
checkdupes /path/to/*.java
So what this series of commands does is to first sort the duplicates (if any) in whatever file you choose. It takes those duplicates and uses grep and awk to create an array of line numbers which each duplicate is found. Looping through the array (while skipping the first value) will allow the duplicates to be replaced by a new UUID and then re-saving the file.
Using a duplicate list file:
If you want to use a file with a list of dupes to search other files and replace the UUID in each of them that match it's just a matter of changing two lines:
Replace:
for i in "${arr[#]:1}"; do
With:
for i in "${arr[#]}"; do
Replace:
done< <( sort "${filename}" | uniq -d )
With:
done< <( cat /path/to/dupes_list )
NOTE: If you don't want to overwrite the file, then remove sed -i '' at the beginning of the command.
This worked for me:
#!/bin/bash
duplicate_uuid=$1
# store file names in array
find . -name "*.java" > file_names
IFS=$'\n' read -d '' -r -a file_list < file_names
# store file duplicate uuids from file to array
IFS=$'\n' read -d '' -r -a dup_uuids < $duplicate_uuid
# loop through all files
for file in "${file_list[#]}"
do
echo "$file"
# Loop through all repeated uuids
for old_uuid in "${dup_uuids[#]}"
do
START=1
# Get the number of times uuid present in this file
END=$(grep -c $old_uuid $file)
if (( $END > 0 )) ; then
echo " Replacing $old_uuid"
fi
# Loop through them one by one and change the uuid
for (( c=$START; c<=$END; c++ ))
do
uuid=$(uuidgen)
echo " [$c of $END] with $uuid"
sed -i '.original' -e "1,/$old_uuid/s/$old_uuid/$uuid/" $file
done
done
rm $file.original
done
rm file_names
I have many fq.gz files in a directory and I want to loop through each filename, delete the last 8 characters and print this to a sample name text file
eg.
984674-TAATGAGC-GCTGTAGA--R1.fq.gz
984674-TAATGAGC-GCTGTAGA--R2.fq.gz
will become:
984674-TAATGAGC-GCTGTAGA--
984674-TAATGAGC-GCTGTAGA--
At the moment I have a bash script to put each filename into an array and then print the array to a sample.txt file. I have then tried to mv the filename to the new desired filename:
#!/bin/bash
declare -a FILELIST
for f in *.gz; do
#FILELIST[length_of_FILELIST + 1]=filename
FILELIST[${#FILELIST[#]}+1]=$(echo "$f");
done
printf '%s\n' "${FILELIST[#]:1:10}" > sample.txt
sample_info=sample.txt
sample_files=($(cut -f 1 "$sample_info"))
for file in "${sample_files[#]}"
do
mv "$file" "${file::(-8)}"
echo $file
done
But the script isn't removing any characters. Can you help?
to loop through each filename, delete the last 8 characters and print this to a sample name, would this work for you:
for i in *fq.gz
do
echo ${i:0:-8}
done
Using substring removal, here. Assuming you want exactly 8 characters out from the end:
for n in *.fq.gz
do
echo "${n%%??.fq.gz}"
done
For a test,
$ n="984674-TAATGAGC-GCTGTAGA--R1.fq.gz"
$ echo "${n%%??.fq.gz}"
984674-TAATGAGC-GCTGTAGA--
OR
$ echo "${n%%????????}"
984674-TAATGAGC-GCTGTAGA--