This question already has answers here:
How can I use a file in a command and redirect output to the same file without truncating it?
(14 answers)
Redirect output from sed 's/c/d/' myFile to myFile
(10 answers)
Using the same file for stdin and stdout with redirection
(3 answers)
How to redirect and replace the input file with the output (don't erase myfile when doing "cat myfile > myfile")
(3 answers)
Why piping to the same file doesn't work on some platforms?
(5 answers)
Closed 5 years ago.
There's a way to cat a file, filter it by something and then output a new file with the same name? I'm doing that and I'm getting an empty file, but if I create it with different file name is working. I don't want to create a new file.
Example:
File="My_test_file.txt"
cat ${File} | grep -v "test" > ${File}
in that way is not working, I have to create another file to make it work, as follow:
File="My_test_file.txt"
cat ${File} | grep -v "test" > ${File}.tmp
any idea?
There's a package called moreutils that contains the tool sponge for this exact purpose:
grep -v test foo.txt | sponge foo.txt
If installing tools is not an option, you can implement a naive version that first reads all data into memory, and then finally writes it out:
#!/bin/sh
sponge() (
var="$(cat; printf x)"
printf '%s' "${var%x}" > "$1"
)
grep -v test foo.txt | sponge foo.txt
What you are attempting to do with cat and grep -v can be easily done using sed -i '/pattern/d' and that allows to save changes inline as well:
sed -i.bak '/test/d' "$file"
Related
This question already has answers here:
How can I remove the extension of a filename in a shell script?
(15 answers)
Closed 11 months ago.
I have a filename with the format yyyymmdd.txt. How can I output only yyyymmdd without the .txt extension?
Example
20220414.txt (before output)
20220414 (after the output)
basename has an option to remove a suffix:
basename -s .txt 20220414.txt
gives:
20220414
Or, if your filename is stored in a variable, bash can help:
a=20220414.txt
echo ${a%.*}
gives:
20220414
You can user awk with flag -F to specify the separator . and then print the first part with $1
echo "20220414.txt" | awk -F "." ' {print $1}'
output
20220414
grep doesn't manipulate anything, it shows what you have in a file. So, you can't modify that file using grep, but you can modify what it shows, using the -o switch, as you can see here:
Prompt> echo "20220414.txt" | grep -o "[0-9]*"
20220414
The [0-9]* means "a list of integers, going from character '0' to character '9'.
This question already has answers here:
How to pass command output as multiple arguments to another command
(5 answers)
Closed 1 year ago.
I have a csv called test.csv that contains urls to be dowloaded in its first column. I want to download the urls using wget. How can I do this in shell?
I have used the command below but no success:
awk -F ',' '{print $1}' test.csv | wget -P download_dir
I don't think you can pipe the filenames into wget, but you can run the command for each item with the filename appended to the end
try this and review the list of commands it will run
awk -F ',' '{print $1}' test.csv | xargs -n1 echo wget -P download_dir
then remove echo and run it again, and it'll execute the commands instead of printing them
This question already has answers here:
How can I use a file in a command and redirect output to the same file without truncating it?
(14 answers)
Closed 3 years ago.
If I try to redirect output for a command into a file that is open for reading within the command, I get an empty file.
For example, suppose I have a file named tmp.txt:
ABC
123
Now if I do this:
$ grep --color=auto A tmp.txt > out.txt
$ cat out.txt
ABC
But if I do this:
$ grep --color=auto A tmp.txt > tmp.txt
$ cat out.txt
$
I get no output.
I'd like to be able to redirect to a file that I am reading within the same command.
Okay, so I have my answer and would like to share it with you all.
You simply have to use a pipe with tee.
$ grep --color=auto A tmp.txt | tee tmp.txt
ABC
$cat tmp.txt
ABC
Perhaps someone who understands pipes well can explain why.
This question already has answers here:
Why does reading and writing to the same file in a pipeline produce unreliable results?
(2 answers)
Closed 3 years ago.
echo "hello" | tee test.txt
cat test.txt
sudo sed -e "s|abc|def|g" test.txt | tee test.txt
cat test.txt
Output:
The output of 2nd command and last command are different, where as the command is same.
Question:
The following line in above script gives an output, but why it is not redirected to output file?
sudo sed -e "s|abc|def|g" test.txt
sudo sed -e "s|abc|def|g" test.txt | tee test.txt
Reading from and writing to test.txt in the same command line is error-prone. sed is trying to read from the file at the same time that tee wants to truncate it and write to it.
You can use sed -i to modify a file in place. There's no need for tee. (There's also no need for sudo. You made the file, no reason to ask for root access to read it.)
sed -e "s|abc|def|g" -i test.txt
You shouldn't use the same file for both input and output.
tee test.txt is emptying the output file when it starts up. If this happens before sed reads the file, sed will see an empty file. Since you're running sed through sudo, it's likely to take longer to start up, so this is very likely.
This question already has answers here:
Remove empty lines in a text file via grep
(11 answers)
Closed 4 years ago.
Exists way to remove empty lines with cat myfile | grep -w #something ?
I looking for simple way for remove empty lines from my output like in the way the presented above.
This really belongs on the codegolfing stackexchange because it's not related to how anyone would ever write a script. However, you can do it like this:
cat myfile | grep -w '.*..*'
It's equivalent to the more canonical grep ., but adds explicit .*s on either side so that it will always match the complete line, thereby satisfying the word boundary conditions imposed by -w
You can pipe your output to awk to easily remove empty lines
cat myfile | grep -w #something | awk NF
EDIT: so... you just want cat myfile | awk NF?
if you have to use grep, you can do grep myfile -v '^[[:blank:]]*$'