I am trying to create a string eg (file1.txt,file2.txt,file3.txt).
All these 3 files names are within a file.
ls file*.txt > lstfiles.txt
while read filename; do
filename+=$line","
done <lstfiles.txt
This returns me with output:
file1.txt,file2.txt,file3.txt,
How can I find the last iteration of the loop so I dont add another comma at the end.
Required output:
file1.txt,file2.txt,file3.txt
For your use case I would rather get rid of the while loop and combine sedand tr commands like so:
sed -e '$ ! s/$/,/g' lstfiles.txt | tr -d '\n'
Where sed command replace each line endings execept the last one with a comma and tr command remove the linebreaks.
Probably avoid using ls in scripts though.
printf '%s,' file*.txt |
sed 's/,$/\n/'
assuming your sed recognizes \n to be a newline, and copes with input which doesn't have a final newline.
Related
How do I modify my bash file to achieve the expected result shown below ?
#!/bin/bash
filename=$1
var="$(<$filename)" | tr -d '\n'
sed -i 's/;/,/g' $var
Convert this input file
a,b;c^d"e}
f;g,h!;i8j-
To this output file
a,b,c,d,e,f,g,h,i,j
How to convert separators using regex in bash
You would, well, literally, do exactly that - convert any of the separators using regex. This consists of steps:
most importantly, figure out the exact definition of what consists of a "separator"
writing a regex for it
writing an algorithm for it
running and testing the code
For example, assuming a separator is a sequence of of any of \n,;^"}!8- characters, you could do:
sed -zi 's/[,;^"}!8-]\+/,/g; s/,$/\n/' input_file
Or similar with first tr '\n' , for example when -z is not available with your sed, and then pass the result of tr to sed. The second regex adds a trailing newline on the output instead of a trailing ,.
Additionally, in your code:
var is unset on sed line. Parts of | pipeline are running in a subshell.
var=$(<$filename) contains the contents of the file, whereas sed wants a filename as argument, not file contents.
var=.... | ... is pipeing the result of assignment to tr. The output of assignment is empty, so that line produces nothing, and its output is unused.
Remember to check bash scripts with shellcheck.
For a somewhat portable solution, maybe try
tr -cs A-Za-z , <input_file | sed '$s/,$/\n/' >output_file
The use of \n to force a final newline is still not entirely reliable; there are some sed versions which interpret the sequence as a literal n.
You'd move output_file back on top of input_file after this command if you want to replace the original.
I have lines and want to do sed operation, on string which comes after it has read '|'character three times. How can I do this in Shell Script?
Input: aaaa|bbbbb|ccccc|hello
Desired Ouput: aaaa|bbbbb|ccccc|hel
This is be done on hello which is after three '|'
-> sed 's/({.3}).*/\1/g'
You don't specify what you want to do with the last field to transform "hello" into "hel". Here's one way:
sed -r 's/^(([^|]+\|){3})(...).*/\1\3/' file
([^|]+\|) denotes a pipe delimited field (with the pipe)
(([^|]+\|){3}) denotes three such fields
requires sed's -r option
on OSX or BSD-ish implementations of sed, use -E instead)
I capture the next three characters with (...)
then replace all with the first and third set of capturing parentheses
Use the cut command instead of sed:
$ echo "aaaa|bbbbb|ccccc|hello" | cut -d '|' -f 4
hello
We are receiving a file that is delimited into rows with the \(newline) and columns with the \(tab) character.
When there is a manual newline present in one of the "fields" of the file, it comes in as a special newline with two backslashes (\\newline).
To remove the special tabs \(tab), we are using this sed command, which works correctly:
sed "s/$(printf '\\\\\t')/ /g"
The corresponding command for newlines, however does not:
sed "s/$(printf '\\\\\n')/ /g"
It does not remove the \n, only the backslash before it. Is there special handling that needs to be done to remove \(newline)?
Clarification: normal newlines are formatted like this:
\(newline)
Wheras the special characters that need removal are
\\(newline)
Here you go:
echo -e 'hello\\\nthere' | perl -ne 's/\\\n/ /; print'
It would be difficult (but probably possible) to do this in sed, because sed processes input line by line, and your data is broken into multiple lines. This perl one-liner processes the input line by line, and since it treats the newline character as part of the line, it can perform a substitution with space, which I think has the effect that you want.
Or if you prefer awk:
echo -e 'hello\\\nthere' | awk '{ if (gsub(/\\$/, " ")) printf; else print }'
At first I suspected your "special newline" character is just the string \\n like in the output of this command:
echo 'hello\\nthere'
You can replace the string \\n with a space like this:
echo 'hello\\nthere' | sed -e 's/\\\\n/ /g'
You can use tr (translate) command as well to do this, like
tr '\n' ' ' < inputfile.txt
EdIT: In that case use it like
tr '\\\n' ' ' < inputfile.txt
I need to replace 2 things from a csv file using Unix shell scripting ; one is Y with 'Y and the newline character by ',' value..
my csv:(values will be one below another vertically and all values start with Y)
YC1234
YC5678
expected output is NEEDED IN A LINE horizontally like :
'YC1234','YC5678'
kindly help as i am new to shell scripting...
i tried sed by its difficult in removing newline
cat master_upd.csv | sed -e 's//'\'','\''/g' | sed 's/\n/ /'
tr '\n' ',' < master.csv
The first command 'cat' is probably unnecessary.
Here is a thread on replacing newlines with something else using sed: How can I replace a newline (\n) using sed?
sed -e 's/^.*$/'\''&'\''/' master_upd.csv | sed ':a;N;$!ba;s/\n/,/g'
The first sed puts in the single quotes, the second one replaces newlines with commas. Notice that I removed the cat and just told sed to open your file directly.
awk -v q="'" 'NR>1{printf ","}{printf "%s",q$0q}END{print ""}' file
I work on shell script and I want to remove all space characters in a string.
On another question I saw a sed command for replacing and tried to use it with sending null character in it:
echo \0 | sed "s/ /${text}/"
But it did not work.
Any other way to do this?
This deletes all space characters in the input:
echo some text with spaces | tr -d ' '
Another way using sed:
echo some text with spaces | sed -e 's/ //g'
But... In your example there are no spaces, and it looks like you want to replace spaces with the content of the variable $text... So not 100% sure this is what you're looking for. So if not, then please clarify.
This was my use case:
Multiple key values that should go on a single string. I'd like to split the string to make them easier to read.
SPLIT=" \
KEY_1=$VALUE_1, \
KEY_2=$VALUE_2, \
KEY_3=$VALUE_3 \
"
JOINED="$(echo "$SPLIT" | sed "s/ //g")" # REMOVE SPACES
echo $JOINED