I am currently trying to create a bash script that will change a text document of one line into multiple lines.
Example:
TextFile: Header~someHeaderInfo|Object~someObjectInfo|SubObject~someSubObjectInfo|Object~someObjectInfo|SubObject~someSubObjectInfo|...|Tail~someInfo
Again, this above is only a single line.
This should be called through a bash script and be converted into:
Header~someHeaderInfo
Object~someObjectInfo|SubObject~someSubObjectInfo
Object~someObjectInfo|SubObject~someSubObjectInfo
...
Tail~someInfo
In the real use case, each Object has upwards of 20 subObjects, each of which may have more subObjects themselves.
How can I go about this separation?
if textfile contains:
Header~someHeaderInfo|Object~someObjectInfo|SubObject~someSubObjectInfo|Object~someObjectInfo|SubObject~someSubObjectInfo|...|Tail~someInfo
The following bash command:
sed "s/|/\n/g" textfile
Will produce the following output:
Header~someHeaderInfo
Object~someObjectInfo
SubObject~someSubObjectInfo
Object~someObjectInfo
SubObject~someSubObjectInfo
...
Tail~someInfo
But the OP wants the SubObject on the same line (see comments), so I suggest:
sed "s/|\([^S]\)/\n\1/g" textfile
That will produce the following output:
Header~someHeaderInfo
Object~someObjectInfo|SubObject~someSubObjectInfo
Object~someObjectInfo|SubObject~someSubObjectInfo
...
Tail~someInfo
Related
Although they do give the same results, I wonder if there is some difference between them and which is the most appropriate way to sort something contained in a file.
Another thing which intrigues me is the use of delimiters, I noticed that the sort filter only works if you separate the strings with a new line, are there any ways to do this without having to write the new strings in a separate line
The sort(1) command reads lines of text, analyzes and sorts them, and writes out the result. The command is intended to read lines, and lines in unix/linux are terminated by a new line.
The command takes its first non-option argument as the file to read; if there is no specification it reads standard input. So:
sort file_name
is a command line with such argument. The other two examples, "... | sort" and "sort < ..." do not specify the file to read directly to sort(1), but use its standard input. The effect, for what sort(1) is concerned, is the same.
ways to do this without having to write the new strings in a separate line
Ultimately no. But if you want you can feed sort using another filter (a program), which reads the file non-linefeed-separated and creates lines to pass to sort. If such program exists and is named "myparse", you can do:
myparse non-linefeed-separated-file | sort
The solution using cat involves creating a second process unnecessarily. This could be a performance issue if you perform many of such operation in a loop.
When doing input redirection to your file, the shell is setting up the association of file with std input. If the file would not exist, the shell complains about the file being missing.
When passing the file name as explicit argument, the sort process has to care about opening the file and to report an error if there is an accessability problem with it.
I have a task where I need to parse through files and extract information. I can do this easy using bash but I have to get it done through unix commands only.
For example, I have a file similar to the following:
Set<tab>one<tab>two<tab>three
Set<tab>four<tab>five<tab>six
ENDSET
Set<tab>four<tab>two<tab>nine
ENDSET
Set<tab>one<tab>one<tab>one
Set<tab>two<tab>two<tab>two
ENDSET
...
So on and so forth. I want to be able to extract a certain number of sets, say the first 10. Also, I want to be able to extract info from the columns.
Once again, this is a trivial thing to do using bash scripting, but I am unsure of how to do this with unix commands only. I can combine the commands together in a shell script but, once again, only unix commands.
Without an output example, it's hard to know your goal, but anyway, one UNIX command you can use is AWK.
Examples:
Extract 2 sets from your data sample (without include "ENDSET" nor blank lines):
$ awk '/ENDSET/{ if(++count==2) exit(0);next; }NF{print}' file.txt
Set one two three
Set four five six
Set four two nine
Extract 3 sets and print 2nd column only (Note 1st column is always "Set"):
$ awk '/ENDSET/{ if(++count==3) exit(0);next; }$2{print $2}' file.txt
two
five
two
one
two
And so on... (more info: $ man awk)
Basically I need to execute a curl command multiple times and redirect the output to a .csv file, each time the command is executed a term that is used in two separate places in the command is changed. I do have a list of these terms (arguments?) contained in a separate text file. Each time the command runs for a different term, the output needs to be appended to the file.
The command is basically:
curl "http://someURL/standardconditions+AND+(TERM_exact+OR+TERM_related)" > testfile.csv
So each time the command is run, TERM changes in both places (TERM_exact and TERM_related). As I mentioned, I have a text file that has a list of all 60 or so terms, what I want is the script to execute the command using the first term on the list, write the output to the specified .csv file and then repeat with the second term on the list, append that to the file and so on and so forth until it's been run for every single term.
I imagine there is a simple way to do this, I'm just not sure how.
Here's one way to do it,
This assumes that listFile.csv is your list of 60 items, and that each line is a comma-separated pair of values (no commas allowed in values!)
while IFS=, read exact related ; do
curl "http://someURL/standardconditions+AND+(TERM_${exact}+OR+TERM_${related})" >> testfile.csv
done < listFile.csv
It's not clear if you wanted one output file, or multiple.
You could replace the >> testfile.csv with >>testfile.${exact}_${related}.csv
to have separate files.
IHTH
You can set a variable to store TERM, and use concat function to get a string like
"http://someURL/standardconditions+AND+(TERM_exact+OR+TERM_related)", and run a python(may be other language)script including loop structure to handle 60 terms.
Before to write, of course I read many other similar cases. Example I used #!/bin/bash instead of #!/bin/sh
I have a very simple script that reads lines from a template file and wants to replace some keywords with real data. Example the string <NAME> will be replaced with a real name. In the example I want to replace it with the word Giuseppe. I tried 2 solutions but they don't work.
#!/bin/bash
#read the template and change variable information
while read LINE
do
sed 'LINE/<NAME>/Giuseppe' #error: sed: -e expression #1, char 2: extra characters after command
${LINE/<NAME>/Giuseppe} #error: WORD(*) command not found
done < template_mail.txt
(*) WORD is the first word found in the line
I am sorry if the question is too basic, but I cannot see the error and the error message is not helping.
EDIT1:
The input file should not be changed, i want to use it for every mail. Every time i read it, i will change with a different name according to the receiver.
EDIT2:
Thanks your answers i am closer to the solution. My example was a simplified case, but i want to change also other data. I want to do multiple substitutions to the same string, but BASH allows me only to make one substitution. In all programming languages i used, i was able to substitute from a string, but BASH makes this very difficult for me. The following lines don't work:
CUSTOM_MAIL=$(sed 's/<NAME>/Giuseppe/' template_mail.txt) # from file it's ok
CUSTOM_MAIL=$(sed 's/<VALUE>/30/' CUSTOM_MAIL) # from variable doesn't work
I want to modify CUSTOM_MAIL a few times in order to include a few real informations.
CUSTOM_MAIL=$(sed 's/<VALUE1>/value1/' template_mail.txt)
${CUSTOM_MAIL/'<VALUE2>'/'value2'}
${CUSTOM_MAIL/'<VALUE3>'/'value3'}
${CUSTOM_MAIL/'<VALUE4>'/'value4'}
What's the way?
No need to do the loop manually. sed command itself runs the expression on each line of provided file:
sed 's/<NAME>/Giuseppe/' template_mail.txt > output_file.txt
You might need g modifier if there are more appearances of the <NAME> string on one line: s/<NAME>/Giuseppe/g
I have a bash script that runs and outputs to a text file however the colour codes it uses are also included what i'd like to know is how to remove them from the file, ie
^[[38;1;32mHello^[[39m
^[[38;1;31mUser^[[39m
so I just want to be left with Hello and User, so something like sed -r "special characters" from file A save to file B
sed 's/\^\[\[[^m]*m//g'
remove (all) part of line starting with ^[[ until first m
Some like this:
awk '{sub(/\^\[\[38;1;[0-9][0-9]m/,x);sub(/\^\[\[39m/,x)}1'
Hello
User