parsing results in bash - bash

I have 2 query which give me the results:
service
number of service
example:
root#:~/elo# cat test | grep name | grep -v expand | cut -c 22- | rev | cut -c 3- | rev
service1
service2
root#:~/elo# cat test | grep customfield | cut -c 31- | rev | cut -c 2- | rev
2.3.4
55.66
I want to connect first value from first query with first value from second query etc. In this example should be:
service1:2.3.4
service2:55.66

Without a sample file, it is hard to write a working example. But I see, both values are from the same text file and the same line. Therefore I would use awk to do it:
$ cat text
service1;some_other_text;2.3.4
service2;just_some_text;55.66
$awk -F ";" '{printf "%s:%s\n", $1, $3}' test
service1:2.3.4
For a JSON file, it would be easier if you can use jg (e.g. apt-get install jg):
$ cat test.json
[
{
"name": "service1",
"customfield_10090": "1.2.3"
},
{
"name": "service2",
"customfield_10090": "23.3.2"
}
]
$jq '.[] | .name + ":" + .customfield_10090' test.json | sed 's/"//g'
service1:1.2.3
service2:23.3.2
The sed is necessary to eliminate the quotes.

You can use paste:
paste -d: <(grep name test| grep -v expand | cut -c 22- | rev | cut -c 3- | rev) \
<(grep customfield test | cut -c 31- | rev | cut -c 2- | rev)
But there might be better ways. If the input is json, you can probably use jq for a shorter and more efficient solution.

Related

Filtered Windows comand works on it's own inside WSL, but not in a script

I have this command which returns an IP successfully:
user#laptop:~$ systeminfo.exe | sed -n '/Connection Name: vEthernet (WSL)/,+4p' | egrep --word-regexp '\[01\]:' | awk '{print $2}'
172.22.0.1
I am trying to concatenate and export an environmental variable DISPLAY using a script with this content:
LOCAL_IP=$(systeminfo.exe | sed -n '/Connection Name: vEthernet (WSL)/,+4p' | egrep --word-regexp '\[01\]:' | awk '{print $2}')
export DISPLAY=$LOCAL_IP:0
But after this script runs, DISPLAY doesn't look like expected:
user#laptop:~$ echo $DISPLAY
:02.22.0.1
I was expecting an answer 172.22.0.1:0. What went wrong?
LOCAL_IP appears to have a trailing \r; od -c <<< "${LOCAL_IP}" should show the value ending in a \r
One fix using parameter substitution:
$ export DISPLAY="${LOCAL_IP//$'\r'/}:0"
$ echo "${DISPLAY}"
172.22.0.1:0
Another option would be to add an additional pipe on the end of OP's current command, a couple ideas (dos2unix, tr -d '\r'); 3rd option modifies the awk script to remove the \r:
systeminfo.exe | sed -n '/Connection Name: vEthernet (WSL)/,+4p' | egrep --word-regexp '\[01\]:' | awk '{print $2}' | dos2unix
# or
systeminfo.exe | sed -n '/Connection Name: vEthernet (WSL)/,+4p' | egrep --word-regexp '\[01\]:' | awk '{print $2}' | tr -d '\r'
# or
systeminfo.exe | sed -n '/Connection Name: vEthernet (WSL)/,+4p' | egrep --word-regexp '\[01\]:' | awk '{gsub(/\r/,"");print $2}'
Another option would be to replace the sed/egrep/awk/tr with a single awk call. If OP wants to go this route I'd recommend asking a new question, making sure to provide the complete output from systeminfo.exe to better understand the parsing requirements.

How to save the output of an sh to a Groovy variable?

I need to store the output of this command into a variable:
sh "curl -s 'http://nexus-cicd.stgcloud.xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.xml' | grep '<version>.*</version>' | sort --version-sort | uniq | tail -n1 | sed -e 's#\\(.*\\)\\(<version>\\)\\(.*\\)\\(</version>\\)\\(.*\\)#\\3#g'"
I tried the following, but echo output null
parentLast = sh ("curl -s 'http://nexus-cicd.stgcloud.xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.xml' | grep '<version>.*</version>' | sort --version-sort | uniq | tail -n1 | sed -e 's#\\(.*\\)\\(<version>\\)\\(.*\\)\\(</version>\\)\\(.*\\)#\\3#g'")
echo "$parentLast"

How to pass environment variable to a command passed to xmllint?

I'm trying to extract the db element's value attributes from an xml file using xmllint
My XPath Query needs to navigate to the correct dbtype, which I wish to store in an environment variable
Without an environment variable (hard coded), this command works
echo 'cat //rdbmsinfo/dbtype[#value="sqlserver"]/db/#value' | xmllint --shell "config.xml" | grep -v ">" | cut -f 2 -d "=" | tr -d \ | sed 's/"//g'
I am trying with below command:
echo 'cat //rdbmsinfo/dbtype[#value="$ldb_source_typ"]/db/#value' | xmllint --shell "config.xml" | grep -v ">" | cut -f 2 -d "=" | tr -d \ | sed 's/"//g'
where ldb_source_typ is a variable to get the value from shell parameter as "sqlserver". But the above syntax is not giving any output.
Why isn't this working, and how can I fix it?
My config.xml XML looks like this:
<?xml version="1.0"?>
<rdbmsinfo>
<dbtype value="sqlserver">
<db value="sqlsrv1">
<dbhostip>192.168.0.1</dbhostip>
<dbhostportno>2000</dbhostportno>
<dbusername>sample</dbusername>
<dbpassword>sample</dbpassword>
</db>
<db value="sqlsrv2">
<dbhostip>192.168.0.2</dbhostip>
<dbhostportno>2000</dbhostportno>
<dbusername>sample</dbusername>
<dbpassword>sample</dbpassword>
</db>
</dbtype>
<dbtype value="postgresql">
<db value="postsql1">
</db>
</dbtype>
</rdbmsinfo>
As I just found out, you need to set the environment variable between single quotes ( ' ).
So the following should work:
echo 'cat //rdbmsinfo/dbtype[#value="'$ldb_source_typ'"]/db/#value' | xmllint --shell "config.xml" | grep -v ">" | cut -f 2 -d "=" | tr -d \ | sed 's/"//g'
I was able to accomplish this with the following:
echo "cat //rdbmsinfo/dbtype[#value="ldb_source_type"]/db/#value" | sed 's/ldb_source_type/"'$SQLSERVER'"/g' | xmllint --shell config.xml| grep -v ">" | cut -f 2 -d "=" | tr -d \ | sed 's/"//g'
Which produces the following output:
sqlsrv1
-------
sqlsrv2
I didn't care enough to figure out how to make the value placeholder use an $ symbol, I may come back and edit it eventually.
Edit: here's one with a $, but you need to escape it with a :
echo "cat //rdbmsinfo/dbtype[#value="\$ldb_source_type"]/db/#value" | sed 's/\$ldb_source_type/"'$SQLSERVER'"/g'| xmllint --shell config.xml| grep -v ">" | cut -f 2 -d "=" | tr -d \ | sed 's/"//g'

Generate User List From AV output File In Bash

I want to generate list of users having malware in their public html
I am using avgscan for scanning,
/opt/avg/av/bin/avgscan -a -c --ignerrors --report=avoutput.txt
but it generates report like
/home/someuser/mail/info/cur/1395054106.H396740P84180,S=47470:2,S:/form_ident.rar Trojan horse Inject2.WPP
/home/someuser/public_html/__swift/files/attach_pq2ar348en1z435o5jhqy37de2xfb391 Trojan horse Zbot.BMI
I tried some thing but it didnt workout as list also have /backup folder which I don't want to be counted in list
I just need list of users, how to do?
#!/bin/bash
in="your_av_report file.txt" # in this case avoutput.txt
a=$(cat $in | grep -i "/mail/" | grep -v "/backup/" | cut -d'/' -f3 | awk '!a[$0]++' | uniq)
b=$cat $in | grep -i "/public_html/" | grep -v "/backup/" | cut -d'/' -f3 | awk '!a[$0]++' | uniq)
echo $a >>foo.txt
echo $b >>foo.txt
I hope it Helps :)

OpenStreetMap Bash / CGI Script

I am trying to build a Bash CGI Script that takes in coordinates as parameters from the url and uses osmosis to extract the map and the splitter and mkgmap to make the map so that it can be opened with Qlandkarte. My problem being is that when i type wget localhost/cgi-bin/script.pl?top=42&left=10&bottom=39&right=9&file=map.osm the linux terminal reads the file with the coordinates. How can I make wget just activate the script so it takes the coordinates and executes the commands. And also when the map is created at the end how can a return the file that was created by the script.
Thanks
#!/bin/bash
TOP=`echo "$QUERY_STRING" | grep -oE "(^|[?&])top=[0.0-9.0]+" | cut -f 2 -d "=" | head -n1`
LEFT=`echo "$QUERY_STRING" | grep -oE "(^|[?&])left=[0.0-9.0]+" | cut -f 2 -d "=" | head -n1`
BOTTOM=`echo "$QUERY_STRING" | grep -oE "(^|[?&])bottom=[0.0-9.0]+" | cut -f 2 -d "=" | head -n1`
RIGHT=`echo "$QUERY_STRING" | grep -oE "(^|[?&])right=[0.0-9.0]+" | cut -f 2 -d "=" | head -n1`
FILE=`echo "$QUERY_STRING" | grep -oE "(^|[?&])file=[^&]+" | sed "s/%20/ /g" | cut -f 2 -d "="`
$(sudo osmosis --read-xml file=bulgaria.osm --bounding-box top=$TOP left=$LEFT bottom=$BOTTOM right=$RIGHT --write-xml file=$FILE)
$(sudo java -Xmx900m -jar splitter.jar --max-nodes=110000 $FILE)
$(sudo java -ea -Xmx900m -jar mkgmap.jar --tdbfile --route -c template.args)
echo "Content-type: text/html"
echo ""

Resources