I am running following command and output is shown below
and nova list
+--------------------------------------+---------------------------------------------------------------------+----------------+--------------------------------------------------+
| id | name | cidr | allocation_pools |
+--------------------------------------+---------------------------------------------------------------------+----------------+--------------------------------------------------+
| 1865fd7d-8971-4b2a-aaa3-c810189c2363 | test-inter-netwk | 192.168.1.0/24 | {"start": "192.168.1.2", "end": "192.168.1.254"} |
| 20214812-48f9-4938-bbd3-1b54cd796425 | App-Mgmt-subnet | 10.20.0.0/24 | {"start": "10.20.0.2", "end": "10.20.0.254"} |
| 7ab22397-e6a9-4a3d-96f0-9ebbce638f1e | vms2.1-net | 192.168.0.0/24 | {"start": "192.168.0.2", "end": "192.168.0.254"} |
| 8f97fc5e-53c6-4f29-9e4e-04f38a283452 | ddddd | 10.24.0.0/24 | {"start": "10.24.0.2", "end": "10.24.0.254"} |
| 990b338c-c980-456e-ade1-fc1a90f233e5 | Orch-Mgmt-subnet | 10.32.1.0/24 | {"start": "10.32.1.2", "end": "10.32.1.254"} |
| f386aca1-4c18-4b8e-9df7-a70111dea849 | TRANSIT-NEXTIP-Skyfall-eb07721aa407423e83e9641855f51e35-mseg-SUBNET | 10.22.0.0/24 | {"start": "10.22.0.2", "end": "10.22.0.254"} |
+--------------------------------------+---------------------------------------------------------------------+----------------+--------------------------------------------------+
Command
neutron subnet-list | awk -F "|" '{print $2}'
Output
output starts with two empty line and the text "id" and empty line.I need only id I am struggling to remove first few line.
id
1865fd7d-8971-4b2a-aaa3-c810189c2363
20214812-48f9-4938-bbd3-1b54cd796425
7ab22397-e6a9-4a3d-96f0-9ebbce638f1e
8f97fc5e-53c6-4f29-9e4e-04f38a283452
990b338c-c980-456e-ade1-fc1a90f233e5
f386aca1-4c18-4b8e-9df7-a70111dea849
You can add a conditional to your awk statement:
neutron subnet-list | awk -F "|" 'NR>2{print $2}'
That just says "IF the Record Number (NR) is greater than 2, then go ahead and print.
If you want to remove the blank after the id you could add that:
neutron subnet-list | awk -F "|" 'NR>2 && NR!=4{print $2}'
Which adds to the first conditional " And Record Number is not 4"
You could also just strip out any blank $2 fields to keep it simple:
neutron subnet-list | awk -F "|" '$2!=""{print $2}'
Use sed to print only lins that match (-n option and p at the end).
Look for a match at the beginning of the line, right after |.
Search for a string starting with at least 3 non-spaces (skipping ip but perhaps a problem with other neutron output) using [^ ]\{3\}.
Add the next characters until a space in the string to be remembered in \(..\).
Throw away the remaining characters with .*.
Put back the first (and only) remembered string with \1.
neutron subnet-list |sed -n 's/^| \([^ ]\{3\}[^ ]*\).*/\1/p'
Related
I have the output of query result into a bash variable, stored as a single line.
-------------------------------- | NAME | TEST_DATE | ----------------
--------------------- | TESTTT_1 | 2019-01-15 | | TEST_2 | 2018-02-16 | | TEST_NAME_3 | 2020-03-17 | -------------------------------------
I would like to ignore the column names(NAME | TEST_DATE) and store actual values of each name and test_date as a tuple in an array.
So here is the logic I am thinking, I would like to extract third string onwards between two '|' characters. These strings are comma separated and when a space is encountered we start the next tuple in the array.
Expected output:
array=(TESTTT_1,2019-01-15 TEST_2,2018-02-16 TEST_NAME_3,2020-03-17)
Any help is appreciated. Thanks.
let say your
String is stored in variable a (or pipe our query output to below command
echo "$a"
-------------------------------- | NAME | TEST_DATE | ----------------
--------------------- | TESTTT_1 | 2019-01-15 | | TEST_2 | 2018-02-16 | | TEST_NAME_3 | 2020-03-17 | ------------------------------------
Command to obtain desired results is:
array="$(echo "$a" | cut -d '|' -f2,3,5,6,8,9 | tail -n1 | sed 's/ | /,/g')
Above will store ourput in variable named array as you expected
Output of above command is:
echo "$array"
TESTTT_1,2019-01-15,TEST_2,2018-02-16,TEST_NAME_3,2020-03-17
Explanation of command: output of echo $a will be piped into cut and using '|' as delimeter it will cut fields 2,3,5,6,8,9 then the output is piped into tail to remove the undesired NAME and TEST_DATE columns and provide values only and then as per your expected output | will be converted to , using sed.
Here in this string you are having only three dates if you have more then just in cut command add more field numbers and as per format of your string field numbers will be in following style 2,3,5,6,8,9,11,12,14,15 .... and so on.
Hope it solved your problem.
echo "$a" | awk -F "|" '{ for(i=2; i<=NF; i++){ print $i }}' | sed -e '1,3d' -e '$d' | tr ' ' '\n' | sed '/^$/d' | sed 's/^/,/g' | sed -e 'N;s/\n/ /' | sed 's/^.//g' | xargs | sed 's/ ,/, /g'
Above is awk based solution
Output:
TESTTT_1, 2019-01-15 TEST_2, 2018-02-16 TEST_NAME_3, 2020-03-17
Is it ok.
First of all it amazes me there is so little information about openstack and script examples but that is not the question i have
I want to create a snapshot and an simple way to restore the snapshot. Because the way of our hosting provider uses underlying storage i am unable to use the rebuild command so i need to destroy the running vm and recreate it with the snapshot image as a base. The creation of the image only works when all information about the running vm is provided as input parameters and here comes the troubles i have.
the information needed is provided by 3 commands
command1: nova show
Output:
+--------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Property | Value |
+--------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| NWFINFRA_1600 network | 10.0.0.39 |
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | gn3a |
| OS-EXT-STS:power_state | 1 |
| OS-EXT-STS:task_state | - |
| OS-EXT-STS:vm_state | active |
| OS-SRV-USG:launched_at | 2019-10-02T14:25:21.000000 |
| OS-SRV-USG:terminated_at | - |
| accessIPv4 | |
| accessIPv6 | |
| config_drive | |
| created | 2019-10-02T14:25:05Z |
| description | - |
| flavor:disk | 0 |
| flavor:ephemeral | 0 |
| flavor:extra_specs | {"ostype": "win", "hw:cpu_cores": "1", "hw:cpu_sockets": "2"} |
| flavor:original_name | win.2large |
| flavor:ram | 8192 |
| flavor:swap | 0 |
| flavor:vcpus | 2 |
| hostId | 18aa94c61106a53b2d9e672e93619a6fce76abb1ee6ba9da471491f9 |
| id | 70941fbf-9143-4f1c-a5e7-979f818ace23 |
| image | IFW039-InstanceSnapshot (8ee1104d-55e4-4c99-93e5-ceb4a53ce13f) |
| key_name | - |
| locked | False |
| metadata | {} |
| name | IWF039 |
| os-extended-volumes:volumes_attached | [{"id": "1134fe12-777b-4c26-ac2b-e6ecb6ad4f70", "delete_on_termination": false}, {"id": "f610a46e-46ad-460f-81b3-e2b34acfbbfc", "delete_on_termination": false}] |
| progress | 0 |
| status | ACTIVE |
| tags | [] |
| tenant_id | 4c15fd467dde4bd6a25427d6bab64a7f |
| trusted_image_certificates | - |
| updated | 2019-10-02T14:25:21Z |
| user_id | ddff2ce854114bef873bac9a1476805e |
+--------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+
Command 2: ./Scripts/openstack port show NWFINFRA_1600_IWF039
where NWFINFRA_1600_IWF039 is a combination of previous output NWFINFRA_1600 network and the server name IWF039
Output:
+-------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+-------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| admin_state_up | UP |
| allowed_address_pairs | |
| binding_host_id | None |
| binding_profile | None |
| binding_vif_details | None |
| binding_vif_type | None |
| binding_vnic_type | normal |
| created_at | 2019-10-02T06:49:08Z |
| data_plane_status | None |
| description | |
| device_id | 70941fbf-9143-4f1c-a5e7-979f818ace23 |
| device_owner | compute:gn3a |
| dns_assignment | fqdn='iwf039.rijkscloud.local.', hostname='iwf039', ip_address='10.0.0.39' |
| dns_domain | |
| dns_name | iwf039 |
| extra_dhcp_opts | |
| fixed_ips | ip_address='10.0.0.39', subnet_id='3298e8d0-b317-465c-8757-c1a4f2cad298' |
| id | b49a7d3a-bb0d-49cb-a04b-64c5dbf9df20 |
| location | cloud='', project.domain_id='default', project.domain_name=, project.id='4c15fd467dde4bd6a25427d6bab64a7f', project.name='vws-pgb', region_name='Groningen3', zone= |
| mac_address | fa:16:3e:99:cc:3c |
| name | NWFINFRA_1600_IWF039 |
| network_id | 450dcc7a-5e55-4e38-9f4e-de9a9c685502 |
| port_security_enabled | False |
| project_id | 4c15fd467dde4bd6a25427d6bab64a7f |
| propagate_uplink_status | None |
| qos_policy_id | None |
| resource_request | None |
| revision_number | 15 |
| security_group_ids | |
| status | ACTIVE |
| tags | |
| trunk_details | None |
| updated_at | 2019-10-03T11:10:00Z |
+-------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------+
with these outputs i can create the build command to restore the snapshot:
nova boot --poll --flavor win.2large --image IFW039-InstanceSnapshot --security-groups default --availability-zone gn3a --nic net-id=450dcc7a-5e55-4e38-9f4e-de9a9c685502 IWF039
note: the image is the name of the created snapshot
I try to script this so i can have a simple snapshot create and restore procedure
but i get stuck on the table layout of the output. This shows really nice but i cannot use it in my scripting to redirect the output to input variables.
I tryed using this: { read foo ; read ID Name MAC IP status;} < <(./Scripts/openstack port list --server IWF039 | sed 's/+--------------------------------------+----------------------+-------------------+--------------------------------------------------------------------------+-------- +//' | sed 's/|//' | sed 's/MAC Address/MAC/' | sed 's/Fixed IP Addresses/IP/')
But the variables get contents like '|' char etc.
So echo $Name gives '|' as output.
There must be a simpler way but i am unable to see it.
Please help ...
I managed to get it almost working by using awk instead of grep:
i have now this code:
#/bin/bash # # Query needed variables # echo -e "\nQuery needed information" NETWORK=$(nova show IWF039 | awk '/network/ {print $2}') ZONE=$(nova show IWF039 | awk '/OS-EXT-AZ:availability_zone/ {print $4}') FLAVOR=$(nova show IWF039 | awk '/flavor:original_name/ {print $4}') SERVERID=$(nova show IWF039 | awk -F '|' '/id/ {print $3; exit}') NETWORKPORT=$(nova interface-list IWF039 | awk -F '|' '/ACTIVE/ {print $3}') # Print out variables echo "network: $NETWORK" echo "zone: $ZONE" echo "flavor: $FLAVOR" echo "server_id: $SERVERID" echo "network_port_id: $NETWORKPORT" # Remove current instance echo -e "\nRemove current instance" nova delete $SERVERID # Rebuild instance from snapshot image echo -e "\nRebuild instance from snapshot" nova boot --poll --flavor $FLAVOR --image IFW039-InstanceSnapshot2 --security-groups default --availability-zone $ZONE --nic port-id=$NETWORKPORT IWF039
If i run the script the last item however e.g. IWF039 which is the name of the instance i want to use throws me an error:
error: unrecognized arguments: IWF039
anyone can tell me why ?
If i run the line on the commandline it works, only not from the bash script
#/bin/bash
#
# Script restores snapshot created in OpenStack
# Script created by Lex IT, Alex Flora
# usage: snapshot-restore.sh <snapshot image name> <server name to restore to>
# to use: make sure to install the following python openstack modules:
# pip install python-openstackclient python-keystoneclient python-glanceclient python-novaclient python-neutronclient
#
#
# Query needed variables
#
if [ "$#" -eq "0" ]
then
echo -e "usage: restore_snapshot <name of snapshot> <name of server>"
echo -e "Querying available snapshots, one moment please ..."
glance image-list
echo -e "\n\033[0;33mGive name of snapshot to restore"
echo -e "\033[0m"
read SNAPSHOT
echo -e "\n\033[0;33mGive server name to restore"
echo -e "\033[0m"
read SERVER
else
SNAPSHOT=$1
SERVER=$2
fi
echo -e "\n\033[0mQuery needed server information from server $SERVER, one moment please ..."
NETWORK=$(nova show IWF039 | awk '/network/ {print $2}' | sed -e 's/^[[:space:]]*//')
ZONE=$(nova show IWF039 | awk '/OS-EXT-AZ:availability_zone/ {print $4}' | sed -e 's/^[[:space:]]*//')
FLAVOR=$(nova show IWF039 | awk '/flavor:original_name/ {print $4}' | sed -e 's/^[[:space:]]*//')
SERVERID=$(nova show IWF039 | awk -F '|' '/\<id\>/ {print $3; exit}' | sed -e 's/^[[:space:]]*//')
NETWORKPORT=$(nova interface-list IWF039 | awk -F '|' '/ACTIVE/ {print $3}' | sed -e 's/^[[:space:]]*//')
# Print out variables
echo -e "\033[0mnetwork: \033[0;32m$NETWORK"
echo -e "\033[0mzone: \033[0;32m$ZONE"
echo -e "\033[0mflavor: \033[0;32m$FLAVOR"
echo -e "\033[0mserver_id: \033[0;32m$SERVERID"
echo -e "\033[0mnetwork_port_id: \033[0;32m$NETWORKPORT"
echo -e "\033[0mSnapshot image: \033[0;32m$SNAPSHOT"
echo -e "\033[0mServer naam: \033[0;32m$SERVER"
# Ask confirmation
echo -e "\n\033[0mGoing to restore snapshot image $SNAPSHOT to server $SERVER"
read -p "Is this correct (y/n) ? " -n 1 -r
if [[ $REPLY =~ ^[Yy]$ ]]
then
# Remove current instance
echo -e "\n\033[0mRemove current instance"
nova delete $SERVERID
sleep 3
# Rebuild instance from snapshot image
echo -e "\nRebuild instance from snapshot using command:"
echo -e "\033[0mnova boot --poll --flavor $FLAVOR --image $SNAPSHOT --security-groups default --availability-zone $ZONE --nic port-id=$NETWORKPORT $SERVER"
nova boot --poll --flavor $FLAVOR --image $SNAPSHOT --security-groups default --availability-zone $ZONE --nic port-id=$NETWORKPORT $SERVER
fi
This is my complete script. Problem was that the output contained spaces in front of the string. I resolved this with the following sed command: sed -e 's/^[[:space:]]*//'
Hopefully someone has some use of the script.
This question already has answers here:
How to clean up masscan output (-oL)
(4 answers)
Closed 6 years ago.
I have a problem with the output L options ("grep-able" output); for instance, it outputs this:
| 14.138.12.21:123 | unknown | disabled |
| 14.138.184.122:123 | unknown | disabled |
| 14.138.179.27:123 | unknown | disabled |
| 14.138.20.65:123 | unknown | disabled |
| 14.138.12.235:123 | unknown | disabled |
| 14.138.178.97:123 | unknown | disabled |
| 14.138.182.153:123 | unknown | disabled |
| 14.138.178.124:123 | unknown | disabled |
| 14.138.201.191:123 | unknown | disabled |
| 14.138.180.26:123 | unknown | disabled |
| 14.138.13.129:123 | unknown | disabled |
The above is neither very readable nor easy to understand.
How can I use Linux command-line utilities, e.g. sed, awk, or grep, to output something as follows, using the file above?
output
14.138.12.21
14.138.184.122
14.138.179.27
14.138.20.65
14.138.12.235
Using awk with field separator as space, and : and getting the second field:
awk -F '[ :]' '{print $2}' file.txt
Example:
% cat file.txt
| 14.138.12.21:123 | unknown | disabled |
| 14.138.184.122:123 | unknown | disabled |
| 14.138.179.27:123 | unknown | disabled |
| 14.138.20.65:123 | unknown | disabled |
| 14.138.12.235:123 | unknown | disabled |
| 14.138.178.97:123 | unknown | disabled |
| 14.138.182.153:123 | unknown | disabled |
| 14.138.178.124:123 | unknown | disabled |
| 14.138.201.191:123 | unknown | disabled |
| 14.138.180.26:123 | unknown | disabled |
| 14.138.13.129:123 | unknown | disabled |
% awk -F '[ :]' '{print $2}' file.txt
14.138.12.21
14.138.184.122
14.138.179.27
14.138.20.65
14.138.12.235
14.138.178.97
14.138.182.153
14.138.178.124
14.138.201.191
14.138.180.26
14.138.13.129
AWK is perfect for cases when you want to split the file by "columns", and you know exactly that the order of values/columns is constant. AWK splits the lines by a field separator (which can be a regular expression like '[: ]'). The column names are accessible by their positions from the left: $1, $2, $3, etc.:
awk -F '[ :]' '{print $2}' src.log
awk -F '[ :|]' '{print $3}' src.log
awk 'BEGIN {FS="[ :|]"} {print $3}' src.log
You can also filter the lines with a regular expression:
awk -F '[ :]' '/138\.179\./ {print $2}' src.log
However, it is impossible to capture substrings with the regular expression groups.
SED is more flexible in regard to regular expressions:
sed -r 's/^[^0-9]*([0-9\.]+)\:.*/\1/' src.log
However, it lacks many useful features of the Perl-like regular expressions we used to use in every day programming. For example, even the extended syntax (-r) fails to interpret \d as a number.
Perhaps, Perl is the most flexible tool for parsing files. You can opt to simple expressions:
perl -n -e '/^\D*([^:]+):/ and print "$1\n"' src.log
or make the matching as strict as you like:
perl -n -e '/^\D*((?:\d{1,3}\.){3}\d{1,3}):/ and print "$1\n"' src.log
using sed
sed -r 's/^ *[|] *([0-9]+[.][0-9]+[.][0-9]+[.][0-9]+):[0-9]{3}.*/\1/
Is there a way to use shell script to get only the name and net from the result as below:
Result
6cb7f14e-6466-4211-9a09-2b8e7ad92703 | name-erkoev4ja3rv | 2e3900ff36574cf9937d88223403da77 | ACTIVE | Running | net0=10.1.1.2; ing-net=10.1.1.3; net=10.1.1.4;
Expected Result
name-erkoev4ja3rv: 10.1.1.4
$ input="6cb7f14e-6466-4211-9a09-2b8e7ad92703 | name-erkoev4ja3rv | 2e3900ff36574cf9937d88223403da77 | ACTIVE | Running | net0=10.1.1.2; ing-net=10.1.1.3; net=10.1.1.4;"
$ echo "$input" | sed -E 's,^[^|]+ \| ([^ ]+).* net=([0-9.]+).*$,\1: \2,g'
name-erkoev4ja3rv: 10.1.1.4
echo "6cb7f14e-6466-4211-9a09-2b8e7ad92703 | name-erkoev4ja3rv | 2e3900ff36574cf9937d88223403da77 | ACTIVE | Running | net0=10.1.1.2; ing-net=10.1.1.3; net=10.1.1.4;" | awk -F ' ' '{print $3}{print $13}'
Does this satisfy your case?
Formatting the output based on the maximum column length. How can I achieve this?
Shell script or any tools is fine.
Input
| date | ID | Typ | Actn |
| 11/29/13 | ID660011 | DP | A |
| 11/29/13 | ID6600123 | DP | A |
Output
| date | ID | Typ| Actn|
| 11/29/13| ID660011 | DP | A |
| 11/29/13| ID6600123| DP | A |
EDIT:
If I use column -t, these are the errors:
$ column -t -s'|' -o'|'
input_file .feature > input_file _check.feature
column: illegal option -- o
usage: column [-tx] [-c columns] [-s sep] [file ...]
$ echo $SHELL /usr/local/bin/bash
$ column -t input_file .feature > input_file _check.feature column:
line too long
On your shell terminal, try this one:
$ awk '
{
for(i=1;i<=NF;i++)
printf("%-40s%c", $i, (i==NF) ? ORS : "")
}' FS=, file.txt