Shell Function to get value of a property from YAML file - shell

I am trying to create a unix shell/bash function to get a specific value from a YAML file.
I don't want to use Python/Ruby or other scripting languages or do not want to install any packages that are not natively available in bash.
The function should take the property name name and yaml file name as input and return the value of the property.
Sample YAML is given below.
---
Action: start
Version: 642
Domains:
Domain:
Name: SanityTest
AppSpaces:
AppSpace:
Name: SanityAppSpace
AppNodes:
AppNode:
Name: SanityAppnode
Applications:
Application:
Name: InstagramTest.application
Version: "1.0"
AppNode:
Name: SanityAppnode_1
Applications:
Application:
Name: InstagramTest.application
Version: "1.0"
I am looking for some function that works like:
DomainName=getValue Domains_Domain[1]_Name /path/my_yaml.yml
AppSpaceName=getValue Domains_Domain[$DomainName]_AppSpaces_AppSpace[1]_Name /path/my_yaml.yml
AppNodeName=getValue Domains_Domain[$DomainName]_AppSpaces_AppSpace[$AppSpaceName]_AppNodes_AppNode[1]_Name /path/my_yaml.yml
AppName=getValue Domains_Domain[$DomainName]_AppSpaces_AppSpace[$AppSpaceName]_AppNodes_AppNode[$AppNodeName]_Applications_Application[1]_Name /path/my_yaml.yml
AppVersion=getValue Domains_Domain[$DomainName]_AppSpaces_AppSpace[$AppSpaceName]_AppNodes_AppNode[$AppNodeName]_Applications_Application[$AppName]_Version /path/my_yaml.yml
I came up with the below code so far, but it is not working as expected. Appreciate any help. Please advise if my approach is wrong and if there is better way.
function getValue {
local ymlFile=$2
local Property=$1
IFS='_' read -r -a props_tokens <<< "${Property}"
local props_index=0
echo "${props_tokens[props_index]}"
CheckingFor="Domains"
currEntity_Index=0
while IFS= read -r line; do
curr_prop="${props_tokens[props_index]}"
curr_prop_Index=0
IFS=':' read -r -a curr_line_props <<< "${line}"
curr_line_prop = ${curr_line_props[0]}
if [ "${props_tokens[props_index]}" == *"["*"]"* ]
then
IFS='[' read -r -a prop_tokens <<< "${props_tokens[props_index]}"
curr_prop=${prop_tokens[0]}
curr_prop_Index=${prop_tokens[1]::-1}
echo "Index processed"
echo $curr_prop
echo $curr_prop_Index
else echo "No Index for this property"
fi
if [ "$curr_line_prop" != "$CheckingFor" ]
then continue
fi
if [ "|Action|Name|Version|" == *"$curr_prop"* ]
then
return curr_line_props[1]
fi
if [ "$curr_prop" == "$CheckingFor" ]
then
if [ "|Domains|AppSpaces|AppNodes|Applications|" == *"$curr_prop"* ]
then
props_index++
case $CheckingFor in
Domains)
CheckingFor="Domain";;
Domain)
CheckingFor="AppSpaces";;
AppSpaces)
CheckingFor="AppSpace";;
AppSpace)
CheckingFor="AppNodes";;
AppNodes)
CheckingFor="AppNode";;
AppNode)
CheckingFor="Applications";;
Applications)
CheckingFor="Application";;
*) echo "$curr_prop - not switched";;
esac
continue
else
if [ "$curr_prop_Index" == 0 ]
then
case $CheckingFor in
Domains)
CheckingFor="Domain";;
Domain)
CheckingFor="AppSpaces";;
AppSpaces)
CheckingFor="AppSpace";;
AppSpace)
CheckingFor="AppNodes";;
AppNodes)
CheckingFor="AppNode";;
AppNode)
CheckingFor="Applications";;
Applications)
CheckingFor="Application";;
*) echo "$curr_prop - not switched";;
esac
props_index++
continue
else
Integer_regex='^[0-9]+$'
if [[ $curr_prop_Index =~ $Integer_regex ]]
then
# Iterate until we get to the nth Item
currEntity_Index++
if [ $currEntity_Index == $curr_prop_Index ]
then
case $CheckingFor in
Domains)
CheckingFor="Domain";;
Domain)
CheckingFor="AppSpaces";;
AppSpaces)
CheckingFor="AppSpace";;
AppSpace)
CheckingFor="AppNodes";;
AppNodes)
CheckingFor="AppNode";;
AppNode)
CheckingFor="Applications";;
Applications)
CheckingFor="Application";;
*) echo "$curr_prop - not switched";;
esac
currEntity_Index=0
else
fi
props_index++
continue
else
# Iterate until the name of the entity match and continue with next property token
# How to handle if the search sipllied into next object?
fi
fi
fi
else
props_index++
continue
fi
done < $ymlFile
return "${props_tokens[props_index]}"
}

Related

obtain dynamically executable path depending on script variable avoiding many ifs

I want to run determined executable depending on a variable of my script cur_num
I have this code, where the paths are "random", meaning they have nothing to do with the secuential cur_num:
run() {
if [ $cur_num = "0" ]
then ~/pathDirName0/execFile0
elif [ $cur_num = "1" ]
then ~/pathDirName1/execFile1
elif [ $cur_num = "2" ]
then ~/pathDirName2/execFile2
elif [ $cur_num = "3" ]
then ~/pathDirName3/execFile3
elif [ $cur_num = "4" ]
then ~/pathDirName4/execFile4
fi
}
In case there are lots more of cases, that resulys in a very long if - elif statement.
As there are no enums in bash to build the cur_num- path relation, is there a cleaner way to obtain the desired path dynamically instead of with lots of ifs?
Try case
case $cur_num in
0) ~/pathDirName0/execFile0;;
1) ~/pathDirName1/execFile1;;
2) ~/pathDirName2/execFile2;;
...
esac
set allows you to create arrays in a portable manner, so you can use that to create an ordered array:
set -- ~/pathDirName0/execFile0 ~/pathDirName1/execFile1 ~/pathDirName2/execFile2 ~/pathDirName3/execFile3 ~/pathDirName4/execFile4
Then to access these items via an index you do $x where x is the index of the item.
Now your code would look something like this:
run() {
original="$#"
: $((cur_num+=1)) # Positional param indexing starts from 1
set -- ~/pathDirName0/execFile0 ~/pathDirName1/execFile1 \
~/pathDirName2/execFile2 ~/pathDirName3/execFile3 \
~/pathDirName4/execFile4
eval "$"$cur_num"" # cur_num = 1, accesses $1 == execFile0
set -- $original # Restore original args
}
A method that is both less and more hacky that would work for indexes above 9 and not mess with the original positional parameters nor use eval
run() {
count=0
for item in "$#"; do
[ "$count" = "$cur_num" ] && {
"$item"
return
}
: "$((count+=1))"
done
echo "No item found at index '$cur_num'"
}
cur_num="$1" # Assuming first arg of script.
run ~/pathDirName0/execFile0 ~/pathDirName1/execFile1 \
~/pathDirName2/execFile2 ~/pathDirName3/execFile3 \
~/pathDirName4/execFile4

How to parse a Terraform file (versions.tf) in Bash?

I'd like to parse a Terraform file named versions.tf which give provider requirements for the project. This should be in bash script.
This file is formatted like this :
terraform {
required_providers {
archive = {
source = "hashicorp/archive"
version = "~> 1.3.0"
}
aws = {
source = "hashicorp/aws"
version = "~> 3.0.0"
}
local = {
source = "hashicorp/local"
version = "~> 2.0.0"
}
template = {
source = "hashicorp/template"
version = "~> 2.2.0"
}
random = {
source = "hashicorp/random"
version = "~> 3.0.1"
}
}
required_version = ">= 0.13"
}
The goal is to have 3 variables likes so in a for loop :
$name = "archive"
$source = "hashicorp/archive"
$version = "1.3.0" # we don't take care of the version constraints
The structure is almost the same for different projects we have.
I've already tried to parse it, but as I'm a noob in text parsing, nothing concrete.
I also tried to use the command terraform providers schema -json but it doesn't work while you don't have initialized the terraform script (it need to be not initialized).
As I will use the script in my corporate Jenkins pipeline, I can't access Internet and I don't have access to binaries like jq or something not "red hat" native.
Thanks you for your help !
This script scans for the lines with the name, which are identifiable by the third field being {, and after each of these reads the two lines with source and version, assigning to variables with the same name:
while read name _ rest
do if [ "$rest" = "{" ]
then read var _ val; eval $var=$val
read var _ val; eval $var=$val
echo $name $source ${version#~> }
fi
done <versions.tf
Might do what you wanted.
#!/usr/bin/env bash
while read -r line; do
[[ $line != *'='* ]] && continue
if [[ $line == *'= {' ]]; then
line=${line%=*\{}
printf '$name = "%s"\n' "${line% }"
else
printf '$%s\n' "${line//~> }"
fi
done < versions.tf
If the required_version line needs to be omitted add
[[ $line == 'required_version'* ]] && continue
Below the line that has continue

how to process last x lines of a file

I want to Analyse a logfile for specific Errors.
Therefore i want to be able to loop through the last x lines of the files and check every line with a specific REGEX Pattern and then define a specific return value.
The logfile Looks in case of success as follows at the Moment when i want to check it.
….
sftp> get blahblah/blahblah
sftp> bye
In case of an Error there is something between the two sftp lines.
What i allready tried is to solve the Problem with a specific regex which worked fine on some online Regex testers but couldn´t get it to work in ksh.
My current Approach is the following
LOG_FIL="test_log"
MODE="${1}"
check_log_file() {
ERRNBR=${1}
REGEX=${2}
TAIL=${3}
RETURN="0"
echo "ERRNBR = ${ERRNBR}"
echo "REGEX = ${REGEX}"
echo "TAIL = ${TAIL}"
while read line; do
echo "${line}"
if [[ "${line}" =~ ${REGEX} ]]; then
RETURN="0"
echo "bin hier"
else
RETURN=${ERRNBR}
echo "bin wo anders"
break
fi
done <<<$(tail -${TAIL} ${LOG_FIL})
echo "${RETURN}"
return ${RETURN}
}
echo "sftp> get cwi/cdk_final*" >> ${LOG_FIL}
if [ "${MODE}" == "1" ]; then
echo "Werner ist der beste" >>${LOG_FIL}
fi
check_log_file "22" "^(sftp> ).*$" "1"
echo "$?"
echo "sftp> bye" >> ${LOG_FIL}
check_log_file "21" "((sftp> ).*|(sftp> bye))" "2"
echo "$?"
The results i get are the following
edv> sh cdk_test4sftp.sh 1
ERRNBR = 22
REGEX = ^(sftp> ).*$
TAIL = 1
Werner ist der beste
bin wo anders
22
22
ERRNBR = 21
REGEX = ((sftp> ).*|(sftp> bye))
TAIL = 2
Werner ist der beste sftp> bye
bin hier
0
0
What i hoped to achieve was that the Output coming from the tail command would be seperated. So that i ccould test each line individually.
Your second regex:
((sftp> ).*|(sftp> bye))
Matches the following line, which is why your function returns 0:
Werner ist der beste sftp> bye
Since you want to match the following pattern on each line:
sftp> get blahblah/blahblah
sftp> bye
Your regex should look more like the first one you used to match:
^(sftp> ).*$

I am not able to replace content from a file using shell script

(function(){})();
window.getJetPaths=function(b){var c={knockout:"/emsaasui/uifwk/libs/4.2.0-41239.001032001/js/oraclejet/js/libs/knockout/knockout-3.4.0","knockout.mapping":"/emsaasui/uifwk/libs/4.2.0-41239.001032001/js/oraclejet/js/libs/knockout/knockout.mapping-latest",jquery:"/emsaasui/uifwk/libs/4.2.0-41239.001032001/js/oraclejet/js/libs/jquery/jquery.min",jqueryui:"/emsaasui/uifwk/libs/4.2.0-41239.001032001/js/oraclejet/js/libs/jquery/jquery-ui-1.12.0.custom.min","jqueryui-amd":"/emsaasui/uifwk/libs/4.2.0-41239.001032001/js/oraclejet/js/libs/jquery/jqueryui-amd-1.12.0.min",promise:"/emsaasui/uifwk/libs/4.2.0-41239.001032001/js/oraclejet/js/libs/es6-promise/es6-promise.min",
require:"/emsaasui/uifwk/libs/4.2.0-41239.001032001/js/oraclejet/js/libs/require/require",hammerjs:"/emsaasui/uifwk/libs/4.2.0-41239.001032001/js/oraclejet/js/libs/hammer/hammer-2.0.8.min",ojs:"/emsaasui/uifwk/libs/4.2.0-41239.001032001/js/oraclejet/js/libs/oj/v4.2.0/min",ojL10n:"/emsaasui/uifwk/libs/4.2.0-41239.001032001/js/oraclejet/js/libs/oj/v4.2.0/ojL10n",ojtranslations:"/emsaasui/uifwk/libs/4.2.0-41239.001032001/js/oraclejet/js/libs/oj/v4.2.0/resources",ojdnd:"/emsaasui/uifwk/libs/4.2.0-41239.001032001/js/oraclejet/js/libs/dnd-polyfill/dnd-polyfill-1.0.0.min",
signals:"/emsaasui/uifwk/libs/4.2.0-41239.001032001/js/oraclejet/js/libs/js-signals/signals.min",customElements:"/emsaasui/uifwk/libs/4.2.0-41239.001032001/js/oraclejet/js/libs/webcomponents/custom-elements.min",crossroads:null,history:null,text:"/emsaasui/uifwk/libs/4.2.0-41239.001032001/js/oraclejet/js/libs/require/text"},a;for(a in c)c[a]?b[a]=c[a]:delete b[a];return b};
window.getJetBundles=function(){return window.isDevMode?[]:"customElements hammerjs jqueryui jqueryui-amd jqueryui-amd/data jqueryui-amd/disable-selection jqueryui-amd/focusable jqueryui-amd/ie jqueryui-amd/keycode jqueryui-amd/plugin jqueryui-amd/position jqueryui-amd/safe-active-element jqueryui-amd/safe-blur jqueryui-amd/scroll-parent jqueryui-amd/tabbable jqueryui-amd/unique-id jqueryui-amd/version jqueryui-amd/widget jqueryui-amd/widgets/draggable jqueryui-amd/widgets/mouse jqueryui-amd/widgets/sortable knockout knockout.mapping ojL10n ojdnd ojs/internal-deps/dvt/DvtAxis ojs/internal-deps/dvt/DvtChart ojs/internal-deps/dvt/DvtDiagram ojs/internal-deps/dvt/DvtGauge ojs/internal-deps/dvt/DvtLegend ojs/internal-deps/dvt/DvtNBox ojs/internal-deps/dvt/DvtOverview ojs/internal-deps/dvt/DvtPanZoomCanvas ojs/internal-deps/dvt/DvtPictoChart ojs/internal-deps/dvt/DvtSubcomponent ojs/internal-deps/dvt/DvtTagCloud ojs/internal-deps/dvt/DvtThematicMap ojs/internal-deps/dvt/DvtTimeAxis ojs/internal-deps/dvt/DvtTimeComponent ojs/internal-deps/dvt/DvtTimeline ojs/internal-deps/dvt/DvtToolkit ojs/internal-deps/dvt/DvtTreeView ojs/ojaccordion ojs/ojanimation ojs/ojarraydatagriddatasource ojs/ojarraypagingdatasource ojs/ojarraytabledatasource ojs/ojbutton ojs/ojchart ojs/ojcheckboxset ojs/ojcollapsible ojs/ojcollectiondatagriddatasource ojs/ojcollectionpagingdatasource ojs/ojcollectiontabledatasource ojs/ojcollectiontreedatasource ojs/ojcomponentcore ojs/ojcomponents ojs/ojconveyorbelt ojs/ojcore ojs/ojcube ojs/ojcustomelement ojs/ojdatacollection-common ojs/ojdatacollection-utils ojs/ojdatagrid ojs/ojdatagrid-model ojs/ojdataprovider ojs/ojdataprovideradapter ojs/ojdatasource-common ojs/ojdatetimepicker ojs/ojdefer ojs/ojdiagram ojs/ojdialog ojs/ojdomscroller ojs/ojdvt-base ojs/ojeditablevalue ojs/ojeventtarget ojs/ojfilmstrip ojs/ojflattenedtreedatagriddatasource ojs/ojflattenedtreetabledatasource ojs/ojgauge ojs/ojindexer ojs/ojinputnumber ojs/ojinputtext ojs/ojjquery-hammer ojs/ojjsontreedatasource ojs/ojkeyset ojs/ojknockout ojs/ojknockout-model ojs/ojknockout-validation ojs/ojkoshared ojs/ojlabel ojs/ojlegend ojs/ojlistdataproviderview ojs/ojlistview ojs/ojmasonrylayout ojs/ojmenu ojs/ojmessaging ojs/ojmodel ojs/ojmodule ojs/ojmoduleanimations ojs/ojnavigationlist ojs/ojnbox ojs/ojoffcanvas ojs/ojoptgroup ojs/ojoption ojs/ojpagingcontrol ojs/ojpagingcontrol-model ojs/ojpagingdatagriddatasource ojs/ojpagingtabledatasource ojs/ojpictochart ojs/ojpopup ojs/ojpopupcore ojs/ojprogress ojs/ojprogressbar ojs/ojpulltorefresh ojs/ojradiocheckbox ojs/ojradioset ojs/ojrouter ojs/ojrowexpander ojs/ojselectcombobox ojs/ojslider ojs/ojsunburst ojs/ojswipetoreveal ojs/ojswitch ojs/ojtable ojs/ojtable-model ojs/ojtabs ojs/ojtagcloud ojs/ojtemplateengine ojs/ojthematicmap ojs/ojtime-base ojs/ojtimeline ojs/ojtoolbar ojs/ojtouchproxy ojs/ojtrain ojs/ojtree ojs/ojtree-model ojs/ojtreemap ojs/ojvalidation ojs/ojvalidation-base ojs/ojvalidation-datetime ojs/ojvalidation-number ojtranslations/nls/localeElements ojtranslations/nls/ojtranslations promise signals text".split(" ")};
window.getUifwkBundles=function(){return window.isDevMode?[]:"uifwk/js/util/ajax-util uifwk/js/util/df-util uifwk/js/util/logging-util uifwk/js/sdk/logging-feature-usage-util uifwk/js/util/message-util uifwk/js/util/mobile-util uifwk/js/util/preference-util uifwk/js/util/screenshot-util uifwk/js/util/typeahead-search uifwk/js/util/usertenant-util uifwk/js/util/zdt-util uifwk/js/util/uifwk-preload-util uifwk/js/sdk/context-util uifwk/js/sdk/menu-util uifwk/js/widgets/aboutbox/js/aboutbox uifwk/js/widgets/brandingbar/js/brandingbar uifwk/js/widgets/datetime-picker/js/datetime-picker uifwk/js/widgets/navlinks/js/navigation-links uifwk/js/widgets/timeFilter/js/timeFilter uifwk/js/widgets/widgetselector/js/widget-selector uifwk/js/widgets/tooltip/tooltip uifwk/js/sdk/widget-selector-util text!uifwk/js/widgets/aboutbox/html/aboutbox.html text!uifwk/js/widgets/navlinks/html/navigation-links.html text!uifwk/js/widgets/brandingbar/html/brandingbar.html text!uifwk/js/widgets/timeFilter/html/timeFilter.html text!uifwk/js/widgets/datetime-picker/html/datetime-picker.html text!uifwk/js/widgets/widgetselector/html/widget-selector.html".split(" ")};
//# sourceMappingURL=jetLoader.map
This is the file content here i want to replace window.getJetPaths whole defenition with a return statement only.
It is giving errors for not able to replace "/,{,}"
Below is my script:
#!/bin/bash
filePath='D:/emsaasui/uifwk/libs/4.2.0-41239/js/jetLoader.js'
line=$(awk '/getJetPaths/{print NR}' $filePath)
str=$(tail -n +$line $filePath)
#echo $str
tempStr=""
count=0
for (( i=0 ; i < ${#str} ; i++ )) {
arr[$i]=${str:i:1}
#printf "%s\n" "${arr[#]}"
if [[ ${arr[$i]} = '{' ]]
then
count=$((count+1))
echo $count
fi
if [[ $count -eq "1" || $count -eq "2" ]]
then
tempStr=${tempStr}${arr[$i]}
fi
if [[ ${arr[$i]} = '}' ]]
then
count=$((count-1))
if [[ $count = 0 ]]
then
break
fi
fi
}
echo $tempStr
sed 's/\\$tempStr{\([^}]*\)}/\1 %/g'
Thanks in advance.
Simplest case - it's all on one line anyway. Just do a literal substitution.
Assuming file is named x -
sed 's/window.getJetPaths=function.*return b};/window.getJetPaths=function(b){return b};/' x > y
Watch the metacharacters. Sed gives special meaning to those, and they don't match unless carefully qualified, or included in a broad match.

Sometimes Getting Incorrectly Generated Passwords

I am using the following function to generate passwords:
genpasswd() {
local l=$1
[[ "$l" == "" ]] && l=15
echo $(tr -dc "[:print:]" < /dev/urandom | head -c ${l})
}
This works almost all of the time; however, every now and then I get a password with a strange character or a password that is not the correct length. This can be demonstrated by running the following command:
COUNTER=0; while [ $COUNTER -lt 1000000 ]; do; pass=$(genpasswd); if [ ${#pass} -ne 15 ]; then; echo $pass; fi; done
It generated the following output for me for instance (I cut it off after a little while):
G/bteEpm
U^e^!7oT\hGC3)
S7Dcpio1GlQTM
CYOyMMA
$Ze^li
&EKl}'o]u7[]T5
i|y9#6P7?
I9)y
3HIZ"0S%65X[md
sIiWTHk2&0>k}e
O
iy3WET^1q+|(
Mh'6A:Z^;lP-x`
e
htQ%sGl4LWl2
:hhHeoG(Hd%(ct
lp?[Qw'~Z994iy
{SK"
giUI|L4Vz
d0G]+?x|
baR
G0B[sqBr,sXiZ:
o&vi[D4HA/uRwg
^(a(UX40ubapQJ
wh6ZW)]"=\O_:n
pNUapE`t$k#b1K
A`6^)?|'
Xt$vA
M8t!#T(J2#
a
B]FazJBd6#2iY1
\7d-V$O+LQcn8#
6<f-\%GNXKNqU'
W}kY{,N&WU[Gc1
]GPrZn678Vy0c`
O#"o#0zQh^g.('
1Sy).
]5J#rA-S
nk9g
(0{z][,M[
P5vB9[5A1C#h`o
_PC+r1S9$hg%CS
T1p-M6iB9t`-_F
o8RR-V`(|&M#'8
OD-k=SMu}7N$#C
2#qus0),FNMhIq
z$mzQ>kW9;HZ}!
dn#KA
Q>6,35gu
2Uj]_&|TbG"ZNo
1:U)JFjou{SaR9
vq}O}IV}s9:cT#
*[%(1U:lbB.NR!
w{znznAf.'6#UC
6_1m/E0\XaHv`V
q3#yuQ0mY+\pF&
%=,l^vE+]')[gO
AT4cMA5{p<Q'M:
xC9jR'I'Vx=LFQ
+RyVQ6%j{nTz>u
=oVpz+Uw+#CV?Q
KRkH
s'sl$Bk:$
f[}2*>^|q_5_|)
u>l}%*I.O/1ea!
&+joM"k-0`Dz-u
t
3
#`H[/?X!]A
[(p)*,4P>OB.K]
PVv}w#/RAw4#$M
x.S[GjK2Y?-O|A
f5NoAZ\ofTrkg&
DYcij8)]d3?<Yi
F+d#pK>kc7KtoP
7#EPtAe$QSw#%s
[P*I.k?DdM}lZc
jkuL37]OGu`CW\
q.c2%qm"'7r(se
vFwGk5;pj08>5
.DXE
lHhj%8.~
W<}wun]hMym%/<
!}7<JpP`,-h~%^
9UW?L=sEvymmI^
4#w4/E|=}5bQpb
^C
Why is it doing this? I assume some sort of escape character issue or something but I can't figure it out. How can I fix it?
Thanks to #hek2mgl I now have the following function which does not have the same problem:
genpasswd() {
local len=$1
[[ -z $len ]] && len=15
pwgen -syN 1 "$len"
}

Resources