I have a situation in which i have to list file which is of the type as
databaseName.schemaName#1234sdf2323.lock where _Database_Name and _target_schema_name
_lockFolder are variables
# is a token then it is followed by the random alphanumeric number and the same is end with .lock .
I have acheived this in batch file through the code as
FOR /R %_lockFolder% %%F in (%_Database_Name%.%_target_schema_name%#*.lock) do (
for /f "tokens=1* delims=# " %%G IN ("%%~nF") DO (
SET _no=%%H
)
)
but when i am changing it into bash enviorment so that it can run on unix enviorment
for entry in "${_lockFolder}"/*
do
echo ENTRY "$entry"
name='${_lockFolder}/${_Database_Name}.${_target_schema_name}#*.lock'
ls -l $name > "${lockFolder}"
if [ "$?" -eq "0" ]
then
echo "Do your work here"
else
echo "No files are there for the given pattern"
fi
# exit 21
done
It is not able to recognize the pattern . The files are present in the folder which i have specified
you can simple write a for loop like,
for file in `find ${lockdir} -name "^${_lockFolder}/${_Database_Name}.${_target_schema_name}#*.lock$"`
do
echo $file
:
:
your job
done
example,
[root#giam20 unix]# ls
checksumupdator.sh GIAMMEFProcessor.sh GIAMRoleExtractor.sh
GIAMAccountExtractor.sh GIAMMetaDataLoader.sh GIAMRoleLoader.sh
GIAMAccountLoader.sh GIAMOOPControlledAttrExtractor.sh GIAMRoleMappingLoader.sh
GIAMAccountTransferLoader.sh GIAMOOPControlledAttrsLoader.sh
[root#giam20 unix]# find . -name "GIAM*.sh"
./GIAMAccountTransferLoader.sh
./GIAMIntermediateCodeUpgrader.sh
./GIAMServiceUpdator.sh
./GIAMOOPControlledAttrsUpdator.sh
./GIAMRoleUpdator.sh
./GIAMProvisioningPolicyExtractor.sh
./GIAMCompExemptionExtractor.sh
./GIAMApprovalNotificationLoader.sh
In Unix shell scripting, variable references are not resolved in a string that is enclosed in single quotes. Therefore, in this line
name='${_lockFolder}/${_Database_Name}.${_target_schema_name}#*.lock'
the value will be stored into name literally, including all the ${name} references. It is no surprise then that the pattern is not matched later.
So, just change the single quotes to double quotes:
name="${_lockFolder}/${_Database_Name}.${_target_schema_name}#*.lock"
Related
I have an array of empty subdirectories and when I loop through all of the subdirectories I want to do something different depending on if it is in the array of empty subdirectories or not.
My issue is that I don't think my script is picking up on if a value is in the empty directories array. When I loop through the arrays and echo their values I can see that some values are in both arrays so I don't know why they aren't being picked up.
Code:
readarray empty_dirs < <(find /local/documents/guests/* -maxdepth 0 -empty)
for f in "${empty_dirs[#]}"
do
echo $f
echo "------------------------"
done
guest_path=/local/documents/guests/guest*
guest_arr=( $guest_path )
for dir in "${guest_arr[#]}"
do
echo "$dir"
if [[ " ${empty_dirs[#]} " =~ " ${dir} " ]];then
echo "--- $dir found ---"
fi
done
Output:
/local/documents/guests/guest10
------------------------
/local/documents/guests/guest12
------------------------
/local/documents/guests/guest15
------------------------
/local/documents/guests/guest18
------------------------
/local/documents/guests/guest20
------------------------
/local/documents/guests/guest1
/local/documents/guests/guest10
/local/documents/guests/guest11
/local/documents/guests/guest12
/local/documents/guests/guest13
/local/documents/guests/guest14
/local/documents/guests/guest15
/local/documents/guests/guest16
/local/documents/guests/guest17
/local/documents/guests/guest18
/local/documents/guests/guest19
/local/documents/guests/guest2
/local/documents/guests/guest20
/local/documents/guests/guest21
/local/documents/guests/guest22
/local/documents/guests/guest23
/local/documents/guests/guest24
/local/documents/guests/guest25
/local/documents/guests/guest3
/local/documents/guests/guest4
/local/documents/guests/guest5
/local/documents/guests/guest6
/local/documents/guests/guest7
/local/documents/guests/guest8
/local/documents/guests/guest9
The problem is ${empty_dirs[#]} contains the names including trailing newlines.
You can stop storing them using the -t option of readarray, or remove them before running the final loop:
empty_dirs=${empty_dirs[#]%$'\n'}
First of all I'm trying to obtain a certain property from a KML file. For now, I tried
ogrinfo C:/test.kml -so -al | findstr "Extent"
which was recommended to me and outputs
Extent: (-100.054053, 33.702234) - (-94.647180, 37.125712)
I would require this in the form
-100.054053,-94.647180,33.702234,37.125712 for which I thought to use regex.
I tried the following just to see what it outputted:
ogrinfo C:/test.kml -so -al | findstr "Extent" | findstr /r /c:"-*[0-9]*\.[0-9]*"
but this still outputs
Extent: (-100.054053, 33.702234) - (-94.647180, 37.125712)
I read somewhere that Windows' FINDSTR only outputs the line where it matched and not the regex matches themselves. Is there some other way of doing it?
If I get that working I would save the matches in different variables somehow in a shell script. I'm no expert in shell scripting but I've been looking around and was thinking of doing something like this
#!/bin/bash
for /f "tokens=*" %%a in ('ogrinfo C:/test.kml -so -al ^| findstr "Extent" ^| findstr /r /c:"-*[0-9]*\.[0-9]*"') do (
echo %%a
#do something
)
done >output
but running this causes the shell to immediately disappears and can't even see the error.
Assumptions
You have a kml file with raw data.
You can extract a single line which starts with "Extent: " to get the values you want
Single line => there is only 1 line with that format in the kml file
The format of that line is:
Extent: (NUMBER1, NUMBER2) - (NUMBER3, NUMBER4)
A number can have the following characters: 0 1 2 3 4 5 6 7 8 9 . -
The output you want is:
NUMBER1,NUMBER3,NUMBER2,NUMBER4
Using Linux tools only, you can do this:
#!/bin/bash
#
datafile="data.kml"
# Ensure the data file exists
if [[ ! -f "$datafile" ]]
then
echo "ERROR: the data file does not exist."
exit 1
fi
# Extract the "Extent:" line
dataline=$(grep "Extent: " "$datafile")
# Make sure the line is of a valid format, and assign the number variables
if [[ $dataline =~ "Extent: ("([0-9.-]+)", "([0-9.-]+)") - ("([0-9.-]+)", "([0-9.-]+)")" ]] && number1="${BASH_REMATCH[1]}" && number2="${BASH_REMATCH[2]}" && number3="${BASH_REMATCH[3]}" && number4="${BASH_REMATCH[4]}"
then
echo "-----DEBUG-----"
echo "line==$dataline"
echo "1==$number1"
echo "2==$number2"
echo "3==$number3"
echo "4==$number4"
echo "-- END DEBUG --"
echo ""
echo "$number1,$number3,$number2,$number4"
else
echo "ERROR: there is no \"Extent: \" line in the data file ($datafile)"
fi
Details:
Everything is done in the if line.
=~ matches the left side with the pattern on the right side.
In the regular expression, you can define sections you want to reuse with ( ).
Ex: abcd(1)efgh(2)ijkl. The sections you can reuse are 1 and 2.
So in the if, each number is surrounded by parentheses.
When the =~ is processed, the BASH_REMATCH array is defined with each section.
The "DEBUG" echo statements can be removed or commented out.
If you have more than one "Extent: ..." in the KML file, you can loop on the lines and process each one at a time.
This question already has answers here:
Extract filename and extension in Bash
(38 answers)
Closed 1 year ago.
I'm trying to create a bash script that will move all files recursively from a source folder to a target folder, and simply rename files if they already exist. Similar to the way M$ Windows does, when a file exists it auto-renames it with "<filemame> (X).<ext>", etc. except for ALL files.
I've create the below, which works fine for almost all scenarios except when a folder has a (.) period in its name and a file within that folder has no extension (no period in its name).
eg a folder-path-file such as: "./oldfolder/this.folder/filenamewithoutextension"
I get (incorrectly):
"./newfolder/this (1).folder/filenamewithoutextension"
if "./newfolder/this.folder/filenamewithoutextension" already exist in the target location (./newfolder),
instead of correctly naming the new file: "./oldfolder/this.folder/filenamewithoutextension (1)"
#!/bin/bash
source=$1 ; target=$2 ;
if [ "$source" != "" ] && [ "$target" != "" ] ; then
#recursive file search
find "$source" -type f -exec bash -c '
#setup variables
oldfile="$1" ; osource='"${source}"' ; otarget='"${target}"' ;
#set new target filename with target path
newfile="${oldfile/${osource}/${otarget}}" ;
#check if file already exists at target
[ -f "${newfile}" ] && {
#get the filename and fileextension for numbering - ISSUE HERE?
filename="${newfile%/}" ; newfileext="${newfile##*.}" ;
#compare filename and file extension for missing extension
if [ "$filename" == "$newfileext" ] ; then
#filename has no ext - perhaps fix the folder with a period issue here?
newfileext="" ;
else
newfileext=".$newfileext" ;
fi
#existing files counter
cnt=1 ; while [ -f "${newfile%.*} (${cnt})${newfileext}" ] ; do ((cnt+=1)); done
#set new filename with counter - New Name created here *** Needs re-work, as folder with a period = fail
newfile="${newfile%.*} (${cnt})${newfileext}";
}
#show mv command
echo "mv \"$oldfile\" \"${newfile}\""
' _ {} \;
else
echo "Requires source and target folders";
fi
I suspect the issue is, how to properly identify the filename and extension, found in this line:
filename="${newfile%/}" ; newfileext="${newfile##*.}" which doesn't identify a filename properly (files are always after the last /).
Any suggestion on how to make it work properly?
UPDATED: Just some completion notes - Issues fixes with:
Initially Splitting each full path filename: path - filename - (optional ext)
Reconstructing the full path filename: path - filename - counter - (optional ext)
fixed the file move to ensure directory structure exists with mkdir -p (mv does not create new folders if they do not exist in the target location).
Maybe you could try this instead?
filename="${newfile##*/}" ; newfileext="${filename#*.}"
The first pattern means: remove the longest prefix (in a greedy way) up to the last /.
The second one: remove the prefix up to the first dot (the greedy mode seems unnecessary here) − and as you already noted, in case the filename contains no dot, you will get newfileext == filename…
Example session:
newfile='./oldfolder/this.folder/filenamewithoutextension'
filename="${newfile##*/}"; newfileext="${filename#*.}"
printf "%s\n" "$filename"
#→ filenamewithoutextension
printf "%s\n" "$newfileext"
#→ filenamewithoutextension
newfile='./oldfolder/this.folder/file.tar.gz'
filename="${newfile##*/}"; newfileext="${filename#*.}"
printf "%s\n" "$filename"
#→ file.tar.gz
printf "%s\n" "$newfileext"
#→ tar.gz
I'm trying to make a batch file that reads a txt file "ayylmao.txt" and find a specific word "hello" and replaces it with "xello".
The thing is that the "ayylmao.txt" contains specific characters.
Ayylmao.txt looks something like this:
‹‹R‹Ę‹/M‹;Ču‹č˙˙˙‹‹#‰‹‹#CëC;Đu‹čq˙˙˙‹‹#C‹D$‰;7u®‹Ó‹Ćčúţ˙˙„Ŕu3Ŕ‰YZ]_^[ĂŤ# SVWUÄđ‰$‹ô‹‰D$‹
‹‹#;Č‚† ‹Ř‹>_‹ůz;ßrv;Ču!‹B‹A‹B‹)B‹x uV‹čđţ˙˙ëM‹Ř‹>_‹ůz;ßu
‹B‹)Bë3‹Z‰\$‹>‹‹.}+ű‰|$+Č‹‰HŤT$‹čMţ˙˙„Ŕu3 hello Ŕë°ë‹‹ ‰‹;D$…Y˙˙˙3ŔÄ]_^[ĂSVW‹Ú‹đţ }ľ ëĆ˙˙ ć ˙˙‰sjh Vj
You can see the "hello" word in the last line. I want the batch to go to the process and give me a ayylmao1.txt that looks like this:
‹‹R‹Ę‹/M‹;Ču‹č˙˙˙‹‹#‰‹‹#CëC;Đu‹čq˙˙˙‹‹#C‹D$‰;7u®‹Ó‹Ćčúţ˙˙„Ŕu3Ŕ‰YZ]_^[ĂŤ# SVWUÄđ‰$‹ô‹‰D$‹
‹‹#;Č‚† ‹Ř‹>_‹ůz;ßrv;Ču!‹B‹A‹B‹)B‹x uV‹čđţ˙˙ëM‹Ř‹>_‹ůz;ßu
‹B‹)Bë3‹Z‰\$‹>‹‹.}+ű‰|$+Č‹‰HŤT$‹čMţ˙˙„Ŕu3 xello Ŕë°ë‹‹ ‰‹;D$…Y˙˙˙3ŔÄ]_^[ĂSVW‹Ú‹đţ }ľ ëĆ˙˙ ć ˙˙‰sjh Vj
You can see that "hello" is now "xello".
I found this batch file that replaces a word from a text file:
#echo off
REM -- Prepare the Command Processor --
SETLOCAL ENABLEEXTENSIONS
SETLOCAL DISABLEDELAYEDEXPANSION
if "%~1"=="" findstr "^::" "%~f0"&GOTO:EOF
for /f "tokens=1,* delims=]" %%A in ('"type %3|find /n /v """') do (
set "line=%%B"
if defined line (
call set "line=echo.%%line:%~1=%~2%%"
for /f "delims=" %%X in ('"echo."%%line%%""') do %%~X
) ELSE echo.
)
This code works for files that don't have specific characters very good if use it like this:
code.bat "hello" "xello" "ayylmao.txt">"ayylmao1.txt"
This code only types in ayylmao1.txt few special characters but replaces hello. I want all the special characters typed in there.
I made it like this:
chcp 1252
code.bat "hello" "xello" "ayylmao.txt">"ayylmao1.txt"
But it didn't work. It worked just like the first code.
If there is a way in PowerShell to do this I'd be glad to hear it.
What you have there looks like a binary file, not a text file, despite the extension. Batch is no good for editing binary files. In PowerShell it's doable, but you need to resort to working with the data bytes instead of simple text.
This is a basic example that will find the first occurrence of the string "hello" in your file and replace it with "xhello":
$f = 'C:\path\to\ayylmao.txt'
$stext = 'hello'
$rtext = [char[]]'xhello'
$len = $stext.Length
$offset = $len - 1
$data = [IO.File]::ReadAllBytes($f)
# find first occurrence of $stext in byte array
for ($i=0; $i -lt $data.Count - $offset; $i++) {
$slice = $data[$i..($i+$offset)]
if (-join [char[]]$slice -eq $stext) { break }
}
# Once you know the beginning ($i) and length ($len) of the array slice
# containing $stext you can "cut up" $data and concatenate the slices before
# and after $stext to the byte sequence you want to insert ($rtext):
#
# |<-- $stext -->|
# [...]['h','e','l','l','o'][...] <-- $data
# ^ ^ ^ ^
# | | | |
# | $i | $i+$len
# $i-1 $i+$offset (== $i+$len-1)
#
$rdata = $data[0..($i-1)] + [byte[]]$rtext + $data[($i+$len)..($data.Count-1)]
[IO.File]::WriteAllBytes($f, $rdata)
You'll need to adjust this code if you want the replacement to work differently (replace other occurrences as well, replace a different occurrence, …).
But it didn't work. It worked just like the first code. Help ?
This batch code is coming from this site and there is a link to discussion why it doesn't work with special characters.
Yes, the PowerShell replace command can replace the string and keep the special characters. To call it from within your batch script, use the following line
powershell -command "(get-content Ayylmao.txt) -replace 'hello','xello' | set-content Ayylmao.txt"
If you want to enter your parameters from the command line, then the line would be
powershell -command "(get-content %3) -replace '%1','%2' | set-content %4"
And if you want to use variables defined in the batch script, it is the same as you would for any batch script
set file=Ayylmao.txt
set Search_criteria=hello
set Replace_criteria=xello
powershell -command "(get-content %file%) -replace '%Search_criteria%','%Replace_criteria%' | set-content %file%"
I have to filter a text file filter.tmp containing two types of lines, this shows the difference:
findstr /r "^[0-9][0-9]*.*$" filter.tmp > filter-numbers.tmp
findstr /r "^[^0-9][^0-9]*.*$" filter.tmp > filter-text.tmp
What I need to do is to append lines containing text together like this and if line does contain number just put it to output file:
IF "current line" contains text THEN
previous line = concatenate "previous line" + "/" + "current line"
ELSE
echo "previous line" >> filter.out
echo "current line" >> filter.out
filter.tmp contains something like:
Hello
World
Foo
Bar
45: this is some line
Trouble
with code
66: another line
filter.out should look like:
Hello/World/Foo/Bar
45: this is some line
Trouble/with code
66: another line
I realize, this is very simple, but I just can not get it working. As I am thinking about it, it would be much easier to use C++....
This is a quite verbatim translation of your pseudocode and your regexes, based on the assumption that »contains numbers« really means »starts with two digits« (which is what your regexes show):
#echo off
setlocal enabledelayedexpansion
set Prev=
for /f "delims=" %%x in (filter.tmp) do (
set "Line=%%x"
if "!Line:~0,2!" GEQ "00" if "!Line:~0,2!" LEQ "99" (
if not "!Prev!"=="" (>>filter.out echo !Prev!)
>>filter.out echo !Line!
set Prev=
) else (
if "!Prev!"=="" (set "Prev=!Line!") else (set "Prev=!Prev!/!Line!")
)
)
if not "!Prev!"=="" (>>filter.out echo !Prev!)
This uses several things. First of all, we need delayed expansion which enables us to manipulate environment variables within the loop. Then we iterate over the lines in the file with for /f. Note that this will skip empty lines in the file, but you cannot avoid that. Inside the for /f loop the variable Line holds the current line and Prev the previous one (if there has been a previous one). I swapped the then and else branches of the condition since numbers at the start of the line are easier to check for than non-numbers.
With the echo you'll notice that I moved the redirection to the start of the line; this is to prevent trailing numbers in Prev or Line from having an effect on the redirection (and also to avoid trailing spaces).
If you're not adverse to PowerShell, you can use the following:
$(switch -Regex -File filter.tmp {
'^\D' { if ($prev) { $prev += "/$_" } else { $prev = $_ } }
'^\d{2}' { if ($pref) {$prev}; $_; $prev = '' }
}
if ($prev) { $prev }
) | Set-Content filter.out