WSL win bash For command replace substring (former extention) - bash

I am using a program that creates one image type from another using a command line that calls it. I do like that:
for FILE in *.bmp; do (my-path-to-app).exe (program tasks) "$FILE" "$FILE.png"
The result is Myimage.bmp.png
Obviously I want the end result to be from Myimage.bmp to Myimage.png and not myimage.bmp.png. So how do I remove the .bmp before? Do I need to specify some trimming before the DO or after? I cant get this right

This can be done with :
for FILE in *.bmp; do echo "$FILE" "${FILE%bmp}png"; done
${FILE%bmp} removes bmp from the end of "$FILE"

Related

Use inotifywait to change filename and further loop through sql loader

Objective: The moment multiple.csv files are uploaded to the folder, code should check each filename, if appropriate filename, file should be further used by sqlloader to get data uploaded in the database. Once file is uploaded, code should delete the file processed. Next time, same process repeats.
I have some parts of the code working but some are creating problem, especially related to inotifywait. Please help.
In first loop, I am trying to monitor the /uploads folder, the moment it finds the .csv file, it checks if the filename has space. If yes, it wants to change the space to underscore in the filename. I have been trying to find a way to find "space, () or ," in the filename but only could do the 'space' part change. This is giving me an error that file cannot be moved, no such file or directory.
Second loop works separately but not when incorporated with first loop as there are errors which I have not been able to debug. If I run second loop separately, it is working correctly. But if there is a way to optimize the code better in one loop, I would be happy to know. Thanks!
Example: folder name: /../../upload
filenames: abc_123.csv (code should not make any change) , pqr(12 Apr).csv (code should change it to pqr_12_Apr.csv), May 12.csv (code should change it to May_12.csv) etc.
Once these 3 files have proper naming, it should be ready to be uploaded through sql loader and once files are processed, they get deleted.
My code is:
#!bin/bash
inotifywait -mqe create /../../upload | while read file; do
if [[ $file = '* *'.csv]]; then
mv "$file" ${file// /_}
fi
done
for file in /../..upload/*.csv
do
sqlcommand="sqlldr user/pwd control="/../xxx.ctl" data=$file silent=feedback, header"
$sqlcommand
rm $file
done
Thank you!
I have modified your script to this,
#!/usr/bin/env bash
while IFS= read -r file; do
filename=${file#* CREATE }
pathname=${file%/*}
if [[ $pathname/$filename = *\ *.csv ]]; then
echo mv -v "$pathname/$filename" "$pathname/${filename// /_}"
fi
done < <(inotifywait -mqe create /../../upload)
Remove the echo if you think the output is correct.
I just don't know how you can integrate the other parts of your script with that, probably create a separate script or remove the -m (which you don't want to do most probably). Well you could use a named pipe if mkfifo is available.
EDIT: as per OP's message add another parameter expansion for another string removal.
Add the code below the if [[ ... ]]; then
newfilename=${filename//\(\)}
Then change "${filename// /_}" to "${newfilename// /_}"

Adding extension to file using file command in linux

I have a file that does not have an extension and would like to add an extension to it programmatically. I know the file command gives information about the extension of a file. How can I utilize this to add the extension to a file? The files I'm downloading can be assumed to be image files (png, jpg, etc.)
My desired outcome would be:
Input: filename
Output: filename.ext
All inside a bash script
Something like this should get you started:
#!/bin/bash
for f in "$#"; do
if [[ $f == *'.'* ]]; then continue; fi # Naive check to make sure we don't add duplicate extensions
ext=''
case $(file -b "$f") in
*ASCII*) ext='.txt' ;;
*JPEG*) ext='.jpg' ;;
*PDF*) ext='.pdf' ;;
# etc...
*) continue ;;
esac
mv "${f}" "${f}${ext}"
done
You'll have to check the output of file for each potential file type to find an appropriate case label.
You can try to find or create a map of file-type to file extension name but there's no universal way. Think about JPEG images, you can either have .jpg or .jpeg extension, and they both mean the same thing. Same for MP4 video containers...
Also, on linux the file extension doesn't even matter to most programs so you could just not care about it, but if you still want to do it for certain types of files, you can check this answer : https://stackoverflow.com/a/6115923/9759362

Loop through a folder of PDF files and append a single PDF to each

This code just seems to replace the first file, not append file1.pdf to it.
I need the file to append not replace.
#!/bin/bash
FILES=("/Users/a/folder/"*.pdf)
for f in "${FILES[#]}"
do
echo "${f}"
"/System/Library/Automator/Combine PDF Pages.action/Contents/Resources/join.py" -o "${f}" "${f}" "/Users/a/folder2/file1.pdf"
done
I noticed, if I run the code manually, but use a different name for the first and second parameters, it seems to work. However, I do not know how to change the name of the first parameter without making it a constant.
It seems to me your problem has nothing to do with Ruby. As I'm understanding it, you are trying to use the command line on MacOS X El Capitan to merge a PDF file with other PDF files.
If I understood your problem correctly, then you probably should heed the advice of this weblog and use the command "/System/Library/Automator/Combine PDF Pages.action/Contents/Resources/join.py" which is available from MacOS X Tiger onwards.
Note that if the file you want to append is in the same directory where all the files are you want to append to, you'll run into problems: the script join.py does not seem to appreciate being given the same file thrice, so place your file elsewhere (the one you want to append to all files).
Try something along the lines of:
#!/bin/bash
for f in /Absolute/Path/To/The/PDFS/*.pdf;
do /System/Library/Automator/Combine\ PDF\ Pages.action/Contents/Resources/join.py -o $f $f /Absolute/Path/To/The/File/To/Append; done
Solution:
#!/bin/bash
FILES=("/Users/a/folder/"*.pdf)
for f in "${FILES[#]}"
do
echo "${f}"
a="${f%.pdf}"
"/System/Library/Automator/Combine PDF Pages.action/Contents/Resources/join.py" -o "${a}_x.pdf" "${f}" "/Users/a/folder2/file1.pdf"
done

automatically renaming files

I have a bunch of files (more than 1000) on this like the followings
$ ls
org.allenai.ari.solvers.termselector.BaselineLearnersurfaceForm-dev.lc
org.allenai.ari.solvers.termselector.BaselineLearnersurfaceForm-dev.lex
org.allenai.ari.solvers.termselector.BaselineLearnersurfaceForm-train.lc
org.allenai.ari.solvers.termselector.BaselineLearnersurfaceForm-train.lex
org.allenai.ari.solvers.termselector.BaselineLearnersurfaceForm.lc
org.allenai.ari.solvers.termselector.BaselineLearnersurfaceForm.lex
org.allenai.ari.solvers.termselector.ExpandedLearner.lc
org.allenai.ari.solvers.termselector.ExpandedLearner.lex
org.allenai.ari.solvers.termselector.ExpandedLearnerSVM.lc
org.allenai.ari.solvers.termselector.ExpandedLearnerSVM.lex
....
I have to rename these files files by adding a learners right before the capitalized name. For example
org.allenai.ari.solvers.termselector.BaselineLearnersurfaceForm.lex
would change to
org.allenai.ari.solvers.termselector.learners.BaselineLearnersurfaceForm.lex
and this one
org.allenai.ari.solvers.termselector.ExpandedLearner.lc
would change to
org.allenai.ari.solvers.termselector.learners.ExpandedLearner.lc
Any ideas how to do this automatically?
for f in org.*; do
echo mv "$f" "$( sed 's/\.\([A-Z]\)/.learner.\1/' <<< "$f" )"
done
This short loop outputs an mv command that renames the files in the manner that you wanted. Run it as-is first, and when you are certain it's doing what you want, remove the echo and run again.
The sed bit in the middle takes a filename ($f, via a here-string, so this requires bash) and replaces the first occurrence of a capital letter after a dot with .learner. followed by that same capital letter.
There is a tool called perl-rename, sometimes rename. Not to be confused with rename from util-linux.
It's very good for tasks like this as it takes a perl expression and renames accordingly:
perl-rename 's/(?=\.[A-Z])/.learners/' *
You can play with the regex online
Alternative you can a for loop and $BASH_REMATCH:
for file in *; do
[ -e "$file" ] || continue
[[ "$file" =~ ^([^A-Z]*)(.*)$ ]]
mv -- "$file" "${BASH_REMATCH[1]}learners.${BASH_REMATCH[2]}"
done
A very simple approach (useful if you only need to do this one time) is to ls >dummy them into a text file dummy, and then use find/replace in a text editor to make lines of the form mv xxx.yyy xxx.learners.yyy. Then you can simple execute the resulting file with ./dummy.
The exact find/replace commands depend on the text editor you use, but something like
replace org. with mv org.. That gets you the mv in the beginning.
replace mv org.allenai.ari.solvers.termselector.$1 with mv org.allenai.ari.solvers.termselector.$1 org.allenai.ari.solvers.termselector.learner.$1 to duplicate the filename and insert the learner.
There is also syntax with a for, which can do it probably in one line, (long) but I cannot explain it - try help for if you want to learn about it.

how to change the extension of multiple files using bash script?

I am very new with linux usage maybe this is my first time so i hope some detailed help please.
I have more than 500 files in multiple directories on my server (Linux) I want to change their extensions to .xml using bash script
I used a lot of codes but none of them work some codes i used :
for file in *.txt
do
mv ${file} ${file/.txt}/.xml
done
or
for file in *.*
do
mv ${file} ${file/.*}/.xml
done
i do not know even if the second one is valid code or not i tried to change the txt extension beacuse the prompt said no such file '.txt'
I hope some good help for that thank you
Explanation
For recursivity you need Bash >=4 and to enable ** (i.e. globstar) ;
First, I use parameter expansion to remove the string .txt, which must be anchored at the end of the filename (%) :
the # anchors the pattern (plain word or glob) to the beginning,
and the % anchors it to the end.
Then I append the new extension .xml
Be extra cautious with filename, you should always quote parameters expansion.
Code
This should do it in Bash (note that I only echothe old/new filename, to actually rename the files, use mv instead of echo) :
shopt -s globstar # enable ** globstar/recursivity
for i in **/*.txt; do
[[ -d "$i" ]] && continue; # skip directories
echo "$i" "${i/%.txt}.xml";
done
If its a matter of a one or two sub-directories, you can use the rename command:
rename .txt .xml *.txt
This will rename all the .txt to .xml files in the directory from which the command is executed.
If all the files are in same directory, it can be done using a single command. For example you want to convert all jpg files to png, go to the related directory location and then use command
rename .jpg .png *
I wanted to rename "file.txt" to "file.jpg.txt", used rename easy peezy:
rename 's/.txt$/.jpg.txt/' *.txt
man rename will tell you everything you need to know.
Got to love Linux, there's a tool for everything :-)
passing command line argument for dir path
#!/bin/sh
cd $1
names_1=`ls`
for file in ${names_1}
do
mv ${file} ${file}.jpg
done

Resources