Extract a tar file with no directory information - bash

I have a folder I am turning into a tar file for transfer over the network it looks like this
/foo/bar/file1
/foo/file2
/foo/baz/bin/file3
I want to extract it with no directory information so the extracted contents look like this
file1
file2
file3
How does one accomplish this

The GNU tar manual elaborates on --transform:
tar --show-transformed --transform 's,.*/,,' tfv myarchive.tar
tar --transform 's,.*/,,' xf myarchive.tar

What about the Unix philosophy of stringing together tools?
Use tar to extract as usual, then
find foo -type f | xargs mv '{}' .
rm -rf foo # optional, nuke the empty dirs.
or variations thereof with find ... -print0 and xargs -0.

Related

How to tar compress specific and multiple files without changing of name in Bash?

I need do some simple compresion multiple files to .tar in bash.
Conditions:
My archive must include only files with extension .exe.
Unfortunetly, when I am trying this:
find ./myDir -name "*.exe" | tar -cf archive -T -
The file names are changed like ./file1.exe How can I compress this without change of file names?
Suggesting:
find "$PWD/myDir" -name "*.exe" | tar -cf archive -T -
This will not do anything interesting.
Better suggesting to feed tar command with results from find command
tar czf your_compressed_file.tar.gz $(find ./myDir -name "*.exe")
Notice tar cf is not compressed. But tar czf is compressed.
You can also add -v option to see what are the compressed files:
tar czfv your_compressed_file.tar.gz $(find ./myDir -name "*.exe")

How to untar all tar files in current directory using Putty

How can I untar all tar files in one command using Putty.
I Tried the following but its not un-tarring (all files start with alcatelS*)
tar -xfv alcatelS*.tar
It is not working i don't get no errors and it is not un-tarring.
Thank you,
-xfv is wrong since v is being referred as the file instead. Also, tar can't accept multiple files to extract at once. Perhaps -M can be used but it's a little stubborn when I tried it. Also, it would be difficult to pass multiple arguments that were extracted from pathname expansion i.e. you have to do tar -xvM -f file1.tar -f file2.tar.
Do this instead:
for F in alcatelS*.tar; do
tar -xvf "$F"
done
Or one-line: (EDIT: Sorry that -is- a "one"-liner but I find that not technically a real one-liner, just a condensed one so I should haven't referred to that as a one-liner. Avoid the wrong convention.)
for F in alcatelS*.tar; do tar -xvf "$F"; done
You can use following command for extract all tar.gz files in directory in unix
find . -name 'alcatelS*.tar.gz' -exec tar -xvf {} \;
Following is my favorite way to untar multiple tar files:
ls *tar.gz | xargs -n1 tar xvf
Can be done in one line:
cat *.tar | tar -xvf - -i

How to restrict pattern matching to the first matching file

I have a very delicate bash problem:
given a directory foo
$ ls
foo.tgz bar.tgz baz.tgz
I want to generate a bash single liner, that extracts the first tarball of a pattern like:
bash -c "tar -zxvf foo.tgz file1" # fine
bash -c 'tar -zxvf *.tgz file1" # oups trying to extract bar.tgz from foo.tgz!
Is there a possibility to restrict pattern matching to the first expanded parameter?
Refinement:
find -iname '*.tgz' | xargs tar -zxvf # oups! cannot add restriction to only extract file1
Use the -quit primary with find:
find -iname '*.tgz' -exec tar -zxvf '{}' \; -quit
As the actions are processed from left-to-right, this will run tar on the first match, then end the find command.
Sure, try this:
tar -zxvf $(ls *.tgz | head -1) file1
You should consider what happens if nothing matches the pattern....

Making archive from files with same names in different directories

I have some files with same names but under different directories. For example, path1/filea, path1/fileb, path2/filea, path2/fileb,....
What is the best way to make the files into an archive? Under these directories, there are many other files under these directories that I don't want to make into the archive. Off the top of my head, I think of using Bash, probably ar, tar and other commands, but am not sure how exactly to do it.
Renaming the files seems to make the file names a little complicated. I tend to keep the directory structure inside the archive. Or I might be wrong. Other ideas are welcome!
Thanks and regards!
EDIT:
Examples would be really nice!
you can use tar with --exclude PATTERN option. See the man page for more.
To exclude files, you can see this page for examples.
You may give the find command multiple directories to search through.
# example: create archive of .tex files
find -x LaTeX-files1 LaTeX-files2 -name "*.tex" -print0 | tar --null --no-recursion -uf LaTeXfiles.tar --files-from -
To recursively copy only files with filename "filea" or "fileb" from /path/to/source to /path/to/archive, you could use:
rsync -avm --include='file[ab]' -f 'hide,! */' /path/to/source/ /path/to/archive/
'*/' is a pattern which matches 'any directory'
'! */' matches anything which is not a directory (i.e. a file)
'hide,! */' means hide all files
Filter rules are applied in order, and the first rule that matches is applied.
--include='file[ab]' has precedence, so if a file matches 'file[ab]', it is included.
Any other file gets excluded from the list of files to transfer.
Another alternative is to use the find...exec pattern:
mkdir /path/to/archive
cd /path/to/source
find . -type f -iname "file[ab]" -exec cp --parents '{}' /path/to/archive ";"
What I have used to make a tar ball for the files with same name in different directories is
$find <path> -name <filename> -exec tar -rvf data.tar '{}' \;
i.e. tar [-]r --append
Hope this helps.

How do I rename the extension for a bunch of files?

In a directory, I have a bunch of *.html files. I'd like to rename them all to *.txt
How can I do that? I use the bash shell.
If using bash, there's no need for external commands like sed, basename, rename, expr, etc.
for file in *.html
do
mv "$file" "${file%.html}.txt"
done
For an better solution (with only bash functionality, as opposed to external calls), see one of the other answers.
The following would do and does not require the system to have the rename program (although you would most often have this on a system):
for file in *.html; do
mv "$file" "$(basename "$file" .html).txt"
done
EDIT: As pointed out in the comments, this does not work for filenames with spaces in them without proper quoting (now added above). When working purely on your own files that you know do not have spaces in the filenames this will work but whenever you write something that may be reused at a later time, do not skip proper quoting.
rename 's/\.html$/\.txt/' *.html
does exactly what you want.
This worked for me on OSX from .txt to .txt_bak
find . -name '*.txt' -exec sh -c 'mv "$0" "${0%.txt}.txt_bak"' {} \;
You want to use rename :
rename -S <old_extension> <new_extension> <files>
rename -S .html .txt *.html
This does exactly what you want - it will change the extension from .html to .txt for all files matching *.html.
Note: Greg Hewgill correctly points out this is not a bash builtin; and is a separate Linux command. If you just need something on Linux this should work fine; if you need something more cross-platform then take a look at one of the other answers.
On a Mac...
Install rename if you haven't: brew install rename
rename -S .html .txt *.html
For Ubuntu Users :
rename 's/\.html$/\.txt/' *.html
This is the slickest solution I've found that works on OSX and Linux, and it works nicely with git too!
find . -name "*.js" -exec bash -c 'mv "$1" "${1%.js}".tsx' - '{}' \;
and with git:
find . -name "*.js" -exec bash -c 'git mv "$1" "${1%.js}".tsx' - '{}' \;
This question explicitly mentions Bash, but if you happen to have ZSH available it is pretty simple:
zmv '(*).*' '$1.txt'
If you get zsh: command not found: zmv then simply run:
autoload -U zmv
And then try again.
Thanks to this original article for the tip about zmv.
Here is an example of the rename command:
rename -n ’s/\.htm$/\.html/’ *.htm
The -n means that it's a test run and will not actually change any files. It will show you a list of files that would be renamed if you removed the -n. In the case above, it will convert all files in the current directory from a file extension of .htm to .html.
If the output of the above test run looked ok then you could run the final version:
rename -v ’s/\.htm$/\.html/’ *.htm
The -v is optional, but it's a good idea to include it because it is the only record you will have of changes that were made by the rename command as shown in the sample output below:
$ rename -v 's/\.htm$/\.html/' *.htm
3.htm renamed as 3.html
4.htm renamed as 4.html
5.htm renamed as 5.html
The tricky part in the middle is a Perl substitution with regular expressions, highlighted below:
rename -v ’s/\.htm$/\.html/’ *.htm
One line, no loops:
ls -1 | xargs -L 1 -I {} bash -c 'mv $1 "${1%.*}.txt"' _ {}
Example:
$ ls
60acbc4d-3a75-4090-85ad-b7d027df8145.json ac8453e2-0d82-4d43-b80e-205edb754700.json
$ ls -1 | xargs -L 1 -I {} bash -c 'mv $1 "${1%.*}.txt"' _ {}
$ ls
60acbc4d-3a75-4090-85ad-b7d027df8145.txt ac8453e2-0d82-4d43-b80e-205edb754700.txt
The command mmv seems to do this task very efficiently on a huge number of files (tens of thousands in a second). For example, to rename all .xml files to .html files, use this:
mmv ";*.xml" "#1#2.html"
the ; will match the path, the * will match the filename, and these are referred to as #1 and #2 in the replacement name.
Answers based on exec or pipes were either too slow or failed on a very large number of files.
In Linux or window git bash or window's wsl, try below command to change every file's extension in current directory or sub-directories or even their sub-directories with just one line of code
find . -depth -name "*.html" -exec sh -c 'mv "$1" "${1%.html}.txt"' _ {} \;
Try this
rename .html .txt *.html
usage:
rename [find] [replace_with] [criteria]
After someone else's website crawl, I ended up with thousands of files missing the .html extension, across a wide tree of subdirectories.
To rename them all in one shot, except the files already having a .html extension (most of them had none at all), this worked for me:
cd wwwroot
find . -xtype f \! -iname *.html -exec mv -iv "{}" "{}.html" \; # batch rename files to append .html suffix IF MISSING
In the OP's case I might modify that slightly, to only rename *.txt files, like so:
find . -xtype f -iname *.txt -exec filename="{}" mv -iv ${filename%.*}.{txt,html} \;
Broken down (hammertime!):
-iname *.txt
- Means consider ONLY files already ending in .txt
mv -iv "{}.{txt,html}"
- When find passes a {} as the filename, ${filename%.*} extracts its basename without any extension to form the parameters to mv. bash takes the {txt,html} to rewrite it as two parameters so the final command runs as: mv -iv "filename.txt" "filename.html"
Fix needed though: dealing with spaces in filenames
This is a good way to modify multiple extensions at once:
for fname in *.{mp4,avi}
do
mv -v "$fname" "${fname%.???}.mkv"
done
Note: be careful at the extension size to be the same (the ???)
Rename file extensions for all files under current directory and sub directories without any other packages (only use shell script):
Create a shell script rename.sh under current directory with the following code:
#!/bin/bash
for file in $(find . -name "*$1"); do
mv "$file" "${file%$1}$2"
done
Run it by ./rename.sh .old .new.
Eg. ./rename.sh .html .txt
A bit late to the party. You could do it with xargs:
ls *.html | xargs -I {} sh -c 'mv $1 `basename $1 .html`.txt' - {}
Or if all your files are in some folder
ls folder/*.html | xargs -I {} sh -c 'mv $1 folder/`basename $1 .html`.txt' - {}
Similarly to what was suggested before, this is how I did it:
find . -name '*OldText*' -exec sh -c 'mv "$0" "${0/OldText/NewText}"' {} \;
I first validated with
find . -name '*OldText*' -exec sh -c 'echo mv "$0" "${0/OldText/NewText}"' {} \;
Nice & simple!
find . -iname *.html -exec mv {} "$(basename {} .html).text" \;
If you prefer PERL, there is a short PERL script (originally written by Larry Wall, the creator of PERL) that will do exactly what you want here:
tips.webdesign10.com/files/rename.pl.txt.
For your example the following should do the trick:
rename.pl 's/html/txt/' *.html
The easiest way is to use rename.ul it is present in most of the Linux distro
rename.ul -o -v [oldFileExtension] [newFileExtension] [expression to search for file to be applied with]
rename.ul -o -v .oldext .newext *.oldext
Options:
-o: don't overwrite preexisting .newext
-v: verbose
-n: dry run
Unfortunately it's not trivial to do portably. You probably need a bit of expr magic.
for file in *.html; do echo mv -- "$file" "$(expr "$file" : '\(.*\)\.html').txt"; done
Remove the echo once you're happy it does what you want.
Edit: basename is probably a little more readable for this particular case, although expr is more flexible in general.
Here is what i used to rename .edge files to .blade.php
for file in *.edge; do mv "$file" "$(basename "$file" .edge).blade.php"; done
Works like charm.
You can also make a function in Bash, add it to .bashrc or something and then use it wherever you want.
change-ext() {
for file in *.$1; do mv "$file" "$(basename "$file" .$1).$2"; done
}
Usage:
change-ext css scss
Source of code in function: https://stackoverflow.com/a/1224786/6732111
Here is a solution, using AWK. Make sure the files are present in the working directory. Else, cd to the directory where the html files are located and then execute the below command:
for i in $(ls | grep .html); do j=$(echo $i | grep -oh "^\w*." | awk '{print $1"txt"}'); mv $i $j; done
I wrote this code in my .bashrc
alias find-ext='read -p "Path (dot for current): " p_path; read -p "Ext (unpunctured): " p_ext1; find $p_path -type f -name "*."$p_ext1'
alias rename-ext='read -p "Path (dot for current): " p_path; read -p "Ext (unpunctured): " p_ext1; read -p "Change by ext. (unpunctured): " p_ext2; echo -en "\nFound files:\n"; find $p_path -type f -name "*.$p_ext1"; find $p_path -type f -name "*.$p_ext1" -exec sh -c '\''mv "$1" "${1%.'\''$p_ext1'\''}.'\''$p_ext2'\''" '\'' _ {} \;; echo -en "\nChanged Files:\n"; find $p_path -type f -name "*.$p_ext2";'
In a folder like "/home/<user>/example-files" having this structure:
/home/<user>/example-files:
file1.txt
file2.txt
file3.pdf
file4.csv
The commands would behave like this:
~$ find-text
Path (dot for current): example-files/
Ext (unpunctured): txt
example-files/file1.txt
example-files/file2.txt
~$ rename-text
Path (dot for current): ./example-files
Ext (unpunctured): txt
Change by ext. (unpunctured): mp3
Found files:
./example-files/file1.txt
./example-files/file1.txt
Changed Files:
./example-files/file1.mp3
./example-files/file1.mp3
~$
You could use a tool designed for renaming files in bulk, e.g. renamer.
To rename all file extensions in the current folder:
$ renamer --find ".html" --replace ".txt" --dry-run *
Many more usage examples here.

Resources