I've one file ABC_123.csv in my app directory and I want to display its full name. I found two ways to do it (see below code snippet): one using ??? and the other using asterisk at the end of required text ABC_.
But, both ways are also displaying the path along with the name. Both below commands are producing results in this format: path + name. I only need the name. Is there any special character (like ? or *) to display the name file only?
[input]$ ls /usr/opt/app/ABC_???.csv
[output] /usr/opt/app/ABC_123.csv
[input]$ ls /usr/opt/app/ABC_*.csv
[output] /usr/opt/app/ABC_123.csv
I cannot do this:
[input]$ cd /usr/opt/app
[input]$ ls ABC_???.csv
[output] ABC_123.csv
Required output:
[input]$ ls /usr/opt/app/ABC_(some-special-character).csv
[output] ABC_123.csv
[Edited] basename is working, but, I want to achieve this using ls and some special character (as highlighted above in Required output). Is there any way to this?
[input]$ basename /usr/opt/app/ABC_???.csv
[output] ABC_123.csv
You can use
find /usr/opt/app/ -type f -name "ABC_*.csv" -exec basename '{}' \;
basename will isolate the file name. find will search the specified directory for files that match the provided pattern.
If you were limited to ls, then this might help.
file="$(echo /usr/opt/app/ABC_???.csv)"; echo "${file##*/}"
Pipe every csv file to basename:
ls /usr/opt/app/ABC_*.csv | xargs basename -a
Without a need for basename:
(cd /usr/opt/app/ && ls ABC_*.csv)
"I cannot do this" was similar but didn't explain why, so maybe one-liner is doable. Doing in sub-shell prevents current dir from changing.
And no - there is no special character that could be used there. It's globbing: https://en.wikipedia.org/wiki/Glob_(programming)
Pure bash:
files=( /usr/opt/app/ABC_???.csv ) # Expands to array of matching filename(s)
printf "%s\n" "${file[#]##*/}"
The ## part of the expansion removes the longest leading string that matches the following pattern from each element of the array (So it works if this pattern matches only a single file like your question says, or if it matches more than one). */ will thus cut off everything up to and including the last slash in the string.
Related
Operating System - Linux (Ubuntu 20.04)
I have a directory with thousands of files in it. The file names range anything from a.daily.csv to a.b.daily.csv to a.b.c.daily.csv to a.b.c.d.daily.csv to a.b.c.d.e.daily.csv
The challenge I'm having is in listing just a.daily.csv or a.b.daily.csv and so on. That is to say with "daily.csv" as the fixed part, I would like to be able to wildcard what is in front of it with "." being the delimiter between the fields
I tried a few wildcards such as ? [a-zA-Z0-9] & so on but unable to achieve this. Please could I get some guidance
Please note a,b,c etc are placeholders I'm using to post the question. In real world, a,b,c are alphanumeric words
Example -
PAHKY.daily.csv
TYUI.GHJ.WE.daily.csv
WGGH.FGH.daily.csv
98KJL-GHR.YUI.daily.csv
67HJE.HJQ.ATD.HJ.daily.csv
If I want to list all those files that are like PAHKY.daily.csv where thre is only one filed (dot being the delimiter) in front of daily.csv, how could I do this?
If you enable the extglob option:
$ shopt -s extglob
you can use extended pattern matching operators like *(pattern) for zero or more of pattern. Knowing that [^.] matches any character but a dot, this leads to:
$ ls *([^.]).daily.csv
PAHKY.daily.csv
to obtain all a.daily.csv files. For the next group:
$ ls *([^.]).*([^.]).daily.csv
WGGH.FGH.daily.csv 98KJL-GHR.YUI.daily.csv
and so on. Replace *(pattern) by +(pattern) if you want to match one or more of pattern instead of zero or more.
You use grep with ls, as grep works well with regex
Try something like this,
^a\.b\.c\.data\.csv$
ls | grep 'Your Expression'
Fact, you can even use find without piping to grep
This should work:
ls |grep -Po '([A-Za-z0-9\-\.]?)+.daily.csv'
Explanation:
-P, --perl-regexp
-o, --only-matching
[A-Za-z0-9\-\.] --match the group of characters : (A-Z,a-z,0-9,-,.)
() -- to capture a group
? -- matches zero or one of the previous RE.
+ -- matches one or more of the previous RE
Output:
67HJE.HJQ.ATD.HJ.daily.csv
98KJL-GHR.YUI.daily.csv
PAHKY.daily.csv
TYUI.GHJ.WE.daily.csv
WGGH.FGH.daily.csv
I have a root directory that I need to run a find and/or grep command on to return a list of files that contain a specific string.
Here's an example of the file and directory set up. In reality, this root directory contains a lot of subdirectories that each have a lot of subdirectories and files, but this example, I hope, gets my point across.
From root, I need to go through each of the children directories, specifically into subdir/ and look through file.html for the string "example:". If a result is found, I'd like it to print out the full path to file.html, such as website_two/subdir/file.html.
I figured limiting the search to subdir/file.html will greatly increase the speed of this operation.
I'm not too knowledgeable with find and grep commands, but I have tried the following with no luck, but I honestly don't know how to troubleshoot it.
find . -name "file.html" -exec grep -HI "example:" {} \;
EDIT: I understand this may be marked as a duplicate, but I think my question is more along the lines of how can I tell the command to only search a specific file in a specific path, looping through all root-> level directories.
find ./ -type f -iname file.html -exec grep -l "example:" {} \+;
or
grep -Rl "example:" ./ | grep -iE "file.htm(l)*$" will do the trick.
Quote from GNU Grep 2.25 man page:
-R, --dereference-recursive
Read all files under each directory, recursively. Follow all symbolic links, unlike -r.
-l, --files-with-matches
Suppress normal output; instead print the name of each input file from which output would normally have
been printed. The scanning will stop on the first match.
-i, --ignore-case
Ignore case distinctions in both the PATTERN and the input files.
-E, --extended-regexp
Interpret PATTERN as an extended regular expression.
I know how I can rename files and such, but I'm having trouble with this.
I only need to rename test-this in a for loop.
test-this.ext
test-this.volume001+02.ext
test-this.volume002+04.ext
test-this.volume003+08.ext
test-this.volume004+16.ext
test-this.volume005+32.ext
test-this.volume006+64.ext
test-this.volume007+78.ext
If you have all of these files in one folder and you're on Linux you can use:
rename 's/test-this/REPLACESTRING/g' *
The result will be:
REPLACESTRING.ext
REPLACESTRING.volume001+02.ext
REPLACESTRING.volume002+04.ext
...
rename can take a command as the first argument. The command here consists of four parts:
s: flag to substitute a string with another string,
test-this: the string you want to replace,
REPLACESTRING: the string you want to replace the search string with, and
g: a flag indicating that all matches of the search string shall be replaced, i.e. if the filename is test-this-abc-test-this.ext the result will be REPLACESTRING-abc-REPLACESTRING.ext.
Refer to man sed for a detailed description of the flags.
Use rename as shown below:
rename test-this foo test-this*
This will replace test-this with foo in the file names.
If you don't have rename use a for loop as shown below:
for i in test-this*
do
mv "$i" "${i/test-this/foo}"
done
Function
I'm on OSX and my bash doesn't come with rename as a built-in function. I create a function in my .bash_profile that takes the first argument, which is a pattern in the file that should only match once, and doesn't care what comes after it, and replaces with the text of argument 2.
rename() {
for i in $1*
do
mv "$i" "${i/$1/$2}"
done
}
Input Files
test-this.ext
test-this.volume001+02.ext
test-this.volume002+04.ext
test-this.volume003+08.ext
test-this.volume004+16.ext
test-this.volume005+32.ext
test-this.volume006+64.ext
test-this.volume007+78.ext
Command
rename test-this hello-there
Output
hello-there.ext
hello-there.volume001+02.ext
hello-there.volume002+04.ext
hello-there.volume003+08.ext
hello-there.volume004+16.ext
hello-there.volume005+32.ext
hello-there.volume006+64.ext
hello-there.volume007+78.ext
Without using rename:
find -name test-this\*.ext | sed 'p;s/test-this/replace-that/' | xargs -d '\n' -n 2 mv
The way it works is as follows:
find will, well, find all files matching your criteria. If you pass -name a glob expression, don't forget to escape the *.
Pipe the newline-separated* list of filenames into sed, which will:
a. Print (p) one line.
b. Substitute (s///) test-this with replace-that and print the result.
c. Move on to the next line.
Pipe the newline-separated list of alternating old and new filenames to xargs, which will:
a. Treat newlines as delimiters (-d '\n').
b. Call mv repeatedly with up to 2 (-n 2) arguments each time.
For a dry run, try the following:
find -name test-this\*.ext | sed 'p;s/test-this/replace-that/' | xargs -d '\n' -n 2 echo mv
*: Keep in mind it won't work if your filenames include newlines.
to rename index.htm to index.html
rename [what you want to rename] [what you want it to be] [match on these files]
rename .htm .HTML *.htm
renames index.htm to index.html
It will do this for all files that match *.htm in the folder.
thx for your passion and answers. I also find a solution for me to rename multiple files on my linux terminal and directly add a little counter. With this I have a very good chance to have better SEO names.
Here is the command
count=1 ; zmv '(*).jpg' 'new-seo-name--$((count++)).jpg'
I also do a live coding video and publush it to YouTube
I want to search for folders by part of their name, which i know and it's common among these kind of folders. i used 'find' command in bash script like this
find . -type d -name "*.hg"
it just print out the whole path from current directory to the found folder itself. the foldr name has '.hg'.then i tried to use 'sed' command but i couldn't address the last part of the path. i decided to get the folder name ends in .hg save it in a variable then use 'sed' command to remove the last directory from output. i use this to get the last part, and try to save the result to a varable, no luck.
find . -type d -name "*.hg"|sed 's/*.hg$/ /'
find . -type d -name "*.hg"|awk -F/ '{print $NF}
this just print out the file names, here the folder with .hg at the end.
then i use different approach
for i in $(find . -type d -name '*.hg' );
do
$DIR = $(dirname ${i})
echo $DIR
done
this didin't work neither. can anyone point me any hint to make this works.
and yes it's homework.
You could use parameter expansion:
d=path/to/my/dir
d="${d#*/}" # remove the first dir
d="${d%/*}" # remove the last dir
echo $d # "to/my"
one problem that you have is with the pattern you are using in your sed script - there is a different pattern language used by both bash and the find command.
They use a very simple regular expression language where * means any number of any character and ? means any single character. The sed command uses a much richer regular expression language where * means any number of the previous character and . means any character (there's a lot more to it than that).
So to remove the last component of the path delivered by find you will need to use the following sed command: sed -e 's,/[^/].hg,,'
Alternatively you could use the dirname command. Pipe the output of the find command to xargs (which will run a command passing standard input as arguments to the command:
xargs -i dirname
#Pamador - that's strange. It works for me. Just to explain: the sed command needs to be quoted in single quotes just to protect against any unwanted shell expansions. The character following the 's' is a comma; what we're doing here is changing the character that sed uses to separate the two parts of the substitute command, this means that we can use the slash character without having to escape it without a preceding backslash. The next part matches any sequence of characters apart from a slash followed by any character and then hg. Honestly I should have anchored the pattern to the end of line with a $ but apart from that it's fine.
I tested it with
echo "./abc/xxx.hg" | sed -e 's,/[^/]\.hg$'
And it printed ./abc
Did I misunderstand what you wanted to do?
find . -type d -name "*.hg" | awk -v m=1 -v n=1 'NR<=m{};NR>n+m{print line[NR%n]};{line[NR%n]=$0}'
awk parameters:
m = number of lines to remove from beginning of output
n = number of
lines to remove from end of output
Bonus: If you wanted to remove 1 line from the end and you have coreutils installed, you could do this: find . -type d -name "*.hg" | ghead -n -1
I know you can do it with a find, but is there a way to send the output of ls to mv in the unix command line?
ls is a tool used to DISPLAY some statistics about filenames in a directory.
It is not a tool you should use to enumerate them and pass them to another tool for using it there. Parsing ls is almost always the wrong thing to do, and it is bugged in many ways.
For a detailed document on the badness of parsing ls, which you should really go read, check out: http://mywiki.wooledge.org/ParsingLs
Instead, you should use either globs or find, depending on what exactly you're trying to achieve:
mv * /foo
find . -exec mv {} /foo \;
The main source of badness of parsing ls is that ls dumps all filenames into a single string of output, and there is no way to tell the filenames apart from there. For all you know, the entire ls output could be one single filename!
The secondary source of badness of parsing ls comes from the broken way in which half the world uses bash. They think for magically does what they would like it to do when they do something like:
for file in `ls` # Never do this!
for file in $(ls) # Exactly the same thing.
for is a bash builtin that iterates over arguments. And $(ls) takes the output of ls and cuts it apart into arguments wherever there are spaces, newlines or tabs. Which basically means, you're iterating over words, not over filenames. Even worse, you're asking bask to take each of those mutilated filename words and then treat them as globs that may match filenames in the current directory. So if you have a filename which contains a word which happens to be a glob that matches other filenames in the current directory, that word will disappear and all those matching filenames will appear in its stead!
mv `ls` /foo # Exact same badness as the ''for'' thing.
One way is with backticks:
mv `ls *.boo` subdir
Edit: however, this is fragile and not recommended -- see #lhunath's asnwer for detailed explanations and recommendations.
None of the answers so far are safe for filenames with spaces in them. Try this:
for i in *; do mv "$i" some_dir/; done
You can of course use any glob pattern you like in place of *.
Not exactly sure what you're trying to achieve here, but here's one possibility:
The "xargs" part is the important piece everything else is just setup. The effect of this is to take everything that "ls" outputs and add a ".txt" extension to it.
$ mkdir xxx #
$ cd xxx
$ touch a b c x y z
$ ls
a b c x y z
$ ls | xargs -Ifile mv file file.txt
$ ls
a.txt b.txt c.txt x.txt y.txt z.txt
$
Something like this could also be achieved by:
$ touch a b c x y z
$ for i in `ls`;do mv $i ${i}.txt; done
$ ls
a.txt b.txt c.txt x.txt y.txt z.txt
$
I sort of like the second way better. I can NEVER remember how xargs works without reading the man page or going to my "cute tricks" file.
Hope this helps.
Check out find -exec {} as it might be a better option than ls but it depends on what you're trying to achieve.
/bin/ls | tr '\n' '\0' | xargs -0 -i% mv % /path/to/destdir/
"Useless use of ls", but should work. By specifying the full path to ls(1) you avoid clashes with aliasing of ls(1) mentioned in some of the previous posts. The tr(1) command together with "xargs -0" makes the command work with filenames containing (ugh) whitespace. It won't work with filenames containing newlines, but having filenames like that in the file system is to ask for trouble, so it probably won't be a big problem. But filenames with newlines could exist, so a better solution would be to use "find -print0":
find /path/to/srcdir -type f -print0 | xargs -0 -i% mv % dest/
You shouldn't use the output of ls as the input of another command. Files with spaces in their names are difficult as is the inclusion of ANSI escape sequences if you have:
alias ls-'ls --color=always'
for example.
Always use find or xargs (with -0) or globbing.
Also, you didn't say whether you want to move files or rename them. Each would be handled differently.
edit: added -0 to xargs (thanks for the reminder)
Backticks work well, as others have suggested. See xargs, too. And for really complicated stuff, pipe it into sed, make the list of commands you want, then run it again with the output of sed piped into sh.
Here's an example with find, but it works fine with ls, too:
http://github.com/DonBranson/scripts/blob/f09d24629ab6eb3ce509d4d3078818430306b063/jarfinder.sh
#!/bin/bash
for i in $( ls * );
do
mv $1 /backup/$1
done
else, it's the find solution by sybreon, and as suggested NOT the green mv ls solution.
Just use find or your shells globing!
find . -depth=1 -exec mv {} /tmp/blah/ \;
..or..
mv * /tmp/blah/
You don't have to worry about colour in the ls output, or other piping strangeness - Linux allows basically any characters in the filename except a null byte.. For example:
$ touch "blah\new|
> "
$ ls | xargs file
blahnew|: cannot open `blahnew|' (No such file or directory)
..but find works perfectly:
$ find . -exec file {} \;
./blah\new|
: empty
So this answer doesn't send the output of ls to mv but as #lhunath explained using ls is almost always the wrong tool for the job. Use shell globs or a find command.
For more complicated cases (often in a script), using bash arrays to build up the argument list from shell globs or find commands can be very useful. One can create an array and push to it with the appropriate conditional logic. This also handles spaces in filenames properly.
For example:
myargs=()
# don't push if the glob does not match anything
shopt -s nullglob
myargs+=(myfiles*)
To push files matching a find to the array: https://stackoverflow.com/a/23357277/430128.
The last argument should be the target location:
myargs+=("Some target directory")
Use myargs in the invocation of a command like mv:
mv "${myargs[#]}"
Note the quoting of the array myargs to pass array elements with spaces correctly.
You surround the ls with back quotes and put it after the mv, so like this...
mv `ls` somewhere/
But keep in mind that if any of your file names have spaces in them it won't work very well.
Also it would be simpler to just do something like this: mv filepattern* somewhere/