I just want to automate some process, and I need to be able to format the output from pass ls
Which is the listing command for https://www.passwordstore.org
Current output looks like so:
➜ ~ pass ls
Password Store
├── README.md
├── folder
│ └── subfolder
│ └── subfolder2
│ └── key1
│ └── key2
│ └── key3
├── anotherfolder
│ ├── subfolder
│ │ └── subfolder2
│ │ └── subfolder3
│ │ └── key1
│ │ └── key2
│ │ └── key3
I want the output to look like:
➜ ~ pass ls | some magic sed/grep/replace/etc
folder/subfolder/subfolder2/key1
folder/subfolder/subfolder2/key2
folder/subfolder/subfolder2/key3
anotherfolder/subfolder/subfolder2/subfolder3/key1
anotherfolder/subfolder/subfolder2/subfolder3/key2
anotherfolder/subfolder/subfolder2/subfolder3/key3
I am trying to use sed to do so, but couldn't replace the increasing spaces/tabs as the subfolder levels gets deeper (for example folder/subfolder/subfolder/subfolder/key)
Here is what I am trying so far:
pass ls | sed -e 's/├──[ \t]*/\\/g' | sed -e 's/│ └──[ \t]*/\\/g'
EDIT AFTER COMMENTS:
Seems like pass ls is just a simple tree command on the password store directory, so I can run find on my directory to achieve the above format I want
I will try that, and the scope of the question can be changed to:
What's the proper listing command that can produce the above format?
find . -type d -name '.git' -prune -o -name '*.gpg' -type f -print
Did the trick for me, it excludes all files in .git directory, and only prints files with .gpg extension (which are the actual keys)
Related
What is the easiest way to get a list of all hidden and non-hidden directories (only directories) in a single call?
Obviously I can do it by connecting 2 different commands with && like this:
ls -d ./.*/ && ls -d ./*/
but shouldn't there be a better way?
EDIT: I do not want the current directory to be included in the list.
Also, ls -d ./.*/ ./*/ is a better alternative to what I have up there.
In bash you don't need to call any expternal utility to list all directories (including hidden ones). Just use:
# make sure we match entries starting with dot and return empty when list is empty
shopt -s dotglob nullglob
# print all the directories including ones starting with dot
printf '%s\n' */
Simple find should do it:
find /some/path -mindepth 1 -maxdepth 1 -type d
-maxdepth 1 ensures you don't descend into subdirectories. However, if that's what you want, you can just remove it.
-mindepth 1 ensures find does not include the directory itself.
If you have the tree command, you can get a nice view of your directories:
tree -a -d -L 3
where:
a -> shows hidden and nonhidden
d -> only shows directories
L 3 -> limit depth (if you want infinite depth, simply remove this option)
Output example:
.
├── bin
├── Common
├── .git
│ ├── branches
│ ├── hooks
│ ├── info
│ ├── logs
│ │ └── refs
│ ├── objects
│ │ ├── 08
│ │ ├── 38
│ │ ├── 62
│ │ ├── 6f
│ │ ├── 8c
│ │ ├── 9e
│ │ ├── a0
│ │ ├── cb
│ │ ├── d9
│ │ ├── info
│ │ └── pack
│ └── refs
│ ├── heads
│ ├── remotes
│ └── tags
├── Setup
├── test
└── tools
27 directories
You also get the number of directories. If you do not want it, add the --noreport option.
It is also possible to exclude pattern etc... man tree is your friend there.
Another example: a flat list of directories, excluding obj* and refs
tree -a -d -L 3 -if -I "obj*|refs" --noreport
returns:
.
./bin
./Common
./.git
./.git/branches
./.git/hooks
./.git/info
./.git/logs
./Setup
./test
./tools
The directory structure is as follows.
├── input
│ ├── config.ac
│ ├── dir1
│ │ ├── image2.png
│ │ └── image3.jpg
│ ├── dir2
│ └── image1.png
├── main.sh
└── output
Essentially, I am trying to run the following ./main.sh input output which I want to produce the following:
├── input
│ ├── config.ac
│ ├── dir1
│ │ ├── image2.png
│ │ └── image3.jpg
│ ├── dir2
│ └── image1.png
├── main.sh
└── output
├── dir1
│ └── image2.png
├── dir2
└── image1.png
After trying several thing such as find -exec, I went through it step by step. I'm trying to copy the internal directory structure of input into output while at the same time only copying .png files and nothing else.
Here is what I have tried:
for DIR in $1/; do
mkdir $2/$DIR
cp $1/$DIR*.png $1/$DIR $2/$DIR
done
Here is my logic, the for loop will go through every directory structure in the source directory($1), it will then make the exact same directory in the destination directory($2). Now, it looks for any .png files in the current directory it is in and copies it to the exact same corresponding directory that was just created in the destination.
Note, I do plan on doing some conversions to the files later on in the for loop
This doesn't seem to work at all. I get the following errors:
cp: cannot stat 'input/input/*.png': No such file or directory
cp: cannot stat 'input/input/': No such file or directory
Change to the directory you are copying from
Then
tar cf - .| ( cd /other directory; tar xf -)
You'll be probably happy with rsync:
rsync -a --include '*/' --include '*.png' --exclude '*' input/ output
This should copy all directories AND png files. If you want to see what would be copied, add -nv options.
(See the notes on rsync versions here.)
My favorite for copying directory trees (using your input/output structure):
cd input
find . | cpio -pdm ../output
Adjust the 'find' options as required.
In Linux I have a folder with the following structure:
.
├── a
│ ├── 00000400000000030bfd.dat
│ ├── 10000400000000030bfd.dat
│ ├── 20000400000000030bfd.dat
│ ├── etc....
├── b
│ ├── 00000401000000030bfd.dat
│ ├── 10000401000000030bfd.dat
│ ├── 20000401000000030bfd.dat
│ ├── etc....
├── c
│ ├── 00000402000000030bfd.dat
│ ├── 10000402000000030bfd.dat
│ ├── 20000402000000030bfd.dat
│ ├── etc....
├── d
│ ├── etc....
├── e
│ ├── etc....
├── f
And so on until the "p" folder. I want to rename every .dat file in every directory to .html file with a bash script. How i can do it?
Use a loop.
for file in {a..p}/*.dat ; do
mv "$file" "${file%.dat}.html"
done
${file%.dat} removes .dat from the end of the value of the $file.
Here is a version that uses all traditional commands:
find log -name "*.dat" |
sed 's/.log$//;s/mv &.dat &.html/' |
bash
Essentially, the find creates the target name list, the sed makes the names generic and then generates a mv command that does the rename and then pipes the results to bash for execution.
The bash command can be omitted to merely get a list of the mv commands for eyeball testing. You can also add a -x to the bash command to get a log of each mv command executed.
I have a large number of files in sub-folders that I need to rename. For example, I have:
ParentFolder/sub-Folders/*.jpg
How can I copy the files with a new naming convention as follows?
ParentFolder1.jpg
ParentFolder2.jpg
One way to do this is via GNU parallel. Tutorial here:
find ./ParentFolder -name "*.jpg" | parallel "mv {} DESTINATION/ParentFolder{#}.jpg"
To view the commands to be run before executing them, try:
find ./ParentFolder -name "*.jpg" | parallel --dryrun "mv {} DESTINATION/ParentFolder{#}.jpg"
Use rename over the full path using file globbing :
*/*
If you don't understand, you can test it with :
echo */*
First * is your directory, second * is your file name. Catch them in a regular expression :
(.*)/(.*)
Now $1 is your parent folder name and $2 is your file name. You can easily build your solution like this :
rename -n "s/(.*)\/(.*)/\$1\/\$1\$2/" */*
It keeps directory structure and adds directory name as a prefix to each of its files. You could move your files up by simply changing \$1\/\$1\$2 to \$1\$2. You then just have to delete empty directories using rmdir.
I voluntarily added the option -n so you don't do a mess if you copy-paste. Simply remove the option when you think it's good.
adrien#adrienLT:~/Documents/PEV$ cp -r Holiday/ Holiday_copy/
adrien#adrienLT:~/Documents/PEV$ tree Holiday*
Holiday
├── France
│ ├── 1.jpg
│ ├── 2.jpg
│ └── 3.jpg
├── Italy1
│ ├── 1.jpg
│ ├── 2.jpg
│ └── 3.jpg
└── Italy2
├── 1.jpg
├── 2.jpg
└── 3.jpg
Holiday_copy
├── France
│ ├── 1.jpg
│ ├── 2.jpg
│ └── 3.jpg
├── Italy1
│ ├── 1.jpg
│ ├── 2.jpg
│ └── 3.jpg
└── Italy2
├── 1.jpg
├── 2.jpg
└── 3.jpg
6 directories, 18 files
adrien#adrienLT:~/Documents/PEV$ cd Holiday_copy/
adrien#adrienLT:~/Documents/PEV/Holiday_copy$ rename "s/(.*)\/(.*)/\$1\/\$1\$2/" */*
adrien#adrienLT:~/Documents/PEV/Holiday_copy$ tree .
.
├── France
│ ├── France1.jpg
│ ├── France2.jpg
│ └── France3.jpg
├── Italy1
│ ├── Italy11.jpg
│ ├── Italy12.jpg
│ └── Italy13.jpg
└── Italy2
├── Italy21.jpg
├── Italy22.jpg
└── Italy23.jpg
3 directories, 9 files
Much research has turned almost similar questions yet nothing close enough to give me an idea of how to accomplish part my task. I'll try to keep this clear and short, while explaining the situation and desired result. My structure would be as follows:
-mobile
--Docs
--Downloads
--SomeFile
----this.is.crazy_0.0.1-1_named-silly.txt
----dont.touch.me.pdf
----leave.me.alone.png
----this.is.crazy_0.0.1-2_named-silly.txt
----this.is.crazy_0.0.1-3_named-silly.txt <---- file to keep
--SomeFileA
----this.is.crazy_0.0.1-1_also-silly.txt
----this.is.crazy_0.0.1-2_also-silly.txt
----dont.touch.me.either.pdf
----leave.me.alone.too.png
----this.is.crazy_0.0.1-3_also-silly.txt
----this.is.crazy_0.0.1-11_also-silly.txt <----file to keep
The first part of my script to find the .txt files ignores every directory that is constant in this working directory and prints them to a list (which is a completely ugly hack and most likely a hinder to the way most would accomplish this task) "SomeFileB and SomeFileC" could come along with the same file structure and I'd like to catch them in this script as well.
The idea is to keep the newest .txt file in each directory according to its time stamp which obviously isn't in the filename. The files to keep will continue to change of course. To clarify the question again, how to go about keeping the newest .txt file in each variable directory with variable crazy name, according to timestamp which isn't in the filename? Hopefully I've been clear enough for help. This script should be in bash.
I'm not with the current code right now, as i said its ugly but heres a snippet of what I have find /path/to/working/directory -maxdepth 0 -not -path "*Docs*" -not -path "*Downloads* -name "*.txt" >list
Assuming the question was understood correctly, the task could be expressed as:
Recursively remove all files *.txt except the newest in each respective directory
#!/bin/bash
# Find all directories from top of tree
find a -type d | while read -r dir; do
# skip $dir if doesn't contain any files *.txt
ls "$dir"/*.txt &>/dev/null || continue
# list *.txt by timestamp, skipping the newest file
ls -t "$dir"/*.txt | awk 'NR>1' | while read -r file; do
rm "$file"
done
done
Assuming this directory tree, where a.txt is always the newest:
$ tree -t a
a
├── otherdir
├── b
│ ├── d e
│ │ ├── a.txt
│ │ ├── b.txt
│ │ ├── c.txt
│ │ ├── bar.txt
│ │ └── foo.pdf
│ ├── c
│ │ ├── a.txt
│ │ ├── b.txt
│ │ └── c.txt
│ ├── a.txt
│ ├── b.txt
│ ├── c.txt
│ └── foo.pdf
├── a.txt
├── b.txt
└── c.txt
This is the result after running the script:
$ tree -t a
a
├── b
│ ├── c
│ │ └── a.txt
│ ├── d e
│ │ ├── a.txt
│ │ └── foo.pdf
│ ├── a.txt
│ └── foo.pdf
├── otherdir
└── a.txt
Change rm "$file" to echo rm "$file" to check what would be removed before running "for real"