On macbook I'm trying to compile all proto files in a folder that do not contain the word server. Zsh and bash seem to be working a bit different than on linux. My coworker could get it wo work by using
[^_server]
in the path on linux. But that does not seem to work on mac. I managed to get this
protoc --dart_out=grpc:proto/ ./api/(core|editor|google)/**/*(_server).proto --proto_path=./api
which compiles all the proto files containing _server in the filename in the folders core, editor and google. But I need it to be all files that don't contain _server in the name in the folders core, editor and google.
Try this:
setopt extendedglob
protoc --dart_out=grpc:proto/ \
./api/(core|editor|google)/**/(^*_server).proto \
--proto_path=./api
The demonstrations of the glob patterns below use this directory tree:
api/
├── core/
│ ├── core_notserver.proto
│ ├── core_server.proto
│ ├── core_server.txt
│ └── coresub/
│ ├── coresub_notserver.proto
│ └── coresub_server.proto
├── editor/
│ ├── ed_notserver.proto
│ └── ed_server.proto
├── google/
│ ├── ends with excluded letter e.proto
│ ├── ends with included letter a.proto
│ ├── gl_server.proto
│ └── gl_server.txt
└── other/
├── other_notserver.proto
└── other_server.proto
With _server in the filename:
> print -l ./api/(core|editor|google)/**/*_server.proto
./api/core/core_server.proto
./api/core/coresub/coresub_server.proto
./api/editor/ed_server.proto
./api/google/gl_server.proto
Without _server in the filename.
There are two ways to do a pattern negation with zsh - they both require enabling extended glob patterns, with setopt extendedglob. The option just needs to be set once, e.g. in ~/.zshrc:
> setopt extendedglob
> print -l ./api/(core|editor|google)/**/(*~*_server).proto
./api/core/core_notserver.proto
./api/core/coresub/coresub_notserver.proto
./api/editor/ed_notserver.proto
./api/google/ends with excluded letter e.proto
./api/google/ends with included letter a.proto
> print -l ./api/(core|editor|google)/**/(^*_server).proto
./api/core/core_notserver.proto
./api/core/coresub/coresub_notserver.proto
./api/editor/ed_notserver.proto
./api/google/ends with excluded letter e.proto
./api/google/ends with included letter a.proto
The pattern [^_server] will not give the results you're looking for. With square braces, the values are used as individual characters to include, or with ^ exclude, from the results. It is not treated as the sequence _server. Your coworker may have had a set of filenames that gave the appearance of working properly.
Here, any filename with a base part that ends with _, s, e, r, or v will be excluded from the glob results:
> print -l ./api/(core|editor|google)/**/*[^_server].proto
./api/google/ends with included letter a.proto
More info on globbing here.
Related
I have a problem which I think needs loops to solve.
I have fastq files that follow the naming conventions, all in a generic directory e.g 'allpools' :
ls allpools
2022_pool_1_Seq_GEX.fastq.gz
2022_pool_1_Seq_CMO.fastq.gz
2022_pool_2_Seq_GEX.fastq.gz
2022_pool_2_Seq_CMO.fastq.gz
I need one loop to make directories in 'allpools' for as many pool numbers as there are in the fastqs, where the resulting directories will be called :
pool1
pool2
and then another loop whereby the original fastqs are moved and placed in respective directories for gex and cmo. As an example :
allpools/pool1/pool1_GEX #containing all gex fastqs of pool_1
allpools/pool1/pool1_CMO #containing all cmo fastqs of pool_1
####
allpools/pool2/pool2_GEX #containing all gex fastqs of pool_2
allpools/pool2/pool2_CMO #containing all cmo fastqs of pool_2
Before:
$ tree allpools/
allpools/
├── 2022_pool_1_Seq_CMO.fastq.gz
├── 2022_pool_1_Seq_GEX.fastq.gz
├── 2022_pool_2_Seq_CMO.fastq.gz
└── 2022_pool_2_Seq_GEX.fastq.gz
Use sed to parse the filename and generate the directory name:
$ for f in allpools/*; do d=$(sed -E 's%.+(pool_[0-9]+).+(GEX|CMO).+%\1/\1_\2%' <<<$f); mkdir -p allpools/$d; mv -vi $f allpools/$d/; done
renamed 'allpools/2022_pool_1_Seq_CMO.fastq.gz' -> 'allpools/pool_1/pool_1_CMO/2022_pool_1_Seq_CMO.fastq.gz'
renamed 'allpools/2022_pool_1_Seq_GEX.fastq.gz' -> 'allpools/pool_1/pool_1_GEX/2022_pool_1_Seq_GEX.fastq.gz'
renamed 'allpools/2022_pool_2_Seq_CMO.fastq.gz' -> 'allpools/pool_2/pool_2_CMO/2022_pool_2_Seq_CMO.fastq.gz'
renamed 'allpools/2022_pool_2_Seq_GEX.fastq.gz' -> 'allpools/pool_2/pool_2_GEX/2022_pool_2_Seq_GEX.fastq.gz'
After:
$ tree allpools/
allpools/
├── pool_1
│ ├── pool_1_CMO
│ │ └── 2022_pool_1_Seq_CMO.fastq.gz
│ └── pool_1_GEX
│ └── 2022_pool_1_Seq_GEX.fastq.gz
└── pool_2
├── pool_2_CMO
│ └── 2022_pool_2_Seq_CMO.fastq.gz
└── pool_2_GEX
└── 2022_pool_2_Seq_GEX.fastq.gz
I'm using OHMYZSH but I'm thinking about building a minimal configuration to Zsh.
So here's the problem. Ohmyzsh has special behaviour for completion. For example: if I'm in my Home Directory and type cd mus and hit a TAB it will expand to cd Music, even I'm typing the sentence with small letters.
I've founded this:
Have zsh return case-insensitive auto-complete matches, but prefer exact matches
Which is my problem but on the contrary. I want to get case insensitive without install OHMYZSH.
Looking at the OHMYZSH structure, I've found this:
.
├── ./cache
├── ./CODE_OF_CONDUCT.md
├── ./CONTRIBUTING.md
├── ./custom
├── ./lib
├── ./LICENSE.txt
├── ./log
├── ./oh-my-zsh.sh
├── ./plugins
├── ./README.md
├── ./templates
├── ./themes
└── ./tools
8 directories, 5 files
Inside the lib folder, there are some config files.
lib
├── bzr.zsh
├── clipboard.zsh
├── cli.zsh
├── compfix.zsh
├── completion.zsh
├── correction.zsh
├── diagnostics.zsh
├── directories.zsh
├── functions.zsh
├── git.zsh
├── grep.zsh
├── history.zsh
├── key-bindings.zsh
├── misc.zsh
├── nvm.zsh
├── prompt_info_functions.zsh
├── spectrum.zsh
├── termsupport.zsh
└── theme-and-appearance.zsh
0 directories, 19 files
I've already tried to source the completion.zsh using Zinit(plugin manager for ZSH which enables load some OHMYZSH stuff) without success. I don't know what is the correct file for the behaviour I want.
This is my config:
#exports
export EDITOR=nvim
export VISUAL=code
export SUDO_EDITOR=nvim
# Theme
ZSH_THEME="spaceship"
#PLUGINS
#==============================================================
source "$HOME/.zinit/bin/zinit.zsh"
autoload -Uz _zinit
(( ${+_comps} )) && _comps[zinit]=_zinit
zinit light zdharma/fast-syntax-highlighting
zinit light zsh-users/zsh-autosuggestions
zinit light zsh-users/zsh-completions
zinit light agkozak/zsh-z
#THIS LINE IS MY ATTEMPT TO load OHMYZSH FILES
zinit snippet OMZL::completion.zsh
#==============================================================
### End of Zinit's installer chunk
What's the correct file to load? Or is there another way to get case insensitive completions?
After reading some comments I found a solution. Add these two lines to the Zsh config file:
zstyle ':completion:*' matcher-list '' 'm:{a-zA-Z}={A-Za-z}' 'r:|=*' 'l:|=* r:|=*'
autoload -Uz compinit && compinit
I have a directory containing many sub-directories, each with lots of files having different extensions.
Is it possible for me to filter the output of the tree command so that files having certain extensions like say .log and .msh or certain named directories are not in the output
Consider the example below
.
├── dir1
│ ├── bar.log
│ ├── blahblah
│ │ └── blah.txt
│ ├── hello.txt
│ └── test.msh
├── foo.log
├── out
├── test.py
└── test.txt
2 directories, 8 files
I would like to filter away certain directories (and obviously their contents) and files having certain extensions. For instance I would like to filter away the contents of directory blahblah and all files having extensions .msh and .log
So I would like the output to be
.
├── dir1
│ ├── hello.txt
├── out
├── test.py
└── test.txt
tree command has -I option that allows you to filter out both files and directories that match the given pattern.
tree -I 'blahblah|*.msh|*.log'
First of all, I've read numerous post on here regarding this topic, but none have helped so far, e.g.
Zipping without creating parent folder
Unix zipping sub directory not including parent directory
Here's what I want. I have the following folder structure:
- root
-- subroot
--- subsubroot
---- file1.foo
---- file2.foo
---- file3.foo
---- file4.foo
And the result of the zipped file should be, when unzipped
file1.foo
file2.foo
file3.foo
file4.foo
The crux is that the file*.foo are just packages, thus basically being folders itself, so I tried using the -j option in conjunction with my recursive option (-r), but then I just get a flat hierarchy of all the files.
My current shell command looks like this:
find root/subroot/subsubroot -name '*.foo' | grep -v somestring | xargs -I % zip -r foos.zip %
However, the result of that will be exactly like the original structure with all the parent folder.
I don't necessarily have to use the zip command, but the resulting file has to be a zip.
Your command is nearly correct, although you shouldn't need to use the -I option.
Directory Structure:
.
└── root
└── subroot
└── subsubroot
├── foo1
│ └── foo1.pkg
├── foo2
│ └── foo2.pkg
├── foo3
│ └── foo3.pkg
├── fooA.pkg
├── fooB.pkg
├── fooC.pkg
└── nope.pkg
Command:
find root/subroot/subsubroot -name '*.pkg' | grep -v "nope" | xargs zip -jr foo
Resulting foo.zip:
fooA.pkg
fooB.pkg
fooC.pkg
foo1.pkg
foo2.pkg
foo2.pkg
There is one caveat.
If you have this type of directory structure you'll get an error:
.
└── root
└── subroot
└── subsubroot
├── foo1
│ └── foo1.pkg
├── foo2
│ └── foo2.pkg
├── foo3
│ └── fooC.pkg
├── fooA.pkg
├── fooB.pkg
├── fooC.pkg
└── nope.pkg
Error:
zip warning: first full name: root/subroot/subsubroot/foo3/fooC.pkg
second full name: root/subroot/subsubroot/fooC.pkg
name in zip file repeated: fooC.pkg
this may be a result of using -j
You'll need to be mindful that files/packages in sub-directories do not have the same filenames otherwise you'll run into problems as shown by fooC.pkg. It's assumed this is not the case, since you are attempting to create a flat archive.
Between web server environments we have data asset folder structures similar to the below:
/Volumes/data-acct/assets/
├── audio
│ ├── section1
│ │ └── nested files and folders
│ └── section2
├── css
│ ├── section1
│ │ └── nested files and folders
│ └── section2
├── images
│ ├── section1
│ │ └── nested files and folders
│ └── section2
└── videos
├── section1
└── section2
I've been trying to find an include filter that will allow me to match a particular section folder for each of the areas, although I get no results when changing from --include="*/" (to include all folders and --include="*/section1/**/*.*"
rsync --dry-run --verbose --recursive --update --include="*/section1/**/*.*" --exclude="*" /Volumes/data-acct/assets/ /Volumes/data-live/assets/
Should I be running several commands for this? Or should I be making use of the --filters argument (which doesn't seem to be documented)
I'm sure there's a good solution using rsync only (since it is very powerful), yet when in doubt I like to fall back on well known tools like so:
cd /Volumes/data-acct/assets &&
find . | # get all files
grep /section1/ | # that contain section1
grep -v videos/ | # but without videos
rsync --dry-run --verbose --recursive --update --files-from=- \
/Volumes/data-acct/assets/ /Volumes/data-live/assets/
The --files-from option allows you to create your own custom list without using any rsync filters at all.