fish shell how to tokenize variable to a list - shell

It's weird that others are complaining that fish is always splitting their variables to lists. But to me it's just having the multiline variable as a single string.
I'm trying to write a nautilus script. The nautilus should set a variable called $NAUTILUS_SCRIPT_SELECTED_FILE_PATHS with the selected files separated with newlines.
I'm trying to get them as a list to loop over them with fish. But they just behave as a single element.
set -l files $NAUTILUS_SCRIPT_SELECTED_FILE_PATHS
for i in (seq (count $files))
echo (count $files) >> koko
end
and the file koko now shows the number 1.

Fish does not split variables after they have been set (this is known as "word splitting").
What it does, however, do is split command substitutions on newlines, so
set files (echo $files)
will work.
Or, if you wish to make it clear that you're doing this to split it, you can use string split like
set files (string split \n -- $files)
which will then end up the same (because currently string split only adds newlines), but looks a bit clearer. (The "--" is the option separator, so nothing in $files is interpreted as an option)
The latter requires fish >= 2.3.0.

Related

Using space-separated arguments from a field in a tab-separated file

I'm writing a shell script intended to edit audio files using the sox command. I've been running into a strange problem I never encountered in bash scripting before: When defining space separated effects in sox, the command will work when that effect is written directly, but not when it's stored in a variable. This means the following works fine and without any issues:
sox ./test.in.wav ./test.out.wav delay 5
Yet for some reason the following will not work:
IFS=' ' # set IFS to only have a tab character because file is tab-separated
while read -r file effects text; do
sox $file.in.wav $file.out.wav $effects
done <in.txt
...when its in.txt is created with:
printf '%s\t%s\t%s\n' "test" "delay 5" "other text here" >in.txt
The error indicates this is causing it to see the output file as another input.
sox FAIL formats: can't open input file `./output.wav': No such file or directory
I tried everything I could think of: Using quotation marks (sox "$file.in.wav" "$file.out.wav" "$effects"), echoing the variable in-line (sox $file.in.wav $file.out.wav $(echo $effects)), even escaping the space inside the variable (effects="delay\ 5"). Nothing seems to work, everything produces the error. Why does one command work but not the other, what am I missing and how do I solve it?
IFS does not only change the behavior of read; it also changes the behavior of unquoted expansions.
In particular, unquoted expansions' content are split on characters found in IFS, before each element resulting from that split is expanded as a glob.
Thus, if you want the space between delay and 5 to be used for word splitting, you need to have a regular space, not just a tab, in IFS. If you move your IFS assignment to be part of the same simple command as the read, as in IFS=$'\t' read -r file effects text; do, that will stop it from changing behavior in the rest of the script.
However, it's not good practice to use unquoted expansions for word-splitting at all. Use an array instead. You can split your effects string into an array with:
IFS=' ' read -r -a effects_arr <<<"$effects"
...and then run sox "$file.in.wav" "$file.out.wav" "${effects_arr[#]}" to expand each item in the array as a separate word.
By contrast, if you need quotes/escapes/etc to be allowed in effects, see Reading quoted/escaped arguments correctly from a string

How to batch rename files in a terminal?

I want to rename the files in a directory to sequential numbers.
stars_01_012.png
stars_01_014.png
stars_01_015.png
stars_01_017.png
stars_02_012.png
stars_02_014.png
stars_02_015.png
stars_02_017.png
And change it into
stars_01_001.png
stars_01_002.png
stars_01_003.png
stars_01_004.png
stars_02_001.png
stars_02_002.png
stars_02_003.png
stars_02_004.png
Relatives but not completely:
How to Batch Rename Files in a macOS Terminal?
How can I batch rename files using the Terminal?
You can do it with rename which you can install on macOS with homebrew using:
brew install rename
The command then looks like this:
rename --dry-run -X -e 'my #parts=split /_/; my $this=$parts[0].$parts[1]; if(!defined($ENV{previous}) || $this ne $ENV{previous}){$ENV{previous}=$this; $ENV{N}=0}; $ENV{N}=$ENV{N}+1; $_ = $parts[0]."_".$parts[1]."_".$ENV{N}' *png
Sample Output
'stars_01_012.png' would be renamed to 'stars_01_1.png'
'stars_01_014.png' would be renamed to 'stars_01_2.png'
'stars_01_015.png' would be renamed to 'stars_01_3.png'
'stars_01_017.png' would be renamed to 'stars_01_4.png'
'stars_02_012.png' would be renamed to 'stars_02_1.png'
'stars_02_014.png' would be renamed to 'stars_02_2.png'
'stars_02_015.png' would be renamed to 'stars_02_3.png'
'stars_02_017.png' would be renamed to 'stars_02_4.png'
'stars_88_099.png' would be renamed to 'stars_88_1.png'
Explanation:
my #parts=split /_/ splits the filename into 3 parts using the underscore as the separator,
my $this=$parts[0].$parts[1] saves the first two parts simply concatenated together, e.g. "stars01",
the if statement tests if either of the first two parts have changed, then
$ENV{previous}=$this; $ENV{N}=0 saves the current stem of the filename in an environment variable called previous and the current sequential counter in another environment variable N,
$ENV{N}=$ENV{N}+1 increments the sequential counter, and
$_ = $parts[0]."_".$parts[1]."_".$ENV{N} creates the new output filename from the various parts.
If that all looks correct, remove the --dry-run and run it again - probably in a spare directory with a copy of your files until you are sure all is ok :-)
The above may be easier to read like this:
#!/bin/bash
rename --dry-run -X -e '
my #parts=split /_/; # split filename into parts based on underscore
my $this=$parts[0].$parts[1]; # save first two parts of filename in $this
# see if it is a new stem
if(!defined($ENV{previous}) || $this ne $ENV{previous}){
$ENV{previous}=$this; $ENV{N}=0}; # save new initial part, reset N
$ENV{N}=$ENV{N}+1; # increment N
$_ = $parts[0]."_".$parts[1]."_".$ENV{N} # formulate new filename from parts
' *png
Change the last line to the following if you want to zero-pad the numbers out to three digits:
$_ = sprintf("%s_%s_%03d",$parts[0],$parts[1],$ENV{N}) # formulate new filename from parts
Note:
I save the previous file prefix and sequential counter into environment variables to preserve them between files - there may be easier ways - if anyone knows, please ping me! Thanks.
You can also create an applescript script, using the Terminal commandline.
below a script to copy in the script editor
set thefile to do shell script "ls The_path_of_your_files" that you wish to rename '"with in common the extension (in your example .png) which gives:
**set thefile to do shell script "ls The_path_of_your_files/*.png"
set AppleScript's text item delimiters to return
set chFile to text items of thefile
set AppleScript's text item delimiters to ""
set n to count chFile
repeat with i from 1 to n
set rnFile to item i of chFile
set nwname to (do shell script "echo" & quoted form of rnFile & "| sed 's # stars_01_01 # stars_01_00 #'")
set endFile to (do shell script "mv -f" & quoted form of rnFile & "" & quoted form of nwname)
end repeat**
which is equivalent to the rename multiple files function of the Finder
Just a rough answer, why use terminal script?
You can just use the Finder with its rename function
by selecting the common part of all your files in your example "stars_01_01" and replacing it with "stars_01_00", this will lead to the same result without having to write a script with :
sed 's#stars_01_01#stars_01_00#g'

Single file contain files name and scores | text processing

I have a folder called files that has 100 files, each one has one value inside;such as: 0.974323
This my code to generate those files and store the single value inside:
DIR="/home/XX/folder"
INPUT_DIR="/home/XX/folder/eval"
OUTPUT_DIR="/home/XX/folder/files"
for i in $INPUT_DIR/*
do
groovy $DIR/calculate.groovy $i > $OUTPUT_DIR/${i##*/}_rates.txt
done
That will generate 100 files inside /home/XX/folder/files, but what I want is one single file that has in each line two columns separated by tab contain the score and the name of the file (which is i).
the score \t name of the file
So, the output will be:
0.9363728 \t resultFile.txt
0.37229 \t outFile.txt
And so on, any help with that please?
Assuming your Groovy program outputs just the score, try something like
#!/bin/sh
# ^ use a valid shebang
# Don't use uppercase for variables
dir="/home/XX/folder"
input_dir="/home/XX/folder/eval"
output_dir="/home/XX/folder/files"
# Always use double quotes around file names
for i in "$input_dir"/*
do
groovy "$dir/calculate.groovy" "$i" |
sed "s%^%$i\t%"
done >"$output_dir"/tabbed_file.txt
The sed script assumes that the file names do not contain percent signs, and that your sed recognizes \t as a tab (some variants will think it's just a regular t with a gratuitous backslash; replace it with a literal tab, or try ctrl-v tab to enter a literal tab at the prompt in many shells).
A much better fix is probably to change your Groovy program so that it accepts an arbitrary number of files as command-line arguments, and includes the file name in the output (perhaps as an option).

Zsh array of strings in for loop

Am trying to print a bunch of strings in a script (in zsh) and it doesn't seem to work. The code would work if I place the array in a variable and use it instead. Any ideas why this doesn't work otherwise?
for string in (some random strings to print) ; echo $string
The default form of the for command in zsh does not use parentheses (if there are any they are not interpreted as part of the for statement):
for string in some random strings to show
do
echo _$string
done
This results in the following output:
_some
_random
_strings
_to
_show
So, echo _$string was run for each word after in. The list ends with the newline.
It is possible to write the whole statement in a single line:
for string in some random strings to show; do echo _$string; done
As usual when putting multiple shell commands in the same line, newlines just need to be replaced by ;. The exception here is the newline after do; while zsh allows a ; to be placed after do, it is usually not done, and in bash it would be a syntax error.
There are also several short forms available for for, all of which are equivalent to the default form above and produce the same output:
for single commands (to be exact: single pipelines or multiple pipelines linked with && or ||, where a pipeline can also be just a single command), there are two options:
the default form, just without do or done:
for string in some random strings to show ; echo _$string
without in but with parentheses, also without do or done
for string (some random strings to show) ; echo _$string
for a list of commands (like in the default form), foreach instead of for, no in, with parentheses and terminated by end:
foreach string (some random strings to show) echo _$string ; end
In your case, you mixed the two short forms for single commands. Due to the presence of in, zsh did not take the parentheses as a syntactic element of the for command. Instead they are interpreted as a glob qualifier. Aside from the fact that you did not intend any filename expansions, this fails for two reasons:
there is no pattern (with or without actual globs) before the glob qualifier. So any matching filename would have to exactly match an empty string, which is just not possible
but mainly "some random strings to print" is not a valid glob qualifier. You probably get an error like "zsh: unknown file attribute: i" (at least with zsh 5.0.5, it may depend on the zsh version).
Check the zsh forloop documentation:
for x (1 2 3); do echo $x; done
for x in 1 2 3; do echo $x; done
You are probably trying to do this:
for string in some random strings to print ;do
echo $string
done

bash parameter expansion changes original string

I have inherited a shell script. One of the things it does is parsing of a list of filenames. For every filename in the list it does following command:
fs_item="`echo ${fs_item%/}`"
This command (a part from doing it's job which in this case, I think, is to remove everything after last slash) replaces spaces in filename with one space:
in: aa bbbb ccc
out: aa bbbb ccc
From this point filename is broken.
So, the question is: can I somehow tell bash not to replace spaces?
Get rid of the backticks and the echo command. It is worse than useless in this situation because it adds nothing, and causes the problem you are trying to solve here.
fs_item="${fs_item%/}"
Is the echo really necessary? You could simply remove it:
fs_item="${fs_item%/}"
If your actual problem is something different, and you cannot get rid of the echo (or some other command invocation), adding some quotes should work:
fs_item="`echo \"${fs_item%/}\"`"
The spaces vanish when running the backticked echo command. The internal field separator includes the space character, so words separated by a sequence of one or more spaces will be passed as separate arguments to echo. Then, echo just prints it's arguments separated by a single space.
Since we're on the internal field separator subject, changing the IFS should also work (but usually has other possibly undesirable effects elsewhere in your script):
IFS=$'\n'
This sets the internal field separator to the newline character. After this, the spaces are no longer considered to be separators for lists. The echo command will receive just one argument (unless you have file names with the newline character in them) and spaces will stay intact.
Try setting IFS to something else, e.g. IFS=","

Resources