I am working on completion for a command that takes argument like "one:two:three".
In the simplest terms, I want ':' to be handled just like a space character is by default. Is there a simple way to do this that I am missing?
I've found the ':' is in COMP_WORDBREAKS, but that the character in COMP_WORDBREAKS are also treated as words as well.
So if the commandline is:
cmd one:tw[TAB]
COMP_CWORD will be 3 and COMP_WORDS[COMP_CWORD-1] will be ':'
For comparison, if the commandline is:
cmd one tw[TAB]
COMP_CWORD will be 2 and COMP_WORDS[COMP_CWORD-1] will be 'one'
Even worse is that if you hit the [TAB] right after the ':' delimiter it acts mostly like a space:
cmd one:[TAB]
Now COMP_CWORD will be 2 and COMP_WORDS[COMP_CWORD-1] will be 'one'.
I can parse the commandline myself from COMP_LINE easily enough, but nicer to find a way to just make ':' act like ' ' in my custom completion. Possible?
Unfortunately, not really. This is actually a 'feature' of bash.
While you could modify COMP_WORDBREAKS, modifying COMP_WORDBREAKS could cause other issues as it is a global variable and will affect the behavior of other completion scripts.
If you take a look at the source for bash-completion, two helper methods exist that can help with this:
_get_comp_words_by_ref with the -n option gets the word-to-complete without considering the characters in EXCLUDE as word breaks
# Available VARNAMES:
# cur Return cur via $cur
# prev Return prev via $prev
# words Return words via $words
# cword Return cword via $cword
#
# Available OPTIONS:
# -n EXCLUDE Characters out of $COMP_WORDBREAKS which should NOT be
# considered word breaks. This is useful for things like scp
# where we want to return host:path and not only path, so we
# would pass the colon (:) as -n option in this case.
# -c VARNAME Return cur via $VARNAME
# -p VARNAME Return prev via $VARNAME
# -w VARNAME Return words via $VARNAME
# -i VARNAME Return cword via $VARNAME
#
__ltrim_colon_completions removes colon containing prefix from COMPREPLY items
# word-to-complete.
# With a colon in COMP_WORDBREAKS, words containing
# colons are always completed as entire words if the word to complete contains
# a colon. This function fixes this, by removing the colon-containing-prefix
# from COMPREPLY items.
# The preferred solution is to remove the colon (:) from COMP_WORDBREAKS in
# your .bashrc:
#
# # Remove colon (:) from list of word completion separators
# COMP_WORDBREAKS=${COMP_WORDBREAKS//:}
#
# See also: Bash FAQ - E13) Why does filename completion misbehave if a colon
# appears in the filename? - http://tiswww.case.edu/php/chet/bash/FAQ
# #param $1 current word to complete (cur)
# #modifies global array $COMPREPLY
For example:
{
local cur
_get_comp_words_by_ref -n : cur
__ltrim_colon_completions "$cur"
}
complete -F _thing thing
First take on a custom parsed solution. Love to know if there is a better way:
parms=$(echo "$COMP_LINE" | cut -d ' ' -f 2)
vals="${parms}XYZZY"
IFS=$":"
words=( $vals )
unset IFS
count=${#words[#]}
cur="${words[$count-1]%%XYZZY}"
Related
Basically what foo(**bar) does in python, here I’d want something like
foo **bar.yaml
and that would become
foo --bar1=1 --bar2=2
Where bar.yaml would be
bar1: 1
bar2: 2
You could use a combination of sed and xargs:
sed -E 's/^(.+):[[:space:]]+(.+)$/--\1=\2/' bar.yaml | xargs -d '\n' foo
sed converts the format of bar.yaml lines (e.g. bar1: 1 -> --bar1=1) and xargs feeds the converted lines as arguments to foo.
You could of course modify/extend the sed part to support other formats or single-dash options like -v.
To test if this does what you want, you can run this Bash script instead of foo:
#!/usr/bin/env bash
echo "Arguments: $#"
for ((i=1; i <= $#; i++)); do
echo "Argument $i: '${!i}'"
done
Here's a version for zsh. Run this code or add it to ~/.zshrc:
function _yamlExpand {
setopt local_options extended_glob
# 'words' array contains the current command line
# yaml filename is the last value
yamlFile=${words[-1]}
# parse 'key : value' lines from file, create associative array
typeset -A parms=("${(#s.:.)${(f)"$(<${yamlFile})"}}")
# trim leading and trailing whitespace from keys and values
# requires extended_glob
parms=("${(kv#)${(kv#)parms##[[:space:]]##}%%[[:space:]]##}")
# add -- and = to create flags
typeset -a flags
for key val in "${(#kv)parms}"; do
flags+=("--${key}='${val}'")
done
# replace the value on the command line
compadd -QU -- "$flags"
}
# add the function as a completion and map it to ctrl-y
compdef -k _yamlExpand expand-or-complete '^Y'
At the zsh shell prompt, type in the command and the yaml file name:
% print -l -- ./bar.yaml▃
With the cursor immediately after the yaml file name, hit ctrl+y. The yaml filename will be replaced with the expanded parameters:
% print -l -- --bar1='1' --bar2='2' ▃
Now you're set; you can hit enter, or add parameters, just like any other command line.
Notes:
This only supports the yaml subset in your example.
You can add more yaml parsing to the function, possibly with yq.
In this version, the cursor must be next to the yaml filename - otherwise the last value in words will be empty. You can add code to detect that case and then alter the words array with compset -n.
compadd and compset are described in the zshcompwid man page.
zshcompsys has details on compdef; the section on autoloaded files describes another way to deploy something like this.
I have a some file names in bash that I have acquired with
$ ones=$(find SRR*pass*1*.fq)
$ echo $ones
SRR6301033_pass_1_trimmed.fq
SRR6301034_pass_1_trimmed.fq
SRR6301037_pass_1_trimmed.fq
...
I then converted into an array so I can iterate over this list and perform some operations with filenames:
# convert to array
$ ones=(${ones// / })
and the iteration:
for i in $ones;
do
fle=$(basename $i)
out=$(echo $fle | grep -Po '(SRR\d*)')
echo "quants/$out.quant"
done
which produces:
quants/SRR6301033
SRR6301034
...
...
SRR6301220
SRR6301221.quant
However I want this:
quants/SRR6301033.quant
quants/SRR6301034.quant
...
...
quants/SRR6301220.quant
quants/SRR6301221.quant
Could somebody explain why what I'm doing doesn't work and how to correct it?
Why do you want this be done this complicated? You can get rid of all the unnecessary roundabouts and just use a for loop and built-in parameter expansion techniques to get this done.
# Initialize an empty indexed array
array=()
# Start a loop over files ending with '.fq' and if there are no such files
# the *.fq would be un-expanded and checking it against '-f' would fail and
# in-turn would cause the loop to break out
for file in *.fq; do
[ -f "$file" ] || continue
# Get the part of filename after the last '/' ( same as basename )
bName="${file##*/}"
# Remove the part after '.' (removing extension)
woExt="${bName%%.*}"
# In the resulting string, remove the part after first '_'
onlyFir="${woExt%%_*}"
# Append the result to the array, prefixing/suffixing strings 'quant'
array+=( quants/"$onlyFir".quant )
done
Now print the array to see the result
for entry in "${array[#]}"; do
printf '%s\n' "$entry"
done
Ways your attempt could fail
With ones=$(find SRR*pass*1*.fq) you are storing the results in a variable and not in an array. A variable has no way to distinguish if the contents are a list or a single string separated by spaces
With echo $ones i.e. an unquoted expansion, the string content is subject to word splitting. You might not see a difference as long as you have filenames with spaces, having one might let you interpret parts of the filename as different files
The part ${ones// / } makes no-sense in converting the string to an array as the attempt to use an unquoted variable $ones itself would be erroneous
for i in $ones; would be error prone for the said reasons above, the filenames with spaces could be interpreted as separated files instead of one.
I'm writing a for loop in bash to run a command and I need to add a comma after one of my variables. I can't seem to do this without an extra space added. When I move "," right next to $bams then it outputs *.sorted,
#!/bin/bash
bams=*.sorted
for i in $bams
do echo $bams ","
done;
Output should be this:
'file1.sorted','file2.sorted','file3.sorted'
The eventual end goal is to be able to insert a list of files into a --flag in the format above. Not sure how to do that either.
First, a literal answer (if your goal were to generate a string of the form 'foo','bar','baz', rather than to run a program with a command line equivalent to somecommand --flag='foo','bar','baz', which is quite different):
shopt -s nullglob # generate a null result if no matches exist
printf -v var "'%s'," *.sorted # put list of files, each w/ a comma, in var
echo "${var%,}" # echo contents of var, with last comma removed
Or, if you don't need the literal single quotes (and if you're passing your result to another program on its command line with the single quotes being syntactic rather than literal, you absolutely don't want them):
files=( *.sorted ) # put *.sorted in an array
IFS=, # set the comma character as the field separator
somecommand --flag "${files[*]}" # run your program with the comma-separated list
try this -
lst=$( echo *.sorted | sed 's/ /,/g' ) # stack filenames with commas
echo $lst
if you really need the single-ticks around each filename, then
lst="'$( echo *.sorted | sed "s/ /','/g" )'" # commas AND quotes
#!/bin/bash
bams=*.sorted
for i in $bams
do flag+="${flag:+,}'$i'"
done
echo $flag
I have a variable equal to a string, which is a series of key/value pairs separated by newlines.
I want to then replace these newline characters with spaces, and set a new variable equal to the result
From various answers on the internet I've arrived at the following:
#test.txt has the content:
#test=example
#what=s0omething
vars="$(cat ./test.txt)"
formattedVars= $("$vars" | tr '\n' ' ')
echo "$taliskerEnvVars"
Problem is when I try to set formattedVars it tries to execute the second line:
script.sh: line 7: test=example
what=s0omething: command not found
I just want formattedVars to equal test=example what=s0omething
What trick am I missing?
Change your line to:
formattedVars=$(tr '\n' ' ' <<< "$secretsContent")
Notice the space of = in your code, which is not permitted in assignment statements.
I see that you are not setting secretsContent in your code, you are setting vars instead.
If possible, use an array to hold contents of the file:
readarray -t vars < ./test.txt # bash 4
or
# bash 3.x
declare -a vars
while IFS= read -r line; do
vars+=( "$line" )
done < ./test.txt
Then you can do what you need with the array. You can make your space-separated list with
formattedVars="${vars[*]}"
, but consider whether you need to. If the goal is to use them as a pre-command modifier, use, for instance,
"${vars[#]}" my_command arg1 arg2
I wrote a simple bash function which would read value from an ini file (defined by variable CONF_FILE) and output it
getConfValue() {
#getConfValue section variable
#return value of a specific variable from given section of a conf file
section=$1
var="$2"
val=$(sed -nr "/\[$section\]/,/\[/{/$var/p}" $CONF_FILE)
val=${val#$var=}
echo "$val"
}
The problem is that it does not ignore comments and runs into troubles if multiple variables within a section names share common substrings.
Example ini file:
[general]
# TEST=old
; TEST=comment
TEST=new
TESTING=this will be output too
PATH=/tmp/test
Running getConfValue general PATH would output /tmp/test as expected, but running getConfValue general TEST shows all the problems this approach has.
How to fix that?
I know there are dedicated tools like crudini or tools for python, perl and php out in the wild, but I do not want to add extra dependencies for simple config file parsing. A solution incorporating awk instead of sed would be just fine too.
Sticking with sed you could anchor your var search to the start of the record using ^ and end it with an equal sign:
"/\[$section\]/,/\[/{/^$var=/p}"
If you are concerned about whitespace in front of your record you could account for that:
"/\[$section\]/,/\[/{/^(\W|)$var=/p}"
That ^(\W|)$var= says "if there is whitespace at the beginning (^(\W) or nothing (|)) before your variable concatenated with an equal sign ($var=)."
If you wanted to switch over to awk you could use something like:
val=$(awk -F"=" -v section=$section -v var=$var '$1=="["section"]"{secFound=1}secFound==1 && $1==var{print $2; secFound=0}' $CONF_FILE)
That awk command splits the record by equal -F"=". Then if the first field in the record is your section ($1=="["section"]") then set variable secFound to 1. Then... if secFound is 1 and the first field is exactly equal to your var variable (secFound==1 && $1==var) then print out the second field ({print $2}) and sets secFound to 0 so we don't pick up any other Test keys.
I encountered this problem and came up with a solution similar to others here.
The main difference is it uses a single awk call to get a response suitable for creating an associative array of the property/value pairs for a section.
This will not ignore the commented properties. Though adding something to do that should not be to hard.
Here's a testing script demonstrating the awk and declare statements used;
#!/bin/bash
#
# Parse a INI style properties file and extract property values for a given section
#
# Author: Alan Carlyle
# License: CC0 (https://creativecommons.org/about/cclicenses/)
#
# Example Input: (example.properties)
# [SEC1]
# item1=value1
# item2="value 2"
#
# [Section 2]
# property 1="Value 1 of 'Section 2'"
# property 2='Single "quoted" value'
#
# Usage:
# $ read_props example.properties "Section 2" property\ 2
# $ Single "quoted" value
#
# Section names and properties with spaces do not need to be quoted.
# Values with spaces must be quoted. Values can use single or double quotes.
# The following characters [ = ] can not be used in names or values.
#
# If the property is not provided the the whole section is outputed.
#
propertiesFile=$1
section=$2
property=$3
# Extract the propetites for the section formated as for associtive array
sectionData="( "$(awk -F'=' -v s="$section" '/^\[/{ gsub(/[\[\]]/, "", $1); f = ($1 == s); next }
NF && f{ print "["$1"]="$2 }' $propertiesFile)" )"
# Create associtive array from extracted section data
declare +x -A "properties=$sectionData"
if [ -z "$property" ]
then
echo $sectionData
else
echo ${properties[$property]}
fi