Old stuff:
Background:
- Ultimate goal is to put a script in my .bash_profile that warns me by changing text color if I'm typing a commit message and it gets too
long (yes I'm aware vim has something like this).
Progress:
- I found the read -n option which led me to write this:
while true; do
# This hits at the 53rd character
read -rn53 input
# I have commit aliased to gc so the if is just checking if I'm writing a commit
if [ "${input:0:2}" = "gc" ]; then
printf "\nMessage getting long"
fi
done
Question:
- However, running this takes the user out of the bash prompt. I need a way to do something like this while at a normal prompt. I can't find
information on anything like this. Does that mean it's not possible?
Or am I just going about it the wrong way?
New progress:
I found the bind -x option which led me to write this:
check_commit() {
if [ "${READLINE_LINE:0:13}" == 'git commit -m' ] && [ ${#READLINE_LINE} -gt 87 ]; then
echo "Commit is $((${#READLINE_LINE} - 87)) characters too long!"
fi
READLINE_LINE="$READLINE_LINE$1"
READLINE_POINT=$(($READLINE_POINT+1))
}
bind -x '"\"": check_commit "\""'
It listens for a double quote and if I'm writing a long commit message tells me how many characters I am over the limit. Also puts the character I typed into the current line since it is eaten by the bind.
New question:
Now I just need a way to put in a regex, character list or at least a variable instead of \" so I can listen on more keys (Yes, I'm aware bind -x probably wasn't intended to be used this way. I can check performance/footprint/stability myself). I tried "$char", "${char}", "$(char)" and a few other things, but none seem to work. What is the correct approach here ?
AFAIK, not possible in a sane way if you want this to happen during your normal prompt (when PROMPT_COMMAND and PS1 are evaluated). That would involved binding a custom compiled readline function for every insert-self and alike.
If you want this to happen in a script using prompt builtin, this is crudely possible with a loop of
read -e -i $(munge_buf $buf) -n $(buf_warn_len $buf) -p $(buf_warning $buf) buf
like commands. This will allow you to create munge_buf() to alter the currently typed text if needed, buf_warn_len() to calculate a new len to warn at (which may be very large if warning was already displayed), and buf_warn_msg() to derive a warning message based upon the buffer.
this is my very first post on Stackoverflow, and I should probably point out that I am EXTREMELY new to a lot of programming. I'm currently a postgraduate student doing projects involving a lot of coding in various programs, everything from LaTeX to bash, MATLAB etc etc.
If you could explicitly explain your answers that would be much appreciated as I'm trying to learn as I go. I apologise if there is an answer else where that does what I'm trying to do, but I have spent a couple of days looking now.
So to the problem I'm trying to solve: I'm currently using a selection of bioinformatics tools to analyse a range of genomes, and I'm trying to somewhat automate the process.
I have a few sequences with names that look like this for instance (all contained in folders of their own currently as paired files):
SOL2511_S5_L001_R1_001.fastq
SOL2511_S5_L001_R2_001.fastq
SOL2510_S4_L001_R1_001.fastq
SOL2510_S4_L001_R2_001.fastq
...and so on...
I basically wish to automate the process by turning these in to variables and passing these variables to each of the programs I use in turn. So for example my idea thus far was to assign them as wildcards, using the R1 and R2 (which appears in all the file names, as they represent each strand of DNA) as follows:
#!/bin/bash
seq1=*R1_001*
seq2=*R2_001*
On a rudimentary level this works, as it returns the correct files, so now I pass these variables to my first function which trims the DNA sequences down by a specified amount, like so:
# seqtk is the program suite, trimfq is a function within it,
# and the options -b -e specify how many bases to trim from the beginning and end of
# the DNA sequence respectively.
seqtk trimfq -b 10 -e 20 $seq1 >
seqtk trimfq -b 10 -e 20 $seq2 >
So now my problem is I wish to be able to append something like "_trim" to the output file which appears after the >, but I can't find anything that seems like it will work online.
Alternatively, I've been hunting for a script that will take the name of the folder that the files are in, and create a variable for the folder name which I can then give to the functions in question so that all the output files are named correctly for use later on.
Many thanks in advance for any help, and I apologise that this isn't really much of a minimum working example to go on, as I'm only just getting going on all this stuff!
Joe
EDIT
So I modified #ghoti 's for loop (does the job wonderfully I might add, rep for you :D ) and now I append trim_, as the loop as it was before ended up giving me a .fastq.trim which will cause errors later.
Is there any way I can append _trim to the end of the filename, but before the extension?
Explicit is usually better than implied, when matching filenames. Your wildcards may match more than you expect, especially if you have versions of the files with "_trim" appended to the end!
I would be more precise with the wildcards, and use for loops to process the files instead of relying on seqtk to handle multiple files. That way, you can do your own processing on the filenames.
Here's an example:
#!/bin/bash
# Define an array of sequences
sequences=(R1_001 R2_001)
# Step through the array...
for seq in ${sequences[#]}; do
# Step through the files in this sequence...
for file in SOL*_${seq}.fastq; do
seqtk trimfq -b 10 -e 20 "$file" > "${file}.trim"
done
done
I don't know how your folders are set up, so I haven't addressed that in this script. But the basic idea is that if you want the script to be able to manipulate individual filenames, you need something like a for loop to handle the that manipulation on a per-filename basis.
Does this help?
UPDATE:
To put _trim before the extension, replace the seqtk line with the following:
seqtk trimfq -b 10 -e 20 "$file" > "${file%.fastq}_trim.fastq"
This uses something documented in the Bash man page under Parameter Expansion if you want to read up on it. Basically, the ${file%.fastq} takes the $file variable and strips off a suffix. Then we add your extra text, along with the suffix.
You could also strip an extension using basename(1), but there's no need to call something external when you can use something built in to the shell.
Instead of setting variables with the filenames, you could pipe the output of ls to the command you want to run with these filenames, like this:
ls *R{1,2}_001* | xargs -I# sh -c 'seqtk trimfq -b 10 -e 20 "$1" > "${1}_trim"' -- #
xargs -I# will grab the output of the previous command and store it in # to be used by seqtk
I have a bash script which get details from many servers. It works good but i want that the lines get updated and not get written new.
while [ true ] ; do
for i in $(seq ${#details[#]}); do
.... more code (irrelevant)
echo ${server[$i]}
echo $stat1
echo $stat2
echo $stat3
echo $stat4
done
done
How can i do, that all lines get constantly updated into same line?
I try with echo -ne but this makes that everything is in one long line.
I want that the line keep the place and just get updated with new value.
Would be great if somebody knows a trick.
Thank you!
UPDATE 1
#cbuckley:
Thanks for your answer, but its not working correctly. In this way with -ne i tryed it already. Result is (it always create new lines):
10.0.0.2
100310.0.0.1
72710.0.0.3
368310.0.0.2
100310.0.0.1
72710.0.0.3
Should be
10.0.0.1
17
1003
10.0.0.2
319
727
10.0.0.3
157
3683
values under IP should get updated constantly. (i think this should normaly work with -ne, but in my case it dont).
If you've already outputted across multiple lines, you can't remove those lines without clearing the screen. You have two options:
Using watch
You can write a script that outputs the stats once, and then use watch to repeatedly that script:
watch -n 10 ./script.sh # calls script every 10 seconds.
Clearing the screen
If that is not suitable, you'll need to clear the screen yourself:
while [ true ] ; do
clear # clear the screen
for i in $(seq ${#details[#]}); do
# ...
done
sleep 10 # don't update the screen too often
done
However, at this point, you've pretty much implemented a basic version of watch anyway.
You might want to try using sed with the -i option to edit a file 'in place' (i.e. to change the existing file instead of writing a new file)
I need to process a big number of data file with gnuplot in order to produce images which are collected in a movie. As the procedure is time consuming I would like to produce the frames in parallel and a small message should be printed from time to time to inform the user of the progress.
I tried the makefile approach:
SOURCES=$(wildcard ./*.in)
OBJECTS=$(SOURCES:.in=.out)
all: $(OBJECTS)
%.out: %.in
./worker.sh $< $#
where worker.sh is:
gnuplot << EOF
set some_gnuplot_options
set output "$2"
plot "$1"
EOF
But:
I cannot print the progress messages,
I would prefer a single file solution (I have not succeeded in having the content of worker.sh directly in the makefile),
This solution introduces pretty much overhead with respect to a single gnuplot script wich contains all the instructions.
Probably the definitive solution would be to have a nice c++ interface to gnuplot, but I don't know very well the existing ones and I'm not sure how to do the job. Any other idea? Please avoid to imply new or not so common programs like GNU parallel as I cannot have them on some machines I use.
From your comment it sounds as if you are allowed to use your own scripts. GNU Parallel can be used as a script and does not need to be installed, and you can then create a file parallel_plotter:
#!/home/tange/bin/parallel --shebang-wrap -v A={} /usr/bin/gnuplot
name=system("echo $A")
set term png
set output name.".png"
plot sin(x*name)/x
Substitute /home/tange/bin/parallel with the full path to where you put the script parallel.
Then:
chmod 755 parallel_plotter
./parallel_plotter 1 2 3 4 5
This will print a line for each completed run.
To avoid the full path to /home/tange/bin/parallel I can come up with this solution:
#!/usr/bin/env gnuplot
name=system("echo $A")
set term png
set output name.".png"
plot sin(x*name)/x
Then:
chmod 755 parallel_plotter
parallel -v A={} ./parallel_plotter ::: 1 2 3 4 5
You are worried that spawning gnuplot will give a lot of overhead. I tested the above with:
./parallel_plotter {1..1000}
That took 10 secs. So the overhead of starting gnuplot on my system is less than 100 ms per job.
How can you get similar highlightings to Zsh's Less than Bash's Less in Ubuntu?
I switched from OS X to Ubuntu. My Less do not work as expected in Zsh.
Manuals in my Less are green and black with or without the following code.
# comment these out in Ubuntu
export LESS_TERMCAP_mb=$'\E[01;31m' # begin blinking
export LESS_TERMCAP_me=$'\E[0m' # end mode
export LESS_TERMCAP_se=$'\E[0m' # end standout-mode
export LESS_TERMCAP_so=$'\E[38;5;246m' # begin standout-mode - info box
export LESS_TERMCAP_ue=$'\E[0m' # end underline
export LESS_TERMCAP_us=$'\E[04;33;146m' # begin underline is now yellow
# | | |
# | |----------------- yellow
# |-------------------- underline
# to have the indication of cursor's location and line numbers, and R
export LESS="-mNR"
# |--------- only ASCII color
The code makes manuals readable in OS X, but it does not work for Ubuntu in Zsh.
Ubuntu has excellent highlightings in Bash's Less. My manuals have the colors yellow, green and black in Bash without my code. Both Zsh and Bash use the same Less at /usr/bin/less. This suggests me that Ubuntu's Bash has some dot-files which configure it somewhere.
Where are highlightings for Ubuntu's Less in Bash?
This works for me in zsh on archlinux:
$ mkdir ~/.terminfo/ && cd ~/.terminfo
Now get the terminfo description:
$ wget http://nion.modprobe.de/mostlike.txt
Now compile it using tic (the terminfo entry-description compiler)
$ tic mostlike.txt
(you may want to delete the mostlike.txt file after compiling)
mostlike.txt is this
# Reconstructed via infocmp from file: /usr/share/terminfo/x/xterm-pcolor
mostlike|manpages with color looking like most,
am, hs, km, mir, msgr, xenl,
cols#80, it#8, lines#24, wsl#40,
acsc=``aaffggiijjkkllmmnnooppqqrrssttuuvvwwxxyyzz{{||}}~~,
bel=^G, bold=\E[1m\E[31m, clear=\E[H\E[2J, cr=^M,
csr=\E[%i%p1%d;%p2%dr, cub=\E[%p1%dD, cub1=^H,
cud=\E[%p1%dB, cud1=^J, cuf=\E[%p1%dC, cuf1=\E[C,
cup=\E[%i%p1%d;%p2%dH, cuu=\E[%p1%dA, cuu1=\E[A,
dch=\E[%p1%dP, dch1=\E[P, dl=\E[%p1%dM, dl1=\E[M,
dsl=\E]0;\007, ed=\E[J, el=\E[K, enacs=\E)0, fsl=^G,
home=\E[H, ht=^I, hts=\EH, il=\E[%p1%dL, il1=\E[L, ind=^J,
is2=\E7\E[r\E[m\E[?7h\E[?1;3;4;6l\E[4l\E8\E>, kbs=^H,
kcub1=\EOD, kcud1=\EOB, kcuf1=\EOC, kcuu1=\EOA,
kdch1=\E[3~, kf1=\E[11~, kf10=\E[21~, kf11=\E[23~,
kf12=\E[24~, kf13=\E[25~, kf14=\E[26~, kf15=\E[28~,
kf16=\E[29~, kf17=\E[31~, kf18=\E[32~, kf19=\E[33~,
kf2=\E[12~, kf20=\E[34~, kf3=\E[13~, kf4=\E[14~,
kf5=\E[15~, kf6=\E[17~, kf7=\E[18~, kf8=\E[19~, kf9=\E[20~,
kfnd=\E[1~, kich1=\E[2~, kmous=\E[M, knp=\E[6~, kpp=\E[5~,
kslt=\E[4~, rc=\E8, rev=\E[7m\E[34m, ri=\EM, rmacs=^O,
rmcup=\E[2J\E[?47l\E8, rmir=\E[4l, rmkx=\E[?1l\E>,
rmso=\E[m, rmul=\E[m,
rs2=\E7\E[r\E8\E[m\E[?7h\E[?1;3;4;6l\E[4l\E>, sc=\E7,
sgr0=\E[m, smacs=^N, smcup=\E7\E[?47h, smir=\E[4h,
smkx=\E[?1h\E=, smso=\E[1;30m\E[47m, smul=\E[32m,
tbc=\E[3g, tsl=\E]0;, u6=\E[%i%d;%dR, u7=\E[6n,
u8=\E[?1;2c, u9=\E[c,
And then just define an alias in the rc file of your favorite shell:
alias man="TERMINFO=~/.terminfo/ LESS=C TERM=mostlike PAGER=less man"
My default shell is bash so take this with a grain of salt. Start with /etc/profile and see how it sources bash-specific files. You need to re-create that logic for zsh. Maybe the zsh-lovers package can help, at least its title of tips, tricks and examples for the zsh
is suggestive.