In Makefile, when I write to file using echo -e "text" >, -e is also passed:
APIM_5 = echo -e "[Desktop Entry]\nName=$(MAIN)\nExec=$(MAIN)\nIcon=$(MAIN)\nType=Application\nVersion=1.0\nCategories=Utility;" > AppDir/usr/share/applications/$(MAIN).desktop;
But the file I echo into ($(MAIN).desktop) looks like below:
-e [Desktop Entry]
Name=main
Exec=main
Icon=main
Type=Application
Version=1.0
Categories=Utility;
All definitions together and how I call them:
APIM_1 = cd output;
APIM_2 = $(RM) AppDir appimage-build;
APIM_3 = mkdir -p AppDir/usr/bin AppDir/usr/share/applications AppDir/usr/share/icons/hicolor/256x256/apps/ AppDir/usr/lib;
APIM_4 = touch AppDir/usr/share/applications/$(MAIN).desktop;
APIM_5 = echo -e "[Desktop Entry]\nName=$(MAIN)\nExec=$(MAIN)\nIcon=$(MAIN)\nType=Application\nVersion=1.0\nCategories=Utility;" > AppDir/usr/share/applications/$(MAIN).desktop;
APIM_6 = cp $(MAIN) AppDir/usr/bin/;
APIM_7 = cp ../meta/icon/$(MAIN).png AppDir/usr/share/icons/hicolor/256x256/apps/;
APIM_8 = appimage-builder --skip-test;
appimage: all
$(APIM_1) $(APIM_2) $(APIM_3) $(APIM_4) $(APIM_5) $(APIM_6) $(APIM_7) $(APIM_8)
#echo Executing 'appimage' complete!
What causes this?
Related
If I'm in a deep directory, let's say
/run/media/PhoenixFlame101/Coding/Projects/react-app
the fish prompt currently looks like this:
/r/m/Ph/C/P/react-app >
How do I change it to show only the current directory? Like this:
react-app >
I am also using tide, if that makes any difference.
Edit:
Since #glenn-jackman asked here's the outputs of type fish_prompt:
fish_prompt is a function with definition
# Defined in /home/PhoenixFlame101/.config/fish/functions/fish_prompt.fish # line 2
function fish_prompt
_tide_status=$status _tide_pipestatus=$pipestatus if not set -e _tide_repaint
jobs -q && set -lx _tide_jobs
/usr/bin/fish -c "set _tide_pipestatus $_tide_pipestatus
set _tide_parent_dirs $_tide_parent_dirs
PATH=$(string escape "$PATH") CMD_DURATION=$CMD_DURATION fish_bind_mode=$fish_bind_mode set _tide_prompt_4007 (_tide_2_line_prompt)" &
builtin disown
command kill $_tide_last_pid 2>/dev/null
set -g _tide_last_pid $last_pid
end
math $COLUMNS-(string length -V "$_tide_prompt_4007[1]$_tide_prompt_4007[3]")+5 | read -lx dist_btwn_sides
echo -ns \n''(string replace #PWD# (_tide_pwd) "$_tide_prompt_4007[1]")''
string repeat -Nm(math max 0, $dist_btwn_sides-$_tide_pwd_len) ' '
echo -ns "$_tide_prompt_4007[3]"\n"$_tide_prompt_4007[2] "
end
and type prompt_pwd:
prompt_pwd is a function with definition
# Defined in /usr/share/fish/functions/prompt_pwd.fish # line 1
function prompt_pwd --description 'short CWD for the prompt'
set -l options h/help d/dir-length= D/full-length-dirs=
argparse -n prompt_pwd $options -- $argv
or return
if set -q _flag_help
__fish_print_help prompt_pwd
return 0
end
set -q argv[1]
or set argv $PWD
set -ql _flag_d
and set -l fish_prompt_pwd_dir_length $_flag_d
set -q fish_prompt_pwd_dir_length
or set -l fish_prompt_pwd_dir_length 1
set -l fulldirs 0
set -ql _flag_D
and set fish_prompt_pwd_full_dirs $_flag_D
set -q fish_prompt_pwd_full_dirs
or set -l fish_prompt_pwd_full_dirs 1
for path in $argv
# Replace $HOME with "~"
set -l realhome ~
set -l tmp (string replace -r '^'"$realhome"'($|/)' '~$1' $path)
if test "$fish_prompt_pwd_dir_length" -eq 0
echo $tmp
else
# Shorten to at most $fish_prompt_pwd_dir_length characters per directory
# with full-length-dirs components left at full length.
set -l full
if test $fish_prompt_pwd_full_dirs -gt 0
set -l all (string split -m (math $fish_prompt_pwd_full_dirs - 1) -r / $tmp)
set tmp $all[1]
set full $all[2..]
else if test $fish_prompt_pwd_full_dirs -eq 0
# 0 means not even the last component is kept
string replace -ar '(\.?[^/]{'"$fish_prompt_pwd_dir_length"'})[^/]*' '$1' $tmp
continue
end
string join / (string replace -ar '(\.?[^/]{'"$fish_prompt_pwd_dir_length"'})[^/]*/' '$1/' $tmp) $full
end
end
end
I'm not sure what exactly this does, but I hope it helps!
I am trying to download some files with snakemake. The files (http://snpeff.sourceforge.net/SnpSift.html#dbNSFP) I would like to download are on a google site/drive and my usual wget approach does not work. I found a bash function that does the job (https://www.zachpfeffer.com/single-post/wget-a-Google-Drive-file):
function gdrive_download () { CONFIRM=$(wget --quiet --save-cookies /tmp/cookies.txt --keep-session-cookies --no-check-certificate "https://docs.google.com/uc?export=download&id=$1" -O- | sed -rn 's/.*confirm=([0-9A-Za-z_]+).*/\1\n/p') wget --load-cookies /tmp/cookies.txt "https://docs.google.com/uc?export=download&confirm=$CONFIRM&id=$1" -O $2 rm -rf /tmp/cookies.txt }
gdrive_download 120aPYqveqPx6jtssMEnLoqY0kCgVdR2fgMpb8FhFNHo test.txt
I have tested this function with my ids in a plain bash script and was able to download all the files. To add a bit to the complexity, I must use a workplace template, and incorporate the function into it.
rule dl:
params:
url = 'ftp://ftp.ncbi.nlm.nih.gov/pub/clinvar/vcf_{genome}/{afile}'
output:
'data/{genome}/{afile}'
params:
id1 = '0B7Ms5xMSFMYlOTV5RllpRjNHU2s',
f1 = 'dbNSFP.txt.gz'
shell:
"""CONFIRM=$(wget --quiet --save-cookies /tmp/cookies.txt --keep-session-cookies --no-check-certificate "https://docs.google.com/uc?export=download&id={{params.id1}}" -O- | sed -rn "s/.*confirm=([0-9A-Za-z_]+).*/\1\n/p") && wget --load-cookies /tmp/cookies.txt "https://docs.google.com/uc?export=download&confirm=$CONFIRM&id={{params.id1}}" -O {{params.f1}} && rm -rf /tmp/cookies.txt"""
#'wget -c {params.url} -O {output}'
rule checksum:
input:
i = 'data/{genome}/{afile}'
output:
o = temp('tmp/{genome}/{afile}.md5')
shell:
'md5sum {input} > {output}'
rule file_size:
input:
i = 'data/{genome}/{afile}'
output:
o = temp('tmp/{genome}/{afile}.size')
shell:
'du -csh --apparent-size {input} > {output}'
rule file_info:
"""md5 checksum and file size"""
input:
md5 = 'tmp/{genome}/{afile}.md5',
s = 'tmp/{genome}/{afile}.size'
output:
o = temp('tmp/{genome}/info/{afile}.csv')
run:
with open(input.md5) as f:
md5, fp = f.readline().strip().split()
with open(input.s) as f:
size = f.readline().split()[0]
with open(output.o, 'w') as fout:
print('filepath,size,md5', file=fout)
print(f"{fp},{size},{md5}", file=fout)
rule manifest:
input:
expand('tmp/{genome}/info/{suffix}.csv', genome=('GRCh37','GRCh38'), suffix=('dbNSFP.txt.gz', 'dbNSFP.txt.gz.tbi'))
#expand('tmp/{genome}/info/SnpSift{suffix}.csv', genome=('GRCh37','GRCh38'), suffix=('dbNSFP.txt.gz', 'dbNSFP.txt.gz.tbi'))
output:
o = 'MANIFEST.csv'
run:
pd.concat([pd.read_csv(afile) for afile in input]).to_csv(output.o, index=False)
There are four downloadable files for which I have ids (I only show one in params), however I don't know how to call the bash functions as written by ZPfeffer for all the ids I have with snakemake. Additionally, when I run this script, there are several errors, the most pressing being
sed: -e expression #1, char 31: unterminated `s' command
I am far from a snakemake expert, any assistance on how to modify my script to a) call the functions with 4 different ids, b) remove the sed error, and c) verify whether this is the correct url format (currently url = 'https://docs.google.com/uc?export/{afile}) will be greatly appreciated.
You would want to use raw string literal so that snakemake doesn't escape special characters, such as backslash in sed command. For example (notice r in front of shell command):
rule foo:
shell:
r"sed d\s\"
You could use --printshellcmds or -p to see how exactly shell: commands get resolved by snakemake.
Here is how I "solved" it:
import pandas as pd
rule dl:
output:
'data/{genome}/{afile}'
shell:
"sh download_snpsift.sh"
rule checksum:
input:
i = 'data/{genome}/{afile}'
output:
o = temp('tmp/{genome}/{afile}.md5')
shell:
'md5sum {input} > {output}'
rule file_size:
input:
i = 'data/{genome}/{afile}'
output:
o = temp('tmp/{genome}/{afile}.size')
shell:
'du -csh --apparent-size {input} > {output}'
rule file_info:
"""md5 checksum and file size"""
input:
md5 = 'tmp/{genome}/{afile}.md5',
s = 'tmp/{genome}/{afile}.size'
output:
o = temp('tmp/{genome}/info/{afile}.csv')
run:
with open(input.md5) as f:
md5, fp = f.readline().strip().split()
with open(input.s) as f:
size = f.readline().split()[0]
with open(output.o, 'w') as fout:
print('filepath,size,md5', file=fout)
print(f"{fp},{size},{md5}", file=fout)
rule manifest:
input:
expand('tmp/{genome}/info/{suffix}.csv', genome=('GRCh37','GRCh38'), suffix=('dbNSFP.txt.gz', 'dbNSFP.txt.gz.tbi'))
output:
o = 'MANIFEST.csv'
run:
pd.concat([pd.read_csv(afile) for afile in input]).to_csv(output.o, index=False)
And here is the bash script.
function gdrive_download () {
CONFIRM=$(wget --quiet --save-cookies /tmp/cookies.txt --keep-session-cookies --no-check-certificate "https://docs.google.com/uc?export=download&id=$1" -O- | sed -rn 's/.*confirm=([0-9A-Za-z_]+).*/\1\n/p')
wget --load-cookies /tmp/cookies.txt "https://docs.google.com/uc?export=download&confirm=$CONFIRM&id=$1" -O $2
rm -rf /tmp/cookies.txt
}
gdrive_download 0B7Ms5xMSFMYlSTY5dDJjcHVRZ3M data/GRCh37/dbNSFP.txt.gz
gdrive_download 0B7Ms5xMSFMYlOTV5RllpRjNHU2s data/GRCh37/dbNSFP.txt.gz.tbi
gdrive_download 0B7Ms5xMSFMYlbTZodjlGUDZnTGc data/GRCh38/dbNSFP.txt.gz
gdrive_download 0B7Ms5xMSFMYlNVBJdFA5cFZRYkE data/GRCh38/dbNSFP.txt.gz.tbi
I try to use one tool and I need to use a wildcard present on input.
This is an example:
aDict = {"120":"121" } #tumor : normal
rule all:
input: expand("{case}.mutect2.vcf",case=aDict.keys())
def get_files_somatic(wildcards):
case = wildcards.case
control = aDict[wildcards.case]
return [case + ".sorted.bam", control + ".sorted.bam"]
rule gatk_Mutect2:
input:
get_files_somatic,
output:
"{case}.mutect2.vcf"
params:
genome="ref/hg19.fa",
target= "chr12",
name_tumor='{case}'
log:
"logs/{case}.mutect2.log"
threads: 8
shell:
" gatk-launch Mutect2 -R {params.genome} -I {input[0]} -tumor {params.name_tumor} -I {input[1]} -normal {wildcards.control}"
" -L {params.target} -O {output}"
I Have this error:
'Wildcards' object has no attribute 'control'
So I have a function with case and control. I'm not able to extract code.
The wildcards are derived from the output file/pattern. That is why you only have the wildcard called case. You have to derive the control from that. Try replacing your shell statement with this:
run:
control = aDict[wildcards.case]
shell(
"gatk-launch Mutect2 -R {params.genome} -I {input[0]} "
"-tumor {params.name_tumor} -I {input[1]} -normal {control} "
"-L {input.target2} -O {output}"
)
You could define control in params. Also {input.target2} in shell command would result in error. May be it's supposed to be params.target?
rule gatk_Mutect2:
input:
get_files_somatic,
output:
"{case}.mutect2.vcf"
params:
genome="ref/hg19.fa",
target= "chr12",
name_tumor='{case}',
control = lambda wildcards: aDict[wildcards.case]
shell:
"""
gatk-launch Mutect2 -R {params.genome} -I {input[0]} -tumor {params.name_tumor} \\
-I {input[1]} -normal {params.control} -L {params.target} -O {output}
"""
Sorry, I'm from Brazil and my english is not fluent.
I wanna concatenate 20 files using a shellscript through cat command. However when I run it from a file, all content of files are showed on the screen.
When I run it directly from terminal, works perfectly.
That's my code above:
#!/usr/bin/ksh
set -x -a
. /PROD/INCLUDE/include.prod
DATE=`date +'%Y%m%d%H%M%S'`
FINAL_NAME=$1
# check if all paremeters are passed
if [ -z $FINAL_NAME ]; then
echo "Please pass the final name as parameter"
exit 1
fi
# concatenate files
cat $DIRFILE/AI6LM760_AI6_CF2_SLOTP01* $DIRFILE/AI6LM761_AI6_CF2_SLOTP02* $DIRFILE/AI6LM763_AI6_CF2_SLOTP04* \
$DIRFILE/AI6LM764_AI6_CF2_SLOTP05* $DIRFILE/AI6LM765_AI6_CF2_SLOTP06* $DIRFILE/AI6LM766_AI6_CF2_SLOTP07* \
$DIRFILE/AI6LM767_AI6_CF2_SLOTP08* $DIRFILE/AI6LM768_AI6_CF2_SLOTP09* $DIRFILE/AI6LM769_AI6_CF2_SLOTP10* \
$DIRFILE/AI6LM770_AI6_CF2_SLOTP11* $DIRFILE/AI6LM771_AI6_CF2_SLOTP12* $DIRFILE/AI6LM772_AI6_CF2_SLOTP13* \
$DIRFILE/AI6LM773_AI6_CF2_SLOTP14* $DIRFILE/AI6LM774_AI6_CF2_SLOTP15* $DIRFILE/AI6LM775_AI6_CF2_SLOTP16* \
$DIRFILE/AI6LM776_AI6_CF2_SLOTP17* $DIRFILE/AI6LM777_AI6_CF2_SLOTP18* $DIRFILE/AI6LM778_AI6_CF2_SLOTP19* \
$DIRFILE/AI6LM779_AI6_CF2_SLOTP20* > CF2_FINAL_TEMP
mv $DIRFILE/CF2_FINAL_TEMP $DIRFILE/$FINAL_NAME
I solved the problem putting the cat block inside a function, and redirecting stdout to the final file.
Ex:
concatenate()
I'm converting a makefile so it can be used with gnu make. In the other make program (can't find what was the version...) this part was executed :
#******************************************************************************
# Explicit Rules
#******************************************************************************
#
# This section contents the explicit rules to create the Virtual File system.
# This file system actualy contains the files used by the Web interface.
#
#******************************************************************************
CFGFILE = $(WEBDST)/vfs.cfg
# Compiling vfsdata
$(OBJDIR)vfsdata.o : vfsdata.c
#echo .
#echo ::. COMPILING1 : vfsdata.c
#echo ::::.
$(NOECHOC)$(CC) $(CFLAGS) $(DEBUGFLASG) $(subst \,/,$(VFSDIR))/vfsdata.c -o"$#" $(INCLUDE)
# Dependance of vfsdata.c
vfsdata.c : $(addprefix $(WEBSRC)\,$(WEBGLOBAL)) $(addprefix $(WEBDST)\,$(WEBLOCAL))
# Make vfsdata.c writable
#if exist $(VFSDIR)\vfsdata.c \
chmod 777 $(VFSDIR)\vfsdata.c
#
# Generate the configuration file needed by vfscomp
#
#echo [general] > $(CFGFILE)
#echo vfs_root_dir = $(WEBDST)>> $(CFGFILE)
#echo vfs_data_file = $(VFSDIR)\vfsdata>> $(CFGFILE)
#echo vfs_image_var = vfsimage>> $(CFGFILE)
#echo far_pointer = yes>> $(CFGFILE)
#echo compress = yes>> $(CFGFILE)
#echo [file] >> $(CFGFILE)
#echo $(sort $(WEBGLOBAL) $(WEBLOCAL)) | $(TR) -s " " "[\n*]" \
| $(SED) "s/gif/gif -u/g" \
>> $(CFGFILE)
#
# Create the local directory tree
#
#for %%f in ($(subst /,\,$(dir $(WEBGLOBAL)))) do \
if not exist $(WEBDST)\%%f mkdir $(WEBDST)\%%f
#
# Copy the WEB pages directory tree localy
#
#for %%f in ($(subst /,\,$(WEBGLOBAL))) do \
cp -rf $(WEBSRC)\%%f $(WEBDST)\%%f
#
# Generate the virtual file system
#
#$(TOOLDIR)/vfscomp $(subst \,/,$(CFGFILE))
#
# Delete the local directory tree
#
#delete the directory recursively exept the one in the exclusion list
#for %%f in ($(filter-out $(EXCLUSION), $(WEBDIR))) do \
if exist $(WEBDST)\%%f \
rm -rf $(WEBDST)\%%f
#delete the remaining files.
#for %%f in ($(subst /,\,$(WEBGLOBAL))) do \
if exist $(WEBDST)\%%f \
rm -f $(WEBDST)\%%f
#del $(subst /,\,$(CFGFILE))
When I build my code, I get this error
The syntax of the command is incorrect.
I have the feeling that some commands like #for #del #if exist are command that are not supported(or with a different syntax) in the gnu make.
I checked in the online GNU make documentation but I didn't found commands that are similar to those used below.
Is there any equivalent in gnu make to the command used in this part of the makefile?
Thanks in advance!
EDIT :
I removed the # and executed make. I can see when the message "The syntax of the command is incorrect" appear :
if exist ../Pa/Filesys\vfsdata.c \
chmod 777 ../Pa/Filesys\vfsdata.c
echo [general] > ../Pa/Filesys/vfs.cfg
echo vfs_root_dir = ../Pa/Filesys>> ../Pa/Filesys/vfs.cfg
echo vfs_data_file = ../Pa/Filesys\vfsdata>> ../Pa/Filesys/vfs.cfg
echo vfs_image_var = vfsimage>> ../Pa/Filesys/vfs.cfg
echo far_pointer = yes>> ../Pa/Filesys/vfs.cfg
echo compress = yes>> ../Pa/Filesys/vfs.cfg
echo [file] >> ../Pa/Filesys/vfs.cfg
echo behavior/datasync/datasync.htc behavior/defered/defered.htc behavior/inputs/indirect.htc behavior/inputs/inputs.htc behavior/progress/progress.htc behavior/slider/slBottom.gif behavior/slider/slLeft.gif behavior/slider/slRight.gif behavior/slider/slTop.gif behavior/slider/slider.htc css/env.css html/cfgmain.html html/changepass.html html/diagpa.html html/firmhlp.html html/firmware.html html/home.html html/logpage.html html/progressend.html html/progressstart.html html/soundhlp.html html/support.html html/uplmain.html html/uploadstart.html img/button.gif img/logo.gif index.html xslt/activate.xslt xslt/main.xslt xslt/netmenu.xslt xslt/page.xslt xslt/page_func.xslt xslt/progress.xslt xslt/tlmenu.xslt xslt/upload.xslt | tr -s " " "[\n*]" \
The syntax of the command is incorrect.
The syntax of the command is incorrect.
The syntax of the command is incorrect.
The syntax of the command is incorrect.
The syntax of the command is incorrect.
The syntax of the command is incorrect.
The syntax of the command is incorrect.
The syntax of the command is incorrect.
The syntax of the command is incorrect.
The syntax of the command is incorrect.
The syntax of the command is incorrect.
The syntax of the command is incorrect.
The syntax of the command is incorrect.
The syntax of the command is incorrect.
The syntax of the command is incorrect.
The syntax of the command is incorrect.
The syntax of the command is incorrect.
The syntax of the command is incorrect.
The syntax of the command is incorrect.
The syntax of the command is incorrect.
The syntax of the command is incorrect.
| sed "s/gif/gif -u/g" \
>> ../Pa/Filesys/vfs.cfg
for %%f in (.\ css\ img\ img\ xslt\ xslt\ xslt\ xslt\ xslt\ xslt\ xslt\ xslt\ html\ html\ html\ html\ html\ html\ html\ html\ html\ html\ html\ html\ behavior\slider\ behavior\slider\ behavior\slider\ behavior\slider\ behavior\slider\ behavior\datasync\ behavior\progress\ behavior\inputs\ behavior\inputs\ behavior\defered\) do \
It appear that the syntax error is somewhere on the line
echo $(sort $(WEBGLOBAL) $(WEBLOCAL)) | $(TR) -s " " "[\n*]" \
| $(SED) "s/gif/gif -u/g" \
To be clear, none of the commands you mention are make commands. They are all shell commands. Make is invoking a shell and passing those commands to the shell. As mentioned elsewhere, the "#" is a special command to make (supported by all POSIX-based make implementations including GNU make) that tells make to not print the command before sending it to the shell (the "#" is removed by make before sending the command to the shell).
Second, the error message "the syntax of the command is incorrect" is not anything GNU make ever prints. Either you have paraphrased the error message rather than repeating it verbatim (never a good idea when asking for help), or that message is being printed not by make but by some shell command that make invokes.
This will take a few iterations (especially since you haven't given us a complete sample).
There are two rules here; try them separately.
Try the second rule with only the first command:
# Make vfsdata.c writable
#if exist $(VFSDIR)\vfsdata.c \
chmod 777 $(VFSDIR)\vfsdata.c
Let us know the results of these experiments, and we'll go from there.
EDIT:
Interesting. I'd guess that either tr doesn't like your syntax, or the length of that command has exceeded some limit of your Make or OS.
Try doing it from the command line:
echo behavior/datasync/datasync.htc behavior/defered/defered.htc behavior/inputs/indirect.htc behavior/inputs/inputs.htc behavior/progress/progress.htc behavior/slider/slBottom.gif behavior/slider/slLeft.gif behavior/slider/slRight.gif behavior/slider/slTop.gif behavior/slider/slider.htc css/env.css html/cfgmain.html html/changepass.html html/diagpa.html html/firmhlp.html html/firmware.html html/home.html html/logpage.html html/progressend.html html/progressstart.html html/soundhlp.html html/support.html html/uplmain.html html/uploadstart.html img/button.gif img/logo.gif index.html xslt/activate.xslt xslt/main.xslt xslt/netmenu.xslt xslt/page.xslt xslt/page_func.xslt xslt/progress.xslt xslt/tlmenu.xslt xslt/upload.xslt | tr -s " " "[\n*]" \
| sed "s/gif/gif -u/g" \
>> ../Pa/Filesys/vfs.cfg
If that succeeds, it's a Make length limit; try with fewer file names.If it fails, try omitting the last term:
echo blah blah blah \
| sed "s/gif/gif -u/g"
Tell us the results, and we'll proceed...