Well i have a pipeline which runs some data for me with the help of a makefile.
This pipeline creates an enormous amount of redundant files which i want to clean.
I have 1 makefile to run the pipeline. And the pipeline itself is connected to much other makefiles. so i added this code to the pipeline chipcap.mk file:
.PHONY cleanintermediate
cleanintermediate: $(CHIPCAP_OUTPUT)
rm -rf $(SAMPLE)clipsync.trimsync.fastq
rm -rf $(SAMPLE)clipsync.fastq
rm -rf $(SAMPLE)clip.fastq
rm -rf $(SAMPLE).fastq
rm -rf $(SAMPLE).wig && $(RM) -rf $(SAMPLE).wig.idx
rm -rf $(SAMPLE).sam
Now i run my file like this make -f run_samples.mk
This script will invoke the pipeline and start running all samples separately the command that run_samples.mk gives to the pipeline is:
all: pool1_negCTRL pool1R2R1 pool1SP1 pool2Posctrl pool2R2R2 pool2Input
getCommand = $(MAKE) -f /data/DIV5/SASC/project-064-ronald-svdz/analysis/pipelines/chipcap/chipcap.mk \
IN_DIR=/data/DIV5/SASC/project-064-ronald-svdz/input/$(1) \
OUT_DIR=/data/DIV5/SASC/project-064-ronald-svdz/analysis/runs/$(2) \
CHIPCAP_VER=0.1.2 \
FASTQ_EXT=fastq \
CHIPCAP_INPUT=$(3) \
CHIPCAP_QC_MODE=cliptrim \
GZIP_EXT=gz \
MACS14_EXE=/home/sajvanderzeeuw/.virtualenvs/ronald/bin/macs \
PYTHON_EXE=/home/sajvanderzeeuw/.virtualenvs/ronald/bin/python2.7 \
OPT_MACS14_mfold=$(4),$(5)
#OPT_MACS14_control=control.bam
#Negative control sample
pool1_negCTRL:
$(call getCommand,pool1,pool1_negCTRL_monday_test,pool1-negCTRL_S1_L001_R1_001.fastq.gz,15,30)
How can i say to run_samples.mk that also cleanintermediate(which is in chipcap.mk) should be executed. I've been puzzling a lot but cant find the right way to do it.
cleanintermediate: $(CHIPCAP_OUTPUT)
in chipcap.mk should become cleanintermediate: all
In runchipcap.mk add cleanintermediate to getCommand
getCommand = $(MAKE) -f /data/DIV5/SASC/project-064-ronald-svdz/analysis/pipelines/chipcap/chipcap.mk \
IN_DIR=/data/DIV5/SASC/project-064-ronald-svdz/input/$(1) \
OUT_DIR=/data/DIV5/SASC/project-064-ronald-svdz/analysis/runs/$(2) \
CHIPCAP_VER=0.1.2 \
FASTQ_EXT=fastq \
CHIPCAP_INPUT=$(3) \
CHIPCAP_QC_MODE=cliptrim \
GZIP_EXT=gz \
MACS14_EXE=/home/sajvanderzeeuw/.virtualenvs/ronald/bin/macs \
PYTHON_EXE=/home/sajvanderzeeuw/.virtualenvs/ronald/bin/python2.7 \
OPT_MACS14_mfold=$(4),$(5) cleanintermediate
#OPT_MACS14_control=control.bam
Related
The tutorial generates 3 functions with different target and pack into a single static library along with halide runtime.
The question is, how do I call it ?
To my understanding, I should check the cpu feature before calling the functions.
What is the best way the dispatch these functions ?
./lesson_15_generate \
-g my_first_generator \
-f my_first_generator_basic \
-e object,c_header\
-o . \
target=host-x86-64-no_runtime
./lesson_15_generate \
-g my_first_generator \
-f my_first_generator_sse41 \
-e object,c_header\
-o . \
target=host-x86-64-sse41-no_runtime
./lesson_15_generate \
-g my_first_generator \
-f my_first_generator_avx \
-e object,c_header\
-o . \
target=host-x86-64-avx-no_runtime
You can get Halide to do the dispatch for you at runtime using a comma-separated list of targets:
./lesson_15_generate \
-g my_first_generator \
-f my_first_generator \
-e static_library,c_header\
-o . \
target=x86-64-avx-no_runtime,x86-64-sse41-no_runtime,x86-64-no_runtime
That makes it compile three different pieces of code for the three targets. Calling "my_first_generator" will then automatically dispatch to the first thing in the list that it thinks it can run, based on checking the CPU ID on the first call.
I need to comment a line in this Ruby code (I'm using Atom to edit files ... )
The line is the follow .. -DPROJ4_LIBRARY:FILEPATH=#{prefix_dir}/lib/libproj.so and here you're the code ...
bash 'build-and-install-libgeotiff' do
user "root"
code <<-EOH
cd "/tmp"
tar xzf libgeotiff-#{version}.tar.gz
cd libgeotiff-#{version}
export MAKEFLAGS='-j2'
[ -d build ] || mkdir build
cd build
cmake3 .. \
-DCMAKE_INSTALL_PREFIX=#{prefix_dir} \
-DWITH_JPEG=ON \
-DWITH_ZLIB=ON \
-DWITH_PROJ4=ON \
-DPROJ4_INCLUDE_DIR:PATH=#{prefix_dir}/include \
-DPROJ4_LIBRARY:FILEPATH=#{prefix_dir}/lib/libproj.so
-DPROJ4_LIBRARY:FILEPATH=/usr/lib64/libproj.so
make && make install
EOH
I've tried to use # but it seems not right because my editor put automatically #{} ... here you're the transformed code ...
bash 'build-and-install-libgeotiff' do
user "root"
code <<-EOH
cd "/tmp"
tar xzf libgeotiff-#{version}.tar.gz
cd libgeotiff-#{version}
export MAKEFLAGS='-j2'
[ -d build ] || mkdir build
cd build
cmake3 .. \
-DCMAKE_INSTALL_PREFIX=#{prefix_dir} \
-DWITH_JPEG=ON \
-DWITH_ZLIB=ON \
-DWITH_PROJ4=ON \
-DPROJ4_INCLUDE_DIR:PATH=#{prefix_dir}/include \
#{}-DPROJ4_LIBRARY:FILEPATH=#{prefix_dir}/lib/libproj.so
-DPROJ4_LIBRARY:FILEPATH=/usr/lib64/libproj.so
make && make install
EOH
How may I comment my line in this code?
In Bash, if you comment out something in a sequence of lines joined by \, you comment out everything after the #.
Here's a quick and dirty workaround.
code <<-EOH
cd "/tmp"
tar xzf libgeotiff-#{version}.tar.gz
cd libgeotiff-#{version}
export MAKEFLAGS='-j2'
[ -d build ] || mkdir build
cd build
so_arg="-DPROJ4_LIBRARY:FILEPATH=#{prefix_dir}/lib/libproj.so"
cmake3 .. \
-DCMAKE_INSTALL_PREFIX=#{prefix_dir} \
-DWITH_JPEG=ON \
-DWITH_ZLIB=ON \
-DWITH_PROJ4=ON \
-DPROJ4_INCLUDE_DIR:PATH=#{prefix_dir}/include \
$so_arg \
-DPROJ4_LIBRARY:FILEPATH=/usr/lib64/libproj.so
make && make install
EOH
Now, you can simply sed -i s/so_arg=/\1#/' file to comment it out.
I am trying to upload files from my local computer to a server via ssh for deployment. In the upload, I want to exclude some files like .pyc and BUILD.
I have managed to exclude all the files, but the ones called BUILD.
This is currently my (dry-run) terminal command:
rsync -e ssh --dry-run \
--recursive --archive --verbose \
--delete \
--exclude='*.pyc' \
--exclude='*.scss' \
--exclude='__*.js' \
--exclude='*BUILD' \
--exclude='*.jar' \
--exclude='*.DS_Store' \
--exclude='__pycache__' \
local_folder/ \
server:server_folder/
All the exclusions work, except BUILD.
I tried:
--exclude='*/BUILD'
--exclude='*BUILD'
--exclude='BUILD'
None of the previous seems to have detected and deleted the existing BUILD files.
Any ideas on how I can exclude these files?
Thank you!
The command seems to be working but could be that the BUILD files already existed previously.
If you have excluded files or directories from being transferred, --delete-excluded will remove them from the destination side, so this should work:
rsync -e ssh --dry-run \
--recursive --archive --verbose \
--exclude='*.pyc' \
--exclude='*.scss' \
--exclude='__*.js' \
--exclude='*BUILD' \
--exclude='*.jar' \
--exclude='*.DS_Store' \
--exclude='__pycache__' \
--delete-excluded \
local_folder/ \
server:server_folder/
To complement check also this answer which explain the delete options in rsync https://superuser.com/a/156702/284722
I'm trying to use Xcode 5 bots for my Continuos Integration. I need to create a zip-file of my app-file.
In my scheme under Archive I use this script:
LATESTBUILD=$(ls -1rt /Library/Server/Xcode/Data/BotRuns | tail -1)
APP="/Library/Server/Xcode/Data/BotRuns/${LATESTBUILD}/output/Archive.xcarchive/Products/Applications/${PRODUCT_NAME}.app"
echo "Zipping .app for ${PRODUCT_NAME}"
/usr/bin/zip -r "${APP}.zip" "${APP}"
echo "Sending to *HockeyApp*"
curl \
-F "status=2" \
-F "notify=0" \
-F "notes=Testing CI" \
-F "notes_type=0" \
-F "ipa=${APP}.zip" \
-H "X-HockeyAppToken: myToken \
https://rink.hockeyapp.net/api/2/apps/myAppID/app_versions/upload
This will create a .zip-file. However, not the actual app but the whole folder structure for ${app}.
How can I create a zip-file that only contains the actual app?
To specify a file parameter with curl, you have to prefix the filename with #. Correct would be:
echo "Sending to *HockeyApp*"
curl \
-F "status=2" \
-F "notify=0" \
-F "notes=Testing CI" \
-F "notes_type=0" \
-F "ipa=#${APP}.zip" \
-H "X-HockeyAppToken: myToken \
https://rink.hockeyapp.net/api/2/apps/myAppID/app_versions/upload
Fixed this by stepping in to the directory before zipping it.
cd "/tmp/Archive.xcarchive/Products/Applications/"
I'm thinking about installing magento in automatical way. I suppose that I need to create some script or something... but I guess I'm not first person whom need it. So do you know about any good resource or solution how to it? It would work in Windows and Linux OS. Thanks. Jaro.
There are probably others out there but here is a quick and dirty script I use form time to time to install Magento checkout my svn repo and initialise modman. It could be extended to create database if required etc, but it works fine for me as is:
#!/bin/bash
# Required Script Variables
DB_NAME=
DB_USER=
DB_HOST=
DB_PASS=
URL=
MAGENTO_VERSION="1.7.0.0"
ADMIN_FIRSTNAME=
ADMIN_SURNAME=
ADMIN_EMAIL=
ADMIN_USER=
ADMIN_PASS=
SVN_REPO=
# Download and install Magento
wget http://www.magentocommerce.com/downloads/assets/$MAGENTO_VERSION/magento-$MAGENTO_VERSION.tar.gz
printf "\n\nUnpacking and preparing to install Magento...\n"
tar -zxvf magento-$MAGENTO_VERSION.tar.gz
mv magento/* magento/.htaccess .
chmod -R o+w media var
chmod o+w app/etc
rm -rf downloader/pearlib/cache/* downloader/pearlib/download/*
rm -rf magento/ magento-$MAGENTO_VERSION.tar.gz
printf "\n\nInstalling Magento...\n"
/usr/local/bin/php -f install.php -- \
--license_agreement_accepted "yes" \
--locale "en_GB" \
--timezone "Europe/London" \
--default_currency "GBP" \
--db_host "$DB_HOST" \
--db_name "$DB_NAME" \
--db_user "$DB_USER" \
--db_pass "$DB_PASS" \
--url "$URL" \
--use_rewrites "yes" \
--use_secure "no" \
--secure_base_url "" \
--use_secure_admin "no" \
--skip_url_validation "yes" \
--admin_firstname "$ADMIN_FIRSTNAME" \
--admin_lastname "$ADMIN_SURNAME" \
--admin_email "$ADMIN_EMAIL" \
--admin_username "$ADMIN_USER" \
--admin_password "$ADMIN_PASS"
# Setup svn and modman
modman init
mkdir .modman/modules
svn co $SVN_REPO .modman/modules
modman update-all