How can I check if a directory doesn't exists or empty then run a target?
Example:
.PHONY: all
all: build
build: install
# ...
# ...
install:
git submodule update --recursive --remote
bash install.sh
basically, I need to check for the lib/ directory. If it exists and not empty, do nothing. If it doesn't exists or empty, run the install target.
I'd go for something like this (untested):
# if lib is empty, run install
ifeq ($(wildcard lib/*),)
git submodule update --recursive --remote
bash install.sh
endif
It's not technically running a target, but I reckon it's what you ask for.
For running an actual target you'd setup prerequisites accordingly:
# some program using something in lib/
program: lib/libhello.a
# "making" the lib
lib/libhello.a:
git submodule update --recursive --remote
bash install.sh
Not sure I understand the role of the build target. Let's assume it is another intermediate phony target. Let's also assume that your default goal is all, and you want all to fire build, that itself fires install if and only if lib does not exist or is empty. Finally, let's assume that it is:
git submodule update --recursive --remote
bash install.sh
that creates lib and populates it. You could use a dummy empty file to mark that the lib directory exists and is not empty. Example:
.PHONY: all build install
all: build
build: install
# ...
# ...
install: lib/.exists
lib/.exists:
git submodule update --recursive --remote
bash install.sh
touch "$#"
Related
I'm building the riscv-gnu-toolchain here: https://github.com/riscv-collab/riscv-gnu-toolchain like this:
git clone https://github.com/riscv/riscv-gnu-toolchain.git
cd riscv-gnu-toolchain
git submodule update --init --recursive
export RISCV=/opt/riscv
./configure --prefix="${RISCV}" --enable-multilib
make linux
When compiling during the build of gcc I get this error:
/mnt/data/opt/riscv/sysroot/usr/include/gnu/stubs.h:8:11: fatal error:
gnu/stubs-ilp32.h: No such file or directory
8 | # include <gnu/stubs-ilp32.h>
| ^~~~~~~~~~~~~~~~~~~
I have glibc-devel.i686 and glibc-devel.x86_64 installed on CentOS Stream 8. It's a new install and all up to date. I did follow the instructions and installed the prerequisites.
I think there is something I'm missing in the prerequisites (not listed). Can someone point out what that might be?
I do need to enable multilib since this is for seL4. Here: https://docs.sel4.systems/Hardware/spike.html
I was able to get this to build with the exact steps above:
git clone https://github.com/riscv/riscv-gnu-toolchain.git
cd riscv-gnu-toolchain
git submodule update --init --recursive
export RISCV=/opt/riscv
./configure --prefix="${RISCV}" --enable-multilib
make linux
What I did differently is not building newlib first. I use both versions of toolchain--one with newlib and one with glibc. I guess some left over configs from newlib cause this error.
I wiped out my /opt/riscv directory and the local git repo. I recloned and followed the above steps. It builds after that.
I'm trying to use the similarity function on a Greemplum system using postgres 9.4.24 version.
The Greenplum System is running on a CentOS 7 cluster (CentOS Linux release 7.9.2009 (Core))
I've managed to install the postgresql-contrib package by running this:
sudo yum install postgresql-contrib.x86_64
However, when I run this command on a DB:
create extension pg_trgm();
I get the following:;
SQL Error [58P01]: ERROR: could not open extension control file "/usr/local/greenplum-db-6.13.0/share/postgresql/extension/pg_trgm.control": No such file or directory
I looked in the directory and the pg_trgm.control wasn't there but I was expecting it since I installed the contrib package.
Is there anything I missed?
I found the contrib repository for Greenplum and downloaded the code here:
https://github.com/greenplum-db/gpdb/tree/master
which includes the contrib directory with all the extensions..
however, when I try to make && make install, I get an error:
[gpadmin#mdw pg_trgm]$ sudo make
Makefile:22: ../../src/Makefile.global: No such file or directory
Makefile:23: /contrib/contrib-global.mk: No such file or directory
make: *** No rule to make target `/contrib/contrib-global.mk'. Stop.
Looking at the directory structure and files, Makefile.global doesn't exists and contrib-global.mk does but not sure if the path is correct, I can try changing the path in the Makefile but not sure.
These are the contents of the Makefile file:
# contrib/pg_trgm/Makefile
MODULE_big = pg_trgm
OBJS = trgm_op.o trgm_gist.o trgm_gin.o trgm_regexp.o $(WIN32RES)
EXTENSION = pg_trgm
DATA = pg_trgm--1.3--1.4.sql \
pg_trgm--1.3.sql pg_trgm--1.2--1.3.sql pg_trgm--1.1--1.2.sql \
pg_trgm--1.0--1.1.sql pg_trgm--unpackaged--1.0.sql
PGFILEDESC = "pg_trgm - trigram matching"
REGRESS = pg_trgm pg_word_trgm pg_strict_word_trgm
REGRESS_OPTS += --init-file=$(top_srcdir)/src/test/regress/init_file
ifdef USE_PGXS
PG_CONFIG = pg_config
PGXS := $(shell $(PG_CONFIG) --pgxs)
include $(PGXS)
else
subdir = contrib/pg_trgm
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
include $(top_srcdir)/contrib/contrib-global.mk
endif
I have checked out the code in my /tmp folder and it created the structure under /tmp/gpdb
UPDATE:
I did follow Bradford suggestions and was able to compile and install, however, I'm getting this when I run:
create extension pg_trgm;
ERROR: could not open extension control file "/usr/local/greenplum-db-6.13.0/share/postgresql/extension/pg_trgm.control": No such file or directory (seg57 172.28.8.8:6001 pid=146663)
I did run manually once I ran the make and make install:
/usr/bin/install -c -m 755 pg_trgm.so '/usr/local/greenplum-db-6.13.0/lib/postgresql/pg_trgm.so'
/usr/bin/install -c -m 644 pg_trgm.control '/usr/local/greenplum-db-6.13.0/share/postgresql/extension/'
/usr/bin/install -c -m 644 pg_trgm--1.1.sql pg_trgm--1.0--1.1.sql pg_trgm--unpackaged--1.0.sql '/usr/local/greenplum-db-6.13.0/share/postgresql/extension/'
I can see the .control file in there so not sure what's going on, it seems it was copied only to the master but not the segment hosts, would that be an issue? Do I need to compile this on every segment host?
UPDATE:
I did install on each segment and now I'm getting this:
SQL Error [0A000]: ERROR: query plan with multiple segworker groups is not supported
Hint: likely caused by a function that reads or modifies data in a distributed table
would this extension not be supported by greenplum?
At a high level, you will want to download the source for the release of GPDB that you are running. You can do this either by downloading the tarball from the GitHub release page or cloning the repository and checking-out the release tag.
Once you have done that, source greenplum_path.sh from your installation of GPDB, change into the the contrib/pg_trgm directory and run
USE_PGXS=true make
USE_PGXS=true make install
Setting USE_PGXS will pull in the missing Makefiles from the installation of GPDB.
If the install location of GPDB is not owned by gpadmin user, then the make install will fail for permission issues; using sudo make install will most likely fail because pg_config won't be found on the path. In that case, I just run make -n install and then manually run the commands prefixed with sudo.
Here are all the steps in a psuedo-shell session:
git clone https://github.com/greenplum-db/gpdb.git
cd gpdb
# use git checkout 6.13.0 for older versions of git
git switch --detach 6.13.0
git submodule update --init --recursive
source /usr/local/greenplum-db-6.13.0/greenplum_path.sh
cd contrib/pg_trgm
USE_PGXS=true make
USE_PGXS=true make -n install
Edit: I should clarify that I haven't tested if pg_trgm works with GPDB, only that it builds and compiles.
I have all my git project in the current directory
tox
Python-pattern
Click
...
How to update all with git pull using make ?
A) Add a target that contains a shell loop over all subdirectories and execute git pull in every subdirectory.
Mark the target as .PHONY so it is executed everytime it is called.
When I run this, files changes from the copied /first-repo are being ignored by bundle exec. How can I make sure that the files from /first-repo are included?
#!/bin/bash
cd /first-repo;
git pull;
cd /second-repo;
git pull;
# copy folder from first repo to second repo
cp -rf /first-repo /second-repo/destination_folder;
git add -A;
git commit -m "update destination folder from first repo";
git push;
# build
bundle exec rake build_site;
Sounds like you could use a Git Submodule.
Continuing the example, in second-repo you can instantiate a git submodule of first-repo to get the whole first-repo repository as a directory inside second-repo. You can also use the Jekyll config include variable to specify what files from first-repo to use in second-repo.
UPDATE: This is working, but needs fine-tuning (see below)
Repo Link: https://github.com/oneezy/sweet-element
Travis CI Link: https://travis-ci.org/oneezy/sweet-element
Automation: Travis CI + Github Pages
To explain in more detail what my goals are, I'm trying to simplify my Github + Travis CI workflow w/ 2 environment variables GH_TOKEN + GH_REF stored in a .travis.yml and an executable gpages_build.sh shell script that's used to auto deploy a demo & docs to Github pages on_success: when I git push from command line. I want to create my .travis.yml in such a way that it never has to be manually edited so that it's smart enough to generate the values it needs for GH_REF depending on information that already exists (more details below).
I've followed 2 blog posts that have gotten me to the point I'm at now:
Tutorial 1: How to build Polymer components with Github and Travis CI
Tutorial 2: Automatic github pages deployment for Polymer elements with Travis CI
Current Working Setup ✔
✔ Create "Personal Access Token" from Github
✔ Use travis encrypt command to add GH_TOKEN environment variable to .travis.yml
✔ Add GH_REF: github.com/oneezy/sweet-element environment variable
✔ Use .travis.yml to set everything else up
✔ Use gpages_build.sh to make auto deployment to Github Pages
✔ Make gpages_build.sh executable w/ git update-index --chmod=+x gpages_build.sh
✔ git push and watch it all come together
The Problem ✖
✖ Manually writing GH_REF value in .travis.yml is repetitive/ tedious
✖ Manually writing git update-index --chmod=+x gpages_build.sh is repetitive/ tedious
Possible Solutions & Questions ?
? Can I dynamically generate GH_REF value w/ information that already exists?
? Can I pull information from bower.json or package.json to generate GH_REF value?
? Can I pull information from git config to generate GH_REF value?
? Does Travis CI provide any variables that can be used instead of GH_REF?
? How can I make the gpages_build.sh script executable for all projects?
? Can I execute the gpages_build.sh script from an entire different repo?
? Can I make the gpages_build.sh script executable from the .travis.yml?
The Codes...
.travis.yml (Github Link)
language: node_js
node_js: stable
sudo: required
dist: trusty
addons:
firefox: latest
apt:
sources:
- google-chrome
packages:
- google-chrome-stable
install:
- npm install -g bower polymer-cli
- npm install
- bower install
script:
- xvfb-run polymer test
after_success:
- if [ "$TRAVIS_BRANCH" = "master" ]; then
./gpages_build.sh oneezy sweet-element "Justin O'Neill'" justin#oneezy.com; fi
env:
global:
- GH_REF: github.com/oneezy/sweet-element
- secure: p1OHpnsMIpMjQ4yiFAZJoDZBr/5VHXel+HHC9s8O+MvIqyv5IdxNexkmQKYJneDfYG8XZ/8aNoP4Bsiycysw5POCX1Z9BAwkEBIQ8rdgslzNoWronbtAZUBQAUFxQoVaKt1hBLXNpfyrfRGIfNtAKgA8pekurvIcgjnPmzsNGWr1ztj2y7/5mR7VZZQy3bcM9cZNZLUymyr+RENOXufJnPG2ve/yha/VynUz2mPWEIQPPhg17ar2ICWZL0EZjG6lajR5g83TtSrDxRN2tTGpVWlKVi6udDB/JU+RLt744qhblXwRpFqh1E7r2xUxJWvibQt+UtuRwi6iNJxAy40/XW6Ss/unkwjmRZgU+G98Z3ojJj8Nrp9xah0H2S6M2CH8ySYHnBO6FhunQ3oeXYUn7mYyRiWRz1sjBn0rhWorD67pBFRKIRKFjPPlD9BuI/l/mD8ulgLa7IJFnkt5ZHJx3cWU/BGQ8xLcfor4SgkE4sxlZQWkkn2m2gwvw33JJP6Vv97cs/mgEYORVlBGdG5eAQc1+k18sbGTbbZZojJr5wp4c9VrnDKA7Ztt70ygSHn4etQRogngKPsrKHWnK2q1zBlWoTDq5zjdYZFQt+VC8fET87VUH5Rl7Cn9Chjg8ybY1a4Dq4zKM4uJVKsAYtL+GYNS/kQ/Vgpsd+UTVGx/lDA=
gpages_build.sh (Github Link)
#
# Modified to work with Travis CI automated builds
# Original license follows
#
# #license
# Copyright (c) 2014 The Polymer Project Authors. All rights reserved.
# This code may only be used under the BSD style license found at http://polymer.github.io/LICENSE.txt
# The complete set of authors may be found at http://polymer.github.io/AUTHORS.txt
# The complete set of contributors may be found at http://polymer.github.io/CONTRIBUTORS.txt
# Code distributed by Google as part of the polymer project is also
# subject to an additional IP rights grant found at http://polymer.github.io/PATENTS.txt
#
# This script pushes a demo-friendly version of your element and its
# dependencies to gh-pages.
# usage gp Polymer core-item [branch]
# Run in a clean directory passing in a GitHub org and repo name
org=$1
repo=$2
name=$3
email=$4
branch=${5:-"master"} # default to master when branch isn't specified
mkdir temp && cd temp
# make folder (same as input, no checking!)
mkdir $repo
git clone "https://${GH_TOKEN}#${GH_REF}" --single-branch
# switch to gh-pages branch
pushd $repo >/dev/null
git checkout --orphan gh-pages
# remove all content
git rm -rf -q .
# use bower to install runtime deployment
bower cache clean $repo # ensure we're getting the latest from the desired branch.
git show ${branch}:bower.json > bower.json
echo "{
\"directory\": \"components\"
}
" > .bowerrc
bower install
bower install $org/$repo#$branch
git checkout ${branch} -- demo
rm -rf components/$repo/demo
mv demo components/$repo/
# redirect by default to the component folder
echo "<META http-equiv="refresh" content=\"0;URL=components/$repo/\">" >index.html
git config user.name $name
git config user.email $email
# send it all to github
git add -A .
git commit -am 'Deploy to GitHub Pages'
git push --force --quiet -u "https://${GH_TOKEN}#${GH_REF}" gh-pages > /dev/null 2>&1
popd >/dev/null
I'm trying to simplify my Github + Travis CI workflow w/ 2 environment variables GH_TOKEN + GH_REF stored in .travis.yml ... used by
git clone "https://${GH_TOKEN}#${GH_REF}" --single-branch
Your script can use the TRAVIS_REPO_SLUG environment variable referenced in the documentation
I managed to do EVERYTHING I wanted to.
My Goals
Automating GH_REF value in gpages_build.sh script w/ the TRAVIS_REPO_SLUG (thanks for the tip #osowskit !)
Writing git update-index --chmod=+x gpages_build.sh once and storing it into it's own repo...when needing it, just bower install it!
✔ The Process
Here's my exact approach I took for creating and testing <custom-elements> w/ Travis CI and auto deploying to Github Pages after a successful build.
gpages_build.sh (script):
In order to move away from having to make my script executable inside every project w/ the git update-index --chmod=+x gpages_build.sh command, I needed to centralize this process inside one handy Github repo (oneezy/build-tools). The benefits of this are:
Only having to make the script executable once
Being able to update the script as needed in one place (not every repo)
Ensuring that when others clone the project, they'll always have the executable file
In order to make the GH_REF variable automatic and not have to write it out each time, Travis CI gives us the TRAVIS_REPO_SLUG variable, which is basically username/repo for whatever Github Repository you're working in. You write it out like this: GH_REF="github.com/${TRAVIS_REPO_SLUG}"
```
# This script pushes a demo-friendly version of your element and its
# dependencies to gh-pages.
# usage gp Polymer core-item [branch]
# Run in a clean directory passing in a GitHub org and repo name
GH_REF="github.com/${TRAVIS_REPO_SLUG}"
org=`echo ${TRAVIS_REPO_SLUG} | cut -f 1 -d /`
repo=`echo ${TRAVIS_REPO_SLUG} | cut -f 2 -d /`
name="Travis CI"
email="builds#oneezy.com"
branch=${3:-"master"} # default to master when branch isn't specified
mkdir temp && cd temp
# make folder (same as input, no checking!)
mkdir $repo
git clone "https://${GH_TOKEN}#${GH_REF}" --single-branch
# switch to gh-pages branch
pushd $repo >/dev/null
git checkout --orphan gh-pages
# remove all content
git rm -rf -q .
# use bower to install runtime deployment
bower cache clean $repo # ensure we're getting the latest from the desired branch.
git show ${branch}:bower.json > bower.json
echo "{
\"directory\": \"components\"
}
" > .bowerrc
bower install
bower install $org/$repo#$branch
git checkout ${branch} -- demo
rm -rf components/$repo/demo
mv demo components/$repo/
# redirect by default to the component folder
echo "<META http-equiv="refresh" content=\"0;URL=components/$repo/\">" >index.html
git config user.name $name
git config user.email $email
# send it all to github
git add -A .
git commit -am 'Deploy to GitHub Pages'
git push --force --quiet -u "https://${GH_TOKEN}#${GH_REF}" gh-pages > /dev/null 2>&1
popd >/dev/null
```
Travis.yml Configuration:
One GOTCHA that was throwing me off here was the GH_TOKEN variable, which is the Personal Access Token you need from Github for read/write ability to your repositories. Due to the power this token has, I needed to encrypt it w/ the Travis CLI (Travis Command Line Tool). travis encrypt GH_TOKEN=********* --add, which will automatically add the encrypted variable to your .travis.yml file. I was under the impression that I could use the same encrypted GH_TOKEN for every repo, but I found out the hard way that's not the case. When you encrypt anything w/ Travis, it appears that you're essentially tieing that encryption to the repository you're in, E.G.: you must travis encrypt GH_TOKEN=********* --add inside every repository. One possible workaround would be to configure an alias/ command inside a config file w/ this information stored inside, but I haven't yet explored the theory.
The next thing to do is bower install -s gpages_build.sh script and run it from .travis.yml w/ ./bower_components/build-tools/gpages_build.sh. This will automatically pull the script from our build-tools repo and run it.
```
language: node_js
node_js: stable
sudo: required
dist: trusty
addons:
firefox: latest
apt:
sources:
- google-chrome
packages:
- google-chrome-stable
cache:
directories:
- node_modules
- "$HOME/.cache/bower"
install:
- npm install -g bower polymer-cli
- npm install
- bower install
script:
- xvfb-run polymer test
after_success:
- if [ ${TRAVIS_PULL_REQUEST} = "false" ] && [ "$TRAVIS_BRANCH" = "master" ]; then
bower install -s oneezy/build-tools && ./bower_components/build-tools/gpages_build.sh;
fi
# DO EVERY TIME:
# travis encrypt GH_TOKEN=**************** --add
# env:
# global:
# secure: ZwNuFN1cryC5dff4c3a1qePkoRZoug+HDiN55dFATTt7Opk20C8SgO+RGEWqYWelFkUN2jhAyoJ91GFMxOyYqbqZP+mQfBaFWgBZoKIcGcDur5in4z6ZaWfw65X03K0HIaaKaunpO4C1d/d++zMhqvudhaJ4JgXJXts5cUdmXGxCIEhKE+mH5d76VK4fbpyrtpewllqHeaiIE88oFZ0L8xQP8K7SUXukvVmE1Re0Kl0UjXjsdSUftcj+gnOcBxqGjVlSjQ9Bk0zmP+2nHYo8Gx=
```
And voila!