Running Word similarity Glove - stanford-nlp

I have tried to run Glove on my Ubuntu machine. It was Ok until it gave me the following error that complains about matlab :
./demo.sh: line 37: matlab: command not found
Can anybody help me how to set up matlab on my machine to run Glove?

Solution-1:
matlab command part is only the intrinsic test part of glove tool.
You don't need to run that part of demo.sh to generate word vectors.
Solution-2:
Moreover, with new release of GloVe which is version 1.2, there is python code to do that. Matlab is not necessary, python will handle it.
demo.sh has an if elif code part which checks if there exists a matlab running on machine. If exists, it runs matlab, if not checks for octave. If octave exists, it runs octave. If octave doesnot exist it runs with python. demo.sh has following code, and matlab part is the last part of it:
#!/bin/bash
# Makes programs, downloads sample data, trains a GloVe model, and then evaluates it.
# One optional argument can specify the language used for eval script: matlab, octave or [default] python
make
if [ ! -e text8 ]; then
if hash wget 2>/dev/null; then
wget http://mattmahoney.net/dc/text8.zip
else
curl -O http://mattmahoney.net/dc/text8.zip
fi
unzip text8.zip
rm text8.zip
fi
CORPUS=text8
VOCAB_FILE=vocab.txt
COOCCURRENCE_FILE=cooccurrence.bin
COOCCURRENCE_SHUF_FILE=cooccurrence.shuf.bin
BUILDDIR=build
SAVE_FILE=vectors
VERBOSE=2
MEMORY=4.0
VOCAB_MIN_COUNT=5
VECTOR_SIZE=50
MAX_ITER=15
WINDOW_SIZE=15
BINARY=2
NUM_THREADS=8
X_MAX=10
$BUILDDIR/vocab_count -min-count $VOCAB_MIN_COUNT -verbose $VERBOSE < $CORPUS > $VOCAB_FILE
if [[ $? -eq 0 ]]
then
$BUILDDIR/cooccur -memory $MEMORY -vocab-file $VOCAB_FILE -verbose $VERBOSE -window-size $WINDOW_SIZE < $CORPUS > $COOCCURRENCE_FILE
if [[ $? -eq 0 ]]
then
$BUILDDIR/shuffle -memory $MEMORY -verbose $VERBOSE < $COOCCURRENCE_FILE > $COOCCURRENCE_SHUF_FILE
if [[ $? -eq 0 ]]
then
$BUILDDIR/glove -save-file $SAVE_FILE -threads $NUM_THREADS -input-file $COOCCURRENCE_SHUF_FILE -x-max $X_MAX -iter $MAX_ITER -vector-size $VECTOR_SIZE -binary $BINARY -vocab-file $VOCAB_FILE -verbose $VERBOSE
if [[ $? -eq 0 ]]
then
**if [ "$1" = 'matlab' ]; then
matlab -nodisplay -nodesktop -nojvm -nosplash < ./eval/matlab/read_and_evaluate.m 1>&2
elif [ "$1" = 'octave' ]; then
octave < ./eval/octave/read_and_evaluate_octave.m 1>&2
else
python eval/python/evaluate.py
fi**
fi
fi
fi
fi
IN SHORT:
New version of Glove which is Glove 1.2 solves your problem. Just run it.
matlab, octave or python, one of them is enough for running the script. If you have one of them, demo.sh will handle it.

Related

clang++: command not found

clang++: command not found
OS: Ubuntu 20.4 LTS
clang --version: 10.0.0
clang++ work outside of this program. But when I run this program show this error message clang++ command not found
PATH=/home/musleh/programming/cpp
DIR=''
FILE=''
execute () {
cd ${PATH}/${DIR}
clang++ ${FILE} -o a
time ./a
rm a
if [[ $? -ne 0 ]]
then
echo "***************************Program Fail***************************"
fi
}
while getopts i:d: OPTION
do
case ${OPTION} in
d)
DIR=${OPTARG}
;;
i)
FILE=${OPTARG}
;;
?)
usage
;;
esac
done
if [[ $# -lt 4 ]]
then
usage
elif [[ ! -d ${PATH}/${DIR} ]]
then
echo "${DIR} dir not found!" >&2
elif [[ ! -f ${PATH}/${DIR}/${FILE} ]]
then
echo "${FILE} file not found!" >&2
else
execute
fi```
PATH=/home/musleh/programming/cpp
Is very probably wrong and should be instead perhaps
PATH=/usr/bin:/bin:/usr/local/bin:$HOME/programming/cpp
export PATH
Read much more more about the PATH variable and execvp(3) (which most shells use)
Use strace(1) on your shell script. Read Advanced Linux Programming and more about syscalls(2).
Study for inspiration the source code of GNU bash and read its documentation. It is free software so your are allowed to study (and perhaps improve) its source code.
Of course, clang++ needs to be installed. Check by using the which command. Or view your PATH variable using echo $PATH
See also this

In bash script Symlink are not getting created even though the commands are executed successfully

I am new to RPM Package enhancement/development and working on post-install script.
I want to achieve the symbolic links creation on execution of post-install script but stuck on a issue.
The script execution is working fine for symbolic link creation but for Upgrade part when I check the Symbolic links in "$RPM_pckg_home/bin" they not getting created though the commands are executed successfully.
Here is the sample code;
Original_bin_path=/a/b/c
RPM_pckg_home=/d/e/f
if [[ "$1" -eq 1 ]]; then # 1 for install
cd $RPM_pckg_home/bin
for cmd in `ls Original_bin_path` ; do
ln -s $Original_bin_path/${cmd} ${cmd}
done
elif [[ "$1" -eq 2 ]]; then # 2 for Upgrade
cd $RPM_pckg_home/bin
for cmd in `ls Original_bin_path` ; do
rm ${cmd}
ln -s $Original_bin_path/${cmd} ${cmd}
done
fi
Could you please suggest where would be the issue.
Aside from the possible typo, this is how you should write your loop:
if [[ "$1" -eq 1 ]]; then # 1 for install
for cmd in "$Original_bin_path"/* ; do
ln -s "${cmd}" "$RPM_pckg_home/bin"
done
elif [[ "$1" -eq 2 ]]; then # 2 for Upgrade
for cmd in "$Original_bin_path"/*; do
rm "${cmd}"
ln -s "${cmd}" "$RPM_pckg_home/bin"
done
fi
Instead of iterating over the output of ls, just iterate over the files that match the glob, and modify your rm and ln commands to accommodate the change in the value of $cmd.

Travis CI build Racket Installation for Container-based

My before_install in my .travis.yml reads
before_install:
- . scripts/get_racket.sh
- alias racket="${RACKET_DIR}/bin/racket"
I also have a script get_racket.sh which reads
#!/bin/bash
if [[ -z "$RACKET_VERSION" ]]; then
echo "Racket version environment variable not set, setting default"
export RACKET_VERSION=HEAD # set default Racket version
echo "Version: $RACKET_VERSION"
fi
if [[ -z "$RACKET_DIR" ]]; then
echo "Racket directory environment variable not set, setting default"
export RACKET_DIR='/usr/racket' # set default Racket directory
echo "Directory: $RACKET_DIR"
fi
if [ ! -e cache ] || [ ! -d cache ]; then
echo "Creating cache folder ..."
mkdir cache
fi
cd cache
INSTALL=$(ls | grep '^racket*.sh' | tr -d '[:blank:]')
if [[ ! -e "$RACKET_DIR" ]] || [[ ! -d "$RACKET_DIR" ]]; then
if [[ -z "$INSTALL" ]]; then
echo "Racket installation script not found, building."
if [ ! -e travis-racket ] || [ ! -d travis-racket ] \
|| [ ! -e travis-racket/install-racket.sh ] \
|| [ ! -f travis-racket/install-racket.sh ]; then
git clone https://github.com/greghendershott/travis-racket.git
fi
bash < travis-racket/install-racket.sh
else
"./$INSTALL"
fi
fi
which racket &>/dev/null
ESTATUS=$?
if [[ -n "$ESTATUS" ]]; then
echo "Adding racket to PATH"
export PATH="${PATH}:${RACKET_DIR}/bin"
fi
alias racket='$RACKET_DIR/bin/racket'
cd ..
but in a script that uses racket later in my build chain, I keep getting
racket: command not found
As you can see in the above snippets, I have tried a few workarounds to install (and later cache for faster builds) racket without sudo privileges (because this is a restriction of Travis CI's Container-based infrastructure). Any help would be much appreciated, I'm stumped.
You need to figure out whether this install script you've shown successfully puts a working Racket binary anywhere on the disk. Maybe it didn't even compile, or maybe it tried to install in /usr/bin, where you don't have write access without sudo, or maybe there's something wrong with the binary. Find the binary, make sure it works.
If it does work, you need to pay attention to where your script puts Racket. Does it go to /usr/bin, $HOME, or someplace else entirely?
Finally, you need to figure out where the failing script is looking for Racket. The line where you set the $PATH will not affect the $PATH as seen from another shell script. I'd bet it's installing somewhere that's not in the default $PATH, and your failing script is looking only in the default $PATH.

Using 'grep' in nested if-statement

I'm still finding my way around bash scripting so please bear with me.
At the moment I am trying to write a script that checks a few on a server.
Once check is to see if the GPU driver has is the latest version.
However regardless of the installed GPU driver on the server, the script returns GPU is not upgraded
Here is the code:
#!/bin/bash -x
######################################################
#GENERAL VARIABLES
GPU_DRIVER=270.41.19
######################################################
#Checking if Packsges are Installed
if [ $(uname -r) != $KERNEL_VERSION ]
then
echo "Kernel is not Upgraded"
#INSTALL KENRENL!
#REBOOT!
else if [ ! $(nvidia-smi -q |grep -q $GPU_DRIVER) ]
then
echo "GPU is not Upgraded"
else if [ $(cat /usr/ort/build_number) != $CODE_RELEASE ]
then
echo "Code Release526 Has not Been Installed"
fi
fi
fi
I would like to know why the condition in the if-statement does not apply?
NOTE:
The output of the nvidia-smi looks similar to below:
:~/script$ nvidia-smi -q|grep Driver
Driver Version : 270.41.19
Driver Model
You want to test whether a grep succeeded or failed. That does not require [...] or $(...). You merely need to execute the grep. Contrary to popular belief, [ is not part of the if statement syntax; it is a bash command which succeeds or fails based on the evaluation of a conditional expression. (Usually, you would want to use [[, which is a better conditional evaluator.) The if statement is followed by a series of ordinary bash statements; followed by the keyword then. If the last statement succeeds, the then branch is taken; otherwise the else branch is taken.
Change
else if [ ! $(nvidia-smi -q |grep -q $GPU_DRIVER) ]
to
elif ! nvidia-smi -q | grep -q -F "$GPU_DRIVER"; then
(And the elif will remove the need for the fi matching that if.)
Aside from removing the test built-in ([), I fixed a couple of other things:
grep normally expects patterns to be regexes. In a regex, a . matches any character. I think you are looking for a precise match, so I added the -F flag.
And I put quotes around the $GPU_DRIVER, just in case.
To explain the if ... then ... elif ... fi syntax, here's the entire if statement:
if [[ $(uname -r) != $KERNEL_VERSION ]]; then
echo "Kernel is not Upgraded"
#INSTALL KENRENL!
#REBOOT!
elif ! nvidia-smi -q |grep -q -F "$GPU_DRIVER"; then
echo "GPU is not Upgraded"
elif [[ $(cat /usr/ort/build_number) != $CODE_RELEASE ]]; then
echo "Code Release526 Has not Been Installed"
fi
The grep -q does't return/print anything. It actually sets the return value as 0 or 1. You can check this using $?. So effectively your if statement becomes
[ ! $() ]
The $() returns false always. This results in the behavior you have defined.

Check if a file is executable

I am wondering what's the easiest way to check if a program is executable with bash, without executing it ? It should at least check whether the file has execute rights, and is of the same architecture (for example, not a windows executable or another unsupported architecture, not 64 bits if the system is 32 bits, ...) as the current system.
Take a look at the various test operators (this is for the test command itself, but the built-in BASH and TCSH tests are more or less the same).
You'll notice that -x FILE says FILE exists and execute (or search) permission is granted.
BASH, Bourne, Ksh, Zsh Script
if [[ -x "$file" ]]
then
echo "File '$file' is executable"
else
echo "File '$file' is not executable or found"
fi
TCSH or CSH Script:
if ( -x "$file" ) then
echo "File '$file' is executable"
else
echo "File '$file' is not executable or found"
endif
To determine the type of file it is, try the file command. You can parse the output to see exactly what type of file it is. Word 'o Warning: Sometimes file will return more than one line. Here's what happens on my Mac:
$ file /bin/ls
/bin/ls: Mach-O universal binary with 2 architectures
/bin/ls (for architecture x86_64): Mach-O 64-bit executable x86_64
/bin/ls (for architecture i386): Mach-O executable i386
The file command returns different output depending upon the OS. However, the word executable will be in executable programs, and usually the architecture will appear too.
Compare the above to what I get on my Linux box:
$ file /bin/ls
/bin/ls: ELF 64-bit LSB executable, AMD x86-64, version 1 (SYSV), for GNU/Linux 2.6.9, dynamically linked (uses shared libs), stripped
And a Solaris box:
$ file /bin/ls
/bin/ls: ELF 32-bit MSB executable SPARC Version 1, dynamically linked, stripped
In all three, you'll see the word executable and the architecture (x86-64, i386, or SPARC with 32-bit).
Addendum
Thank you very much, that seems the way to go. Before I mark this as my answer, can you please guide me as to what kind of script shell check I would have to perform (ie, what kind of parsing) on 'file' in order to check whether I can execute a program ? If such a test is too difficult to make on a general basis, I would at least like to check whether it's a linux executable or osX (Mach-O)
Off the top of my head, you could do something like this in BASH:
if [ -x "$file" ] && file "$file" | grep -q "Mach-O"
then
echo "This is an executable Mac file"
elif [ -x "$file" ] && file "$file" | grep -q "GNU/Linux"
then
echo "This is an executable Linux File"
elif [ -x "$file" ] && file "$file" | grep q "shell script"
then
echo "This is an executable Shell Script"
elif [ -x "$file" ]
then
echo "This file is merely marked executable, but what type is a mystery"
else
echo "This file isn't even marked as being executable"
fi
Basically, I'm running the test, then if that is successful, I do a grep on the output of the file command. The grep -q means don't print any output, but use the exit code of grep to see if I found the string. If your system doesn't take grep -q, you can try grep "regex" > /dev/null 2>&1.
Again, the output of the file command may vary from system to system, so you'll have to verify that these will work on your system. Also, I'm checking the executable bit. If a file is a binary executable, but the executable bit isn't on, I'll say it's not executable. This may not be what you want.
Seems nobody noticed that -x operator does not differ file with directory.
So to precisely check an executable file, you may use
[[ -f SomeFile && -x SomeFile ]]
Testing files, directories and symlinks
The solutions given here fail on either directories or symlinks (or both). On Linux, you can test files, directories and symlinks with:
if [[ -f "$file" && -x $(realpath "$file") ]]; then .... fi
On OS X, you should be able to install coreutils with homebrew and use grealpath.
Defining an isexec function
You can define a function for convenience:
isexec() {
if [[ -f "$1" && -x $(realpath "$1") ]]; then
true;
else
false;
fi;
}
Or simply
isexec() { [[ -f "$1" && -x $(realpath "$1") ]]; }
Then you can test using:
if `isexec "$file"`; then ... fi
Also seems nobody noticed -x operator on symlinks. A symlink (chain) to a regular file (not classified as executable) fails the test.
First you need to remember that in Unix and Linux, everything is a file, even directories. For a file to have the rights to be executed as a command, it needs to satisfy 3 conditions:
It needs to be a regular file
It needs to have read-permissions
It needs to have execute-permissions
So this can be done simply with:
[ -f "${file}" ] && [ -r "${file}" ] && [ -x "${file}" ]
If your file is a symbolic link to a regular file, the test command will operate on the target and not the link-name. So the above command distinguishes if a file can be used as a command or not. So there is no need to pass the file first to realpath or readlink or any of those variants.
If the file can be executed on the current OS, that is a different question. Some answers above already pointed to some possibilities for that, so there is no need to repeat it here.
To test whether a file itself has ACL_EXECUTE bit set in any of permission sets (user, group, others) regardless of where it resides, i. e. even on a tmpfs with noexec option, use stat -c '%A' to get the permission string and then check if it contains at least a single “x” letter:
if [[ "$(stat -c '%A' 'my_exec_file')" == *'x'* ]] ; then
echo 'Has executable permission for someone'
fi
The right-hand part of comparison may be modified to fit more specific cases, such as *x*x*x* to check whether all kinds of users should be able to execute the file when it is placed on a volume mounted with exec option.
This might be not so obvious, but sometime is required to test the executable to appropriately call it without an external shell process:
function tkl_is_file_os_exec()
{
[[ ! -x "$1" ]] && return 255
local exec_header_bytes
case "$OSTYPE" in
cygwin* | msys* | mingw*)
# CAUTION:
# The bash version 3.2+ might require a file path together with the extension,
# otherwise will throw the error: `bash: ...: No such file or directory`.
# So we make a guess to avoid the error.
#
{
read -r -n 4 exec_header_bytes 2> /dev/null < "$1" ||
{
[[ -x "${1%.exe}.exe" ]] && read -r -n 4 exec_header_bytes 2> /dev/null < "${1%.exe}.exe"
} ||
{
[[ -x "${1%.com}.com" ]] && read -r -n 4 exec_header_bytes 2> /dev/null < "${1%.com}.com"
}
} &&
if [[ "${exec_header_bytes:0:3}" == $'MZ\x90' ]]; then
# $'MZ\x90\00' for bash version 3.2.42+
# $'MZ\x90\03' for bash version 4.0+
[[ "${exec_header_bytes:3:1}" == $'\x00' || "${exec_header_bytes:3:1}" == $'\x03' ]] && return 0
fi
;;
*)
read -r -n 4 exec_header_bytes < "$1"
[[ "$exec_header_bytes" == $'\x7fELF' ]] && return 0
;;
esac
return 1
}
# executes script in the shell process in case of a shell script, otherwise executes as usual
function tkl_exec_inproc()
{
if tkl_is_file_os_exec "$1"; then
"$#"
else
. "$#"
fi
return $?
}
myscript.sh:
#!/bin/bash
echo 123
return 123
In Cygwin:
> tkl_exec_inproc /cygdrive/c/Windows/system32/cmd.exe /c 'echo 123'
123
> tkl_exec_inproc /cygdrive/c/Windows/system32/chcp.com 65001
Active code page: 65001
> tkl_exec_inproc ./myscript.sh
123
> echo $?
123
In Linux:
> tkl_exec_inproc /bin/bash -c 'echo 123'
123
> tkl_exec_inproc ./myscript.sh
123
> echo $?
123

Resources