HFST, python setup - macos

I am building a morphological parser with hfst, but am running into problems with installation.
I have successfully downloaded the hfst package, and it is present in the site-packages directory of my python framework. I can import it in python without a problem.
However, when I try to run some of the commands from the quickstart page, I get syntax errors:
>>> import hfst
>>> hfst-lexc -v -f foma finntreebank.lexc -o finntreebank.inverted.hfst
File "<stdin>", line 1
hfst-lexc -v -f foma finntreebank.lexc -o finntreebank.inverted.hfst
^
SyntaxError: invalid syntax
Is the problem that I need to install foma? Or is the interface between the C++ and python not working? It's been difficult to figure this out with the documentation.
Is there a resource for guidance on how to install and use C++ libraries in python?

hfst-lexc -v -f foma finntreebank.lexc -o finntreebank.inverted.hfst is a command that you can run in a Unix shell / command line. It is not Python code.
If you're on Ubuntu or Mac, you can open the Terminal and cd to the directory of that file and run the command there.
See https://hfst.github.io/python/3.12.1/QuickStart.html for some examples of how to use HFST from Python. I don't know if a lexc command is available from the Python API, but it would probably be simplest to just run that example from the command line. You can still use the resulting .hfst file from Python to do lookups and such.

Related

Perl installed via conda snakemake shell error: Can´t open perl script .. no such file or directory

I´m currently writing a snakemake pipeline, for which I want to include a perl script.
The script is not written by me, but from a github page. I never worked with perl before.
I installed perl (5.32.1) via conda. I have installed miniconda and am working on my universities unix server.
The code for my perl script rule looks like this:
rule r1_filter5end:
input:
config["arima_mapping"] + "unprocessed_bam/{sample}_R1.sam"
output:
config["arima_mapping"] + "filtered_bam/{sample}_R1.bam"
params:
conda:
"../envs/arima_mapping.yaml"
log:
config["logs"] + "arima_mapping/r1_filter5end/{sample}_R1.log"
threads:
12
shell:
"samtools view --threads {threads} -h {input} -b | perl ../scripts/filter_five_end.pl | samtools -b -o {output} 2> log"
When I run this I receive the following error:
Can't open perl script "../scripts/filter_five_end.pl": no such file
or directory found
From what I learned while researching is that the 1. line of a perl script sets the path to my perl executable. The script I downloaded had the following path:
#!/usr/bin/perl
And since I use perl installed via conda this is probably wrong. So I set the path to:
#!/home/mi/my_user/miniconda3/bin/perl
However this did still not work, regardless of if I call
perl ../scripts/filter_five_end.pl
or
../scripts/filter_five_end.pl
Maybe it´s just not possible to run perl scripts via snakemake?
Anyone who had encountered this specific similar case?^^
The problem is not with the shebang. The interpreter path in the shebang doesn't matter because you're calling it with perl ../path directly. The shell that this command is executed in will resolve the path to the perl program (which is very likely the conda one) and then run the script, only taking flags (like -T or -w) from the shebang inside the script.
The error message means it cannot find the actual script file. I suspect when you run that shell command, it's in the wrong directory. Try a fully qualified path.
As stated by OP in their comment:
I forgot that snakemake always looks files up from the Snakemake file not the directory the rules are saved in.
Not quite an answer but perhaps relevant:
I forgot that snakemake always looks files up from the Snakemake file not the directory the rules are saved in.
This is not entirely correct, I think. The reference point is the directory set by -d/--directory which by default is where you execute snakemake:
--directory DIR, -d DIR
Specify working directory (relative paths in the snakefile will use this as their origin). (default: None)

How to run Shell Script using CHMOD in a folder [duplicate]

I simply want to run an executable from the command line, ./arm-mingw32ce-g++, but then I get the error message,
bash: ./arm-mingw32ce-g++: No such file or directory
I'm running Ubuntu Linux 10.10. ls -l lists
-rwxr-xr-x 1 root root 433308 2010-10-16 21:32 arm-mingw32ce-g++
Using sudo (sudo ./arm-mingw32ce-g++) gives
sudo: unable to execute ./arm-mingw32ce-g++: No such file or directory
I have no idea why the OS can't even see the file when it's there. Any thoughts?
This error can mean that ./arm-mingw32ce-g++ doesn't exist (but it does), or that it exists and is a dynamically linked executable recognized by the kernel but whose dynamic loader is not available. You can see what dynamic loader is required by running ldd /arm-mingw32ce-g++; anything marked not found is the dynamic loader or a library that you need to install.
If you're trying to run a 32-bit binary on an amd64 installation:
Up to Ubuntu 11.04, install the package ia32-libs.
On Ubuntu 11.10, install ia32-libs-multiarch.
Starting with 12.04, install ia32-libs-multiarch, or select a reasonable set of :i386 packages in addition to the :amd64 packages.
I faced this error when I was trying to build Selenium source on Ubuntu. The simple shell script with correct shebang was not able to run even after I had all pre-requisites covered.
file file-name # helped me in understanding that CRLF ending were present in the file.
I opened the file in Vim and I could see that just because I once edited this file on a Windows machine, it was in DOS format. I converted the file to Unix format with below command:
dos2unix filename # actually helped me and things were fine.
I hope that we should take care whenever we edit files across platforms we should take care for the file formats as well.
This error may also occur if trying to run a script and the shebang is misspelled. Make sure it reads #!/bin/sh, #!/bin/bash, or whichever interpreter you're using.
I had the same error message when trying to run a Python script -- this was not #Warpspace's intended use case (see other comments), but this was among the top hits to my search, so maybe somebody will find it useful.
In my case it was the DOS line endings (\r\n instead of \n) that the shebang line (#!/usr/bin/env python) would trip over. A simple dos2unix myfile.py fixed it.
I found my solution for my Ubuntu 18 here.
sudo dpkg --add-architecture i386
Then:
sudo apt-get update
sudo apt-get install libc6:i386 libncurses5:i386 libstdc++6:i386
I got this error “No such file or directory” but it exists because my file was created in Windows and I tried to run it on Ubuntu and the file contained invalid 15\r where ever a new line was there.
I just created a new file truncating unwanted stuff
sleep: invalid time interval ‘15\r’
Try 'sleep --help' for more information.
script.sh: 5: script.sh: /opt/ag/cont: not found
script.sh: 6: script.sh: /opt/ag/cont: not found
root#Ubuntu14:/home/abc12/Desktop# vi script.sh
root#Ubuntu14:/home/abc12/Desktop# od -c script.sh
0000000 # ! / u s r / b i n / e n v b
0000020 a s h \r \n w g e t h t t p : /
0000400 : 4 1 2 0 / \r \n
0000410
root#Ubuntu14:/home/abc12/Desktop# tr -d \\015 < script.sh > script.sh.fixed
root#Ubuntu14:/home/abc12/Desktop# od -c script.sh.fixed
0000000 # ! / u s r / b i n / e n v b
0000020 a s h \n w g e t h t t p : / /
0000400 / \n
0000402
root#Ubuntu14:/home/abc12/Desktop# sh -x script.sh.fixed
As mentioned by others, this is because the loader can't be found, not your executable file. Unfortunately the message is not clear enough.
You can fix it by changing the loader that your executable uses, see my thorough answer in this other question: Multiple glibc libraries on a single host
Basically you have to find which loader it's trying to use:
$ readelf -l arm-mingw32ce-g++ | grep interpreter
[Requesting program interpreter: /lib/ld-linux.so.2]
Then find the right path for an equivalent loader, and change your executable to use the loader from the path that it really is:
$ ./patchelf --set-interpreter /path/to/newglibc/ld-linux.so.2 arm-mingw32ce-g++
You will probably need to set the path of the includes too, you will know if you want it or not after you try to run it. See all the details in that other thread.
I got the same error for a simple bash script that wouldn't have 32/64-bit issues. This is possibly because the script you are trying to run has an error in it. This ubuntu forum post indicates that with normal script files you can add sh in front and you might get some debug output from it. e.g.
$ sudo sh arm-mingw32ce-g++
and see if you get any output.
In my case the actual problem was that the file that I was trying to execute was in Windows format rather than Linux.
Below command worked on 16.4 Ubuntu
This issue comes when your .sh file is corrupt or not formatted as per unix protocols.
dos2unix converts the .sh file to Unix format!
sudo apt-get install dos2unix -y
dos2unix test.sh
sudo chmod u+x test.sh
sudo ./test.sh
I had the same problem with a file that I've created on my mac.
If I try to run it in a shell with ./filename I got the file not found error message.
I think that something was wrong with the file.
what I've done:
open a ssh session to the server
cat filename
copy the output to the clipboard
rm filename
touch filename
vi filename
i for insert mode
paste the content from the clipboard
ESC to end insert mode
:wq!
This worked for me.
Added here for future reference (for users who might fall into the same case):
This error happens when working on Windows (which introduces extra characters because of different line separator than Linux system) and trying to run this script (with extra characters inserted) in Linux. The error message is misleading.
In Windows, the line separator is CRLF (\r\n) whereas in linux it is LF (\n). This can be usually be chosen in text editor.
In my case, this happened due to working on Windows and uploading to Unix server for execution.
I just had this issue in mingw32 bash. I had execuded node/npm from Program Files (x86)\nodejs and then moved them into disabled directory (essentially removing them from path). I also had Program Files\nodejs (ie. 64bit version) in path, but only after the x86 version. After restarting the bash shell, the 64bit version of npm could be found. node worked correctly all the time (checked with node -v that changed when x86 version was moved).
I think bash -r would've worked instead of restarting bash: https://unix.stackexchange.com/a/5610
I had this issue and the reason was EOL in some editors such as Notepad++. You can check it in Edit menu/EOL conversion. Unix(LF) should be selected.
I hope it would be useful.
Hit this error trying to run terraform/terragrunt (Single go binary).
Using which terragrunt to find where executable was, got strange error when running it in local dir or with full path
bash: ./terragrunt: No such file or directory
Problem was that there was two installations of terragrunt, used brew uninstall terragrunt to remove one fixed it.
After removing the one, which terragrunt showed the new path /usr/bin/terragrunt everything worked fine.
For those encountering this error when running a java program, it's possible that you're trying to run a 64-bit java program using on a 32-bit linux operating system.
I only realised when I ran ldd on 64-bit java which reported:
ldd /usr/java/jdk1.8.0_05/bin/java
'not a dynamic executable'
Whereas the old 32 bit java reported sensible results:
ldd /usr/java/jdk1.8.0_05/bin/java
In my case, it turns out the file was a symlink:
$ cat deluge-gtk.lock
cat: deluge-gtk.lock: No such file or directory
$ file deluge-gtk.lock
deluge-gtk.lock: broken symbolic link to 32309
Misleading errors like this are fairly common on Linux. Related discussion: https://lwn.net/Articles/532771/
Give it a try by changing the name of file or folder which is not showing in terminal/command prompt.
step1 : change the name of file or folder.
step2 : cd filename/foldername
For future readers, I had this issue when trying to launch a Django server using gunicorn. I was using AWS CodeBuild to build the virtual environment and run tests and using CodeDeploy to put the built artifacts onto the production server and launch the new version (all environments were Ubuntu 20.04). I had mistakenly thought that env/bin/... contained actual binaries of native libraries but that was not the case. It was just Python scripts with a shebang of the path to the Python interpreter on the build machine. In my case, the machine installing the packages and actually running the packages was different. To be more specific, all of the files in env/bin had the shebang #!/codebuild/output/src715682316/src/env/bin/python, so of course running env/bin/gunicorn on the production server would fail. The cryptic error message was when Ubuntu would tell me that env/bin/gunicorn didn't exist as opposed to saying /codebuild/output/src715682316/src/env/bin/python didn't exist. I was able to fix this problem by starting gunicorn using python3 env/bin/gunicorn instead of env/bin/gunicorn.
In a .sh script, each line MUST end with a single character - newline (LF or "\n").
Don't make mistakes like me, because my text-editor of choice is Notepad++ in Win.

How can I run pydoc when it doesn't appear to be on the PATH?

I am using MACOSX 10.12.3 and Python 3.52
I am running into issues when running pydoc in a bash script in python IDLE.
For example:
myFile = ‘/home/user/afile.py’
import subprocess
subprocess.run([‘pydoc’, ‘-w’, myFile])
Inevitably does not create the html file as would be expected.
On the other hand, when I run pydoc in the terminal
pydoc3 -w /home/user/afile.py
it always creates the wanted html file. But pydoc -w /home/user/afile.py generates an error
This is due to the fact that:
the python /home/user/afile.py contains a few print statements in python3 style. That is: print('blabla') instead of: print 'blabla' as would be the case on python2.
the bash script in IDLE does not load the correct pydoc. I should also mention that running subprocess.run([‘pydoc3’, ‘-w’, myFile]) generates an error, for example:
subprocess.run(['pydoc3', '-w', myFile]) File
"/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/subprocess.py",
line 693, in run
with Popen(*popenargs, **kwargs) as process: File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/subprocess.py",
line 947, in init
restore_signals, start_new_session) File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/subprocess.py",
line 1551, in _execute_child
raise child_exception_type(errno_num, err_msg) FileNotFoundError: [Errno 2] No such file or directory: 'pydoc3'
Any one has a way to circumvent this please?
Question 1:
Regarding running the python 3 version of pydoc against python 2 code- That will not work. Pydoc will execute (import) the target module. You would need to use python 2 version of pydoc by modifying your path or using your favorite virtual environment tool.
Question 2:
If you can get a reference to your desired installation of python (by setting the path or activating a virtual environment), then you can handle missing or broken entry points with this syntax:
python -m pydoc -w myfile
You will also need to make sure myfile is in the current directory or on the PYTHONPATH or otherwise importable

How to run cython command line options

I have recently started using cython and now want to use the -a tag as shown http://docs.cython.org/src/quickstart/cythonize.html#determining-where-to-add-types to see how my code is doing. However to use this I need to access the cython command line program. My question is how to do this. I am running windows and tried adding C:\Python32\Lib\site-packages\Cython to my path environmental variable and the typing cython on command line, but this didn't work. Thank you for your time
Scripts and executables are stored in the Scripts/ directory in the python install directory. In my case, I installed python3 in C:\Python31, so I added C:\Python31\Scripts to my path. This allows me to use cython on the command line.
Note that I have to type "cython.py" and not "cython" on the command line, since the Scripts/ directory contains a "cython.py" file, and not a "cython" file.
If you just want the -a (annotation) HTML file, you do NOT necessarily need to use the command line. See https://stackoverflow.com/a/11075375/1272672

"No such file or directory" but it exists

I simply want to run an executable from the command line, ./arm-mingw32ce-g++, but then I get the error message,
bash: ./arm-mingw32ce-g++: No such file or directory
I'm running Ubuntu Linux 10.10. ls -l lists
-rwxr-xr-x 1 root root 433308 2010-10-16 21:32 arm-mingw32ce-g++
Using sudo (sudo ./arm-mingw32ce-g++) gives
sudo: unable to execute ./arm-mingw32ce-g++: No such file or directory
I have no idea why the OS can't even see the file when it's there. Any thoughts?
This error can mean that ./arm-mingw32ce-g++ doesn't exist (but it does), or that it exists and is a dynamically linked executable recognized by the kernel but whose dynamic loader is not available. You can see what dynamic loader is required by running ldd /arm-mingw32ce-g++; anything marked not found is the dynamic loader or a library that you need to install.
If you're trying to run a 32-bit binary on an amd64 installation:
Up to Ubuntu 11.04, install the package ia32-libs.
On Ubuntu 11.10, install ia32-libs-multiarch.
Starting with 12.04, install ia32-libs-multiarch, or select a reasonable set of :i386 packages in addition to the :amd64 packages.
I faced this error when I was trying to build Selenium source on Ubuntu. The simple shell script with correct shebang was not able to run even after I had all pre-requisites covered.
file file-name # helped me in understanding that CRLF ending were present in the file.
I opened the file in Vim and I could see that just because I once edited this file on a Windows machine, it was in DOS format. I converted the file to Unix format with below command:
dos2unix filename # actually helped me and things were fine.
I hope that we should take care whenever we edit files across platforms we should take care for the file formats as well.
This error may also occur if trying to run a script and the shebang is misspelled. Make sure it reads #!/bin/sh, #!/bin/bash, or whichever interpreter you're using.
I had the same error message when trying to run a Python script -- this was not #Warpspace's intended use case (see other comments), but this was among the top hits to my search, so maybe somebody will find it useful.
In my case it was the DOS line endings (\r\n instead of \n) that the shebang line (#!/usr/bin/env python) would trip over. A simple dos2unix myfile.py fixed it.
I found my solution for my Ubuntu 18 here.
sudo dpkg --add-architecture i386
Then:
sudo apt-get update
sudo apt-get install libc6:i386 libncurses5:i386 libstdc++6:i386
I got this error “No such file or directory” but it exists because my file was created in Windows and I tried to run it on Ubuntu and the file contained invalid 15\r where ever a new line was there.
I just created a new file truncating unwanted stuff
sleep: invalid time interval ‘15\r’
Try 'sleep --help' for more information.
script.sh: 5: script.sh: /opt/ag/cont: not found
script.sh: 6: script.sh: /opt/ag/cont: not found
root#Ubuntu14:/home/abc12/Desktop# vi script.sh
root#Ubuntu14:/home/abc12/Desktop# od -c script.sh
0000000 # ! / u s r / b i n / e n v b
0000020 a s h \r \n w g e t h t t p : /
0000400 : 4 1 2 0 / \r \n
0000410
root#Ubuntu14:/home/abc12/Desktop# tr -d \\015 < script.sh > script.sh.fixed
root#Ubuntu14:/home/abc12/Desktop# od -c script.sh.fixed
0000000 # ! / u s r / b i n / e n v b
0000020 a s h \n w g e t h t t p : / /
0000400 / \n
0000402
root#Ubuntu14:/home/abc12/Desktop# sh -x script.sh.fixed
As mentioned by others, this is because the loader can't be found, not your executable file. Unfortunately the message is not clear enough.
You can fix it by changing the loader that your executable uses, see my thorough answer in this other question: Multiple glibc libraries on a single host
Basically you have to find which loader it's trying to use:
$ readelf -l arm-mingw32ce-g++ | grep interpreter
[Requesting program interpreter: /lib/ld-linux.so.2]
Then find the right path for an equivalent loader, and change your executable to use the loader from the path that it really is:
$ ./patchelf --set-interpreter /path/to/newglibc/ld-linux.so.2 arm-mingw32ce-g++
You will probably need to set the path of the includes too, you will know if you want it or not after you try to run it. See all the details in that other thread.
I got the same error for a simple bash script that wouldn't have 32/64-bit issues. This is possibly because the script you are trying to run has an error in it. This ubuntu forum post indicates that with normal script files you can add sh in front and you might get some debug output from it. e.g.
$ sudo sh arm-mingw32ce-g++
and see if you get any output.
In my case the actual problem was that the file that I was trying to execute was in Windows format rather than Linux.
Below command worked on 16.4 Ubuntu
This issue comes when your .sh file is corrupt or not formatted as per unix protocols.
dos2unix converts the .sh file to Unix format!
sudo apt-get install dos2unix -y
dos2unix test.sh
sudo chmod u+x test.sh
sudo ./test.sh
I had the same problem with a file that I've created on my mac.
If I try to run it in a shell with ./filename I got the file not found error message.
I think that something was wrong with the file.
what I've done:
open a ssh session to the server
cat filename
copy the output to the clipboard
rm filename
touch filename
vi filename
i for insert mode
paste the content from the clipboard
ESC to end insert mode
:wq!
This worked for me.
Added here for future reference (for users who might fall into the same case):
This error happens when working on Windows (which introduces extra characters because of different line separator than Linux system) and trying to run this script (with extra characters inserted) in Linux. The error message is misleading.
In Windows, the line separator is CRLF (\r\n) whereas in linux it is LF (\n). This can be usually be chosen in text editor.
In my case, this happened due to working on Windows and uploading to Unix server for execution.
I just had this issue in mingw32 bash. I had execuded node/npm from Program Files (x86)\nodejs and then moved them into disabled directory (essentially removing them from path). I also had Program Files\nodejs (ie. 64bit version) in path, but only after the x86 version. After restarting the bash shell, the 64bit version of npm could be found. node worked correctly all the time (checked with node -v that changed when x86 version was moved).
I think bash -r would've worked instead of restarting bash: https://unix.stackexchange.com/a/5610
I had this issue and the reason was EOL in some editors such as Notepad++. You can check it in Edit menu/EOL conversion. Unix(LF) should be selected.
I hope it would be useful.
Hit this error trying to run terraform/terragrunt (Single go binary).
Using which terragrunt to find where executable was, got strange error when running it in local dir or with full path
bash: ./terragrunt: No such file or directory
Problem was that there was two installations of terragrunt, used brew uninstall terragrunt to remove one fixed it.
After removing the one, which terragrunt showed the new path /usr/bin/terragrunt everything worked fine.
For those encountering this error when running a java program, it's possible that you're trying to run a 64-bit java program using on a 32-bit linux operating system.
I only realised when I ran ldd on 64-bit java which reported:
ldd /usr/java/jdk1.8.0_05/bin/java
'not a dynamic executable'
Whereas the old 32 bit java reported sensible results:
ldd /usr/java/jdk1.8.0_05/bin/java
In my case, it turns out the file was a symlink:
$ cat deluge-gtk.lock
cat: deluge-gtk.lock: No such file or directory
$ file deluge-gtk.lock
deluge-gtk.lock: broken symbolic link to 32309
Misleading errors like this are fairly common on Linux. Related discussion: https://lwn.net/Articles/532771/
Give it a try by changing the name of file or folder which is not showing in terminal/command prompt.
step1 : change the name of file or folder.
step2 : cd filename/foldername
For future readers, I had this issue when trying to launch a Django server using gunicorn. I was using AWS CodeBuild to build the virtual environment and run tests and using CodeDeploy to put the built artifacts onto the production server and launch the new version (all environments were Ubuntu 20.04). I had mistakenly thought that env/bin/... contained actual binaries of native libraries but that was not the case. It was just Python scripts with a shebang of the path to the Python interpreter on the build machine. In my case, the machine installing the packages and actually running the packages was different. To be more specific, all of the files in env/bin had the shebang #!/codebuild/output/src715682316/src/env/bin/python, so of course running env/bin/gunicorn on the production server would fail. The cryptic error message was when Ubuntu would tell me that env/bin/gunicorn didn't exist as opposed to saying /codebuild/output/src715682316/src/env/bin/python didn't exist. I was able to fix this problem by starting gunicorn using python3 env/bin/gunicorn instead of env/bin/gunicorn.
In a .sh script, each line MUST end with a single character - newline (LF or "\n").
Don't make mistakes like me, because my text-editor of choice is Notepad++ in Win.

Resources