PyCharm: Can't find spark-submit - bash

I am trying to run a test file in PyCharm with unittest.
It errors with this message:
FileNotFoundError: [Errno 2] No such file or directory: 'myfolder/opt/spark/./bin/spark-submit': 'myfolder/opt/spark/./bin/spark-submit'
I think it has something to do with my .bash_profile which a coworker helped configure. It looks like this:
export SPARK_HOME=myfolder/opt/spark
export PATH=$SPARK_HOME/bin:$PATH
export PATH=$HOME/.node_modules_global/bin:$PATH
# added by Anaconda3 5.1.0 installer
export PATH="/Users/myname/anaconda3/bin:$PATH"
Important to note, the path myfolder/opt/spark/./bin/spark-submit does not exist, but myfolder/opt/spark/bin/spark-submit does. I have no idea why the extra ./ is being added to the path.

Related

'protoc-gen-go-grpc: program not found or is not executable', despite all the efforts

I'm trying to run makefile command 'gen' from this project https://github.com/penthaapatel/grpcblog. The command is: protoc --go_out=. --go_opt=paths=source_relative --go-grpc_out=. --go-grpc_opt=paths=source_relative blog/blog.proto However I get an error:
protoc-gen-go-grpc: program not found or is not executable
--go-grpc_out: protoc-gen-go-grpc: Plugin failed with status code 1.
make: *** [/home/deckard/go/src/grpcblog/Makefile:2: gen] Error 1
I use Ubuntu 20.04? Golang-1.13.8 with $GOROOT=/usr/lib/go-1.13 and $GOPATH=/home/deckard/go ($HOME is /home/deckard). Protoc is libprotoc v.3.6.1.. I installed protobuf-compiler both with:
sudo apt install protobuf-compiler
and
go get -u github.com/golang/protobuf/{proto,protoc-gen-go}
There's a folder /home/deckard/go/src/github.com/golang/protobuf and /home/deckard/go/src/google.golang.org/protobuf with /home/deckard/go/src/google.golang.org/protobuf, with, I guess, everything i need.
My bashrc configurations (tried few methods, so things overlap each other):
export GOROOT=/usr/lib/go-1.13
export GOPATH=$HOME/go
export GOBIN=$GOPATH/bin
export PATH=$PATH:$GOROOT:$GOPATH:$GOBIN
export PATH=$PATH:$GOPATH/bin
export GO_PATH=~/go
export PATH=$PATH:/$GO_PATH/bin
So i tried solutions from here: protoc-gen-go: program not found or is not executable, but none of them helped me. Any clue would be helpful, because I stuck and am very tired.
UPDATE: ls -l /home/deckard/go/bin/protoc-gen-go-grpc shows rwxrwxr-x 1 deckard deckard 7970222 мая 1 14:55 /home/deckard/go/bin/protoc-gen-go-grpc, and printenv PATH returns /home/deckard/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/deckard/.dotnet/tools:/usr/lib/go-1.13:/home/deckard/go:/home/deckard/go/bin:/home/deckard/go/bin://home/deckard/go/bin
UPDATE: which protoc returns /usr/bin/protoc (tried to install it from precompiled binary to /home/deckard/.local/bin/protoc, didn't helped). In both cases protoc-gen-go-grpc --version returns protoc-gen-go-grpc 1.1.0. I also tried to copy everything from google.golang.org/protobuf to google.golang.org/protobuf, no result.
So I deleted github.com/golang/protobuf folder, and seems like google.golang.org/protobuf has started indexing, and everything just started working.

protoc-gen-go-grpc: program not found or is not executable [protoc-gen-go is in PATH]

I am trying to generate Go source code from .proto file through this command:
protoc greet/greetpb/greet.proto --go-grpc_out=plugins=grpc:.
and, this is the directory structure:
greet/greetpb/greet.proto
I have tried all the answers mentioned here
But, none of them seems to be working.
I have verified that protoc-gen-go is in PATH.
Still, I am getting this:
protoc-gen-go-grpc: program not found or is not executable
Please specify a program using absolute path or make sure the program is available in your PATH system variable
--go-grpc_out: protoc-gen-go-grpc: Plugin failed with status code 1.
Additional details:
Go Version: go version go1.14.6 darwin/amd64
protoc version: libprotoc 3.12.3
My .zshrc file:
export GOROOT=/usr/local/go
export GOPATH=$HOME/go
export GOBIN=$GOPATH/bin
export PATH=$PATH:$GOROOT:$GOPATH:$GOBIN

How to write a Python 3 script to do .\activate in Windows 10 OS?

I'm trying to write a script to activate the virtual environment under a path, which is doing the same thing as the following command line:
cd C:\go\to\my\venv\path\Scripts
.\activate
My idea is first to change the directory to my Scripts folder, then execute the .\activate file. The problem is I have tried
1. os.system(".\\activate") #".\activate" is an error
2. subprocess.run(".\\activate")
3. subprocess.call(".\\activate")
4. subprocess.Popen(".\\activate", shell=True)
but all failed, and because I'm new to Python 3, so any suggestion will be appreciated, thanks.
Update 1:
Right now, it seems like can install one package successfully, but I have a list of packages need to install and I will figure it out how to do it. When I print out os.getcwd() after I activated the venv, there is no (venv) as the prefix in the result, so I don't know if the package was installed under venv or not?
Update 2:
Now, my problem is I'm using from pip._internal import main as pip
pip(['install', package_name])
It only can install one package successfully, if I pass a list of packages and use for loop to install them then they will fail. The error will be
Could not install packages due to an EnvironmentError: [Errno 2] No such file or directory: 'C:\Users\BO7D71~1.CHE\AppData\Local\Temp\pip-req-tracker-1nc0ofuc\5fc2bc8999692b42fb54fd13e7cbdf5f855b9718f5363d30a270bbe1'
Update 3:
Problem solved!

Dynamic Shared Library error when creating exe from cx_freeze

I have a python file using OpenCV library.I want this in as an executable file.
Came across this cx_freeze
Example:
ABC.py(file using opencv library)
Content of setup.py is given below
from cx_Freeze import setup, Executable
setup( name = "ABC",
version = "0.1",
description = "Testing",
executables = [Executable("ABC.py")],
)
I ran this command in the terminal
python setup.py install
It is stopping after showing this error message
copying /Users/name/.virtualenvs/test/lib/python3.6/site-packages/cv2.so -> build/exe.macosx-10.12-x86_64-3.6/lib/cv2.so
copying /Users/name/.virtualenvs/test/bin/../lib/libopencv_reg.3.3.dylib -> build/exe.macosx-10.12-x86_64-3.6/libopencv_reg.3.3.dylib
error: [Errno 2] No such file or directory: '/Users/name/.virtualenvs/test/bin/../lib/libopencv_reg.3.3.dylib'
I know from the error message that libopencv_reg.3.3.dylib is not found.
I'm running this on a python virtual environment.
This is my bash_profile content
#Homebrew
export PATH=/usr/local/bin:$PATH
#export WORKON_HOME=$HOME/.virtualenvs
# Virtualenv/VirtualenvWrapper
source /usr/local/bin/virtualenvwrapper.sh
#export PROJECT_HOME=$HOME/Devel
#export VIRTUALENVWRAPPER_PYTHON=/usr/local/bin/python
#export VIRTUALENVWRAPPER_VIRTUALENV=/usr/local/bin/virtualenv
#export VIRTUALENVWRAPPER_VIRTUALENV_ARGS='--no-site-packages'
I'm confused please help me out fix this.
Guidance would really be helpful, if this question can be posted else where let me know rather than down voting it.
can you search your filesystem for the missing file? When you find it you can set PERL5LIB or use -I switch to set the location into the perl path
perl library path
Shared the same on OpenCV forum
here's the response I got there
Looks like we have build OpenCV in such a way that other shared objects(.so) files are not dependent.

Virtualenv: No module named virtualenvwrapper, path is wrong

Why after installing virtualenv and virtualenvwrapper and adding
export WORKON_HOME=$HOME/.virtualenvs
export PROJECT_HOME=$HOME/Devel
source /Users/<user>/Library/Enthought/Canopy_64bit/User/bin/virtualenvwrapper.sh
to my .bash_profile and running $ source ~/.bash_profile, I encounter:
/Library/Frameworks/Python.framework/Versions/7.3/Resources/Python.app/Contents/MacOS/Python: No module named virtualenvwrapper
virtualenvwrapper.sh: There was a problem running the initialization hooks.
If Python could not import the module virtualenvwrapper.hook_loader,
check that virtualenv has been installed for
VIRTUALENVWRAPPER_PYTHON=/Library/Frameworks/Python.framework/Versions/Current/bin/python and that PATH is set properly.
?
My .bash_profile looks like:
# Setting PATH for EPD_free-7.3-2
# The orginal version is saved in .bash_profile.pysave
PATH="/Library/Frameworks/Python.framework/Versions/Current/bin:${PATH}"
export PATH
##
# Your previous /Users/<user>/.bash_profile file was backed up as /Users/<user>/.bash_profile.macports-saved_2012-11-06_at_11:39:22
##
# MacPorts Installer addition on 2012-11-06_at_11:39:22: adding an appropriate PATH variable for use with MacPorts.
export PATH=/opt/local/bin:/opt/local/sbin:$PATH
# Finished adapting your PATH environment variable for use with MacPorts.
# virtualenvwrapper
export WORKON_HOME=$HOME/.virtualenvs
export PROJECT_HOME=$HOME/Devel
source /Users/<user>/Library/Enthought/Canopy_64bit/User/bin/virtualenvwrapper.sh
# Added by Canopy installer on 2014-01-26
# VIRTUAL_ENV_DISABLE_PROMPT can be set to '' to make bashprompt show that Canopy is active, otherwise 1
VIRTUAL_ENV_DISABLE_PROMPT=1 source /Users/<user>/Library/Enthought/Canopy_64bit/User/bin/activate
So figured it out. It turns out that when I installed EPD for python a while back, it added the first few lines:
# Setting PATH for EPD_free-7.3-2
# The orginal version is saved in .bash_profile.pysave
PATH="/Library/Frameworks/Python.framework/Versions/Current/bin:${PATH}"
export PATH
in my .bash_profile. Having updated my python to Enthought Canopy, I needed to change these lines to this:
# set path for current python
PATH="/Users/<user>/Library/Enthought/Canopy_64bit/User/bin:${PATH}"
export PATH
and everything works now. However as it turns out that won't work at the end of the day because with Enthought, it backports venv, which is a python 3 standard as talked about here.

Resources