I try to import and run selfmade modules under Spyder.
The main.py programm
import rot
f=rotor(b_x,b_y,b_z,x,y,z)
The rot.py programm
def rotor(bx,by,bz,x,y,z):
import numpy as np
...
The issue is:
File "C:\Users\Professional.spyder-py3\my_pro\jgut\main.py", line 93, in
f=rotor(b_x,b_y,b_z,x,y,z)
NameError: name 'rotor' is not defined
Problem doesn't disappear even after starting brut force start of any imported modules.
Is there a way to start all imported modules simultaniously? Or maybesomeone faced this problem?
P.S. Problem appeared today after Anaconda reinstall, becouse of incopatebilities with IK-Multimedia apps (I just removed them). I was starting all the mudules by brut force from the begining.
1) Please, do not import modules in functions!
2) rotor is not defined because it is in the rot namespace. You can call it by rot.rotor(args). If you wanted to do call it the way you do, you have to import it like this: from rot import rotor or from rot import * (which imports everything from the rot module and is a bad practise - you should only import what you need).
The error has nothing to do with anything written in the last 2 paragraphs.
Related
I'm running a script which imports a module from a file in the same directory. The first time I run the script after starting the cluster the script runs as expected. Any subsequent times I run the script I get the following error: ModuleNotFoundError: No module named 'ex_cls'
How do I get Ray to recognize modules I'm importing after the first run?
I am using Ray 1.11.0 on a redhat Linux cluster.
Here are my scripts. Both are located in the /home/ray_experiment directory:
--ex_main.py
import sys
sys.path.insert(0, '/home/ray_experiment')
from ex_cls import monitor_wrapper
import ray
ray.init(address='auto')
from ray.util.multiprocessing import Pool
def main():
pdu_infos = range(10)
with Pool() as pool:
results = pool.map(monitor_wrapper, [pdu for pdu in pdu_infos])
for pdu_info, result in zip(pdu_infos, results):
print(pdu_info, result)
if __name__ == "__main__":
main()
--ex_cls.py
import sys
from time import time, sleep
from random import randint
import collections
sys.path.insert(0, '/home/ray_experiment')
MonitorResult = collections.namedtuple('MonitorResult', 'key task_time')
def monitor_wrapper(args):
start = time()
rando = randint(0, 200)
lst = []
for i in range(10000 * rando):
lst.append(i)
pause = 1
sleep(pause)
return MonitorResult(args, time() - start)
-- Edit
I've found that by adding these two environment variables I no longer see the ModuleNotFoundError.
export PYTHONPATH="${PYTHONPATH}:/home/ray_experiment/"
export RAY_RUNTIME_ENV_WORKING_DIR_CACHE_SIZE_GB=0
Is there another solution that doesn't require disabling the working environment caching?
The issue here is that Ray's worker processes may be run from different working directories than your driver python script. In fact, on a cluster, they may even be run from different machines. This is coupled by the fact that python looks for the module based on a relative path (to be precise, cloudpickle serializes definitions in other modules by reference).
The "intended" solution to this problem is to use runtime environments.
In particular, you should do ray.init(address='auto', runtime_env={"working_dir": "./"}) when starting Ray to ensure that the module is passed to other processes.
After searching for similar error I found this in Ray docs:
https://docs.ray.io/en/latest/ray-core/handling-dependencies.html#library-development
Adding __init__.py to root of these directories and handing them over with this kwarg solved my issue with ModuleNotFoundError. Another part of solution is to install all requirements to the container the ray is running in and probably starting it in same workdir as app (or passing workdir in runtime kwargs)
To answer OPs problem I would try something like this:
import ex_cls
# it cannot pass classes, just whole module
ray.init(address='auto', runtime_env={"py_modules": [ex_cls]})
Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 3 years ago.
Improve this question
I have a Python 2 Pyramid App that is setup using buildout and a project folder activated using Mr.Developer; I am trying to futurize this project as part of Python 3 migration which changes:
import test
to:
from __future__ import absolute_import
from . import test
However for some reason ./bin/pserve development.ini doesn't come up, it says:
...
File "/apps/src/project/engine/config.py", line 3, in <module>
import utilities
File "/apps/src/project/engine/utilities.py", line 9, in <module>
from project.engine import spreadsheets
File "/apps/src/project/engine/spreadsheets.py", line 16, in <module>
from project.engine import utilities
ImportError: cannot import name utilities
It is a proper package as far as I am aware because it does have a setup.py and I ran develop activate project + buildout again.
If I type print(__package__) in that code it prints fine, but if I add a line from __future__ import absolute_import it prints None. Is it possible that future's absolute_import can simply clears out the __package__ variable and that is why it is not detecting this as a package?
This is the MWE of the real issue: Why does this circular import fail in Python 2 but not in Python 3?, it seems to be a bug in Python 2 when there is circular import. If anyone has a clean solution please let me know.
When __package__ is set to None, that's just a flag value for not yet set. It is not a problem.
From PEP 366 – Main module explicit relative imports, on the subject of __package__:
When the import system encounters an explicit relative import in a module without __package__ set (or with it set to None), it will calculate and store the correct value (__name__.rpartition('.')[0] for normal modules and __name__ for package initialisation modules). If __package__ has already been set then the import system will use it in preference to recalculating the package name from the __name__ and __path__ attributes.
If you see this set to None in the main module and it was meant to be part of a package, then set it yourself:
if __name__ == "__main__" and __package__ is None:
__package__ = "foo.bar.baz"
So I have been working for 2 months on a project in python (it is the first time I'm using python).
The project is basically:
- 1 main file,
- 1 GUI file using PyQt4
- and a few classes for the calculation functions.
Here is my setup.py file:
from distutils.core import setup
import py2exe
from glob import glob
import matplotlib
dataf = [
(r'mpl-data',
[r'C:\Python27\Lib\site-packages\matplotlib-1.4.3-py2.7-win32.egg\matplotlib\mpl-data\matplotlibrc']),
(r'mpl-data\stylelib',
glob(r'C:\Python27\Lib\site-packages\matplotlib-1.4.3-py2.7-win32.egg\matplotlib\mpl-data\stylelib\*.*')),
(r'mpl-data\images',
glob(r'C:\Python27\Lib\site-packages\matplotlib-1.4.3-py2.7-win32.egg\matplotlib\mpl-data\images\*.*')),
(r'mpl-data\fonts',
glob(r'C:\Python27\Lib\site-packages\matplotlib\mpl-data\fonts\*.*')),
(r'Qt\imageformats',
glob(r'C:\Python27\Lib\site-packages\PyQt4\plugins\imageformats\*.*')),
(r'Resource',
glob(r'C:\PythonWS\Lince5M\*.png')),
]
includes = ['sip', 'PyQt4.QtCore', 'PyQt4.Qt', 'PyQt4.QtGui', 'PyQt4.QtNetwork',]
pack = ['numpy.core', 'matplotlib.backends', 'matplotlib.axes', 'zmq', 'PyQt4.QtCore', 'PyQt4.Qt',
'PyQt4.QtGui', 'PyQt4.QtNetwork', 'CaptureImage', 'ClusterDetectionIterative', 'DisplayPanel',
'ImageProcessAvg', 'WriteOutputFiles',
]
excludes = ['_gtkagg', '_agg2', '_cairo', '_cocoaagg', '_fltkagg', '_gtk', '_gtkcairo', '_thread',
'_macosx',
]
#I admit not understand really much those excludes, that's why I don't put them in the setup
opts={
'py2exe': {
'compressed': 2,
'optimize': 2,
'includes':includes,
'packages':pack,
#'excludes':excludes,
'bundle_files': 1,
}
}
win = [{
'script': 'Application.py',
'icon_resourses': [(1,'icone\icone.png')]
}
]
setup(
version = "0.1",
console=win,
options=opts,
data_files=dataf,
)
My programs works fine when launched with Eclipse.
The making of the executable seems to be done fine too, except for the list of missing modules there is no error, and my .exe is created...
But when I start it nothing happens... I have been looking for a few days now and I really have no idea on what is wrong since nothing gives me a notice on the problem..
If it could be of any use, my project uses those imports:
PyQt4, os, decimal, time, numpy, sys, clr, matplotlib, cv2, and a dll we made in the company.
So here is my question:
Anyone ever got this kind of error?
Does any one know how to get an output, or anything that could tell me where is the problem?
There should be a [.exe name].log file in the "dist" folder that py2exe created.
Try looking to see if that can glean some information on your problem.
I'm stuck on running the BeeGO app using "bee run" it says
The thing is I've already have setup properly my GOPATH to D:/Web Dev/GO/BeeGO/test-project/
and also routers path does exist and I've tried to manual build the file but it doesn't generate an .exe file.
Anyone knows how to fix this?
I'm using Windows 8.1 Pro (64-bit)
Thanks
GO expects the directory structure under $GOPATH in following ways as described in code organization:
$GOPATH/src <--- where your source code goes
/pkg
/bin
Instead of placing your source files directly under $GOPATH (D:/Web Dev/GO/BeeGO/test-project/ for your case), you want to move your code under $GOPATH/src.
D:/Web Dev/GO/BeeGO/test-project/src/main.go
D:/Web Dev/GO/BeeGO/test-project/src/quickstart/routers/routers.go
D:/Web Dev/GO/BeeGO/test-project/src/quickstart/controllers/controllers.go
import path should be always starting from $GOPATH/src. routers.go can be always imported as import "quickstart/routers" and controllers.go can be imported as import "quickstart/controllers".
That's not how you import a package.
The import path is relative to $GOPATH/src. use:
import "quickstart/routers"
Finally fixed the bug from the framework,
What I did:
in main.go import from
"D:/Web Dev/GO/BeeGO/test-project/quickstart/routers"
I changed it to _"../quickstart/routers" make sure to include _ this means to import the library even if it is not used,
Then in the routers/router.go I changed the import path
"D:/Web Dev/GO/BeeGO/test-project/quickstart/controllers" to "../controllers"
It seems BeeGO doesn't generate the template properly and caused the build to fail.
Another possiblity for this error, is when you copy-paste code from the internet,
and
import "quickstart/routers"
became
import "quickstart/routers "
due to bugs in some CMS/Blog systems (notice the space at the end before the closing quote...).
At long last(!) I've compiled Boost::Python and have gotten my XCode project to import a local module. This module starts with the line from xml.dom import minidom, but when it executes, I'm given this error:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "FeedStore.py", line 1, in <module>
from xml.dom import minidom
ImportError: No module named xml.dom
However, I know that I've installed the xml Python module -- when I open Python from my command prompt and type from xml.dom import minidom, everything goes smoothly. Moreover, when I import the module, it behaves as I would expect.
I suspected that there was something wrong with sys.path, so I compared the one I get from the prompt to the one that's being used in my embedded module. The only difference is that the embedded sys.path does not include ''. I've tried appending it, but that didn't change the behavior.
I also suspected that the embedded version was accessing a different version of Python than I was using from the prompt, but sys.prefix matched between both executions.
Here's the code that imports my module and runs it. It's pretty bare-bones at the moment (not even reference counting yet) because at this point I'd just like to make sure I'll be able to embed my module (I'm a total newbie C++ programmer).
Py_Initialize();
//PyRun_SimpleString("import sys");
//PyRun_SimpleString("sys.path.append('')"); //tried this to no avail!
PySys_SetPath("/Users/timoooo/Documents/Code/TestEmbed/"); //this allows me to import my local module
PyRun_SimpleString("import FeedStore as fs"); //here's where it whines about the lack of xml.dom
PyRun_SimpleString("store = fs.feedStore()");
PyRun_SimpleString("print store.next()");
Py_Finalize();
I'm probably misunderstanding something essential about boost::python. Can anyone help me out?
Despite having identical sys.path values, calling
PyRun_SimpleString("sys.path.append(\"<<path>>\")");
with the places I needed fixed the problem.