Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 3 years ago.
Improve this question
I have a Python 2 Pyramid App that is setup using buildout and a project folder activated using Mr.Developer; I am trying to futurize this project as part of Python 3 migration which changes:
import test
to:
from __future__ import absolute_import
from . import test
However for some reason ./bin/pserve development.ini doesn't come up, it says:
...
File "/apps/src/project/engine/config.py", line 3, in <module>
import utilities
File "/apps/src/project/engine/utilities.py", line 9, in <module>
from project.engine import spreadsheets
File "/apps/src/project/engine/spreadsheets.py", line 16, in <module>
from project.engine import utilities
ImportError: cannot import name utilities
It is a proper package as far as I am aware because it does have a setup.py and I ran develop activate project + buildout again.
If I type print(__package__) in that code it prints fine, but if I add a line from __future__ import absolute_import it prints None. Is it possible that future's absolute_import can simply clears out the __package__ variable and that is why it is not detecting this as a package?
This is the MWE of the real issue: Why does this circular import fail in Python 2 but not in Python 3?, it seems to be a bug in Python 2 when there is circular import. If anyone has a clean solution please let me know.
When __package__ is set to None, that's just a flag value for not yet set. It is not a problem.
From PEP 366 – Main module explicit relative imports, on the subject of __package__:
When the import system encounters an explicit relative import in a module without __package__ set (or with it set to None), it will calculate and store the correct value (__name__.rpartition('.')[0] for normal modules and __name__ for package initialisation modules). If __package__ has already been set then the import system will use it in preference to recalculating the package name from the __name__ and __path__ attributes.
If you see this set to None in the main module and it was meant to be part of a package, then set it yourself:
if __name__ == "__main__" and __package__ is None:
__package__ = "foo.bar.baz"
Related
I'm running a script which imports a module from a file in the same directory. The first time I run the script after starting the cluster the script runs as expected. Any subsequent times I run the script I get the following error: ModuleNotFoundError: No module named 'ex_cls'
How do I get Ray to recognize modules I'm importing after the first run?
I am using Ray 1.11.0 on a redhat Linux cluster.
Here are my scripts. Both are located in the /home/ray_experiment directory:
--ex_main.py
import sys
sys.path.insert(0, '/home/ray_experiment')
from ex_cls import monitor_wrapper
import ray
ray.init(address='auto')
from ray.util.multiprocessing import Pool
def main():
pdu_infos = range(10)
with Pool() as pool:
results = pool.map(monitor_wrapper, [pdu for pdu in pdu_infos])
for pdu_info, result in zip(pdu_infos, results):
print(pdu_info, result)
if __name__ == "__main__":
main()
--ex_cls.py
import sys
from time import time, sleep
from random import randint
import collections
sys.path.insert(0, '/home/ray_experiment')
MonitorResult = collections.namedtuple('MonitorResult', 'key task_time')
def monitor_wrapper(args):
start = time()
rando = randint(0, 200)
lst = []
for i in range(10000 * rando):
lst.append(i)
pause = 1
sleep(pause)
return MonitorResult(args, time() - start)
-- Edit
I've found that by adding these two environment variables I no longer see the ModuleNotFoundError.
export PYTHONPATH="${PYTHONPATH}:/home/ray_experiment/"
export RAY_RUNTIME_ENV_WORKING_DIR_CACHE_SIZE_GB=0
Is there another solution that doesn't require disabling the working environment caching?
The issue here is that Ray's worker processes may be run from different working directories than your driver python script. In fact, on a cluster, they may even be run from different machines. This is coupled by the fact that python looks for the module based on a relative path (to be precise, cloudpickle serializes definitions in other modules by reference).
The "intended" solution to this problem is to use runtime environments.
In particular, you should do ray.init(address='auto', runtime_env={"working_dir": "./"}) when starting Ray to ensure that the module is passed to other processes.
After searching for similar error I found this in Ray docs:
https://docs.ray.io/en/latest/ray-core/handling-dependencies.html#library-development
Adding __init__.py to root of these directories and handing them over with this kwarg solved my issue with ModuleNotFoundError. Another part of solution is to install all requirements to the container the ray is running in and probably starting it in same workdir as app (or passing workdir in runtime kwargs)
To answer OPs problem I would try something like this:
import ex_cls
# it cannot pass classes, just whole module
ray.init(address='auto', runtime_env={"py_modules": [ex_cls]})
I'm facing an issue in a ROS (Python 2.7) project.
Strucure
I'm working on a ROS project. The structure is the following:
probable-adventure
|
|-catkin_ws
|-src
|-ros_rover
|-include
|-launch
|-msg
|-__init__.py
|-Commands.msg
|-Teleoperation.msg
|-scripts
|-__init__.py
|-keyboard.py
|-test
|-__init__.py
|-testing.py
|-__init__.py
|-CMakeLists.txt
|-package.xml
Keyboard node (secondary problem)
keyboard.py starts as follows:
#!/usr/bin/env python
import rospy
from ros_rover.msg import Teleoperation
I describe it as a secondary problem because if I compile the project from catkin_ws directory using catkin_make command, everything works well and I can run the keyboard ROS node perfectly even though I get a pylint error in the from ros_rover.msg ... line.
Unable to import 'ros_rover.msg'
Main Problem
I just wrote the previous as context. The point is that I have to test the code within the scripts directory. I'm using unittest and I create python files in test directory.
I have create the following simple test (testing.py):
#!/usr/bin/env python
import unittest
from scripts.keyboard import GetKey
class TestBareBones(unittest.TestCase):
def test_one_equals_one(self):
self.assertEquals(1, 1, "1!=1")
if __name__ == '__main__':
unittest.main()
I think I have a problem with the imports because if I run the following python commands (from the ros_rover directory):
user#dubuntu:~/probable-adventure/catkin_ws/src/ros_rover$ python testing.py
user#dubuntu:~/probable-adventure/catkin_ws/src/ros_rover$ python -m testing
I get the following error:
File "/home/diego/Documents/probable-adventure/catkin_ws/src/ros_rover/test/prueba.py", line 5, in <module>
from scripts.keyboard import GetKey
File "scripts/keyboard.py", line 14, in <module>
from ros_rover.msg import Teleoperation
ImportError: No module named ros_rover.msg
My thought
I think the problem is related with imports because I have read that python imports depends on the directory from where you run the python command.
I run the test from ros_rover directory because the test's import is scripts.keyboard so it can found it (scripts is in ros_rover). But when the keyboard.py tries to import ros_rover.msg fails because I'm already in ros_rover directory and cannot find ros_rover.
I have read a lot here in StackOverflow and out of it but I didn't find a solution (I have added a empty __init__.py file to ros_rover, scripts and test directories).
I try to import and run selfmade modules under Spyder.
The main.py programm
import rot
f=rotor(b_x,b_y,b_z,x,y,z)
The rot.py programm
def rotor(bx,by,bz,x,y,z):
import numpy as np
...
The issue is:
File "C:\Users\Professional.spyder-py3\my_pro\jgut\main.py", line 93, in
f=rotor(b_x,b_y,b_z,x,y,z)
NameError: name 'rotor' is not defined
Problem doesn't disappear even after starting brut force start of any imported modules.
Is there a way to start all imported modules simultaniously? Or maybesomeone faced this problem?
P.S. Problem appeared today after Anaconda reinstall, becouse of incopatebilities with IK-Multimedia apps (I just removed them). I was starting all the mudules by brut force from the begining.
1) Please, do not import modules in functions!
2) rotor is not defined because it is in the rot namespace. You can call it by rot.rotor(args). If you wanted to do call it the way you do, you have to import it like this: from rot import rotor or from rot import * (which imports everything from the rot module and is a bad practise - you should only import what you need).
The error has nothing to do with anything written in the last 2 paragraphs.
When ever I try to doctest in python, basically whenever I run the code
if __name__ =="__main__":
import doctest
doctest.testmod()
I get this response from the interpreter
AttributeError: 'module' object has no attribute 'testmod'
I can run this code just fine, but whenever I run it on my windows machine, it doesn't work.
My machine is running Windows theirs is OS X, but are running python 2.7.5.
Thank you :)
Make sure that you are not trying to save your test file as doctest.py. The print statement suggested above will show it. If the file name is doctest.py, then rename it and try again.
AttributeError: 'module' object has no attribute 'testmod'
Clearly stats that the doctest module you're importing do not has the testmod() method.
Possible reasons can be:
You have more than one doctest modules in the lib.
and, it is the other one (without the testmod() method) which is getting imported as result of import doctest.
Solution: Look for the path of standard doctest module.
if __name__ =="__main__":
import doctest
if doctest.__file__ == "/path/to/standard/doctest-module":
doctest.testmod()
It looks like there is a different module called doctest that is being imported instead of the standard one.
To find out which module is being imported exactly, simply add the following print:
if __name__ =="__main__":
import doctest
print doctest.__file__ # add this
doctest.testmod()
The print should produce something similar to C:\Python27\lib\doctest.pyc, depending on the location and version of Python you're using. Any other output means you are importing the wrong module, and explain why you're getting the error.
At long last(!) I've compiled Boost::Python and have gotten my XCode project to import a local module. This module starts with the line from xml.dom import minidom, but when it executes, I'm given this error:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "FeedStore.py", line 1, in <module>
from xml.dom import minidom
ImportError: No module named xml.dom
However, I know that I've installed the xml Python module -- when I open Python from my command prompt and type from xml.dom import minidom, everything goes smoothly. Moreover, when I import the module, it behaves as I would expect.
I suspected that there was something wrong with sys.path, so I compared the one I get from the prompt to the one that's being used in my embedded module. The only difference is that the embedded sys.path does not include ''. I've tried appending it, but that didn't change the behavior.
I also suspected that the embedded version was accessing a different version of Python than I was using from the prompt, but sys.prefix matched between both executions.
Here's the code that imports my module and runs it. It's pretty bare-bones at the moment (not even reference counting yet) because at this point I'd just like to make sure I'll be able to embed my module (I'm a total newbie C++ programmer).
Py_Initialize();
//PyRun_SimpleString("import sys");
//PyRun_SimpleString("sys.path.append('')"); //tried this to no avail!
PySys_SetPath("/Users/timoooo/Documents/Code/TestEmbed/"); //this allows me to import my local module
PyRun_SimpleString("import FeedStore as fs"); //here's where it whines about the lack of xml.dom
PyRun_SimpleString("store = fs.feedStore()");
PyRun_SimpleString("print store.next()");
Py_Finalize();
I'm probably misunderstanding something essential about boost::python. Can anyone help me out?
Despite having identical sys.path values, calling
PyRun_SimpleString("sys.path.append(\"<<path>>\")");
with the places I needed fixed the problem.