How to import python script using ROS and unittest - python-2.x

I'm facing an issue in a ROS (Python 2.7) project.
Strucure
I'm working on a ROS project. The structure is the following:
probable-adventure
|
|-catkin_ws
|-src
|-ros_rover
|-include
|-launch
|-msg
|-__init__.py
|-Commands.msg
|-Teleoperation.msg
|-scripts
|-__init__.py
|-keyboard.py
|-test
|-__init__.py
|-testing.py
|-__init__.py
|-CMakeLists.txt
|-package.xml
Keyboard node (secondary problem)
keyboard.py starts as follows:
#!/usr/bin/env python
import rospy
from ros_rover.msg import Teleoperation
I describe it as a secondary problem because if I compile the project from catkin_ws directory using catkin_make command, everything works well and I can run the keyboard ROS node perfectly even though I get a pylint error in the from ros_rover.msg ... line.
Unable to import 'ros_rover.msg'
Main Problem
I just wrote the previous as context. The point is that I have to test the code within the scripts directory. I'm using unittest and I create python files in test directory.
I have create the following simple test (testing.py):
#!/usr/bin/env python
import unittest
from scripts.keyboard import GetKey
class TestBareBones(unittest.TestCase):
def test_one_equals_one(self):
self.assertEquals(1, 1, "1!=1")
if __name__ == '__main__':
unittest.main()
I think I have a problem with the imports because if I run the following python commands (from the ros_rover directory):
user#dubuntu:~/probable-adventure/catkin_ws/src/ros_rover$ python testing.py
user#dubuntu:~/probable-adventure/catkin_ws/src/ros_rover$ python -m testing
I get the following error:
File "/home/diego/Documents/probable-adventure/catkin_ws/src/ros_rover/test/prueba.py", line 5, in <module>
from scripts.keyboard import GetKey
File "scripts/keyboard.py", line 14, in <module>
from ros_rover.msg import Teleoperation
ImportError: No module named ros_rover.msg
My thought
I think the problem is related with imports because I have read that python imports depends on the directory from where you run the python command.
I run the test from ros_rover directory because the test's import is scripts.keyboard so it can found it (scripts is in ros_rover). But when the keyboard.py tries to import ros_rover.msg fails because I'm already in ros_rover directory and cannot find ros_rover.
I have read a lot here in StackOverflow and out of it but I didn't find a solution (I have added a empty __init__.py file to ros_rover, scripts and test directories).

Related

using "pythonw", ChromeDriverManager().install() is not working

from selenium.webdriver.chrome.service import Service as chromeService
from webdriver_manager.chrome import ChromeDriverManager
chromeServiceObj = chromeService(ChromeDriverManager().install())
this is my "test_install_driver.py" file.
and when i use "python" in a terminal(or command window), it works.
but using "pythonw", it doesn't.
I made a test code file.
And I was using the command below.
pythonw c:\test_install_driver.py
it seems like that the code doesn't install the web driver file.

ModuleNotFoundError seen after the first time a job is run on a Ray cluster

I'm running a script which imports a module from a file in the same directory. The first time I run the script after starting the cluster the script runs as expected. Any subsequent times I run the script I get the following error: ModuleNotFoundError: No module named 'ex_cls'
How do I get Ray to recognize modules I'm importing after the first run?
I am using Ray 1.11.0 on a redhat Linux cluster.
Here are my scripts. Both are located in the /home/ray_experiment directory:
--ex_main.py
import sys
sys.path.insert(0, '/home/ray_experiment')
from ex_cls import monitor_wrapper
import ray
ray.init(address='auto')
from ray.util.multiprocessing import Pool
def main():
pdu_infos = range(10)
with Pool() as pool:
results = pool.map(monitor_wrapper, [pdu for pdu in pdu_infos])
for pdu_info, result in zip(pdu_infos, results):
print(pdu_info, result)
if __name__ == "__main__":
main()
--ex_cls.py
import sys
from time import time, sleep
from random import randint
import collections
sys.path.insert(0, '/home/ray_experiment')
MonitorResult = collections.namedtuple('MonitorResult', 'key task_time')
def monitor_wrapper(args):
start = time()
rando = randint(0, 200)
lst = []
for i in range(10000 * rando):
lst.append(i)
pause = 1
sleep(pause)
return MonitorResult(args, time() - start)
-- Edit
I've found that by adding these two environment variables I no longer see the ModuleNotFoundError.
export PYTHONPATH="${PYTHONPATH}:/home/ray_experiment/"
export RAY_RUNTIME_ENV_WORKING_DIR_CACHE_SIZE_GB=0
Is there another solution that doesn't require disabling the working environment caching?
The issue here is that Ray's worker processes may be run from different working directories than your driver python script. In fact, on a cluster, they may even be run from different machines. This is coupled by the fact that python looks for the module based on a relative path (to be precise, cloudpickle serializes definitions in other modules by reference).
The "intended" solution to this problem is to use runtime environments.
In particular, you should do ray.init(address='auto', runtime_env={"working_dir": "./"}) when starting Ray to ensure that the module is passed to other processes.
After searching for similar error I found this in Ray docs:
https://docs.ray.io/en/latest/ray-core/handling-dependencies.html#library-development
Adding __init__.py to root of these directories and handing them over with this kwarg solved my issue with ModuleNotFoundError. Another part of solution is to install all requirements to the container the ray is running in and probably starting it in same workdir as app (or passing workdir in runtime kwargs)
To answer OPs problem I would try something like this:
import ex_cls
# it cannot pass classes, just whole module
ray.init(address='auto', runtime_env={"py_modules": [ex_cls]})

Can future's absolute_import clear __package__? [closed]

Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 3 years ago.
Improve this question
I have a Python 2 Pyramid App that is setup using buildout and a project folder activated using Mr.Developer; I am trying to futurize this project as part of Python 3 migration which changes:
import test
to:
from __future__ import absolute_import
from . import test
However for some reason ./bin/pserve development.ini doesn't come up, it says:
...
File "/apps/src/project/engine/config.py", line 3, in <module>
import utilities
File "/apps/src/project/engine/utilities.py", line 9, in <module>
from project.engine import spreadsheets
File "/apps/src/project/engine/spreadsheets.py", line 16, in <module>
from project.engine import utilities
ImportError: cannot import name utilities
It is a proper package as far as I am aware because it does have a setup.py and I ran develop activate project + buildout again.
If I type print(__package__) in that code it prints fine, but if I add a line from __future__ import absolute_import it prints None. Is it possible that future's absolute_import can simply clears out the __package__ variable and that is why it is not detecting this as a package?
This is the MWE of the real issue: Why does this circular import fail in Python 2 but not in Python 3?, it seems to be a bug in Python 2 when there is circular import. If anyone has a clean solution please let me know.
When __package__ is set to None, that's just a flag value for not yet set. It is not a problem.
From PEP 366 – Main module explicit relative imports, on the subject of __package__:
When the import system encounters an explicit relative import in a module without __package__ set (or with it set to None), it will calculate and store the correct value (__name__.rpartition('.')[0] for normal modules and __name__ for package initialisation modules). If __package__ has already been set then the import system will use it in preference to recalculating the package name from the __name__ and __path__ attributes.
If you see this set to None in the main module and it was meant to be part of a package, then set it yourself:
if __name__ == "__main__" and __package__ is None:
__package__ = "foo.bar.baz"

Q: Getting Build Error "Invalid Import Path"

I'm stuck on running the BeeGO app using "bee run" it says
The thing is I've already have setup properly my GOPATH to D:/Web Dev/GO/BeeGO/test-project/
and also routers path does exist and I've tried to manual build the file but it doesn't generate an .exe file.
Anyone knows how to fix this?
I'm using Windows 8.1 Pro (64-bit)
Thanks
GO expects the directory structure under $GOPATH in following ways as described in code organization:
$GOPATH/src <--- where your source code goes
/pkg
/bin
Instead of placing your source files directly under $GOPATH (D:/Web Dev/GO/BeeGO/test-project/ for your case), you want to move your code under $GOPATH/src.
D:/Web Dev/GO/BeeGO/test-project/src/main.go
D:/Web Dev/GO/BeeGO/test-project/src/quickstart/routers/routers.go
D:/Web Dev/GO/BeeGO/test-project/src/quickstart/controllers/controllers.go
import path should be always starting from $GOPATH/src. routers.go can be always imported as import "quickstart/routers" and controllers.go can be imported as import "quickstart/controllers".
That's not how you import a package.
The import path is relative to $GOPATH/src. use:
import "quickstart/routers"
Finally fixed the bug from the framework,
What I did:
in main.go import from
"D:/Web Dev/GO/BeeGO/test-project/quickstart/routers"
I changed it to _"../quickstart/routers" make sure to include _ this means to import the library even if it is not used,
Then in the routers/router.go I changed the import path
"D:/Web Dev/GO/BeeGO/test-project/quickstart/controllers" to "../controllers"
It seems BeeGO doesn't generate the template properly and caused the build to fail.
Another possiblity for this error, is when you copy-paste code from the internet,
and
import "quickstart/routers"
became
import "quickstart/routers "
due to bugs in some CMS/Blog systems (notice the space at the end before the closing quote...).

Embedded Python loads module but does not load that module's internal import statements

At long last(!) I've compiled Boost::Python and have gotten my XCode project to import a local module. This module starts with the line from xml.dom import minidom, but when it executes, I'm given this error:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "FeedStore.py", line 1, in <module>
from xml.dom import minidom
ImportError: No module named xml.dom
However, I know that I've installed the xml Python module -- when I open Python from my command prompt and type from xml.dom import minidom, everything goes smoothly. Moreover, when I import the module, it behaves as I would expect.
I suspected that there was something wrong with sys.path, so I compared the one I get from the prompt to the one that's being used in my embedded module. The only difference is that the embedded sys.path does not include ''. I've tried appending it, but that didn't change the behavior.
I also suspected that the embedded version was accessing a different version of Python than I was using from the prompt, but sys.prefix matched between both executions.
Here's the code that imports my module and runs it. It's pretty bare-bones at the moment (not even reference counting yet) because at this point I'd just like to make sure I'll be able to embed my module (I'm a total newbie C++ programmer).
Py_Initialize();
//PyRun_SimpleString("import sys");
//PyRun_SimpleString("sys.path.append('')"); //tried this to no avail!
PySys_SetPath("/Users/timoooo/Documents/Code/TestEmbed/"); //this allows me to import my local module
PyRun_SimpleString("import FeedStore as fs"); //here's where it whines about the lack of xml.dom
PyRun_SimpleString("store = fs.feedStore()");
PyRun_SimpleString("print store.next()");
Py_Finalize();
I'm probably misunderstanding something essential about boost::python. Can anyone help me out?
Despite having identical sys.path values, calling
PyRun_SimpleString("sys.path.append(\"<<path>>\")");
with the places I needed fixed the problem.

Resources