I'm using dask_jobqueue to establish a SLURMCluster. I'm trying to pass python files in the parent directory to the workers. I tried different ways including sys.path.append, setting PYTHONPATH in my .bashrc file, and setting PYTHONPATH in env_extra (it seems DASK version can't recognize job_script_prologue), but they all fail. The script works only when the imported Python files are in the same directory as the script. I just wonder if this is really the case (which means I should re-write my code to keep all files in the same directory), or I'm missing something here?
Also, I tried to use with performance_report to record some statistics in an HTML file, but it seems it does not work with dask_jobqueue, right?
Related
I'm following the Scripts with Support Files answer from here https://stackoverflow.com/a/46479538/4771016 which works great but running into a problem during the update of my script.
If not found, my script creates an .env file for the users to pass some variables in the same directory as the .sh file lives: /home/linuxbrew/.linuxbrew/Cellar/myscript/1.0.2/libexec/.env the problem is that upon releasing a new version, the .env file won't be in the new directory i.e. /home/linuxbrew/.linuxbrew/Cellar/myscript/1.0.3/libexec/ and thus will be recreated losing the modifications.
Any ideas for keeping that .env file during updates or an acceptable design pattern for my use case? I was thinking about keeping the .env file outside that directory somewhere, but I don't know the Homebrew directory structure well enough to store it in the right place.
I have a CGI Perl script that will run a select statement from a Oracle Database to get records. This scripts runs on Apache with cgi-bin linked. It was running fine.
Due to failover we moved the hard disk to the backup server with similar setups. When we run the script however, the following error is shown:
install_driver(Oracle) failed: Can't load '/u02/system/perl/usr/local/lib64/perl5/auto/DBD/Oracle/Oracle.so' for module DBD::Oracle: libclntsh.so.11.1: cannot open shared object file: No such file or directory at /u02/system/perl/usr/lib64/perl5/DynaLoader.pm line 200.
I had checked my ORACLE_HOME and LD_LIBRARY_PATH variables and they both pointing to the correct oracle client. Also try locate libclntsh.so.11.1 and managed to find the file in the correct directory, with permission granted.
Also added a oracle.conf file that has the path to oracle lib directory in /etc/ld.so.conf.d
All these and still it's showing the same error. Am running out of ideas....
Any pointers or suggestion are appreciated. Thank you.
You've covered the basics already which is good: make sure the libraries exist, make sure your program has read/execute permissions, and setting environment variables for ORACLE_HOME and LD_LIBRARY_PATH.
You many need to check the dependencies recursively.
ldd /u02/system/perl/usr/local/lib64/perl5/auto/DBD/Oracle/Oracle.so
should show you any dependencies of the .so that can't be loaded. DBD::Oracle is also version sensitive to the Oracle client. If the version or location of the Oracle client has changed (or when all else fails), you may need to recompile DBD::Oracle.
I've written a piece of code in PHP to generate PDF using WKHTMLTOPDF binary file. It was working fine till I had to recompile my Apache. Now it fails with error Error: Unable to create temporery file (this is the exact wording).
The situation in which the error is reproducible is a little complicated. I managed to narrow down the error and now I'm pretty sure that the error happens because of the user that Apache runs as. It seems to me that when WKTHMLTOPDF is running as a user with no home folder, it's unable to access a temporary folder within the user's home folder.
Surely I can change the Apache's user but I would rather resolve this problem once and for all. To this end it would be great if I could somehow set the temp folder for WKHTMLTOPDF or at least print its current value to make it valid! Does anyone know how to do any of these two?
BTW, I'm using WKHTMLTOPDF 0.11.0 rc1.
I saw the same error today in Rails4 + pdfkit gem(0.8.2) + wkhtmltopdf(0.12.2.1) under CentOS 6.7.
This error came from wkhtmltopdf and the reason was it couldn't create temporary file. wkhtmltopdf depends on some temporary filename creation API (I'm not sure), but probably following shows some hints:
$ man tempfile
$ man tempnam
In my case, my TMPDIR environment variable showed wrong path (I had accidentaly deleted the directory!) so that wkhtmltopdf couldn't create work file.
When I unset TMPDIR, then it worked! Of course, setting correct existence directory to TMPDIR should be OK too.
I am writing a system script in Ruby.
I'm using the classic gem structure: lib, bin, spec for RSpec.
I want to build a configurable script: I want to be able to provide options like --set-stuff and alike. A perfect example is:
git config --global user.name "Andrea"
which writes the given information out to a file, in order to be able to retrieve this information later.
How can I do this in a clean way?
I'd rather not use the environment variable solution: I know I could just set an env variable to point to a configuration file, but then I'd have to save this env variable in, say, .bashrc. Then again, how do I deal with zsh? Or how do I deal with people (like me) who keep their .bashrcs super-neat or even have a separate .env-variables file in their system?
Just stick the configuration into a Hash and serialize it into a file in the user's home directory as YAML or JSON...
This might not be a valid question. I was going through source of copy module at github.
While I could understand what and how it is doing one thing I am not able to get. I see following two lines
if not os.path.exists(src)
and
if os.path.exists(dest):
While I get that these lines are checking the presence of source and dest directories, how does python knows where to look these on as they are on different machines (the ansible server and host). How does python differentiates them and looks for them on their repective machines?
Can someone please help?
I think this copy module (liblary/file/copy) doesn't work.
Usually, when we use a command like this,
ansible webservers -m copy -a "src=/tmp/foo.conf dest=/tmp/bar.conf"
ansible will use this runnner module(lib/ansible/runner/action_plugins/copy.py) preferentially.
I tried running the same command, hiding the runner module. Then, (ansible/liblary/file/copy) module was executed. However, this did not do the work that is expected. There are several problems, this code is one of a cause.
if not os.path.exists(src)
if os.path.exists(dest):
Both check the file on the remote host.