I have an application development server that is automatically updated every night with a massive shell script that we run with crontab. The script specifies #!/bin/sh at the top of the file and I am not able to change that. The basic purpose of the script is to go through the machine and download the latest code in each of the directories that we list in the script. After all of the repositories are updated, we execute a number of scripts to update the relevant databases using the appropriate virtual environment (Django manage.py commands) by calling that virtualenv's python directly.
The issue that I am having is that we have all the necessary Sphinx plugins installed in one of the virtual environments to allow us to build the documentation from the code at the end of the script, but I cannot seem to figure out how to allow the make command to run inside of the virtualenv so that it has access to the proper packages and libraries. I need a way to run the make command inside of the virtual environment and if necessary deactivate that environment afterwards so that the remainder of the script can run.
My current script looks like the below and gives errors on the latter 3 lines, because sh does not have workon or deactivate, and because make can't find the sphinx-build.
cd ${_proj_root}/dev/docs
workon dev
make clean && make html
deactivate
I was able to find the answer to this question here. The error message that is shown when you attempt to build the sphinx documentation from the root is as follows, and leads to the answer that was provided there:
Makefile:12: *** The 'sphinx-build' command was not found. Make sure
you have Sphinx installed, then set the SPHINXBUILD environment
variable to point to the full path of the 'sphinx-build' executable.
Alternatively you can add the directory with the executable to your
PATH. If you don't have Sphinx installed, grab it from
http://sphinx-doc.org/. Stop.
The full command for anyone looking to build sphinx documentation through a cron when all tools are installed in various virtual environments are listed below. You can find the location of your python and sphinx-build commands by using which while the environment is activated.
make html SPHINXBUILD='<virtualenv-path-to>/python <virtualenv-path-to>/sphinx-build'
Related
pyenv-virtualenv offers a nice way of activating the environment on the very instant of entering or leaving the directory which contains a .python-version text file which specifies the environment to activate. It works for the directory it is in and all directories contained in it.
The environment is deactivated once we change the directory to something above it. This allows to easily switch between projects or analyses using different python versions (just by changing the directories).
Is there a way of achieving the same behaviour with (ana)conda?
Edit: added bash tag, because - as far as I understand - pyenv achieves this by hooking a custom script into .bashrc (which allows it to monitor the directory changes). If there is no build-in way in conda, how to create a script which would make it possible?
As mentioned in my comment, this is currently not supported. There is however an open issue on conda's GitHub asking for this feature.
In the meantime you could use autoenv, a small tool that'll automatically run the code in a .env file when entering a directory and that in a .env.leave when leaving the directory (supports bash/zsh and a couple others).
A simple example taken from their readme which illustrates the feature quite nicely:
$ echo "echo 'whoa'" > project/.env
$ cd project
whoa
To load a conda environment your .env would simply look like this:
conda activate <my_env>
Note 1: Check out the Configuration section of their GitHub readme before you start using it.
Note 2: The author of autoenv actually suggests trying direnv instead. However I've never used it, so I can't comment on it.
From autoenv's readme:
you should probably use direnv instead. Simply put, it is higher quality software. But, autoenv is still great, too. Maybe try both? :)
Web development is new to me and I'm trying to grasp the meaning and usage of environment variables once and for all. In my research the most simple explanation I've come across is that it is comparable to 'configuration settings'.
Through the terminal I've been exploring what feels like the computer's innards by typing printenv etc etc.
But I'm still not sure when it is necessary to set up env var. For example, I use fish as my shell. Often when I try to do an npm install it seems like the package didn't take. Here is a recent example:
user#iMac-van-user ~/P/v/v/public_html> npm install -g modernizr
/usr/local/Cellar/node/10.4.0/bin/modernizr -> /usr/local/Cellar/node/10.4.0/lib/node_modules/modernizr/bin/modernizr
+ modernizr#3.6.0
updated 1 package in 2.316s
user#iMac-van-user ~/P/v/v/public_html> modernizr
When I try to use modernizr as a command fish will tell me the modernizr is an unknown command and the color remains red. Valid commands show up in white in fish. Thus I have a suspicion that modernizr will only be available and valid once I've set up the configs. I've had this happen many times with various attempts to install package managers and things like composer, vue-cli, etc. My failures to get it working boils down to my meager knowledge of environment variables and what they do, I think.
This is from the documentation on the modernizr site: modernizr -c modernizr-config.json
Note that you will need to give the command line config the file path
to the configuration you downloaded from the site. In the above
example, we are running the modernizr command from the same folder
that we downloaded the modernizr-config.json file to.
What does the sentence: "Note that you will need to give the command line config the file path to the configuration you downloaded from the site" imply? I copied the file into my project folder but there is no change.
Is there is someone that can explain the following to me in layman's terms, so like you would to a 5 year old, it would be great:
use of environment variables
setting up configs - why, where and how (I've done it once through VIM)
how to know when to set up environment variables
Thank you in advance.
Developing on macOS 10.
I have setup multiple python environment using Anaconda.
Usually, to run a script "manually", I would open a command line and then type:
activate my-env
python path/to/my/script.py
Fine.
Now I am trying to run a script automatically using a scheduler and I was wondering what the difference was between
Writing a batch which activates the environment and the executes the script (like in the snippet above)
Calling directly the python executable from the environment (within the envs/my-enjv/ directory) like below:
/path/to/envs/my-env/python.exe path/to/my/script.py
Both seem to work fine. Is there any difference?
I don't claim to be an expert but here's my 2 cents.
For small scripts, no, there isn't a difference.
You should notice a difference when calling external modules / packages. conda activate alters the system path to change how the command shell searches for the appropriate capabilities.
If you supply a full path to an interpreter and the full path to an isolated script, then the shell doesn't need to do a lookup as this has priority over the path. This means you could be in a situation where the interpreter can see the script but cannot see dependencies.
If you follow the conda activate process, and the environment is correctly packaged, then the shell will be able to trace any additional resources.
EDIT: The idea behind this is portability. If an admin has been careful in setting up a system, then scripts should have the appropriate visibility - i.e. see everything in it's environment plus everything in the main system installation.
It's possible to full-path every call to an interpreter and a script or package location, but then what happens when you need to move it to another machine? You would need to spend a lot of time setting everything up exactly as it was before. On the other hand, you can follow the package process and the system path will trace everything for you.
Simply checkout the PATH variable in your environment. After conda activation it has been extended by
\Anaconda3;
\Anaconda3\Library\mingw-w64\bin;
\Anaconda3\Library\usr\bin;
\Anaconda3\Library\bin;
\Anaconda3\Scripts;
\Anaconda3\bin;
This doesn't make much of a difference, if you are just using the standard library in your code. However, if you rely on external packages like pandas, it's a prerequisite so that the modules can be found.
Good morning, I am reading about the prepared scripts in MacOSX to use when creating a pkg for my application.
In particular, I have some doubts how to make sure that postupgrade script is used.
What I read till now is:
from here
The postupgrade script is run after files have been installed and before the postflight script if one is defined. This script is run only if the component has been previously installed. If the script does not return an exit status of zero, Installer will declare the installation has failed.
Ok then it seems that postupgrade will just run in automatic when an upgrade is done. BUT...from man pkgbuild, section --scripts scripts-path
Archive the entire contents of scripts-path as the package scripts. If this directory contains scripts named preinstall and/or postinstall, these will be run as the top-level scripts of the package. If you want to run scripts for specific bundles, you must specify those in a component property list; see more at COMPONENT PROPERTY LIST. Any other files under scripts-path will be used only if the top-level or component-specific scripts invoke them.
So, it seems I should add it to the component.plist, since they do not say anything about postupgrade. BUT it seems strange, I would put there more specific script, not the postupgrade script.
Reading more, I found it that refers to this, where there is written:
To determine whether a Package has already been installed or not, Installer.app is having a look at the content of the following directory: /Library/Receipts. If there's a file named PackageName.pkg within it, then the Package has already been installed, otherwise it's the first install.
Well, my application leaves no pkg file there, but yes it is present in the InstallHistory.plist.
Well, finally the question: should I set the upgrade script somewhere, for example in the component.plist file? The last link seems to be out of date, something has changed? How can I put a pkg file inside /Library/Receipts? Or better, how can be sure if my installation is indeed an installation and not an upgrade, or viceversa?
Thanks everyone, I am a bit confused...
I'm following the instructions from various Wikis on how to compile Node so I can eventually get it running as a service on Windows.
My steps so far:
https://github.com/joyent/node/wiki/Installation
(which lead to...)
http://blog.tatham.oddie.com.au/2011/03/16/node-js-on-windows/
(successfully compiled via cygwin, but lead to...)
https://github.com/joyent/node/wiki/Building-node.js-on-mingw
(which apparently is better than the so far successful cygwin compile)
So - I've managed to compile Node.exe using Cygwin but not the preferred Mingw. I concur this isn't an ideal situation, building on Windows isn't the ideal. Nevertheless.
The error I see in Mingw, once I've followed all of the steps above, occurs when I try to ./configure --without-ssl. The error message is:
Danjah#PC /c/cygwin/home/Danjah/node-v0.4.7/node
$ ./configure –without-ssl
/usr/bin/env: python: No such file or directory
I understand from step 3's URL, that I must take steps to provide the environment variables for both Python and Git - using help from the provided URL I managed to input the Python path var, but I don't think I have the Git path var right. Either way, in no install directories for Python, Cygwin or Mingw32 do I see the path specified in the error msg: "/usr/bin/env".
Googling didn't really bring much to the table in terms of env variables or Mingw32, best I got was: PATH=C:\MinGW\bin;C:\MinGW\msys\1.0\bin where my install directory is at C:\MingW\.
The path I added to Windows environment vars for Python was: PythonPath=C:\Python27;C:\Python27\DLLs;C:\Python27\Lib;C:\Python27\Lib\lib-tk where Python 2.7 is installed in C:\Python27\.
I hate it when a file path stops you from doing things, as I suspect is the problem here. So please set me straight here - is it a file path problem I have or something else? And if its something else, please try and help me to get Node up and running... keen as to get experimenting.
I should probably also mention that I do also have a previously installed version of Git on my Windows XP SP3 machine, but had not previously had Cygwin, Mingw32 or Python installed, and I do not have IIS running as a service - my usual testing environment is a WAMP stack.
Windows uses the PATH environment variable to locate programs that are invoked without a fully qualified file path, i.e. 'python' rather than 'C:\Python27\python'.
So you need to add python's home directory to the Windows PATH variable, as well as MinGW, git and anything else your script requires.
Also by setting the PATH variable explicitly in your shell session or script, you are overwriting its original contents (in the local context) which limits which programs your shell can find to only those available within the PATH which is usually a bad idea.
See http://www.java.com/en/download/help/path.xml for details on modifying your PATH so you can always run your Python scripts from the command line.