Layman's explanation of environment variables and OS shells - macos

Web development is new to me and I'm trying to grasp the meaning and usage of environment variables once and for all. In my research the most simple explanation I've come across is that it is comparable to 'configuration settings'.
Through the terminal I've been exploring what feels like the computer's innards by typing printenv etc etc.
But I'm still not sure when it is necessary to set up env var. For example, I use fish as my shell. Often when I try to do an npm install it seems like the package didn't take. Here is a recent example:
user#iMac-van-user ~/P/v/v/public_html> npm install -g modernizr
/usr/local/Cellar/node/10.4.0/bin/modernizr -> /usr/local/Cellar/node/10.4.0/lib/node_modules/modernizr/bin/modernizr
+ modernizr#3.6.0
updated 1 package in 2.316s
user#iMac-van-user ~/P/v/v/public_html> modernizr
When I try to use modernizr as a command fish will tell me the modernizr is an unknown command and the color remains red. Valid commands show up in white in fish. Thus I have a suspicion that modernizr will only be available and valid once I've set up the configs. I've had this happen many times with various attempts to install package managers and things like composer, vue-cli, etc. My failures to get it working boils down to my meager knowledge of environment variables and what they do, I think.
This is from the documentation on the modernizr site: modernizr -c modernizr-config.json
Note that you will need to give the command line config the file path
to the configuration you downloaded from the site. In the above
example, we are running the modernizr command from the same folder
that we downloaded the modernizr-config.json file to.
What does the sentence: "Note that you will need to give the command line config the file path to the configuration you downloaded from the site" imply? I copied the file into my project folder but there is no change.
Is there is someone that can explain the following to me in layman's terms, so like you would to a 5 year old, it would be great:
use of environment variables
setting up configs - why, where and how (I've done it once through VIM)
how to know when to set up environment variables
Thank you in advance.
Developing on macOS 10.

Related

stupid nube cant even do one simple code test [closed]

Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 10 months ago.
Improve this question
I decided recently that I want to start learning to code. I chose python as my first language because of it's versatility and Open Source connections. Now, of course to learn any programming language, first you have to be able to practice with it by writing code and running it.
I have Python beginner tutorials I am using but as soon as I got to the point where I wanted to run a simple Hello World message, I couldn't figure out how to get my document to actually RUN. I downloaded LiClipse with the hopes of using PyDev. I read that you can run a code directly from PyDev.
Wow, what a confusing mess! I downloaded the instant install version of LiClipse and I had to manually update the pip because it was out of date and that took an hour of research all on it's own to figure out how to do it.
Now that I have the interpreter set up correctly with the most recent pip...(I THINK), when I try to run any code I write, it doesn't let me. It wants me to save it first, which of course is quite logical. But when I try to save it, it says there is no folders to save to. How can there be no folders? I have been reading the install help, but it doesn't say anything about this issue. Or maybe it did explain how to fix this issue and I simply didn't understand all the jargon included in the configuration documentation. It kept instructing me to do technical things I couldn't understand.
Can anyone tell me what is going on? Shouldn't this all be ready to go when I used the "instant install" ?? I am just a beginner here, I don't know all this technical installation jargon. If anyone can help me get this setup, please let me know.
I really want to start learning. But I can't if I don't have the right tools.
Yeah, the technical jargon can be really confusing in the beginning. I've never heard of LiClipse but I can give you a few tips to get started:
Use a good IDE or Editor. For beginning VsCode with the Python PlugIn is quite good. You can also use PyCharm (the free version is enough) but that IDE is quite confusing.
Get comfortable with the commandline. On Windows you can invoke Python by running py on Linux with python3. You open the commandline on Windows by opening the startmenu, entering cmd and hitting enter.
You will have to make sure Python is in your "PATH" enviroment variable. That is a variable telling the Operating System where to find your programms. It should have asked you whether you want to add Python to your PATH during the install.
You run your python programm by saving it as a .py file and running py <filename.py> or python3 <filename.py> on Windows or linux respectively. You have to be in the directory where you saved your file of course.
I hope I could help you. Please mark this answer as accepted if it did answer your question or helped you with your issue.
I want to start by saying I'm mainly experienced in C & C++, not Python, but with programming in general many of the concepts carry over. I also don't have enough rep to comment, or I'd ask for some clarification on how familiar you are with the commandline.
I'm going to assume you're on Windows since you didn't mention the platform.
I wrote this like a tutorial, because I found those to be very helpful when I first started programming.
I can't recommend this enough, start off by using the commandline!
I think you complicated things more by searching for a text editor that can also run code. In order to get something like that working for your setup you need to know what's happening behind-the-scenes first.
Once you have a grasp of how to write & execute Python code, then you can move on to an IDE or advanced text editor like Visual Studio Code or LiClipse with confidence.
R+Click on the Start Menu, then click on Windows PowerShell to open a terminal.
Take a moment to note the filepath shown before the blinking cursor.
This is known as the Working Directory, which works similar to how the file explorer only allows you to "view" one directory at any given time; You can open multiple explorer windows side-by-side, but you can't view multiple directories at the same time in a single window.
You can run commands by typing them and pressing Enter.
You can also use the Tab key to autocomplete things like directory/file names, commands, etc. after typing a few characters.
Now let's create a workspace directory for your first project.
This can be anywhere that you choose, but for the sake of simplicity I'll be using my User directory which is usually where PowerShell will start by default.
To create a new directory named _workspace, use the mkdir (Make Directory) command like so:
mkdir _workspace
Now let's change our working directory to _workspace by using the cd (Change Directory) command like so:
cd _workspace
This directory is empty, but if you want to view the contents of a directory you can use the ls (List Directory) command:
ls
Since this is an empty directory, nothing will happen.
Next, we'll create a Python script file using the touch command, which creates empty files:
touch HelloWorld.py
Now that we have a script file, open HelloWorld.py in Notepad by running this command:
notepad HelloWorld.py
If you use a dark desktop theme, now is the time when your retinas may be seared out by the intensity of Notepad's white color scheme.
Now, let's add the code to print "Hello World!" in Python:
print("Hello World!")
Once you're done, you can save & close Notepad to save your retinas.
Now it's time to run the program! In the PowerShell window, enter this command:
py HelloWorld.py
which will show:
Hello World!
If you see an error saying this:
The term 'py' is not recognized as the name of a cmdlet, function, script file, or operable program.
You should re-install Python and make sure you check this box in the installer:
After re-installing Python and checking the Add to PATH box, close and reopen the PowerShell window, enter the cd _workspace command again, followed by py HelloWorld.py, which should be successful this time.
I hope I could be of some help, if so, I'd greatly appreciate marking this answer as accepted so I can write comments. Happy coding!
If you are using Linux following will work.
Install python3 first (Its better to stuck on python3 now. Once you get use-to then you can code in python2.7 also.)
Open a terminal, go the folder contains your code. If your source code is in hello.py then type python3 hello.py on terminal. It will execute and print the output on terminal.
The suggestions given by both #radj and #C. Dautermann are excellent.
Like others here, I will be assuming that you are on Windows.
Before jumping into LiClipse and PyDev, I highly recommend making sure that Python and packages (such as pip) are functioning on their own first. Command line is a great tool for this.
Windows Command Line can be accessed in a variety of ways, one of which is to go to your start menu and search "cmd." Command Prompt should return. In the command prompt, run the following:
python
This should return the version of Python installed on your machine if it is configured properly. For example, you may see the following:
Python 3.10.1 (...)[...] on win32
If this does not appear, I recommend uninstalling Python and reinstalling. In the installer, you will want to check the "Add Python ... to PATH" option. This will configure your environment variables on your machine. If you plan on running multiple versions of Python down the road, don't worry; you can use batch files to set the PATH variable to another version.
Once you have Python working, your command prompt will lead with the following:
>>>
Notice that there is no longer a file path followed by >>. This is because you are now in the Python interpreter. Congratulations!
To exit the Python interpreter, enter Ctrl+Z. It should appear as the following:
^Z
After we know Python works, we will check to see that pip works. Ensure you have pip installed by entering the following:
python -m pip --version
You should then be able to see the pip version. For example:
pip 22.0.3 from C:\...
If pip is not working, you can consult this documentation on pip to make sure pip gets installed properly.
Once pip is working, again, congratulations! You've gotten over some of the biggest hurdles. At this point, you can open up any text editor, type in:
print('Hello World!')
save the file with the extension .py, then run the script through command prompt. To run a script through the command line, you have to cd to the file location, then run the script with python, like so:
cd filepath
python filename.py
Where filepath is the path to the Python file and filename is the name of the file. For example:
cd C:\Users\name\Documents\PythonPractice
python helloworld.py
Usually, command line is all that is needed when starting out with Python outside of a text editor. Notepad comes standard on Windows computers, but there are plenty of other editors out there. Sublime text and Notepad++ are some of my favorites, personally.
Now, if you're able to run everything prior, the good news is that your Python and pip are both working, and you have everything you need to get started learning Python. If Python and/or pip are not functioning, there's thankfully a huge body of troubleshooting tips out there.
Unfortunately, from what I get of your initial post, you might be struggling with configuring LiClipse. I personally do not use LiClipse, nor do I use PyDev, so my help at this point may be limited. You may want to uninstall and reinstall after confirming your Python and pip work as a first step. After that, perhaps check that you are following the practices outlined in the LiClipse manual. Worst case scenario, you can try a different version of Python (I believe it has some Python3 incompatibilities), try running PyDev independent of LiClipse, or look at a different tool. If you believe the problem lies with file locations/existence, I would look at radj's post or play around in File Explorer to see what is going on for yourself.
Again, to get familiar with Python, you don't even need these extra tools. All you need is a text editor and the command line to get started.
Some other tools to help you get started:
PyPI
A venv virtual environment tutorial
PyDev
A comparison of LiClipse alternatives in case you continue to struggle
Hope this helps. Happy Coding!

Activate/deactivate conda virtualenvs on entering/leaving directories

pyenv-virtualenv offers a nice way of activating the environment on the very instant of entering or leaving the directory which contains a .python-version text file which specifies the environment to activate. It works for the directory it is in and all directories contained in it.
The environment is deactivated once we change the directory to something above it. This allows to easily switch between projects or analyses using different python versions (just by changing the directories).
Is there a way of achieving the same behaviour with (ana)conda?
Edit: added bash tag, because - as far as I understand - pyenv achieves this by hooking a custom script into .bashrc (which allows it to monitor the directory changes). If there is no build-in way in conda, how to create a script which would make it possible?
As mentioned in my comment, this is currently not supported. There is however an open issue on conda's GitHub asking for this feature.
In the meantime you could use autoenv, a small tool that'll automatically run the code in a .env file when entering a directory and that in a .env.leave when leaving the directory (supports bash/zsh and a couple others).
A simple example taken from their readme which illustrates the feature quite nicely:
$ echo "echo 'whoa'" > project/.env
$ cd project
whoa
To load a conda environment your .env would simply look like this:
conda activate <my_env>
Note 1: Check out the Configuration section of their GitHub readme before you start using it.
Note 2: The author of autoenv actually suggests trying direnv instead. However I've never used it, so I can't comment on it.
From autoenv's readme:
you should probably use direnv instead. Simply put, it is higher quality software. But, autoenv is still great, too. Maybe try both? :)

What is the difference between activating an anaconda environment and running its python executable directly?

I have setup multiple python environment using Anaconda.
Usually, to run a script "manually", I would open a command line and then type:
activate my-env
python path/to/my/script.py
Fine.
Now I am trying to run a script automatically using a scheduler and I was wondering what the difference was between
Writing a batch which activates the environment and the executes the script (like in the snippet above)
Calling directly the python executable from the environment (within the envs/my-enjv/ directory) like below:
/path/to/envs/my-env/python.exe path/to/my/script.py
Both seem to work fine. Is there any difference?
I don't claim to be an expert but here's my 2 cents.
For small scripts, no, there isn't a difference.
You should notice a difference when calling external modules / packages. conda activate alters the system path to change how the command shell searches for the appropriate capabilities.
If you supply a full path to an interpreter and the full path to an isolated script, then the shell doesn't need to do a lookup as this has priority over the path. This means you could be in a situation where the interpreter can see the script but cannot see dependencies.
If you follow the conda activate process, and the environment is correctly packaged, then the shell will be able to trace any additional resources.
EDIT: The idea behind this is portability. If an admin has been careful in setting up a system, then scripts should have the appropriate visibility - i.e. see everything in it's environment plus everything in the main system installation.
It's possible to full-path every call to an interpreter and a script or package location, but then what happens when you need to move it to another machine? You would need to spend a lot of time setting everything up exactly as it was before. On the other hand, you can follow the package process and the system path will trace everything for you.
Simply checkout the PATH variable in your environment. After conda activation it has been extended by
\Anaconda3;
\Anaconda3\Library\mingw-w64\bin;
\Anaconda3\Library\usr\bin;
\Anaconda3\Library\bin;
\Anaconda3\Scripts;
\Anaconda3\bin;
This doesn't make much of a difference, if you are just using the standard library in your code. However, if you rely on external packages like pandas, it's a prerequisite so that the modules can be found.

Making Sphinx documentation inside of a virtual environment with cron

I have an application development server that is automatically updated every night with a massive shell script that we run with crontab. The script specifies #!/bin/sh at the top of the file and I am not able to change that. The basic purpose of the script is to go through the machine and download the latest code in each of the directories that we list in the script. After all of the repositories are updated, we execute a number of scripts to update the relevant databases using the appropriate virtual environment (Django manage.py commands) by calling that virtualenv's python directly.
The issue that I am having is that we have all the necessary Sphinx plugins installed in one of the virtual environments to allow us to build the documentation from the code at the end of the script, but I cannot seem to figure out how to allow the make command to run inside of the virtualenv so that it has access to the proper packages and libraries. I need a way to run the make command inside of the virtual environment and if necessary deactivate that environment afterwards so that the remainder of the script can run.
My current script looks like the below and gives errors on the latter 3 lines, because sh does not have workon or deactivate, and because make can't find the sphinx-build.
cd ${_proj_root}/dev/docs
workon dev
make clean && make html
deactivate
I was able to find the answer to this question here. The error message that is shown when you attempt to build the sphinx documentation from the root is as follows, and leads to the answer that was provided there:
Makefile:12: *** The 'sphinx-build' command was not found. Make sure
you have Sphinx installed, then set the SPHINXBUILD environment
variable to point to the full path of the 'sphinx-build' executable.
Alternatively you can add the directory with the executable to your
PATH. If you don't have Sphinx installed, grab it from
http://sphinx-doc.org/. Stop.
The full command for anyone looking to build sphinx documentation through a cron when all tools are installed in various virtual environments are listed below. You can find the location of your python and sphinx-build commands by using which while the environment is activated.
make html SPHINXBUILD='<virtualenv-path-to>/python <virtualenv-path-to>/sphinx-build'

ImageMagick deployment on Mac OS

I'm trying to deploy ImageMagick with my own software. On windows I've just included all the core dlls with coders dlls at the exe path and it works well.
But on mac os I have troubles with coders. I installed ImageMagick via macports and found it with the help of CMake. CMake does all the job of copying and fixing up all the core libs I've linked against. Then I copied all the coder libs and fixed them up also, but when I start my application it just can't find any coder. So I'd like to know what am I missing there.
Note: if I didn't fix up any paths it works well. It is only my deployment that is in trouble. Maybe I should include some kind of config file?
P.S. I have all ImageMagick libs including coders SOs near the executable in MacOS bundle sub-folder.
How about setting the MAGICK_CODER_MODULE_PATH in your bundle?
see here: http://www.imagemagick.org/script/resources.php
EDIT:
To improve the information:
Originally when embedding IM in our own app bundle we had three problems:
our app and the IM dylibs not finding referenced IM dylibs,
IM not finding its config files,
IM not finding coders (the No Decode Delegate error)
We tried changing the hardcoded paths in the dylibs using the install_name_tool but finally when doing some tests with moving the IM around to different directories and testing
convert -debug configuration
we found out the all three above problems could be solved just by setting and exporting at least these three environment variables in the terminal console before running convert:
DYLD_LIBRARY_PATH
MAGICK_CONFIGURE_PATH
MAGICK_CODER_MODULE_PATH
With this experience, we returned back to our bundle and in the beginning tried to use the Info.plist fiel to set these variables but it didn't seem to work - probably because there were problems with making the paths to IM inside the bundle relative.
Finally we created a simple sh script and put it into our bundle and configured this bundle to run this script instead of the main app:
#!/bin/sh
CURR_DIR="$( cd -P "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
IMAGE_MAGICK_PATH=$CURR_DIR/../Resources/ImageMagick
export DYLD_LIBRARY_PATH=$IMAGE_MAGICK_PATH/lib
export MAGICK_CONFIGURE_PATH=$IMAGE_MAGICK_PATH/lib/ImageMagick-6.8.0/config
export MAGICK_CODER_MODULE_PATH=$IMAGE_MAGICK_PATH/lib/ImageMagick-6.8.0/modules-Q16/coders
# run application
exec $CURR_DIR/OurAppName
The key thing to make it working was properly getting the CURR_DIR of the app bundle (thanks to this post).
And as came out of our tests, setting the environment variables this way makes them visible only for this application execution context - i.e. when we started our app using the bundle, opened terminal and typed
env
the above three variables were missing from the output.
Hope this will help others save couple of days of research and pulling hairs out of their heads ;)
I've found a full solution for deploying ImageMagick in a bundle with the help of CMake. If you don't use CMake then #Tomasz's answer will be of help also.
So let's start:
First of all you need to know what and where ImageMagick is trying to locate when it is used from your own code. To find it out you can use MAGICK_DEBUG environmental variable which could be set to those parameters. It really helps when you debug ImageMagick.
Prerequisites:
I assume that you used FIND_PACKAGE and FIXUP_BUNDLE to find ImageMagick and set its binary paths inside the bundle. The only thing left is to deploy coders. Also I assume that you've installed ImageMagick from Mac Ports.
We need to get ImageMagick version string to correctly locate the coders:
STRING(REGEX REPLACE "-.+" "" ImageMagick_SHORT_VERSION ${ImageMagick_VERSION_STRING})
Now ImageMagick_SHORT_VERSION contains full version without any sub versions.
Then we need to copy all the coders to some predefined folder(I've used ImageMagick/coders subfolder under MacOS part of the bundle)
FILE(COPY /opt/local/lib/ImageMagick-${ImageMagick_SHORT_VERSION}/modules-Q16/coders/ DESTINATION ${PATH_TO_YOUR_BUNDLE}/Contents/MacOS/ImageMagick/coders/)
Now we need to fixup all the *.so libs we have, so we list it and pass to fixup_bundle
FILE(GLOB IMAGEMAGICK_CODERS ${PATH_TO_YOUR_BUNDLE}/Contents/MacOS/ImageMagick/coders/*.so)
Now we should update *.la files which accompanies coders *.so. To achieve it I've used script:
INSTALL(SCRIPT LaScript.cmake COMPONENT Runtime)
Script content:
SET(TARGET_BINARY_DIR "${PATH_TO_YOUR_BUNDLE}")
FILE(GLOB IMAGEMAGICK_CODERS_LA ${TARGET_BINARY_DIR}/Contents/MacOS/ImageMagick/coders/*.la)
FOREACH(file ${IMAGEMAGICK_CODERS_LA})
FILE(READ ${file} FILE_CONTENT)
STRING(REGEX REPLACE "dependency_libs='.*'" " " MODIFIED_FILE_CONTENT ${FILE_CONTENT})
STRING(REGEX REPLACE "libdir='.*'" " " MODIFIED_FILE_CONTENT ${MODIFIED_FILE_CONTENT})
FILE(WRITE ${file} ${MODIFIED_FILE_CONTENT})
ENDFOREACH()
We almost ready the only thing left to be done is to change the way we launch the application. But let's digress a little bit and find out where ImageMagick searches for the coders:
It tries to get the content of MAGICK_CODER_MODULE_PATH environmental variable
Then it checks if MAGICKCORE_CODER_PATH macro is defined(and in fact it does!) and use its value.
Then it will try to use MAGICK_HOME environmental variable and MAGICKCORE_CODER_RELATIVE_PATH to get path to the modules but we don't care since we will stop on #2 anyway!(NOTE: that it is true for Mac Ports installation)
So the only way we can interfere with search is to set MAGICK_CODER_MODULE_PATH environmental variable(Well we can also edit libMagickCodre and replace MAGICKCORE_CODER_PATH with some static path we need but it is too britle way to do things and it won't save us if someone set MAGICK_CODER_MODULE_PATH anyway)
We shouldn't set it system wide since we can break some user installtion so we have 2 options:
Use LSEnvironment to set the MAGICK_CODER_MODULE_PATH to some predefined location
Use script to launch our app and set this variable inside it.
I've chose the later since it is more flexible,
I have the following script:
#!/bin/bash
working_dir="${0%/*}"
export MAGICK_CODER_MODULE_PATH=$working_dir/ImageMagick/coders
executable="${working_dir}/ApplicationName"
"$executable"
and set CFBundleExecutable to the name of the script.
That's all and I hope it will help someone to save his/her time.
You should follow the Mac OS X-specific Build instructions but specifying --enable-shared in the configure options (see this document for details).
I guess that your application can't find the codecs because they have been statically linked to ImageMagick tools. This is usually done to address portability issues. To make codecs available in your application, you should build them as shared objects.

Resources