I know that I have Node installed but I don't remember if I used NVM, Brew or if I downloaded it from the official Node website.
Is there any command I can use to figure out how I installed it in the first place?
Note: I use a zsh shell on macOS
A good indicator is to check where it is installed:
ls -l $(which node)
For brew, this will yield something like
lrwxr-xr-x 1 flyx admin 30 Sep 12 17:02 /usr/local/bin/node -> ../Cellar/node/16.9.1/bin/node
NVM seems to install it somewhere in ~/.nvm but I don't know the specifics. I have no idea what the installer does.
What's the best way to download a python package and it's dependencies from pypi for offline installation on another machine? Is there any easy way to do this with pip or easy_install? I'm trying to install the requests library on a FreeBSD box that is not connected to the internet.
On the system that has access to internet
The pip download command lets you download packages without installing them:
pip download -r requirements.txt
(In previous versions of pip, this was spelled pip install --download -r requirements.txt.)
On the system that has no access to internet
Then you can use
pip install --no-index --find-links /path/to/download/dir/ -r requirements.txt
to install those downloaded modules, without accessing the network.
If you want install python libs and their dependencies offline, finish following these steps on a machine with the same os, network connected, and python installed:
1) Create a requirements.txt file with similar content (Note - these are the libraries you wish to download):
Flask==0.12
requests>=2.7.0
scikit-learn==0.19.1
numpy==1.14.3
pandas==0.22.0
One option for creating the requirements file is to use pip freeze > requirements.txt. This will list all libraries in your environment. Then you can go in to requirements.txt and remove un-needed ones.
2) Execute command mkdir wheelhouse && pip download -r requirements.txt -d wheelhouse to download libs and their dependencies to directory wheelhouse
3) Copy requirements.txt into wheelhouse directory
4) Archive wheelhouse into wheelhouse.tar.gz with tar -zcf wheelhouse.tar.gz wheelhouse
Then upload wheelhouse.tar.gz to your target machine:
1) Execute tar -zxf wheelhouse.tar.gz to extract the files
2) Execute pip install -r wheelhouse/requirements.txt --no-index --find-links wheelhouse to install the libs and their dependencies
If the package is on PYPI, download it and its dependencies to some local directory.
E.g.
$ mkdir /pypi && cd /pypi
$ ls -la
-rw-r--r-- 1 pavel staff 237954 Apr 19 11:31 Flask-WTF-0.6.tar.gz
-rw-r--r-- 1 pavel staff 389741 Feb 22 17:10 Jinja2-2.6.tar.gz
-rw-r--r-- 1 pavel staff 70305 Apr 11 00:28 MySQL-python-1.2.3.tar.gz
-rw-r--r-- 1 pavel staff 2597214 Apr 10 18:26 SQLAlchemy-0.7.6.tar.gz
-rw-r--r-- 1 pavel staff 1108056 Feb 22 17:10 Werkzeug-0.8.2.tar.gz
-rw-r--r-- 1 pavel staff 488207 Apr 10 18:26 boto-2.3.0.tar.gz
-rw-r--r-- 1 pavel staff 490192 Apr 16 12:00 flask-0.9-dev-2a6c80a.tar.gz
Some packages may have to be archived into similar looking tarballs by hand. I do it a lot when I want a more recent (less stable) version of something. Some packages aren't on PYPI, so same applies to them.
Suppose you have a properly formed Python application in ~/src/myapp. ~/src/myapp/setup.py will have install_requires list that mentions one or more things that you have in your /pypi directory. Like so:
install_requires=[
'boto',
'Flask',
'Werkzeug',
# and so on
If you want to be able to run your app with all the necessary dependencies while still hacking on it, you'll do something like this:
$ cd ~/src/myapp
$ python setup.py develop --always-unzip --allow-hosts=None --find-links=/pypi
This way your app will be executed straight from your source directory. You can hack on things, and then rerun the app without rebuilding anything.
If you want to install your app and its dependencies into the current python environment, you'll do something like this:
$ cd ~/src/myapp
$ easy_install --always-unzip --allow-hosts=None --find-links=/pypi .
In both cases, the build will fail if one or more dependencies aren't present in /pypi directory. It won't attempt to promiscuously install missing things from Internet.
I highly recommend to invoke setup.py develop ... and easy_install ... within an active virtual environment to avoid contaminating your global Python environment. It is (virtualenv that is) pretty much the way to go. Never install anything into global Python environment.
If the machine that you've built your app has same architecture as the machine on which you want to deploy it, you can simply tarball the entire virtual environment directory into which you easy_install-ed everything. Just before tarballing though, you must make the virtual environment directory relocatable (see --relocatable option). NOTE: the destination machine needs to have the same version of Python installed, and also any C-based dependencies your app may have must be preinstalled there too (e.g. say if you depend on PIL, then libpng, libjpeg, etc must be preinstalled).
Let me go through the process step by step:
On a computer connected to the internet, create a folder.
$ mkdir packages
$ cd packages
open up a command prompt or shell and execute the following command:
Suppose the package you want is tensorflow
$ pip download tensorflow
Now, on the target computer, copy the packages folder and apply the following command
$ cd packages
$ pip install 'tensorflow-xyz.whl' --no-index --find-links '.'
Note that the tensorflow-xyz.whl must be replaced by the original name of the required package.
offline python. for doing this I use virtualenv (isolated Python environment)
1) install virtualenv
online with pip:
pip install virtualenv --user
or offline with whl: go to this link , download last version (.whl or tar.gz) and install that with this command:
pip install virtualenv-15.1.0-py2.py3-none-any.whl --user
by using --user you don't need to use sudo pip….
2) use virtualenv
on online machine select a directory with terminal cd and run this code:
python -m virtualenv myenv
cd myenv
source bin/activate
pip install Flask
after installing all the packages, you have to generate a requirements.txt so while your virtualenv is active, write
pip freeze > requirements.txt
open a new terminal and create another env like myenv2.
python -m virtualenv myenv2
cd myenv2
source bin/activate
cd -
ls
now you can go to your offline folder where your requirements.txt and tranferred_packages folder are in there. download the packages with following code and put all of them to tranferred_packages folder.
pip download -r requirements.txt
take your offline folder to offline computer and then
python -m virtualenv myenv2
cd myenv2
source bin/activate
cd -
cd offline
pip install --no-index --find-links="./tranferred_packages" -r requirements.txt
what is in the folder offline [requirements.txt , tranferred_packages {Flask-0.10.1.tar.gz, ...}]
check list of your package
pip list
note: as we are in 2017 it is better to use python 3. you can create python 3 virtualenv with this command.
virtualenv -p python3 envname
I had a similar problem. And i had to make it install the same way, we do from pypi.
I did the following things:
Make a directory to store all the packages in the machine that have internet access.
mkdir -p /path/to/packages/
Download all the packages to the path
Edit: You can also try:
python3 -m pip wheel --no-cache-dir -r requirements.txt -w /path/to/packages
pip download -r requirements.txt -d /path/to/packages
Eg:- ls /root/wheelhouse/ # **/root/wheelhouse** is my **/path/to/packages/**
total 4524
-rw-r--r--. 1 root root 16667 May 23 2017 incremental-17.5.0-py2.py3-none-any.whl
-rw-r--r--. 1 root root 34713 Sep 1 10:21 attrs-18.2.0-py2.py3-none-any.whl
-rw-r--r--. 1 root root 3088398 Oct 15 14:41 Twisted-18.9.0.tar.bz2
-rw-r--r--. 1 root root 133356 Jan 28 15:58 chardet-3.0.4-py2.py3-none-any.whl
-rw-r--r--. 1 root root 154154 Jan 28 15:58 certifi-2018.11.29-py2.py3-none-any.whl
-rw-r--r--. 1 root root 57987 Jan 28 15:58 requests-2.21.0-py2.py3-none-any.whl
-rw-r--r--. 1 root root 58594 Jan 28 15:58 idna-2.8-py2.py3-none-any.whl
-rw-r--r--. 1 root root 118086 Jan 28 15:59 urllib3-1.24.1-py2.py3-none-any.whl
-rw-r--r--. 1 root root 47229 Jan 28 15:59 tqdm-4.30.0-py2.py3-none-any.whl
-rw-r--r--. 1 root root 7922 Jan 28 16:13 constantly-15.1.0-py2.py3-none-any.whl
-rw-r--r--. 1 root root 164706 Jan 28 16:14 zope.interface-4.6.0-cp27-cp27mu-manylinux1_x86_64.whl
-rw-r--r--. 1 root root 573841 Jan 28 16:14 setuptools-40.7.0-py2.py3-none-any.whl
-rw-r--r--. 1 root root 37638 Jan 28 16:15 Automat-0.7.0-py2.py3-none-any.whl
-rw-r--r--. 1 root root 37905 Jan 28 16:15 hyperlink-18.0.0-py2.py3-none-any.whl
-rw-r--r--. 1 root root 52311 Jan 28 16:15 PyHamcrest-1.9.0-py2.py3-none-any.whl
-rw-r--r--. 1 root root 10586 Jan 28 16:15 six-1.12.0-py2.py3-none-any.whl
Tar the packages directory and copy it to the Machine that doesn't have internet access. Then do,
cd /path/to/packages/
tar -cvzf packages.tar.gz . # not the . (dot) at the end
Copy the packages.tar.gz into the destination machine that doesn't have internet access.
In the machine that doesn't have internet access, do the following (Assuming you copied the tarred packages to /path/to/package/ in the current machine)
cd /path/to/packages/
tar -xvzf packages.tar.gz
mkdir -p $HOME/.config/pip/
vi $HOME/.config/pip/pip.conf
and paste the following content inside and save it.
[global]
timeout = 10
find-links = file:///path/to/package/
no-cache-dir = true
no-index = true
Finally, i suggest you use, some form of virtualenv for installing the packages.
virtualenv -p python2 venv # use python3, if you are on python3
source ./venv/bin/activate
pip install <package>
You should be able to download all the modules that are in the directory /path/to/package/.
Note: I only did this, because i couldn't add options or change the way we install the modules. Otherwise i'd have done
pip install --no-index --find-links /path/to/download/dir/ -r requirements.txt
Download the tarball, transfer it to your FreeBSD machine and extract it, afterwards run python setup.py install and you're done!
EDIT: Just to add on that, you can also install the tarballs with pip now.
Using wheel compiled packages.
bundle up:
$ tempdir=$(mktemp -d /tmp/wheelhouse-XXXXX)
$ pip wheel -r requirements.txt --wheel-dir=$tempdir
$ cwd=`pwd`
$ (cd "$tempdir"; tar -cjvf "$cwd/bundled.tar.bz2" *)
copy tarball and install:
$ tempdir=$(mktemp -d /tmp/wheelhouse-XXXXX)
$ (cd $tempdir; tar -xvf /path/to/bundled.tar.bz2)
$ pip install --force-reinstall --ignore-installed --upgrade --no-index --no-deps $tempdir/*
Note wheel binary packages are not across machines.
More ref. here: https://pip.pypa.io/en/stable/user_guide/#installation-bundles
For Windows I have used below things
Internet Connection
1.Create directory with any name.I have created with "repo"
2.Download libraries using below command (it will download not install)
pip download libraray_name -d"C:\repo"
pip download openpyxl -d"C:\repo"
Then you will find multiple .whl extension files
copy all the filename in requirements.txt
No Internet Connection
Now Move this folder and files to PC where no internet connection and run the below command.
pip install -r requirements.txt --find-links=C:\repo --no-index
You can read the detailed blog Link
As a continues to #chaokunyang answer, I want to put here the script I write that does the work:
Write a "requirements.txt" file that specifies the libraries you want to pack.
Create a tar file that contains all your libraries (see the Packer script).
Put the created tar file in the target machine and untar it.
run the Installer script (which is also packed into the tar file).
"requirements.txt" file
docker==4.4.0
Packer side: file name: "create-offline-python3.6-dependencies-repository.sh"
#!/usr/bin/env bash
# This script follows the steps described in this link:
# https://stackoverflow.com/a/51646354/8808983
LIBRARIES_DIR="python3.6-wheelhouse"
if [ -d ${LIBRARIES_DIR} ]; then
rm -rf ${LIBRARIES_DIR}/*
else
mkdir ${LIBRARIES_DIR}
fi
pip download -r requirements.txt -d ${LIBRARIES_DIR}
files_to_add=("requirements.txt" "install-python-libraries-offline.sh")
for file in "${files_to_add[#]}"; do
echo "Adding file ${file}"
cp "$file" ${LIBRARIES_DIR}
done
tar -cf ${LIBRARIES_DIR}.tar ${LIBRARIES_DIR}
Installer side: file name: "install-python-libraries-offline.sh"
#!/usr/bin/env bash
# This script follows the steps described in this link:
# https://stackoverflow.com/a/51646354/8808983
# This file should run during the installation process from inside the libraries directory, after it was untared:
# pythonX-wheelhouse.tar -> untar -> pythonX-wheelhouse
# |
# |--requirements.txt
# |--install-python-libraries-offline.sh
pip3 install -r requirements.txt --no-index --find-links .
For Pip 8.1.2 you can use pip download -r requ.txt to download packages to your local machine.
Download the wheel file (for example dlb-0.5.0-py3-none-any.whl) from Pypi and
pip install dlb-0.5.0-py3-none-any.whl
I need compile some LEX/YACC files(*.l) under OS X. And GNU flex is needed as a scanner.
However, I was stuck while installing GNU flex.
Run brew install flex, but got an error:
Error: You must `brew link xz' before flex can be installed
Then I run brew link xz, still got an error:
Error: Could not symlink lib/pkgconfig/liblzma.pc
/usr/local/lib/pkgconfig is not writable.
How to install flex on OS X 10.10 correctly? Is this problem caused by my home brew?
Some details about my 'brew':
Run brew doctor
Warning: /usr/local/lib/pkgconfig isn't writable.
This can happen if you "sudo make install" software that isn't managed by
by Homebrew. If a formula tries to write a file to this directory, the
install will fail during the link step.
You should probably `chown` /usr/local/lib/pkgconfig
Run ls command
yeze#yezedeMacBook-Pro:~$ ls -la /usr/local/lib/pkgconfig
total 16
drwxr-xr-x 4 root wheel 136 Mar 31 2013 .
drwxr-xr-x 30 yeze admin 1020 Oct 1 21:05 ..
-rw-r--r-- 1 root wheel 405 Mar 30 2013 tcl.pc
-rw-r--r-- 1 root wheel 404 Mar 30 2013 tk.pc
This question is caused by brew.
When you got /usr/local/lib/pkgconfig is not writable., you should run:
chown [YourAccountName] /usr/local/lib/pkgconfig
Then follow the instruction, run brew link xz. You may get a response like that :Linking /usr/local/Cellar/xz/5.2.1... 53 symlinks created
Finally, try brew install flex again, it will work.
Best thanks #IKavanagh.
Ok, well it's just been one of those nights where you spend hours and hours trying to get something to work, and you just keep getting weird errors, so if someone could help me I would greatly appreciate. After hours of trying to update Maven from 3.0.4 to 3.1.1 or 3.2.2 I've decided it's just not going to happen (I've tried almost everything I can find online, but I'm afraid to try to do to much in terminal) and I'm trying to install homebrew to make it easier. When I try to do the normal homebrew install I get an error that says:
dyld: lazy symbol binding failed: Symbol not found: ___strlcpy_chk
Referenced from: /usr/local/git/bin/git
Expected in: /usr/lib/libSystem.B.dylib
dyld: Symbol not found: ___strlcpy_chk
Referenced from: /usr/local/git/bin/git
Expected in: /usr/lib/libSystem.B.dylib
Failed during: git init -q
So then, I tried to do the alternative install method, but at this point I'm just so annoyed, and I don't get what it wants me to do. If anyone could give me some at least kind of detailed explanation for what to do I would be extremely grateful.
Here's where I'm looking at alternative installs: https://github.com/Homebrew/homebrew/wiki/Installation
The problem is I don't even really know what they mean by "untar" and "extract."
Thanks so much to anyone who can help
When I enter
ls -l /usr/local | pbcopy
I get:
total 0
drwxr-xr-x 2 root wheel 68 Aug 11 03:34 apache-maven
drwxrwxr-x 81 root admin 2754 Jan 17 2014 bin
drwxrwxr-x 3 root admin 102 Feb 21 2013 etc
drwxr-xr-x 9 root wheel 306 Jul 25 14:54 git
drwxrwxr-x 3 root admin 102 Feb 21 2013 lib
drwxrwxr-x 4 root admin 136 Feb 20 2013 share
drwxr-xr-x 4 root wheel 136 Dec 14 2013 texlive
Edited
Ok, so let's do an alternative install of homebrew:
cd /usr/local
mkdir homebrew
curl -L https://github.com/Homebrew/homebrew/tarball/master | tar xz --strip 1 -C homebrew
If that works, you just need to find the directory where the file brew is located and then add that to your PATH
So
find /usr/local -name brew
Let's suppose the previous command results in
/usr/local/homebrew/bin/brew
we take brew off the end (because we want to know its directory only) and we add that to the start of our PATH
export PATH=/usr/local/homebrew/bin:$PATH
Now we should be able to run
brew doctor
Also, we need to add that export PATH=.... command to our login sequence so our shell knows how to find brew every time we login. So add that line to the end of your ~/.profile
Original Answer
Ok, take a deep breath, and relax :-)
homebrew is a great choice on the Mac, so the pain should be worth it. I suspect you have a customised PATH and customised environment variables which are stopping the homebrew installation. You can either set your PATH and environment variables back to their default settings, or, if that is simpler, just add a new user to your Mac and log in as the new user with a standard environment and then install homebrew using the standard method.
To look at your PATH and environment variables, use these commands:
echo $PATH
set
or
set | grep -i LIB
to look for any customised DYLD_LIBRARY_PATH
Once you have it installed, try running
brew doctor
to check your setup before adding maven and other packages.
I am running ant 1.8.2 and I want to up date to the latest on my mac. What is the best way to do this? Should I remove then install or can i write over the previous?
I use it in both Eclipse and from the command line. I also have jenkins call ant targets. It's currently set in /usr/local
I found these two links to update Ant on mac:
Seventy6.com
Ninjascript.com
I had Ant 1.8.4 installed on my Mac 10.8.5 and upgraded to:
Apache Ant(TM) version 1.9.4 compiled on April 29 2014
The first website from seventy6 said all of the below, with a few updates from myself.
To see what version you currently have installed run this:
$ ant -version
Apache Ant version 1.7.1 compiled on February 11 2010
I was advised to use the simple OSX package installer homebrew. I’m not going to discuss how to install this, as they have a really simple guide. However, it’s not obvious how to install Apache Ant as it’s not one of the packages they list. So here is the brew to get started:
$brew install https://raw.github.com/adamv/homebrew-alt/master/duplicates/ant.rb
EDIT: You can also just install now with homebrew just by typing the following
$brew install ant
Beware, you might need to install Apple’s XCode, which if you’re still running OS X 10.6 or lower (like me) you can’t install the latest version (4) via the wonderful (?) AppStore. Ha! fun and games! Luckily, I have an Apple Developer account which allows you to download archived releases of Apple’s software. I created this ages ago and assume they’re still free to do… Once installed it puts the package here:
/usr/local/Cellar/ant/1.8.2
I then scratched my head for a while as to how to get OSX to use the newly installed version. It seems the easiest option is to remove the symlink currently set for the system. Run this to see where this is:
$ whereis ant
/usr/bin/ant //returned message
So we need to remove the symlink and set it to our new installed copy. I found this article which got me started. Useful, but not perfect for a homebrew package install. So here is the final commands to change the symlink to the homebrew version:
$ cd /usr/share
$ sudo rm /usr/share/ant
Password:
************
$ ln -s /usr/local/Cellar/ant/1.8.2 ant
You now need to close your current terminal session and open a new one. In theory you should be able to run 'ant -version' to get the latest version…
$ ant -version
Apache Ant(TM) version 1.8.2 compiled on December 20 2010
I’m not sure if it’s best way to do it. But it’s a simple technique which can easily be updated if needed. At least you haven’t removed the base Java install of Ant from your system!
DONE
Also from blog.ninjascript.com, I saw this was interesting to know about:
Now Ant is kind of buried in OS X; the $PATH variable points to a symlink which points to another symlink. To find out where ant really is, just follow the chain:
$ which ant
/usr/bin/ant
$ ls -la /usr/bin/ant
lrwxr-xr-x 1 root wheel 22 Nov 11 18:04 /usr/bin/ant -> /usr/share/ant/bin/ant
$ ls -la /usr/share/ant
lrwxr-xr-x 1 root wheel 14 Nov 11 18:04 /usr/share/ant -> java/ant-1.7.1
$ ls -la /usr/share/java/ant-1.7.1
total 40
drwxr-xr-x 8 root wheel 272 Feb 27 12:32 .
drwxr-xr-x 8 root wheel 272 Nov 11 18:04 ..
-rw-r--r-- 1 root wheel 15289 Feb 10 2010 LICENSE.txt
-rw-r--r-- 1 root wheel 1270 Feb 10 2010 NOTICE.txt
drwxr-xr-x 8 root wheel 272 Feb 10 2010 bin
drwxr-xr-x 3 root wheel 102 Feb 10 2010 docs
drwxr-xr-x 15 root wheel 510 Feb 10 2010 etc
drwxr-xr-x 44 root wheel 1496 Feb 27 12:09 lib