A friend sent me along this great tutorial on webscraping The New York Times with R. I would really love to try it. However, the first step is to install a package called [RJSONIO][2] from source.
I know R reasonably well, but I have no idea how to install a package from source.
I'm running macOS (OS X).
If you have the file locally, then use install.packages() and set the repos=NULL:
install.packages(path_to_file, repos = NULL, type="source")
Where path_to_file would represent the full path and file name:
On Windows it will look something like this: "C:\\RJSONIO_0.2-3.tar.gz".
On UNIX it will look like this: "/home/blah/RJSONIO_0.2-3.tar.gz".
Download the source package, open Terminal.app, navigate to the directory where you currently have the file, and then execute:
R CMD INSTALL RJSONIO_0.2-3.tar.gz
Do note that this will only succeed when either: a) the package does not need compilation or b) the needed system tools for compilation are present. See: R for Mac OS X
You can install directly from the repository (note the type="source"):
install.packages("RJSONIO", repos = "http://www.omegahat.org/R", type="source")
A supplementarily handy (but trivial) tip for installing older version of packages from source.
First, if you call "install.packages", it always installs the latest package from repo. If you want to install the older version of packages, say for compatibility, you can call install.packages("url_to_source", repo=NULL, type="source"). For example:
install.packages("http://cran.r-project.org/src/contrib/Archive/RNetLogo/RNetLogo_0.9-6.tar.gz", repo=NULL, type="source")
Without manually downloading packages to the local disk and switching to the command line or installing from local disk, I found it is very convenient and simplify the call (one-step).
Plus: you can use this trick with devtools library's dev_mode, in order to manage different versions of packages:
Reference: doc devtools
From CRAN, you can install directly from a GitHub repository address. So if you want the package at https://github.com/twitter/AnomalyDetection, using
library(devtools)
install_github("twitter/AnomalyDetection")
does the trick.
In addition, you can build the binary package using the --binary option.
R CMD build --binary RJSONIO_0.2-3.tar.gz
If you have source code you wrote yourself, downloaded (cloned) from GitHub, or otherwise copied or moved to your computer from some other source, a nice simple way to install the package/library is:
In R
It's as simple as:
# install.packages("devtools")
devtools::install('path/to/package')
From terminal
From here, you can clone a GitHub repo and install it with:
git clone https://github.com/user/repo.git
R -e "install.packages('devtools');devtools::install('path/to/package')"
Or if you already have devtools installed, you can skip that first bit and just clone the repo and run:
R -e "devtools::install('path/to/package')"
Note that if you're on ubuntu, install these system libraries before installing devtools (or devtools won't install properly).
apt-get update
apt-get install build-essential libcurl4-gnutls-dev libxml2-dev libssl-dev libfontconfig1-dev libharfbuzz-dev libfribidi-dev libfreetype6-dev libpng-dev libtiff5-dev libjpeg-dev -y
Related
I recently developed a package my_package and am hosting it on GitHub. For easy installation and use, I have following setup.py:
from setuptools import setup
setup(name='my_package',
version='1.0',
description='My super cool package',
url='https://github.com/my_name/my_package',
packages=['my_package'],
python_requieres='3.9',
install_requires=[
'some_package==1.0.0'
])
Now I am trying to install this package in a conda environment:
conda create --name myenv python=3.9
conda activate myenv
pip install git+'https://github.com/my_name/my_package'
So far so good. If I try to use it in the project folder, everything works perfectly. If I try to use the packet outside the project folder (still inside the conda environment), I get the following error:
ModuleNotFoundError: No module named 'my_package'
I am working on windows, if that matters.
EDIT:
I'm verifying that both python and pip are pointing towards the correct version with:
which pip
which python
/c/Anaconda3/envs/my_env/python
/c/Anaconda3/envs/my_env/Scripts/pip
Also, when I run:
pip show my_package
I get a description of my package. So pip finds it, but as soon as I try to import my_package in the script, I get the described error.
I also verified that the package is installed in my environment. So in /c/Anaconda3/envs/my_env/lib/site-packages there is a folder my_package-1.0.dist-info/
Further: python "import sys, print(sys.path)"
shows, among other paths, /c/Anaconda3/envs/my_env/lib/site-packages. So it is in the path.
Check if you are using some explicit shebang in your script pointing to other Python interpreters.
Eg. using the system default Python:
#!/bin/env python
...
While inside your environment myenv, try to uninstall your package first, to do a clean test:
pip uninstall my_package
Also, you have a typo in your setup.py: python_requieres --> python_requires.
And I actually tried to install with your setup.py, and also got ModuleNotFoundError - but because it didn't properly install due to install_requires:
ERROR: Could not find a version that satisfies the requirement some_package==1.0.0
So, check also that everything installs without errors and warnings.
Hope that helps.
First thing I would like to point out (not the solution) regards the following statement you made:
If I try to use it in the project folder [...] If I try to use the packet outside the project folder [...]
I understand "project folder" means the "my_package" folder (inside the git repository). If that is the case, I would like to point out that you are mixing two situations: that of testing a (remote) package installation, while in your (local) repository. Which is not necessarily wrong, but error-prone.
Whenever testing the setup/install process of a package, make sure to move far from your repository (say, "/tmp/" equivalent in Windows) and, preferably, use a fresh environment. That will eliminate "noise" in your tests.
First thing I would tell you to do -- if not already -- is to create a fresh conda env and install your package from an empty/new folder. Eg,
$ conda env create -n test_my_package ipython pip
$ cd /tmp # equivalent temporary or new in your Windows
$ pip install git+https://github.com/my_name/my_package
If that doesn't work (maybe a problem with your pip' git+http code), do another way: create a release for your package (eg, "v1") and then install the released version by indicating the zip package URL (that you get from your "my_package" releases page on Github):
$ pip install https://github.com/my_name/my_package/archive/v1.zip
How would I install things on Pepper, since I don't know what package manager it uses. I usually use apt on my Ubuntu machine and want to install some packages on Pepper. I'm not sure what package manager Pepper has (if any) and want to install some packages, but also only know the name of the package using apt (not sure if the package name is the same on other package managers). And if possible, would I be able to install apt on Pepper. Thanks.
Note: From the research I've done, Pepper is using NaoQi which is based off Gentoo which uses portage.
You don't have root access on Pepper, which limits what you can install (and apt isn't on the robot anyway).
Some possibilities:
Include your content in Choregraphe projects - when you install a package, the whole directory structure is installed (more exactly, what's listed in the .pml); so you can put arbitrary files on your robot, and you can usually include whatever dependencies your code needs.
Install python packages with pip.
In NAOqi 2.5, a slightly older version of pip is installed that will not always work out of the box; I recommend upgrading it:
pip install --user --upgrade pip
... you can then use the upgraded pip to install other packages, using the upgraded pip, and always --user:
/home/nao/.local/bin/pip install --user whatever-package-you-need
Note however that if you do this and use your packages in your code running on Pepper, that code won't work on other robots until you do pip on them, which is why I usually only do this for tests; for production code I prefer packaging all dependencies in my app's package.
As a workaround if you need to install software (or just newer versions of software) using Gentoo Prefix is an option.
Gentoo Prefix builds a Gentoo OS on any location (no need of root, can be any folder). It includes it's own portage (package manager) to install new software.
I maintain a few projects to work with Pepper and use "any" software I want. Note that they are built for 64b (amd64) and 32b (x86) even though for Pepper only the 32b matter.
gentoo_prefix_ci and gentoo_prefix_ci_32b Which builds nightly the bootstrap of the Gentoo Prefix system. This is a process that takes a while to compile (3-6h depending on your machine) and that breaks from time to time (as upstream packages are updated and bugs are found, Gentoo is a rolling release distribution). Every night updated binary images ready to use can be found in the Releases section.
For ROS users that want to run it on the robot, based on the previous work, I maintain also ros_overlay_on_gentoo_prefix and ros_overlay_on_gentoo_prefix_32b. They provide nightly builds with binary releases of ROS Kinetic and ROS Melodic over Gentoo Prefix using ros-overlay. You can find ready-to-use 'ros_base' and 'desktop' releases.
For purposes related to the RoboCup#Home Social Standard Platform League where the Pepper robot is used I also maintain a specific build that contains a lot of additional software. This project is called pepper_os and it builds 270+ ROS packages, a lot of Python packages (250+ including Theano, dlib, Tensorflow, numpy...) and all the necessary dependencies for these to build (750+ packages). Note that the base image (it's built with Docker) is the actual Pepper 2.5.5.5 image, so it can be used for debugging as it if it was in the real robot (although without sensors and such).
Maybe this approach, or these projects, are useful.
The package manager on pepper is disabled. But you can copy the files to the robot and write your own service that imports any package you might need.
As a supplement on importing:
http://www.about-robots.com/how-to-import-python-files-in-your-pepper-apps.html
To get rid of error :
" SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
".
If you use python and requests package, just add verify=False at the end of your parameters.
r=requests.get(URL,params,header,verify=False)
Works with my Pepper
To get rid of
InsecurePlatformWarning: A true SSLContext object is not available.
install
/home/nao/.local/bin/pip install --user requests[security]
To get rid of:
CryptographyDeprecationWarning: Support for your Python version is deprecated.
install
/home/nao/.local/bin/pip install --user cryptography==2.2.2
If it based on Gentoo maybe we could try to install portage with pip.
pip install portage
Just a thought.
Jython Package installation issue, using pip
Hi, I have installed Jython2.7 configured with pydev in eclipse neon, also configured python 3.6 package
I am able to install packages for python using pip installer?
pip install "packagename"
Below are some of the packages in python/Lib/Site-packages directory
I was able to install all the packages
How do I use pip installer to install packages for jython?
I tried to install Jip package with
jython install setup.py
The binary File got installed in the Jython/Lib/Site-packages folder
However, I am not able to use it.
where and how do I get Jython package binaries like jip?
Also, Please let me know how to search jython packages?
Also, How to make pip install library packages in jython?
Any other configuration like jython home, etc that should be made?
This answer is going to be really generic but I just recently have slogged my way through the setup for jython/jip/pip and here's roughly what I had to do.
Firstly, I'm running Windows 7 64 Bit from behind a proxy (work machine.)
Had to install jython 2.7.0 instead of 2.7.1 because (I think anyway) 2.7.1 requires admin privileges which I don't have on my work PC.
Pip didn't install correctly during the Jython installation and I spent an obscene amount of time trying to get it installed and functioning as I knew it from my cpython days. NOTE: Just because you get pip installed, doesn't mean you can use any package on a python package repo. As of 2.7.0, Jython doesn't have end to end capability to interpret/compile some libraries that rely on certain python wrappers of native OS function calls. I believe 2.7.1 makes solid progress in the direction of supporting all needed native calls but don't quote me on that. For example, I tried to use wxPython to make a simple GUI to test my jython install. Trying to install it from pip kept causing really non-specific error info that took me a lot of time to figure out that the cause was jython simply couldn't compile the wxPython source so beware.
I had to set environment variables 'http_proxy' and 'https_proxy' in the form of http://proxyhosturl:port and https://proxyhosturl:port respectively to get out from behind the proxies without having to invoke pip with the proxy switch every time I called it.
To actually install pip, have a look here. These instructions are for Python and Linux/Unix but the principle is roughly the same. Just use jython -m instead of python -m and ignore the '$' at the start of each command line.
Also be sure to CD to your python_home/bin folder when invoking the ez_install exe.
If that doesn't work (didn't for me), try using get-pip.py script with these instructions https://pip.pypa.io/en/stable/installing/ (remember jython instead of python etc.). Download it, cd to the download location and follow the noted install steps. Worth noting is about half way down the install instructions where it details installing from local archives (source/binary zip or tar.gz archives of pip and setuptools as better described here: https://packaging.python.org/tutorials/installing-packages/#installing-from-local-archives).
The links to the bin archives of pip and setuptools are here:
https://pypi.python.org/pypi/setuptools
https://pypi.python.org/pypi/pip
It may also be worth making sure that your PATH environment variable has the jython/bin path in the variable value. The jython installer should do this but, again, mine did not.
If all goes well, you should be able to invoke pip with the --version switch and if it prints a line with the installed pip version info then you should be good to go
Another quirky issue I had was I could invoke a function of pip one time and any subsequent times I would get a stack trace ending with something along the lines of an object not having a certain property. I fixed that by finding my temp directory by opening a windows explorer instance and typing %TEMP% in the address bar and hitting enter, it should take you to a subdir of your AppData folder and there you may see a folder with the name of the package you were trying to install and the text "_pip" somewhere in the directory name. Delete the directory and try the pip install command again. I had to do this + invoke pip install pip -U to update my install to the latest version. Then pip began behaving correctly in my instance.
pip search numpy (or your library name) will generate a list of results with the same logic it uses to locate your desired package when you call pip install but, again, just because it returns a matching package doesn't mean it will compile when you install it (numpy doesn't work because of the missing java to C native function calls I described earlier.) The trade off is that you can import code artifacts from Java JAR files in your Jython script files and leverage their functionality with relative ease. Between the public Java APIs available and the python packages that work with the jython interpretor, you can (in my experience) come up with a way to accomplish your task. See the following info on JIP, Maven and IDEs.
IDE and jython integration (Eclipse)
- If you are stuck using Eclipse (like me) it actually has pretty decent support for python development. Install the PyDev plugin for Eclipse from Help -> Install Software. Put in this URL https://marketplace.eclipse.org/content/pydev-python-ide-eclipse, hit tab, and select the PyDev plugin and hit 'finish.'
- Setup the jython interpretor info from Windows -> Preferences -> PyDev. Provide the path to your jython.jar file.
- You should now be able to use File -> New PyDev project to create a basic python project and configure it to use your version of Jython and Java.
Brief Overview of Jip and Maven
- jip is a jython package that is invoked very similarly to pip but instead will download JAR files from the Maven Central Repository instead of python packages from pypi.com, for example. See the install instructions described here. Note the install procedure for a global jip install which differ from just pip install jip. https://pypi.python.org/pypi/jip/
- I never got jip to work exactly as I wished because there's not a ton of documentation on it outside of what I already linked. However, if you install a JAR using jip, you have to go to your project in Eclipse and actually add the JARs themselves to your PYTHONPATH in order for import statements and editing to have intellisense and so that you don't get a classnotfound exception at runtime. See following screen shot.
- There is a JIP config file that you can use similar to the pip config ini file but I have yet to find any exhaustive documentation on it's setup.
Note in the above screen shot the first entry in the External libraries entries. By default, pip places installed packages in that directory so to enable eclipse to find them, you need to also ensure that location is entered.
In Conclusion
- I have more to add to this answer and I will do so as soon as possible. In the meantime, see this example project I've loaded into github.
https://github.com/jheidlage1222/jython_java_integration_example
It shows basic config and how to interface with JARs from python code. I used the apache httpcomponents library as an example. Good luck amigo.
When I go get a package (say, godoc), it wants to create and install it in /usr/lib/go/bin (this is an Ubuntu system using the stock Go packages).
This isn't cool, because like a good Debian user I only let .deb packages install stuff into /usr.
How do I convince it to install into either my GOPATH workspace, or somewhere under /usr/local?
The correct answer turned out to be installing the golang-go.tools package.
If you install Go with GVM instead of from apt they will be placed in ~/.gvm/gos/ or your $GOPATH instead.
https://github.com/moovweb/gvm
Also I don't think godoc comes with 1.3 anymore, it has to be installed from go.tools
http://godoc.org/code.google.com/p/go.tools/cmd/godoc
http://golang.org/doc/go1.2#go_doc
The "go doc" command is deleted.
I would like to install the older version of Google Protocol Buffers (protobuf-2.4.1) on Mac using the command line/Terminal app.
I tried with brew install protobuf, but the latest version 2.5.0 has been installed.
Is it possible to install the older version from the terminal?
There are some issues with building protobuf 2.4.1 from source on a Mac. There is a patch that also has to be applied. All this is contained within the homebrew protobuf241 formula, so I would advise using it.
To install protocol buffer version 2.4.1 type the following into a terminal:
brew tap homebrew/versions
brew install protobuf241
If you already have a protocol buffer version that you tried to install from source, you can type the following into a terminal to have the source code overwritten by the homebrew version:
brew link --force --overwrite protobuf241
Check that you now have the correct version installed by typing:
protoc --version
It should display 2.4.1
This is not via brew, but the end result is the same.
Download the protobuf-2.4.1 from https://github.com/protocolbuffers/protobuf/releases/tag/v2.4.1
Extract the tar.gz file.
$cd ~/Downloads/protobuf-2.4.1
$./configure
$make
$make check
$sudo make install
$which protoc
$protoc --version
Steps 4-7 are from the README.txt file from the protobuf tarball.
If you landed here looking for how to install Protocol Buffers on Mac, it can be done using Homebrew by running the command below
brew install protobuf
It installs the latest version of protobuf available. For me, at the time of writing, this installed the v3.7.1
If you'd like to install an older version, please look up the available ones from the package page Protobuf Package - Homebrew and install that specific version of the package.
The oldest available protobuf version in this package is v3.6.1.3
It's a new year and there's a new mismatch between the version of protobuf in Homebrew and the cutting edge release. As of February 2016, brew install protobuf will give you version 2.6.1.
If you want the 3.0 beta release instead, you can install it with:
brew install --devel protobuf
For some reason I need to use protobuf 2.4.1 in my project on OS X El Capitan. However homebrew has removed protobuf241 from its formula. I install it according #kksensei's answer manually and have to fix some error during the process.
During the make process, I get 3 error like following:
google/protobuf/message.cc:130:60: error: implicit instantiation of undefined template 'std::__1::basic_istream<char, std::__1::char_traits<char> >'
return ParseFromZeroCopyStream(&zero_copy_input) && input->eof();
^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/c++/v1/iosfwd:108:28: note: template is declared here
class _LIBCPP_TYPE_VIS basic_istream;
^
google/protobuf/message.cc:135:67: error: implicit instantiation of undefined template 'std::__1::basic_istream<char, std::__1::char_traits<char> >'
return ParsePartialFromZeroCopyStream(&zero_copy_input) && input->eof();
^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/c++/v1/iosfwd:108:28: note: template is declared here
class _LIBCPP_TYPE_VIS basic_istream;
^
google/protobuf/message.cc:175:16: error: implicit instantiation of undefined template 'std::__1::basic_ostream<char, std::__1::char_traits<char> >'
return output->good();
^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/c++/v1/iosfwd:110:28: note: template is declared here
class _LIBCPP_TYPE_VIS basic_ostream;
^
(Sorry, I dont know how to attach code when the code contains '`' )
If you get the same error, please edit src/google/protobuf/message.cc, add #include <istream> at the top of the file and do $ make again and should get no errors. After that do $ sudo make install. When install finished $protoc --version should display the correct result.
From https://github.com/paulirish/homebrew-versions-1 . Works for me!
brew install https://raw.githubusercontent.com/paulirish/homebrew-versions-1/master/protobuf241.rb
I used macports
sudo port install protobuf-cpp
To install Protocol Buffer (as of today version v3.7.0)
Go to this website
download the zip file according to your OS (e.g.: protoc-3.7.0-osx-x86_64.zip). This applies also to other OS.
Move the executable in protoc-3/bin/protoc to one of your directories in PATH. In Mac I suggest to put it into /usr/local/bin
Now your good to go 💪🏾
(optional) There is also an include file, you can add. This is a snippet of the README.md
If you intend to use the included well known types then don't forget to
copy the contents of the 'include' directory somewhere as well, for example
into '/usr/local/include/'.
Please refer to our official github site for more installation instructions:
https://github.com/protocolbuffers/protobuf
brew install --devel protobuf
If it tells you "protobuf-2.6.1 already installed":
1. brew uninstall --devel protobuf
2. brew link libtool
3. brew install --devel protobuf
There is another official way by Google, as mentioned by another user.
Read it fully before trying.
Here are the steps:
Open Terminal and type the following
PROTOC_ZIP=protoc-3.7.1-osx-x86_64.zip
curl -OL https://github.com/google/protobuf/releases/download/v3.7.1/$PROTOC_ZIP
sudo unzip -o $PROTOC_ZIP -d /usr/local bin/protoc
rm -f $PROTOC_ZIP
Worked for me.
P.S.
This is for version 3.7.1 in osx only.
If you want to install some other version/platform, visit the releases link and check out the details of the latest version, and use those information.
Reference
FWIW., the latest version of brew is at protobuf 3.0, and doesn't include any formulae for the older versions. This is somewhat "inconvenient".
While protobuf may be compatible at the wire level, it is absolutely not compatible at the level of generated java classes: you can't use .class files generated with protoc 2.4 with the protobuf-2.5 JAR, etc. etc. This is why updating protobuf versions is such a sensitive topic in the Hadoop stack: it invariably requires coordination across different projects, and is traumatic enough that nobody likes to do it.
HomeBrew versions has been removed and formulaes have been emptied. Therefore, my advice is to install it manually following the following steps.
For the time being you will need to build and install the Protocol Buffers toolset manually.
Download source code: https://github.com/google/protobuf/releases/download/v2.4.1/protobuf-2.4.1.tar.gz
tar xvfz protobuf-2.4.1.tar.gz
cd protobuf-2.4.1
Run ./configure
Edit src/google/protobuf/message.cc, add #include at the top of the file
Run make command from root of the folder, i.e. protobuf-2.4.1/
Run sudo make install
Run /usr/local/bin/protoc --version to check the version of protobuf compiler version
The terminal output should be:
Version: libprotoc 2.4.1
you can install from official link page provided by google http://google.github.io/proto-lens/installing-protoc.html
For v3 users.
http://google.github.io/proto-lens/installing-protoc.html
PROTOC_ZIP=protoc-3.7.1-osx-x86_64.zip
curl -OL https://github.com/protocolbuffers/protobuf/releases/download/v3.7.1/$PROTOC_ZIP
sudo unzip -o $PROTOC_ZIP -d /usr/local bin/protoc
sudo unzip -o $PROTOC_ZIP -d /usr/local 'include/*'
rm -f $PROTOC_ZIP
There should be better ways but what I did today was:
Download from https://github.com/protocolbuffers/protobuf/releases (protoc-3.14.0-osx-x86_64.zip at this moment)
Unzip (double click the zip file)
Here, I added a symbolic link
ln -s ~/Downloads/protoc-3.14.0-osx-x86_64/bin/protoc /usr/local/bin/protoc
Check if works
protoc --version