I would like to create a requirements_all.txt file using make commands. I am using Anaconda.
I have tried the following but none have worked as intended.
requirements_all.txt:
conda list --export
This displays all of the packages I have installed in the terminal window, and does not create requirements_all.txt.
requirements_all.txt:
#echo conda list --export
This displays #echo conda list --export in the terminal window, and does not create requirements_all.txt.
requirements_all.txt:
#echo conda list --export > requirements_all.txt
This created a file called requirements_all.txt containing conda list --export.
Related
I want to create a Python environment with the data science libraries NumPy, Pandas, Pytorch, and Hugging Face transformers. I use miniconda to create the environment and download and install the libraries. There is a flag in conda install, --download-only to download the required packages without installing them and install them afterwards from a local directory. Even when conda just downloads the packages without installing them, it also extracts them.
Is it possible to download the packages without extracting them and extract them afterwards before installation?
There is no simple command in the CLI to prevent the extraction step. The extraction is regarded as part of the FETCH operation to populate the package cache before running the LINK operation to transfer the package to the specified environment.
The alternative would be to do something manually. Naively, one could search Anaconda Cloud and manually download, however, it would probably be better to go through the solver to ensure package compatibility. All the info for operations to be run can be viewed by including the --json flag. This could be filtered to just the tarball URLs and then downloaded directly. Here's a script along these lines (assuming Linux/Unix):
File: conda-download.sh
#!/bin/bash -l
conda create -dn null --json "$#" |\
grep '"url"' | grep -oE 'https[^"]+' |\
xargs wget -c
which can be used as
./conda-download.sh -c conda-forge -c pytorch numpy pandas pytorch transformers
that is, it accepts all arguments conda create would, and will download all the tarballs locally.
Ignoring Cached Packages
If you already have some packages cached then the above will not redownload them. Instead, if you wish to download all tarballs needed for an environment, then you could use this alternate version which overrides the package cache using an empty temporary directory:
File: conda-download-all.sh
#!/bin/bash -l
tmp_dir=$(mktemp -d)
CONDA_PKGS_DIRS=$tmp_dir conda create -dn null --json "$#" |\
grep '"url"' | grep -oE 'https[^"]+' |\
xargs wget -c
rm -r $tmp_dir
Do you really want to use conda-pack? That lets you archive a conda-environment for reproducing without using the internet or re-solving for dependencies. To just prevent re-solving you can also use conda env export --explict but that still ties you to the source (internet or local disk repository).
If you have a static environment (read-only) and want to really reduce docker size, you can volume mount the environment at runtime. You would need to match the file paths (ie: /opt/anaconda => /opt/anaconda).
Description of Problem:
I'm trying to use terminator in a bash script but I keep getting an error stating that No such file or directory: ~/.config/terminator/config. and its keeping the terminals from popping up on screen. However when I run terminator on my terminal the window pops up. So I definitely have terminator installed I just don't know where terminator/config is.
Questions and summary of issue:
How would I check if terminator/config exists on my system?
Where would I find it since it isn't in ~/.config?
Why would this be happening?
Solutions that I know don't work and extra information
Uninstalling and reinstalling. I did it before I came to this site to ask.
These are the commands I used to install it the first time:
sudo add-apt-repository ppa:gnome-terminator
sudo apt-get update
sudo apt-get install terminator
I don't think I was at the home directory when I installed terminator if that matters.
How would I check if terminator/config exists on my system?
Either find the file using 'locate':
$ sudo apt install locate -y
$ sudo updatedb # this is to update the file database
$ locate config
Or even better, use stat:
$ stat ~/.config/terminator/config
Where would I find it since it isn't in ~/.config?
If the file does not exist, you cannot find it ofcourse.
Perhaps create the file yourself.
$ mkdir -p ~/.config/terminator/
$ touch ~/.config/terminator/config
Why would this be happening?
The systems provides an error message. Because it expects there to be file, which isn't.
Please read the documentation.
the config file is created automatically if you make same permanent changes.
start terminator
right click -> preferences
go to tab profiles -> sub tab colors
choose a different color profile e.g. "Solirized light" https://i.stack.imgur.com/QYFyp.png
close preferences and confirm the color of the terminator has
changed.
now the file "~/.config/terminator/config" has been created, containing the current user configuration.
When I start bash, the following description appears:
-bash: /usr/share/virtualenvwrapper/virtualenvwrapper_lazy.sh: No such file or directory
-bash: /usr/share/virtualenvwrapper/virtualenvwrapper_lazy.sh: No such file or directory
i'm using Microsoft-Windows-Subsystem-Linux, with ubuntu distro.
I already tried uninstalling by powershell pip uninstall virtualenvwrapper or in bash sudo pip uninstall virtualenvwrapper.
I also entered the share directory to delete the virtualenvwrapper folder, but it does not appear. I'm pressing LS and the folder does not appear.
I am also trying "which virtualenvwrapper" or "which virtualenvwrapper_lazy.sh" but this simply returns the same folder as I am.
i just need help to remove that message of bash
Most likely, there is a reference (potentially with source or .) to virtualenvwrapper_lazy.sh in one of bash startup files. Look at the usual suspects: ~/.bashrc, ~/.profile, /etc/profile, /etc/bash.bash.rc, /etc/.bash_login
To help with debugging, consider getting by adding set -x to ~/.bashrc, or by running interactive session
bash -ix /dev/null
UPDATE, based on log:
Per https://superuser.com/questions/893448/bash-shell-error-no-such-file-or-directory/1049989
I just had the same problem on Linux Mint.
The following file was causing the issue:
/etc/bash_completion.d/virtualenvwrapper
I made a backup (just in case) and deleted it, and everything seems to
be working fine now.
I've created a custom channel on a windows box following the steps detailed here.
Now I'd like to access it from a different machine but the channel parameter is a URI and I don't know what form it should take with Windows.
Here's the command I tried to execute:
conda search -c file://machine\C\channel --override-channels scipy
which failed with the following error message:
Fetching package metadata: Error: Invalid index file
I have been trying to do the same thing, and the answer by Paul made me a bit pessimistic.
It turns out that it is possible to use a UNC-path.
After trying a few hundred combinations of slashes and backslashes, I found this combination to work:
conda search -c "file://\\DOMAIN\SERVER\SHARE\conda\channel" --override-channels
Similarly,
conda config --add channels "file://\\DOMAIN\SERVER\SHARE\conda\channel"
Adds the channel to your config file.
Let's say that your custom channel is located in the following directory:
N:\conda\channel. Then we would expect to see the following in this directory (1) the win-64 directory (2) the index files inside, in this case the directory N:\conda\channel\win-64\, of repodata.json and repodata.json.bz2 (3) any packages that you have added to your channel. A search on this channel for the scipy package, ignoring all other channels, would look like this conda search -c file://N:\conda\channel --override-channels scipy
Did you add the scipy package into your custom channel? If you did, then did you run conda index on that directory?
I'm a little confused by your directory structure but, if your channel is machine\C\channel, then what happens when you do dir machine\C\channel?
I had no success with the other answers though #gDexter42 helped me in the right direction. Perhaps the API has changed. Testing several different options I was somewhat surprised that
you can use / or \ interchangably
you do not need to escape spaces
you do not need to enclose paths with spaces in quotes
After creating a custom channel in a network accessible directory, you can search for a conda package using the file path, excluding the file:// mentioned in other posts and in the documentation.
For a UNC Path:
$ conda search -c //my_nas/some/path with spaces/channel --override-channels
or
$ conda search -c \my_nas\some\path with spaces\channel --override-channels
If the folder is local, or you have mounted a network directory to a local path (D:\ in this example), you would use that file path.
$ conda search -c D:/some/path with spaces/channel --override-channels
or
$ conda search -c D:\some\path with spaces\channel --override-channels
I tested these commands using both Git Bash for Windows and Anaconda Prompt (which I think is just cmd.exe with the path modified so base/root is the active environment).
Note that if you then want to add that path to your .condarc file, you can use the same path.
channels:
- \\my_nas\some\path with spaces\channel # UNC
- D:/some/path with spaces/channel # local drive
- defaults # this gives defaults lower priority
ssl_verify: true
If you are trying to search for a conda package in a local directory (not on UNC), the following two approaches worked for me.
Navigate to the drive containing the package and try
conda search -c file://folder_path/channel --override-channels
the better way is to drop the file flag which allows you to search from any drive. Type
conda search -c Drive://folder_path/channel --override-channels thus if you are searching from D: drive you would type this as
conda search -c D://folder_path/channel --override-channels
If your conda channel is at C:\conda-channel then do:
conda search -c file://\conda-channel --override-channels
There is currently a bug in conda 4.6+ where file://C:\conda-channel will not work as it removes the colon when parsing. And downgrading to 4.5 is dangerous and can mess up your installation.
I have followed the following approach in order to install Torch in my machine (Mac).
http://torch.ch/docs/getting-started.html#_
When I am done with the installation, I type:
$ luarocks install image
or $ luarocks lis
or $th
in order to load the th or to make updates on the lua packages. It says "command not found". Do you have any idea how I can resolve this issue?
If you're on a Mac using the bash terminal, make sure that you've permanently added /Users/you/torch/install/bin to your PATH.
To do this:
Navigate in your terminal to the root directory by running the command:
$ cd
Using the text editor of your choice (emacs, vim, etc.) open the .bash_profile file for editing. For example:
$ emacs .bash_profile
Add the following line to the end of the file (replacing 'you' with your Mac username):
PATH=$PATH\:/Users/you/torch/install/bin ; export PATH
Save and exit the text editor
Source the changes by running:
$ source .bash_profile
Check that your PATH has been updated (look for /Users/you/torch/install/bin in the string returned):
$ echo $PATH
To make sure it has been changed permanently, completely quit Terminal, open it and run echo $PATH again
Now try th and it should run Torch!
For more help on PATH:
https://kb.iu.edu/d/acar
The Torch installation (at least for me) added the line . /Users/jb/torch/install/bin/torch-activate to my .profile file, not .bash_profile. I tried adding that exact line to .bash_profile but it didn't work, so based on the recommendations here I got rid of the trailing directory and such.
Have you updated your PATH? It should include something like
/home/user/torch/install/bin
I faced the same issue and following this post deleted and reinstalled everything. However in the end what helped was adding /home/user/torch/install/bin/ to the PATH variable.
I have resolved the issue. I have deleted torch and I have installed it again. I have updated my PATH, and I have ran the $ luarocks install image command. After all of these, I was able to ran $ th command and in general torch.