Is there any way in Ruby where I can do a global search for a given file?
I tried Dir.glob but it searches in the current directory. And for the Find module I need to pass a list of directories to search. In my case, I need to search for a particular file which might be in any directory.
How can I do this in Ruby?
Find recurses into subdirectories, so just start at the root path, it'll go everywhere:
Find.find('/') do |path|
# look for your filename
end
On systems that have the locate command line tool, like Linux and Mac OSX, you can find files very fast like this:
filename = "test"
array_of_files_found = `locate #{filename}`.split("\n")
Be aware of the dangers of passing user-supplied parameters to the command line. See this answer for details
Also note that on OSX, you might need to create the database that powers the locate command like this:
sudo launchctl load -w /System/Library/LaunchDaemons/com.apple.locate.plist
Related
When running scripts in bash, I have to write ./ in the beginning:
$ ./manage.py syncdb
If I don't, I get an error message:
$ manage.py syncdb
-bash: manage.py: command not found
What is the reason for this? I thought . is an alias for current folder, and therefore these two calls should be equivalent.
I also don't understand why I don't need ./ when running applications, such as:
user:/home/user$ cd /usr/bin
user:/usr/bin$ git
(which runs without ./)
Because on Unix, usually, the current directory is not in $PATH.
When you type a command the shell looks up a list of directories, as specified by the PATH variable. The current directory is not in that list.
The reason for not having the current directory on that list is security.
Let's say you're root and go into another user's directory and type sl instead of ls. If the current directory is in PATH, the shell will try to execute the sl program in that directory (since there is no other sl program). That sl program might be malicious.
It works with ./ because POSIX specifies that a command name that contain a / will be used as a filename directly, suppressing a search in $PATH. You could have used full path for the exact same effect, but ./ is shorter and easier to write.
EDIT
That sl part was just an example. The directories in PATH are searched sequentially and when a match is made that program is executed. So, depending on how PATH looks, typing a normal command may or may not be enough to run the program in the current directory.
When bash interprets the command line, it looks for commands in locations described in the environment variable $PATH. To see it type:
echo $PATH
You will have some paths separated by colons. As you will see the current path . is usually not in $PATH. So Bash cannot find your command if it is in the current directory. You can change it by having:
PATH=$PATH:.
This line adds the current directory in $PATH so you can do:
manage.py syncdb
It is not recommended as it has security issue, plus you can have weird behaviours, as . varies upon the directory you are in :)
Avoid:
PATH=.:$PATH
As you can “mask” some standard command and open the door to security breach :)
Just my two cents.
Your script, when in your home directory will not be found when the shell looks at the $PATH environment variable to find your script.
The ./ says 'look in the current directory for my script rather than looking at all the directories specified in $PATH'.
When you include the '.' you are essentially giving the "full path" to the executable bash script, so your shell does not need to check your PATH variable. Without the '.' your shell will look in your PATH variable (which you can see by running echo $PATH to see if the command you typed lives in any of the folders on your PATH. If it doesn't (as is the case with manage.py) it says it can't find the file. It is considered bad practice to include the current directory on your PATH, which is explained reasonably well here: http://www.faqs.org/faqs/unix-faq/faq/part2/section-13.html
On *nix, unlike Windows, the current directory is usually not in your $PATH variable. So the current directory is not searched when executing commands. You don't need ./ for running applications because these applications are in your $PATH; most likely they are in /bin or /usr/bin.
This question already has some awesome answers, but I wanted to add that, if your executable is on the PATH, and you get very different outputs when you run
./executable
to the ones you get if you run
executable
(let's say you run into error messages with the one and not the other), then the problem could be that you have two different versions of the executable on your machine: one on the path, and the other not.
Check this by running
which executable
and
whereis executable
It fixed my issues...I had three versions of the executable, only one of which was compiled correctly for the environment.
Rationale for the / POSIX PATH rule
The rule was mentioned at: Why do you need ./ (dot-slash) before executable or script name to run it in bash? but I would like to explain why I think that is a good design in more detail.
First, an explicit full version of the rule is:
if the path contains / (e.g. ./someprog, /bin/someprog, ./bin/someprog): CWD is used and PATH isn't
if the path does not contain / (e.g. someprog): PATH is used and CWD isn't
Now, suppose that running:
someprog
would search:
relative to CWD first
relative to PATH after
Then, if you wanted to run /bin/someprog from your distro, and you did:
someprog
it would sometimes work, but others it would fail, because you might be in a directory that contains another unrelated someprog program.
Therefore, you would soon learn that this is not reliable, and you would end up always using absolute paths when you want to use PATH, therefore defeating the purpose of PATH.
This is also why having relative paths in your PATH is a really bad idea. I'm looking at you, node_modules/bin.
Conversely, suppose that running:
./someprog
Would search:
relative to PATH first
relative to CWD after
Then, if you just downloaded a script someprog from a git repository and wanted to run it from CWD, you would never be sure that this is the actual program that would run, because maybe your distro has a:
/bin/someprog
which is in you PATH from some package you installed after drinking too much after Christmas last year.
Therefore, once again, you would be forced to always run local scripts relative to CWD with full paths to know what you are running:
"$(pwd)/someprog"
which would be extremely annoying as well.
Another rule that you might be tempted to come up with would be:
relative paths use only PATH, absolute paths only CWD
but once again this forces users to always use absolute paths for non-PATH scripts with "$(pwd)/someprog".
The / path search rule offers a simple to remember solution to the about problem:
slash: don't use PATH
no slash: only use PATH
which makes it super easy to always know what you are running, by relying on the fact that files in the current directory can be expressed either as ./somefile or somefile, and so it gives special meaning to one of them.
Sometimes, is slightly annoying that you cannot search for some/prog relative to PATH, but I don't see a saner solution to this.
When the script is not in the Path its required to do so. For more info read http://www.tldp.org/LDP/Bash-Beginners-Guide/html/sect_02_01.html
All has great answer on the question, and yes this is only applicable when running it on the current directory not unless you include the absolute path. See my samples below.
Also, the (dot-slash) made sense to me when I've the command on the child folder tmp2 (/tmp/tmp2) and it uses (double dot-slash).
SAMPLE:
[fifiip-172-31-17-12 tmp]$ ./StackO.sh
Hello Stack Overflow
[fifi#ip-172-31-17-12 tmp]$ /tmp/StackO.sh
Hello Stack Overflow
[fifi#ip-172-31-17-12 tmp]$ mkdir tmp2
[fifi#ip-172-31-17-12 tmp]$ cd tmp2/
[fifi#ip-172-31-17-12 tmp2]$ ../StackO.sh
Hello Stack Overflow
I have a /.cust_dev_cmds/ directory on my MBP machine that is part of a parent sysadmin-work directory that uses a git repo here. I would like to be able to:
Not have to use a for loop in my .bash_profile to source all the *.sh files.
Add the directory to PATH with and export line in the .bash_profile instead.
# from my .bash_profile
export PATH="/Users/<my-usr-name>/Public/sharable-dev-scripts:$PATH"
This does show up with a echo $PATH but when I try to invoke a function from within the scripts I have created that worked with sourcing directly within the .bash_profile in a loop (like with point #1 above) like this
# create a directory with a builtin command
mkdir test-dir
# use one of my custom ones to create a simple readme.md within that directory:
mkr !!:1
# I am getting >>> mkr: command not found
Use whatever type of link to not have a duplicated directory structure on the machine.
Any good explanations to read up on here without using $10 words would be great.
Define a means to test the link works and works through PATH. It would also be nice that something like declare -F would be available to see that the scripts within the directory are in fact becoming part of callable functions in the shell.
is there a command anyone knows to do this?
Step this up a notch for a shared network directory. I have created a shared directory through apple > System Preferences > Sharing, and turned on the ability to share this directory in the Public folder.
Is there a tutorial that can outline this with something like VirtualBox and an Ubuntu guest that is accessing the commands from the MBP shared directory?
I have realized point #1, so really the question begins with #2 so no one would suggest the first one. I have read a bit on links but the way most of the articles I come across describing them are difficult to wrap my head around- especially when wishing to add this functionality to PATH. I believe the answer may revolve around how links are followed, but it may be better to back up and punt- dig back into linking first- then export my directory appropriately without a link, and eventually get the proper resolution to this situation.
The last thought on links before I try a few hacks on my own is do I need to only add a link to the Public directory and somehow place a flag to look at all the directories within the /Public, or is it better to drill all the way down to the /Public/shared-directory/.cust_dev_cmds? Any direction would be greatly appreciated. My goal is to be able to have a few custom command directories for various tasks, and eventually have them across networks/instances.
When you want all functions that you wrote in files in /.cust_dev_cmds/, the normal way would be sourcing all the files.
When you want to avoid a loop, you can use
utildir="$HOME/.cust_dev_cmds/" # I think the path is relative to your home).
source <(cat ${utildir}/*)
When you want the functions found with the PATH, you should make a file for each function.
Before:
# cat talk
ask() { echo "How are you?"; }
answer() { echo "Fine, thank you"; }
After:
# cat ask
echo "How are you?"
# cat answer
echo "Fine, thank you"
When you want all users to use the same set of functions, consider a master script that sources all scripts (the masterfile can use user=dependent settings like HOME or VERSION):
# cat /Public/shared-directory/setup_functions
utildir="$HOME/.cust_dev_cmds/" # I think the path is relative to your home).
source <(cat ${utildir}/*)
source some_other_file
Now each user only needs to source one file.
I have a c5 project with a directory C5PROJECTDIR it has a bin directory. Is there a way (without adding C5PROJECTDIR/bin to $PATH) to access commands from any other subdirectory of C5PROJECTDIR?
For example:
There is an x command located here: C5PROJECTDIR/bin/x.
I want to run this command from a bash script from C5PROJECTDIR/packages/some-other-dir.
UPDATE: The sub directory can be more or less deeper in the tree...
How can it be possible?
There are a few ways:
Use relative paths:
Essentially call the binary directly with its path, but relative to the current script. If your script is located in /home/user/foo/C5PROJECTDIR/packages/somedir/script, then you call the script relatively:
#!/usr/bin/env bash
../../bin/x
The problem with this method is maintainability. Imagine you want to move your script to /home/user/foo/C5PROJECTDIR/packages/scripts, then you would have to update all your scripts.
Use a shell variable:
Instead of relative paths, you can define a variable PROJECTHOME which contains the base value.
#!/usr/bin/env bash
PROJECTHOME=/home/user/foo/C5PROJECTDIR
$PROJECTHOME/bin/x
This generally solves most problems unless you move project location from /home/user/foo/C5PROJECTDIR to /home/user/random-dir. But this can be solved with a simple sed command that searches for the line /home/user/foo/C5PROJECTDIR and replaces it.
PATH variable only make things easier.
You able to run any commands with absolute path like /bin/echo or /home/jon/myscript. Relative path works as well ~/../../bin/ls.
Script have its own PATH variable, so you can add the desired location to PATH in your script if you like.
Apart from this you have to deal with permissions.
The user that have script executed by, must have execute permission on the x script.
To find and execute command:
find C5PROJECTDIR -type f -executable -name "x" -exec {} \;
Find's must be strict to match only one command else it will execute all that it fund. You cannot have script with the same name under C5PROJECTDIR in this case.
From security point of view. I do not recommend this, as anyone can put an executable file under C5PROJECTDIR with a name used in the script. things can go nasty.
When running scripts in bash, I have to write ./ in the beginning:
$ ./manage.py syncdb
If I don't, I get an error message:
$ manage.py syncdb
-bash: manage.py: command not found
What is the reason for this? I thought . is an alias for current folder, and therefore these two calls should be equivalent.
I also don't understand why I don't need ./ when running applications, such as:
user:/home/user$ cd /usr/bin
user:/usr/bin$ git
(which runs without ./)
Because on Unix, usually, the current directory is not in $PATH.
When you type a command the shell looks up a list of directories, as specified by the PATH variable. The current directory is not in that list.
The reason for not having the current directory on that list is security.
Let's say you're root and go into another user's directory and type sl instead of ls. If the current directory is in PATH, the shell will try to execute the sl program in that directory (since there is no other sl program). That sl program might be malicious.
It works with ./ because POSIX specifies that a command name that contain a / will be used as a filename directly, suppressing a search in $PATH. You could have used full path for the exact same effect, but ./ is shorter and easier to write.
EDIT
That sl part was just an example. The directories in PATH are searched sequentially and when a match is made that program is executed. So, depending on how PATH looks, typing a normal command may or may not be enough to run the program in the current directory.
When bash interprets the command line, it looks for commands in locations described in the environment variable $PATH. To see it type:
echo $PATH
You will have some paths separated by colons. As you will see the current path . is usually not in $PATH. So Bash cannot find your command if it is in the current directory. You can change it by having:
PATH=$PATH:.
This line adds the current directory in $PATH so you can do:
manage.py syncdb
It is not recommended as it has security issue, plus you can have weird behaviours, as . varies upon the directory you are in :)
Avoid:
PATH=.:$PATH
As you can “mask” some standard command and open the door to security breach :)
Just my two cents.
Your script, when in your home directory will not be found when the shell looks at the $PATH environment variable to find your script.
The ./ says 'look in the current directory for my script rather than looking at all the directories specified in $PATH'.
When you include the '.' you are essentially giving the "full path" to the executable bash script, so your shell does not need to check your PATH variable. Without the '.' your shell will look in your PATH variable (which you can see by running echo $PATH to see if the command you typed lives in any of the folders on your PATH. If it doesn't (as is the case with manage.py) it says it can't find the file. It is considered bad practice to include the current directory on your PATH, which is explained reasonably well here: http://www.faqs.org/faqs/unix-faq/faq/part2/section-13.html
On *nix, unlike Windows, the current directory is usually not in your $PATH variable. So the current directory is not searched when executing commands. You don't need ./ for running applications because these applications are in your $PATH; most likely they are in /bin or /usr/bin.
This question already has some awesome answers, but I wanted to add that, if your executable is on the PATH, and you get very different outputs when you run
./executable
to the ones you get if you run
executable
(let's say you run into error messages with the one and not the other), then the problem could be that you have two different versions of the executable on your machine: one on the path, and the other not.
Check this by running
which executable
and
whereis executable
It fixed my issues...I had three versions of the executable, only one of which was compiled correctly for the environment.
Rationale for the / POSIX PATH rule
The rule was mentioned at: Why do you need ./ (dot-slash) before executable or script name to run it in bash? but I would like to explain why I think that is a good design in more detail.
First, an explicit full version of the rule is:
if the path contains / (e.g. ./someprog, /bin/someprog, ./bin/someprog): CWD is used and PATH isn't
if the path does not contain / (e.g. someprog): PATH is used and CWD isn't
Now, suppose that running:
someprog
would search:
relative to CWD first
relative to PATH after
Then, if you wanted to run /bin/someprog from your distro, and you did:
someprog
it would sometimes work, but others it would fail, because you might be in a directory that contains another unrelated someprog program.
Therefore, you would soon learn that this is not reliable, and you would end up always using absolute paths when you want to use PATH, therefore defeating the purpose of PATH.
This is also why having relative paths in your PATH is a really bad idea. I'm looking at you, node_modules/bin.
Conversely, suppose that running:
./someprog
Would search:
relative to PATH first
relative to CWD after
Then, if you just downloaded a script someprog from a git repository and wanted to run it from CWD, you would never be sure that this is the actual program that would run, because maybe your distro has a:
/bin/someprog
which is in you PATH from some package you installed after drinking too much after Christmas last year.
Therefore, once again, you would be forced to always run local scripts relative to CWD with full paths to know what you are running:
"$(pwd)/someprog"
which would be extremely annoying as well.
Another rule that you might be tempted to come up with would be:
relative paths use only PATH, absolute paths only CWD
but once again this forces users to always use absolute paths for non-PATH scripts with "$(pwd)/someprog".
The / path search rule offers a simple to remember solution to the about problem:
slash: don't use PATH
no slash: only use PATH
which makes it super easy to always know what you are running, by relying on the fact that files in the current directory can be expressed either as ./somefile or somefile, and so it gives special meaning to one of them.
Sometimes, is slightly annoying that you cannot search for some/prog relative to PATH, but I don't see a saner solution to this.
When the script is not in the Path its required to do so. For more info read http://www.tldp.org/LDP/Bash-Beginners-Guide/html/sect_02_01.html
All has great answer on the question, and yes this is only applicable when running it on the current directory not unless you include the absolute path. See my samples below.
Also, the (dot-slash) made sense to me when I've the command on the child folder tmp2 (/tmp/tmp2) and it uses (double dot-slash).
SAMPLE:
[fifiip-172-31-17-12 tmp]$ ./StackO.sh
Hello Stack Overflow
[fifi#ip-172-31-17-12 tmp]$ /tmp/StackO.sh
Hello Stack Overflow
[fifi#ip-172-31-17-12 tmp]$ mkdir tmp2
[fifi#ip-172-31-17-12 tmp]$ cd tmp2/
[fifi#ip-172-31-17-12 tmp2]$ ../StackO.sh
Hello Stack Overflow
I'm trying to create a homebrew formula for an application that doesn't need to be compiled. I've tried looking through the formula cookbook, but I'm missing something to make things properly work. Below is my use-case with more generic filenames.
Inside the container is two files: one is the script for the application, the other being a file for man pages. We'll use the following filenames to keep things generic:
myapp.py (executable script)
resource.txt (a resource file that the script needs)
myapp.1 (man page)
What are the best ways to get these into the correct locations? Presume I have the ability to modify the code in the script for choosing the location to load the resource.
Thanks.
You can either modify your script at install time to use the correct location of the resource or just assume it’s in the same directory and let Homebrew do the magic for you. I wrote an example formula for the latter case in another answer.
Here’s how it looks like for your needs:
class Foo < Formula
desc "Blah blah"
url "https://github.com/foo/foo/archive/master.zip"
version "1.2.3"
def install
man1.install "myapp.1"
libexec.install Dir["*"]
bin.write_exec_script (libexec/"myapp.py")
end
end
It installs myapp.1 in the correct directory. You can also use man2, man3, etc for other man directories.
It then installs all the remaining files under libexec then create an exec script in bin/myapp.py. This will be a simple shell script that execs your script in libexec. That way, your script will executes from libexec and thus will be able to find resource.txt that’s located in the same directory.
If you’d like to call it myapp and not myapp.py it’d look like that:
def install
man1.install "myapp.1"
libexec.install "resource.txt"
libexec.install "myapp.py" => "myapp"
bin.write_exec_script (libexec/"myapp")
end