Hi I'm learning greemplum and using gpfdist to loading files, however there are some syntax that I couldn't find explanations on docs.
gpfdist -d ~/gpdb-sandbox-tutorials/faa -p 8081 > /tmp/gpfdist.log 2>&1 &
The above is the example, I would like to ask
What does ">" means
What does "2>&1" means
At the end "&" means?
Can anyone tell me the detail of how these syntax works.
Thanks a lot!!
What does ">" means
This redirects standard out (stdout) to a file (gpfdist.log).
What does "2>&1" means
This redirects standard error (stderr) to stdout so that errors are also logged to the same file normal messages are sent.
At the end "&" means?
This means run the program in the background.
This is all standard Unix stuff and not directly related to gpfdist.
Related
i know what they are, but i dunno when i should use them. Are they useful? I think yes, but I want you to tell me in which situations a file descriptor could be useful. Thanks :D
The most obvious case which springs to mind is:
myProgram >myProgram.output_and_error 2>&1
which sends both standard output and error to the same file.
I've also used:
myProgram 2>&1 | less
which will allow me to page through the output and error in sequence (rather than having error got to the terminal in "arbitrary" places in the output).
Basically, any time when you need to get at an already existing file descriptor, you'll find yourself using this.
I am running Windows 10 and am trying to save the error output of a test.sh file to a text file.
So I created the test.sh file and wrote an unkown command in it (i.e. "blablubb").
After that I open the terminal (cmd.exe), switch to the directory and type test.sh 2>> log.txt.
Another window opens with "/usr/bin/bash --login -i \test.sh" in the title bar, shows me "bash: blablubb: command not found" and then closes immediately.
I want to save that output because the bash-window just opens for a split second. Every google search brings me to websites talking about redirecting the output and that Stream2 ist STDERR and therefore I should use test.sh 2>> log.txt or something smiliar that takes care of the STDERR stream.
If I try the same with a test.sh file and the content:
#!/bin/bash
echo hi there
I get the output in the briefly open bash-window:
bash: #!/bin/bash: No such file or directory
hi there
But the log.txt file is empty.
If I only have echo hi therein the test.sh file I get bash: echo: command not found in the bash-window.
The log.txt also empty.
If I type the following directly in the terminal, the output is written in the log.txt:
echo hi > log.txt 2>&1
If I type directly in the terminal:
echdo hi > log.txt 2>&1
I get 'Der Befehl "echdo" ist entweder falsch geschrieben oder konnte nicht gefunden werden.' in the log.txt file.
So I guess the redirecting of the output works fine until I use test.sh.
I know that .sh files are something from the unix world and that the problem might lie there but I don't know why I can not redirect the output briefly shown in the bash-console to a text file.
The 2>> redirection syntax only works if the command line containing that syntax is interpreted by bash. So it won't work from the Windows command prompt, even if the program you are running happens to be written in bash. By the time bash is running, it's too late; it gets the arguments as they were interpreted by CMD or whatever your Windows command interpreter is. (In this case, I'm guessing that means the shell script will find it has a command line argument [$1] with the value "2".)
If you open up a bash window (or just type bash in the command one) and then type the test.sh 2>>log.txt command line in that shell, it will put the error message in the file as you expect.
I think you could also do it in one step by typing bash -c "test.sh 2>>log.txt" at the Windows command prompt, but I'm not sure; Windows quoting is different than *nix quoting, and that may wind up passing literal quotation marks to bash, which won't work.
Note that CMD does have the 2>> syntax as well, and if you try to run a nonexistent windows command with 2>>errlog.txt, the "is not recognized" error message goes to the file. I think the problem comes from the fact that CMD and bash disagree on what "standard error" means, so redirecting the error output from Windows doesn't catch the error output by bash. But that's just speculation; I don't have a bash-on-Windows setup handy to test.
It would help to know if you are running Windows Subsystem for Linux (Beta). Or if you are doing something else. I'm assuming this is what you are doing on windows 10.
If this is the case are you using bash to run the script?
Are you using win-bash?
If it is win-bash I'm not very familiar and would recommend Windows Subsystem for Linux (Beta) for reasons like this. win-bash, while cool, might not be compatible with redirection operators like 2>>.
You have stdout and stderr, by default (if you don't specify) the >> (or append) will only append standard output into the txt file.
If you use 2 it will append the standard error into the txt file. Example: test.sh 2>> log.txt
This could be better described at this site.
To get exactly the command for append both stdout and stderr, go to this page.
Please tell me if this doesn't answer your question. Also, it could be more valuable to attempt a search for this answer first and explain why your search found nothing or give more extensive clarification as to what the problem is. I love answering questions and helping, but creating a new forum page for what might be an easy answer may be ineffective. I've had a bunch of fun with your question. I hope that I've helped.
That's makes a lot of sense. Thanks Mark!
Taking what mark says into account I would get Windows Subsystem for Linux (Beta). There are instructions here. Then run your script from there.
I need to start up a Golang web server and leave it running in the background from a bash script. If the script in question in syntactically correct (as it will be most of the time) this is simply a matter of issuing a
go run /path/to/index.go &
However, I have to allow for the possibility that index.go is somehow erroneous. I should explain that in Golang this for something as "trival" as importing a module that you then fail to use. In this case the go run /path/to/index.go bit will return an error message. In the terminal this would be something along the lines of
index.go:4:10: expected...
What I need to be able to do is to somehow change that command above so I can funnel any error messages into a file for examination at a later stage. I tried variants on go run /path/to/index.go >> errors.txt with the terminating & in different positions but to no avail.
I suspect that there is a bash way to do this by altering the priority of evaluation of the command via some judiciously used braces/brackets etc. However, that is way beyond my bash capabilities. I would be most obliged to anyone who might be able to help.
Update
A few minutes later... After a few more experiments I have found that this works
go run /path/to/index.go &> errors.txt &
Quite apart from the fact that I don't in fact understand why it works there remains the issue that it produces a 0 byte errors.txt file when the command goes to completion without Golang throwing up any error messages. Can someone shed light on what is going on and how it might be improved?
Taken from man bash.
Redirecting Standard Output and Standard Error
This construct allows both the standard output (file descriptor 1) and the standard error output (file descriptor 2) to be redirected to the file whose name is the expansion of word.
There are two formats for redirecting standard output and standard error:
&>word
and
>&word
Of the two forms, the first is preferred. This is semantically equivalent to
>word 2>&1
Appending Standard Output and Standard Error
This construct allows both the standard output (file descriptor 1) and the standard error output (file descriptor 2) to be appended to the file whose name is the expansion of word.
The format for appending standard output and standard error is:
&>>word
This is semantically equivalent to
>>word 2>&1
Narūnas K's answer covers why the &> redirection works.
The reason why the file is created anyway is because the shell creates the file before it even runs the command in question.
You can see this by trying no-such-command > file.out and seeing that even though the shell errors because no-such-command doesn't exist the file gets created (using &> on that test will get the shell's error in the file).
This is why you can't do things like sed 'pattern' file > file to edit a file in place.
I am compiling a kernel module and it has many compilation errors in it. After running "make", the errors thrown out are too many to fit in the screen. Scrolling up doesn't reach the first error. I tried capturing the errors by doing make &2 > log which didn't work (log file was empty and the error messages were still dumped on screen).
Can someone please tell me how to go about logging all the messages generated during compilation/make into a logfile?
If you want to watch it scroll past, too:
make 2>&1 | tee log
(/bin/sh, bash and related) This sends the standard error to the same place as the standard output, then pipes them through tee to capture the result and still get screen action.
Try doing:
make >&log
the & after the > tells the shell to dump both stdout and stderr to the log. This can also be used with pipes.
I think stdout, so you can easily grep, what do you think?
Only errors go to stderr. This is in no way an error, it does exactly what the user had in mind, which is print usage information.
Always stdout, makes it easier to pipe to less, grep it etc.
If you are showing the help text because there was a problem with parsing the command line arguments, then you might use stderr.
Well, it's an explicit request for help so it's output. If for some reason you can't output the help or the user mis-spells "help" then, by all means, send that to error :-)
Users that know what they're doing can use the infamous "2>&1" if they want errors on standard output.
It's not an error, so I'd say stdout....
netcat is the only application I can think of that would redirect -h to stderr, and I can't for the life of me fathom why.
I suppose if you're outputting the help information because someone used improper arguments, you might want to redirect it to stderr, but personally even then I wouldn't use stderr because I don't think spamming error logs with fullblown help text is useful - I'd rather just output a single error pointing out the arguments were malformed to stderr. If someone is explicitly calling your application using -h or --help, then you really shouldn't redirect it to stderr.