While using the ESAPI executor to run a simple command such as:
C:\\Windows\\System32\\cmd.exe /c echo test >> D:\\wow\\test.txt
while the C:\Windows\System32\cmd.exe is the executable file.
and the /c echo test >> D:\wow\test.txt are parameters.
The problem is that the codec ruins the operator >> (which means avoiding it or referring to it as a String. so the output will be just be printing the "test >> D:\wow\test.txt".
The codec is doing exactly what it should be doing. Maybe an overly simple example would make that more clear as to why this is the correct behavior.
Let's suppose you have a web application where you have a user upload a document and then you run a spell-check on it for them and send them back the corrected results. For sake of simplicity, left's assume that you have set the current working directory to be specific to each user using your service and that you have a "spellcheck.bat" that takes a single filename as an argument and you planned to execute the string:
C:\Windows\System32\cmd.exe /c C:\spelling\spellcheck.bat userFilename > spelling-errors.txt
where userFilename is the name of the users uploaded file. Your intent is to run spellcheck.bat with the user's uploaded file and filename and the read and display the 'spelling-errors.txt' file back to the user. (IRL, that would probably be a temp file in a different directory, but I want to make this simple.)
So what happens if 'userFilename' contains a '>' character? Surely you want that quoted appropriately, do you not? Because otherwise an attacker might be able to overwrite some file that you don't want to be overwritten, including things like your web.xml or other configuration / properties files. Because running something like (say)
C:\Windows\System32\cmd.exe /c C:\spelling\spellcheck.bat myDoc.txt >sucker.txt > spelling-errors.txt
is going to create 'sucker.txt' and if that filename exists, it will be truncated and replaced by the output of "spellcheck.bat myDoc.txt". That is probably NOT what you want.
So preventing file I/O redirection operations via '<', '<<', '>', and '>>' is intended. If you really want to allow them (and I recommend against it), then you will have to parse your own command string based on what you want to allow.
Also, while I'm not sure it is true for Window's cmd prompt shell, it certainly is true for *nix shells that one can also do command execution in the redirection as part of the file redirection. For example, in bash, I can do something like this
cat < $(ls foo)
and if the file 'foo' exists, the 'ls foo' command will list the output foo, so this command becomes:
cat < foo
and we just end up taking stdin for the 'cat' command from the file 'foo'. But an attacker could do something much more nefarious with it. Of course, the OS codec also prevents '$(...)' style command substitutions, but I hopefully have made my point.
Related
Examples:
Create an ISO image and burn it directly to a CD.
mkisofs -V Photos -r /home/vivek/photos | cdrecord -v dev=/dev/dvdrw -
Change to the previous directory.
cd -
Listen on port 12345 and untar data sent to it.
nc -l -p 12345 | tar xvzf -
What is the purpose of the dash and how do I use it?
If you mean the naked - at the end of the tar command, that's common on many commands that want to use a file.
It allows you to specify standard input or output rather than an actual file name.
That's the case for your first and third example. For example, the cdrecord command is taking standard input (the ISO image stream produced by mkisofs) and writing it directly to /dev/dvdrw.
With the cd command, every time you change directory, it stores the directory you came from. If you do cd with the special - "directory name", it uses that remembered directory instead of a real one. You can easily switch between two directories quite quickly by using that.
Other commands may treat - as a different special value.
It's not magic. Some commands interpret - as the user wanting to read from stdin or write to stdout; there is nothing special about it to the shell.
- means exactly what each command wants it to mean. There are several common conventions, and you've seen examples of most of them in other answers, but none of them are 100% universal.
There is nothing magic about the - character as far as the shell is concerned (except that the shell itself, and some of its built-in commands like cd and echo, use it in conventional ways). Some characters, like \, ', and ", are "magical", having special meanings wherever they appear. These are "shell metacharacters". - is not like that.
To see how a given command uses -, read the documentation for that command.
It means to use the program's standard input stream.
In the case of cd, it means something different: change to the prior working directory.
The magic is in the convention. For millennia, people have used '-' to distinguish options from arguments, and have used '-' in a filename to mean either stdin or stdout, as appropriate. Do not underestimate the power of convention!
I'm writing commands that do something like ./script > output.txt so that I can use the files in later scripts like ./script2 output.txt otherFile.txt > output2.txt. I remove them all at the end of the script, but when I'm testing certain things or debugging it's tricky to search through all my sub directories and files which have been created in the script.
Is the best option just to create a hidden file?
As always, there are numerous ways to do so. If you want to avoid files altogether, you can save the output (STDOUT) of a command in a variable and pass it to the next command as a file using the <() operator:
output=$(cat /usr/include/stdio.h)
cat <(echo "$output")
Alternatively, you can do so in a single command line:
cat <(cat /usr/include/stdio.h)
This assumes that the next command strictly requires a file for input.
I tend to avoid temporary files whenever possible to eliminate the need for a cleanup step that gets executed in all cases unless large amounts of data have to be processed.
I need to start up a Golang web server and leave it running in the background from a bash script. If the script in question in syntactically correct (as it will be most of the time) this is simply a matter of issuing a
go run /path/to/index.go &
However, I have to allow for the possibility that index.go is somehow erroneous. I should explain that in Golang this for something as "trival" as importing a module that you then fail to use. In this case the go run /path/to/index.go bit will return an error message. In the terminal this would be something along the lines of
index.go:4:10: expected...
What I need to be able to do is to somehow change that command above so I can funnel any error messages into a file for examination at a later stage. I tried variants on go run /path/to/index.go >> errors.txt with the terminating & in different positions but to no avail.
I suspect that there is a bash way to do this by altering the priority of evaluation of the command via some judiciously used braces/brackets etc. However, that is way beyond my bash capabilities. I would be most obliged to anyone who might be able to help.
Update
A few minutes later... After a few more experiments I have found that this works
go run /path/to/index.go &> errors.txt &
Quite apart from the fact that I don't in fact understand why it works there remains the issue that it produces a 0 byte errors.txt file when the command goes to completion without Golang throwing up any error messages. Can someone shed light on what is going on and how it might be improved?
Taken from man bash.
Redirecting Standard Output and Standard Error
This construct allows both the standard output (file descriptor 1) and the standard error output (file descriptor 2) to be redirected to the file whose name is the expansion of word.
There are two formats for redirecting standard output and standard error:
&>word
and
>&word
Of the two forms, the first is preferred. This is semantically equivalent to
>word 2>&1
Appending Standard Output and Standard Error
This construct allows both the standard output (file descriptor 1) and the standard error output (file descriptor 2) to be appended to the file whose name is the expansion of word.
The format for appending standard output and standard error is:
&>>word
This is semantically equivalent to
>>word 2>&1
Narūnas K's answer covers why the &> redirection works.
The reason why the file is created anyway is because the shell creates the file before it even runs the command in question.
You can see this by trying no-such-command > file.out and seeing that even though the shell errors because no-such-command doesn't exist the file gets created (using &> on that test will get the shell's error in the file).
This is why you can't do things like sed 'pattern' file > file to edit a file in place.
While debugging I want to display console output both on console and save a backup in file.
Windows doesn't have tee, but you can add one. Say the folder is c:\bin\ and it works fine. And I have added it into system's PATH.
Problem is setting "[ ]| tee[.exe] output.txt" or " | tee[.exe] output.txt" won't work -- the output.txt is just nowhere to be found. I also tried to add the c:\bin\ path explicitly in VC Directories or environment under debugging and merge environment to be yes.
"> output.txt" works fine.
Anyone has any idea how I can resolve this? Many thanks!
I assume that you're putting the | tee.exe output.txt string in the project property "Debugging | Command Argument".
Unfortunately, that property only supports the redirection operators, not the pipe operator. If you have the | tee.exe output.txt string in the preoperty and run a program that dumps the command line arguments, you'll see that that information is just passed on as the arguments. The "Debugging | Command Argument" doesn't actually get processed by a full-fledged shell (such as cmd.exe) - it's just the IDE supporting some simple redirection (actually, it seems to support more than I expected):
From http://msdn.microsoft.com/en-us/library/kcw4dzyf.aspx:
You can use the following redirection operators in this box:
< file
Reads stdin from file.
> file
Writes stdout to file.
>> file
Appends stdout to file.
2> file
Writes stderr to file.
2>> file
Appends stderr to file.
2> &1
Sends stderr (2) output to same location as stdout (1).
1> &2
Sends stdout (1) output to same location as stderr (2).
You can have a limited version of what you're looking for by redirecting the program's output to a file using >> and using a tail-f command to display whatever gets added to the file. If you do this you'll probably want to call setvbuf( stdout, NULL, _IONBF, 0 ) first thing in main() so that I/O is unbuffered. Otherwise tail -f won't see it until the buffer gets flushed, and I imagine that you'd like to see each output operation as it occurs.
Another option is to crank the console window's "Screen Buffer Height" property up to a large number - one of the first things I do when I get a new Windows machine is set that value to 3000 or so - then debug the program normally and copy/paste the contents of the console window before it closes.
You better NOT use printf for this purpose. Instead, write your own function; taking formatted-input, like printf - having variable number of arguments (...). That function will use printf to display on console, get the buffer written on file, would send to output to debug window and all. You may customize it depending on Debug/Release build.
It may go like (may have some minor mistakes):
void PrintDebuggingInfo(const char* pFormatString, ...)
{
va_list arguments;
char OutputString[1024];
va_start(pFormatString, argument);
vsprintf(OutputString, pFormatString, argument); // Generate string
// Now use `OutputString` as you wish!
}
You may use other variant of vsprintf. Infact all formatted-functions use this function only!
I am a newbie to unix scripting, I want to do following and I have little clue how to proceed.
I want to log the input and output of certain set of commands, given on the terminal, to a trace file. I should be able to switch it on and off.
E.g.
switch trace on
user:echo Hello World
user:Hello World
switch trace off
Then the trace log file, e.g. trace.log, it's content should be
echo Hello World
Hello World
One thing that I can think to do is to use set -x, redirecting its output to some file, but couldn't find a way to do that. I did man set, or man -x but I found no entry. Maybe I am being too naive, but some guidance will be very helpful.
I am using bash shell.
See script(1), "make typescript of terminal session". To start a new transcript in file xyz: script xyz. To add on to an existing transcript in file xyz: script -a xyz.
There will be a few overhead lines, like Script started on ... and Script done on ... which you could use awk or sed to filter out on printout. The -t switch allows a realtime playback.
I think there might have been a recent question regarding how to display a transcript in less, and although I can't find it, this question and this one address some of the same issues of viewing a file that contains control characters. (Captured transcripts often contain ANSI control sequences and usually contain Returns as well as Linefeeds.)
Update 1 A Perl program script-declutter is available to remove special characters from script logs.
The program is about 45 lines of code found near the middle of the link. Save those lines of code in a file called script-declutter, in a subdirectory that's on your PATH (for example, $HOME/bin if that's on your search path, else (eg) /usr/local/bin) and make the file executable. After that, a command like
script-declutter typescript > out
will remove most special characters from file typescript,
while directing the result to file out.