AIX Version 6.1
I'm trying to write a script to pull times out of a program to send to Zabbix, but I'm wanting to modify the formatting of the times returned.
At present, when I pull the time it returns like so: [15:48:30]
My goal is to remove the brackets ([]) to then be able to pull the time apart with awk to do calculations on the time to render it down into seconds and draw relevant information from that.
AIX is continually giving me errors with every form of text formatting I can think of/find.
Ex: echo $unformattedtime | awk '{print substr($0,1,8)}'
gives me permissions errors when I've already chmod 777 the script. I've seen fixes that require going in and making root changes, and while that's possible for me to do, the script needs to run as a non-root user for what it's being designed to do.
Failing manipulating the unformattedtime variable, I tried putting it into a text file and manipulating it with tr.
Ex: tr -s [] '' < timet.txt > timetformat.txt
Where timet.txt simply had the '[15:48:30]' put in using vi. This simply returned a "tr:Error 0"
Is there some sort of AIX specific method of doing modifications that I'm missing? Or just anything that would accomplish the goal here?
Thanks!
Answer from my comments:
tr -s is going to leave single square brackets unmolested, this isn't what you want. (Though I don't understand that error unless it is a quoting issue on [] in the shell. Try '[]' instead.)
tr -d '[]' is probably more what you want there.
It might also be possible, depending on the shell, to avoid the temporary files by using a HEREDOC:
tr -d '[]' <<EOF
echo "$unformattedtime"
EOF
There can be no leading spaces before the final EOF there for the record.
Related
I have a bash script that includes a line like this:
matches="`grep --no-filename $searchText $files`"
In other words, I am assigning the result of a grep to a variable.
I recently found that that line of code seems to have a vulnerability: if the grep finds too many results, it annoyingly simply freezes execution.
First, if anyone can confirm that excessive output (and exactly what constitutes excessive) is a known danger with command substitution, please provide a solid link for me. I web searched, and the closest reference that I could find is in this link:
"Do not set a variable to the contents of a long text file unless you have a very good reason for doing so."
That hints that there is a danger, but is very inadequate.
Second, is there a known best practice for coping with this?
The behavior that I really want is for excessive output in command substitution
to generate a nice human readable error message followed by an error exit code so that my script will terminate instead of freeze. (Note: I always run my scripts with "set -e" as one of the initial lines). Is there any way that I can get this behavior?
Currently, the only solution that I know of is a hack that sorta works just for my immediate case: I can limit the output from grep using its --max-count option.
Ideally, you shouldn't capture data of unknown length into memory at all; if you read it as you need it, then grep will wait until the content is ready to use.
That is:
while IFS= read -r match; do
echo "Found a match: $match"
# example: maybe we want to look at whether a match exists on the filesystem
[[ -e $match ]] && { echo "Got what we needed!" >&2; break; }
done < <(grep --no-filename "$searchText" "${files[#]}")
That way, grep only writes a line when read is ready to consume it (and will block instead of needing to continue to read input if it has more output already produced than can be stored in the relatively small pipe buffer) -- so the names you don't need don't even get generated in the first place, and there's no need to allocate memory or deal with them in any other way.
I'm trying to script something that isn't outputting quite correctly with the date command. Here's the contents of what I have thus far:
#!/bin/bash
# Get RPM manifest
# Output written to /tmp
NOW=$(date +%D)
rpm -qa --qf="%{NAME}.%{ARCH}\n" | sort > /tmp/$HOSTNAME.RPM_Manifest.$NOW.txt
When I run this script, I get this message:
[root#linmachine1 ~]# sh /usr/local/bin/rpm_manifest.sh
/usr/local/bin/rpm_manifest.sh: line 7: /tmp/linmachine1.RPM_Manifest.03/01/17.txt: No such file or directory
I suspect the problem is in how the date formatting within the NOW variable I'm defining may be the culprit. I've tried with and without quotes and get the same thing. Looking at the man pages, I didn't see a way to change the default behavior such that the forward slashes would be replaced by dots, as I believe this is where the problem lies.
EDIT: Thanks for all of your responses. I'm not real sure why this was downvoted though. I asked a legitimate question. What gives?
Yes, you shouldn't have slashes in a file name.
Use:
now=$(date "+%d.%m.%Y")
rpm -qa --qf="%{NAME}.%{ARCH}\n" | sort > "/tmp/$HOSTNAME.RPM_Manifest.$now.txt"
instead, or replace the . with whatever you prefer
I have an output file (namely a log from screen) containing several control characters. Inside the screen, I have programs running that use control characters to refresh certain lines (examples would be top or anything printing progress bars).
I would like to output a tail of this file using PHP. If I simply read in that file and echo its contents (either using PHP functions or through calling tail, the output is messy and much more than these last lines as it also includes things that have been overwritten. If I instead run tail in the command line, it returns just what I want because the terminal evaluates the control characters.
So my question is: Is there a way to evaluate the control characters, getting the output that a terminal would show me, in a way that I could then use elsewhere (e.g., write to a file)?
#5gon12eder's answer got rid of some control characters (thanks for that!) but it did not handle the carriage return part that was even more important to me.
I figured out that I could just delete anything from the beginning of a line to the last carriage return inside that line and simply keep everything after that, so here is my sed command accomplishing that:
sed 's/^.*\r\([^\r]\+\)\r\?$/\1\r/g'
The output can then be further cleaned using #5gon12eder's answer:
cat screenlog.0 | sed 's/^.*\r\([^\r]\+\)\r\?$/\1\r/g' | sed 's,\x1B\[[0-9?;]*[a-zA-Z],,g'
Combined, this looks exactly like I wanted.
I'm not sure what you mean by “evaluating” the control characters but you could remove them easily.
Here is an example using sed but if you are already using PHP, its internal regex processing functionality seems more appropriate. The command
$ sed 's,\x1B\[[0-9?;]*[a-zA-Z],,g' file.dat
will dump the contents of file.dat to standard output with all ANSI escape sequences removed. (And I'm pretty sure that nothing else is removed except if your file contains invalid escape sequences in which case the operation is ill-defined anyway.)
Here is a little demo:
$ echo -e "This is\033[31m a \033[umessy \033[46mstring.\033[0m" > file.dat
$ cat file.dat
# The output of the above command is not shown to protect small children
# that might be browsing this site.
$ reset # your terminal
$ sed 's,\x1B\[[0-9?;]*[a-zA-Z],,g' file.dat
This is a messy string.
The less program has some more advanced logic built in to selectively replace some escape sequences. Read the man page for the relevant options.
In a shell, I run following commands without problem,
ls -al
!ls
the second invocation to ls also list files with -al flag. However, when I put the above script to a bash script, complaints are thrown,
!ls, command not found.
how to realise the same effects in script?
You would need to turn on both command history and !-style history expansion in your script (both are off by default in non-interactive shells):
set -o history
set -o histexpand
The expanded command is also echoed to standard error, just like in an interactive shell. You can prevent that by turning on the histverify shell option (shopt -s histverify), but in a non-interactive shell, that seems to make the history expansion a null-op.
Well, I wanted to have this working as well, and I have to tell everybody that the set -o history ; set -o histexpand method will not work in bash 4.x. It's not meant to be used there, anyway, since there are better ways to accomplish this.
First of all, a rather trivial example, just wanting to execute history in a script:
(bash 4.x or higher ONLY)
#!/bin/bash -i
history
Short answer: it works!!
The spanking new -i option stands for interactive, and history will work. But for what purpose?
Quoting Michael H.'s comment from the OP:
"Although you can enable this, this is bad programming practice. It will make your scripts (...) hard to understand. There is a reason it is disabled by default. Why do you want to do this?"
Yes, why? What is the deeper sense of this?
Well, THERE IS, which I'm going to demonstrate in the follow-up section.
My history buffer has grown HUGE, while some of those lines are script one-liners, which I really would not want to retype every time. But sometimes, I also want to alter these lines a little, because I probably want to give a third parameter, whereas I had only needed two in total before.
So here's an ideal way of using the bash 4.0+ feature to invoke history:
$ history
(...)
<lots of lines>
(...)
1234 while IFS='whatever' read [[ $whatever -lt max ]]; do ... ; done < <(workfile.fil)
<25 more lines>
So 1234 from history is exactly the line we want. Surely, we could take the mouse and move there, chucking the whole line in the primary buffer? But we're on *NIX, so why can't we make our life a bit easier?
This is why I wrote the little script below. Again, this is for bash 4.0+ ONLY (but might be adapted for bash 3.x and older with the aforementioned set -o ... stuff...)
#!/bin/bash -i
[[ $1 == "" ]] || history | grep "^\s*$1" |
awk '{for (i=2; i<=NF; i++) printf $i" "}' | tr '\n' '\0'
If you save this as xselauto.sh for example, you may invoke
$ ./xselauto.sh 1234
and the contents of history line #1234 will be in your primary buffer, ready for re-use!
Now if anyone still says "this has no purpose AFAICS" or "who'd ever be needing this feature?" - OK, I won't care. But I would no longer want to live without this feature, as I'm just too lazy to retype complex lines every time. And I wouldn't want to touch the mouse for each marked line from history either, TBH. This is what xsel was written for.
BTW, the tr part of the pipe is a dirty hack which will prevent the command from being executed. For "dangerous" commands, it is extremely important to always leave the user a way to look before he/she hits the Enter key to execute it. You may omit it, but ... you have been warned.
P.S. This scriptlet is in fact a workaround, simulating !1234 typed on a bash shell. As I could never make the ! work directly in a script (echo would never let me reveal the contents of history line 1234), I worked around the problem by simply greping for the line I wanted to copy.
History expansion is part of the interactive command-line editing features of a shell, not part of the scripting language. It's not generally available in the context of a script, only when interacting with a (pseudo-)human operator. (pseudo meaning that it can be made to work with things like expect or other keystroke repeating automation tools that generally try to play act a human, not implying that any particular operator might be sub-human or anything).
I'm implementing an interactive bash script similar to the MySQL client, /usr/bin/mysql. In this script I need to issue various types of 'commands'. I also need to provide a history mechanism whereby the user can use the up/down arrow keys to scroll through the commands entered so far.
The snippet listed here (Example 15-6, Detecting the arrow keys) does not exactly do what I want it to. I really want the following:
The up/down arrow keys should operate in silent mode. Meaning, they should not echo their character codes on the terminal.
The other keys however (which will be used to read the command names and their arguments) must not operate in silent mode.
The problem with read -s -n3 is that it does not satisfy my simultaneously conflicting requirements of silent mode and echo mode, based solely on the character code. Also, the value -n3 will work for arrow keys but, for other/regular keys, it won't 'return control' to the calling program until 3 characters have been consumed.
Now, I could try -n1 and manually assemble the input, one character at a time (yuck!). But the character-code based silent-/echo-mode switching problem would still persist!
Has anyone attempted this thing in bash? (Note: I cannot use C, nor other scripting languages like Perl, Python, etc.)
EDIT
Continuing with Dennis' answer... You will also need to manually add your desired entries to your history via history -s, like so...
while read -e x; do
history -s "$x"
# ...
done
You can use read -e to have read use readline. It will process your cursor keys and maintain the history for you. You will also need to manually add your desired entries to your history via history -s, like so:
while read -e x; do
history -s "$x"
# ...
done
MySQL and Bash use the Readline library to implement this. Maybe you could use something like rlwrap or rlfe?
rlwrap has a special "one-shot" mode to act as a replacement for the 'read' shell command. If you wish, every occurrence of this command in your script can be given its own history and completion word list.
Use it like this:
REPLY=$(rlwrap -o cat)
or, specifying a history file and a completion wordlist:
REPLY=$(rlwrap -H my_history -f my_completions -o cat)