In the OSX terminal :
du -h | sort -nr
-bash: sort: command not found
which sort
/usr/bin/sort
The weird thing is: I tried reproducing the error and it seems to be totally random. My PATH echoed:
/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/X11/bin:/usr/local/git/bin:/usr/texbin:/Users/sytycs/.rvm/bin
This only occurs when piping and happens with grep, more, less etc.
Any idea what is causing this?
This likely happens because you use a keyboard layout with a non-US layout (happened to me too). On German layouts, the pipe is typed with Alt+7, so chances are high that you press Alt+Space afterwards, which will produce a non-breaking space.
A quick solution: Add the line
"\xC2\xA0": " "
to your ~/.inputrc (if you are using bash). This will map non-breaking spaces to normal ones which should solve the problem.
If you want more detail (or if you are interested in how you can track down these kinds of issues), I wrote a blog post about it some time ago.
That space is not a space. Erase and replace it.
The trick with ~/.inputrc doesn't work for zsh. But here you can configure iTerm to send a space when you type alt+space, for instance
Bash is so sensitive to space when you are piping. Remove them all.
I was facing the same problem by running this command:
|awk '{$1=$1};1' | tr '[:upper:]' '[:lower:]' | sort | uniq
| awk '{ print length, $0 }' | sort -n -s | cut -d" " -f2- > dest_file
the errors were as here:
zsh: command not found: tr
zsh: command not found: sort
then I removed all the spaced and it got resolved:
|awk '{$1=$1};1'|tr '[:upper:]' '[:lower:]'|sort|uniq|awk '{ print length, $0
}'|sort -n -s|cut -d" " -f2- >
Related
I want to cut everything with a delimiter ":" The input file is in the following format:
data1:data2
data11:data22
...
I have a linux command
cat merged.txt | cut -f1 -d ":" > output.txt
On mac terminal it gives an error:
cut: stdin: Illegal byte sequence
what is the correct way to do it on a mac terminal?
Your input file (merged.txt) probably contains bytes/byte sequences that are not valid in your current locale. For example, your locale might specify UTF-8 character encoding, but the file be in some other encoding and cannot be parsed as valid UTF-8. If this is the problem, you can work around it by telling tr to assume the "C" locale, which basically tells it to process the input as a stream of bytes without paying attention to encoding.
BTW, cat file | is what's commonly referred to as a Useless Use of Cat (UUOC) -- you can just use a standard input redirect < file instead, which cleaner and more efficient. Thus, my version of your command would be:
LC_ALL=C cut -f1 -d ":" < merged.txt > output.txt
Note that since the LC_ALL=C assignment is a prefix to the tr command, it only applies to that one command and won't mess up other operations that should assume UTF-8 (or whatever your normal locale is).
Your cut command works for me on my Mac, you can try awk for the same result
awk -F: '{print $1}' merged.txt
data1
data11
I am trying to run this command on MacOSX terminal , which was initially intended to run on Linux
sed '1 i VISPATH=/mnt/local/gdrive/public/3DVis' init.txt >> ~/.bash_profile
but it gives me the error:
command i expects \ followed by text.
is there any way I could modify the above command to work on MacOSX terminal
Shelter is right but there's another way to do it. You can use the bash $'...' quoting to interpret the escapes before passing the string to sed.
So:
sed -iold '1i\'$'\n''text to prepend'$'\n' file.txt
^^^^^^^^ ^
/ |\|||/ \ |__ No need to reopen
| | \|/ | string to sed
Tells sed to | | | |
escape the next _/ | | +-----------------------------+
char | +-------------+ |
| | |
Close string The special bash Reopen string to
to sed newline char to send to sed
send to sed
This answer on unix.stackexchange.com led me to this solution.
Had the same problem and solved it with brew:
brew install gnu-sed
gsed YOUR_USUAL_SED_COMMAND
If you want to use the sed command, then you can set an alias:
alias sed=gsed
The OSX seds are based on older versions, you need to be much more literal in your directions to sed, AND you're lucky, in this case, sed is telling you exactly what to do. Untested as I don't have OSX, but try
sed '1 i\
VISPATH=/mnt/local/gdrive/public/3DVis
' init.txt >> ~/.bash_profile
Input via the i cmd is terminated by a blank line. Other sed instructions can follow after that. Note, NO chars after the \ char!
Also, #StephenNiedzielski is right. Use the single quote chars to wrap your sed statements. (if you need variable expansion inside your sed and can escape other uses of $, then you can also use dbl-quotes, but it's not recommended as a normal practices.
edit
As I understand now that you're doing this from the command-line, and not in a script or other editor, I have tested the above, and.... all I can say is that famous line from tech support ... "It works for me". If you're getting an error message
sed: -e expression #1, char 8: extra characters after command
then you almost certainly have added some character after the \. I just tested that, and I got the above error message. (I'm using a linux version of sed, so the error messages are exactly the same). You should edit your question to include an exact cut-paste of your command line and the new error message. Using curly-single-quotes will not work.
IHTH
Here's how I worked it out on OS X. In my case, I needed to prepend text to a file. Apparently, modern sed works like this:
sed -i '1i text to prepend' file.txt
But on OS X I had to do the following:
sed -i '' '1i\
text to prepend
' file.txt
It looks like you copied rich text. The single quotes should be straight not curly:
sed '1 i VISPATH=/mnt/local/gdrive/public/3DVis'
I am writing shell script for embedded Linux in a small industrial box. I have a variable containing the text pid: 1234 and I want to strip first X characters from the line, so only 1234 stays. I have more variables I need to "clean", so I need to cut away X first characters and ${string:5} doesn't work for some reason in my system.
The only thing the box seems to have is sed.
I am trying to make the following to work:
result=$(echo "$pid" | sed 's/^.\{4\}//g')
Any ideas?
The following should work:
var="pid: 1234"
var=${var:5}
Are you sure bash is the shell executing your script?
Even the POSIX-compliant
var=${var#?????}
would be preferable to using an external process, although this requires you to hard-code the 5 in the form of a fixed-length pattern.
Here's a concise method to cut the first X characters using cut(1). This example removes the first 4 characters by cutting a substring starting with 5th character.
echo "$pid" | cut -c 5-
Use the -r option ("use extended regular expressions in the script") to sed in order to use the {n} syntax:
$ echo 'pid: 1234'| sed -r 's/^.{5}//'
1234
Cut first two characters from string:
$ string="1234567890"; echo "${string:2}"
34567890
pipe it through awk '{print substr($0,42)}' where 42 is one more than the number of characters to drop. For example:
$ echo abcde| awk '{print substr($0,2)}'
bcde
$
Chances are, you'll have cut as well. If so:
[me#home]$ echo "pid: 1234" | cut -d" " -f2
1234
Well, there have been solutions here with sed, awk, cut and using bash syntax. I just want to throw in another POSIX conform variant:
$ echo "pid: 1234" | tail -c +6
1234
-c tells tail at which byte offset to start, counting from the end of the input data, yet if the the number starts with a + sign, it is from the beginning of the input data to the end.
Another way, using cut instead of sed.
result=`echo $pid | cut -c 5-`
I found the answer in pure sed supplied by this question (admittedly, posted after this question was posted). This does exactly what you asked, solely in sed:
result=\`echo "$pid" | sed '/./ { s/pid:\ //g; }'\``
The dot in sed '/./) is whatever you want to match. Your question is exactly what I was attempting to, except in my case I wanted to match a specific line in a file and then uncomment it. In my case it was:
# Uncomment a line (edit the file in-place):
sed -i '/#\ COMMENTED_LINE_TO_MATCH/ { s/#\ //g; }' /path/to/target/file
The -i after sed is to edit the file in place (remove this switch if you want to test your matching expression prior to editing the file).
(I posted this because I wanted to do this entirely with sed as this question asked and none of the previous answered solved that problem.)
Rather than removing n characters from the start, perhaps you could just extract the digits directly. Like so...
$ echo "pid: 1234" | grep -Po "\d+"
This may be a more robust solution, and seems more intuitive.
This will do the job too:
echo "$pid"|awk '{print $2}'
I am working with an idea using the unix who command. As we all know, there does not seem to be a direct switch that gives just the username and line (terminal) info without the date and screen info... eg: the output is mneedham tty7...2012-02-19 11:26 (:0)
What I am trying to get is just the mneedham tty7 part. The solution needs to work no matter how long the username and terminal information.
I tried using tr -s ' ' (one space) like who | tr -s ' ' and that gave me one space between everything. Not quite what I was seeking. Tried cut -d" " -f1 gets the username only. So I am hopeful someone can help me find the right command to get both bits of information.
Thanks.
Using cut:
who | cut -d " " -f1,2
Using awk:
who | awk '{ print $1, $2 }'
I'm trying to pipe data into "cut" to, say, cut away the first column of text. This works
$ cat test.txt | cut -d\ -f2-
Reading from stdin also works:
$ cut -d\ -f2- -
? doc/html/analysis.html
? doc/html/classxytree-members.html
<CTRL+D>
However, as soon as a pipe is involved, it doesn't accept my <CTRL+D> anymore, and I can't signal "end of file":
$ cut -d\ -f2- - | xargs echo
Update: This is apparently a bug in an old version of bash (3.00.15). It does work in more recent versions (tried 4.0.33 and 3.2.25). It would be nice to have some workaround, though, since I can't easily upgrade.
Background: I've got a script/oneliner that gives me a condensed output of cvs status (I know, CVS...) in the form
? filename
e.g. for a file not committed yet. I'd like to be able to copy+paste parts of the output from that command and use this as an input to another command, that adds these files to cvs. Say:
$ cut -d\ -f2- | xargs cvs add
<paste lines>
<CTRL-D> # <-- doesn't work
Ideas?
have you tried
$ cat | cut -d\ -f2- | xargs cvs add
<paste lines>
<CTRL-D> # <-- doesn't work
Your examples work fine for me. What shell are you using? What utilities?
One thing that sometimes trips people up is that Ctrl-D only works if it's the first character in the line. If you copy and paste, you might sometimes accidentally have whitespace as the first character of the line, or no newline at the end of the pasted block, in which case Ctrl-D won't work. Just hit return and then try Ctrl-D again and see if that fixes your problem.