I'm piping some hex ascii into perl and would like to use it's print function to print this stuff to a file. however the hex is not printed as ascii but as normal string
echo "\x48\x32\x35\x45" "\x90\xc1\xe2\x7a" | perl -e' #string = split " ",<>; print ${string[0]},chr(0),"$string[1]";' > input.txt
This gives however the following in input.txt
\x48\x32\x35\x45^#\x90\xc1\xe2\x7a
where it should more look like
H25E^#<90><C1><E2>z
You need evals (and extra double quotes) to make perl "interpret" the hex escapes:
echo "\x48\x32\x35\x45 \x90\xc1\xe2\x7a" | perl -e '#string = split " ",<>; print eval "\"$string[0]\"",chr(0),eval "\"$string[1]\"";'
Related
Hello everyone I'm a beginner in shell coding. In daily basis I need to convert a file's data to another format, I usually do it manually with Text Editor. But I often do mistakes. So I decided to code an easy script who can do the work for me.
The file's content like this
/release201209
a1,a2,"a3",a4,a5
b1,b2,"b3",b4,b5
c1,c2,"c3",c4,c5
to this:
a2>a3
b2>b3
c2>c3
The script should ignore the first line and print the second and third values separated by '>'
I'm half way there, and here is my code
#!/bin/bash
#while Loops
i=1
while IFS=\" read t1 t2 t3
do
test $i -eq 1 && ((i=i+1)) && continue
echo $t1|cut -d\, -f2 | { tr -d '\n'; echo \>$t2; }
done < $1
The problem in my code is that the last line isnt printed unless the file finishes with an empty line \n
And I want the echo to be printed inside a new CSV file(I tried to set the standard output to my new file but only the last echo is printed there).
Can someone please help me out? Thanks in advance.
Rather than treating the double quotes as a field separator, it seems cleaner to just delete them (assuming that is valid). Eg:
$ < input tr -d '"' | awk 'NR>1{print $2,$3}' FS=, OFS=\>
a2>a3
b2>b3
c2>c3
If you cannot just strip the quotes as in your sample input but those quotes are escaping commas, you could hack together a solution but you would be better off using a proper CSV parsing tool. (eg perl's Text::CSV)
Here's a simple pipeline that will do the trick:
sed '1d' data.txt | cut -d, -f2-3 | tr -d '"' | tr ',' '>'
Here, we're just removing the first line (as desired), selecting fields 2 & 3 (based on a comma field separator), removing the double quotes and mapping the remaining , to >.
Use this Perl one-liner:
perl -F',' -lane 'next if $. == 1; print join ">", map { tr/"//d; $_ } #F[1,2]' in_file
The Perl one-liner uses these command line flags:
-e : Tells Perl to look for code in-line, instead of in a file.
-n : Loop over the input one line at a time, assigning it to $_ by default.
-l : Strip the input line separator ("\n" on *NIX by default) before executing the code in-line, and append it when printing.
-a : Split $_ into array #F on whitespace or on the regex specified in -F option.
-F',' : Split into #F on comma, rather than on whitespace.
SEE ALSO:
perldoc perlrun: how to execute the Perl interpreter: command line switches
I would like to convert first character to capital and character coming after dash(-) needs to be converted to capital using bash.
I can split individual elements using - ,
echo "string" | tr [:lower:] [:upper:]
and join all but that doesn't seem effect. Is there any easy way to take care of this using single line?
Input string:
JASON-CONRAD-983636
Expected string:
Jason-Conrad-983636
I recommend using Python for this:
python3 -c 'import sys; print("-".join(s.capitalize() for s in sys.stdin.read().split("-")))'
Usage:
capitalize() {
python3 -c 'import sys; print("-".join(s.capitalize() for s in sys.stdin.read().split("-")))'
}
echo JASON-CONRAD-983636 | capitalize
Output:
Jason-Conrad-983636
In pure bash (v4+) without any third party utils
str=JASON-CONRAD-983636
IFS=- read -ra raw <<<"$str"
final=()
for str in "${raw[#]}"; do
first=${str:0:1}
rest=${str:1}
final+=( "${first^^}${rest,,}" )
done
and print the result
( IFS=- ; printf '%s\n' "${final[*]}" ; )
This might work for you (GNU sed):
sed 's/.*/\L&/;s/\b./\u&/g' file
Lowercase everything. Uppercase first characters of words.
Alternative:
sed -E 's/\b(.)((\B.)*)/\u\1\L\2/g' file
Could you please try following(in case you are ok with awk).
var="JASON-CONRAD-983636"
echo "$var" | awk -F'-' '{for(i=1;i<=NF;i++){$i=substr($i,1,1) tolower(substr($i,2))}} 1' OFS="-"
Although the party is mostly over, please let me join with a perl solution:
perl -pe 's/(^|-)([^-]+)/$1 . ucfirst lc $2/ge' <<<"JASON-CONRAD-983636"
It may be cunning to use the ucfirst function :)
I'm trying to convert a string that has either - (hyphen) or _ (underscore) to Capital_Case string.
#!/usr/bin/env sh
function cap_case() {
[ $# -eq 1 ] || return 1;
_str=$1;
_capitalize=${_str//[-_]/_} | sed -E 's/(^|_)([a-zA-Z])/\u\2/g'
echo "Capitalize:"
echo $_capitalize
return 0
}
read string
echo $(cap_case $string)
But I don't get anything out.
First I am replacing any occurrence of - and _ with _ ${_str//[-_]/_}, and then I pipe that string to sed which finds the first letter, or _ as the first group, and then the letter after the first group in the second group, and I want to uppercase the found letter with \u\2. I tried with \U\2 but that didn't work as well.
I want the string some_string to become
Some_String
And string some-string to become
Some_String
I'm on a mac, using zsh if that is helpful.
EDIT: More generic solution here to make each field's first letter Capital.
echo "some_string_other" | awk -F"_" '{for(i=1;i<=NF;i++){$i=toupper(substr($i,1,1)) substr($i,2)}} 1' OFS="_"
Following awk may help you.
echo "some_string" | awk -F"_" '{$1=toupper(substr($1,1,1)) substr($1,2);$2=toupper(substr($2,1,1)) substr($2,2)} 1' OFS="_"
Output will be as follows.
echo "some_string" | awk -F"_" '{$1=toupper(substr($1,1,1)) substr($1,2);$2=toupper(substr($2,1,1)) substr($2,2)} 1' OFS="_"
Some_String
This being zsh, you don't need sed (or even a function, really):
$ s=some-string-bar
$ print ${(C)s:gs/-/_}
Some_String_Bar
The (C) flag capitalizes words (where "words" are defined as sequences of alphanumeric characters separated by other characters); :gs/-/_ replaces hyphens with underscores.
If you really want a function, it's cap_case () { print ${(C)1:gs/-/_} }.
pure bash:
#!/bin/bash
camel_case(){
local d display string
declare -a strings # = scope local
[ "$2" ] && d="$2" || d=" " # optional output delimiter
ifs_ini="$IFS"
IFS+='_-' # we keep initial IFS
strings=( "$1" ) # array
for string in ${strings[#]} ; do
display+="${string^}$d"
done
echo "${display%$d}"
IFS="$ifs_ini"
}
camel_case "some-string_here" "_"
camel_case "some-string_here some strings here" "+"
camel_case "some-string_here some strings here"
echo "$BASH_VERSION"
exit
output:
Some_String_Here
Some+String+Here+Some+Strings+Here
Some String Here Some Strings Here
4.4.18(1) release
You can try this gnu sed
echo 'some_other-string' | sed -E 's/(^.)/\u&/;s/[_-](.)/_\u\1/g'
Explains :
s/(^.)/\u&/
(^.) match the first char and \u& put the match in capital letter.
s/[_-](.)/_\u\1/g
[_-](.) capture a char preceded by _ or - and replace it by _ and the matched char in capital letter.
The g at the end tell sed to make the replacement for each char which meet the criteria
You didn't assign to _capitalize - you set a _capitalize environment variable for the empty command that you piped into sed.
You probably meant
_capitalize=$(<<<"${_str//[-_]/_}" sed -E 's/(^|_)([a-zA-Z])/\1\u\2/g')
Note also that ${//} isn't standard shell, so you really ought to specify an interpreter other than sh.
A simpler approach would be simply:
#!/bin/sh
cap_case() {
printf "Capitalize: "
echo "$*" | sed -e 'y/-/_/' -e 's/\(^\|_\)[[:alpha:]]/\U&/g'
}
echo $(cap_case "snake_case")
Note that the \u / \U replacement is a GNU extension to sed - if you're using a non-GNU implementation, check whether it supports this feature.
I have a CSV file which contains strings like this:
ID1;banana
| apple
| oranges
and I want that every time there is a pipe at the beginning of the line, the string will be appended to the previous line, the output should be like this:
ID1;banana | apple | oranges
how can remove the newlines that precede a line begining with a pipe |?
In a hackish one liner, removing returns before pipes:
perl -ne '$s = do {local $/; <>}; $s =~ s/\n\|/ |/g; print $s' file.csv
Instead of trying to backspace/erase what's already been printed, you could instead only print the carriage return when a | isn't the first char:
perl -n -e 'chomp; /^\s*\|/? print " $_": print "\n$_" ' yourfile.txt
Is the string you show the value of a single CSV field? If so then you should be using Text::CSV to divide each line into fields (as its getline method is the simplest way to cope with data that contains embedded newlines) and you can use the substitution s/\n(?=\|)/ /g to change a newline into a space if it precedes a pipe character.
Here's an example
use strict;
use warnings;
use Text::CSV;
my $csv = Text::CSV->new({ binary => 1, eol => $/ });
while (my $row = $csv->getline(*DATA)) {
s/\n(?=\|)/ /g for #$row;
print "$_: $row->[$_]\n" for 0 .. $#$row;
print "\n";
}
__DATA__
"ID1;banana
| apple
| oranges",f2,f3
g1,g2,g3
output
0: ID1;banana | apple | oranges
1: f2
2: f3
0: g1
1: g2
2: g3
If your circumstance is different from that then you need to explain.
What I am trying to achieve is pass the Base64 encoded value captured in the sed regex to the base64 and have it decoded.
But the problem is, even though it seems like the correct value is being passed to the function using backreference, base64 complains that the input is invalid.
Following is my script -
#!/bin/bash
decodeBaseSixtyFour() {
echo "$1 is decoded to `echo $1 | base64 -d`"
}
echo Passing direct value ...
echo SGVsbG8gQmFzZTY0Cg== | sed -r "s/(.+)$/$(decodeBaseSixtyFour SGVsbG8gQmFzZTY0Cg==)/"
echo Passing captured value ...
echo SGVsbG8gQmFzZTY0Cg== | sed -r "s/(.+)$/$(decodeBaseSixtyFour \\1)/"
And when ran it produces the following output -
Passing direct value ...
SGVsbG8gQmFzZTY0Cg== is decoded to Hello Base64
Passing captured value ...
base64: invalid input
SGVsbG8gQmFzZTY0Cg== is decoded to
I think the output explains what I mean.
Is it possible to do what I am trying to do? If not, why?
Perl s/// can do what you want, but I don't think what you're asking for is what you need.
$ echo SGVsbG8gQmFzZTY0Cg== | perl -MMIME::Base64 -pe 's/(.+)/decode_base64($1)/e'
Hello Base64
What's actually happening:
echo SGVsbG8gQmFzZTY0Cg== | sed -r "s/(.+)$/$(decodeBaseSixtyFour \\1)/"
Before sed starts reading input, the shell notices the process substitution in the double quoted string
the decodeBaseSixtyFour function is called with the string "\\1"
base64 chokes on the input \1 and emits the error message
the function returns the string "\1 is decoded to "
now the sed script is 's/(.+)$/\1 is decoded to /' which is how you get the last line.
As I commented sed cannot do an equivalent of replace_callback which is esentially what you're trying to do.
Following awk does close to what you're trying to do:
s="My string is SGVsbG8gQmFzZTY0Cg== something"
awk '{for(i=1; i<=NF; i++) if ($i~/==$/) "base64 -D<<<"$i|getline $i}1'<<<"$s"
My string is Hello Base64 something