I'd like to insert a random number after specific text using a shell script. I can generate a random number (32 char):
cat /dev/urandom | tr -dc 'a-z0-9' | fold -w 32 | head -n 1
How do I insert this random number (32_char_random_number) after text (e.g. "alphabet" in a file?:
sed '/\balphabet \b/& 32_char_random_number/' file
Assumed your rand number is assigned to $char_random_number.
char_random_number=$(cat /dev/urandom | tr -dc 'a-z0-9' | fold -w 32 | head -n 1)
And you may use double quote to access the variable,
sed "s/\(alphabet\)/\1$char_random_number/g" file
Robust awk solution:
Sample input.txt contents:
Lorem ipsum dolor alphabet sit amet, consectetuer adipiscing elit alphabet. Aenean commodo ligula eget dolor.
Aenean massa. Cum sociis natoque penatibus et alphabet magnis dis parturient montes, nascetur ridiculus mus.
Donec quam felis, alphabet ultricies nec, pellentesque eu, alphabet pretium quis, sem. Nulla consequat massa quis enim.
Donec pede justo, fringilla no alphabet vel, aliquet nec, alphabet vulputate eget, arcu. In enim justo, rhoncus ut, imperdiet a, venenatis vitae, justo.
Nullam dictum felis eu pede alphabet mollis pretium.
Integer tincidunt, alphabet. Cras dapibus.
The job:
rnum=$(< /dev/urandom tr -dc [[:alnum:]] | head -c 32)
awk -v rnum="$rnum" '{ gsub(/\<alphabet\>/,"& "rnum) }1'
The output:
Lorem ipsum dolor alphabet g9vgamG[slhPKJgOxqphM[KFOdL1qDo5 sit amet, consectetuer adipiscing elit alphabet g9vgamG[slhPKJgOxqphM[KFOdL1qDo5. Aenean commodo ligula eget dolor.
Aenean massa. Cum sociis natoque penatibus et alphabet g9vgamG[slhPKJgOxqphM[KFOdL1qDo5 magnis dis parturient montes, nascetur ridiculus mus.
Donec quam felis, alphabet g9vgamG[slhPKJgOxqphM[KFOdL1qDo5 ultricies nec, pellentesque eu, alphabet g9vgamG[slhPKJgOxqphM[KFOdL1qDo5 pretium quis, sem. Nulla consequat massa quis enim.
Donec pede justo, fringilla no alphabet g9vgamG[slhPKJgOxqphM[KFOdL1qDo5 vel, aliquet nec, alphabet g9vgamG[slhPKJgOxqphM[KFOdL1qDo5 vulputate eget, arcu. In enim justo, rhoncus ut, imperdiet a, venenatis vitae, justo.
Nullam dictum felis eu pede alphabet g9vgamG[slhPKJgOxqphM[KFOdL1qDo5 mollis pretium.
Integer tincidunt, alphabet g9vgamG[slhPKJgOxqphM[KFOdL1qDo5. Cras dapibus.
Related
I have a .txt file where the word 'picture:' is found multiple times in the file. How can I extract all words after the 'pictures:' word and save in a text file
I tried the follow code,but doesn't work:
cat users_sl.txt |awk -F: '/^login:"/{print $2}' cookies.txt
user_sl.txt:
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Quis picture lobortis scelerisque fermentum dui faucibus in ornare quam. Est ullamcorper eget nulla facilisi etiam dignissim diam quis. Quis viverra nibh cras pulvinar mattis nunc sed. Turpis massa sed elementum picture tempus egestas. Condimentum vitae sapien pellentesque habitant. Et molestie ac feugiat sed lectus vestibulum mattis ullamcorper. Tincidunt lobortis feugiat vivamus at augue eget arcu picture dictum varius. Donec massa sapien faucibus et molestie ac feugiat sed. Tincidunt eget nullam non nisi est. Ornare arcu dui vivamus arcu. Mattis enim ut tellus elementum sagittis vitae et leo duis
picturelist.txt:
lobortis
dictum
tempus
Well, I'm assuming you actually just have picture instead of **picture:**, and that you may need to deal with line breaks, so...
$ cat sl.txt
Lorem ipsum dolor sit amet, consectetur adipiscing elit,
sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.
Quis picture lobortis scelerisque fermentum dui faucibus in ornare quam.
Est ullamcorper eget nulla facilisi etiam dignissim diam quis.
Quis viverra nibh cras pulvinar mattis nunc sed.
Turpis massa sed elementum picture tempus egestas.
Condimentum vitae sapien pellentesque habitant.
Et molestie ac feugiat sed lectus vestibulum mattis ullamcorper.
Tincidunt lobortis feugiat vivamus at augue eget arcu picture
dictum varius. Donec massa sapien faucibus et molestie ac feugiat sed.
Tincidunt eget nullam non nisi est.
Ornare arcu dui vivamus arcu.
Mattis enim ut tellus elementum sagittis vitae et leo duis
$ cat sl.txt | tr '\n' ' ' | grep -o 'picture [^ ]*' | cut -d' ' -f2
lobortis
tempus
dictum
Edit: Explanation:
tr '\n' ' ' replaces every (unix) line break with a space -- makes the whole thing one line.
The -o flag tells grep to return only the matched string. The search pattern starts with picture and a space picture , and then everything that follows that is not a space: [^ ]*.
Finally cut using the space character for a delimiter -d ' ' prints the second field: -f 2
Here is a bash solution with a clean shellcheck. Tested with bash version 5.2.2 on a MacOS Ventura system.
#!/usr/bin/env bash
IFS=" " read -r -a WORDS <<< "$(tr '\n' ' ' < users_sl.txt)"
echo processing ${#WORDS[#]} words
for (( i=0; i < ${#WORDS[#]}; i++ ))
do
if [ "${WORDS[$i]}" = "picture" ]; then
echo "${WORDS[i+1]}"
fi
done | tee picturelist.txt
With bash:
#!/bin/bash
arr=( $(<user_sl.txt) )
for ((i=0; i<${#arr[#]}; i++)); do
if [[ ${arr[i]} == picture ]]; then
printf '%s\n' "${arr[i+1]}"
fi
done | tee picturelist.txt
Output
lobortis
tempus
dictum
With perl:
$ perl -nE 'say for /\bpicture\b\s+(\w+)\b/g' user_sl.txt | tee picturelist.txt
lobortis
tempus
dictum
With awk:
$ awk '{
for (i=1; i<=NF; i++) {
if ($i == "picture") print $(i+1)
}
}' user_sl.txt | tee picturelist.txt
or
$ printf '%s\n' $(< users_sl.txt) |
awk '/picture/{p=1;next} {if (p==1) {print;p=0}}' > picturelist.txt
lobortis
tempus
dictum
I have 3 paragraphs of text:
my_value:
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nunc
ullamcorper dolor nibh, ut vestibulum purus vestibulum eu. Nulla in
elit sed ante maximus efficitur eu eget orci. Donec fermentum diam at
ornare auctor. Aenean porttitor, est ac dignissim sagittis, ligula
justo luctus risus, ac auctor turpis magna id ex.
Nunc imperdiet dictum mi efficitur malesuada. Sed in ipsum imperdiet,
aliquam massa vitae, vehicula nisl. Morbi eu odio imperdiet, auctor
nulla in, convallis turpis. Pellentesque habitant morbi tristique
senectus et netus et malesuada fames ac turpis egestas. Proin id
egestas massa. Maecenas finibus erat ac cursus auctor.
Nullam elementum accumsan massa id finibus. Praesent ipsum lectus,
venenatis nec congue vel, dignissim rutrum eros. Suspendisse potenti.
Vivamus et sodales ipsum. Mauris eu erat luctus nibh posuere sodales
in ut diam.
What I want as a result is a string that has 3 newlines - the 2 between the paragraphs, and the final newline. The newlines within each paragraph should be folder into spaces.
If I use my_value: >, every newline is folded, so the newlines between the 3 paragraphs are not preserved. If I use my_value: |, the newlines within each paragraph are preserved, which isn't what I want either.
Is there any way to represent this to get the right output, short of just having super long lines in my yaml file?
If I use my_value: >, every newline is folded
That's not true. Maybe you are using a broken YAML library?
> is the indicator for the "folded block scalar".
Empty lines in a folded block scalar are not folded.
So a plain scalar like you showed and a folded block scalar can be the same:
plain:
a
b
c
folded block: >-
a
b
c
double quoted: "a b\nc"
So you can use a plain scalar or a folded block scalar in your case.
(More about Block Scalars in my article about quoting)
I wrote a batch file where I put in a bunch of text that gets set to some variable and then next line echoes that variable into a text file.
Problem is that it echoes all of it into one long line and it's impossible to read (it forces you to toggle wrap mode, but what if software doesn't support it).
echo "Lorem ipsum dolor sit amet, consectetur adipiscing elit. Phasellus at elit eu felis gravida pretium. Morbi suscipit eu metus quis facilisis. Sed aliquet eget sem ac semper. Fusce tempor magna dui, nec ullamcorper nulla rutrum eget. Suspendisse mattis lorem ut nulla placerat vestibulum. Vivamus sollicitudin nisl in lorem suscipit luctus. Nam a nisi vestibulum, lacinia nulla faucibus, condimentum eros. Donec fringilla neque et massa sagittis sollicitudin. Pellentesque vestibulum, metus maximus lacinia varius, nisi tortor facilisis ante, et porttitor magna urna sed odio. Maecenas ut mi sed ipsum pretium mattis et a urna. Nunc vulputate ornare bibendum. Pellentesque libero lacus, porttitor ut sem vitae, lobortis dictum erat. Sed viverra euismod nisl, nec malesuada turpis pellentesque eget." > text_file.txt
(just look at this example, it's unreadable)
So I need to print it into something like:
"Lorem ipsum dolor sit amet, consectetur adipiscing elit. Phasellus at elit
eu felis gravida pretium. Morbi suscipit eu metus quis facilisis. Sed
aliquet eget sem ac semper. Fusce tempor magna dui, nec ullamcorper nulla
rutrum eget. Suspendisse mattis lorem ut nulla placerat vestibulum. Vivamus
sollicitudin nisl in lorem suscipit luctus. Nam a nisi vestibulum, lacinia
nulla faucibus, condimentum eros. Donec fringilla neque et massa sagittis
sollicitudin. Pellentesque vestibulum, metus maximus lacinia varius, nisi
tortor facilisis ante, et porttitor magna urna sed odio. Maecenas ut mi sed
ipsum pretium mattis et a urna. Nunc vulputate ornare bibendum. Pellentesque
libero lacus, porttitor ut sem vitae, lobortis dictum erat. Sed viverra
euismod nisl, nec malesuada turpis pellentesque eget."
As you can see, the longest line is 76 characters. Is it possible to use it as a breakpoint or something?
But that would also mean it can't chop it in the middle of the word.
UPDATE: I found same question and answer on Linux https://unix.stackexchange.com/questions/60219/is-there-a-command-line-tool-to-insert-line-breaks-into-a-long-string the command is called fold. Any alternative to Windows?
Coincidentally, I wrote something like this about 4 years ago when I was first starting to learn batch. It almost certainly doesn't accept &s or )s, but you don't have any in your sample text and I don't care enough to test for them.
Pass the string in as a single parameter.
#echo off
setlocal enabledelayedexpansion
cls
set unformattedText=Lorem ipsum dolor sit amet, consectetur adipiscing elit. Phasellus at elit eu felis gravida pretium. Morbi suscipit eu metus quis facilisis. Sed aliquet eget sem ac semper. Fusce tempor magna dui, nec ullamcorper nulla rutrum eget. Suspendisse mattis lorem ut nulla placerat vestibulum. Vivamus sollicitudin nisl in lorem suscipit luctus. Nam a nisi vestibulum, lacinia nulla faucibus, condimentum eros. Donec fringilla neque et massa sagittis sollicitudin. Pellentesque vestibulum, metus maximus lacinia varius, nisi tortor facilisis ante, et porttitor magna urna sed odio. Maecenas ut mi sed ipsum pretium mattis et a urna. Nunc vulputate ornare bibendum. Pellentesque libero lacus, porttitor ut sem vitae, lobortis dictum erat. Sed viverra euismod nisl, nec malesuada turpis pellentesque eget.
set width=76
set counter=0
set slashN=^
if not [%1]==[] set unformattedText=%~1
:wordCounter
for /f "tokens=1,* delims= " %%A in ("%unformattedText%") do (
if not "%%A"=="" (
set words[!counter!]=%%A
set /a counter=!counter!+1
set unformattedText=%%B
goto wordCounter
) else (
goto loopBreak
)
)
:loopBreak
echo There are !counter! words in your sentence.
set /a counter=!counter!-1
for /l %%A in (0,1,!counter!) do (
echo !words[%%A]!>tmp.txt
for %%J in (tmp.txt) do (
set /a len[%%A]=%%~zJ
set /a len[%%A]-=2
del tmp.txt
if !len[%%A]! gtr %width% set /a width=!len[%%A]!+1
)
)
:: -- Concatenate the words and spaces --
set newString=
set stringLength=0
for /l %%A in (0,1,!counter!) do (
if %%A lss !counter! (
set /a possLength=!stringLength!+!len[%%A]!+1
if !possLength! leq %width% (
set newString=!newString!!words[%%A]!
set /a stringLength+=!len[%%A]!+1
) else (
set newString=!newString!!slashN!!words[%%A]!
set /a stringLength=!len[%%A]!+1
)
) else (
set /a possLength=!stringLength!+!len[%%A]!
if !possLength! leq %width% (
set newString=!newString!!words[%%A]!
set /a stringLength+=!len[%%A]!
) else (
set newString=!newString!!slashN!!words[%%A]!
set /a stringLength=!len[%%A]!
)
)
)
echo !newString!
This is my version...
EDIT: I modified the code, so the split method is now isolated in a subroutine that may be called by any program.
The subroutine cut the string at a space previous to the maximum allowed length, and repeat this procedure as long as there is a remaining string.
#echo off
setlocal
set longString="Lorem ipsum dolor sit amet, consectetur adipiscing elit. Phasellus at elit eu felis gravida pretium. Morbi suscipit eu metus quis facilisis. Sed aliquet eget sem ac semper. Fusce tempor magna dui, nec ullamcorper nulla rutrum eget. Suspendisse mattis lorem ut nulla placerat vestibulum. Vivamus sollicitudin nisl in lorem suscipit luctus. Nam a nisi vestibulum, lacinia nulla faucibus, condimentum eros. Donec fringilla neque et massa sagittis sollicitudin. Pellentesque vestibulum, metus maximus lacinia varius, nisi tortor facilisis ante, et porttitor magna urna sed odio. Maecenas ut mi sed ipsum pretium mattis et a urna. Nunc vulputate ornare bibendum. Pellentesque libero lacus, porttitor ut sem vitae, lobortis dictum erat. Sed viverra euismod nisl, nec malesuada turpis pellentesque eget."
rem To show lines in screen:
call :splitLines longString 76
rem To store lines in a file:
call :splitLines longString 76 > file.txt
goto :EOF
:splitLines string width
setlocal EnableDelayedExpansion
set "string=!%~1!"
:nextLine
set /A "i=%~2+1"
:prevChar
set /A i-=1
if "!string:~%i%,1!" neq " " goto prevChar
echo !string:~0,%i%!
set /A i+=1
set "string=!string:~%i%!"
if "!string:~%~2!" neq "" goto nextLine
echo !string!
exit /B
I'm working on a TeamCity server, one of my build commands is:
xcodebuild -scheme "<myscheme>" archive
I need to retrieve the .dSYM file
code=$(cat <<-'CODE'
$lines = file("php://stdin");
foreach($lines as $line){
if(preg_match("#Touch (.*dSYM)#",$line,$m))echo "$m[1]\n";
}
CODE
)
dsym=$(xcodebuild -scheme "<myscheme>" archive | php -r "$code")
This will work. However, my issue is, I would like the logs of xcodebuild to be piped to stdout AND php -r "$code"
xcodebuild -scheme "<myscheme>" archive | tee >(php -r "$code" --)
This also works, the build log shows, and if I change php -r "$code" -- to php -r "$code" -- | cat, it logs the .dSYM file location.
But, the following doesn't work:
xcodebuild -scheme "<myscheme>" archive | tee >(dsym=$(php -r "$code" --))
#this one is the closest but is the wrong way around,
#dsym = all the output, the filename is sent to stdout
exec 5>&1
dsym=$(xcodebuild -scheme "<myscheme>" archive | tee >(php -r "$code" >&5))
And I am unable to get my head around how read -u X dsym works or is meant to be working. Does anyone know how I would go about:
Piping all output to stdout
Piping all output to an intermediate program/script (grep)
Storing the above intermediate program/script output into a variable
To test: save a file scheme.out and replace xcodebuild... with cat scheme.out
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Vivamus nibh
nulla, tempor nec dolor ac, eleifend imperdiet diam. Mauris tristique
congue condimentum. Nullam commodo erat fringilla vestibulum tempus.
Aenean mattis varius erat in venenatis. Donec eu tellus urna. Morbi
lacinia vulputate purus, eu egestas tortor varius eget. Curabitur
vitae commodo elit, vitae ullamcorper leo.
Touch some_test_dsym_file.dSYM
Nunc malesuada, nisi at ultricies lobortis, odio diam rhoncus urna,
sed scelerisque enim ipsum eget quam. Nunc ut iaculis sem. Pellentesque
massa odio, sodales nec lacinia nec, rutrum eu neque. Aenean quis neque
magna. Nam quis dictum quam. Proin ut libero tortor. Class aptent taciti
sociosqu ad litora torquent per conubia nostra, per inceptos himenaeos.
Vivamus vehicula fringilla consequat. Curabitur tincidunt est sed magna
congue tristique. Maecenas aliquam nibh eget pellentesque pellentesque.
Quisque gravida cursus neque sed interdum. Proin ornare dapibus
dignissim.
Desired output
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Vivamus nibh
nulla, tempor nec dolor ac, eleifend imperdiet diam. Mauris tristique
congue condimentum. Nullam commodo erat fringilla vestibulum tempus.
Aenean mattis varius erat in venenatis. Donec eu tellus urna. Morbi
lacinia vulputate purus, eu egestas tortor varius eget. Curabitur
vitae commodo elit, vitae ullamcorper leo.
Touch some_test_dsym_file.dSYM
Nunc malesuada, nisi at ultricies lobortis, odio diam rhoncus urna,
sed scelerisque enim ipsum eget quam. Nunc ut iaculis sem. Pellentesque
massa odio, sodales nec lacinia nec, rutrum eu neque. Aenean quis neque
magna. Nam quis dictum quam. Proin ut libero tortor. Class aptent taciti
sociosqu ad litora torquent per conubia nostra, per inceptos himenaeos.
Vivamus vehicula fringilla consequat. Curabitur tincidunt est sed magna
congue tristique. Maecenas aliquam nibh eget pellentesque pellentesque.
Quisque gravida cursus neque sed interdum. Proin ornare dapibus
dignissim.
Desired output of echo $dsym
some_test_dsym_file.dSYM
Your code has a lot of dependencies. I will illustrate what I think that you need without using anything beyond standard unix tools.
This runs a command, seq 4, and sends all of its output to stdout and also sends all of its output to another command, sed 's/3/3-processed/', the output of which is captured in a variable, var:
$ exec 3>&1
$ var=$(seq 4 | tee >(cat >&3) | sed 's/3/3-processed/')
1
2
3
4
To illustrate that we successfully captured the output of the sed command:
$ echo "$var"
1
2
3-processed
4
Explanation: var=$(...) captures the output of file handle 1 (stdout) and assigns it to var. Thus, to make the output also appear on stdout, we need to duplicate stdout to another file handle before $(...) redirects it. Thus, we use exec to duplicate stdout as file handle 3. In this way, tee >(cat >&3) sends the output of the command both the original stdout (now called 3) and to file handle 1 which is passed on the the next stage in the pipeline.
So, using your toolchain, try:
exec 5>&1
dsym=$(xcodebuild -scheme "<myscheme>" archive | tee >(cat >&5) | php -r "$code")
I need to pass a string against an external file that contains a list of substitutions to perform at every occurrence.
The substitution file will look like this (I'm open to suggestions on the structure, it can be a csv, a yaml, etc...)
"ipsum" "foobar"
"elit" ""
"sit amet" "2312"
My ruby code should be implemented like this:
mystring = "Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aliquam quis elit augue. Nulla tempus magna nec ligula dapibus malesuada. Fusce at orci augue, sit amet suscipit sem. Suspendisse potenti."
newstring = mystring.somemagichappenshere
And the newstring value should be "Lorem foobar dolor 2312, consectetur adipiscing . Aliquam quis augue. Nulla tempus magna nec ligula dapibus malesuada. Fusce at orci augue, 2312 suscipit sem. Suspendisse potenti."
How should I implement that?
Using a csv:
require 'csv'
str = "Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aliquam quis elit augue. Nulla tempus magna nec ligula dapibus malesuada. Fusce at orci augue, sit amet suscipit sem. Suspendisse potenti."
replacements = "ipsum,foobar
elit,
sit amet,2312"
#construct a hash from the csv:
transform_table = Hash[CSV.parse(replacements)]
#Take the keys from the hash and use them for a regular expression:
re = Regexp.union(transform_table.keys)
#Do all substituions in one go:
p str.gsub(re, transform_table)
It's quite simple
Read the file
Iterate each line in the file and for each entry use mystring.gsub!(find, replace) to replace the value with the substitution