I'm a beginner in bash script and cannot solve the following problem: I have a file where each line is a xml file. I would like to divide each line into several lines.
For instance, I would like to put the following line:
<LumiBlockCollection><Run>201556</Run><LBRange Start="1020" End="1030"/></LumiBlockCollection>
into the format:
<LumiBlockCollection>
<Run>201556</Run>
<LBRange Start="1020" End="1030"/>
</LumiBlockCollection>
Does anyone know how to solve this problem?
In general, for a robust solution that works with varying input data, you should use an XML parser for this task:
A solution based on xmllint - xmllint is a standard utility on OS X and some Linux distros (e.g., Fedora):
echo '<LumiBlockCollection><Run>201556</Run><LBRange Start="1020" End="1030"/></LumiBlockCollection>' \
| XMLLINT_INDENT= xmllint --format - | tail -n +2
If your Linux distro does not come with xmllint, chances are that it can be installed with your platform's package manager; e.g., on Debian-based distros such as Ubuntu:
sudo apt-get install libxml2-utils
Another solution, based on third-party utility xmlstarlet:
echo '<LumiBlockCollection><Run>201556</Run><LBRange Start="1020" End="1030"/></LumiBlockCollection>' \
| xmlstarlet fo --omit-decl --noindent
Obtaining xmlstarlet:
OSX: Install via Homebrew with brew install xmlstarlet
Linux: chances are that it can be installed with your platform's package manager; e.g., on Debian-based distros such as Ubuntu: sudo apt-get install xmlstarlet
Here's a simple solution using sed. Note that if you have CDATA sections, this will place them on their own line:
$ xml='<LumiBlockCollection><Run>201556</Run><LBRange Start="1020" End="1030"/></LumiBlockCollection>'
$ echo $xml | sed 's/></>\n</g'
<LumiBlockCollection>
<Run>201556</Run>
<LBRange Start="1020" End="1030"/>
</LumiBlockCollection>
$
Related
I'm trying to create a code which will fetch the latest version of anaconda and install it.
Currently we can do this to install the latest version:
mkdir tmp
cd tmp
wget https://repo.anaconda.com/archive/Anaconda3-2020.11-Linux-x86_64.sh
bash Anaconda3-2020.11-Linux-x86_64.sh
I want the script to be more generalized such that the code would automatically find the latest version of anaconda, download the shell script file and install it.
You can use this to get the latest version:
wget https://repo.anaconda.com/archive/ -q -O- |\
grep 'Anaconda3'| \
sed -n 's|.*>Anaconda3-\([0-9]\{4\}\.[0-9]\{2\}\)-.*|\1|p'
uniq |\
sort -r |\
head -1
This solution works only for those versions that use the year format (e.g. 2020-07), but since the latest version will presumably be of that format that should be fine.
Some explanation:
wget to fetch the contents of the archive page, which gives us the HTML content containing all the download URLs. -q quiets the output, -O- prints to stdout. Alternatively, you can use curl -s to the same effect.
grep 'Anaconda3' gives us the lines containing Anaconda, which contain the download links.
Use sed to select the version strings from the download links, e.g. 2020-11. That gives you a list of all versions (of the format YYYY-MM).
Sort that lists and select the first entry, which is the latest version.
Use the version in the rest of your script and you are done. A complete solution would be:
version=$(wget https://repo.anaconda.com/archive/ -q -O- |\
grep 'Anaconda3'|\
sed -n 's|.*>Anaconda3-\([0-9]\{4\}\.[0-9]\{2\}\)-.*|\1|p' |\
uniq |\
sort -r |\
head -1)
wget "https://repo.anaconda.com/archive/Anaconda3-$version-Linux-x86_64.sh"
I'm sure fetching the latest version could be made more efficient, but this should be sufficient for your use case.
I need to install GNU grep on my Mac but I'm finding some difficulties.
I tried doing this:
brew install grep --with-default-names
But this is no longer an option since Homebrew removed --with-default-names.
Can anyone provide a solution for this?
Yes, --with-default-names was removed.
But some formulas, like grep, provided a workaround for this:
$ brew info grep
...
==> Caveats
All commands have been installed with the prefix "g".
If you need to use these commands with their normal names, you
can add a "gnubin" directory to your PATH from your bashrc like:
PATH="/usr/local/opt/grep/libexec/gnubin:$PATH"
...
First, to install, just do install without --with-default-names.
$ brew install grep
...
==> Summary
🍺 /usr/local/Cellar/grep/3.3: 21 files, 880.7KB
You should also see that same "Caveats" info I mentioned at the start. Now, by default, the Homebrew grep would be prefixed by a "g", so it's accessible as ggrep.
$ ggrep -V
ggrep (GNU grep) 3.3
Packaged by Homebrew
Copyright (C) 2018 Free Software Foundation, Inc.
...
This prevents it from shadowing the built-in grep that comes with Mac.
$ grep -V
grep (BSD grep) 2.5.1-FreeBSD
If you really need to use grep and not ggrep, just follow the instructions and put /usr/local/opt/grep/libexec/gnubin at the start of your PATH. You have to do this in your .bashrc or .bash_profile (whichever one you use).
$ echo 'export PATH="/usr/local/opt/grep/libexec/gnubin:$PATH"' >> ~/.bash_profile
$ source ~/.bash_profile
$ grep -V
grep: warning: GREP_OPTIONS is deprecated; please use an alias or script
grep (GNU grep) 3.3
Packaged by Homebrew
...
I have been trying to run the usr/bin/time command in my terminal (Bash) with the verbose flag --verbose or -v but have repeatedly been getting this error:
/usr/bin/time: illegal option -- v
usage: time [-lp] command.
The command I have been running looks like basically like this:
/usr/bin/time -v python practice.py
Any ideas how to get this to work properly on a Mac? (I have OS X Yosemite)?
If you have homebrew, you can get GNU time by installing the gnu-time package:
brew install gnu-time
After that, it’s available as the gtime command:
$ gtime
Usage: gtime [-apvV] [-f format] [-o file] [--append] [--verbose]
[--portability] [--format=format] [--output=file] [--version]
[--help] command [arg...]
The case is similar for a lot of other homebrew-packaged GNU utilities for OSX; e.g., you can get the GNU df command with gdf, du with gdu, readlink with greadlink, etc.
The homebrew package that has most of those is coreutils, which installs about a hundred different GNU-flavored commands. Other useful packages: findutils, gnu-sed, gnu-tar.
If you don’t have homebrew installed yet, you can get it with just a single command:
Command to download and install homebrew
ruby -e "$(curl -fsSL\
https://raw.githubusercontent.com/Homebrew/install/master/install)"
I think looking at the man page the verbose flag is GNU only. Unfortunately, OSX implementation simply differs.
I don't want to just try it and mess something up.
$ dlpackages=$(ls -l anaconda3/bin | awk '{print $9}')
$ for package in $dlpackages; do sudo apt-get install $package; done
or as root: $ for package in $dlpackages; do apt-get install $package; done
Add a safety check for each package, to see if it can be located.
dlpackages=$(ls -l anaconda3/bin | awk '{print $9}')
for package in $dlpackages; do
[[ $(apt-cache search $package) ]] && sudo apt-get install $package
done
Now for every string, the install will only be executed if the package can be found.
Alternatively use the -s option of install as Eric Renouf suggested.
In general things in bin aren't the same as package names. conda list may be closer, but you'll ultimately probably have to figure out the translation of package names manually.
I'm writing some small bash scripts for copiyng certain files/directories in GNU/Linux and Solaris. Everything is OK in Linux, but cp command hasn't the same options in Linux and Solaris.
Copy command is something like this:
cp -ruv $source $dest
Unfortunately I don't know how to achieve copy verbose and copy update in Solaris. Any idea?
Thanks
Unfortunately, cp under Solaris doesn't have that option. man solaris should reveal that.
Are you comfortable making your script depend on rsync?
Or, if possible, you can install the coreutils package and use GNU's cp.
I ran into a similar issue myself and found that gcp takes care of it too. I've made installing coreutils part of my standard system setup.
I run these on a new Solaris install:
pkgadd -d http://get.opencsw.org/now
pkgutil -U
pkgutil -i -y coreutils
pkgutil -a vim
pkgutil -i -y vim
pkgutil -i -y findutils
Remember to add the path - and the documentation path - to your profile, and possibly to the system profile at /etc/profile:
# Set the program path
PATH=$PATH:/usr/sfw/bin:/usr/sfw/sbin:/usr/openwin/bin:/opt/csw/bin:/usr/ccs/bin:/usr/local/bin:/usr/local
export PATH
# Set the documentation path
MANPATH="$MANPATH:/usr/share/man:/opt/sfw/man:/opt/csw/man"
export MANPATH
It sounds like you might be new to Solaris - as I am relatively new. I also do these, which shouldn't affect anything.
I set VIM as the default editor instead of VI - it's compatible, but has more features, including ANSI color, and some terminal emulators will pass your mouse clicks and scrolling through for even more flexibility:
# Set the default editor
EDITOR=vim
export EDITOR
Then if you are still using the default prompt that doesn't say anything, you might want to add some information - this version requires a Bash shell:
# Set the command prompt, which includes the username, host name, and the current path.
PS1='\u#\h:\w>'
export PS1
To recreate verbose mode, you can tee the output to the controlling terminal (/dev/tty) while the stdoout output of tee itself is passed to cp via xargs.
find /some/source/directory -type f | \
tee /dev/tty | xargs -I {} cp {} /copy/to/this-directory/
Replace the find with whatever you like, so long as it passes the paths to the files to be copied through the pipe to tee.
Tested on a standard Solaris 10 system without extra GNU utils.