I keep getting an error when trying to configure Datastax Enterprise (my first Cassandra cluster) on Google Cloud Platform, specifically when following the tutorial here:
DataStax Enterprise Deployment Guide for Google Compute Engine - Manual
ssh into new machine called customizer
When I copy and paste the script, or completely retype the script on pico or vi (script found under "Create a customized OS image") and try to run it I get the error ./customizer.sh: line 21: syntax error near unexpected token 'newline' because of this line:
patch --backup /usr/share/google/safe_format_and_mount < #MOUNT_OPTIONS="discard,defaults" > MOUNT_OPTIONS="defaults,discard,noauto,noatime,barrier=0"
SFAM
Any idea what this line should look like to not get this error? When I try to remove the < and > the terminal gets hung up and i have to ctrl+c to get out.
Here is the full script:
#!/bin/bash
#This script can be applied to a running GCE instance
#to prep it for running DSE on SSD based storage, assumed
#to be mounted at /dev/sdb. After this script has been applied,
#a GCE image can be created accordin to the instructions at the
#Image creation guide: https://developers.google.com/compute/docs/images#creatingimage
#Base OS list: https://developers.google.com/compute/docs/operating-systems#backportsimages
apt-get update
apt-get install -y less htop patch libjna-java sysstat iftop binutils pssh pbzip2 zip unzip openssl curl liblzo2-dev ntp git python-pip tree unzip dstat ethtool
#Don't need to disable swap
#Disable Swap
#swapoff -a
#Need to mount SSD
mkdir -p /var/lib/cassandra
#https://developers.google.com/compute/docs/disks#formatting
#/usr/share/google/safe_format_and_mount -m "mkfs.ext4 -F"
patch --backup /usr/share/google/safe_format_and_mount < #MOUNT_OPTIONS="discard,defaults"
> MOUNT_OPTIONS="defaults,discard,noauto,noatime,barrier=0"
SFAM
patch --backup /etc/rc.local < echo deadline > /sys/block/sdb/queue/scheduler
> echo 0 > /sys/block/sdb/queue/rotational
> blockdev --setra 0 /dev/sdb
> /usr/share/google/safe_format_and_mount -m "mkfs.ext4 -F" /dev/sdb /var/lib/cassandra
>
END
cat >> /etc/sysctl.conf <
Thanks for pointing this out and sorry for the inconvenience - this is actually an issue with our documentation. Some of the characters were stripped out when we moved to a new content management system. We are in the process of fixing this as soon as possible.
In the meantime, please use this latest deployment guide for deploying Datastax Enterprise to Google:
https://academy.datastax.com/demos/deployment-guide-google
Replace line 21 and 22 with the following:
From
patch --backup /usr/share/google/safe_format_and_mount < #MOUNT_OPTIONS="discard,defaults"
> MOUNT_OPTIONS="defaults,discard,noauto,noatime,barrier=0"
to this (note that this is still 2 lines):
patch --backup /usr/share/google/safe_format_and_mount
> MOUNT_OPTIONS="defaults,discard,noauto,noatime,barrier=0"
or this (note that this 1 lines would replace 21 and 22):
patch --backup /usr/share/google/safe_format_and_mount < MOUNT_OPTIONS="defaults,discard,noauto,noatime,barrier=0"
The reason for the error is because the end of line 21 is commented out (everything after the # symbol) so it is technically ending in the < character... which is used for command substation.
Related
I am trying to set up GOPATH GOROOT in my AWS EC2 Ubuntu 20.04 user data, but it never worked, every time I connect to the AWS EC2 and view the log in /var/log/cloud-init-output.log it always says
go: not found, but if I key in the echo part it will work.
I am trying to set up multiple EC2 with this basis, so I can't key in every instance myself.
The CloudFormation yaml user data part is below:
UserData:
Fn::Base64: |
#!/bin/bash
wget https://dl.google.com/go/go1.14.4.linux-amd64.tar.gz
tar -C /usr/local -zxvf go1.14.4.linux-amd64.tar.gz
mkdir -p ~/go/{bin,pkg,src}
echo 'export GOPATH=$HOME/go' >> ~/.bashrc
echo 'export GOROOT=/usr/local/go' >> ~/.bashrc
echo 'export PATH=$PATH:$GOPATH/bin:$GOROOT/bin' >> ~/.bashrc
echo 'export GO111MODULE=auto' >> ~/.bashrc
source ~/.bashrc
apt -y update
apt -y install mongodb wget git
systemctl start mongodb
apt -y install git gcc cmake autoconf libtool pkg-config libmnl-dev libyaml-dev
go get -u github.com/sirupsen/logrus
cd ~
git clone --recursive https://github.com/williamlin0504/free5gcWithOCF.git
cd free5gcWithOCF
make
And here is the error inside /var/log/cloud-init-output.log
Error while user data runs
Is there anyone is familiar with this, please I need some help~
In your error message, in the Makefile at line 30 there is a program bin/amf being used
This program appears to be a shell script with a problem in line 1
The nature of the problem is "go: not found"
If you have the bare word "go" in line 1 of the shell script and the path cannot find it then this is what will happen
Probably you need to alter the last line of your userdata shell script to say
PATH=/usr/local/go/bin:$PATH make
I know you have a source command earlier in the script that is supposed to set this up but it doesn't do what you think it does
This is my first post.
I searched a lot and could not find all these pieces in one place. I gathered them from various posts on SO and managed to create a working script. All the credits go to their respective creators.
It's a bash script rather than a question.Hope it helps someone. Please feel free to suggest improvements.
#!/bin/bash
# Author: krishna - and thanks to all the helpful folks on SO.
# date: apr-03-2018
# script
# Notes: unattented script to install percona server. Pieced together from all over SO.
# see versions below next to each item. they are current as on this script creation dates.
# 2. percona server 5.7 or later
# 4. ubuntu 64bit 16.4.3
#
# This script works for me but may not be the best bash script, Please help, to make it better.
#
# you have to run this script as sudo
# if you saved this script as let's say install-percona.sh
# then do this
# chmod +x install-percona.sh
# sudo ./install-percona.sh
#
# do everything in downloads
mkdir -p ~/downloads
cd ~/downloads
# Variables
# Passwords are like under garments.
# 1. Dont leave them lying around
# 2. Change Often
# 3. Do not share it with others
# 4. Dont be chaep. Invest in STRONG AND high quality one's.
# So I hope you know what to do with the next 3 lines.
MYSQL_ROOT_PASSWORD="SecretRootPassword123"
DB_USERNAME="MickeyMouse"
DB_PASSWORD="SecretDBPassword123"
export DEBIAN_FRONTEND=noninteractive
sudo apt-get install -y debconf-utils
# Install software
# We are going to install Percona Server.
#
# By default, percona-server is going to ask for the root password and we automate that with debconf-set-selections
# begin percona stuff
# you need to change the next 2 lines, visit the percona server site to get latest values.
wget https://repo.percona.com/apt/percona-release_0.1-4.$(lsb_release -sc)_all.deb
dpkg -i percona-release_0.1-4.$(lsb_release -sc)_all.deb
#
apt-get update
# also change the version number accordingly. the -5.7 part
# Note. its root-pass and re-root-pass unlike root_password and root_password_again for other flavors of mysql.
#
# !!!!! CAUTION !!!!
# Be sure to check /var/cache/debconf/passwords.dat file for these 2 entries. After installation is completed.
# The value fields were clear in my case but check anyway.
# Y.M.M.V - dont want to leave passwords hanging in the clear.
#
#
echo "percona-server-server-5.7 percona-server-server-5.7/root-pass password ${MYSQL_ROOT_PASSWORD}" | debconf-set-selections
echo "percona-server-server-5.7 percona-server-server-5.7/re-root-pass password ${MYSQL_ROOT_PASSWORD}" | debconf-set-selections
apt install -y percona-server-server-5.7
# Configure MySQL ie Percona
# We are going to create a user also and the 3 CREATE FUNCTION lines are what percona suggests.
# these may happen silently
echo "lets configure percona."
mysql -u root -p${MYSQL_ROOT_PASSWORD} <<EOF
CREATE FUNCTION fnv1a_64 RETURNS INTEGER SONAME 'libfnv1a_udf.so';
CREATE FUNCTION fnv_64 RETURNS INTEGER SONAME 'libfnv_udf.so';
CREATE FUNCTION murmur_hash RETURNS INTEGER SONAME 'libmurmur_udf.so';
CREATE USER '${DB_USERNAME}'#'localhost' IDENTIFIED BY '${DB_PASSWORD}';
GRANT ALL ON *.* TO '${DB_USERNAME}'#'localhost' with grant option;
FLUSH Privileges;
EOF
echo "make sure you run mysql_secure_installation after install"
# run the secure installation script
mysql_secure_installation
echo "done."
This is driving me crazy...
I have Win10 and I have installed the Docker Toolbox with
Docker=1.10.2
Compose=1.6.0
VirtualBox=5.0.14
I have successfully launched the LAMP in Linux [Amazon linux] but when I try to do the same the terminal responds with "ERROR: Container command not found or does not exist"
As I understand, there is something wrong with the way Windows interpreter the CMD syntax.
I have tried
- CMD ["/run.sh"]
- ENTRYPOINT ["/run.sh"]
- CMD /run.sh
- CMD '/run.sh'
- CMD run.sh
- CMD "/run.sh"
but nothing seems to work.
Note: When I run CMD /run.sh the error does not appear but the container exits immediately.
Note2: I have exactly the same problem when trying to setup the LAMP with Docker-Machine on AWS
I have this DockerfileLamp :
FROM ubuntu
# -- Install needed packages --
ENV DEBIAN_FRONTEND noninteractive
# -- Install additional utilities --
RUN apt-get update && \
apt-get install -y supervisor git curl apache2 mcrypt cron wget nano unzip
# -- Install PHP 5.5 --
RUN apt-get -y update && \
apt-get -y install php5 libapache2-mod-php5 mysql-server-5.5 php5-mysql pwgen php-apc php5-mcrypt php5-xdebug php5-gd php5-curl php-pear openssh-server php5-cli php5-apcu php5-intl php5-imagick php5-json
# -- Set localhost to apache conf file --
RUN echo "ServerName localhost" >> /etc/apache2/apache2.conf
# -- Add image configuration and scripts --
ADD ./lamp/start-apache2.sh /start-apache2.sh
ADD ./lamp/start-mysqld.sh /start-mysqld.sh
ADD ./lamp/run.sh /run.sh
RUN chmod 755 /*.sh
ADD ./lamp/my.cnf /etc/mysql/conf.d/my.cnf
ADD ./lamp/supervisord-apache2.conf
/etc/supervisor/conf.d/supervisord-apache2.conf
ADD ./lamp/supervisord-mysqld.conf
/etc/supervisor/conf.d/supervisord-mysqld.conf
# -- Remove pre-installed database --
RUN rm -rf /var/lib/mysql/*
# -- Add MySQL utils --
ADD ./lamp/setup_MySQL.sh /setup_MySQL.sh
RUN chmod 755 /*.sh
# -- config to enable .htaccess --
##ADD apache_default /etc/apache2/sites-available/000-default.conf
RUN a2enmod rewrite
# -- Environmental variables to configure php --
ENV PHP_UPLOAD_MAX_FILESIZE 10M
ENV PHP_POST_MAX_SIZE 10M
# -- Add volumes for MySQL --
##VOLUME ["/etc/mysql", "/var/lib/mysql" ]
# -- Set up SSH server --
RUN mkdir /var/run/sshd
RUN echo 'root:root' |chpasswd
RUN sed -ri 's/^PermitRootLogin\s+.*/PermitRootLogin yes/'
/etc/ssh/sshd_config
RUN sed -ri 's/UsePAM yes/#UsePAM yes/g' /etc/ssh/sshd_config
ADD ./lamp/supervisord-openssh-server.conf
/etc/supervisor/conf.d/supervisord-openssh-server.conf
# -- Install Python & pip --
RUN apt-get update && \
apt-get upgrade -y && \
apt-get install -y python python-pip python-dev && \
pip install --upgrade pip
# -- Install xvfb --
RUN apt-get install -y xvfb
EXPOSE 80 3306 22
CMD /run.sh
and the run.sh :
#!/bin/bash
VOLUME_HOME="/var/lib/mysql"
sed -ri -e "s/^upload_max_filesize.*/upload_max_filesize = ${PHP_UPLOAD_MAX_FILESIZE}/" \
-e "s/^post_max_size.*/post_max_size = ${PHP_POST_MAX_SIZE}/" /etc/php5/apache2/php.ini
if [[ ! -d $VOLUME_HOME/mysql ]]; then
echo "=> An empty or uninitialized MySQL volume is detected in $VOLUME_HOME"
echo "=> Installing MySQL ..."
mysql_install_db > /dev/null 2>&1
echo "=> Done!"
/setup_MySQL.sh
else
echo "=> Using an existing volume of MySQL"
fi
exec supervisord -n
and the docker-compose.yml :
lamp: # apache + mysql/php
build: .
dockerfile: DockerfileLamp
ports:
- "8181:80" # open apache to public
- "3333:3306" # open mysql to public
- "2222:22" # open SSH to public
Docker is process centric, in other words your containers dies when your CMD script dies. At the end of your script run ...
tail -f logfile (where logfile is some logfile you are interested in)
This will
1 - stop your container exiting
2 - allow you to do
docker logs -f containerName
To help u debug
3 - allow you to enter into the container with
docker exec -it bash containerName
Then u can run the command that you think is failing inside the container and try n sort this out
Whilst this doesn't directly answer your question it should give u sufficient weaponry to attack this issue
For another project I tried to get to work on Windows with Docker Machine I ran into the same ambiguous error message of docker-compose Container command not found or does not exist.
Your comment about line endings triggered me to try dos2unix ./*/*.sh within git-bash (multiple scripts, in subfolders), which fixed the issue for me.
My suspicion is that git clone saves the files with DOS line endings, which results in incorrect syntax for the top line !#/bin/bash.
$ docker-compose -v
docker-compose version 1.6.2, build e80fc83
$ docker version
Client:
Version: 1.10.3
API version: 1.22
Go version: go1.5.3
Git commit: 20f81dd
Built: Thu Mar 10 21:49:11 2016
OS/Arch: windows/amd64
Server:
Version: 1.10.3
API version: 1.22
Go version: go1.5.3
Git commit: 20f81dd
Built: Thu Mar 10 21:49:11 2016
OS/Arch: linux/amd64
I solved it by simplifying the file. I commented out all the controls because whatever I tried it would keep throwing Syntax Errors
#!/bin/bash
VOLUME_HOME="/var/lib/mysql"
sed -ri -e "s/^upload_max_filesize.*/upload_max_filesize = ${PHP_UPLOAD_MAX_FILESIZE}/" \
-e "s/^post_max_size.*/post_max_size = ${PHP_POST_MAX_SIZE}/" /etc/php5/apache2/php.ini
#if [[ ! -d $VOLUME_HOME/mysql ]]; then
echo "=> An empty or uninitialized MySQL volume is detected in $VOLUME_HOME"
echo "=> Installing MySQL ..."
mysql_install_db > /dev/null 2>&1
echo "=> Done!"
/setup_MySQL.sh
#else
# echo "=> Using an existing volume of MySQL"
#fi
exec supervisord -n
It works for my case so I am not going to investigate further. Cheers!
EDITED
The above solution was not so complete.
It worked because I was making changes from INSIDE the container.
The permanent solution goes like this :
I migrated the run.sh file to a private Gist . [It does not need to be private but ok]
I think the problem is that when I try to build the Dockerfile from Windows machine [either locally or on a cloud provider] it messes up the syntax , EOF , line breaks and whatnot.
So I broke out of it by ADDing the gist url
ADD http://gist_url/run.sh /run.sh
Note1: You must use the raw file URL otherwise you are going to get the complete HTML.
Note2: The private gist is not protected.You don't need authentication to fetch the URL.
I am installing hadoop 2.7.1 .
It was written there:
"Disable IPv6 with the command
sudo nano /etc/sysctl.conf
and copy the following lines at the end of the file:
#disable ipv6
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv6.conf.lo.disable_ipv6 = 1
Use the command cat /proc/sys/net/ipv6/conf/all/disable_ipv6 to check to make sure IPv6 is off:
it should say 1. If it says 0, you missed something."
After using cat /proc/sys/net/ipv6/conf/all/disable_ipv6 command I am getting "0". that means it don't get disable.
What i am doing wrong.
sudo apt-get install gksu
gksu gedit /etc/sysctl.conf
If you’re set on using terminal then this’ll do it:
echo "net.ipv6.conf.all.disable_ipv6 = 1" | sudo tee -a
/etc/sysctl.conf
echo "net.ipv6.conf.default.disable_ipv6 = 1" | sudo tee -a
/etc/sysctl.conf
echo "net.ipv6.conf.lo.disable_ipv6 = 1" | sudo tee -a
/etc/sysctl.conf
at this point you can try to run
sudo service networking restart
However, Atheros and Ubuntu seem to have a strange sort of ‘not working’ thing going on and so that command doesn’t work with my wireless driver. If the restart fails, just restart the computer and you should be good.
(if you’re terminal only : sudo shutdown -r now )
Has it worked up to here?
If you’re stout of heart, attempt the following:
su - hduser
ssh localhost
If that’s worked you be greeted with a message along the lines of ‘Are you sure you want to continue connecting?’ The answer you’re looking for at this point is ‘yes’.
If it hasn’t worked at this point run the following command:
cat /proc/sys/net/ipv6/conf/all/disable_ipv6
If the value returned is 0 then you’ve still not got ipv6 disabled – have a re-read of that section and see if you’ve missed anything.
I am trying to set up a local Rails environment, and I am having issues with getting RVM installed.
Here is the error I get:
$ bash < <( curl http://rvm.io/releases/rvm-install-head )
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 185 100 185 0 0 136 0 0:00:01 0:00:01 --:--:-- 0
bash: line 1: html: No such file or directory
bash: line 2: syntax error near unexpected token `<'
'ash: line 2: `<head><title>301 Moved Permanently</title></head>
The contents of my /usr/local folder are as follows:
bin man
etc mysql
hermes mysql-5.1.56-osx10.5-x86_64
include rvm
info share
lib
I have some working knowledge of what's going on here, but help on what I'm missing or whether I'm approaching this the wrong way would be greatly appreciated.
Use the new RVM installer for either user and/or root installs instead, it defaults to head.
user$ curl -L https://get.rvm.io | bash
Alternatively you can install the stable release like this:
user$ curl -L https://get.rvm.io | bash -s stable
RTFM material is located at https://rvm.io/rvm/install and https://rvm.io/rvm/basics
On my Mac OS X 10.5.8 I had to use
bash < <(curl -Lks https://rvm.io/install/rvm) or it failed silently.
The -s switch by itself absolutely does not work. Fails silently.
It's annoying when people answer with the generic info anyone can find at the RVM site. Please try to actually answer this when you've actually succeeded in installing ON A MAC.
Matt if my fix does not work for you then would you let us know? Thanks.
On mac 10.5 I had to run this command
$ echo insecure >> ~/.curlrc
followed by this
$ bash -s stable < <(curl -Lks --insecure https://raw.github.com/wayneeseguin/rvm/master/binscripts/rvm-installer)
I hope this is helpful to someone out there.
You are missing the -s option to curl.
bash < <(curl -s https://rvm.io/releases/rvm-install-head )