How start ejabberdctl from bash script properly? - bash

I need to register many thousands of users in ejabberd from csv file. For this, I wrote a simple script.
#!/bin/sh
OLDIFS=$IFS
IFS=','
[ ! -f $INPUT ] && { echo "$INPUT file not found"; exit 99; }
while read username domain pass p1 p2 p3 p4
do
echo "ejabberdctl register $username $domain $pass"
ejabberdctl register $username $domain $pass
done < users.csv
IFS=$OLDIFS
But in the end, the answer is: Error: cannot_register
If I just run the line copied from the output, everything is ok. The user is created normally.

This is just a little trick for later: once you get your loop working, if you consider ejabberdctl is too slow, you can try using the ReST API. That should be a lot faster when doing many requests.
Configure temporarily something like this (remember to remove this when you finished):
listen:
-
port: 5280
module: ejabberd_http
tls: false
request_handlers:
/api: mod_http_api
api_permissions:
"console commands":
from:
- ejabberd_ctl
- mod_http_api
who: all
what: "*"
modules:
mod_http_api: {}
Then execute this in a shell to register an account:
curl 'localhost:5280/api/register?user=user2&host=localhost&password=somepass123'

Related

How do I loop through elements in two bash arrays at the same time

I have a bash script that should execute mongodump on multiple databases within a single mongo instance.
The script should dynamically inject the Mongo-URIs and backup filenames as below :
mongodump --uri=$endpoint --authenticationDatabase admin --gzip --archive=/tmp/$filename.gz
When the script runs its facing an error - it appears its not injecting the variables correctly :
dump.sh
Starting-BACKUP
dump.sh
platforms-2022-10-25-19:05:20
sports-2022-10-25-19:05:20
mongodb://root:mypassword#mongodb-prom/platforms
mongodb://root:mypassword#mongodb-prom/sports
2022-10-25T19:05:20.392+0000 Failed: error creating intents to dump: error getting collections for database `platformsmongodb://root:mypassword#mongodb-prom/sports`: (InvalidNamespace) Invalid database name: 'platformsmongodb://root:mypassword#mongodb-prom/sports'
2022-10-25T19:05:50.409+0000 Failed: can't create session: could not connect to server: server selection error: server selection timeout, current topology: { Type: Single, Servers: [{ Addr: localhost:27017, Type: Unknown, Last error: connection() error occurred during connection handshake: dial tcp [::1]:27017: connect: connection refused }, ] }
Database backup was successful
Both the Mongo-URIs and backup filenames are being populated from arrays as below :
#declare -a BACKUP_NAMES=()
#declare -a BACKUP_ENDPOINTS=()
#declare –a DATABASES=()
#declare –a RAW_DBNAMES=()
#declare –a DATABASES=()
DATABASES+=("platforms")
DATABASES+=("sports")
BACKUP_RETURN_CODE=${?}
MONGODB_URI="mongodb://root:mypassword#mongodb-prom/"
NOW="$(date +"%F")-$(date +"%T")"
#array for raw db names
for DATABASE in "${DATABASES[#]}";
do
RAW_DBNAMES+=("$DATABASE")
done
#array for constructing backup filenames
for DATABASE in "${DATABASES[#]}";
do
#echo $DATABASE
BACKUP_NAMES+=("$DATABASE")
done
#construct backup filenames
cnt=${#BACKUP_NAMES[#]}
for ((i=0;i<cnt;i++)); do
BACKUP_NAMES[i]="${BACKUP_NAMES[i]}-$NOW"
echo "${BACKUP_NAMES[i]}"
done
#construct db endpoints
for ((i=0;i<cnt;i++)); do
uri="$MONGODB_URI${RAW_DBNAMES[i]}"
echo "$uri"
BACKUP_ENDPOINTS+=$uri
done
#pass BACKUP_ENDPOINTS & BACKUP_FILENAMES to mongodump
for ((i=0; i<${#BACKUP_NAMES[#]}; i++));
do
mongodump --uri="${BACKUP_ENDPOINTS[$i]}" --authenticationDatabase admin --gzip --archive=/tmp/"${BACKUP_NAMES[$i]}".gz
done
#If the return code is non-zero then the backup was not successful
if [[ 0 != ${BACKUP_RETURN_CODE} ]]
then
echo "Database backup has failed"
exit ${BACKUP_RETURN_CODE}
else
echo "Database backup was successful"
fi
exit 0
As a check I used these two sample arrays and I have verified that I can indeed loop through two arrays at the same time although this particular example is concatenating the elements (Is this how it works in general?):
array1=( 1 2 3 )
array2=("toronto""new york")
for ((i=0; i<${#array1[#]}; i++));
do echo "${array1[$i]}${array2[$i]}";
done
but I am not sure why in my case the strings are concatenating so wrongly.
What am I missing ?
You don't need multiple arrays create just one and rearrange it like this:
arr=(
# id item product
1 apple juice
2 banana smuzi
3 'raw potato' chips
# ...
)
N=${#arr[*]} # number of items in array
C=3 # number of 'columns'
Then loop over it like so:
for ((i=0; i<$N; i+=$C)); {
read id item product <<< "${arr[#]:$i:$C}"
echo "id: $id"
echo "item: $item"
echo "product: $product"
echo '-----------------'
}
Result:
id: 1
item: apple
product: juice
-----------------
id: 2
item: banana
product: smuzi
-----------------
id: 3
item: raw
product: potato chips
-----------------
p.s. check out this backup script for example it's for psql but probably could be useful.
Yes, it turned i was instantiating the BACKUP_ENDPOINTS array the wrong way.
This is the correct way :
#construct db endpoints
for ((i=0;i<cnt;i++)); do
# uri="$MONGODB_URI${RAW_DBNAMES[i]}"
# echo "$uri"
# BACKUP_ENDPOINTS+=$uri THIS IS WRONG !!!
BACKUP_ENDPOINTS[i]="$MONGODB_URI${RAW_DBNAMES[i]}"
done
for ENDPOINT in "${BACKUP_ENDPOINTS[#]}"; do
echo "$ENDPOINT"
done
#pass BACKUP_ENDPOINTS & BACKUP_FILENAMES to mongodump
for ((i=0; i<${#BACKUP_NAMES[#]}; i++));
do
mongodump --uri="${BACKUP_ENDPOINTS[$i]}" --authenticationDatabase admin --gzip --archive=/tmp/"${BACKUP_NAMES[$i]}".gz
done

Getting an error for unexpected else in bash

[SOLVED]
I'm pretty new tho bash-/shell-scripting and trying setup a check for ip address on a server which gets about once a week a new ip.
The script will then send the new ip to the users.
My problem is, that I'm getting a syntax-error in the last if-else statement for "unexpected" else and can wrap my head around why.
My first iteration didn't use functions, but instead one multi lined if-else which got me the same error. The functions on their own seem to work just fine.
#!/bin/bash
# script to send the new server ip to the users
# get the recent ip address of the system
new_ip=$(ip route get 8.8.8.8 | awk -F"src " 'NR==1{split($2,a," ");print a[1]}')
file=old_ip.txt
function ip_mail(){
source $file
if [ $new_ip != $old_ip ]
then
# email-address changed for obvious reasons
mail -s "New Server IP" [hidden]#[hidden].com <<< "$new_ip"
echo "old_ip=$new_ip" > old_ip.txt
exit 0
fi
exit 0
}
function set_old(){
touch old_ip.txt
echo "old_ip=$new_ip" > old_ip.txt
exit 0
}
if [ $file ]
then
ip_mail()
else
set_old()
fi

Expect fails but I don't see why

I have a bash script that gets info from Heroku so that I can pull a copy of my database. That script works fine in cygwin. But to run it in cron it halts because the shell that it uses halts as Heroku's authentication through Heroku Toolbelt.
Here is my crontab:
SHELL=/usr/bin/bash
5 8-18 * * 1-5 /cygdrive/c/Users/sam/work/push_db.sh >>/cygdrive/c/Users/sam/work/output.txt
I have read the Googles and the man page within cygwin to come up with this addition:
#!/usr/bin/bash
. /home/sam.walton/.profile
echo $SHELL
curl -H "Accept: application/vnd.heroku+json; version=3" -n https://api.heroku.com/
#. $HOME/.bash_profile
echo `heroku.bat pgbackups:capture --expire`
#spawn heroku.bat pgbackups:capture --expire
expect {
"Email:" { send -- "$($HEROKU_LOGIN)\r"}
"Password (typing will be hidden):" { send -- "$HEROKU_PW\r" }
timeout { echo "timed out during login"; exit 1 }
}
sleep 2
echo "first"
curl -o latest.dump -L "$(heroku.bat pgbackups:url | dos2unix)"
Here's the output from the output.txt
/usr/bin/bash
{
"links":[
{
"rel":"schema",
"href":"https://api.heroku.com/schema"
}
]
}
Enter your Heroku credentials. Email: Password (typing will be hidden): Authentication failed. Enter your Heroku credentials. Email: Password (typing will be hidden): Authentication failed. Enter your Heroku credentials. Email: Password (typing will be hidden): Authentication failed.
As you can see it appears that the output is not getting the result of the send command as it appears it's waiting. I've done so many experiments with the credentials and the expect statements. All stop here. I've seen few examples and attempted to try those out but I'm getting fuzzy eyed which is why I'm posting here. What am I not understanding?
Thanks to comments, I'm reminded to explicitly place my env variables in .bashrc:
[[ -s $USERPROFILE/.pik/.pikrc ]] && source "$USERPROFILE/.pik/.pikrc"
export HEROKU_LOGIN=myEmailHere
export HEROKU_PW=myPWhere
My revised script per #Dinesh's excellent example is below:
. /home/sam.walton/.bashrc echo $SHELL echo $HEROKU_LOGIN curl -H "Accept: application/vnd.heroku+json; version=3" -n https://api.heroku.com/
expect -d -c " spawn heroku.bat pgbackups:capture --expire --app gw-inspector expect {
"Email:" { send -- "myEmailHere\r"; exp_continue}
"Password (typing will be hidden):" { send -- "myPWhere\r" }
timeout { puts "timed out during login"; exit 1 } } " sleep 2 echo "first"
This should work but while the echo of the variable fails, giving me a clue that the variable is not being called, I am testing hardcoding the variables directly to eliminate that as a variable. But as you can see by my output not only is the echo yielding nothing, there is no clue that any diagnostics are being passed which makes me wonder if the script is even being called to run from expect, as well as the result of the spawn command. To restate, the heroku.bat command works outside the expect closure but the results are above. The result of the command directly above is:
/usr/bin/bash
{
"links":[
{
"rel":"schema",
"href":"https://api.heroku.com/schema"
}
]
}
What am I doing wrong that will show me diagnostic notes?
If you are going to use the expect code inside your bash script, instead of calling it separately, then you should have use the -c flag option.
From your code, I assume that you have the environmental variables HEROKU_LOGIN and HEROKU_PW declared in the bashrc file.
#!/usr/bin/bash
#Your code here
expect -c "
spawn <your-executable-process-here>
expect {
# HEROKU_LOGIN & HEROKU_PW will be replaced with variable values.
"Email:" { send -- "$HEROKU_LOGIN\r";exp_continue}
"Password (typing will be hidden):" { send "$HEROKU_PW\r" }
timeout { puts"timed out during login"; exit 1 }
}
"
#Your further bash code here
You should not use echo command inside expect code. Use puts instead. The option of spawning the process inside expect code will be more robust than spawning it outside.
Notice the use of double quotes with the expect -c flag. If you use single quotes, then bash script won't do any form of substitution. So, if you need bash variable substitution, you should use double quotes for the expect with -c flag.
To know about usage of -c flag, have a look at here
If you still have any issue, you can debug by appending -d with the following way.
expect -d -c "
our code here
"

SQUID3 - using multiple auth_param like basic_ncsa_auth & basic_ldap_auth

i tried to setup squid3 with multiple auth_param. Basically, the first choice should be basic_ldap_auth and if this doesnt return OK it should try basic_ncsa_auth with the same values. As far as i know squid doesnt support it however there is the possibility to use "external" ACL
auth_param basic program /usr/lib/squid3/basic_fake_auth
external_acl_type MultAuth %SRC %LOGIN %{Proxy-Authorization} /etc/squid3/multAuth.pl
acl extAuth external MultAuth
my "multAuth.pl"
use URI::Escape;
use MIME::Base64;
$|=1;
while (<>) {
($ip,$user,$auth) = split();
# Retrieve the password from the authentication header
$auth = uri_unescape($auth);
($type,$authData) = split(/ /, $auth);
$authString = decode_base64($authData);
($username,$password) = split(/:/, $authString);
# do the authentication and pass results back to Squid.
$ldap = `/bin/bash auth/ldap.sh`;
if ($ldap == "OK") {
print "OK";
}
$ncsa = `/bin/bash auth/ncsa.sh`;
if ($ncsa == "OK") {
print "OK";
} else {
print "ERR";
}
}
now i am trying to run with ncsa.sh and ldap.sh the "normal" shell command for these auth methods.
./basic_ldap_auth -R -b "dc=domain,dc=de" -D "CN=Administrator,CN=Users,DC=domain,DC=de" -w "password" -f sAMAccountName=%s -h domain.de
user password
and
./basic_ncsa_auth /etc/squid3/users
user password
Therefor i ran:
auth/ncsa.sh
#!/usr/bin/expect
eval spawn [lrange $argv 0 end]
expect ""
send [lindex $argv 1]
send '\r'
expect {
"OK" {
echo OK
exp_continue
}
"ERR" {
echo ERR
exp_continue
}
interact
with
./ncsa.sh "/usr/lib/squid3/basic_ncsa_auth /etc/squid3/users" "user password"
and i generate the following error:
couldn't execute "/usr/lib/squid3/basic_ncsa_auth /etc/squid3/users": no such file or directory
while executing
"spawn {/usr/lib/squid3/basic_ncsa_auth /etc/squid3/users} {user password}"
("eval" body line 1)
invoked from within
"eval spawn [lrange $argv 0 end]"
(file "./ncsa.sh" line 2)
Besides this error, i am not sure how to pass the variables (username & password) forward and i am also not sure how to answer the shell questions like for example the user & pw input for basic_ldap_auth .
Is there a nice way how to solve that? or any other good plan ?
thanks!
FWIW, the following script helped me transition from passwd based to LDAP based authentication.
Contrary to your requirements, my script acts the other way around: It first checks passwd, then LDAP.
#!/usr/bin/env bash
# multiple Squid basic auth checks
# originally posted here: https://github.com/HackerHarry/mSquidAuth
#
# credits
# https://stackoverflow.com/questions/24147067/verify-user-and-password-against-a-file-created-by-htpasswd/40131483
# https://stackoverflow.com/questions/38710483/how-to-stop-ldapsearch1-from-base64-encoding-userpassword-and-other-attributes
#
# requires ldap-utils, openssl and perl
# tested with Squid 4 using a "auth_param basic program /usr/lib/squid/mSquidAuth.sh" line
# authenticate first against squid password file
# if this fails, try LDAP (Active Directory) and also check group membership
# variables
# sLOGFILE=/var/log/squid/mSquidAuth.log
sPWDFILE="/etc/squid/passwd"
sLDAPHOST="ldaps://dc.domain.local:636"
sBASE="DC=domain,DC=local"
sLDS_OPTIONS="-o ldif-wrap=no -o nettimeout=7 -LLL -P3 -x "
sBINDDN="CN=LDAP-read-user,OU=Users,DC=domain,DC=local"
sBINDPW="read-user-password"
sGROUP="Proxy-Users"
# functions
function _grantAccess {
# echo "access granted - $sUSER" >>$sLOGFILE
echo "OK"
}
function _denyAccess {
# echo "access denied - $sUSER" >>$sLOGFILE
echo "ERR"
}
function _setUserAndPass {
local sAuth="$1"
local sOldIFS=$IFS
IFS=' '
set -- $sAuth
IFS=$sOldIFS
# set it globally
sUSER="$1"
sPASS="$2"
}
# loop
while (true); do
read -r sAUTH
sUSER=""
sPASS=""
sSALT=""
sUSERENTRY=""
sHASHEDPW=""
sUSERDN=""
iDNCOUNT=0
if [ -z "$sAUTH" ]; then
# echo "exiting" >>$sLOGFILE
exit 0
fi
_setUserAndPass "$sAUTH"
sUSERENTRY=$(grep -E "^${sUSER}:" "$sPWDFILE")
if [ -n "$sUSERENTRY" ]; then
sSALT=$(echo "$sUSERENTRY" | cut -d$ -f3)
if [ -n "$sSALT" ]; then
sHASHEDPW=$(openssl passwd -apr1 -salt "$sSALT" "$sPASS")
if [ "$sUSERENTRY" = "${sUSER}:${sHASHEDPW}" ]; then
_grantAccess
continue
fi
fi
fi
# LDAP is next
iDNCOUNT=$(ldapsearch $sLDS_OPTIONS -H "$sLDAPHOST" -D "$sBINDDN" -w "$sBINDPW" -b "$sBASE" "(|(sAMAccountName=${sUSER})(userPrincipalName=${sUSER}))" dn 2>/dev/null | grep -cE 'dn::? ')
if [ $iDNCOUNT != 1 ]; then
# user needs a unique account
_denyAccess
continue
fi
# get user's DN
# we need the extra grep in case we get lines back starting with "# refldap" :/
sUSERDN=$(ldapsearch $sLDS_OPTIONS -H "$sLDAPHOST" -D "$sBINDDN" -w "$sBINDPW" -b "$sBASE" "(|(sAMAccountName=${sUSER})(userPrincipalName=${sUSER}))" dn 2>/dev/null | perl -MMIME::Base64 -n -00 -e 's/\n +//g;s/(?<=:: )(\S+)/decode_base64($1)/eg;print' | grep -E 'dn::? ' | sed -r 's/dn::? //')
# try and bind using that DN to check password validity
# also test if that user is member of a particular group
# backslash in DN needs special treatment
if ldapsearch $sLDS_OPTIONS -H "$sLDAPHOST" -D "$sUSERDN" -w "$sPASS" -b "$sBASE" "name=${sGROUP}" member 2>/dev/null | perl -MMIME::Base64 -n -00 -e 's/\n +//g;s/(?<=:: )(\S+)/decode_base64($1)/eg;print' | grep -q "${sUSERDN/\\/\\\\}"; then
_grantAccess
continue
fi
_denyAccess
done

Code to count number of types of files on client in Unix

Hi I am new to shell scripting.
my requirement is:
There is one server & 3 clients. On each client error logs files are generated which are of 4 types.Say type 1 error , type 2 error like this type 4 error.
I want to write a script which read all the 3 clients from server & provide me number of times 4 different type of error logs are genereted on each client.
In short it should use ssh & grep command combination.
I have written the demo script but it's not providing me the number of times different type of logs occured on clients.
#error[1]='Exception: An application error occurred during an address lookup request, please contact IT'
#error[2]='SocketTimeoutException: Read timed out'
#error[3]='Exception: The search has produced too many matches to be returned'
#error[4]='Exception: No matching address found'
error_1='exception 1'
error_2='exception 2'
function get_list_of_clients()
{
NUM_OF_CLIENTS=$(wc -l ${CLIENT_IP_LIST} | awk -F " " '{ print $1 }' )
echo $NUM_OF_CLIENTS
if [ "${NUM_OF_CLIENTS}" -gt 0 ]
then
for ((row=2; row<=$NUM_OF_CLIENTS; row++))
do
CLIENTS_IP=$(sed -n ${row}p ${CLIENT_IP_LIST}| awk -F " " '{print $3 }')
echo ${CLIENTS_IP}
# get_number_of_errors
# copy_count_errors
echo ${$error_$row}
done
fi
}
function get_number_of_errors()
{
for((row_no=1; row_no<=4; row_no++))
do
{
/usr/bin/expect - <<- EndMark
spawn ssh root#${CLIENTS_IP} "grep $error[$row_no] var/error.log |wc -l" >> /tmp/${CLIENTS_IP}_error${row_no}.txt
match_max 50000
expect {
"*yes/no*" {
send -- "yes\r"
send -- "\r"
exp_continue
}
"*?assword:*" {
send -- "${CLIENT_PASSWORD}\r"
send -- "\r"
}
}
expect eof
EndMark
}
done
}
function copy_count_errors()
{
/usr/bin/expect - <<- EndMark
spawn scp root#${CLIENTS_IP}:/tmp/${CLIENTS_IP}* /tmp/
match_max 50000
expect {
"*yes/no*" {
send -- "yes\r"
send -- "\r"
exp_continue
}
"*?assword:*" {
send -- "${CLIENT_PASSWORD}\r"
send -- "\r"
}
}
expect eof
EndMark
}
get_list_of_clients
================================================================================
please help.
This is not really an answer, an attempt to help you getting your own.
The Problem
If I understand it correctly:
Your script runs on the server
You have three clients, each has log files
Your list of clients is in a file named $CLIENT_IP_LIST, where the IP is the third field and the first line is some sort of header, which you want to exclude.
Suggestions
It seems you need to ssh and scp to the clients. If possible, I suggest setting up SSH Public Key Authentication between your server and clients. This will greatly simplify your scripts.
Use the -C flag for compression with the scp and ssh commands.
You should copy the log files from the clients to the server and do processing on the server.
If you can choose a different scripting language such as Python, Tcl (You used Expect, which is really Tcl). Bash can handle your problem, but you will find other languages work better. This is my opinion, of course.
Get one piece of your puzzle to work before moving on to the next. For example, right now, your get_list_of_clients() function does not yet working.
That being said, here is a rewrite of get_list_of_clients, which I tested and it works within my assumptions about your $CLIENT_IP_LIST file:
function get_list_of_clients() {
let row=1
while read line
do
if (( row > 1 ))
then
set $line # Breaks line into pieces
CLIENT_IP="$3"
echo "$CLIENT_IP"
fi
let row=row+1
done < $CLIENT_IP_LIST
}

Resources