I am writing a Jenkins pipeline. I am trying to capture last part of the git url without the git extension. For instance: https://github.hhhh.com/aaaaaa-dddd/xxxx-yyyy.git. I want only xxxx-yyyy to be returned. Below is my code:
String getProjectName() {
echo "inside getProjectName +++++++"
# projectName = sh(
# script: "git config --get remote.origin.url",
# returnStdout: true
# ).trim()
def projectName= sh returnStdout:true, script: '''
#!/bin/bash
GIT_LOG = $(env -i git config --get remote.origin.url)
echo $GIT_LOG
basename -s .git "$GIT_LOG"; '''
echo "projectName: ${projectName}"
return projectName
}
PS: Please ignore the commented lines of code.
There is basic Bourne shell functionality that achieves that:
# strip everything up to the last /
projectName=${GIT_LOG##*/}
# strip trailing .git
projectName=${projectName%.git}
This leaves just the requested name in projectName.
No space before and after =:
x='https://github.hhhh.com/aaaaaa-dddd/xxxx-yyyy.git'
basename "$x" .git
Output:
xxxx-yyyy
I'm trying to add some text to the end of a few files.
I have made a file, where I have 5 servername. Each servername corresponds to a separate config file. (The path of these config files is not known).
I am using below code to get the file path,
MyCode:
#!/bin/bash
for i in $(cat serverlist-file | while read f; do find . -name "*$f*"; done);
do
echo $i
done
Output:
/data/servers/customer01/server01.cfg
/data/servers/customer01/server02.cfg
/data/servers/customer02/server03.cfg
/data/servers/customer03/server04.cfg
/data/servers/customer03/server05.cfg
I am using below code to get the list of servers,
MyCode:
#!/bin/bash
for j in $(cat serverlist-file);
do
echo $j
done
Output:
server01
server02
server03
server04
server05
Now I want to edit those config files and add text to it.
I am using below code to add the required text:
#!/bin/bash
for i in $(cat serverlist-file | while read f; do find . -name "*$f*"; done);
do
for j in $(cat serverlist-file);
do
sed -i -e "\$a\
this\ is\ a\ config\ file\nfor\ $j" $i
done
done
Expected Output:
/data/servers/customer01/server01.cfg
this is a config file
for server01
/data/servers/customer01/server02.cfg
this is a config file
for server02
/data/servers/customer02/server03.cfg
this is a config file
for server03
/data/servers/customer03/server04.cfg
this is a config file
for server04
/data/servers/customer03/server05.cfg
this is a config file
for server05
Edit for a reply to #ShawnMilo:
I am trying to bulk add some config to some nagios config files, but not to all server config files.
So, searching with find . -name '*.config' isn't going to work, because then all the config files will get edited.
I only want specific files to get edited, just the servers from the serverlist-file.
Nagios configs need to have the hostname of the server in them, like:
define service {
use generic-service
host_name server01
service_description SSH
contact_groups linux
check_command check_something
}
Seems like an odd requirement. What are you actually trying to do?
In any case, this will do what was requested:
$ find . -name '*.config' | while read x; do echo $x; cat $x; echo; done
./data/servers/customer02/server03.config
default stuff here
./data/servers/customer03/server05.config
default stuff here
./data/servers/customer03/server04.config
default stuff here
./data/servers/customer01/server01.config
default stuff here
./data/servers/customer01/server02.config
default stuff here
$ find . -name '*.config' | while read x; do name=$(basename $x); echo -e "this is a config file\nfor ${name%%.*}" >> $x; done
$ find . -name '*.config' | while read x; do echo $x; cat $x; echo; done
./data/servers/customer02/server03.config
default stuff here
this is a config file
for server03
./data/servers/customer03/server05.config
default stuff here
this is a config file
for server05
./data/servers/customer03/server04.config
default stuff here
this is a config file
for server04
./data/servers/customer01/server01.config
default stuff here
this is a config file
for server01
./data/servers/customer01/server02.config
default stuff here
this is a config file
for server02
I have a FTP folder receiving files from a remote camera. The camera stores the video file name always as ./rec_YYYY-MM-DD_HH-MM.mkv. The video files are stored all in the same folder, the root folder from the FTP server.
I need to move these files to another folder, with this new scheme:
Remove rec_ from the file name.
Change date format to DD-MM-YY.
Remove date from the file name and make it a folder instead, where that same file and all the others in the same date will be stored in.
Final file path would be: ./DD-MM-YYYY/HH-MM.mkv.
The process would continue to all the files, putting them in the folder corresponding to the day it was created.
Summing up: ./rec_YYYY-MM-DD_HH-MM.mkv >> ./DD-MM-YYYY/HH-MM.mkv. The same should apply to all files that are in the same folder.
As I can't make it happen directly from the camera, this needs to be done with Bash on the server that is receiving the files.
So far, what I got is script, which would get the file's creation date and use it to make a folder, and then get creation time to move the file with the new name, based on it's creation time.:
for f in *.mp4
do
mkdir "$f" "$(date -r "$f" +"%d-%m-%Y")"
mv -n "$f" "$(date -r "$f" +"%d-%m-%Y/%H-%M-%S").mp4"
done
I'm getting this output (with testfile 1.mp4):
It creates the folder based on the file's creation date;
it renames the file to it's creation time;
Then, it returns mkdir: cannot create directory ‘1.mp4’: File exists
If two or more files, only one gets renamed and moved as described. The others stay the same and terminal returns:
mkdir: cannot create directory ‘1.mp4’: File exists
mkdir: cannot create directory ‘2.mp4’: File exists
mkdir: cannot create directory ‘12-12-2018’: File exists
Could someone help me out? Better suggestions? Thanks!
Honestly I would just use Perl or Python for this. Here's how to embed either in a shell script.
Here's a perl script that doesn't use any libraries, even ones that ship with Perl (so it'll work without extra packages on distributions like CentOS that don't ship with the entire Perl library). The perl script launches one new process per file in order to perform the copy.
perl -e '
while (<"*.m{p4,kv}">) {
my $path = $_;
my ($prefix, $year, $month, $day, $hour, $minute, $ext) =
split /[.-_]/, $path;
my $sec = q[00];
die "unexpected prefix ($prefix) in $path"
unless $prefix eq q[rec];
die "unexpected extension ($ext) in $path"
unless $ext eq q[mp4] or $ext eq q[mkv];
my $dir = "$day-$month-$year";
my $name = "$hour-$min-$sec" . q[.] . $ext;
my $destpath = $dir . q[/] . $name;
die "$dir . $name is unexpectedly a directory" if (-d $dir);
system("cp", "--", $path, $destpath);
}
'
Here's a Python example, it's compatible with either Python 2 or Python 3 but does use the standard library. The Python script does not spawn any additional processes.
python3 -c '
import os.path as path
import re
from glob import iglob
from itertools import chain
from os import mkdir
from shutil import copyfile
for p in chain(iglob("*.mp4"), iglob("*.mkv")):
fields = re.split("[-]|[._]", p)
prefix = fields[0]
year = fields[1]
month = fields[2]
day = fields[3]
hour = fields[4]
minute = fields[5]
ext = fields[6]
sec = "00"
assert prefix == "rec"
assert ext in ["mp4", "mkv"]
directory = "".join([day, "-", month, "-", year])
name = "".join([hour, "-", minute, "-", sec, ".", ext])
destpath = "".join([directory, "/", name])
assert not path.isdir(destpath)
try:
mkdir(directory)
except FileExistsError:
pass
copyfile(src=p, dst=destpath)
'
Finally, here's a bash solution. It splits paths using -, ., and _ and then extracts various subfields by indexing into $# inside a function. The indexing trick is portable, although regex substitution on variables is a bash extension.
#!/bin/bash
# $1 $2 $3 $4 $5 $6 $7 $8
# path rec YY MM DD HH MM ext
process_file() {
mkdir "$5-$4-$3" &> /dev/null
cp -- "$1" "$5-$4-$3"/"$6-$7-00.$8"
}
for path in *.m{p4,kv}; do
[ -e "$path" ] || continue
# NOTE: two slashes are needed in the substitution to replace everything
# read -a ARRAYVAR <<< ... reads the words of a string into an array
IFS=' ' read -a f <<< "${path//[-_.]/ }"
process_file "$path" "${f[#]}"
done
If you cd /to/some/directory/containing_your_files then you could use the following script
#!/usr/bin/env bash
for f in rec_????-??-??_??-??.m{p4,kv} ; do
dir=${f:4:10} # skip 4 chars ('rec_') take 10 chars ('YYYY_MM_DD')
fnm=${f:15} # skip 15 chars, take the remainder
test -d "$dir" || mkdir "$dir"
mv "$f" "$dir"/"$fnm"
done
note ① that I have not exchanged the years and the days, if you absolutely need to do the swap you can extract the year like this, year=${dir::4} etc and ② that this method of parameter substitution is a Bash-ism, e.g., it doesn't work in dash.
your problem is: mkdir creates folder but you are giving filename for folder creation.
if you want to use fileName for folder creation then use it without extension.
the thing is you are trying to create folder with the already existing fileName
What I'm trying to do is have salt set an internal host ip based on the current value of $i from the for loop. I've tried the following but was unsuccessful at modifying a network script that contains this line: 192.168.200.100 which resides in all 39 nodes.
for ((i=2; i<=30; i++)); do sudo salt -L "host$i.dev.mysite.com" cmd.run "sed -i "s/192.168.200.100/192.168.200.$i/" /etc/sysconfig/network-scripts/ifcfg-bond1; done"
Results I am looking is to have each hostX.dev.mysite.com bond1 files modified from 192.168.200.100 to 192.168.200.2
host2.dev.mysite.com /etc/sysconfig/network-scripts/ifcfg-bond1 = 192.168.200.2
host3.dev.mysite.com /etc/sysconfig/network-scripts/ifcfg-bond1 = 192.168.200.3
host4.dev.mysite.com /etc/sysconfig/network-scripts/ifcfg-bond1 = 192.168.200.4
host5.dev.mysite.com /etc/sysconfig/network-scripts/ifcfg-bond1 = 192.168.200.5
host6.dev.mysite.com /etc/sysconfig/network-scripts/ifcfg-bond1 = 192.168.200.6
host7.dev.mysite.com /etc/sysconfig/network-scripts/ifcfg-bond1 = 192.168.200.7
etc...
Something like this?
for ((i=2;i<=30;i++)); do
sudo salt -L "host${i}.dev.mysite.com" cmd.run "sed -r -i \"s#([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.)[0-9]{1,3}\$#\1$i#\" /etc/sysconfig/network-scripts/ifcfg-bond1"
done
Sorry, I'm from Brazil and my english is not fluent.
I wanna concatenate 20 files using a shellscript through cat command. However when I run it from a file, all content of files are showed on the screen.
When I run it directly from terminal, works perfectly.
That's my code above:
#!/usr/bin/ksh
set -x -a
. /PROD/INCLUDE/include.prod
DATE=`date +'%Y%m%d%H%M%S'`
FINAL_NAME=$1
# check if all paremeters are passed
if [ -z $FINAL_NAME ]; then
echo "Please pass the final name as parameter"
exit 1
fi
# concatenate files
cat $DIRFILE/AI6LM760_AI6_CF2_SLOTP01* $DIRFILE/AI6LM761_AI6_CF2_SLOTP02* $DIRFILE/AI6LM763_AI6_CF2_SLOTP04* \
$DIRFILE/AI6LM764_AI6_CF2_SLOTP05* $DIRFILE/AI6LM765_AI6_CF2_SLOTP06* $DIRFILE/AI6LM766_AI6_CF2_SLOTP07* \
$DIRFILE/AI6LM767_AI6_CF2_SLOTP08* $DIRFILE/AI6LM768_AI6_CF2_SLOTP09* $DIRFILE/AI6LM769_AI6_CF2_SLOTP10* \
$DIRFILE/AI6LM770_AI6_CF2_SLOTP11* $DIRFILE/AI6LM771_AI6_CF2_SLOTP12* $DIRFILE/AI6LM772_AI6_CF2_SLOTP13* \
$DIRFILE/AI6LM773_AI6_CF2_SLOTP14* $DIRFILE/AI6LM774_AI6_CF2_SLOTP15* $DIRFILE/AI6LM775_AI6_CF2_SLOTP16* \
$DIRFILE/AI6LM776_AI6_CF2_SLOTP17* $DIRFILE/AI6LM777_AI6_CF2_SLOTP18* $DIRFILE/AI6LM778_AI6_CF2_SLOTP19* \
$DIRFILE/AI6LM779_AI6_CF2_SLOTP20* > CF2_FINAL_TEMP
mv $DIRFILE/CF2_FINAL_TEMP $DIRFILE/$FINAL_NAME
I solved the problem putting the cat block inside a function, and redirecting stdout to the final file.
Ex:
concatenate()