Say you copy folder f_1 from your local machine to target machine m_1 into the /tmp directory as mf_1.
console:
[root#m_1 tmp] ls -a | grep mf_1 # => doesn't exist
irb:
options = {recursive: true}
Net::SCP.upload!(host, user, '~/f_1', '/tmp/mf_1', options)
console:
[root#m_1 tmp] ls -a | grep mf_1 # => folder exists, everything is fine
# but then, if you try to overwrite the existing folder...
irb:
Net::SCP.upload!(host, user, '~/f_1', '/tmp/mf_1', options)
console:
[root#m_1 tmp] ls -a | grep mf_1 # => folder exists
[root#m_1 tmp] cd mf_1
[root#m_1 m_f1] ls # => f_1 => /tmp/mf_1/f_1
So, instead of mf_1 being overwritten folder was copied inside of /tmp/mf_1, resulting in /tmp/mf_1/f_1.
The question is pretty simple, how to preserve the behavior so it's consistent and calling
Net::SCP.upload!(host, user, '~/f_1', '/tmp/mf_1', options)
twice in a row would act the same way both when folder exists and doesn't?
I ended up adding a dot, if source is a dir.
This is not ideal, here's an example:
options = {recursive: true}
# target /tmp/mf_1 doesn't exist
Net::SCP.upload!(host, user, '~/f_1/.', '/tmp/mf_1', options)
# target /tmp/mf_1 has been created
# second time
Net::SCP.upload!(host, user, '~/f_1/.', '/tmp/mf_1', options)
# target /tmp/mf_1 has been overwritten
# not dir, but files in it, which is what we usually want
Related
I have multiple folders where two files are present.
For example, 123.jpg, 456.jpg under folder ABC. I want to rename the files to IT1_ABC_123.v1.jpg and IT2_ABC_456.v1.jpg. Similarly, other folders have two files.
How can I do this in shell or Perl?
Try this, using shell and perl:
mkdir /tmp/test; cd $_
mkdir ABC DEF
touch {ABC,DEF}/{123,456}.jpg #creates four files, two in each directory
find|perl -nlE's,((.*)/(.+))/((123|456).jpg),$1/IT#{[++$n]}_$3_$4,&&say"$&\n$_\n"'
./ABC/123.jpg
./ABC/IT1_ABC_123.jpg
./ABC/456.jpg
./ABC/IT2_ABC_456.jpg
./DEF/123.jpg
./DEF/IT3_DEF_123.jpg
./DEF/456.jpg
./DEF/IT4_DEF_456.jpg
Now, after confirming this is what you want, replace the say with a rename:
find|perl -nlE's,((.*)/(.+))/((123|456).jpg),$1/IT#{[++$n]}_$3_$4, and rename$&,$_'
The new filenames:
find -type f
./ABC/IT1_ABC_123.jpg
./ABC/IT2_ABC_456.jpg
./DEF/IT3_DEF_123.jpg
./DEF/IT4_DEF_456.jpg
This will find filenames with 123.jpg or 456.jpg and rename them.
s,,, is the search-replace and it returns 1 (the number of changes it made) which again leads to the right side of the and being done (the rename).
Filenames that doesn't match 123.jpg or 456.jpg isn't renamed since s,,, will return 0 and the and is "short cutted" since it then logically cannot be true with a false (0) left side. So then the rename is not executed.
This variant does the same, but might be easier to read:
find|perl -nlE 'rename$&,$_ if s,((.*)/(.+))/((123|456).jpg),$1/IT#{[++$n]}_$3_$4,'
I have found this pattern useful in many cases of mass renamings. Also, dedicated software for mass renaming with GUIs exists, which for some might be easier to use.
Rewritten as a program abc.pl, it could be:
#!/usr/bin/perl
while(<>){
chomp;
next if not s,((.*)/([A-Z]{3}))/(\d{3}\.jpg),$1/IT#{[++$n]}_$3_$4,;
print "Found: $&\nNew name: $_\n\n";
#rename $&, $_;
}
Run:
find|perl abc.pl
You can do this in core Perl using the File::Find, File::Basename, and File::Copy modules. You can test it out with the script below. It won't make any changes until you uncomment the line with the "move" function.
#! perl
use strict;
use warnings;
use File::Basename;
use File::Copy;
use File::Find;
my $root_dir = '/path/to/main/folder';
# Recursively searches for all files below the $root_dir
my #fileset;
find(
sub {
# Get the absolute file path
my $path = $File::Find::name;
# Only capture the path if not a directory
# You can add any number of conditions here
if (!-d $path) {
push #fileset, $path;
}
},
$root_dir
);
# set the IT counter in new file name
my $int = 1;
# list of all possible file suffixes to have fileparse() look for. It will
# capture the end of the file path verbatim (including the period) if it's
# in this array
my #suffixes = ('.jpg', '.txt');
my $previous_dir;
foreach my $old_path (#fileset) {
# split apart the basename of the file, the directory path, and the file suffix
my ($basename, $parent_dir, $suffix) = fileparse($old_path, #suffixes);
# strip off trailing slash so fileparse() will capture parent dir name correctly
$parent_dir =~ s{[/]$}{};
# capture just the name of the parent directory
my $parent_name = fileparse($parent_dir);
# Assemble the new path
my $new_path = $parent_dir . '/IT' . $int . '_'
. $parent_name . '_' . "$basename.v1" . $suffix;
# Move the file to rename (this is safer than using rename() for cross-platform)
# move $old_path, $new_path;
print "OLD PATH: $old_path\n";
print "NEW PATH: $new_path\n\n";
# Reset counter when dir changes
if (!$previous_dir) {
$previous_dir = $parent_dir; # set previous_dir on first loop
}
elsif($previous_dir ne $parent_dir) {
$previous_dir = $parent_dir; # update previous_dir to check next loop
$int = 0; # reset counter
}
$int++; # iterate the counter
}
Edit 2018-07-12: I've updated the answer to show how to reset the counter when the directory changes by evaluating the current path with the one used in the previous loop and updating accordingly. This is not tested so it may need some adjustments.
Given the abc/def examples given, the output should look something like this:
OLD PATH: /path/to/main/folder/abc/123.jpg
NEW PATH: /path/to/main/folder/abc/IT1_abc_123.v1.jpg
OLD PATH: /path/to/main/folder/abc/456.txt
NEW PATH: /path/to/main/folder/abc/IT2_abc_456.v1.jpg
OLD PATH: /path/to/main/folder/def/123.jpg
NEW PATH: /path/to/main/folder/def/IT1_def_123.v1.jpg
OLD PATH: /path/to/main/folder/def/456.jpg
NEW PATH: /path/to/main/folder/def/IT2_def_456.v1.jpg
I have a bash command that will return either 1 or 0. I want to run said command from puppet:
exec { 'Check if Thinkpad':
command => 'sudo dmidecode | grep -q ThinkPad && echo 1 || echo 0',
path => '/usr/bin/:/bin/bash/',
environment => "HOME=/root"
}
Is there a way I can include a file using puppet only if my command returned 1?
file { '/etc/i3/config':
source => 'puppet:///modules/i3/thinkpad',
owner => 'root',
group => 'root',
mode => '0644',
}
You can use an external fact to use the bash script as is. Inside the module's facts.d directory, you could place the script.
#!/bin/bash
if [ dmidecode | grep -q ThinkPad ]
echo 'is_thinkpad=true'
else
echo 'is_thinkpad=false'
fi
You can also use a custom fact inside the lib/facter directory of your module.
Facter.add(:is_thinkpad) do
confine kernel: linux
setcode do
`dmidecode | grep -q ThinkPad && echo true || echo false`
end
end
In both cases, the fact name of is_thinkpad follows the convention for the nomenclature of boolean facts for types of systems. You can then update the code in your manifest for this boolean.
if $facts['is_thinkpad'] == true {
file { '/etc/i3/config':
source => 'puppet:///modules/i3/thinkpad',
owner => 'root',
group => 'root',
mode => '0644',
}
}
This will provide you with the functionality you desire.
https://docs.puppet.com/facter/3.6/custom_facts.html#adding-custom-facts-to-facter
https://docs.puppet.com/facter/3.6/custom_facts.html#external-facts
You will probably need to turn your bash script into a "custom fact" -- which is something I've only done once and don't fully understand enough to teach you how.
I want to say that the easiest way to set up a custom fact is to put your script into /etc/facter/facts.d/ on the agent machine, and make sure it ends with a line that says
echo "thinkpadcheck=1"
or
echo "thinkpadcheck=0"
You can test it with (note: you must be root)
sudo facter -p | grep think
and it should return
thinkpadcheck => 1
But once you have done that, then your puppet script can say
if $thinkpadcheck == 1
{
file { '/etc/i3/config':
source => 'puppet:///modules/i3/thinkpad',
owner => 'root',
group => 'root',
mode => '0644',
}
}
else
{
notify { "thinkpadcheck failed for $hostname" : }
}
I'd like to share another method I found in the Puppet Cookbook 3rd edition (page 118):
message.rb
#!/usr/bin/env ruby
puts "This runs on the master if you are centralized"
Make your script executable with:
sudo chmod +x /usr/local/bin/message.rb
message.pp
$message = generate('/usr/local/bin/message.rb')
notify { $message: }
Then run:
puppet apply message.pp
This example uses a ruby script but any type of script including a basic shell script, as was needed in my case, can be used to set a variable in puppet.
I'm trying out to copy all *.md files from a directory to another.
With that code:
project = Hoe::Manns.get_projectname
dest = "#{Dir.home}/RubymineProjects/#{project}"
wikipath = "#{Dir.home}/RubymineProjects/#{project}.wiki"
FileUtils.mkdir_p "#{dest}/doc", verbose: true if !File.exist?('doc')
files = Dir.glob("#{wikipath}/*.md")
FileUtils.cp "#{files}", "#{dest}/doc", verbose: true
i'm getting
cp ["/home/sascha/RubymineProjects/hoe-manns.wiki/home.md", "/home/sascha/RubymineProjects/hoe-manns.wiki/behind-the-scene.md"] /home/sascha/RubymineProjects/hoe-manns/doc
rake aborted!
Errno::ENOENT: No such file or directory # rb_sysopen - ["/home/sascha/RubymineProjects/hoe-manns.wiki/home.md", "/home/sascha/RubymineProjects/hoe-manns.wiki/behind-the-scene.md"]
Maybe i missed anything?
FileUtils.cp can take a single filename or an array of filenames as its first argument. Try this:
FileUtils.cp files, "#{dest}/doc", verbose: true
First off, very new to Capistrano, so excuse my newbness.
I am deploying a CodeIgniter application of which I have a data folder that lives outside the current web directory that I am trying to symlink to. This is what I have used to create the symlink
set :shared_dir, %w(/var/www/website/)
set :linked_dirs, %w{43b621acdfc9523443f27b7767e}
This is the result of the deployed application
43b621acdfc9523443f27b7767e -> /var/www/website/shared/43b621acdfc9523443f27b7767e
My goal is to override the shared_dir variable to generate a symlink of
/var/www/website/43b621acdfc9523443f27b7767e
I am using Capistrano 3. Any insight greatly appreciated.
Thanks
You can create an specific task to take care of creating and updating the symlink from anywhere in your application to anywhere in your system on each deploy. For example:
namespace :terminal do
task :link_external_dir, :except => { :no_release => true } do
capifony_pretty_print '--> Generating soft link with external_example_dir'
run "sharedWebDir=#{shared_path}/web; cd $sharedWebDir; if [ -d $sharedWebDir/internal_example_dir -o -f $sharedWebDir/internal_example_dir ]; then rm -rf $sharedWebDir/internal_example_dir; fi; if [ ! -L $sharedWebDir/internal_example_dir ]; then ln -s /srv/example/vhost/external_example_dir $sharedWebDir/internal_example_dir; fi;"
capifony_puts_ok
end
task :rm_shared_example_dir, :except => { :no_release => true } do
capifony_pretty_print '--> Removing sharedDir/internal_example_dir symlink'
run "sharedWebDir=#{shared_path}/internal_example_dir; if [ -L $sharedWebDir/internal_example_dir ]; then rm $sharedWebDir/internal_example_dir; fi;"
capifony_puts_ok
end
end
Now call each function on each deploy:
before "deploy", "terminal:rm_shared_example_dir"
after "deploy", "deploy:cleanup"
after "deploy", "terminal:link_external_dir"
What we are trying to accomplish is:
1.Before Deploy, remove the file from your main sharedDir that links to your external dir.
REMOVE
/var/www/website/shared/externalDir -> /srv/example/vhost/external_example_dir
2.Deploy your app.
3.Generate the custom symlink again.
ln -s /srv/example/vhost/external_example_dir $sharedWebDir/internal_example_dir
I would like to create a package that contains a file but renames it inside the package.
For example:
Rake::PackageTask.new("rake", "1.2.3") do |p|
p.package_files.include("aa.rb")
end
I would like aa.rb to be named bb.rb inside the package.
How can I do this elegantly?
Looking at the PackageTask source, it seems you could define a new task (say rename_files) that depends on on the p.package_dir_path task defined by Rake::PackageTask. In rename_files task you can rename the file link(s) which package_dir_path task made in package_dir. Then you add your new rename_files task as a dependency for each of the "#{package_dir}/#{[tar|zip|etc]_file}" task targets you care about.
With these dependencies, the order of operations should become:
set up package_dir with links to source files from package_files
rename links with your injected dependency
execute archive creation command on package_dir
If this isn't clear enough to get you there, I'll try and post some actual code later.
[LATER] Ok, some code. I made a sample project which looks like this:
$ find .
.
./lib
./lib/aa.rb
./lib/foo.rb
./Rakefile
And in the Rakefile, I define a package task as:
require 'rake/packagetask'
Rake::PackageTask.new('test', '1.2.3') do |p|
p.need_tar = true
p.package_files.include('lib/**/*')
task :rename_files => [ p.package_dir_path ] do
fn = File.join( p.package_dir_path, 'lib', 'aa.rb' )
fn_new = File.join( p.package_dir_path, 'lib', 'bb.rb' )
File.rename( fn, fn_new )
end
[
[p.need_tar, p.tgz_file, "z"],
[p.need_tar_gz, p.tar_gz_file, "z"],
[p.need_tar_bz2, p.tar_bz2_file, "j"],
[p.need_zip, p.zip_file, ""]
].each do |(need, file, flag)|
task "#{p.package_dir}/#{file}" => [ :rename_files ]
end
end
The logic here is what I explained above. Running it, you can see that the hard link made in the package dir is renamed from "aa.rb" to "bb.rb", then we tar the directory and viola!
$ rake package
(in /Users/dbenhur/p)
mkdir -p pkg
mkdir -p pkg/test-1.2.3/lib
rm -f pkg/test-1.2.3/lib/aa.rb
ln lib/aa.rb pkg/test-1.2.3/lib/aa.rb
rm -f pkg/test-1.2.3/lib/foo.rb
ln lib/foo.rb pkg/test-1.2.3/lib/foo.rb
cd pkg
tar zcvf test-1.2.3.tgz test-1.2.3
a test-1.2.3
a test-1.2.3/lib
a test-1.2.3/lib/bb.rb
a test-1.2.3/lib/foo.rb
cd -
Here's the tar manifest with "bb.rb" instead of "aa.rb":
$ tar tf pkg/test-1.2.3.tgz
test-1.2.3/
test-1.2.3/lib/
test-1.2.3/lib/bb.rb
test-1.2.3/lib/foo.rb