I've to rename around 5 000 folders located in a remote storage. Running Dir['**/*/'] returns an error "*** stack smashing detected ***" and invites me to report the bug as it might occurs during the interpretation process (see bug report)
If it can help, here's the script I was planning to run (works fine on a test environment, though it's quite specific to my needs)
#!/usr/bin/env ruby
# Fetch root directories
dirs = Dir['**/*/'].select { |d| d =~ /\d([\.-]{1}\d{2,})?/ }
# Order subdirectories first
dirs = dirs.sort_by { |d| d.count('/') }.reverse
# Substitute "." and "-" placed after the last "/" with "_"
dirs.each do |dir|
File.rename(dir, dir.gsub(/[\.-](?!.*\/.*)/, '_'))
end
Any suggestion for mitigating this issue ?
It's neither a well formed question, nor a generic answer, but I managed to get around the issue by specifying the deepness to look at. Concretely, I replaced Dir['**/*/'] by Dir['*/*/*/*/'].
However I'm open to other suggestions, as others may face similar issues without the possibility to hardcode the deepness to look at.
Related
For my current module, I need to check if php version 5 or 7 is installed and created a fact for this. The fact file is stored in the modules directory in facts.d/packageversion.rb and has the following content:
#!/usr/bin/ruby
require 'facter'
Facter.add(:php_version) do
setcode do
if File.directory? '/etc/php5'
5
else
if File.directory? '/etc/php7'
7
else
0
end
end
end
end
But I can't use it in my module. In Puppet agent log, i get this error:
Fact file /var/lib/puppet/facts.d/packageversion.rb was parsed but
returned an empty data set
How can I solve this?
facts.d is the module directory for external facts. You could place this file into the external facts directory, but the expected output would need to be key-value pairs. This is not happening, so Puppet is not finding a data set for the fact. https://docs.puppet.com/facter/3.6/custom_facts.html#executable-facts-----unix
You have written this fact as a custom fact and not an external fact. Therefore, it needs to be placed inside the lib/facter directory in your module instead. Then it will function correctly. I notice this important information seems to have been removed from the latest Facter documentation, which probably lends to your confusion.
Also, consider using an elsif in your code for clarity and optimization:
if File.directory? '/etc/php5'
5
elsif File.directory? '/etc/php7'
7
else
0
end
What Matt Schuchard said.
Also, you might consider that the Approved Vox Populi Puppet module uses this code for PHP version:
Facter.add(:phpversion) do
setcode do
output = Facter::Util::Resolution.exec('php -v')
unless output.nil?
output.split("\n").first.split(' ').
select { |x| x =~ %r{^(?:(\d+)\.)(?:(\d+)\.)?(\*|\d+)} }.first
end
end
end
Note that Facter::Util::Resolution.exec is deprecated in favour of Facter::Core::Execution.exec.
Aside from that, you might consider this a better way of getting the PHP version.
I want to provide a parser for parsing so called Subversion config auth files (see patch based authorization in the Subversion red book). Here I want to define rules for directories like
[/]
* = r
[/trunk]
#PROJECT = rw
So the part of the grammar I have problems is the path definition. I currently have the following rules in Parslet:
rule(:auth_rule_head) { (str('[') >> path >> str(']') >> newline).as(:arh) }
rule(:top) { (str('/')).as(:top) }
rule(:path) { (top | ((str('/') >> path_ele).repeat)).as(:path) }
rule(:path_ele) { ((str('/').absent? >> any).repeat).as(:path_ele) }
So I want to divide in two cases:
To find only [/] (the root directory)
in all other cases [/<dir>] which may be repeated, but has to end without a /
The problematic rule seems to be the path that defines an alternative, here / XOR something like /trunk
I have defined test cases for those, and get the following error when running the test case:
Failed to match sequence (SPACES '[' PATH ']' NEWLINE) at line 1 char 3.
`- Expected "]", but got "t" at line 1 char 3.
So the problem seems to be, that the alternative (rule :path) is chosen all the time top.
What is a solution (as a grammar) for this problem? I think there should be a solution, and this looks like something idiomatic that should happen from here to there. I am not an expert at all with PEG parsers or parser / compiler generation, so if that is a base problem not solvable, I would like to know that as well.
In short: Swap the OR conditions around.
Parlset rules consume the input stream until they get a match, then they stop.
If you have two possible options (an OR), the first is tried, and only if it doesn't match is the second tried.
In your case, as all your paths start with '/' they all match the first part of the path rule, so the second half is never explored.
You need to try to match the full path first, and only match the 'top' if it fails.
# changing this
rule(:path) { (top | ((str('/') >> path_ele).repeat)).as(:path) }
# to this
rule(:path) { ((str('/') >> path_ele).repeat) | top).as(:path) }
# fixes your first problem :)
Also... Be careful of rules that can consume nothing being in a loop.
Repeat by default is repeat(0). Usually it needs to be repeat (1).
rule(:path) { ((str('/') >> path_ele).repeat(1)) | top).as(:path) }
also...
Is "top" really a special case? All paths end in a "/", so top is just the zero length path.
rule(:path) { (path_ele.repeat(0) >> str('/')).as(:path) }
Or
rule(:path) { (str('/') >> path_ele.repeat(0)).as(:path) }
rule(:path_ele) { ((str('/').absent? >> any).repeat(0)).as(:path_ele) >> str('/') }
# assuming "//" is valid otherwise repeat(1)
Seems to be I have not got the problem right. I have tried to reproduce the problem in creating a small example grammar including some unit tests, but now, the thing is working.
If you are interested in it, have a look at the gist https://gist.github.com/mliebelt/a36ace0641e61f49d78f. You should be able to download the file, and run it directly from the command line. You have to have installed first parslet, minitest should be already included in a current Ruby version.
I have added there only the (missing) rule for newline, and added 3 unit tests to test all cases:
The root: /
A path with only one element: /my
A path with more than one element: /my/path
Works like expected, so I get two cases here:
Top element only
One or more path elements
Perhaps this may help others how to debug a situation like that.
I'm trying to make a program that searches through hopefully every directory, sub directory, sub sub directory and so on in C:\. I feel like I can take care of that part, but there's also the issue of the folder names. There may be case issues like a folder named FOO not being dected when my program searches for Foo or a giant if/else or case statement for multiple search criteria.
My questions are: 1. is there a way to ignore letter case? and 2. is there a way to make a more efficient statement for searching?
My current code:
#foldersniffer by Touka, ©2015
base = Dir.entries("C:\\")
trees = Dir.entries("#{base}")
trees.each do |tree|
if Dir.exist?("Foo")
puts "Found Folder \"Foo\" in C:\\"
elsif Dir.exist?("Bar")
puts "Found Folder \"Bar\" in C:\\"
else
puts "No folders found"
end
end
sleep
any help is appreciated.
edit: it's trying to scan files like bootmgr and it's giving me errors... I'm not sure how to fix that.
Consider using Dir.glob(...) and regular expressions for case insensitive matching:
Dir.glob('c:\\**\*') do |filename|
if filename =~ /c:\\(foo|bar)($|\\)/i
puts "Found #{filename}"
end
end
Case sensitivity for the Dir.glob argument is likely not relevant on Windows systems:
Note that this pattern is not a regexp, it’s closer to a shell glob. See File.fnmatch for the meaning of the flags parameter. Note that case sensitivity depends on your system (so File::FNM_CASEFOLD is ignored), as does the order in which the results are returned.
I am not expert enough to say for sure but I would look into File::FNM_CASEFOLD
https://lostechies.com/derickbailey/2011/04/14/case-insensitive-dir-glob-in-ruby-really-it-has-to-be-that-cryptic/
I have hundreds of files that also have hundreds of 'should' statements.
Is there any sort of automated way to update these files to the new syntax?
I'd like options to both create new files and also modify the existing files inline.
sed is a good tool for this.
The following will process all the files in the current directory and write them out to new files in a _spec_seded directory. This currently handle about 99%+ of the changes but might still leave you with a couple of manual changes to make (the amount will depend on your code and coding style).
As always with a sed script you should check the results, run diffs and look at the files manually. Ideally you are using git which helps make the diffs even easier.
filenum=1
find . -type f -name '*_spec.rb' | while read file; do
mkdir -p ../_spec_seded/"${file%/*}"
echo "next file...$filenum...$file"
let filenum+=1
cp "$file" ../_spec_seded/"$file"
sed -i ' # Exclude:
/^ *describe .*do/! { # -describe...do descriptions
/^ *it .*do/! { # -it...do descriptions
/^[[:blank:]]*\#/! { # -comments
/^ *def .*\.should.*/! { # -inline methods
/\.should/ {
s/\.should/)\.to/ # Change .should to .to
s/\(\S\)/expect(\1/ # Add expect( at start of line.
/\.to\( \|_not \)>\=/ s/>\=/be >\=/ # Change operators for
/\.to\( \|_not \)>[^=]/ s/>/be >/ # >, >=, <, <= and !=
/\.to\( \|_not \)<\=/ s/<\=/be <\=/
/\.to\( \|_not \)<[^=]/ s/</be </
/\.to\( \|_not \)\!\=/ s/\!\=/be \!\=/
}
/\.to +==\( +\|$\)/ s/==/eq/
/=\~/ { # Change match operator
s/=\~/match(/
s/$/ )/
s/\[ )$/\[/
}
s/[^}.to|end.to]\.to /).to / # Add paren
/eq ({.*} )/ s/ ({/ ( {/ # Add space
/to\(_\|_not_\)receive/ s/_receive/ receive/ # receive
/\.to eq \[.*\]/ {
s/ eq \[/ match_array([/
s/\]$/\])/
}
/expect.*(.*lambda.*{.*})/ { # Remove unneeded lambdas
s/( *lambda *{/{/
s/ })\.to / }\.to /
}
/expect *{ *.*(.*) *})\.to/ { # Fix extra end paren
s/})\.to/}\.to/
}
}
}
}
}' ../_spec_seded/"$file"
done
Please use with caution. Currently the script create new files in _seded/ for review first for safety. The script is placed in /spec directory and run from there.
If you have hundreds of files this could save you hours or days of work!
If you use this I recommend that "step 2" is do manually copy files from _spec_seded to spec itself and run them. I recommend that you don't just rename the whole directories. For one thing, files, such as spec_helper.rb aren't currently copied to _spec_seded.
11/18/2013 Note: I continue to upgrade this script. Covering more edge cases and also making matches more specific and also excluding more edge cases, e.g. comment lines.
P.S. The differences which should be reviewed can be seen with (from the project directory root):
diff -r /spec /_spec_seded
git also has nice diff options but I like to look before adding files to git at all.
Belated update, mainly for those who may find their way to this page via a search engine.
Use Yuji Nakayama's excellent Transpec gem for this purpose. I've used it over 10 times now on different projects without issue.
From the website:
Transpec lets you upgrade your RSpec 2 specs to RSpec 3 in no time. It supports conversions for almost all of the RSpec 3 changes, and it’s recommended by the RSpec team.
Also, you can use it on your RSpec 2 project even if you’re not going to upgrade it to RSpec 3 for now.
New to Ruby, probably something silly
Trying to make a directory in order to store files in it. Here's my code to do so
def generateParsedEmailFile
apath = File.expand_path($textFile)
filepath = Pathname.new(apath + '/' + #subject + ' ' + #date)
if filepath.exist?
filepath = Pathname.new(filepath+ '.1')
end
directory = Dir.mkdir (filepath)
Dir.chdir directory
emailText = File.new("emailtext.txt", "w+")
emailText.write(self.generateText)
emailText.close
for attachment in #attachments
self.generateAttachment(attachment,directory)
end
end
Here's the error that I get
My-Name-MacBook-2:emails myname$ ruby etext.rb email4.txt
etext.rb:196:in `mkdir': Not a directory - /Users/anthonydreessen/Developer/Ruby/emails/email4.txt/Re: Make it Brief Report Wed 8 May 2013 (Errno::ENOTDIR)
from etext.rb:196:in `generateParsedEmailFile'
from etext.rb:235:in `<main>'
I was able to recreate the error - it looks like email4.txt is a regular file, not a directory, so you can't use it as part of your directory path.
If you switched to mkdir_p and get the same error, perhaps one of the parents named in '/Users/anthonydreessen/Developer/Ruby/emails/email4.txt/Re: Make it Brief Report Wed 8 May 2013' already exists as a regular file and can't be treated like a directory. Probably that last one named email.txt
You've got the right idea, but should be more specific about the files you're opening. Changing the current working directory is really messy as it changes it across the entire process and could screw up other parts of your application.
require 'fileutils'
def generate_parsed_email_file(text_file)
path = File.expand_path("#{#subject} #{date}", text_file)
while (File.exist?(path))
path.sub!(/(\.\d+)?$/) do |m|
".#{m[1].to_i + 1}"
end
end
directory = File.dirname(path)
unless (File.exist?(directory))
FileUtils.mkdir_p(directory)
end
File.open(path, "w+") do |email|
emailText.write(self.generateText)
end
#attachments.each do |attachment|
self.generateAttachment(attachment, directory)
end
end
I've taken the liberty of making this example significantly more Ruby-like:
Using mixed-case names in methods is highly irregular, and global variables are frowned on.
It's extremely rare to see for used, each is much more flexible.
The File.open method yields to a block if the file could be opened, and closes automatically when the block is done.
The ".1" part has been extended to keep looping until it finds an un-used name.
FileUtils is employed to makes sure the complete path is created.
The global variable has been converted to an argument.