I am trying to create a ruby script that loads 2 .sql files and removes all strings that begin with 'AUTO_INCREMENT='
There are multiple occurrences of this in my .sql files and all I want is them to be removed from both files.
Thanks for any help or input as I am new to ruby and decided to give it a try.
Given the right regexp (the one below might not be the most correct given the syntax), and the answer given there to a similar question, it is rather straightforward to put a script together:
file_names = ['file1.sql', 'file2.sql']
file_names.each do |file_name|
text = File.read(file_name)
File.open(file_name, 'wb') do
|file|
file.write(text.gsub(/\s*AUTO_INCREMENT\s*(\=\s*[0-9]+)?/, ""))
end
end
Have you tried using Regex for this? If you want to remove the whole line, you could simply match ^AUTO_INCREMENT=.+$ and replace it with an empty string. That pattern should match an entire line beginning with AUTO_INCREMENT.
Here's a good site to learn Regex if you aren't familiar with it:
Hope that works for you.
You should read up on IO, String, Array for more details on methods you can use.
Here's how you might read, modify, and save the contents of one file:
# Opens a file for reading.
file = File.open("file1.txt")
# Reads all the contents into the string 'contents'.
contents = file.read
file.close
# Splits contents into an array of strings, one for each line.
lines = contents.split("\n")
# Delete any lines that start with AUTO_INCREMENT=
lines.reject! { |line| line =~ /^AUTO_INCREMENT=/ }
# Join the lines together into one string again.
new_contents = lines.join("\n")
# Open file for writing.
file = File.open("file1.txt", "w")
# Save new contents.
file.write(new_contents)
file.close
Related
I'm trying to write a ruby script that will read through a CSV file and prepend information to certain cells (for instance adding a path to a file). I am able to open and mutate the text just fine, but am having issues writing back to the CSV without overriding everything. This is a sample of what I have so far:
CSV.foreach(path) { |row|
text = row[0].to_s
new_text = "test:#{text}"
}
I would like to add something within that block that would then write new_textback to the same reference cell(row) in the file. The only way I have to found to write to a file is
CSV.open(path, "wb") { |row|
row << new_text
}
But I think that is bad practice since you are reopening the file within the file block already. Is there a better way I could do this?
EX: I have a CSV file that looks something like:
file,destination
test.txt,A101
and need it to be:
file,destination
path/test.txt,id:A101
Hope that makes sense. Thanks in advance!
Depending on the size if the file, you might consider loading the contents of the file into a local variable and then manipulating that, overwriting the original file.
lines = CSV.read(path)
File.open(path, "wb") do |file|
lines.each do |line|
text = line[0].to_s
line[0] = "test:#{text}" # Replace this with your editing logic
file.write CSV.generate_line(line)
end
end
Alternately, if the file is big, you could write each modified line to a new file along the way and then replace the old file with the new one at the end.
Given that you don't appear to be doing anything that draws on CSV capabilities, I'd recommend using Ruby's "in-place" option variable $-i.
Some of the stats software I use wants just the data, and can't deal with a header line. Here's a script I wrote a while back to (appear to) strip the first line out of one or more data files specified on the command-line.
#! /usr/bin/env ruby -w
#
# User supplies the name of one or more files to be "stripped"
# on the command-line.
#
# This script ignores the first line of each file.
# Subsequent lines of the file are copied to the new version.
#
# The operation saves each original input file with a suffix of
# ".orig" and then operates in-place on the specified files.
$-i = ".orig" # specify backup suffix
oldfilename = ""
ARGF.each do |line|
if ARGF.filename == oldfilename # If it's an old file
puts line # copy lines through.
else # If it's a new file remember it
oldfilename = ARGF.filename # but don't copy the first line.
end
end
Obviously you'd want to change the puts line pass-through to whatever edit operations you want to perform.
I like this solution because even if you screw it up, you've preserved your original file as its original name with .orig (or whatever suffix you choose) appended.
more sysadmin (chef) than ruby guy, so this may be a five minute fix.
I am working on a task where i write a ruby script that pulls json data from multiple files, parses it, and writes the desired fields to a single .csv file. Basically pulling metadata about aws accounts and putting it in an accountant friendly format.
Got a lot of help from another stackoverflow on how to solve the problem for a single file, json.parse help.
My issue is that I am trying to pull the same data from multiple JSON files in an array. I can get it to loop through each file with the code below.
require 'csv'
require "json"
delim_file = CSV.open("delimited_test.csv", "w")
aws_account_list = %w(example example2)
aws_account_list.each do |account|
json_file = File.read(account.to_s + "_aws.json")
parsed_json = JSON.parse(json_file)
delim_file = CSV.open("delimited_test.csv", "w")
# This next line could be a problem if you ran this code multiple times
delim_file << ["EbsOptimized", "PrivateDnsName", "KeyName", "AvailabilityZone", "OwnerId"]
parsed_json['Reservations'].each do |inner_json|
inner_json['Instances'].each do |instance_json|
delim_file << [[instance_json['EbsOptimized'].to_s, instance_json['PrivateDnsName'], instance_json['KeyName'], instance_json['Placement']['AvailabilityZone'], inner_json['OwnerId']],[]]
end
delim_file.close
end
end
However, whenever I do it, it overwrites every time to the same single row in the .csv file. I have tried adding a \n string to the end of the array, converting the array to a string with hashes and doing a \n, but all that does is add a line to the same row that it overwrites.
How would I go about writing that it reads each json file, then appending each files metadata to a new row? This looks like a simple case of writing the right loop, but I can't figure it out.
You declared your file like this:
delim_file = CSV.open("delimited_test.csv", "w")
To fix your issue, all you have to do is change "w" to "a":
delim_file = CSV.open("delimited_test.csv", "a")
See the docs for IO#new for a description of the available file modes. In short, w creates an empty file at the filename, overwriting anyothers, and writes to that. a only creates the file if it doesn't exist, and appends otherwise. Because you have it currently at w, it'll overwrite it each time you run the script. With a, it'll append to what's already there.
You need to open file in append mode, use
delim_file = CSV.open("delimited_test.csv", "a")
'a' Write-only, starts at end of file if file exists, otherwise creates a new file for writing.
'a+' Read-write, starts at end of file if file exists, otherwise creates a new file for reading and writing'
Is there a way to edit each line in a file, without involving 2 files? Say, the original file has,
test01
test02
test03
I want to edit it like
test01,a
test02,a
test03,a
Tried something as show in the code block, but it replaces some of the characters.
Writing it to a temporary file and then replace the original file works, However, I need to edit the file quite often and therefore prefer to do it within the file itself .Any pointers are appreciated.
Thank you!
File.open('mytest.csv', 'r+') do |file|
file.each_line do |line|
file.seek(-line.length, IO::SEEK_CUR)
file.puts 'a'
end
end
f = open 'mytest.csv', 'r+'
r = f.readlines.map { |e| e.strip << ',a' }
f.rewind
f.puts r
f.close # you can leave out this line if it's the last one that runs
Here is a one-liner variation, note that in this case 2 descriptors are left open until the program exits.
open(F='mytest.csv','r+').puts open(F,'r').readlines.map{|e|e.strip<<',a'}
Writing to a file doesn't insert; it always overwrites. This makes it awkward to modify text in-place, because you have to rewrite the entire rest of the contents of the file every time you add something new.
If the file is small enough to fit in memory, you can read it in, modify it, and write it back out. Otherwise, you really are better off with the temporary file.
So I want to read in my .bash_profile and append a string to the PATH.
Should I be opening the file and reading per line until I find what I want then replace? Or read in everything first?
File.open("/root/.bash_profile", "w+") do |file|
while line = line.gets
if line =~ /^PATH/
Not sure how to append
end
end
The w+ mode for files erases all content (I found this in a script that tried to modify its source). If you want to be able to write but keep content, use the r+ mode instead.
NOTE: After seing your problem, why can you not append a line to this effect to the end of the bash profile?:
PATH=/some/path:$PATH
Or will this not work? Because the code for that is simple:
f=File.new("~/.bash_profile", "a+")
f.puts "PATH=/some/path:$PATH"
This may work just as well.
I am trying to do a simple find/replace on all text files in a directory, modifying any instance of [RAVEN_START: by inserting a string (in this case 'raven was here') before the line.
Here is the entire ruby program:
#!/usr/bin/env ruby
require 'rubygems'
require 'fileutils' #for FileUtils.mv('your file', 'new location')
class RavenParser
rawDir = Dir.glob("*.txt")
count = 0
rawDir.each do |ravFile|
#we have selected every text file, so now we have to search through the file
#and make the needed changes.
rav = File.open(ravFile, "r+") do |modRav|
#Now we've opened the file, and we need to do the operations.
if modRav
lines = File.open(modRav).readlines
lines.each { |line|
if line.match /\[RAVEN_START:.*\]/
line.gsub!(/\[RAVEN_START:/, 'raven was here '+line)
count = count + 1
end
}
printf("Total Changed: %d\n",count)
else
printf("No txt files found. \n")
end
end
#end of file replacing instructions.
end
# S
end
The program runs and compiles fine, but when I open up the text file, there has been no change to any of the text within the file. count increments properly (that is, it is equal to the number of instances of [RAVEN_START: across all the files), but the actual substitution is failing to take place (or at least not saving the changes).
Is my syntax on the gsub! incorrect? Am I doing something else wrong?
You're reading the data, updating it, and then neglecting to write it back to the file. You need something like:
# And save the modified lines.
File.open(modRav, 'w') { |f| f.puts lines.join("\n") }
immediately before or after this:
printf("Total Changed: %d\n",count)
As DMG notes below, just overwriting the file isn't properly paranoid as you could be interrupted in the middle of the write and lose data. If you want to be paranoid (which all of us should be because they really are out to get us), then you want to write to a temporary file and then do an atomic rename to replace the original file the new one. A rename generally only works when you stay within a single file system as there is no guarantee that the OS's temp directory (which Tempfile uses by default) will be on the same file system as modRav so File.rename might not even be an option with a Tempfile unless precautions are taken. But the Tempfile constructor takes a tmpdir parameter so we're saved:
modRavDir = File.dirname(File.realpath(modRav))
tmp = Tempfile.new(modRav, modRavDir)
tmp.write(lines.join("\n"))
tmp.close
File.rename(tmp.path, modRav)
You might want to stick that in a separate method (safe_save(modRav, lines) perhaps) to avoid further cluttering your block.
There is no gsub! in the post (except the title and question). I would actually recommend not using gsub!, but rather use the result of gsub -- avoiding mutability can help reduce a number of subtle bugs.
The line read from the file stream into a String is a copy and modifying it will not affect the contents of the file. (The general approach is to read a line, process the line, and write the line. Or do it all at once: read all lines, process all lines, write all processed lines. In either case, nothing is being written back to the file in the code in the post ;-)
Happy coding.
You're not using gsub!, you're using gsub. gsub! and gsub different methods, one does replacement on the object itself and the other does replacement then returns the result, respectively.
Change this
line.gsub(/\[RAVEN_START:/, 'raven was here '+line)
to this :
line.gsub!(/\[RAVEN_START:/, 'raven was here '+line)
or this:
line = line.gsub(/\[RAVEN_START:/, 'raven was here '+line)
See String#gsub for more info