I'm capturing/creating user entered text into files from my app, attempting to temporarily store them in my Heroku tmp directory, then upload them to a cloud service such as Google Drive.
In using Tempfile I can successfully upload, but when using File.open I get the following error when attempting to upload:
ArgumentError (wrong number of arguments (1 for 0))
The error is on the call:
#client.upload_file_by_folder_id(save_path, #folder_id)
Where #client is a session with the cloud service, save_path is the location of the attached file for upload and #folder_id is the folder they should go into.
When I use Tempfile.new I am successful in doing so:
tempfile = Tempfile.new([final_filename, '.txt'], Rails.root.join('tmp','text-temp'))
tempfile.binmode
tempfile.write msgbody
tempfile.close
save_path = tempfile.path
upload_file = #client.upload_file_by_folder_id(save_path, #folder_id)
tempfile.unlink
File.open code is:
path = 'tmp/text-temp'
filename = "#{final_filename}.txt"
save_path = Rails.root.join(path, filename)
File.open(save_path, 'wb') do |file|
file.write(msgbody)
file.close
end
upload_file = #client.upload_file_by_folder_id(save_path, #folder_id)
File.delete(save_path)
Could it be that the File.path is a string, and Tempfile.path is the full path (but not as a string)? When I put out each, they look identical.
I'd like to use File as I don't want to change the filename of the existing attachments I'm uploading, whereas Tempfile appends to the filename.
Any and all assistance is greatly appreciated. Thanks!
In order for it to work using File, I needed to set the save_path to a string:
save_path.to_s
Related
I want to upload data binary directly to GCP storage, without writing the file to disk. Below is the code snippet I have created to get to the state that I am going to be at.
require 'google/cloud/storage'
bucket_name = '-----'
data = File.open('image_block.jpg', 'rb') {|file| file.read }
storage = Google::Cloud::Storage.new("project_id": "maybe-i-will-tell-u")
bucket = storage.bucket bucket_name, skip_lookup: true
Now I want to directly put this data into a file on gcs, without having to write a file to disk.
Is there an efficient way we can do that?
I tried the following code
to_send = StringIO.new(data).read
bucket.create_file to_send, "image_inder_11111.jpg"
but this throws an error saying
/google/cloud/storage/bucket.rb:2898:in `file?': path name contains null byte (ArgumentError)
from /home/inder/.gem/gems/google-cloud-storage-1.36.1/lib/google/cloud/storage/bucket.rb:2898:in `ensure_io_or_file_exists!'
from /home/inder/.gem/gems/google-cloud-storage-1.36.1/lib/google/cloud/storage/bucket.rb:1566:in `create_file'
from champa.rb:14:in `<main>'
As suggested by #stefan, It should be to_send = StringIO.new(data), i.e. without .read (which would return a string again)
As part of my larger program, I want to create a logfile with the current time & date as part of the title. I can create it as follows:
malwareLog = open(datetime.datetime.now().strftime("%Y%m%d - %H.%M " + pcName + " Malware scan log.txt"), "w+")
Now, my app is going to call a number of other functions, so I'll need to open the file, write some output to it and close the file, several times. It doesn't seem to work if I simply go:
malwareLog.open(malwareLog, "a+")
or similar. So how should I open a dynamically created txt file that I don't know the actual filename for...?
When you create malwareLog object, it has name attribute which contains the file name.
Here's an example: (my test is your malwareLog)
import random
test = open(str(random.randint(0,999999))+".txt", "w+")
test.write("hello ")
test.close()
test = open(test.name, "a+")
test.write("world!")
test.close()
with open(test.name, "r") as f: print(f.read())
You also can store the file name in a variable before or after creating the file.
###Before
file_name = "123"
malwareLog = open(file_name, "w")
###After
malwareLog = open(random.randint(0,999999), "w")
file_name = malwareLog.name
I am practicing Ruby, and I am trying to copy contents from file "from" to file "to". can you tell me where I did it wrong?
thanks !
from = "1.txt"
to = "2.txt"
data = open(from).read
out = open(to, 'w')
out.write(data)
out.close
data.close
Maybe I am missing the point, but I think writing it like so is more 'ruby'
from = "1.txt"
to = "2.txt"
contents = File.open(from, 'r').read
File.open(to, 'w').write(contents)
Personally, however, I like to use the Operating systems terminal to do File operations like so. Here is an example on linux.
from = "1.txt"
to = "2.txt"
system("cp #{from} #{to}")
And for Windows I believe you would use..
from = "1.txt"
to = "2.txt"
system("copy #{from} #{to}")
Finally, if you were needing the output of the command for some sort of logging or other reason, I would use backticks.
#A nice one liner
`cp 1.txt 2.txt`
Here is the system and backtick methods documentation.
http://ruby-doc.org/core-1.9.3/Kernel.html
You can't perform data.close — data.class would show you that you have a String, and .close is not a valid String method. By opening from the way you chose to, you lost the File reference after using it with your read. One way to fix that would be:
from = "1.txt"
to = "2.txt"
infile = open(from) # Retain the File reference
data = infile.read # Use it to do the read
out = open(to, 'w')
out.write(data)
out.close
infile.close # And finally, close it
I'm building a web tool in Heroku / Ruby Sinatra that scrapes a web domain and downloads all specified filetypes (it should provide a zip file of the sitemap of the domain's filetypes to download).
I am trying to figure out how to build a ZipFile on Heroku. How do I set the home directory? Then once I have the ZipFile, how do I link to it so it's downloadable?
Here is some of the relavent code so far:
anemone.after_crawl do
puts "Crawl finished. Gathering files, preparing download..."
datasets.each do |url|
u = URI.parse(url.to_s)
Net::HTTP.start(u.host) { |http|
resp = http.get(u.path)
if u.path[0] == "/"
u.path[0] = ''
end
full_path = u.path.split("/")
i = 0
len = full_path.size
filename = full_path[-1]
Zip::ZipFile.open(u.host + ".zip", Zip::ZipFile::CREATE) {
|zipfile|
while i < (len-1) do
directory = full_path[i]
unless File.directory?(directory)
zipfile.mkdir(directory)
end
Dir.chdir directory
i+=1
end
zipfile.add(filename);
while (i > 0) do
Dir.chdir File.expand_path("..",Dir.pwd)
i-=1
end
}
}
end
end
The Heroku filesystem is mostly read-only, but you should be able to temporarily stash the zipfile on /tmp:
Zip::ZipFile.open("#{RAILS_ROOT}/tmp/" + u.host + ".zip", Zip::ZipFile::CREATE)
You'll probably want to use send_file in a "downloads" controller to allow users to download the file. You'll want to build in error handling in case the temporary file disappears before the user downloads it (e.g., if the dyno restarted between zipfile creation and download).
EDIT
The documentation I linked is apparently outdated. RAILS_ROOT is the Rails 2 way to refer to the directory root, but the Rails 3 way (Rails.root) doesn't work either--in Heroku it refers to the ./app folder.
However, you can use the Heroku base filesystem /tmp folder, like this:
Zip::ZipFile.open("/tmp/" + u.host + ".zip", Zip::ZipFile::CREATE)
I have downloaded a set of html files and saved the file paths which I saved them to in a .txt file. It has each path on a new line. I wanted to look at the first file in the list and then itterate through the whole list, opening the files and extracting data before going on to the next file.
My code works fine with a single path put in directly (for the first file) as:
path = r'C:\path\to\file.html'
and works if I itterate through the text file using:
file_list_fp = r'C:\path\to\file_with_pathlist.txt'
with open(file_list_fp, 'r') as file_list:
for filepath in file_list:
pathend = filepath.find('\n')
path = file[:pathend]
q = open(path, 'r').read()
but it fails when I try getting a single path using either:
with open(file_list_fp, 'r') as file_list:
path_n = file_list.readline()
end = path_n.find('\n')
path_bad1 = path_n[:end]
or:
with open(file_list_fp, 'r') as file_list:
path_bad2 = file_list.readline().split('\n')[0]
With these two my code exits just after that point. I can't figure out why. Any pointers very welcome. (I'm using Python 3.3.1 on windows.)