I want to take Google Custom Search API results and add them to a database using Datamapper.
I've managed to successfully set up and manually add some items to the database, so it seems like that is all set up correctly.
I'm using HTTParty to make the call the the Google API, which is returning JSON results. I then want to take that JSON and add each link into the database. I'mtrying to use .each as follows;
response["items"].each do |item|
i=DMapperModel.create(city: "London", link: item["link"])
puts i
puts i.saved?
end
Response is a variable holding the HTTParty::response, "items" and "link" are both subsets of the HTTParty::response.
puts i successfully puts the correct DataMapper resource (i.e. <#DMapperModel city: 'London', link: 'example.com'>)
puts i.saved? is a check to see if i saved to the database, at the moment this is returning false...
So it is successfully setting i to the DataMapper resource, but not saving it to the database for some reason, can anyone see where I'm going wrong?
Solved it myself!
I had the links parameter set as a string, which DataMapper sets at a default maximum length of 50 characters. All of the links I was trying to add were longer that 50 characters so the save was failing.
Resolved the problem by setting the maximum length of the link parameter to 2000 characters;
class CityLink
include DataMapper::Resource
property :id, Serial
property :city, String
property :link, String, *length: 2000*
end
I took the length of 2000 characters from this page on the DataMapper docs which is for the URI parameter type and links to this SO question.
Once I'd set this, the above .each method worked a trick.
Related
In the ruby controller, I have two methods in the same controller.
class NotificationsController < ApplicationController
def first
variable_one = xxxx
end
def second
// do something
end
end
I want to use the variable one in the method first, and use it in the method two. I tried to assign the variable one to a session hash. session[:variable_one] = variable_one, and access it in the method two. But it turns out the session[:variable_one] in the method two is nil. These two methods don't have the corresponding views, so I cannot add a link_to and pass parameters. The method one cannot be set as before_action as well.
Could you please have some suggestions on this problem? Thanks so much.
The issue that session is stored via cookie, and therefore it is specific to one device. So, you will have one session between the rails app and your frontend, and another session betweeen the rails app and Twilio (probably the Twilio session will reset between each request). Basically, they're totally separate contexts.
Possibly you could figure out how to pass the information along via Twilio - see https://www.twilio.com/docs/voice/how-share-information-between-your-applications - but as a general-purpose workaround, you could just store the column on the database.
First, make a migration to add the column:
add_column :users, :my_variable, :string
Set this value in the first endpoint:
def first
current_user.update my_variable: "xxxx"
end
Then read it from the second:
def second
# first you would need to load the user, then you can read the value:
my_variable = current_user.my_variable
# you could set the db value to nil here if you wanted
current_user.update my_varible: nil
end
I'm using Anemone to store crawled pages into MongoDB. It mostly works, except for accessing the page headers when I retrieve a page from MongoDB.
When I call collection.find_one("http://stackoverflow.com") I'll get the correct object from the data store, but I can't acecss the headers.
Anemone stores the headers as a hash, so theoretically, after retreiving the document, I should be able to do something like
document["headers"]["content-type"]
but that doesn't work because document["headers"] is a BSON::Binary.
puts document["headers"]
displays a mixture of text and binary characters.
How can I create a usable ruby hash object from the binary data that comes back from MongoDB?
EDIT: I haven't solved the original problem, but was able to modify Anemone so that I can have it load the data for me, which seems to work:
class NewMongo < Anemone::Storage::MongoDB
def initialize(mongo_db, collection_name)
#db = mongo_db
#collection = #db[collection_name]
#Do not delete the collection! I need it!
##collection.remove
#collection.create_index 'url'
end
end
And then later on...
repo = NewMongo.new(db, "pages")
repo.each db |url, page|
puts page.content_type
end
If the data was stored in a Binary format by the Anemone storage backend there isn't much you can do unless you know the format or there is a deserializer they provide. It sounds like that would be a bad choice for storing the header as the hash would be a more natural form for it.
I'm automating a site that has a page with a list of options selected by a radio button. When selecting one of the radios, a text field and a select list are presented.
I created a file (test_contracting.rb) that is the one through which I execute the test (ruby test_contracting.rb) and some other classes to represent my page.
On my class ContractPage, I have the following element declaration:
checkbox(:option_sub_domain, :id => "option_sub_domain")
text_field(:domain, :id => "domain_text")
select_list(:tld, :id => "domain_tld")
I've created in the ContractPage a method that sets the configuration of the domain like this:
def configure_domain(config={})
check_option_sub_domain
domain = config[:domain]
tld = config[:tld]
end
When I call the method configure_domain from the test_contracting.rb, it selects the radio button, but it doesn't fill the field with the values. The params are getting into the method correctly. I've checked it using "puts". Even if I change the params to a general string like "bla" it doesnt work. The annoying point is that if on test_contracting.rb I call the exact same components, it works.
my_page_instance = ContractPage.new(browser)
my_page_instance.domain = "bla"
my_page_instance.tld = ".com"
What I found to work was to in the configure_domain method, implement the following:
domain_element.value = config[:domain]
tld_element.send_keys config[:locaweb_domain]
Then it worked.
The documentation for the PageObjects module that I'm using as reference can be found here: http://rubydoc.info/github/cheezy/page-object/master/PageObject/Accessors#select_list-instance_method
Do you guys have any explation on why the method auto generated by the pageobject to set the value of the object didnt work in this scope/context ?
By the way, a friend tried the same thing with Java and it failed as well.
In ruby all equals methods (methods that end with the = sign) need to have a receiver. Let me show you some code that will demonstrate why. Here is the code that sets a local variable to a value:
domain = "blah"
and here is the code that calls the domain= method:
domain = "blah"
In order for ruby to know that you are calling a method instead of setting a local variable you need to add a receiver. Simply change your method above to this and it will work:
def configure_domain(config={})
check_option_sub_domain
self.domain = config[:domain]
self.tld = config[:tld]
end
I'm pretty new to this world of Selenium and page objects but maybe one of my very recent discoveries might help you.
I found that that assignment methods for the select_list fields only worked for me once I started using "self" in front. This is what I have used to access it within my page object code. e.g., self.my_select_list="my select list value"
Another note - The send_keys workaround you mention is clever and might do the trick for a number of uses, but in my case the select list values are variable and may have several options starting with the same letter.
I hope something in here is useful to you.
UPDATE (Jan 3/12)
On diving further into the actual Ruby code for the page object I discovered that the select_list set is also using send_keys, so in actuality I still have the same limitation here as the one I noted using the send_keys workaround directly. sigh So much to learn, so little time!
I've to add several fields in a mongoid model, I know there is not migration with MongoDB but if I go on without dropping the DB, making rails to "regenerate" the DB entirely, it doesn't display or use the new fields at all !
What's the best way to go here ? Is there something softer than drop/reopen mongodb ?
Thanks in advance
luca
In general it should be possible to update old documents with the new fields at runtime. There is no need for migrations in MongoDB.
You maybe want to write rake tasks to update your old documents with the new fields and default values.
You could find out these documents by checking those new fields which have per default a nil value.
Update
Easy style:
If you define a new field with a default value, this value should always be used as long as you set a new one:
app/models/my_model.rb
class MyModel
include Mongoid::Document
field :name, type: String
field :data, type: String
# NEW FIELD
field :note, type: String, default: "no note given so far!"
end
If you query your database you should get your default value for documents which haven't this field before your extension:
(rails console)
MyModel.first
#=> #<MyModel …other fields…, note: "no note given so far!">
I tested this with a fresh rails stack with a current mongoid on Ruby 1.9.2 - should work with other stacks, too.
More complicated/complex style:
If you didn't set a default value, you'll get nil for this new field.
app/models/my_model.rb
class MyModel
include Mongoid::Document
field :name, type: String
field :data, type: String
# NEW FIELD
field :note, type: String
end
(rails console)
MyModel.first
#=> #<MyModel …other fields…, note: nil>
Then you could set up a rake task and migration file like in this example:
lib/tasks/my_model_migration.rake:
namespace :mymodel do
desc "MyModel migration task"
task :migrate => :environment do
require "./db/migrate.rb"
end
end
db/migrate.rb:
olds = MyModel.where(note: nil)
# Enumerator of documents without a valid :note field (= nil)
olds.each do |doc|
doc.note = "(migration) no note given yet"
# or whatever your desired default value should be
doc.save! rescue puts "Could not modify doc #{doc.id}/#{doc.name}"
# the rescue is only a failsafe statement if something goes wrong
end
Run this migration with rake mymodel:migrate.
This is only a starting point and you can extend this to a full mongoid migration engine.
The task :migrate => :environment do … is necessary, otherwise rake won't load models.
It is a little ridiculous to say that you don't need migrations with mongodb or mongoid. Any sophisticated app needs to be refactored from time to time and that can mean pulling fields out of disparate documents into a new one.
Writing one off rake tasks is way less convenient and error prone than having migrations be part of your deploy script so that it always gets run on every environment.
https://github.com/adacosta/mongoid_rails_migrations brings AR style migrations to mongoid.
You might need them less often, but you will certainly need them as an app grows.
Below is a nice code example for data migration script with mongoid and the ruby mongo driver - to be used when your updated model no longer match production data.
http://pivotallabs.com/users/lee/blog/articles/1548-mongoid-migrations-using-the-mongo-driver
I whish we would stop using "no migrations with mongoid" as slogan. It'll turn people to MongoDB for the wrong reasons, and it's only partially true. No schema, true, but data still needs to be maintained, which IMO is harder with MongoDB than RDBMs. There are other, great reasons for choosing MongoDB and it depends on your problem.
So I need to access this service from my rails app. I'm using soap4r to read the WSDL and dynamically generate methods for accessing the service.
From what I've read, I should be able to chain methods to access the nested XML nodes, but I can't get it to work. I tried using the wsdl2ruby command and read through the generated code. From what I can tell, the soap library is not generating these accessor methods. I'm pretty new to ruby, so I don't know if I'm just missing something?
I know when I inspect the element, I can see the data I want. I just can't get to it.
For instance if I use the following code:
require "soap/wsdlDriver"
wsdl = "http://frontdoor.ctn5.org/CablecastWS/CablecastWS.asmx?WSDL"
driver = SOAP::WSDLDriverFactory.new(wsdl).create_rpc_driver
response = driver.getChannels('nill')
puts response.inspect
I get the following output:
ignored element: {http://schemas.xmlsoap.org/wsdl/soap12/}binding
ignored element: {http://schemas.xmlsoap.org/wsdl/soap12/}operation
ignored element: {http://schemas.xmlsoap.org/wsdl/soap12/}body
ignored element: {http://schemas.xmlsoap.org/wsdl/soap12/}address
#<SOAP::Mapping::Object:0x80b96394 {http://www.trms.com/CablecastWS/}GetChannelsResult=#<SOAP::Mapping::Object:0x80b96178 {http://www.trms.com/CablecastWS/}Channel=[#<SOAP::Mapping::Object:0x80b95f5c {http://www.trms.com/CablecastWS/}ChannelID="1" {http://www.trms.com/CablecastWS/}Name="CTN 5">, #<SOAP::Mapping::Object:0x80b9519c {http://www.trms.com/CablecastWS/}ChannelID="2" {http://www.trms.com/CablecastWS/}Name="PPAC 2">, #<SOAP::Mapping::Object:0x80b94620 {http://www.trms.com/CablecastWS/}ChannelID="14" {http://www.trms.com/CablecastWS/}Name="Test Channel">]>>
So the data is definitely there!
Here is the code generated by wsdl2ruby for the method being used above:
# {http://www.trms.com/CablecastWS/}GetChannels
class GetChannels
def initialize
end
end
# {http://www.trms.com/CablecastWS/}GetChannelsResponse
# getChannelsResult - ArrayOfChannel
class GetChannelsResponse
attr_accessor :getChannelsResult
def initialize(getChannelsResult = nil)
#getChannelsResult = getChannelsResult
end
end
Sorry for the long post, I figured the more info the more likely someone can point me in the right direction.
Thanks
-ray
Answer
require "soap/wsdlDriver"
wsdl = "http://frontdoor.ctn5.org/CablecastWS/CablecastWS.asmx?WSDL"
driver = SOAP::WSDLDriverFactory.new(wsdl).create_rpc_driver
response = driver.getChannels('nill')
for item in response.getChannelsResult.channel
puts item.name
puts item.channelID
end
How I got the Answer
You can figure out the methods of response via
response.methods
This will get you a long list of methods that are hard to sort through, so I like to subtract out the generic methods. Ruby lets you subtract arrays.
response.methods - Object.new.methods
Using this technique, I found the getChannelsResult method for response. I repeated the process
resonse.getChannelsResult.methods - Object.new.methods
I found the channel method for its result. Again!
response.getChannelsResult.channel.methods - Object.new.methods
This returned a bunch of methods including: sort, min, max etc. So I guessed Array. A simple confirmation was in order
response.getChannelsResult.channel.class
Sure enough it returned Array. To make life simple, I just worked with the first item of the array to get its methods
response.getChannelsResult.channel.first.methods - Object.new.methods
Whoalla, I found two more methods "name" and "channelID"