Curl has an option which allows me to specify to which IP a domain should be resolved
e.g. curl --resolve example.com:443:1.2.3.4 https://example.com/foo
to make sure that a very specific server is called
(e.g. when multiple servers have the same vhost with a load balancer usually in front of it and there are multiple applications running on the same port with different vhosts)
How do I set this value when using Ethon? https://github.com/typhoeus/ethon
This is how I'd expect it to work
Ethon::Easy.new(url: "https://example.com/foo", :resolve => "example.com:443:1.2.3.4")
but I'm getting an invalid value exception (I have tried multiple different formats that came to mind)
Ethon::Errors::InvalidValue: The value: example.com:443:1.2.3.4 is invalid for option: resolve.
I took a look at the code but couldn't figure out how I'd have to provide the value - and the documentation on this is a bit scarce
Thanks in advance for any reply that might point me in the right direction
Thanks to i0rek on Github I got the answer I was looking for:
resolve = nil
Ethon::Curl.slist_append(resolve, "example.com:443:1.2.3.4")
e = Ethon::Easy.new(url: "https://example.com/foo", :resolve => resolve)
#=> #<Ethon::Easy:0x007faca2574f30 #url="https://example.com/foo", ...>
Further information can be found here:
https://github.com/typhoeus/ethon/issues/95#event-199961240
Related
I have a collection of Person, stored in a legacy mongodb server (2.4) and accessed with the mongoid gem via the ruby mongodb driver.
If I perform a
Person.where(email: 'some.existing.email#server.tld').first
I get a result (let's assume I store the id in a variable called "the_very_same_id_obtained_above")
If I perform a
Person.find(the_very_same_id_obtained_above)
I got a
Mongoid::Errors::DocumentNotFound
exception
If I use the javascript syntax to perform the query, the result is found
Person.where("this._id == #{the_very_same_id_obtained_above}").first # this works!
I'm currently trying to migrate the data to a newever version. Currently mongodbrestore-ing on amazon documentdb to make tests (mongodb 3.6 compatible) and the issue remains.
One thing I noticed is that those object ids are peculiar:
5ce24b1169902e72c9739ff6 this works anyway
59de48f53137ec054b000004 this requires the trick
The small number of zeroes toward the end of the id seems to be highly correlated with the problem (I have no idea of the reason).
That's the default:
# Raise an error when performing a #find and the document is not found.
# (default: true)
raise_not_found_error: true
Source: https://docs.mongodb.com/mongoid/current/tutorials/mongoid-configuration/#anatomy-of-a-mongoid-config
If this doesn't answer your question, it's very likely the find method is overridden somewhere in your code!
According to README on github, Ruby Whois can be used "as a standalone library to parse WHOIS records fetched previously and/or from different WHOIS clients."
I know how to use the library to directly perform whois query and parse the returning result. But I cannot find anywhere(stackoverflow included) how I can use this library to parse whois data previously fetched ?
I think it's not important but this is how I get my data, anyway: they are fetched through linux whois command and stored in separate files, each file containing one whois query result.
The manual pages on https://whoisrb.org/ are 404. Even the code on the homepage is outdated thus wrong, and the doc pages provide little information.
I tried to scan the source code on github( https://github.com/weppos/whois-parser and https://github.com/weppos/whois). I tried to find the answer on rubydoc ( https://www.rubydoc.info/gems/whois-parser/Whois/Parser, https://www.rubydoc.info/gems/whois/Whois/Record and some related pages). Both failed, partly because this task is the first time and the reason that I use Ruby.
So could anyone help me? I'm really desperate and I'll definitely appreciate any help.
Try it like this,
require 'whois-parser'
domain = 'google.com'
data = 'WHOIS DATA THAT YOU ALREADY HAVE'
whois_server = Whois::Server.guess domain
whois_data = [Whois::Record::Part.new(body: data, host: whois_server.host)]
record = Whois::Record.new(whois_server, whois_data)
parser = record.parser
parser.available? #=> false
parser.registered? #=> true
I just started using the driver org.apache.hive.jdbc.HiveDriver (Version
1.2.1 for spark2) with a Spark Thrift Server (STS) (reference here)
java.sql.ResultSet defines the method absolute() (JavaDoc here)
but HiveBaseResultSet seems to have chosen not to implement the method (source code here)
So now my application (built on top of SmartGWT) was doing a simple operation and I got the following error message:
=== 2017-05-13 18:06:16,980 [3-47] WARN RequestContext - dsRequest.execute() failed:
java.sql.SQLException: Method not supported
at org.apache.hive.jdbc.HiveBaseResultSet.absolute(HiveBaseResultSet.java:70)
at org.apache.commons.dbcp.DelegatingResultSet.absolute(DelegatingResultSet.java:373)
at com.isomorphic.sql.SQLDataSource.executeWindowedSelect(SQLDataSource.java:2970)
at com.isomorphic.sql.SQLDataSource.SQLExecute(SQLDataSource.java:2024)
What is the reason that the driver chose not to implement absolute()?
Are there any workaround for the limitation?
Thanks for the hint from Mark Rotteveel. Now I understand better and let me post an answer to my own question.
Implementation of absolute() is optional
As specified by the Interface of ResultSet#absolute() (link), the implementation for absolute() is optional -- especially when the result set type is TYPE_FORWARD_ONLY.
Workaround
In my case, the result set comes from a Spark Thrift Server (STS) so I guess it is indeed forward-only. So the question became how to instruct my application to NOT making a call to absolute(), which is basically for cursor movement.
SmartGWT-specific answer
For SmartGWT, this is controlled by a property called sqlPaging, which we can specified for an OperationBinding. The right value to use seems to be dropAtServer (more reference here). So I set my SmartGWT DataSource XML file to something like this
<operationBindings>
<operationBinding operationType="fetch" progressiveLoading="false"
sqlPaging="dropAtServer"
>
After that I saw another error, which is now related to HiveConnection#commit():
java.sql.SQLException: Method not supported
at org.apache.hive.jdbc.HiveConnection.commit(HiveConnection.java:742)
at org.apache.commons.dbcp.DelegatingConnection.commit(DelegatingConnection.java:334)
at com.isomorphic.sql.SQLTransaction.commitTransaction(SQLTransaction.java:307)
at com.isomorphic.sql.SQLDataSource.commit(SQLDataSource.java:4673)
After more digging, I realized that the right property for SmartGWT to control the commit behavior is autoJoinTransactions and I should set it to false (more reference here). After these two changes, I could get may application to talk to STS via jdbc.HiveDriver
For anyone out there who are also trying this, here is my full settings for the driver in SmartGWT's server.properties (more reference here)
sql.defaultDatabase: perf2 # this name is picked by me, but it can be anyname
sql.perf2.driver.networkProtocol: tcp
sql.perf2.driver: org.apache.hive.jdbc.HiveDriver # important
sql.perf2.database.type: generic # important
sql.perf2.autoJoinTransactions: false # important
sql.perf2.interface.type: driverManager # important
sql.perf2.driver.url: jdbc:hive2://host:port # important -- pick your host:port
sql.perf2.driver.user: someuser # important -- pick your username
sql.perf2.interface.credentialsInURL: true
sql.perf2.driver.databaseName: someDb
sql.perf2.driver.context:
I'm starting to use whoisrb and I'm noticing domains from some registrars return nil contact information.
For example:
domain_name = ARGV[0]
r = Whois.whois(domain_name)
t=r.registrant_contact
if t == nil
puts 'Registrant Contact is empty.'
end
Will return "Registrant Contact is empty." Trying to access the contact attributes results in an error, like undefined method 'id' for nil:NilClass (NoMethodError).
If I check the raw record that's being returned, puts r, I can see it's getting the thick record, so the contact information is there in the unparsed raw record.
The two registrars I've noticed this for, so far, are onlinenic.com and namesilo.com. If you try to run whois for those two domains, you'll see what I mean.
I'm checking the ICANN compliant sample here:
https://www.icann.org/resources/pages/approved-with-specs-2013-09-17-en#whois
against onlinenic.com and namesilo.com, and I don't see any substantial differences (maybe I'm missing something, though).
Any ideas why it's having trouble parsing these, or pointers on what I could check to fix it? Thanks.
It happens when the registrar has no parser associated, or the parser doesn't have the definition required to parse the contacts.
In other words, unless a parser exists, it's possible that the registrar details are in the response but the library can't find them.
In that case, the solution is to either add/update the parser corresponding to the specific registrar/registry.
Since this behavior is confusing to whoever is not familiar with the internals of the library, also note that the new release 4 will raise an error in this case (instead of silently returning nil). In this way it will be clear when the value is nil vs the value is unknown.
r = Whois.whois(domain_name)
The r here is a Whois::Record object and you can find the available methods here. registrant_contact is not one of them. You probably have to parse it out yourself.
Problem:
Attempting to use the JYTHON command below and I cannot retrieve the id of my active specification defined at a node-server level in Websphere. I believe its a syntax issue but I'm not sure what.
Code:
AdminConfig.getid('/Cell:mycell/Node:mynode/Server:myserver/J2CActivationSpec:myActiveSpecName/')
Problem Notes:
I do not get a invalid object error so I believe I have the syntax right but it just cannot find the resource even though it exists.
I am using the AdminConfig.getid() as a way to check if the resource already exists in order to do a modify or a create.
If I use the following code: AdminConfig.getid('/J2CActivationSpec:myActiveSpecName/') it will find it but not if I use a more specific path listed above.
Reference Material:
IBM Documentation
Containment paths are always a little tricky. In my (limited) experience, even if you can trace the path by AdminConfig.parents, you may not always be able to use getid.
Are you restricted to using getid? If not, here are some alternatives that will get you an ActivationSpec at the /Cell/Node/Server level:
Querying using AdminConfig.list
This approach will list the Activation Specifications at the specified scope (in our case, the server), and grab the one that has it's name attribute equal to 'myActiveSpecName'.
server = AdminConfig.getid('/Cell:mycell/Node:mynode/Server:myserver')
activationSpec = ''
for as in AdminConfig.list('J2CActivationSpec', server).splitlines():
if AdminConfig.showAttribute(as, 'name') == 'myActiveSpecName'
activationSpec = as
print 'found it :)'
Using Wildcards
This approah uses AdminConfig.list as well, but with a pattern to narrow down your list. If you know your activation spec's configuration begins with myActiveSpecName, then you can do the following:
activationSpec = AdminConfig.list('J2CActivationSpec', 'myActiveSpecName*')