I want to resolve the DNS requests issued from within a Ruby script through a DNS server, different from the ones in resolv.conf. While I could do that manualy by using Resolv::DNS or something like that, I'd like to do that for all the requests (like the ones issued by RestClient, for example). Any ideas?
RestClient uses net/http and uses the host name part of the provided URL to open a TCP socket:
https://github.com/ruby/ruby/blob/trunk/lib/net/http.rb?source=cc#L879
The simplest way to change which host is accessed is to manually change the URL to use an IP address by performing the lookup yourself.
Alternatively, you can replace the resolver of the various *Socket classes, and there is actually an example of how to do this here: https://github.com/ruby/ruby/blob/4c2304f0004e9f1784540f3d36976aad9eab1f68/lib/resolv-replace.rb
Related
In my current setup I have two interfaces, one is my normal connection and the other for my VPN, which is the default interface.
My VPN isn't nearly as fast as my connection so for some heavier load I use my normal connection. The problem I am having is that I want to send files to Google Drive by using their api via CMD, but googleapis.com simply resolves to too many IP's to route them all. Currently I am solving this issue by connecting through a proxy so I only have to route that proxy to my normal interface.
What I would rather do is just set the interface of my normal connection to the CMD session like I do with the proxy. I have tried playing around with netsh but so far with no success.
Does anyone have an idea on how to solve this problem?
It isn;t that your VPN is your "Default Interface" it's that it has a Lower Route Metric for Traffic because whoever set it up made it so that all traffic routes t the VPN instead of only internal traffic.
If this is intentional, and you don;t want to change it then your only real option is to use a Proxy as you need to be able to sniff the packet headers to match the API URL, and it's it's HTTPS traffic, that is encrypted, and only passing through a proxy will be able to get around that.
i am using the TIdHTTP component on a multi-device app (building in Rad Studio Tokyo 10.2.3). All i'm doing is downloading a file to my local app folder (iOS). I want to make sure it works with IPv6 but i don't see the "IPVersion" property for TIdHTTP. I see it on other indy components in rad studio (e.g. IdFTP).
Is there a way to set IPVersion in code for the TIdHTTP component? Below is a snip of the code i'm using to download the file. If it fails on IPv4 it is supposed to try IPv6 next:
UnicodeString LFileName = System::Ioutils::TPath::Combine(System::Ioutils::TPath::GetDocumentsPath(), "myfile.txt");
TFileStream* fs = new TFileStream(LFileName, fmCreate);
Form1->TIdHTTP->ConnectTimeout = 8000; // give it 8 seconds
Form1->TIdHTTP->ReadTimeout = 8000;
try
{
UnicodeString URL = "http://myservername.com/myfile.txt";
Form1->TIdHTTP->Get(URL, fs);
Form1->TIdHTTP->Disconnect();
// ShowMessage("Good download via IPv4");
}
catch(const System::Sysutils::Exception &)
{
try
{
Form1->TIdHTTP->Disconnect(); // clean up from IPv4 try
UnicodeString URL = "http://[myservername.com]/myfile.txt";
Form1->TIdHTTP->Get(URL, fs);
Form1->TIdHTTP->Disconnect();
// ShowMessage("Good download via IPv6");
Righ now i just put brackets around the domain name in hopes that this would work for IPv6...i won't know for sure until i can get a truly IPv6 only network setup (working on it).
UPDATE: I just had an app accepted to Apple Store that uses this approach so obviously it passed IPv6 testing. FYI
I found this discussion in which Remy Lebeau states:
TIdHTTP parses IP version info only from the URL itself, so if you need to send a request to an IPv6 target (IP address or hostname), you have to tell TIdHTTP to use IPv6 by wrapping the hostname portion of the URL in square brackets, eg:
s := idHTTP.Post('http://[IPv6 address or hostname]/resource', ...);
Otherwise TIdHTTP uses IPv4 instead.
This bracketed syntax is a requirement when using IPv6 address literals in
URLs, per RFCs 2732 and 3986. But when using a hostname instead, there is
no such syntax requirement. Connecting to a hostname requires a DNS lookup
to discover the host's available IP addresses and their respective IP versions, and then connect to those addresses as needed until one succeeds.
Indy does not currently implement that kind of connection model. It first
allocates a socket based on the client's IPVersion property, THEN performs
a DNS request suited for that IPVersion, and then finally connects to the
first IP address reported by DNS. This is a known limitation of Indy, which
would require a redesign of Indy's internal logic to fix. However, it can
be worked around manually in most Indy client components by performing
your own DNS lookup ahead of time and then looping through the results, setting Indy's Host and IPVersion properties and calling Connect() on each loop iteration.
However, TIdHTTP is a special case, because it derives the Host and IPVersion values from the requested URL, overriding anything the calling code might set manually. However, it is also because of this model that would potentially allow TIdHTTP to be updated to perform its own DNS loop internally without having to rewrite all of Indy's client components to match. Except that Indy does not currently implement a function that returns all of the IP addresses of a hostname, only the first IP address of a particular version. Until such a function is added to Indy in a cross-platform manner, TIdHTTP cannot be updated to auto-detect IPv4/IPv6 hosts.
As such, this confirms my approach should work (trying IPv4 first and then IPv6 if that fails) using the brackets.
Also, I edited my code to show a "Disconnect" prior to trying the IPv6 in case the IPv4 left a mess.
Is it possible to route dns to different servers based on the protocol of the request without using a proxy server?
For example wss://example.com goes to 1 server and https://example.com goes to a different one.
In principle no, this is not possible. Although there are exceptions.
When you an application and request a network address, e.g. example.com the application will (typically) pass on a request to the OS to open a connection to that address.
Because the OS can only make a connection to an IP Address, the first thing it does is to make a DNS request to find out the address it can connect to. There is no mechanism at all, to tell that DNS request what protocol is being requested. At the point the OS makes the DNS request it is simply, 'What is the address for example.com' there is no space in the body of the message for more information.
In that sense it is not possible.
However there are a few services that use SRV records to find the server they need to connect to.
In these cases the application will say to the OS 'get me the SRV record for _somefancyservice.example.com' When it has that record, it will then send another request to the OS to open a connection the whatever it found in the SRV record.
So you could, in theory, write an application that performed the SRV record query first, and then opened a connection to whatever was returned.
I wrote a more detailed answer specifically about wss, which you may find interesting.
We are trying to create a simulation script where we need to send TCP packet data to the server in way that it appears to be coming from different IP every time.
Basically we need to emulate multiple devices ( with different IP) which are constantly sending data to the server.
The server creates a new connection only for request coming in from a new IP.
What is the best possible way to achieve it ? Is there a way of using proxy servers or some sort of virtualization to accomplish this ?
What you want to use is IP aliasing. This allows you to create virtual network interfaces. Each virtual interface can have one or more IP addresses assigned to it.
This link shows how to do it in Linux.
This link shows how to do it in Windows.
Next your clients need to specify which of your addresses to use. Use getifaddrs() to enumerate the available addresses. Then use the bind() system call on the socket before you do a connect(). This way you can have multiple clients and each one will use a different source IP address. This post has the details.
WinINet library caches IP address for any URL accessed over it. Because of this, when IP address for that URL gets changed then also WinInet library's HttpSendRequest goes to older IP address.
And, if older IP is responding, then WinINet will send all http request to older IP only.
Is there any way to force clean DNS cache of WinInet?
Or
Is there any way to force WinINet to send HTTP request to specified IP address (as we are able to get newer IP using gethostbyname())?
Note :
gethostbyname is giving me newer IP address, So this behaviour is happening of WinINet's caching.
I have tried "method 2" suggested in this MS article, but it didn't help
Sample code
You can try to use the flag when connecting:
INTERNET_FLAG_DONT_CACHE = 0x04000000 Does not add the returned entity
to the cache. This is identical to the preferred value,
INTERNET_FLAG_NO_CACHE_WRITE.
Or you can take a look at the DeleteUrlCacheEntry from the WinInet documentation here
I beleave this should do the trick.
--UPDATE
From this doc I've seen that there is an better flag to use it, look at the:
INTERNET_FLAG_PRAGMA_NOCACHE
Forces the request to be resolved by the origin server, even if a cached copy exists on the proxy.
--UPDATE
As tested by #Pradeep you can change this registry keys to work it:
HKCU\Software\Microsoft\Windows\CurrentVersion\Internet Settings\DnsCacheEnabled
ServerInfoTimeOut and DnsCacheTimeout to 0.