My aim is to have a self-contained repository that a user/developer can pull down (on a mac), type docker-compose up -d and have a working development environment via a friendly url like http://myproject.dev/
I have my docker images set up as needed but the local domain is where i've come unstuck. I know this is a bit outside of Docker, as this is a host system thing. But i'm really looking for a way to achieve this without requiring the user to install local apps or make various local system config changes.
Is this something that is achievable, or am i barking up the wrong tree?
**Sorry guys, i missed out some important info. I ideally want to avoid pointing to localhost as this would cause conflicts if/when running multiple projects. So i guess we would need to point to the containers IP, so host entries would need to be dynamic. murky dns waters indeed
I know you said:
... or make various local system config changes.
But if you can lift that requirement, the one change you need to make for local macOS systems would be an addition to /etc/resolver/dev. Then, you can run a DNS server to your docker-compose.yml that automatically adds entries for the services such as https://github.com/ruudud/devdns (specifically, the https://github.com/ruudud/devdns#host-machine--containers bit).
Point a real world DNS record that all your developers can lookup to 127.0.0.1. So myproject.existing.tld would point to 127.0.0.1.
An example is vmware's could application platform:
→ dig +noall +answer vcap.me
vcap.me. 3412 IN A 127.0.0.1
→ dig +noall +answer whatever.vcap.me
vcap.me. 3412 IN A 127.0.0.1
Otherwise your treading into the murky waters of advertising services via
mDNS (or LLMNR) and Zeroconf.
Despite ElasticSearch and Kibana both running on my production server, I'm unable to visit the GUI over the public IP: http://52.4.153.19:5601/
Localhost curls return 200 but console errors on the browser report timeouts after a few images are retrieved.
I've successfully installed, run, and accessed Kibana on my local (Windows 10) and on my staging AWS EC2 Ubuntu 14.04 environment. I'm able to access both over port 5601 on localhost and the staging environment is accessible over the public IP address and all domains addressed accordingly. The reverse proxy also works and all status indicators are green on the dashboard.
I'm running Kibana 4.5, ElasticSearch 2.3.1, Apache 2.4.12
I've used the same exact volume from the working environment to attach to the production instance, so everything is identical on the two volumes, except that the staging environment's apache vhost uses a subdomain while the production environment's servername is the base domain. Both are configured for SSL wildcards. Both are in separate availability zones at Amazon. I've tried altering the server block to use a subdomain on the production server, just to see if the domain was impactful but the error remains.
I also tried running one instance individually, in case EC2 had some kind of networking error with 0.0.0.0 but I'm unable to come to a resolution. All logs and configurations are identical between the two servers for ElasticSearch and Kibana.
I've tried deleting and re-creating the kibana index, tried alternate settings inclusive of the host, elasticsearch url, extending the max ping and timeout, max retries, extended the apache limits, http.cors to allow different origins. I've tried other ports but both servers are indicating that 5601 is listening in the same way.
I also had the same problem on a completely different volume that was previously attached to this instance.
The only difference I can see is that the working version pings fine while the non-working version has a 100% packet loss when pinging the IP, although I can't imagine why that would be, as I'm able to reach the website on 80, just fine. I can also access various other tools running on other ports. I assume there might be some kind of networking conflict. Any ideas?
May be port 5601 is blocked by firewall
Allow incoming connections to port 5601 by:
sudo iptables -I INPUT -p tcp --dport 5601 -j ACCESS
For security:
Modify above mentioned command and accept connection only from specific address. (See man iptables)
or use Shield plugin for elasticseach
Sorry, forgot to update this question. The answer turned out being that I simply needed to deploy a new instance. Simply by creating a clone of the instance, I was able to resolve the issue. I've had networking problems at AWS, before, with their internal dns/ip conflicts, so I've had to do so, in the past and this turned out to be the quickest and cleanest solution, albeit not providing any definitive insight into the cause.
I have a machine running lighttpd to allow me to do a small scale web-based project. Up until now I've been accessing the project by using the private IP of the machine running lighttpd. The problem I have is that if I disconnect the server and attempt to run my code to test for all cases is realize that the site hangs (after an AJAX call) instead of returning a 404 error because of the direct IP.
Besides for ensuring that the server never goes offline, how can I make sure that the website won't hang up. Is there a way for me to look up the private IP but going through a DNS Lookup to ensure that if it isn't there I don't try to load it anyways.
If it is a unix-compatible resolver (Linux, *BSD etc.), just add an entry to your private IP in into /etc/hosts, following the same format of the other entries of the file. Like this:
10.0.1.6 my_private_host_name
That solution will only work individually on the machine you changed the /etc/hosts file.
Your second option is to create a private-DNS-server and add the your.domain.internal domain to that DNS, and enable DNS forwarding to your internal network, and set your clients to use that DNS server. This way if you try to call your private domain, your DNS server will be SOA (Start of Authority) and answer for it, but if you ask for something that it isn't SOA for it will just pass your request to the old DNS server, that will answer it the usual way.
Now if you need to use a "real" DNS name for your private IP (I want it it to answer on www.my_company.com), the first /etc/hosts method will do the trick,but if you need to set up a DNS server, follow the tutorials on "Split DNS".
I've got XAMPP setup on my laptop (OSX 10.6) for dev, and I wanted to use VirtualDocumentRoot so that I could do *.localhost and it would automap to the folder under my sites directory. I've got this all set up fine, and it works great, but when I got to work today, I found an issue with the way our LAN handles DNS.
Long story short, instead of checking the LAN DNS server for local domains, it goes out to the root. Is there a way to get bind to check the DHCP supplied DNS server for addresses it's not responsible for? Or alternatively, is there a way to get my os to use the DHCP DNS server first, and then fall back to the local with minimal performance hit?
Thanks!
I'm using Linux Arch, but as MacOSX is based on some *nix system - may this ideas helps you:
Take a look at the file /etc/resolv.conf. In my setup this file is automatically generated by NetworkManager.
This document writes about ways to update /etc/resolv.conf when dhcpcd, NetworkManager or dhclient is used: https://wiki.archlinux.org/index.php/Dnsmasq#DHCP_Setup
In this way you do just prepend the local dns before the dhcp's dns (or static if you're switching to a static configuration). Make sure you remove all forwarders from your dns-server.
If macos does not use them, may this workaround gives you a hint, even if it's very limited:
Add a global name-server (like google's one 8.8.8.8) to your dns-server's list of forwarders.
Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 9 years ago.
Improve this question
After upgrading to Mac OSX Lion I figured out that /etc/hosts is not looked up in first place for name resolution anymore. This leads to some side effects like:
Entries in /etc/hosts are resolved painfully slow
You can't not override existing domains, e.g. 127.0.0.1 www.google.com
If you get search domain entries from DHCP, let say .lan, and some funny guy configured localhost.lan to something else then 127.0.0.1 in the local DNS you can not reach your localhost anymore.
Is this behavior intended? Does it make any sense? And most important, how can I come back to the old behavior.
I think he matter is Lion handles .local TLD differently because it's reserved for some Multicast DNS features (used by Bonjour). The only way i found to solve this issue is using a different TLD for development hosts (ie: .dev). It works fine for me, hope it's gonna be helpful to others!
With regards to overriding domains in the hosts file, I have found that in some circumstances, Lion queries the IPv6 address for a domain if it senses that a domain is unreachable over the IPv4 network.
I discovered this when I noticed some ads that I had never seen before on Snow Leopard because I had redirected the ad domains to 127.0.0.1. I fired up wireshark and noticed AAAA (IPv6 DNS records) queries following the IPv4 A queries (IPv4). The ad servers indeed have IPv6 addesses and were able to serve me their content.
The solution to this is have a
::1 mydomain.com
entry for every
127.0.0.1 mydomain.com
entry in your hosts file.
Interestingly, if you happen to have a local webserver running on 127.0.0.1:80 and your browser receives a response from the webserver (error or otherwise), no AAAA query is issued, as it seems to be satisfied that a TCP connection was at least possible.
On a related note, if you make heavy use of the hosts file (for adblocking, local web development, etc), you may want to look into running your own local DNS resolver. There is a considerable disk/CPU hit from having to read /etc/hosts on every request, so it is in your best interest to keep that file very light.
One advantage of running something like dnsmasq locally (besides the significant performance boost) is that you can redirect whole top-level domains back to your local machine. This allows you to have the whole *.dev namespace for development (for instance), without having to individually enter every domain you want resolved locally into /etc/hosts
The problem was that I symlinked the /etc/hosts file. If /etc/hosts is a plain file everything is ok.
Update(2): OSX 10.10.5 brings the return of mDNSResponder.
Update: OSX 10.10 Yosemite has replaced mDNSResponder with "discoveryd". I've not upgraded so I am not sure of the discoveryd behavior w/r/t DNS lookups and /etc/hosts.
The system DNS resolver on Lion is the mDNSResponder process.
You may be thinking "but mDNSResponder is the multicast dns responder." You're right; that's what it originally was for, and it still fulfills this function. However, on newer MacOS versions it also does standard host lookups.
In Lion, it does not appear to automatically re-read /etc/hosts when it changes, at least not always. Killing mDNSResponder (and allowing it to be automatically restarted) seems to fix the problem.
sudo killall mDNSResponder
should do the trick.
below is my original answer for posterity. I suppose it might still be an issue in some cases.
Make sure your /etc/hosts file is a unix style text file, with linefeeds as the ending rather than cr's.
Editing with TextWrangler or a unix text editor should preserve the file.
If your file is already messed up, try this to fix
tr '\015' '\012' < /etc/hosts > /tmp/hosts.$$
mv /etc/hosts /etc/hosts.bad
mv /tmp/hosts.$$ /etc/hosts
# fix up permissions while we are at it
chown root:wheel /etc/hosts
chmod 644 /etc/hosts
credit for this fix to:
http://techpatio.com/2011/guides-how-to/fixed-mac-osx-lion-etc-hosts-bugs-dns
ive had this issue for a while, as im working a team of devs it became necessary to actually use .local rather then .dev or .localhost, i found this article to be very useful.
iTand.me - Lion local domains and etc hosts..
In summary;
But if you have to use .local, the most elegant solution I've found is the dscl utility. Using it is very straightforward. To add a host called mydev.local and point it to the localhost, just do this:
sudo dscl localhost -create /Local/Default/Hosts/mydev.local IPAddress 127.0.0.1
To see all the currently defined hosts and their IPs
sudo dscl localhost -list /Local/Default/Hosts IPAddress
And to remove a host:
sudo dscl localhost -delete /Local/Default/Hosts/mydev.local
Overall, pretty straightforward and works well. I still would prefer to be able to edit /etc/hosts instead, but this is a better alternative to having to rename all our .local servers.
Prior to moving from Snow Leopard to Lion, I had several app-specific entries in /etc/hosts, like this:
127.0.0.1 foo.bar.local
After the update, loading my local apps was VERY slow. I noticed that the delay happened before the request showed up in the log file, and that once it did, the app itself was as fast as usual.
Now I have two lines per app, like this:
127.0.0.1 foo.bar.local
::1 foo.bar.local
... and everything is fast again.
Apparently this adds IPv6 addresses? I don't quite get it, really, but it works.
My situation was similar, but the delays, of exactly 5 seconds, only happened for URLs ending with '.local'. When viewing sites that ended in '.dev', there was no delay.
Some of the other developers in my office had this problem, while a few did not. I was hoping for a simple fix and I did not want to rename the site to '.local' due to other dependencies.
I ran the following command in Terminal and diffed my output with a few other users in the office.
scutil --dns
This section was the only difference:
resolver #2
domain : 00000000.members.btmm.icloud.com
options : pdns
timeout : 5
order : 150000
My Mac was linked to my iCloud account and I had Back To My Mac enabled. Once I disabled Back To My Mac, the additional resolver went away and the 5 second delay disappeared.
Wow, what a nightmare. I have read absolutely everything on this subject and everything that has been suggested so far was tantilizingly close to what I was experiencing, but none of the solutions worked for me.
And I figured out why.
Unlike others, I was not using /etc/hosts to set up local domains. My /etc/hosts file was stock, containing only the entries needed for the loopback interface and the broadcast host. Moreover, it was a correctly-encoded unix file, as I'm the sort of person who would only edit that from the command-line using emacs. And, thank goodness, I did not have to resort to running my own DNS server like DNSmasq to get around the problem.
(To be clear, the symptom that brought me here to this issue was that emacs took about 10 seconds to start, but only when I was on wifi. If I turned off wifi, emacs would start up instantly as expected.)
My solution: my laptop has a name, "terminator". (Yes, its shiny aluminum exterior made me think of the Arnold Schwarzenegger character.) I just needed to add entries to /etc/hosts for the name of the machine itself:
127.0.0.1 terminator
::1 terminator
I found the name of my host by running a simple command in the terminal:
hostname
...which came back with the output: "terminator". After changing /etc/hosts to contain those two entries, emacs can now quickly resolve my laptop's name.
I hope this helps someone.
I've had speed issues using OSX Lion as a web development box ... Using a combination of suggestions I resorted to disabling ipv6 networking and routing ipv6 to localhost6 ... things sped up quite a bit ...
sudo networksetup -setv6off Ethernet
/etc/hosts ...
127.0.0.1 localhost
127.0.0.1 dev.aliasdomain.com
...
::1 localhost6
I think there's been some bug fixes. I've seen a lot of problems mentioned, and none of these seem to apply currently (for example, putting multiple aliases on a single line now works fine for me).
At any rate it seems that with Lion, Apple made some drastic changes to mDNSResponder which handles all the DNS lookups, and (with Lion at least) also handles /etc/hosts cacheing. For me forward lookups also now work. But reverse lookups (e.g. looking up 1.2.3.4 instead of google.com) don't work.
After a lot of pain, it looks like mDNSResponder converts this lookup to 4.3.2.1.in-addr.arpa and does a name lookup. This may well be how DNS prefers to operate, but it doesn't work at all with /etc/hosts.
Unless of course you add an alias of 4.3.2.1.in-addr.arpa for each host, where 4.3.2.1 is the ip address in the opposite order from which you are used to seeing it. This fixes everything for me. Here's an example /etc/hosts entry:
1.2.3.4 foo foo.example.com alias.example.com 4.3.2.1.in-addr.arpa