I'm trying to use internal service DNS for service-to-service HTTP communication.
If I try to curl a pod from another deployment pod it is working but unable to use it in golang net/http service
2023/01/27 15:48:37 oauth2.go:90: oauth2 url http://hydra-admin.microservices.svc.cluster.local:4445/oauth2/auth/requests/login/accept
2023/01/27 15:48:37 oauth2.go:101: Unable to make http request Put "http://localhost:4445/admin/oauth2/auth/requests/login/accept?login_challenge=b569006c8b834a298cf1cd72e2424953": dial tcp [::1]:4445: connect: connection refused
hydra-admin.microservices.svc.cluster.local is resolved to localhost when the API call is made
but curl works as you see below
/ # curl -X PUT http://hydra-admin:4445/admin/oauth2/auth/requests/login/accept?login_challenge=6f51146e49c54b739de8a37b25a72349
{"error":"invalid_request","error_description":"The request is missing a required parameter, includes an invalid parameter value, includes a parameter more than once, or is otherwise malformed. Unable to decode body because: EOF"}
What am I missing here?
Per my comment, depending on how you are building your go executable will have an effect on how it behaves within a k8s environment. Are you using a scratch image or a CGO_ENABLED=1 image?
From the dns package docs there's a caveat on DNS behavior
By default the pure Go resolver is used, because a blocked DNS request
consumes only a goroutine, while a blocked C call consumes an
operating system thread. When cgo is available, the cgo-based resolver
is used instead under a variety of conditions:
... when /etc/resolv.conf or /etc/nsswitch.conf specify the use of features
that the Go resolver does not implement, and when the name being
looked up ends in .local or is an mDNS name.
So I would suggest - to maximized your success rate for both external & internal DNS requests - building your go executable for k8s like so:
CGO_ENABLED=1 go build -tags netgo
Related
I have a custom API running on http://127.0.0.1:8080 and I have my own chainlink node running on http://127.0.0.1:6688. I get the error saying "Connections to local/private and multicast networks are disabled by default for security reasons: disallowed IP" when sending requests.
I guess maybe we can enable it by modifying the env file, but I don't know which configuration should I change. Does anyone know if we can enable these local/private connections? and how to do that?
Per the chainlink docs on the http task:
allowUnrestrictedNetworkAccess (optional): permits the task to access a URL at localhost, which could present a security risk. Note that Bridge tasks allow this by default.
ie:
my_http_task [type="http"
method=PUT
url="http://chain.link"
requestData="{\\"foo\\": $(foo), \\"bar\\": $(bar), \\"jobID\\": 123}"
allowUnrestrictedNetworkAccess=true
]
You can also use this flag in JSON
I discovered OpenSIPS and all the possibilities a few days ago. I would just use it as a simple SIP proxy to get started. Proxy between my designated UAC and my UAS (asterisk, not natted). The goal is to use a proxy to prevent bot attacks on my UAS.
After installing OpenSIPS, I tried to configure my XLITE (natted) by simply adding the proxy URL in the configuration. It works, I register and I can see in my UAS peers my extensions with proxy IP. But when I make a call, I got a forbidden error. In debug mode, the log does not talk to me, I see a lot of information but nothing about this error.
I did not make any changes to the default configuration script. Is this behavior normal?
I also tried with VM on public IP as UAC (so not named), same thing.
Thank you for your help.
Olivier
Most likely, your SIP INVITE is hitting this block:
if (!is_myself("$rd")) {
send_reply("403", "Relay Forbidden");
exit;
}
What this means is that your OpenSIPS does not consider itself responsible for the domain (or IP) that your SIP UA has placed in the Request-URI and is trying to route towards. To fix this, just whitelist the Asterisk IP as a local (recognized) domain using the alias statement:
listen = udp:*:5060
alias = 1.1.1.1
For those of you running Go backends in production:
What is your stack / configuration for running a Go web application?
I haven't seen much on this topic besides people using the standard library net/http package to keep a server running. I read using Nginx to pass requests to a Go server - nginx with Go
This seems a little fragile to me. For instance, the server would not automatically restart if the machine was restarted (without additional configuration scripts).
Is there a more solid production setup?
An aside about my intent - I'm planning out a Go powered REST backend server for my next project and want to make sure Go is going to be viable for launching the project live before I invest too much into it.
Go programs can listen on port 80 and serve HTTP requests directly. Instead, you may want to use a reverse proxy in front of your Go program, so that it listens on port 80 and and connects to your program on port, say, 4000. There are many reason for doing the latter: not having to run your Go program as root, serving other websites/services on the same host, SSL termination, load balancing, logging, etc.
I use HAProxy in front. Any reverse proxy could work. Nginx is also a great option (much more popular than HAProxy and capable of doing more).
HAProxy is very easy to configure if you read its documentation (HTML version). My whole haproxy.cfg file for one of my Go projects follows, in case you need a starting pont.
global
log 127.0.0.1 local0
maxconn 10000
user haproxy
group haproxy
daemon
defaults
log global
mode http
option httplog
option dontlognull
retries 3
timeout connect 5000
timeout client 50000
timeout server 50000
frontend http
bind :80
acl is_stats hdr(host) -i hastats.myapp.com
use_backend stats if is_stats
default_backend myapp
capture request header Host len 20
capture request header Referer len 50
backend myapp
server main 127.0.0.1:4000
backend stats
mode http
stats enable
stats scope http
stats scope myapp
stats realm Haproxy\ Statistics
stats uri /
stats auth username:password
Nginx is even easier.
Regarding service control, I run my Go program as a system service. I think everybody does that. My server runs Ubuntu, so it uses Upstart. I have put this at /etc/init/myapp.conf for Upstart to control my program:
start on runlevel [2345]
stop on runlevel [!2345]
chdir /home/myapp/myapp
setgid myapp
setuid myapp
exec ./myapp start 1>>_logs/stdout.log 2>>_logs/stderr.log
Another aspect is deployment. One option is to deploy by just sending binary file of the program and necessary assets. This is a pretty great solution IMO. I use the other option: compiling on server. (I’ll switch to deploying with binary files when I set up a so-called “Continuous Integration/Deployment” system.)
I have a small shell script on the server that pulls code for my project from a remote Git repository, builds it with Go, copies the binaries and other assets to ~/myapp/, and restarts the service.
Overall, the whole thing is not very different from any other server setup: you have to have a way to run your code and have it serve HTTP requests. In practice, Go has proved to be very stable for this stuff.
nginx for:
Reverse HTTP proxy to my Go application
Static file handling
SSL termination
HTTP headers (Cache-Control, et. al)
Access logs (and therefore leveraging system log rotation)
Rewrites (naked to www, http:// to https://, etc.)
nginx makes this very easy, and although you can serve directly from Go thanks to net/http, there's a lot of "re-inventing the wheel" and stuff like global HTTP headers involves some boilerplate you can probably avoid.
supervisord for managing my Go binary. Ubuntu's Upstart (as mentioned by Mostafa) is also good, but I like supervisord as it's relatively distro-agnostic and is well documented.
Supervisord, for me:
Runs my Go binary as needed
Brings it up after a crash
Holds my environmental variables (session auth keys, etc.) as part of a single config.
Runs my DB (to make sure my Go binary isn't running without it)
For those who want simple go app running as a daemon, use systemd (Supported by many linux distros) instead of Upstart.
Create a service file at
touch /etc/systemd/system/my-go-daemon.service
Enter
[Unit]
Description=My Go App
[Service]
Type=simple
WorkingDirectory=/my/go/app/directory
ExecStart=/usr/lib/go run main.go
[Install]
WantedBy=multi-user.target
Then enable and start the service
systemctl enable my-go-daemon
systemctl start my-go-daemon
systemctl status my-go-daemon
systemd has a separate journaling system that will let you tail logs for easy trouble-shooting.
You can bind your binary to a socket to Internet domain privileged ports (port numbers less than 1024) using setcap
setcap 'cap_net_bind_service=+ep' /path/to/binary
This command needs to be escalated. sudo as necessary
Every new version of your program will result in a new binary that will need to be reauthorized by setcap
setcap documentation
cap_net_bind_service documentation
I want to resolve the DNS requests issued from within a Ruby script through a DNS server, different from the ones in resolv.conf. While I could do that manualy by using Resolv::DNS or something like that, I'd like to do that for all the requests (like the ones issued by RestClient, for example). Any ideas?
RestClient uses net/http and uses the host name part of the provided URL to open a TCP socket:
https://github.com/ruby/ruby/blob/trunk/lib/net/http.rb?source=cc#L879
The simplest way to change which host is accessed is to manually change the URL to use an IP address by performing the lookup yourself.
Alternatively, you can replace the resolver of the various *Socket classes, and there is actually an example of how to do this here: https://github.com/ruby/ruby/blob/4c2304f0004e9f1784540f3d36976aad9eab1f68/lib/resolv-replace.rb
Trying to setup an example for node.js chat on Windows x64.
Command line:
D:\Websites\dev\chat>node server.js
Server at http://127.0.0.1:8001/
Now when server part runs, trying http://dev/chat/index.html
After submitting Name, it gives me "error connecting to server".
Same error message on http://chat.nodejs.org/
Does the thing actually work? =)
Do I need to set up an Apache's mod_proxy to handle /join to port 8001?
Some of the issues are with using http://dev/chat/index.html and also, I suspect, with:
Do I need to set up an Apache's mod_proxy to handle /join to port 8001?
Node's http module is more for creating the server than it is for integrating with other servers like Apache. (It's possible, e.g. iisnode, but not the default.)
While node server.js is running, you should be able to access index.html via either:
http://localhost:8001/
http://127.0.0.1:8001/
Then, /join, /recv, /send, etc. should be able to route through the same origin.
Otherwise, using http://dev/ has 2 problems:
Requests will route based on the current address. For example, /join will request http://dev/join rather than http://127.0.0.1:8001/join, likely resulting in a 404 response. And, even if you modified the client script to specify the origin...
Same-origin policy. Pages requested from http://dev/ cannot make Ajax requests to http://127.0.0.1:8001 without exceptions, which this demo does not have established.