Bind kafka to 0.0.0.0:9092 - amazon-ec2

I have set up Kafka on my amazon ec2 machine running ubuntu-18 following this blog plost and this is how it is exposing the ports.
tcp 0 0 127.0.0.53:53 0.0.0.0:* LISTEN 772/systemd-resolve
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1220/sshd
tcp 0 0 0.0.0.0:3004 0.0.0.0:* LISTEN 1041/mongod
tcp6 0 0 :::45827 :::* LISTEN 2059/java
tcp6 0 0 :::9092 :::* LISTEN 2136/java
tcp6 0 0 :::2181 :::* LISTEN 2059/java
tcp6 0 0 :::32851 :::* LISTEN 2136/java
tcp6 0 0 :::22 :::* LISTEN 1220/sshd
how can I bind it to 0.0.0.0:9092?

how can I bind it to 0.0.0.0:9092
:::9092 should be all you need for binding on IPv6.
If you want to force IPv4, please refer kafka binding to ipv6 port even though ipv4 address specified in config
You can also add this to server.properties to explicitly bind to all interfaces
listeners=PLAINTEXT://0.0.0.0:9092
But when set, you also need to set (and uncomment) advertised.listeners to the external interface address (IP or hostname) that clients should use to communicate to that server, as mentioned in the property file.
# If not set, it uses the value for "listeners".
#advertised.listeners=PLAINTEXT://your.host.name:9092
More details here if you need something more complex https://www.confluent.io/blog/kafka-listeners-explained

I am doing this on aws ec2 running ubuntu 18, the blog post shared in the first answer provides detailed information on how to go about this kind of challenge. The main challenge was failing to get a broker connection from the machine.
What worked is to add your machines public DNS(ec2......com) to advertised listeners.
I made the edit in server.properties file and like
advertised.listeners=PLAINTEXT://public DNS(ec2......com):9092

Related

Unable to connect to elastic search remotely

My elastic search is running on server A on port 9200 and 9300.
tcp6 0 0 127.0.0.1:9200 :::* LISTEN 23489/java
tcp6 0 0 ::1:9200 :::* LISTEN 23489/java
tcp6 0 0 127.0.0.1:9300 :::* LISTEN 23489/java
tcp6 0 0 ::1:9300 :::* LISTEN 23489/java
When I try to connect to elastic search from server B which is on same LAN, I get connection refused error. Even I am unable to telnet the server on port 9200 or 9300. Please suggest what I am missing.
This is because your ES is bound to localhost 127.0.0.1.
You need to change the network.host property in elasticsearch.yml in order to be able to connect from remote hosts. Basically, this does the trick and will use the first available IP address in your network:
network.host: 0

cant use MAMP after laravel php artisan

Hi today i was able to run a laravel project of a client, was a big problem, i achieve that after a lot of tests doing a sudo
sudo php artisan serve --host=127.0.0.1 --port=80
After that I restart Apache and MAMP and MYsql and i thougth that everything was ok.... But i try to run MAMP after editing de virtualhost, and PDO is trying to work with ¿laravel project? wtf..
Im desperate, how can i reset ¿mysql?, ¿apache?, i try to reset both , apache at least can go into localhost , and i get yes it work
Pls someone help i have a lot of jobs to do in MAMP
How can i see if something of laravel is still running, how can i close it.
Hopefully I've understood your question correctly the below might help.
You can see what processes are running using the netstat command.
If you do netstat -tapn you should see something like
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN -
tcp 0 0 127.0.0.1:8000 0.0.0.0:* LISTEN 9046/php5
tcp 0 0 0.0.0.0:11300 0.0.0.0:* LISTEN -
tcp 0 0 0.0.0.0:6379 0.0.0.0:* LISTEN -
tcp 0 0 0.0.0.0:11211 0.0.0.0:* LISTEN -
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 10231/nginx: worker
tcp 0 0 X.X.X.X:80 X.X.X.X:33801 TIME_WAIT -
tcp 0 0 X.X.X.X:80 X.X.X.X:44907 TIME_WAIT -
tcp 0 1012 X.X.X.X:22 X.X.X.X:61850 ESTABLISHED -
tcp 0 0 X.X.X.X:49308 X.X.X.X:443 TIME_WAIT -
tcp6 0 0 :::22 :::* LISTEN -
tcp6 0 0 :::3306 :::* LISTEN -
tcp6 0 0 :::6379 :::* LISTEN -
my artisan is the second line down in this case running on port 8000 you can tell this by it being a php5 process. You can kill this by doing kill PID the PID in my case being 9046.
The reason your probably seeing problems in the first place is that depending on your MAMP configuration that probably runs on port 80 your then trying to run artisan serve also on port 80 instead run it on the default 8000 then you can have both running.

Tightvncserver on rapsberry pi not working - connection to mac

today i installed a vncserver on my raspberry pi running rasbian.
I used this tutorial: http://blog.wenzlaff.de/?p=2207 (its german but i think you will understand what they do there anyway ;) )
Everythin worked great but the tightvnc viever jar applet on my mac will not go further than "handshaking with remote host". After typing in my password of my pi, nothing is happening anymore.
I tried to look in the logs of tightvncserver on the pi , but i got a "permission denied" every time.
Can you help me please. I dont know what do to now.
Thanks
Most likely you have some problems with SSH-tunneling ( provided you were following the tutorial).
To diagnose what's happening, first it would be nice to figure out whether your vncserver is running on Raspberry. You can do it by issuing command:
netstat -lnt
The output can look something like below:
pi#raspberrypi ~ $ netstat -lnt
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.1:631 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.1:587 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:5901 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:6001 0.0.0.0:* LISTEN
We are interested to see if there's line:
tcp 0 0 0.0.0.0:5901 0.0.0.0:* LISTEN
which basically tells us that our vncserver instance is listening on port 5901.
If that's true, then you should be able to use any VNC Viewer to connect to your RPI. Maybe you want to avoid SSH-tunnelling option and connect directly.

HDFS using Cloudera Manager in private cloud

This is driving me crazy. I have been working on this for days and just can't seem to solve this issue. I have a private cloud running on eucalyptus for testing and 4 VMs running Ubuntu 12.04. I am trying to get cloudera to run HDFS and map-reduce however when I try to start it up, the data-nodes never seem to be able to communicate with the name-node. It installs fine and passes all the pre-launch checks. Host files are all set up with 127.0.0.1 localhost and the ip and hostnames of the other vms, firewalls are all disable and security groups are set to allow everything. I can connect to the 8022 port from the data-nodes to the name-node with telnet and netstat on the name-node looks like this:
tcp 0 0 172.31.254.119:9000 0.0.0.0:* LISTEN 6519/python
tcp 0 0 0.0.0.0:7432 0.0.0.0:* LISTEN 5672/postgres
tcp 0 0 127.0.0.1:9001 0.0.0.0:* LISTEN 6538/python
tcp 0 0 172.31.254.119:50090 0.0.0.0:* LISTEN 8694/java
tcp 0 0 0.0.0.0:7180 0.0.0.0:* LISTEN 5680/java
tcp 0 0 0.0.0.0:7182 0.0.0.0:* LISTEN 5680/java
tcp 0 0 172.31.254.119:8020 0.0.0.0:* LISTEN 8689/java
tcp 0 0 172.31.254.119:50070 0.0.0.0:* LISTEN 8689/java
tcp 0 0 172.31.254.119:8022 0.0.0.0:* LISTEN 8689/java
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 576/sshd
tcp 0 0 127.0.0.1:5432 0.0.0.0:* LISTEN 5486/postgres
tcp6 0 0 :::7432 :::* LISTEN 5672/postgres
tcp6 0 0 :::22 :::* LISTEN 576/sshd
yet the error I keep getting is:
Failed to publish event: SimpleEvent{attributes={STACKTRACE=[org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.protocol.DisallowedDatanodeException): Datanode denied communication with namenode: DatanodeRegistration(172.31.254.110, storageID=DS-1259113373-172.31.254.110-50010-1378398035331, infoPort=50075, ipcPort=50020, storageInfo=lv=-40;cid=cluster9;nsid=46459994;c=0)
I would greatly appreciate any advice from anyone with more Linux/cloudera/eucalyptus experience then I.
Thanks all.
You have specified that you are using loopback, but the DN is identifying itself as 172.31.254.110. Use proper hostname instead of 127.0.0.1. To be on the safer side add the hostname and IP of each machine into the /etc/hosts files of all other machines. If problem still persists, show me your config files.

Cannot telnet to port 25 on EC2 instance

My security group has the following:
>22 (SSH) 0.0.0.0/0
>80 (HTTP) 0.0.0.0/0
>143 (IMAP) 0.0.0.0/0
>443 (HTTPS) 0.0.0.0/0
>995 (POP3S) 0.0.0.0/0
>465 (SMTPS) 0.0.0.0/0
>25 (SMTP) 0.0.0.0/0
Running a netstat on the server shows the following:
>Active Internet connections (servers and established)
>Proto Recv-Q Send-Q Local Address Foreign Address State
>tcp 0 0 127.0.0.1:9000 0.0.0.0:* LISTEN
>tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN
>tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN
>tcp 0 0 0.0.0.0:25 0.0.0.0:* LISTEN
>tcp 0 0 10.211.30.202:44025 194.14.0.125:6667 ESTABLISHED
>tcp6 0 0 :::995 :::* LISTEN
>tcp6 0 0 :::110 :::* LISTEN
>tcp6 0 0 :::143 :::* LISTEN
>tcp6 0 0 :::22 :::* LISTEN
>tcp6 0 0 :::25 :::* LISTEN
>tcp6 0 0 :::993 :::* LISTEN
And when I try and access the box from the outside world, I get nothing.
>thedude:~ root$ telnet mail.sd0a.com 25<br />
>Trying 107.20.235.215...<br />
>telnet: connect to address 107.20.235.215: Operation timed out<br />
>telnet: Unable to connect to remote host<br />
Anyone have any positive experiences with Amazon EC2 instances and getting mail to a state where it will work? Its worth noting that via command line, mail seems to go through. System is Ubuntu 12.04.1 LTS if that matters.
Might be your ISP filtering outbound connections to port 25/tcp in order to prevent botnet spam.
To eliminate the obvious: Have you tried
connect to another port other than 25?
connect to another new ec2 instance, port 25? (straightforward task to duplicate it on EC2)
connect from another machine (or your friend's PC) to sd0a.com:25?
traceroute to identify where the packets are dropped?
setup postfix on port 2525 (remember to add that into Security Groups)
ufw* on Ubuntu. (Default is off... but good to check)
As far as I can tell, all IP addresses on Amazon EC2 are blacklisted in spamhaus.com (and a lot of other anti-spam list). Hence most likely your ISP is blocking these packets, if so it is IP block or port block?

Resources