I build a socks5 proxy.
url="https://gitlab.com/muttmua/mutt/-/wikis/MailConcept"
curl --socks5-hostname 127.0.0.1:1080 $url
curl command can get the url via socks5 proxy.
To install tsocks:
sudo apt install tsocks
And set the configuration:
vim /etc/socks/tsocks.conf
server = 127.0.0.1
server_type = 5
server_port = 1080
Let curl run in tsocks:
tsocks curl $url
It can't get nothing!
It is no use to set variable with export:
export url="https://gitlab.com/muttmua/mutt/-/wikis/MailConcept"
tsocks curl $url
curl: (7) Failed to connect to gitlab.com port 443: Connection timed out
Curl becomes a subprocess of tsocks and with the way you declare the variable, it can not be accessed from a subprocess:
url="https://gitlab.com/muttmua/mutt/-/wikis/MailConcept"
change this to
export url="https://gitlab.com/muttmua/mutt/-/wikis/MailConcept"
See also:
Defining a variable with or without export
Related
Need some help with a tricky SSH tunnel through a bastion host.
I want to port forward Postgres on the remote server, through the bastion. Our company setup only allows communication over SSH, so we have to port forward everything.
Currently, I use the CLI command to set up the SSH tunnel, then use psql shell command on my laptop to query the remote Postgres. I want to write this same connection in Go, so I can create reports, graphs, etc.
The following command line works, but I can't figure out how to do this with Go SSH.
ssh -o ProxyCommand="ssh -l username1 -i ~/.ssh/privKey1.pem bastionIP -W %h:%p" -i ~/.ssh/privKey2.pem -L 8080:localhost:5432 -N username2#PsqlHostIP
psql -h localhost -P 8000 -U user -W pass
I am using hadoop apache 2.7.1 on centos 7 and I want to delete a file(file1) by using webhdfs commands.
curl -i -x DELETE "http://192.168.25.21:50070/webhdfs/v1/hadoophome/file1/?user.name=root&op=DELETE&recursive=true"
But I am getting this error:
curl: (5) Could not resolve proxy: DELETE; Unknown error
I edited bashrc file as following :
export http_proxy=""
export https_proxy=""
export ftp_proxy=""
And source the file to save changes
source ~/.bashrc
But with the same error.
So I tried to set no proxy in the culr command as
curl -i -x --noproxy localhost DELETE "http://192.168.25.21:50070/webhdfs/v1/hadoophome/file1/?user.name=root&op=DELETE&recursive=true"
With this error:
curl: (5) Could not resolve proxy: --noproxy; Unknown error
What should I edit to exclude this proxy?
Thanks.
-x stands for proxy. You should be using -X to specify the request method.
So the command would be,
curl -i -X DELETE "http://192.168.25.21:50070/webhdfs/v1/hadoophome/file1/?user.name=root&op=DELETE&recursive=true"
Refer curl(1) for options.
My provider currently only provides FTPS as a means of uploading files to the server.
Now I want to publish files from Jenkins to that server. I can access the server using an FTP client that supports FTPS but neither of the FTP-Publisher plugins, seem to be able to publish using FTPS.
The only reference for FTPS and Jenkins that I found was this open bug.
I know that SSH would be a good option, but since my hosting provider does not support this I wonder how I can efficiently upload files to my server through jenkins.
My jenkins server runs on OSX.
Update: According to my own answer below I tried CURL but got a generic error:
curl -v -T index.html ftps://myusername:mypassword#myserver.com:21/www/
Adding handle: conn: 0x7fa9d500cc00
Adding handle: send: 0
Adding handle: recv: 0
Curl_addHandleToPipeline: length: 1
Conn 0 (0x7fa9d500cc00) send_pipe: 1, recv_pipe: 0
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0*
About to connect() to myserver.com port 21 (#0)
Trying xx.xx.xx.xx...
Connected to myserver.com (xx.xx.xx.xx) port 21 (#0)
Unknown SSL protocol error in connection to myserver.com:-9800
Closing connection 0
curl: (35) Unknown SSL protocol error in connection to myserver.com:-9800
There are currently no Jenkins plugins that will handle FTPS (FTP over SSL). Instead the cURL program is capable of uploading with FTPS.
First check that cURL is installed on the Jenkins host.
On a linux environment try the command:
which curl
Now ensure that cURL is in the path for the Jenkins user account. Alternatively fully qualify the path to cURL.
Now using a post build step, task, or with the promoted builds plugin add a shell script that contains the following:
FILEPATH=$WORKSPACE/path/to/some/file
REMOTEPATH=/new/path/for/file
curl -T $FILEPATH -u username:password ftps://myserver.com$REMOTEPATH
Correct the $FILEPATH and $REMOTEPATH to reflect the environment.
Example:
FILEPATH=$WORKSPACE/index.html
REMOTEPATH=/www/index.html
If a self signed certificate is in use on the remote host then cURL needs to skip verification. This is done with the -k parameter.
curl -T $FILEPATH -u username:password -k ftps://myserver.com$REMOTEPATH
One way of uploading might be to do this via CURL, which is not the best of options since I would rather use a Jenkins Plugin, but at least this would allow me to do it for the time being.
From the Curl docs
UPLOADING
FTP / FTPS / SFTP / SCP
Upload all data on stdin to a specified server:
curl -T - ftp://ftp.upload.com/myfile
Upload data from a specified file, login with user and password:
curl -T uploadfile -u user:passwd ftp://ftp.upload.com/myfile
Upload a local file to the remote site, and use the local file name at the remote site too:
curl -T uploadfile -u user:passwd ftp://ftp.upload.com/
Upload a local file to get appended to the remote file:
curl -T localfile -a ftp://ftp.upload.com/remotefile
Note that using FTPS:// as prefix is the "implicit" way as described in the
standards while the recommended "explicit" way is done by using FTP:// and
the --ftp-ssl option.
I have problem to connect to any mysql database using jdbc connector in google apps scripts, I'm using tutorial code:
var conn = Jdbc.getConnection("jdbc:mysql://host(or ip):3306/database", "username", "password");
But in each case ( I've tested 4 different databases on 4 different host names) i get the same error:
Failed to establish a database connection. Check connection string, username and password. (line 2)
I'm looking for some help, I have no idea what could be the problem ;-(
ps. usernames/passwords are ok.
pps. In each database remote access is working ( I've tested by using telnet).
Since your issue was Remote access to a MySQL Database I will post some related documentation so that future viewers like TonyMiao will have a avenue to fix their own related issues.
Step # 1: Login Using SSH (if server is outside your data center)
First, login over ssh to remote MySQL database server. You may need to login to your MySQL server as the root user:
ssh user#server1.cyberciti.biz
login as the root using su or sudo
su
or use sudo
sudo -i
OR directly login as root user if allowed:
ssh root#server1.cyberciti.biz
Step # 2: Edit the my.cnf file
Once connected you need to edit the MySQL server configuration file my.cnf using a text editor such as vi:
If you are using Debian/Ubuntu Linux file is located at /etc/mysql/my.cnf location.
If you are using Red Hat Linux/Fedora/Centos Linux file is located at /etc/my.cnf location.
If you are using FreeBSD you need to create a file /var/db/mysql/my.cnf location.
Edit the /etc/my.cnf, run:
# vi /etc/my.cnf
Step # 3: Once file opened, locate line that read as follows
[mysqld]
Make sure line skip-networking is commented (or remove line) and add following line
bind-address=YOUR-SERVER-IP
For example, if your MySQL server IP is 65.55.55.2 then entire block should be look like as follows:
[mysqld]
user = mysql
pid-file = /var/run/mysqld/mysqld.pid
socket = /var/run/mysqld/mysqld.sock
port = 3306
basedir = /usr
datadir = /var/lib/mysql
tmpdir = /tmp
language = /usr/share/mysql/English
bind-address = 65.55.55.2
# skip-networking
....
..
....
Where,
bind-address: IP address to bind to.
skip-networking : Don’t listen for TCP/IP connections at all. All
interaction with mysqld must be made via Unix sockets. This option is
highly recommended for systems where only local requests are allowed.
Since you need to allow remote connection this line should be removed
from my.cnf or put it in comment state.
Step# 4 Save and Close the file
If you are using Debian / Ubuntu Linux, type the following command to restart the mysql server:
# /etc/init.d/mysql restart
OR
# systemctl restart mysql
If you are using RHEL / CentOS / Fedora / Scientific Linux, type the following command to restart the mysql server:
# /etc/init.d/mysqld restart
OR
# systemctl restart mysqld
If you are using FreeBSD, type the following command to restart the mysql server:
# /usr/local/etc/rc.d/mysql-server restart
OR
# service mysql-server restart
Step # 5 Grant access to remote IP address
Connect to mysql server:
$ mysql -u root -p mysql
GRANT ACCESS TO A NEW DATABASE
If you want to add a new database called foo for user bar and remote IP 202.54.10.20 then you need to type the following commands at mysql> prompt:
mysql> CREATE DATABASE foo;
mysql> GRANT ALL ON foo.* TO bar#'202.54.10.20' IDENTIFIED BY 'PASSWORD';
HOW DO I GRANT ACCESS TO AN EXISTING DATABASE?
Let us assume that you are always making connection from remote IP called 202.54.10.20 for database called webdb for user webadmin, To grant access to this IP address type the following command At mysql> prompt for existing database, enter:
mysql> update db set Host='202.54.10.20' where Db='webdb';
mysql> update user set Host='202.54.10.20' where user='webadmin';
Step # 6: Logout of MySQL
Type exit command to logout mysql:
mysql> exit
Step # 7: Open port 3306
You need to open TCP port 3306 using iptables or BSD pf firewall.
A SAMPLE IPTABLES RULE TO OPEN LINUX IPTABLES FIREWALL
/sbin/iptables -A INPUT -i eth0 -p tcp --destination-port 3306 -j ACCEPT
OR only allow remote connection from your web server located at 10.5.1.3:
/sbin/iptables -A INPUT -i eth0 -s 10.5.1.3 -p tcp --destination-port 3306 -j ACCEPT
OR only allow remote connection from your lan subnet 192.168.1.0/24:
/sbin/iptables -A INPUT -i eth0 -s 192.168.1.0/24 -p tcp --destination-port 3306 -j ACCEPT
Finally save all rules (RHEL / CentOS specific command):
# service iptables save
A SAMPLE FREEBSD / OPENBSD / NETBSD PF FIREWALL RULE ( /ETC/PF.CONF)
Use the following to open port # 3306 on a BSD based systems:
pass in on $ext_if proto tcp from any to any port 3306
OR allow only access from your web server located at 10.5.1.3:
pass in on $ext_if proto tcp from 10.5.1.3 to any port 3306 flags S/SA synproxy state
Step # 8: Test it
From your remote system or your desktop type the following command:
$ mysql -u webadmin -h 65.55.55.2 -p
Where,
-u webadmin: webadmin is MySQL username
-h IP or hostname: 65.55.55.2 is MySQL server IP address or hostname (FQDN)
-p : Prompt for password
You can also use the telnet or nc command to connect to port 3306 for testing purpose:
$ echo X | telnet -e X 65.55.55.2 3306
OR
$ nc -z -w1 65.55.55.2 3306
Sample outputs:
Connection to 65.55.55.2 3306 port [tcp/mysql] succeeded!
Resource information: Click Here!
I want to setup a simple ssh tunnel from a local machine to a machine on the internet.
I'm using
ssh -D 8080 -f -C -q -N -p 12122 <username>#<hostname>
Setup works fine (I think) cause ssh returs asking for the credentials, which I provide.
Then i do
export http_proxy=http://localhost:8080
and
wget http://www.google.com
Wget returns that the request has been sent to the proxy, but no data is received back.
What i need is a way to look at how ssh is processing the request....
To get more information out of your SSH connection for debugging, leave out the -q and -f options, and include -vvv:
ssh -D 8080 -vvv -N -p 12122 <username>#<hostname>
To address your actual problem, by using ssh -D you're essentially setting up a SOCKS proxy which I believe is not supported by default in wget.
You might have better luck with curl which provides SOCKS suport via the --socks option.
If you really really need to use wget, you'll have to recompile your own version to include socks support. There should be an option for ./configure somewhere along the lines of --with-socks.
Alternatively, look into tsock which can intercept outgoing network connections and redirecting them through a SOCKS server.