How to properly use tor-privoxy Ruby gem? - ruby

I am using tor-privoxy Ruby gem. According to this page: https://github.com/pirj/tor-privoxy
I installed "tor" and "privoxy" packages on my Arch Linux installation. I issued commands:
sudo systemctl start privoxy.service
sudo systemctl start tor.service
Status of the services, by "systemctl status privoxy.service" and "systemctl status tor.service":
● tor.service - Anonymizing Overlay Network
Loaded: loaded (/usr/lib/systemd/system/tor.service; enabled)
Active: active (running) since Thu 2014-06-26 16:27:44 CEST; 1 weeks 5 days ago
Main PID: 454 (tor)
CGroup: /system.slice/tor.service
└─454 /usr/bin/tor -f /etc/tor/torrc
Jul 08 16:28:28 bridgelinux Tor[454]: Application request when we haven't used client functionality late...gain.
Jul 08 16:28:40 bridgelinux Tor[454]: We now have enough directory information to build circuits.
Jul 08 16:28:41 bridgelinux Tor[454]: Tor has successfully opened a circuit. Looks like client functiona...king.
Jul 08 17:20:05 bridgelinux Tor[454]: Socks version 65 not recognized. (Tor is not an http proxy.)
Jul 08 17:20:05 bridgelinux Tor[454]: Fetching socks handshake failed. Closing.
Jul 08 18:01:25 bridgelinux Tor[454]: Socks version 65 not recognized. (Tor is not an http proxy.)
Jul 08 18:01:25 bridgelinux Tor[454]: Fetching socks handshake failed. Closing.
Jul 08 18:10:04 bridgelinux systemd[1]: Started Anonymizing Overlay Network.
Jul 08 18:10:13 bridgelinux systemd[1]: Started Anonymizing Overlay Network.
Jul 08 18:14:34 bridgelinux systemd[1]: Started Anonymizing Overlay Network.
Hint: Some lines were ellipsized, use -l to show in full.
and
● privoxy.service - Privoxy Web Proxy With Advanced Filtering Capabilities
Loaded: loaded (/usr/lib/systemd/system/privoxy.service; disabled)
Active: active (running) since Tue 2014-07-08 16:09:16 CEST; 2h 8min ago
Process: 8554 ExecStart=/usr/bin/privoxy --pidfile /run/privoxy.pid --user privoxy.privoxy /etc/privoxy/config (code=exited, status=0/SUCCESS)
Main PID: 8555 (privoxy)
CGroup: /system.slice/privoxy.service
└─8555 /usr/bin/privoxy --pidfile /run/privoxy.pid --user privoxy.privoxy /etc/privoxy/config
Jul 08 16:09:16 bridgelinux systemd[1]: Started Privoxy Web Proxy With Advanced Filtering Capabilities.
Jul 08 18:17:55 bridgelinux systemd[1]: Started Privoxy Web Proxy With Advanced Filtering Capabilities.
My Ruby script looks like:
require 'mechanize'
require 'tor-privoxy'
require 'net/telnet'
def tor
privoxy_agent ||= TorPrivoxy::Agent.new '127.0.0.1', '', {8118 => 9050} do |agent|
sleep 20
puts "New IP is #{agent.ip}"
end
return privoxy_agent
end
def switch_endpoint
localhost = Net::Telnet::new("Host" => "localhost", "Port" => "9050", "Timeout" => 10, "Prompt" => /250 OK\n/)
localhost.cmd('AUTHENTICATE ""') { |c| print c; throw "Cannot authenticate to Tor" if c != "250 OK\n" }
localhost.cmd('signal NEWNYM') { |c| print c; throw "Cannot switch Tor to new route" if c != "250 OK\n" }
localhost.close
end
agent=tor
It shows that my IP adress remained the original one. When I try to call "switch_endpoint" method, I get an error: "ArgumentError: uncaught throw "Cannot authenticate to Tor"
However when I issue this command at bash prompt:
torify wget -qO- https://check.torproject.org/ | grep -i congratulations
I get no error, and it shows that I was able to connect to Tor network.
What can I do to make Tor-Privoxy work with Ruby and Mechanize?

I ran into the same problem, you can see in the logs that your authenticate command was refused by tor :
Socks version 65 not recognized. (Tor is not an http proxy.)
I managed to send telnet command to Tor using Socksify instead of tor-privoxy. You don't need privoxy anymore if you use socksify.
Here is a working example to dynamically swich Tor circuit :
First start Tor specifying password, control port and socks port:
tor --CookieAuthentication 0 --HashedControlPassword "" --ControlPort 9050 --SocksPort 50001
Then you can try this in ruby :
require 'net/telnet'
require 'socksify'
require 'mechanize'
original_ip = Mechanize.new.get("http://bot.whatismyipaddress.com").content
puts "original IP is : #{original_ip}"
# socksify will forward traffic to Tor so you dont need to set a proxy for Mechanize from there
TCPSocket::socks_server = "127.0.0.1"
TCPSocket::socks_port = "50001"
tor_port = 9050
2.times do
#Switch IP
localhost = Net::Telnet::new("Host" => "localhost", "Port" => "#{tor_port}", "Timeout" => 10, "Prompt" => /250 OK\n/)
localhost.cmd('AUTHENTICATE ""') { |c| print c; throw "Cannot authenticate to Tor" if c != "250 OK\n" }
localhost.cmd('signal NEWNYM') { |c| print c; throw "Cannot switch Tor to new route" if c != "250 OK\n" }
localhost.close
sleep 5
new_ip = Mechanize.new.get("http://bot.whatismyipaddress.com").content
puts "new IP is #{new_ip}"
end

Related

Updated Ubuntu and pcntl_fork stopped working (php)

Things that changed recently on my server:
I'm almost sure it's because the dist-upgrade.(few days ago)
I added a new user and added him a library in var/www/html/banana.
so it might be from that too (?) - (2 weeks ago)
Tried installing FastCGI without any success - but this didn't disrupt any regular processing and flow. (2 months ago)
I usually run API queries from my PHP code using forking, and in some point it stopped working for me (it does work, but when getting to heavy query results it stopps).
error.log:
[Sun Aug 28 12:15:03.201994 2016] [:notice] [pid 1882] FastCGI: process manager initialized (pid 1882)
[Sun Aug 28 12:15:03.278176 2016] [mpm_prefork:notice] [pid 1879] AH00163: Apache/2.4.18 (Ubuntu) mod_fastcgi/mod_fastcgi-SNAP-0910052141 configured -- resuming normal operations
running cat /var/mail/root outputs:
From root#banana Sun Aug 28 12:39:01 2016
Return-Path: <root#banana>
X-Original-To: root
Delivered-To: root#banana
Received: by banana (Postfix, from userid 0)
id ABC281005BA; Sun, 28 Aug 2016 12:39:01 +0300 (IDT)
From: root#banana (Cron Daemon)
To: root#banana
Subject: Cron <root#banana> [ -x /usr/lib/php/sessionclean ] && /usr/lib/php/sessionclean
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-Cron-Env: <SHELL=/bin/sh>
X-Cron-Env: <HOME=/root>
X-Cron-Env: <PATH=/usr/bin:/bin>
X-Cron-Env: <LOGNAME=root>
Message-Id: <20160828093901.ABC281005BA#YHserver>
Date: Sun, 28 Aug 2016 12:39:01 +0300 (IDT)
Can someone help me debug the problem better and solve it?
Running this script gets true results:
<?php
echo "Is fork? <br/>";
var_dump (extension_loaded('pcntl'));
echo "<br><br> more checks: <br>";
$supports = array();
if (function_exists("pcntl_fork")) $supports[] = "ispcntl";
echo implode(",", $supports);
for ($i = 1; $i <= 5; ++$i) {
$pid = pcntl_fork();
if (!$pid) {
sleep(1);
print "In child $i\n";
exit;
}
}
?>
EDIT: I Tried running that same script on the server without forking and I got all the results right (after waiting a lot and getting my website stuck for a while..)

rsyslog - Avoid pushing certain logs to /var/log/messages

I'm having an ec2 linux server, and am tracking the logs of my application server using rsyslog so that I can push these logs to loggly.
The problem is, rsyslog is also logging these in /var/log/messages which I don't want. Is there any way to avoid this? Can I filter out certain messages in /etc/rsyslog.conf so that these are not pushed to var/log/messages?
****** UPDATE *******
I tried adding the following lines in rsyslog.conf:
if $programname == 'programName' then {
*.err /var/log/messages
} else {
*.info;mail.none;authpriv.none;cron.none /var/log/messages
}
However, upon restarting rsyslog, I see the following error:
Dec 11 08:01:46 <hostname> rsyslogd: the last error occured in /etc/rsyslog.conf, line 37:"if $programname == 'programName' then {"
Dec 11 08:01:46 <hostname> rsyslogd: warning: selector line without actions will be discarded
Dec 11 08:01:46 <hostname> rsyslogd-3000: unknown priority name "" [try http://www.rsyslog.com/e/3000 ]
Dec 11 08:01:46 <hostname> rsyslogd: the last error occured in /etc/rsyslog.conf, line 39:"} else {"
Dec 11 08:01:46 <hostname> rsyslogd: warning: selector line without actions will be discarded
Dec 11 08:01:46 <hostname> rsyslogd-3000: unknown priority name "" [try http://www.rsyslog.com/e/3000 ]
Dec 11 08:01:46 <hostname> rsyslogd: the last error occured in /etc/rsyslog.conf, line 41:"}"
Dec 11 08:01:46 <hostname> rsyslogd: warning: selector line without actions will be discarded
I suppose my version of rsyslog (5.8.10) doesn't support if / else. Is there any other way to do this?
Thanks.
first send the message to the file that you want.
then use stop to prevent further actions.
if $programname == 'apache2' then {
action(type="omfile" file="/var/log/apache2/rewrite.log" name="action-omfile-apache2-rewrite")
stop
}

why xinetd can't run shell service

guys,i have a problem on using xinetd,the error message is 'xinetd[20126]: execv( /home/fulu/download/mysqlchk_status2.sh ) failed: Exec format error (errno = 8)'
the system operation is : CentOS release 6.2;
i installed the xinetd by the command 'sudo yum install xinetd'
i edited the /etc/services, add my port 6033 for my service named 'mysqlchk'
the service 'mysqlchk' in /etc/xinetd.d/mysqlchk is
service mysqlchk
{
disable = no
flags = REUSE
socket_type = stream
port = 6033
wait = no
user = fulu
server = /home/fulu/download/mysqlchk_status2.sh
log_on_failure += USERID
}
the shell file /home/fulu/download/mysqlchk_status2.sh content is
echo 'test'
6.i can run the command /home/fulu/download/mysqlchk_status2.sh straightly and get the result 'test'
when i telnet 127.0.0.1 6033,i get the output
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.
Connection closed by foreign host.
then i tail the log file /var/log/messages,it shows
Apr 22 22:01:47 AY1304111122016 xinetd[20001]: START: mysqlchk pid=20126 from=127.0.0.1
Apr 22 22:01:47 AY1304111122016 xinetd[20126]: execv( /home/fulu/download/mysqlchk_status2.sh ) failed: Exec format error (errno = 8)
Apr 22 22:01:47 AY1304111122016 xinetd[20001]: EXIT: mysqlchk status=0 pid=20126 duration=0(sec)
i don't know why,can anybody help me ?
I'm sorry, after questioning it i suddenly found the answer. If you want the shell to be run in other program you need add '#!/bin/echo' at the first line of the shell file (of course the echo can be changed)

ruby webrick and cgi

I can't get Webrick to work with the servlet HTTPServlet::CGIHandler--I get an EACCES error:
[2012-12-06 01:38:02] ERROR CGIHandler: /tmp/cgi-bin:
/Users/7stud/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/webrick/httpservlet/cgi_runner.rb:46:in `exec': Permission denied - /tmp/cgi-bin (Errno::EACCES)
from /Users/7stud/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/webrick/httpservlet/cgi_runner.rb:46:in `<main>'
[2012-12-06 01:38:02] ERROR CGIHandler: /tmp/cgi-bin exit with 1
[2012-12-06 01:38:02] ERROR Premature end of script headers: /tmp/cgi-bin
localhost - - [06/Dec/2012:01:38:02 MST] "GET /cgi/my_prog.cgi HTTP/1.1" 500 326
- -> /cgi/my_prog.cgi
Here are the permissions I set:
~/ruby_programs$ cd /
/$ ls -al tmp
lrwxr-xr-x# 1 root wheel 11 Jul 3 2011 tmp -> private/tmp
/$ cd tmp
/tmp$ ls -al
total 0
drwxrwxrwt 8 root wheel 272 Dec 6 01:08 .
drwxr-xr-x# 6 root wheel 204 Mar 27 2010 ..
drwxr-xr-x 3 7stud wheel 102 Dec 6 01:25 cgi-bin
/tmp$ cd cgi-bin/
/tmp/cgi-bin$ ls -al my_prog.cgi
-rwxr-xr-x 1 7stud wheel 123 Dec 6 01:09 my_prog.cgi
My server program(1.rb):
#!/usr/bin/env ruby
require 'webrick'
include WEBrick
port = 12_000
dir = Dir::pwd
server = HTTPServer.new(
:Port => port,
:DocumentRoot => dir + "/html"
)
server.mount("/cgi", HTTPServlet::CGIHandler, "/tmp/cgi-bin")
puts "Listening on port: #{port}"
Signal.trap('SIGINT') { server.shutdown }
server.start
Running my server program:
~/ruby_programs$ ruby 1.rb
[2012-12-06 01:37:58] INFO WEBrick 1.3.1
[2012-12-06 01:37:58] INFO ruby 1.9.3 (2012-04-20) [x86_64-darwin10.8.0]
Listening on port: 12000
[2012-12-06 01:37:58] INFO WEBrick::HTTPServer#start: pid=4260 port=12000
I entered this address in my browser:
http://localhost:12000/cgi/my_prog.cgi
This was displayed in my browser:
Internal Server Error
Premature end of script headers: /tmp/cgi-bin WEBrick/1.3.1
(Ruby/1.9.3/2012-04-20) at localhost:12000
Here's my cgi script(/tmp/cgi-bin/my_prog.cgi):
#!/usr/bin/env ruby
require 'cgi'
cgi = CGI.new
puts cgi.header
puts "<html><body>Hello Webrick</body></html>"
The only way I can get WEBrick to execute cgi files in a directory other than the root, is to use the HTTPServlet::FileHandler servlet:
port = 12_500
...
cgi_dir = File.expand_path("~/ruby_programs/cgi-bin")
server.mount("/cgi", HTTPServlet::FileHandler, cgi_dir)
Then the url used to execute a .cgi file located in the cgi_dir is:
http://localhost:12500/cgi/my_prog.cgi
Apparently, when you write:
server = HTTPServer.new(
:Port => port,
:DocumentRoot => "./html" #Regular files served/.cgi files executed out of this dir
)
Webrick automatically "mounts" an HTTPServlet::FileHandler to handle requests to the :DocumentRoot directory, e.g.
http://localhost:12500/my_html.htm
which will serve files out of the ./html directory (i.e. a directory called html located below the directory from which your program is running). The HTTPServlet::FileHandler will also execute files in that directory if they have a .cgi extension.
If you explicitly use mount() to add an HTTPServlet::FileHandler to another directory, e.g.
cgi_dir = File.expand_path("~/ruby_programs/cgi-bin")
server.mount("/cgi", HTTPServlet::FileHandler, cgi_dir)
then WEBrick will also serve files from that directory and execute files in that directory that have a .cgi extension.
I haven't found a way to configure WEBrick to only serve files out of the :DocumentRoot directory and only execute .cgi files in another directory.
See "Gnome's Guide to WEBrick" here:
http://microjet.ath.cx/webrickguide/html/
In my case I had similar problems because of incorrect file permissions and really incorrect headers. Permissions of CGI-script should be like this:
~/ruby_projects/cgi_webrick/cgi
ls -l
-rwxr-xr-x 1 me me 112 Sep 20 20:29 test.cgi
My server code looks very similar (placed in ~/ruby_projects/cgi_webrick/), but with different handler.
#!/usr/bin/env ruby
require 'webrick'
server = WEBrick::HTTPServer.new :Port => 1234
server.mount "/", WEBrick::HTTPServlet::FileHandler , './'
trap('INT') { server.stop }
server.start
If you run server script ruby my_server_script.cgi, it will serve any scripts from root or other directory. In my case I can access http://localhost:1234/cgi/test.cgi (script placed in cgi subfolder), and http://localhost:1234/test.cgi (placed in the root directory from which server is started).
My test script:
#!/usr/bin/ruby
require 'cgi'
cgi = CGI.new
puts cgi.header
puts "<html><body>This is a test</body></html>"

Ruby 1.8.7 and Net::HTTP: Making an SSL GET request with client cert?

I'm trying to fetch a resource via SSL using Net::HTTP. Here is the relevant code fragment:
req = Net::HTTP::Get.new(ContentURI.path)
https = Net::HTTP.new(ContentURI.host, ContentURI.port)
https.use_ssl = true
https.cert = OpenSSL::X509::Certificate.new(#cert_raw)
https.key = OpenSSL::PKey::RSA.new(#cert_key_raw)
https.verify_mode = OpenSSL::SSL::VERIFY_PEER
https.ca_file = File.join(TestDataPath, 'cacert.pem')
resp = https.start { |cx| cx.request(req) }
or with the alternate last line:
resp = https.get(ContentURI.path)
I have verified that the various bits (cert, key, CA cert, etc.) are correct.
The problem is that the cx.request(req) throws an exception:
OpenSSL::SSL::SSLError: SSL_connect SYSCALL returned=5 errno=0 state=SSLv3 read server session ticket A
The Apache SSL error log on the server contains the following:
[Tue Jan 24 11:47:26 2012] [debug] ssl_engine_kernel.c(1876): OpenSSL: Loop: SSLv3 read finished A
[Tue Jan 24 11:47:26 2012] [debug] ssl_engine_kernel.c(1905): OpenSSL: Exit: error in SSLv3 write session ticket A
[Tue Jan 24 11:47:26 2012] [debug] ssl_engine_kernel.c(1905): OpenSSL: Exit: error in SSLv3 write session ticket A
[Tue Jan 24 11:47:26 2012] [info] [client 10.11.88.53] SSL handshake interrupted by system [Hint: Stop button pressed in browser?!]
[Tue Jan 24 11:47:26 2012] [info] [client 10.11.88.53] Connection closed to child 0 with abortive shutdown (server _SERVERNAME_:443
The cert, key, and CA cert file work with this SSL host through other tools; I'm just having trouble reproducing that success programatically using Net::HTTP[S].
Thanks to anyone who can identify what I'm doing wrong!
I'd say this is matter of your Apache setup rather than problem with Ruby itself.
Check this out:
$ curl https://palladium.wejn.cz/sslcerttest/
curl: (56) SSL read: error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure, errno 0
$ curl https://palladium.wejn.cz/sslcerttest/ -E client.pem
<pre>Yop.</pre>
And then:
#!/usr/bin/ruby
require 'net/http'
require 'net/https'
require 'openssl'
require 'uri'
ContentURI = URI.parse("https://palladium.wejn.cz/sslcerttest/")
#cert_raw = File.read('client.pem')
#cert_key_raw = #cert_raw
TestDataPath = '.'
req = Net::HTTP::Get.new(ContentURI.path)
https = Net::HTTP.new(ContentURI.host, ContentURI.port)
https.use_ssl = true
https.cert = OpenSSL::X509::Certificate.new(#cert_raw)
https.key = OpenSSL::PKey::RSA.new(#cert_key_raw)
https.verify_mode = OpenSSL::SSL::VERIFY_PEER
https.ca_file = File.join(TestDataPath, 'cacert.pem')
resp = https.start { |cx| cx.request(req) }
p resp
p resp.body
results in:
$ ruby a.rb
#<Net::HTTPOK 200 OK readbody=true>
"<pre>Yop.</pre>\n"
$ dpkg -l ruby1.8 | tail -n1 | awk '{print $3}'
1.8.7.302-2squeeze1
$ ruby -v
ruby 1.8.7 (2010-08-16 patchlevel 302) [x86_64-linux]
Of course, the apache config in question is:
<Directory /var/www/sslcerttest/>
SSLVerifyClient require
SSLVerifyDepth 5
SSLCACertificateFile /var/www/sslcerttest/cacert.pem
SSLCACertificatePath /var/www/sslcerttest/
SSLOptions +FakeBasicAuth
SSLRequireSSL
SSLRequire %{SSL_CLIENT_S_DN_O} eq "Wejn s.r.o." and %{SSL_CLIENT_S_DN_OU} eq "Server Certificate"
</Directory>
Maybe posting additional details (apache config to test against) would be of some help when tracking this issue down...

Resources