Being new to Apache Camel, I have been following the examples all over the internet. Near on every book/forum/bog post has an example on how to create a simple route that will consume/produce files to/from an FTP server.
My issue is that the host that I am trying to connect to using the suggested connection
from("ftp://someone#someserver.com?password=secret&localWorkDirectory=/tmp")
OR
<from uri="ftp://someone#someserver/public/reports?password=password"/>
Doesn't work for me when I use the credentials my host supplies me:
For the sake of this example, lets call the FTP details:
FTP username: username#ftpserverhost.com
Password: password
So when I replace the advised URI with my servers credentials it fails to connect. I am assuming this is because it has two # symbols and this is causing the route some confusion.
from("ftp://username#ftpserverhost.com#ftpserverhost.com?password=secret&localWorkDirectory=/tmp")
OR
<from uri="ftp://username#ftpserverhost.com#ftpserverhost.com/public/reports?password=password"/>
Is there a way to escape the # or get this working? I have asked my host if its possible to leave the #ftpserverhost.com but they have advised me that it will not work.
Any ideas?
Use the username option:
from("ftp://ftpserverhost.com?password=secret&localWorkDirectory=/tmp&username=username#ftpserverhost.com")
If this does not work and if you use Camel 2.11 or above, you may use the RAW syntax:
from("ftp://ftpserverhost.com?password=secret&localWorkDirectory=/tmp&username=RAW(username#ftpserverhost.com)")
The RAW syntax also comes in handy if your options contains &, + or other special characters that must not be encoded.
Related
I am using pybliometrics, a Python interface to the Scopus API, to download the abstracts of some papers.
Unfortunately Scopus only works inside the network of the university that subscribed to it. I am currently at home and whenever I try to download something using pybliometrics it gives me the following error:
pybliometrics.scopus.exception.Scopus401Error: The requestor is not authorized to access the requested view or fields of the resource
I need to use my university's proxy in order to enter the internet with the IP address of my university. The proxy has a WPAD configuration file available, but I fail to realize how to use it with pybliometrics. The pybliometrics documentation says to add a block in the configuration file like this:
[Proxy]
ftp = socks5://127.0.0.1:1234
http = socks5://127.0.0.1:1234
https = socks5://127.0.0.1:1234
But this proxy requires authentication. How can I specify the proxy username and password?
EDIT: I have tried setting up the block in config.ini like:
[Proxy]
ftp = http://username:password#proxy.thing.it:8080
http = http://username:password#proxy.thing.it:8080
https = http://username:password#proxy.thing.it:8080
but it still fails with the following error message:
requests.exceptions.ProxyError: HTTPSConnectionPool(host='api.elsevier.com', port=443): Max retries exceeded with url: /content/abstract/scopus_id/84983158344?view=META_ABS (Caused by ProxyError('Cannot connect to proxy.', OSError('Tunnel connection failed: 407 Proxy Authentication Required')))
From our perspective the API will work via a proxy as long as the proxy is configured correctly. I would suggest you speak to the provider of the proxy to see if they can help.
We don't have specific instructions on how to use APIs with a proxy (as there are many potential different versions and potential configurations); however, the general instructions are here:
https://service.elsevier.com/app/answers/detail/a_id/29026/supporthub/elsevieraccess/
To me your new proxy block looks suspicious. It funnels ftp and https requests through the http as well. Maybe try ftp and https as protocols in the corresponding sections.
The other solution is to ask Scopus Integration Support for an InstToken, which you use instead of a proxy. You then specify the InstToken in the configuration file as well.
The problem was that my proxy requires DigestAuth rather than BasicAuth.
HTTPS POST requests are successful in POSTMAN both with and without Port 443. But failing in MULE Application with the error Remotely Closed.
Also, I tried to access the host using commands PING, TELNET and TRACENET for the host server via Command Prompt. Those fail with error Request Timed Out.
Can you please let me know where the issue lies?
Workaround:
For Mule4, remove the Global Element "HTTP Request Configuration" and pass the URL directly.
I am working with SAP Revsym Rest API's and this worked for me.
Plese, share config details. My guess is that config has path as well as path exist in the http request. These 2 pathes combined lead request to wrong place. That would be only possible explanation for provided information.
I've hunted around for this and just can't find a solution.
Currently I have a database.yml connected to a local pgbouncer server on a Unix socket successfully. However, I'm transitioning to setting this to a database_url environment variable and cannot work out at all how to connect to a local Postgres server via a Unix socket. localhost obviously works OK.
I was looking at a URL that looks like this as apparently you can use this with Postgres (http://www.postgresql.org/docs/9.2/interactive/libpq-connect.html):
export DATABASE_URL="postgresql://username#%Fvar%Frun%Fpostgresql%F.s.PGSQL.6432/dbname"
However, this will not get past the URI police:
rubies/ruby-2.1.5/lib/ruby/2.1.0/uri/common.rb:176:in `split': bad URI(is not URI?): postgresql://username#%Fvar%Frun%Fpostgresql%F.s.PGSQL.6432/dbname
Does anyone have any idea about the secret sauce needed for this? I've Googles endlessly and haven't found anything. It must be possible since the application currently connects over a socket now.
Thanks in advance,
31.1.1.2. Connection URIs
The general form for a connection URI is:
postgresql://[user[:password]#][netloc][:port][/dbname][?param1=value1&...]`
The URI scheme designator can be either postgresql:// or postgres://.
Each of the URI parts is optional. The following examples illustrate
valid URI syntax uses:
postgresql://
postgresql://localhost
postgresql://localhost:5433
postgresql://localhost/mydb
postgresql://user#localhost
postgresql://user:secret#localhost
postgresql://other#localhost/otherdb?connect_timeout=10&application_name=myapp
Components of the hierarchical part of the URI can also be given as
parameters. For example:
postgresql:///mydb?host=localhost&port=5433
Percent-encoding may be used to include symbols with special meaning
in any of the URI parts.
Any connection parameters not corresponding to key words listed in
Section 31.1.2 are ignored and a warning message about them is sent to
stderr.
For improved compatibility with JDBC connection URIs, instances of
parameter ssl=true are translated into sslmode=require.
The host part may be either host name or an IP address. To specify an
IPv6 host address, enclose it in square brackets:
postgresql://[2001:db8::1234]/database
The host component is interpreted as described for the parameter host.
In particular, a Unix-domain socket connection is chosen if the host
part is either empty or starts with a slash, otherwise a TCP/IP
connection is initiated. Note, however, that the slash is a reserved
character in the hierarchical part of the URI. So, to specify a
non-standard Unix-domain socket directory, either omit the host
specification in the URI and specify the host as a parameter, or
percent-encode the path in the host component of the URI:
postgresql:///dbname?host=/var/lib/postgresql
postgresql://%2Fvar%2Flib%2Fpostgresql/dbname
You must omit the host to use unix socket, like so:
postgres://username#/dbname
or simply
postgres:///dbname
This works with psql > 9.2.
I am not sure it works with the rails handling of the database URL.
You can pass the socket as a query parameter:
postgresql://user#host/database?socket=/path/to/socket
The answer provided by #blnc is correct if using libpq in general. However, if you are using Activerecord or RubyonRails, at least in version 3.2.21, that module parses the URI to Ruby's URI.parse, instead of directly handing it to libpq. URI.parse cannot handle an empty hostname here.
irb(main):020:0> URI.parse "postgresql://user:pass#host/dbname"
=> #<URI::Generic:0x7f72b17f04d8 URL:postgresql://user:pass#host/dbname>
But, without host:
irb(main):021:0> URI.parse "postgresql://user:pass#/dbname"
URI::InvalidURIError: the scheme postgresql does not accept registry part: user:pass# (or bad hostname?)
from /usr/lib/ruby/1.8/uri/generic.rb:195:in `initialize'
from /usr/lib/ruby/1.8/uri/common.rb:492:in `new'
from /usr/lib/ruby/1.8/uri/common.rb:492:in `parse'
from (irb):21
from /usr/lib/ruby/1.8/uri/generic.rb:556
There appears to be no way to fix this without adding custom URI-parsing code (or altering activerecord).
On Archlinux with default path for unix_socket_directories:
postgres:///<dbname>?host=/run/postgresql/
I want anyone who send a email to test#example.com, and the server could get the mail and so I can use some server side scripting language like Ruby to parser the content of the email.
Ruby solution would be great.
If you've never set up an smtp server, and you're not willing to dig in and learn a lot, you might want to check out some of the services that offer this kind of ability. I have never used any of these, not sure about quality or pricing aspects.
cloudmailin
mailgun
dispatch
postmark
An alternative to setting up and configuring a mail server is to use a service such as sendgrid or postmark. You point your DNS mx records at them and they will turn inbound messages into JSON documents and post them to you
You can install qmail on your server, and configure it to pipe incoming mail to your domain to the STDIN of your ruby script.
If you haven't done so already, create an MX record for your domain that points to your mailserver's IP. Then, login to the server as root, then install qmail by following the instructions at http://lifewithqmail.org/lwq.html#installation
Configure qmail to accept incoming mail for *#yourdomain.tld (catchall), and forward all incoming mail to the script at /home/yourhomedirectory/yourscript.sc:
add the following line to /var/qmail/control/me:
hostname.yourdomain.tld
add the following line to /var/qmail/control/defaultdomain:
yourdomain.tld
addd the following line to /var/qmail/control/virtualdomains
yourdomain.tld:yourusername
add the following line to /var/qmail/control/rcphosts:
yourdomain.tld
restart qmail:
qmailctl stop
qmailctl stat
Logout of the server, then log back in again as yourusername. Create a file /home/yourusername/.qmail-default, consisting of the following line:
| /home/yourhomedirectory/yourscript.sc
Create /home/yourhomedirectory/yourscript.sc script to process incoming mail. Enable executable permissions on the script for all users:
chmod a+x /home/yourhomedirectory/yourscript.sc
That's all there is to it. Incoming mail to your domain will be piped by qmail to this script's STDIN.
Install and configure an SMTP server. SMTP servers can usually be configured quite extensively, what to do with mails that should be deliverd locally. One such option would be to deliver the mail locally by invoking a ruby script. Popular SMTP servers are postfix, courier and qmail. The details about how to setup the mail server depends on which one you use.
I'm trying to get some protocols work through my company's firewall. Until now I have been succesfull in masking either http or https data by setting a http proxy on localhost and one on a remote server I own. The communication is done via $_POSTed and received modified .bmp files that contain a header and the encripted serialised request array.
This works fine, but there are a few drawbacks that make me think I might have taken a wrong approach.
Firstly I do not use apache's mod-proxy. instead I just created a local subdomain (proxy.localhost) and use that in browser's proxy settings. the subdomain's index.php does all the work. This creates some problems. I cannot use http and https simultaneously or the server will complain of using either "http on a https enabled port" or "incoresc ssl response length".
The second problem is, well, other protocols. I could make use of some ftp, sftp, remote deskoptop, ssh, nust name another... I need it
there are 2 solutions I can think of: First is if I run a php script in CLI so that it listens on a predefined port and handles the requests differently, or some sort of ssh tunnel. Problem is I haven't had any success with freeSSHd and putty because of my ignorance.
Thanks in advance for any advice.
I used the free version of bitvise SSH Client and server and it seems to work just fine.