I am trying to post audio data with curl for a HTTP-API which allows to transmit/receive audio files.
First I tried this:
curl -vv --http1.0 -H "Content-Type: audio/basic" -H "Content-Length: 9999999" -H "Connection: Keep-Alive" -H "Cache-Control: no-cache" --data-binary #- 'http://IP/API-Endpoint.cgi'
This seems to work:
* Trying [IP]...
* TCP_NODELAY set
* Connected to [IP] ([IP]) port 80 (#0)
> POST /API-Endpoint.cgi HTTP/1.0
> Host: [IP]
> User-Agent: curl/7.54.0
> Accept: */*
> Content-Type: audio/basic
> Content-Length: 9999999
> Connection: Keep-Alive
> Cache-Control: no-cache
>
* upload completely sent off: 17456 out of 17456 bytes
* HTTP 1.0, assume close after body
< HTTP/1.0 200 OK
< Content-Type: text/plain
< Content-Length: 0
* HTTP/1.0 connection set to keep alive!
< Connection: keep-alive
< Date: Wed, 06 Jun 2018 19:38:37 GMT
< Server: lighttpd/1.4.45
But I can only hear the very last part of the Audio file. (The file has the correct audio format for the API: G.711 μ-law with 8000 Hz) My next guess is, that the audio gets transmitted too fast and has to be sent in real time to the API endpoint. So I tried the --limit-rate parameter of curl, which had no effect. Then I tried piping the data with a rate limit into curl:
cat myfile.wav | pv -L 10k | curl -vv --http1.0 -H "Content-Type: audio/basic" -H "Content-Length: 9999999" -H "Connection: Keep-Alive" -H "Cache-Control: no-cache" --data-binary #- 'http://IP/API-Endpoint.cgi'
but the result is always the same: I can only hear the last part of the audio file. It seems like curl is waiting for the piped input to complete and then sends the request as before.
Is there an option to post audio to a HTTP-API from bash in "real time"?
Update:
Without forcing HTTP 1.0 I get the following result:
curl -vv -H "Content-Type: audio/basic" --data-binary '#myfile.wav' 'http://[IP]/API-Endpoint.cgi'
* Trying [IP]...
* TCP_NODELAY set
* Connected to [IP] ([IP]) port 80 (#0)
> POST /API-Endpoint.cgi HTTP/1.1
> Host: [IP]
> User-Agent: curl/7.54.0
> Accept: */*
> Content-Type: audio/basic
> Content-Length: 15087
> Expect: 100-continue
>
< HTTP/1.1 417 Expectation Failed
< Content-Type: text/html
< Content-Length: 363
< Connection: close
< Date: Wed, 06 Jun 2018 20:34:22 GMT
< Server: lighttpd/1.4.45
<
<?xml version="1.0" encoding="iso-8859-1"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<title>417 - Expectation Failed</title>
</head>
<body>
<h1>417 - Expectation Failed</h1>
</body>
</html>
* Closing connection 0
with -H "Content-Length: 9999999" you say that your audio file is exactly 9999999 bytes long (roughly 10 megabytes), but curl reports that your file is 17456 bytes:
* upload completely sent off: 17456 out of 17456 bytes
(roughly 0.02 megabytes), so either your Content-Length header is wrong (that's my best guess), or the program feeding your audio file to curl is faulty, closing stdin prematurely.
either fix your Content-Length header, or fix the program feeding curl's stdin, hopefully that should send the entire file intact.
EDIT: oh, seems that server can't handle Expect: 100-continue, to disable that header, add the argument -H 'Expect:'
(an empty Expect header will make curl omit the header entirely, instead of sending the header empty)
... but to answer the question in the title, yeah that's the --limit-rate argument.
I'm currently setting up Varnish 4 for my domain.
My Varnish caches all files like charm, except of the DOC-request.
But I can't figure out, why the DOC-request is always a "MISS".
I hope, that you guys can help me.
All details below. If you need more, tell me.
Thank you so much for your help!
Regards,
Hermsi
Varnish-Version:
/ # varnishd -V
varnishd (varnish-4.1.3 revision 5e3b6d2)
Copyright (c) 2006 Verdens Gang AS
Copyright (c) 2006-2015 Varnish Software AS
curl request:
$ curl -s -D - "https://www.example.com" -o /dev/null
HTTP/1.1 200 OK
Accept-Ranges: bytes
Age: 0
Cache-Control: max-age=0
Content-Type: text/html; charset=UTF-8
Date: Wed, 15 Nov 2017 20:25:14 GMT
Expires: Wed, 15 Nov 2017 20:25:14 GMT
Vary: Accept-Encoding
X-Cache: MISS
X-Cache-Age:
X-Cache-Grace:
X-Cache-Ttl-Remaining:
X-Hits: 0
X-Host: example.com
X-Varnish-Cookie-Debug: Cleaned request cookies:
Transfer-Encoding: chunked
Varnishlog:
* << BeReq >> 786983
- Begin bereq 786982 pass
- Timestamp Start: 1510849297.605099 0.000000 0.000000
- BereqMethod GET
- BereqURL /
- BereqProtocol HTTP/1.1
- BereqHeader Host: www.example.com
- BereqHeader User-Agent: curl/7.26.0
- BereqHeader Accept: */*
- BereqHeader X-Forwarded-Host: www.example.com
- BereqHeader X-Forwarded-Port: 443
- BereqHeader X-Forwarded-Proto: https
- BereqHeader X-Forwarded-Server: lb-proxy-1
- BereqHeader X-Real-Ip: XXX.XX.XXX.XXX
- BereqHeader X-Forwarded-For: XXX.XX.XXX.XXX
- BereqHeader Accept-Encoding: gzip
- BereqHeader X-Device: desktop
- BereqHeader X-Varnish: 786983
- VCL_call BACKEND_FETCH
- VCL_return fetch
- BackendOpen 23 boot.default XXX.XX.XXX.XXX 80 10.42.70.133 39122
- BackendStart XXX.XX.XXX.XXX 80
- Timestamp Bereq: 1510849297.619798 0.014699 0.014699
- Timestamp Beresp: 1510849297.676958 0.071859 0.057160
- BerespProtocol HTTP/1.1
- BerespStatus 200
- BerespReason OK
- BerespHeader Date: Thu, 16 Nov 2017 16:21:37 GMT
- BerespHeader Server: Apache
- BerespHeader Cache-Control: max-age=0
- BerespHeader Expires: Thu, 16 Nov 2017 16:21:37 GMT
- BerespHeader Vary: Accept-Encoding
- BerespHeader Content-Encoding: gzip
- BerespHeader Content-Length: 3252
- BerespHeader Content-Type: text/html; charset=UTF-8
- TTL RFC 0 10 -1 1510849298 1510849298 1510849297 1510849297 0
- VCL_call BACKEND_RESPONSE
- BerespHeader X-Varnish-Cookie-Debug: Cleaned request cookies:
- BerespHeader x-url: /
- BerespHeader x-host: www.example.com
- BerespHeader X-Cache: ZIP
- BereqHeader stale-while-revalidate: max-age=0
- TTL VCL -1 7200 0 1510849298
- TTL VCL 120 7200 0 1510849298
- VCL_return deliver
- Storage malloc Transient
- ObjProtocol HTTP/1.1
- ObjStatus 200
- ObjReason OK
- ObjHeader Date: Thu, 16 Nov 2017 16:21:37 GMT
- ObjHeader Server: Apache
- ObjHeader Cache-Control: max-age=0
- ObjHeader Expires: Thu, 16 Nov 2017 16:21:37 GMT
- ObjHeader Vary: Accept-Encoding
- ObjHeader Content-Encoding: gzip
- ObjHeader Content-Length: 3252
- ObjHeader Content-Type: text/html; charset=UTF-8
- ObjHeader X-Varnish-Cookie-Debug: Cleaned request cookies:
- ObjHeader x-url: /
- ObjHeader x-host: www.example.com
- ObjHeader X-Cache: ZIP
- Fetch_Body 3 length stream
- Gzip u F - 3252 10723 80 80 25946
- BackendReuse 23 boot.default
- Timestamp BerespBody: 1510849297.677148 0.072049 0.000190
- Length 3252
- BereqAcct 328 0 328 247 3252 3499
- End
* << Request >> 786982
- Begin req 786981 rxreq
- Timestamp Start: 1510849297.605008 0.000000 0.000000
- Timestamp Req: 1510849297.605008 0.000000 0.000000
- ReqStart 10.42.115.200 35666
- ReqMethod GET
- ReqURL /
- ReqProtocol HTTP/1.1
- ReqHeader Host: www.example.com
- ReqHeader User-Agent: curl/7.26.0
- ReqHeader Accept: */*
- ReqHeader X-Forwarded-For: XXX.XX.XXX.XXX
- ReqHeader X-Forwarded-Host: www.example.com
- ReqHeader X-Forwarded-Port: 443
- ReqHeader X-Forwarded-Proto: https
- ReqHeader X-Forwarded-Server: lb-proxy-1
- ReqHeader X-Real-Ip: XXX.XX.XXX.XXX
- ReqHeader Accept-Encoding: gzip
- ReqUnset X-Forwarded-For: XXX.XX.XXX.XXX
- ReqHeader X-Forwarded-For: XXX.XX.XXX.XXX, 10.42.115.200
- VCL_call RECV
- ReqUnset X-Forwarded-For: XXX.XX.XXX.XXX, 10.42.115.200
- ReqHeader X-Forwarded-For: XXX.XX.XXX.XXX
- ReqHeader Cookie:
- ReqUnset Cookie:
- ReqHeader Cookie:
- ReqUnset Cookie:
- ReqHeader Cookie:
- ReqUnset Cookie:
- ReqHeader Cookie:
- ReqUnset Cookie:
- ReqHeader Cookie:
- ReqUnset Cookie:
- ReqHeader Cookie:
- ReqUnset Cookie:
- ReqHeader Cookie:
- ReqUnset Cookie:
- ReqHeader Cookie:
- ReqUnset Cookie:
- ReqHeader Cookie:
- ReqUnset Cookie:
- ReqHeader Cookie:
- ReqUnset Cookie:
- ReqUnset Accept-Encoding: gzip
- ReqHeader Accept-Encoding: gzip
- ReqHeader X-Device: desktop
- VCL_return hash
- VCL_call HASH
- VCL_return lookup
- Debug "XXXX HIT-FOR-PASS"
- HitPass 786980
- VCL_call PASS
- VCL_return fetch
- Link bereq 786983 pass
- Timestamp Fetch: 1510849297.677182 0.072175 0.072175
- RespProtocol HTTP/1.1
- RespStatus 200
- RespReason OK
- RespHeader Date: Thu, 16 Nov 2017 16:21:37 GMT
- RespHeader Server: Apache
- RespHeader Cache-Control: max-age=0
- RespHeader Expires: Thu, 16 Nov 2017 16:21:37 GMT
- RespHeader Vary: Accept-Encoding
- RespHeader Content-Encoding: gzip
- RespHeader Content-Length: 3252
- RespHeader Content-Type: text/html; charset=UTF-8
- RespHeader X-Varnish-Cookie-Debug: Cleaned request cookies:
- RespHeader x-url: /
- RespHeader x-host: www.example.com
- RespHeader X-Cache: ZIP
- RespHeader X-Varnish: 786982
- RespHeader Age: 0
- RespHeader Via: 1.1 varnish-v4
- VCL_call DELIVER
- RespUnset x-url: /
- RespUnset x-host: www.example.com
- RespUnset X-Cache: ZIP
- RespHeader X-Cache: MISS
- RespHeader X-Hits: 0
- RespHeader X-Host: example.com
- RespUnset Server: Apache
- RespUnset X-Varnish: 786982
- RespUnset Via: 1.1 varnish-v4
- RespHeader X-Cache-TTL-Remaining:
- RespHeader X-Cache-Age:
- RespHeader X-Cache-Grace:
- VCL_return deliver
- Timestamp Process: 1510849297.677222 0.072215 0.000040
- RespHeader Accept-Ranges: bytes
- Debug "RES_MODE 2"
- RespHeader Connection: keep-alive
- Timestamp Resp: 1510849297.677275 0.072268 0.000053
- ReqAcct 290 0 290 446 3252 3698
- End
This is my VCL:
vcl 4.0;
import std;
include "cache_invalidation.vcl";
include "custom_subroutines.vcl";
backend default {
.host = "VARNISH_BACKEND_ADDRESS"; # IP or Hostname of backend
.port = "VARNISH_BACKEND_PORT"; # Port Apache or whatever is listening
.connect_timeout = 5s; # Wait a maximum of 5s for backend connection (Apache, Nginx, etc...)
.first_byte_timeout = 10s; # Wait a maximum of 10s for the first byte to come from your backend
.between_bytes_timeout = 2s; # Wait a maximum of 2s between each bytes sent
.max_connections = 300; # Max parallel connections to our backend
}
sub vcl_recv {
# IP forwarding.
if (req.restarts == 0) {
if (req.http.X-Forwarded-For && !req.http.X-Real-IP) {
set req.http.X-Real-IP = regsub(req.http.X-Forwarded-For, ".*\b(?!10|127|172)(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}).*", "\1");
}else{
set req.http.X-Forwarded-For = req.http.X-Real-IP;
}
}
# Enforce redirect to HTTPS
if (req.http.X-Forwarded-Proto !~ "(?i)https") {
return(synth(720, "https://" + req.http.host + req.url));
}
# Remove the Google Analytics added parameters, useless for our backend
if (req.url ~ "(\?|&)(utm_source|utm_medium|utm_campaign|utm_content|gclid|cx|ie|cof|siteurl)=") {
set req.url = regsuball(req.url, "&(utm_source|utm_medium|utm_campaign|utm_content|gclid|cx|ie|cof|siteurl)=([A-z0-9_\-\.%25]+)", "");
set req.url = regsuball(req.url, "\?(utm_source|utm_medium|utm_campaign|utm_content|gclid|cx|ie|cof|siteurl)=([A-z0-9_\-\.%25]+)", "?");
set req.url = regsub(req.url, "\?&", "?");
set req.url = regsub(req.url, "\?$", "");
}
# Remove anchor or hash references
if(req.url ~ "\#"){
set req.url = regsub(req.url, "\#.*$", "");
}
# Remove incomplete query string
if(req.url ~ "\?$"){
set req.url = regsub(req.url, "\?$", "");
}
# Remove Google Analytics Cookies, our server does not need it
set req.http.Cookie = regsuball(req.http.Cookie, "__utm.=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "_ga=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "_gat=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "utmctr=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "utmcmd.=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "utmccn.=[^;]+(; )?", "");
# Remove DoubleClick offensive cookies
set req.http.Cookie = regsuball(req.http.Cookie, "__gads=[^;]+(; )?", "");
# Remove the Quant Capital cookies (added by some plugin, all __qca)
set req.http.Cookie = regsuball(req.http.Cookie, "__qc.=[^;]+(; )?", "");
# Remove the AddThis cookies
set req.http.Cookie = regsuball(req.http.Cookie, "__atuv.=[^;]+(; )?", "");
# Remove a ";" prefix in the cookie if present
set req.http.Cookie = regsuball(req.http.Cookie, "^;\s*", "");
# Are there cookies left with only spaces or that are empty?
if (req.http.cookie ~ "^\s*$") {
unset req.http.cookie;
}
if (req.url ~ "^[^?]*\.(bmp|bz2|css|doc|eot|flv|gif|gz|ico|jpeg|jpg|js|less|pdf|png|rtf|swf|txt|woff|xml)(\?.*)?$") {
unset req.http.Cookie;
return (hash);
}
if (req.url ~ "^[^?]*\.(mp[34]|rar|tar|tgz|gz|wav|zip|bz2|xz|7z|avi|mov|ogm|mpe?g|mk[av]|webm)(\?.*)?$") {
unset req.http.Cookie;
return (hash);
}
if(req.http.Accept-Encoding)
{
if(req.url ~ "\.(jpg|png|gif|gz|tgz|bz2|tbz|mp[34]|ogg)$")
{
# No point compressing these extensions
unset req.http.Accept-Encoding;
} elsif(req.http.Accept-Encoding ~ "gzip")
{
set req.http.Accept-Encoding = "gzip";
} elsif(req.http.Accept-Encoding ~ "deflate")
{
set req.http.Accept-Encoding = "deflate";
} elsif(req.http.Accept-Encoding ~ "sdch")
{
set req.http.Accept-Encoding = "sdch";
} else
{
# Unknown Algorithm
unset req.http.Accept-Encoding;
}
}
}
# At this point, it is our last chance to make some decision (set|unset|mod) about some header that the backend should see.
sub vcl_backend_fetch {
unset bereq.http.X-Varnish-SESS-ID;
}
sub vcl_hit {
# Custom Headers for debug purposes
set req.http.X-Cache-TTL-Remaining = obj.ttl;
set req.http.X-Cache-Age = obj.keep - obj.ttl;
set req.http.X-Cache-Grace = obj.grace;
}
sub vcl_backend_response {
if (bereq.url ~ "^[^?]*\.(mp[34]|rar|tar|tgz|gz|wav|zip|bz2|xz|7z|avi|mov|ogm|mpe?g|mk[av]|webm)(\?.*)?$") {
unset beresp.http.set-cookie;
set beresp.do_stream = true; # Check memory usage it'll grow in fetch_chunksize blocks (128k by default) if the backend doesn't send a Content-Length header, so only enable it for big objects
set beresp.do_gzip = false; # Don't try to compress it for storage
}
# Do not Gzip image files in Varnish
if (beresp.http.url ~ "\.(jpg|jpeg|png|gif|gz|tgz|bz2|tbz|mp3|mp4|ogg|swf)$") {
set beresp.do_gzip = false;
}
else {
set beresp.do_gzip = true;
set beresp.http.X-Cache = "ZIP";
}
# GZip the cached content if possible
if (beresp.http.content-type ~ "text") {
set beresp.do_gzip = true;
}
# If we are performing a redirection 301 / 302 from the backend and our web server and varnish instances are in the same node, apache mod_rewrite could append it's port
# typically varnish :80 apache/nginx/lighthttp :8080, so the redirect can then often redirect the end-user to a URL on :8080, where it should be :80.
if(beresp.status == 301 || beresp.status == 302) {
set beresp.http.Location = regsub(beresp.http.Location, ":[0-9]+", "");
}
# The example below is explained in detail in load_balancing.vcl as part of one of the strategies that we can take to handle Varnish Grace Mode
set bereq.http.stale-while-revalidate = regsub(beresp.http.Cache-Control, ".*stale-while-revalidate\=([0-9]+).*", "\1");
if(std.real(bereq.http.stale-while-revalidate, 0.0) > 0)
{
set beresp.grace = std.duration(bereq.http.stale-while-revalidate + "s", 10s);
} else {
set beresp.grace = 2h;
}
}
sub vcl_deliver {
# At this point this is our last change to set or unset the response headers before deliver it
if (obj.hits > 0)
{
set resp.http.X-Cache = "HIT";
} else
{
set resp.http.X-Cache = "MISS";
}
set resp.http.X-Hits = obj.hits;
# Remove www of our host in the backend response
set resp.http.X-Host = regsub(req.http.host, "^www\.", "");
# Remove PHP , Apache and Varnish versions
unset resp.http.x-powered-by;
unset resp.http.Server;
unset resp.http.X-Varnish;
unset resp.http.Via;
# Here we are ensuring to add this headers in the response (defined in the hit subroutine) for debug purposes
set resp.http.X-Cache-TTL-Remaining = req.http.X-Cache-TTL-Remaining;
set resp.http.X-Cache-Age = req.http.X-Cache-Age;
set resp.http.X-Cache-Grace = req.http.X-Cache-Grace;
}
The varnish log you are providing does not match the curl request you are doing (wrong user agent for example).
If it is still relevant, in your varnishlog you can see that the backend request contains the headers :
BereqHeader Cache-Control: no-cache
BereqHeader Pragma: no-cache
and the backend response contains the following
BerespHeader Cache-Control: max-age=0
which prevent Varnish from caching the response.
Can you share the varnish log part for the Request (not the BackendRequest) ?
Without it, my guess could be that you always get a miss because the builtin.vcl make your request a pass (you do not do any return (pass) in your conf nonetheless the backend request comes from the pass state
Begin bereq 1048612 pass
It could be triggered by these two rules in the builtin.vcl :
if (req.method != "GET" && req.method != "HEAD") {
/* We only deal with GET and HEAD by default */
return (pass);
}
if (req.http.Authorization || req.http.Cookie) {
/* Not cacheable by default */
return (pass);
}
EDIT
That's weird. You should not have any Cookie using Curl.
Try scoping your cookie handling code and use req.http.Cookie (with an uppercase C) like this :
if (req.http.Cookie) {
# Remove Google Analytics Cookies, our server does not need it
set req.http.Cookie = regsuball(req.http.Cookie, "__utm.=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "_ga=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "_gat=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "utmctr=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "utmcmd.=[^;]+(; )?", "");
set req.http.Cookie = regsuball(req.http.Cookie, "utmccn.=[^;]+(; )?", "");
# Remove DoubleClick offensive cookies
set req.http.Cookie = regsuball(req.http.Cookie, "__gads=[^;]+(; )?", "");
# Remove the Quant Capital cookies (added by some plugin, all __qca)
set req.http.Cookie = regsuball(req.http.Cookie, "__qc.=[^;]+(; )?", "");
# Remove the AddThis cookies
set req.http.Cookie = regsuball(req.http.Cookie, "__atuv.=[^;]+(; )?", "");
# Remove a ";" prefix in the cookie if present
set req.http.Cookie = regsuball(req.http.Cookie, "^;\s*", "");
# Are there cookies left with only spaces or that are empty?
if (req.http.Cookie ~ "^\s*$") {
unset req.http.Cookie;
}
}
Another thing here, in your vcl log you can see that Varnish is doing Hit-For-Pass, that is, Varnish is caching the fact that your request is not cached. If you want to truly test, you'll need to purge the content beforehands.
One last thing : your backend server (apache) responds with a Cache-Control : max-age =0 header which prevent caching, you should change that if you want to cache content.
I am discovering PDF reactor and I'd like to use it as a web service. To test a file, I use cURL
curl -v -X POST --header "Content-Type:application/xml" http://localhost:9423/service/rest/convert/async -d #test.html
Is that correct ?
test.html :
<html>
<body>
Coucou, je suis terrien.
</body>
</html>
Thank you for your help,
Cédrik
edit #1:
response from the comman above :
* About to connect() to localhost port 9423 (#0)
* Trying 127.0.0.1... connected
* Connected to localhost (127.0.0.1) port 9423 (#0)
> POST /service/rest/convert/async HTTP/1.1
> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.14.3.0 zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> Host: localhost:9423
> Accept: */*
> Content-Type:application/xml
> Content-Length: 50
>
< HTTP/1.1 400 Bad Request
< Content-Type: text/plain
< Date: Tue, 15 Dec 2015 11:47:29 GMT
< Content-Length: 307
< Server: Jetty(9.3.2.v20150730)
<
* Connection #0 to host localhost left intact
* Closing connection #0
JAXBException occurred : élément inattendu (URI : "", local : "html"). Les éléments attendus sont <{http://webservice.pdfreactor.realobjects.com/}configuration>. élément inattendu (URI : "", local : "html"). Les éléments attendus sont <{http://webservice.pdfreactor.realobjects.com/}configuration>.
When using the REST API of PDFreactor via cURL you have to send a configuration XML or JSON to the server which includes configuration for PDFreactor and your document, as described here: http://www.pdfreactor.com/product/doc_html/index.html#d0e688
A sample configuration for XML could look like this:
config.xml:
<tns:configuration xmlns:tns="http://webservice.pdfreactor.realobjects.com/">
<document><html> <body> Coucou, je suis terrien. </body> </html></document>
</tns:configuration>
You can then call the following:
curl -v -X POST --header "Content-Type:application/xml" http://localhost:9423/service/rest/convert/async.xml -d #config.xml
The output will look like the following:
* About to connect() to localhost port 9423
* Trying 127.0.0.1... connected
* Connected to localhost (127.0.0.1) port 9423
> POST /service/rest/convert/async.xml HTTP/1.1
> User-Agent: curl/7.15.5 (x86_64-redhat-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5
> Host: localhost:9423
> Accept: */*
> Content-Type:application/xml
> Content-Length: 195
>
> <tns:configuration xmlns:tns="http://webservice.pdfreactor.realobjects.com/"> <document><html><body>Coucou, je suis terrien.</body></html></document></tns:configuration>HTTP/1.1 202 Accepted
< Access-Control-Allow-Credentials: true
< Access-Control-Allow-Headers: Accept, Content-Length, content-type, Host, User-Agent
< Access-Control-Allow-Methods: GET, PUT, POST, DELETE
< Access-Control-Expose-Headers: Location
< Cache-Control: no-cache
< Date: Wed, 16 Dec 2015 16:34:19 GMT
< Location: http://localhost:9423/service/rest/progress/c2a58dbd-ef9d-4b79-87d9-079c139fe9ed
< Content-Length: 0
< Server: Jetty(9.3.2.v20150730)
* Connection #0 to host localhost left intact
* Closing connection #0
The "Location" response header contains the URL which can be used to retrieve the progress of the conversion, so you can retrieve the progress with (the ID will of course vary):
curl -v http://localhost:9423/service/rest/progress/c2a58dbd-ef9d-4b79-87d9-079c139fe9ed
This will return the conversion progress and if the conversion has finished the "Location" repsonse header will contain a new URL to retrieve the document. You can use ".pdf" to retrieve the PDF binary data or ".xml" to retrieve XML data containing the PDF as base64 encoded String, the number of pages of the document, etc.
curl -v http://localhost:9423/service/rest/document/c2a58dbd-ef9d-4b79-87d9-079c139fe9ed.pdf
I'm trying to POST json data to url, decorated with login_required, but django returns redirect to login page
DRF setup:
'DEFAULT_AUTHENTICATION_CLASSES': (
'rest_framework.authentication.SessionAuthentication',
'rest_framework.authentication.TokenAuthentication',
),
and rest_framework.authtoken in INSTALLED_APPS
I can obtain auth token via curl
$ curl -X POST -d "{\"username\" : 7, \"password\" : 1}" -H "Content-Type: application/json" http://127.0.0.1:9000/extapi/get-auth-token/
{"token":"bc61497d98bed02bd3a84af2235365d0b2b549ff"}
But when i POST to the view, decorated with login_required, django returns http 302 with Location header pointing to the login page.
$ curl -v -X POST -d '{"event":"14","user":"7","action":"1868","unit":"","value":"-1"}' -H "Content-Type: application/json" -H "Authorization: Token bc61497d98bed02bd3a84af2235365d0b2b549ff" http://127.0.0.1:9000/zk2015/events/actions/api/uservotejournal/7/
* Hostname was NOT found in DNS cache
* Trying 127.0.0.1...
* Connected to 127.0.0.1 (127.0.0.1) port 9000 (#0)
> POST /zk2015/events/actions/api/uservotejournal/7/ HTTP/1.1
> User-Agent: curl/7.35.0
> Host: 127.0.0.1:9000
> Accept: */*
> Content-Type: application/json
> Authorization: Token bc61497d98bed02bd3a84af2235365d0b2b549ff
> Content-Length: 64
>
* upload completely sent off: 64 out of 64 bytes
< HTTP/1.1 302 FOUND
* Server nginx/1.4.6 (Ubuntu) is not blacklisted
< Server: nginx/1.4.6 (Ubuntu)
< Date: Fri, 18 Sep 2015 11:14:31 GMT
< Content-Type: text/html; charset=utf-8
< Location: http://127.0.0.1:9000/accounts/login/?next=/zk2015/events/actions/api/uservotejournal/7/
< Transfer-Encoding: chunked
< Connection: keep-alive
< Vary: Cookie
< X-Frame-Options: SAMEORIGIN
< ETag: "d41d8cd98f00b204e9800998ecf8427e"
< Set-Cookie: csrftoken=G85fWrKKsIA5a2uGPIn9fS4pqKrS51jK; expires=Fri, 16-Sep-2016 11:14:31 GMT; Max-Age=31449600; Path=/
<
* Connection #0 to host 127.0.0.1 left intact
I've tried to set breakpoints in rest_framework.authentication.SessionAuthentication and rest_framework.authentication.TokenAuthentication, but they were never fired
What is wrong in my setup? Help, please.
You are not passing the Authorization in Header in the curl
curl -X POST -d "{\"username\" : 7, \"password\" : 1}" -H "Authorization: Token bc61497d98bed02bd3a84af2235365d0b2b549ff" http://127.0.0.1:9000/extapi/get-auth-token/
The point is that request.user is AnonymousUser in drf.APIView.dispatch(), but is defined as authorized user in drf.APIView.post() and other similar methods.
This differs from django: request.user is defined as authorized user in django.views.View.dispatch()
Also that is the cause, why django.contrib.auth.decorators.login_required is not compatible whith drf views.
I would like to do some redirects but involving the $args.
I am trying to to the following:
rewrite /aaa?a=1&aa=2 /bbb?b=1&bb=2 permanent;
But it does not work. The line below works fine, though
rewrite /aaa /bbb permanent;
I added those lines to my config file:
proxy_set_header x-request_uri "$request_uri";
proxy_set_header x-args "$args";
And I can see those headers:
GET /aaa?a=1&aa=2 HTTP/1.0
Host: www.example.com
x-request_uri: /aaa?a=1&aa=2
x-args: a=1&aa=2
Connection: close
User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.15.3 zlib/1.2.3 libidn/1.18 libssh2/1.4.2
Accept: */*
What I am doing wrong? is there a way to accomplish redirect taking full $request_uri in consideration?
I've got the answer on irc.freenode.net #nginx:
Mod_rewrite does not match against url-with-args only without, use if or map instead.
I managed to get it working with if:
if ( $request_uri = '/aaa?a=1&aa=2' ){
return 301 $scheme://$host/bbb?b=1&bb=2;
}
Response header:
< HTTP/1.1 301 Moved Permanently
< Server: nginx/1.0.15
< Date: Wed, 02 Jul 2014 20:05:34 GMT
< Content-Type: text/html
< Content-Length: 185
< Connection: keep-alive
< Location: http://www.example.com/bbb?b=1&bb=2
< x-uri: /aaa?a=1&aa=2