Im using Wget 1.2 64bit for Windows 8. In CMD, I used this command to download files from facebook, but it gives error that
SYSTEM_WGETRC = c:/progra~1/wget/etc/wgetrc
syswgetrc = C:\Program Files (x86)\GnuWin32/etc/wgetrc
wget: missing URL
Usage: wget [OPTION]... [URL]...
Try `wget --help' for more options.
Command
wget --header "origin: https://web.facebook.com" --header "accept-encoding: gzip, deflate, br" --header "accept-language: en-US,en;q=0.9,la;q=0.8" --header "user-agent: Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.131 Mobile Safari/537.36" --header "content-type: application/x-www-form-urlencoded" --header "accept: */*" --header "referer: https://web.facebook.com/groups/Booooooks/files/" --header "authority: web.facebook.com" --header "cookie: sb=bKUwXJpWAe1SSNfwIDDsD8XN; datr=bKUwXEDW1b_pukzYrFFidfkz; c_user=100001863466617; xs=42^%^3AdXBOhMFarGS56g^%^3A2^%^3A1557410062^%^3A19008^%^3A5962; spin=r.1000696612_b.trunk_t.1557507467_s.1_v.2_; fr=0w2O4LxVaFNhqyrg3.AWUuDmmLnUnhm_CUzvBpimPX0No.BcMIBy.Zs.FzR.0.0.Bc1fRF.AWWEKwrf; wd=1360x657; presence=EDvF3EtimeF1557525751EuserFA21B01863466617A2EstateFDsb2F1557521405829EatF1557525746283Et3F_5bDiFA2user_3a1B00937247527A2ErF1EoF5EfF8C_5dEutc3F1557525409191G557525751464CEchFDp_5f1B01863466617F403CC; act=1557525832477^%^2F1270; x-src=^%^2Fgroups^%^2FBooooooks^%^2Ffiles^%^2F^%^7Cpagelet_bluebar; pnl_data2=eyJhIjoib25hZnRlcmxvYWQiLCJjIjoiL2dyb3Vwcy9wcm9maWxlLnBocDpmaWxlcyIsImIiOmZhbHNlLCJkIjoiL2dyb3Vwcy9Cb29vb29va3MvZmlsZXMvIiwiZSI6W119 --content-disposition -c -i list.txt
If you look closely at your command:
wget --header "origin: https://web.facebook.com" --header "accept-encoding: gzip, deflate, br" --header "accept-language: en-US,en;q=0.9,la;q=0.8" --header "user-agent: Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.131 Mobile Safari/537.36" --header "content-type: application/x-www-form-urlencoded" --header "accept: */*" --header "referer: https://web.facebook.com/groups/Booooooks/files/" --header "authority: web.facebook.com" --header "cookie: sb=bKUwXJpWAe1SSNfwIDDsD8XN; datr=bKUwXEDW1b_pukzYrFFidfkz; c_user=100001863466617; xs=42^%^3AdXBOhMFarGS56g^%^3A2^%^3A1557410062^%^3A19008^%^3A5962; spin=r.1000696612_b.trunk_t.1557507467_s.1_v.2_; fr=0w2O4LxVaFNhqyrg3.AWUuDmmLnUnhm_CUzvBpimPX0No.BcMIBy.Zs.FzR.0.0.Bc1fRF.AWWEKwrf; wd=1360x657; presence=EDvF3EtimeF1557525751EuserFA21B01863466617A2EstateFDsb2F1557521405829EatF1557525746283Et3F_5bDiFA2user_3a1B00937247527A2ErF1EoF5EfF8C_5dEutc3F1557525409191G557525751464CEchFDp_5f1B01863466617F403CC; act=1557525832477^%^2F1270; x-src=^%^2Fgroups^%^2FBooooooks^%^2Ffiles^%^2F^%^7Cpagelet_bluebar; pnl_data2=eyJhIjoib25hZnRlcmxvYWQiLCJjIjoiL2dyb3Vwcy9wcm9maWxlLnBocDpmaWxlcyIsImIiOmZhbHNlLCJkIjoiL2dyb3Vwcy9Cb29vb29va3MvZmlsZXMvIiwiZSI6W119 --content-disposition -c -i list.txt
... you'll see that the --header "cookie: ... option is missing its closing ".
That's why --content-disposition -c -i list.txt is interpreted as part of the cookie header.
Add the missing " after ...MvIiwiZSI6W119".
I am trying to create a DbcpController service from nifi rest api. I am getting the proper response also but when i go to UI, The controller service is not visible.
curl -i -X POST -H 'Content-Type:application/json'
-d '{"revision":{"clientId":"a09f2359-0165-1000-bf28-1dca98f6e259","version":0},
"permissions":{"canRead":true,"canWrite":true},"bulletins":[],
"component":{"name":"Dynamic-Pool","type":"org.apache.nifi.dbcp.DBCPConnectionPool",
"bundle":{"group":"org.apache.nifi","artifact":"nifi-dbcp-service-nar","version":"1.6.0"},
"controllerServiceApis":[{"type":"org.apache.nifi.dbcp.DBCPService",
"bundle":{"group":"org.apache.nifi","artifact":"nifi-standard-services-api-nar","version":"1.6.0"}}],
"comments":"","state":"DISABLED","persistsState":false,"restricted":false,"deprecated":false,
"multipleVersionsAvailable":true,"properties":{"Database Connection URL":"jdbc:oracle:thin:#144.21.86.182:1521/ORCL.607022672.oraclecloud.internal",
"Database Driver Class Name":"oracle.jdbc.driver.OracleDriver",
"database-driver-locations":"/home/siddharth/Downloads/ojdbc7.jar","Database User":"test","Password":"***",
"Max Wait Time":"500 millis","Max Total Connections":"8","Validation-query":null},"descriptors":{ } } }'
http://localhost:9090/nifi-api/controller/controller-services/
Response :
HTTP/1.1 201 Created
Date: Mon, 03 Sep 2018 18:19:28 GMT
X-Frame-Options: SAMEORIGIN
Location: http://localhost:9090/nifi-api/controller-services/a0a93568-0165-1000-0c3b-ff9494571881
Content-Type: application/json
Vary: Accept-Encoding
Vary: User-Agent
Content-Length: 3614
Server: Jetty(9.4.3.v20170317)
If i click the location the details also comes up.
Is there any parameter in the request which i am missing.
I am not sure what "When I go to UI, the controller service is not visible" means.
When you go to the Apache NiFi UI, and click Global Menu > Controller Settings > Controller Services, you do not see any Controller Services listed -- Possibly refresh your browser and clear the cache? I don't think this is your issue
The Controller Service is listed but it is not available to use in a processor -- this is because the CS is being added to the global level, which means it is available for Reporting Tasks to reference, but not processors on the canvas. You'll need to provide a parentGroupId in the ControllerServiceDTO object in the JSON in order for the CS to be available to a component on the canvas (in that parent group). Your URL will look like this: /nifi-api/process-groups/a0f3dfb7-0165-1000-8310-4fb72628adaa/controller-services. I think this is your issue
The Controller Service is available to use in a processor but it is not enabled -- this is because you are setting the state of the CS to DISABLED. You can issue an additional command which enables the CS, or set the state to ENABLED initially
If you right click on the canvas and select Configure, you'll get a dialog for the Process Group you're in (called "NiFi Flow" by default at the root level), and there is a tab for "Controller Services", which will list the CS you have created.
You can always perform these operations in the UI and use your browser's Developer Tools panel to monitor the requests -- NiFi's UI is a consumer of the REST API, so any activity performed there can be done using the API alone.
Examples:
Add a reporting task controller service
curl 'http://localhost:8080/nifi-api/controller/controller-services' -H 'Origin: http://localhost:8080' -H 'Accept-Encoding: gzip, deflate, br' -H 'Accept-Language: en-US,en;q=0.9' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36' -H 'Content-Type: application/json' -H 'Accept: application/json, text/javascript, */*; q=0.01' -H 'Referer: http://localhost:8080/nifi/' -H 'X-Requested-With: XMLHttpRequest' -H 'Connection: keep-alive' -H 'DNT: 1' --data-binary '{"revision":{"clientId":"a0f490b8-0165-1000-a521-4d24892f12c0","version":0},"disconnectedNodeAcknowledged":false,"component":{"type":"org.apache.nifi.processors.aws.credentials.provider.service.AWSCredentialsProviderControllerService","bundle":{"group":"org.apache.nifi","artifact":"nifi-aws-nar","version":"1.8.0-SNAPSHOT"}}}' --compressed
Enable a reporting task controller service
curl 'http://localhost:8080/nifi-api/controller-services/a0f4dbe2-0165-1000-8cd2-dd50d7dba48d' -X PUT -H 'Origin: http://localhost:8080' -H 'Accept-Encoding: gzip, deflate, br' -H 'Accept-Language: en-US,en;q=0.9' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36' -H 'Content-Type: application/json' -H 'Accept: application/json, text/javascript, */*; q=0.01' -H 'Referer: http://localhost:8080/nifi/' -H 'X-Requested-With: XMLHttpRequest' -H 'Connection: keep-alive' -H 'DNT: 1' --data-binary '{"revision":{"clientId":"a0f490b8-0165-1000-a521-4d24892f12c0","version":1},"disconnectedNodeAcknowledged":false,"component":{"id":"a0f4dbe2-0165-1000-8cd2-dd50d7dba48d","state":"ENABLED"}}' --compressed
Add a controller service available to a component on the canvas
curl 'http://localhost:8080/nifi-api/process-groups/a0f3dfb7-0165-1000-8310-4fb72628adaa/controller-services' -H 'Origin: http://localhost:8080' -H 'Accept-Encoding: gzip, deflate, br' -H 'Accept-Language: en-US,en;q=0.9' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36' -H 'Content-Type: application/json' -H 'Accept: application/json, text/javascript, */*; q=0.01' -H 'Referer: http://localhost:8080/nifi/?processGroupId=root&componentIds=a0f7e36a-0165-1000-ef42-c59f18644d42' -H 'X-Requested-With: XMLHttpRequest' -H 'Connection: keep-alive' -H 'DNT: 1' --data-binary '{"revision":{"clientId":"a0f490b8-0165-1000-a521-4d24892f12c0","version":0},"disconnectedNodeAcknowledged":false,"component":{"type":"org.apache.nifi.ssl.StandardRestrictedSSLContextService","bundle":{"group":"org.apache.nifi","artifact":"nifi-ssl-context-service-nar","version":"1.8.0-SNAPSHOT"},"name":"StandardRestrictedSSLContextService"}}' --compressed
I have tried to export a text file resulted by an request to a .jsp page with Java script enabled from Oracle BI Publisher framework with below command in Unix:
curl 'http://server/xmlpserver/report.xdo&_xt=New+Template+1&_xf=text'
-H 'Accept-Encoding: gzip, deflate, sdch' -H 'Accept-Language: en-US,en;q=0.8'
-H 'Upgrade-Insecure-Requests: 1' -H 'User-Agent: Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.116 Safari/537.36'
-H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8'
-H 'Referer: http://server/xmlpserver/report.xdo'
-H 'Cookie:
JSESSIONID=qq!11!22; ORA_XDO_UI=en_US;
JSESSIONID=qq!11; oracle.uix=0^^GMT+3:00^p'
-H 'Connection: keep-alive'
--compressed -o test.txt
This command is 'copy as bash curl command' from Google Chrome debugger and work it but syntax command use id JSESSION in cookie who will expire.
I need to login on this .jsp page to generate cookie with JSESSION=qq!11!22 and JSESSION=qq!11 (see above cUrl syntax from Google Chrome).
For this I have used command:
curl -c cookies.txt -d "user=user&pass=pass" http://server/xmlpserver/login.jsp, and was generated only one "JSESSIONID=qq!11;".
I need to generate one more JSESSIONID=qq!11!22.
Thanks!
I am developing a module for a web application. To trigger this module, I need to submit some data to the server. For simple forms, Copy as cURL in the Chrome Developer Tools works fine (using curl from msys[git]), but for post requests with multipart/form-data, the copied string is neither usable in the windows shell (cmd) nor with bash (form msys); the copied text is similar to:
curl "http://myserver.local" -H "Origin: http://wiki.selfhtml.org" -H "Accept-Encoding: gzip, deflate" -H "Accept-Language: de-DE,de;q=0.8,en-US;q=0.6,en;q=0.4" -H "User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.130 Safari/537.36" -H "Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryntXdlWbYXAVwCIMU" -H "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8" -H "Cache-Control: max-age=0" -H "Referer: http://wiki.selfhtml.org/extensions/Selfhtml/frickl.php/Beispiel:HTML_form-Element1.html" -H "Connection: keep-alive" --data-binary "------WebKitFormBoundaryntXdlWbYXAVwCIMU"^
"Content-Disposition: form-data; name=""area"""^
"multi"^
"line"^
"------WebKitFormBoundaryntXdlWbYXAVwCIMU--"^
"" --compressed
Is there any way I could use this or convert it to something usable?
Chrome, as well as the other browsers actually, do a rather poor job of translating multi-part formposts into curl command lines.
A much more convenient curl command line would not use --data-binary for that, it would use --form. And then you want one --form per input field.
In your case, it probably would look something like (backslashes inserted here
for visibility):
curl "http://myserver.local" \
--compressed \
-H "Origin: http://wiki.selfhtml.org" \
-A "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.130 Safari/537.36" \
-H "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8" \
-H "Cache-Control: max-age=0" \
-e "http://wiki.selfhtml.org/extensions/Selfhtml/frickl.php/Beispiel:HTML_form-Element1.html" \
-F area=[contents]
I left the [contents] in there, but it should be replaced with what you actually want in the area field. You could also pass it from a file if you prefer to.
I removed two unnecessary -H uses, and I replaced two to use the direct curl options.
h2c - headers to curl
Advice for the future: figure out the exact HTTP header trace you want to reproduce with a curl command line and paste it over at https://curl.se/h2c/ .
I have a curl command in a shell file that looks like below. All I do through another application is rename this file and then I execute the shell file. However, it is very temperamental. For no obvious reason it stops being able to read the variable and treats as blank. Usually (but not always) when I recreate the file it does work. I'm using Notepad++ so I don't think there are spacing or line break issues.
Thanks for your help
vr_id=`basename $0 .sh`
curl "https://www.test.com/control.php/info/loads/? event=showList&wo_id=${vr_id:7}&id=all::0" -H 'Cookie: filemsg=1; var_db=xxxxxxxxxxxxxxxx; __utma=111111.11111111.11111.11111111.1111116.12; __utmb=11111.1.10.1111111; __utmc=1111111; __utmz=111111.111111.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none)' -H 'Accept-Encoding: gzip,deflate,sdch' -H 'Host: www.test.com' -H 'Accept-Language: en-US,en;q=0.8' -H 'User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/29.0.1547.57 Safari/537.36' -H 'Accept: text/javascript, text/html, application/xml, text/xml, */*' -H "Referer: https://www.test.com/control.php/info/loads/?wo_id=${vr_id:7}" -H 'X-Requested-With: XMLHttpRequest' -H 'Connection: keep-alive' -H 'X-Prototype-Version: 1.7' --compressed