I am working on a simple bash script to download images from the website Tumblr. The idea is to use read to get login info from the user, and wget --post-data to log in, and this is what I have:
read -p "Tumblr login email: " EMAIL
read -p "Tumblr login password: " PASSWRD
wget --user-agent=Mozilla/5.0 --save-cookies cookies.txt --post-data 'email=$EMAIL&password=$PASSWRD' --no-check-certificate https://www.tumblr.com/login
However, it is sending "$EMAIL" and "$PASSWRD" instead of the strings for the variables, is there any way to get it to send values that have been inputted by the user?
change:
--post-data 'email=$EMAIL&password=$PASSWRD'
to:
--post-data="email=$EMAIL&password=$PASSWRD"
bash manual about Quoting: http://www.gnu.org/software/bash/manual/bashref.html#Quoting
important:
Do not use :
--header="Content-Type: text/xml"
together with --post-data. It will override
--header="Content-Type: application/x-www-form-urlencoded"
issued by wget. Post-data will not be received by HttpServlet
Related
I use the below curl command to disable the alert which works fine.
curl -k -u admin:password https://URL -X POST
But i am trying to hide the username and password in the below shell script but getting unauthorized exception,
SCRIPT_DIR=/tmp
USER=$(cat $SCRIPT_DIR/.nonprodusr.txt)
PWD=$(cat $SCRIPT_DIR/.nonprod.txt)
curl -k -u $USER:$PWD https://url -X POST
You can use a netrc file : https://everything.curl.dev/usingcurl/netrc
/tmp/netrc :
login admin
password Passw0rd
And use it with this option :
curl -k --netrc-file /tmp/.netrc https://url -X POST
One can use a configuration file to add user/password as well as other configurations.
curl --config /home/me/curl-configuration.txt <url>
Contents of "/home/me/curl-configuration.txt":
--user <username>#<password>
I am trying to use wget to download the yahoo stock history file,
https://query1.finance.yahoo.com/v7/finance/download/ARDM?period1=1504030392&period2=1506708792&interval=1d&events=history&crumb=TKe9axyOsuR
but it always reports an error code":
Username/password authentication Failed.
However, if I visit that stock page
https://finance.yahoo.com/quote/ARDM/history?p=ARDM
I can click the datafile link, and download the CSV file without giving any user or password.
Any clue?
When you are going in via the browser the system is using your cached yahoo login details to go straight to the information using your cookie.
Set your authentication in your wget by using the following:
wget --user user --password pass http://example.com/
If you are not logging in to the server as seems to be the case here then you could try using two WGET using the first one to grab a cookie and the second one to download the data as follows:
wget -qO- --keep-session-cookies --save-cookies cookies.txt https://finance.yahoo.com/quote/ARDM/history?p=ARDM
followed by
wget -qO- --load-cookies cookies.txt https://query1.finance.yahoo.com/v7/finance/download/ARDM?period1=1504030392&period2=1506708792&interval=1d&events=history&crumb=TKe9axyOsuR
I'm trying to download invoice with bash script(wget), because I need to save invoices to my server automatically when order is made. I looked for other solutions but that didnt work, like downloading invoice with changing tcpdf.php.The post was up here: https://www.prestashop.com/forums/topic/465729-automatically-generate-invoices-in-a-folder/
So first I need to login to prestashop adminpage I suppose. Logging in got me confused, because how do I know when I'm logged in? This what I did:
wget --save-cookies cookies.txt --keep-session-cookies --post-data 'email=example#gmail.com&passwd=example' http://www.example.ee/admin/index.php?controller=AdminLogin&token=5a01dc4e606baa6c26e95ddea91d3d14
After saving my cookie, I tried to download the invoice by:
wget --load-cookies cookies.txt -e robots=off -A pdf -r -l1 http://www.example.ee/admin/index.php?controller=AdminPdf&token=35b276c05aa6r5eb516437a8d534eb64&submitAction=generateInvoicePDF&id_order=3197
But I get an error: Deleting, www.example.ee/admin/index.php?controller=AdminPdf, since it should be rejected
I think It's because I'm still not logged in?
It would be very nice that if it's possible without wget.
PS changed my website to example,etc
I want to create an alias to get a file using cURL which prompts for the:
URL
output path
username and password of a proxy
How would I be able to do this interactively rather than hardcoding everything?
The following works as expected:
alias proxyget='read userid?"Userid: "; read passwd?"Password: "; read url?"URL: "; read local?"Local: "; curl -x 1.2.3.4:8080 -U $userid:$passwd $url -o $local;'
References
Using cURL to automate HTTP jobs
curl - Manual
read Man Page - Bash - SS64.com
I want to aoutomate logging in to a website and downloading a file.
Since it is session based and I have to send a token from inside the html with the login credentials as POST, I was wondering if there was a solution to this in bashscript?!
Thanks in advance!
curl may be the answer to your problem. It supports saving and loading cookies to / from a file.
The --cookie-jar <file> option causes curl to dump cookies to the given file after the request.
And the --cookie <file> option causes curl to load cookies from the given file before the request.
So you can use these options like this:
curl --cookie cookies.txt --cookie-jar cookies.txt http://www.example.com/
Posting can be done in many ways, but the --data-urlencode option is the easier:
curl --data-urlencode "username=john.doe" --data-urlencode "password=dummy" --cookie [...] http://www.example.com/