In an Applescript I am trying to pass on a URL that I receive as an argument to a do shell script command to use it with curl.
With regular characters the procedure works fine, but as soon as my argument contains special characters like Umlauts, it gets all funky.
curl does download something, but replaces the letter Ü with à etc., which of course will not get me the correct result.
What do I need to do, to get this to work? I am neither very skilled with Applescript nor with encoding issues.
My setup at the moment is as follows:
set download_URL to item 1 of arguments
do shell script "curl " & download_URL & " > targetFile.html"
Some examples of what happens:
Äquivozität ---> Ãquivozität
Ökolikör ---> Ãkolikör
Übermütigkeit ---> Ãbermütigkeit
Schweißfuß ---> SchweiÃfuÃ
Which makes my confusion even greater. All Ä, Ö, Ü and ß render as Ã, but both in the editing mask here and in the one of the site in question they render as shown in this image.
Also, through some amateurish digging in the html-File, I figured out that instead of the letter Ü, I would need to pass the letters %C3%9C. So the whole procedure does work, if I pass %C3%9Cbermut instead of Übermut. However, I would of course like to avoid creating a translation table for all diacritics.
Can somebody figure out, what specific encoding problem is happening here?
After some more researching, I found out that what I need to urlEncode my string. That way, the letter Ü will be replaced with %C3%9C and it works for my purposes.
Applescript does not seem to support this natively, but one can use php to do the conversion. I found the method here: https://discussions.apple.com/message/9801376#9801376
So, in my case I used it like this:
set keyword to item 1 of arguments
set encodedKeyword to do shell script "php -r 'echo trim(urlencode(" & "\"" & keyword & "" & "\"));'"
do shell script "curl https://www.myUrl.com/" & encodedKeyword & ".html > targetFile"
This way, it works for me.
In case there is a better way - maybe something that works in Applescript directly - feel free to post another answer, then I'll change the accepted answer.
Related
I am trying to download a file from this link with wget.
https://sfirmware.com/downloads/downloader.php?fileid=257457&hash=458cd40aa7824c3d25fe096c0b01d4b7
I am aware that & is a special character in shell environment.
So far i've tried double quote,single quote ,putting %26 replacing '&' (as per some suggestion) but the solution doesn't seem to work with this error-
Hash not exist in db!Back to Home Page
It seems the hash number from the link is ignoring while requesting.
How do I modify the code to download this file with wget?
I ran into a small problem when trying to create a shell line in groovy that is stapled from many variables.
Here is what it looks like:
sh("""
java -jar Report.jar settings.xml "$startTest" "$durationTest" "$opt" empty iReport:gReport:aReport:pReport="$iReport":"$gReport":"$aReport":"$pReport" $reportName
""")
The point is that the command in the shell must run along with the ", which here frames the values of the variables.
However, no matter how much I tried to screen them:
Put \ or \\ in front of "
Convert the string to 'text' + varibale + 'text'
Put the whole shell line in ' and leave " as is, or with the first point taken into account.
None of this works.
Here is an example output:
java -jar Report.jar settings.xml '2022-07-04 11:00' 01:00 'Offset=01:00;' empty iReport:gReport:aReport:pReport=true:true:true:true App.result
You may notice that for some reason two variables are surrounded by quotation marks ' for themselves, although this is not specified anywhere, which is probably the root of the problem.
Because if me use \", the quotes appear, but not everywhere, in general, me get some mishmash of " and '.
To understand, this is how a groovy file works in the Jenkins pipeline.
Hence my question, under what circumstances do variables start framing themselves '' and how to get rid of it?
Or maybe someone has a different solution to this problem, I would appreciate it.
I have a cURL request as follows.
$(curl --request PUT --upload-file "<path to catalog file on your local machine>" "<presigned URL>")
Let's say that I have to upload a bin/test.txt file with the presigned URL being https://www.someurl.com
I execute the command in my terminal
curl --request PUT --upload-file "bin/test.txt" "https://www.someurl.com" and it works fine.
How do I write a piece of Golang which does the same? I have tried
cmd := exec.Command("curl", "--request", "PUT", "--upload-file", fmt.Sprintf("\"%s\"", catalogPath), fmt.Sprintf("\"%s\"", presignedURL))
err = cmd.Run()
but found no success.
I see one obvious problem preventing that curl call from working properly, one quite possible and another one also possible.
The obvious problem is that string quoting — such as executing curl … --upload-file "bin/test.txt" … — in interpreted by the shell which is to execute the command. Quoting — using either double or single quotes — is used to inhibit interpreting of otherwise special characters by the shell; chiefly it's used to prevent the shell from splitting a string into separate "words" on whitespace characters or their series.
The key takeaway is that the command run by the shell after it's fully parsed the command to be executed (and interpreted the quotes) does not "see" these quotes because they are removed by the shell.
os/exec.Cmd calls the specified program directly and does not "pass it through" the shell. Hence if you include double quotes into the command-line parameters of the program to execute, they are passed to that program, unchanged. This means, curl were to try to find the file named test.txt" located in the directory named "bin — which is most probably not what you expected.
The same applied to the URL.
The second — possible — problem is that your call relies on the current directory of your Go program because you pass a relative path to curl.
This might or might not be a problem but you might check this anyway.
The third problem is that you might want to pass your URL through the "percent escaping" algorithm before passing it to curl.
You might look at PathEscape and QueryEscape functions of the net/url package.
Two pieces of advice follow.
First, I would try very hard not to call out to curl to perform such a ridiculously simple task. Go has excellent support for making HTTP requests (and serving them, FWIW) in its standard library, and PUTting a file is really a no-brainer with a solutions googleable in, like, five minutes.
Second, if, for some reason, you intend to stick with calling curl, consider passing it some options to make it fail loudly on errors — otherwise you're doomed to be in that „but found no success” situation in your attempts. For instance, consider passing curl the -s and -S command line options (together).
that's not how you quote shell arguments, that would break if your argument starts with or ends with \ or ", the proper way to quote shell arguments on unix would be
func quoteshellarg(str string) string {
if strings.Contains(str, "\x00") {
panic("argument contains null bytes, it is impossible to escape null bytes in shell arguments!")
}
return "'" + strings.ReplaceAll(str, "'", "'\\''") + "'"
}
and with that, just
cmd := exec.Command("curl --request PUT --upload-file " + quoteshellarg(catalogPath)+ " " + quoteshellarg(presignedURL));
... at least that's how to do it on unix systems. as for how to do it on Windows, it seems nobody knows for sure, not even Microsoft
I'm trying to create a mask and use the bitwise operator "&" to compare to another variable and see the output. Let there be code:
mask=00000
mesk=00010
mosk=$mask&$mesk
echo $mosk
echo meec
I'm trying to expand this functionality to be able to have more characters (different error/success codes), but those lines just don't work: Executing the script will print an empty line, then "meec".
I came from an object oriented programming background, and although I've read through several documents on this subject, it seems there's something I'm missing.
Any help would be appreciated.
Edit: For some reason, turns out the code doesn't work, it says "command 00010 not found" >_>
It's because usually the & character in the shell is the modifier to put a command in the background.
You have to use Arithmetic Expansion of Bash (for example) for it to work:
mosk=$(($mask & $mesk))
I noticed that cmd seems to accept some characters at the ends of commands. for example all of the following function correctly:
cls.
cls;
cls(
cls\
cls+
cls=
cls\"whatever"
cls\$
cls\#
and these do not:
cls'
cls$
cls)
cls-
cls#
cls\/
Does anybody know why this happens?
Thanks in advance.
It depends on the batch parser.
;,= are general batch delimiters, so you can append/prepend them to the most commands without effect.
;,,= ,=; echo hello
;,cls,;,,
The . dot can be appended to the most commands, as the parser will try to find a file named cls (without extension) cls.exe cls.bat, and when nothing is found then it takes the internal command.
The opening bracket is also a special charcter that the parser removes without error.
The \ backslash is used as path delimiter, so sometimes it works but sometimes you could change even the command.
cls\..\..\..\windows\system32\calc.exe