Due to some odd circumstances I have the necessity to use uriQuery() in a Power Automate flow in order to extract the query string from an url.
This works as expected in most circumstances, except when the url contains special characters like accented letters, for example
http://www.example.com/peppers/Jalapeño/recipe #1.docx
In such cases the call triggers an error and the exception message shows a (partially) encoded version of my url (why?).
The template language function 'uriQuery' expects its parameter to be a well-formed absolute URI. The provided value was '......'
Obviously the url was indeed a well-formed, absolute URI.
Since the error only triggers when the url contains special characters I assumed that I had to encode the value before calling uriQuery(), yet nothing I tried seems to work (for example encodeUriComponent() ). And as expected nothing I could find on the web mentioned a similar issue.
As a last attempt I am asking here - does uriQuery() support this use-case? And if it does... how?
I am using the html/template functionality to assemble a page and on of the variables I'm supplying to the template is URI in the form "/some/path/etc" which is used as a parameter to a JS function called in a onClick="..".
No matter what, the string used in this configuration will be escaped with backslashes : "\/some\/path\/etc"
As you can see in the playground example below, I tried all the .HTML(), .JS() etc. functions but nothing seems to stop the escaping.
See Go Playground example here: https://play.golang.org/p/2gdghTpQHKP
How can I get this URI "as is" into the template result?
Thanks to mkopriva for his comment.
As far as I could see there is no way (as mkopriva mentioned) to handling a HRML attribute value fragment in a Go HTML template.
So the options are:
Leave it as is (it seems that at least in my use case the URI even works in the further processing with the escaped forward slashes)
first concatenate the complete attribute, so that the "HTML Attribute" way will accept it
Write a construction function that takes parts and assembles the final attribute value inside the template execution
Hardcode the value in some form
My online research seems to show that firstnames and lastnames should not be heavily validated, to accommodate the variety of names out there. In fact, people have even advocated no validation altogether for the names. However, the possibility of xss attacks via the input fields make me worried. I checked the google naming guidelines, and they seem pretty relaxed and allow unicode characters as well as stuff like "%$#^&*...." !!
So, what would be the best approach to take, and how do I balance this out ?
ps - I don't intend to spark a debate here. I am genuinely confused and need help understanding the best approach to take !
Validation and XSS are two very different concepts. You cannot balance them. You cannot "sometimes allow XSS". You also do not want to allow input that does not make sense, or that you can't use. If you require an email for something, you can allow an user to enter "mailme at gmail dot com", but if you do not know how to parse this, then there is no point in allowing this as an input in the first place.
When you talk about validating a 'first name field', you ask yourself: "What kind of data do I want to accept in this field, and what kind of data do I not want to accept in this field?". I am not aware of a language where "%" can be part of a first name, so it is probably a safe bet to disallow this character. You have to tackle this problem alone, without even thinking about XSS. If a character, or a sequence of characters, does not make sense as a value for the thing you want to know, you should not include it. If a character does make sense to include, you should not decide otherwise because it has some special meaning.
XSS is the problem where incorrectly escaped (user) input is returned to the browser, allowing a possible attacker to load/run third-party scripts. It has nothing to do with validation. If the character "a" is potentially unsafe, would you disallow it from the first name field? The solution can be found in the definition: The problem exists if, and only if, the user input is incorrectly escaped.
Think about how you are going to sent back this data to the user. I take as example an input field: <input value="" />, but if you were going to put it in a textarea for example, you would need to alter your data for that. Inserting it between a <div></div> tag would require something entirely different again, and inserting it inside a script that is in <script></script> tags would require something different than all the previous things. There is no one-size-fits-all-solution.
For the input field example, find out what characters have a special meaning in this input field. The delimiter of the value attribute (" in value="") is one of the characters that has a special meaning. If there are any other special characters, you find them in accompanying documentation. You have to escape such characters. Escaping is the act of removing the special meaning from a character. How you do that can be found in the accompanying documentation. In case of an input element in html, you'll need to turn the special character into it's entity-form (" would become "). Php provides built-in functions to do this, but you should always be wary of what such a function actually does and if this function actually gives the desired output for every use-case.
tl;dr There is no balance. You use validation on a field to get the data you actually want. If you want to present this data to the user, you have to escape the data for the special case where you want to display this data.
Example: Let's look at the following case. We have a textarea. We allow the characters a-z, <, >, (, ), {, }, / and ; in any order. If the textarea contains other characters we consider it invalid. If the textarea is valid, we put the characters in the textarea between <div> and </div> in the html document.
From the definition above, you can derive that asdf is a valid input and that <scri (random nonsense to bypass faulty proxy) pt>alert();</sc (more random nonsense) ript> is also a valid input, but 123 isn't. That is the definition. The logic that handles validation should flawlessly discriminate between those two things. You probably notice that the second valid input may provide a problem, but that is of no concern to the validate function. The validate function only checks if the text matches the description of what we consider valid input.
If the text in the textarea is valid, the definition says we should put it between the div tags. This is where you start worrying about XSS. There are some characters, namely the < and > character that have a special meaning in html. Because they are valid input, we should remove their special meaning when we insert them in the html. If the textarea is invalid, we can't do anything. We would display a descriptive error message how it should be improved.
The pseudo-implementation below shows what I try to explain above. In a real-life application that communicates with the server, the server should do validation too, but it should show how both concepts are separated and should allow you to test things.
$('#billy').on('click', function(e) {
if (validate($('#txt').val())) {
$('#status').text("The textarea is valid. The contents have been inserted as html in the page.");
$('#result').html($('#txt').val());
} else {
$('#status').text("The textarea is INVALID. It contains characters we don't want.");
}
});
$('#betty').on('click', function(e) {
if (validate($('#txt').val())) {
$('#status').text("The textarea is valid. The contents have been inserted as html in the page.");
$('#result').html(escapeforhtml($('#txt').val()));
} else {
$('#status').text("The textarea is INVALID. It contains characters we don't want.");
}
});
function validate(txt) {
return txt.match(/^[a-z{}\/\(\)<>;]*$/);
}
//We know only a limited amount of characters can be inserted.
//From those, < and > are the only characters that have a special
//meaning.
function escapeforhtml(txt) {
return txt.replace(/</g, "<").replace(/>/g, ">");
}
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.9.1/jquery.min.js"></script>
<div id="status"></div>
<textarea id="txt" cols="60" rows="10"></textarea>
<br/>
<input type="button" id="billy" value="Do something">
<input type="button" id="betty" value="Do something while ESCAPING">
<p>Result:</p>
<div id="result"></div>
In my data model myVar contains <b>hello</b> and when I bring it like this ${myVar} the output I get is literally <b>hello</b> rather than hello.
Any idea how to correct this?
Certainly you have HTML escaping on, so try <#noescape>${myvar}</#noescape>.
Update: Since 2.3.24 a new kind of auto-escaping was introduced, which doesn't use #escape, and hence nor #noescape. When that's used, write ${myvar?no_esc} to avoid escaping. Or, you can put the value into the data-model already as a TemplateHTMLOutputModel (created with HTMLOutputFormat.fromMarkup(myString)), and then FreeMarker will know without ?no_esc that it need not be escaped.
I am creating a Joomla component and one of the pages contains a form with a text input for an email address.
When a < character is typed in the input field, that character and everything after is not showing up in the input.
I tried $_POST['field'] and JFactory::getApplication()->input->getCmd('field')
I also tried alternatives for getCmd like getVar, getString, etc. but no success.
E.g. John Doe <j.doe#mail.com> returns only John Doe.
When the < is left out, like John Doe j.doe#mail.com> the value is coming in correctly.
What can I do to also have the < character in the posted variable?
BTW. I had to use & lt; in this question to display it as I want it. This form suffers from the same problem!!
You actually need to set the filtering that you want when you grab the input. Otherwise, you will get some heavy filtering. (Typically, I will also lose # symbols.)
Replace this line:
JFactory::getApplication()->input->getCmd('field');
with this line:
JFactory::getApplication()->input->getRaw('field');
The name after the get part of the function is the filtering that you will use. Cmd strips everything but alphanumeric characters and ., -, and _. String will run through the html clean tags feature of joomla and depending on your settings will clean out <>. (That usually doesn't happen for me, but my settings are generally pretty open to the point of no filtering on super admins and such.
getRaw should definitely work, but note that there is no filtering at all, which can open security holes in your application.
The default text filter trims html from the input for your field. You should set the property
filter="raw"
in your form's manifest (xml) file, and then use getRaw() to retrieve the value. getCmd removes the non-alphanumeric characters.