I am trying to use hclwrite to generate .tf files.
According to the example in hclwrite Example, I can generate variables like foo = env.PATH, but I don't know how to generate more forms of expressions. For example, the following.
stage = "prod"
foo = "hello${var.stage}"
when i set foo with
SetAttributeValue("foo", cty.StringVal("hello${bar}"))
i get
foo = "hello$${bar}"
The hclwrite tool currently has no facility to automatically generate arbitrary expressions. Its helper functions are limited only to generating plain references and literal values. SetAttributeValue is the one for literal values, and so that's why the library correctly escaped the ${ sequence in your string, to ensure that it will be interpreted literally.
If you want to construct a more elaborate expression then you'll need to do so manually by assembling the tokens that form the expression and then calling SetAttributeRaw instead.
In the case of your example, it looks like you'd need to generate the following eight tokens:
TokenOQuote with the bytes "
TokenQuotedLit with the bytes hello
TokenTemplateInterp with the bytes ${
TokenIdent with the bytes var
TokenDot with the bytes .
TokenIdent with the bytes stage
TokenTemplateSeqEnd with the bytes }
TokenCQuote with the bytes "
The SetAttributeValue function is automating the simpler case of generating three tokens: TokenOQuote, TokenQuotedLit, TokenCQuote.
You can potentially automate the creation of tokens for the var.stage portion of this expression by using TokensForTraversal, which is what SetAttributeTraversal does internally. However, unless you already have a parsed hcl.Traversal representing var.stage, or unless you need things to be more dynamic than you've shown in practice, I expect that it would take more code to construct that input traversal than to just write out the three tokens literally as I showed above.
Related
I'm trying to write a short program (short enough that it has a simple main function). First, I should list the dependency in the cargo.toml file:
[dependencies]
passwords = {version = "3.1.3", features = ["crypto"]}
Then when I use the crate in main.rs:
extern crate passwords;
use passwords::hasher;
fn main() {
let args: Vec<String> = std::env::args().collect();
if args.len() < 2
{
println!("Error! Needed second argument to demonstrate BCrypt Hash!");
return;
}
let password = args.get(1).expect("Expected second argument to exist!").trim();
let hash_res = hasher::bcrypt(10, "This_is_salt", password);
match hash_res
{
Err(_) => {println!("Failed to generate a hash!");},
Ok(hash) => {
let str_hash = String::from_utf8_lossy(&hash);
println!("Hash generated from password {} is {}", password, str_hash);
}
}
}
The issue arises when I run the following command:
$ target/debug/extern_crate.exe trooper1
And this becomes the output:
?sC�M����k��ed from password trooper1 is ���Ka .+:�
However, this input:
$ target/debug/extern_crate.exe trooper3
produces this:
Hash generated from password trooper3 is ��;��l�ʙ�Y1�>R��G�Ѡd
I'm pretty content with the second output, but is there something within UTF-8 that could cause the "Hash generat" portion of the output statement to be overwritten? And is there code I could use to prevent this?
Note: Code was developed in Visual Studio Code in Windows 10, and was compiled and run using an embedded Git Bash Terminal.
P.S.: I looked at similar questions such as Rust println! problem - weird behavior inside the println macro and Why does my string not match when reading user input from stdin? but those issues seem to be issues with new-line and I don't think that's the problem here.
To complement the previous, the answer to your question of "is there something within UTF-8 that could cause the "Hash generat" portion of the output statement to be overwritten?" is:
let str_hash = String::from_utf8_lossy(&hash);
The reason's in the name: from_utf8_lossy is lossy. UTF8 is a pretty prescriptive format. You can use this function to "decode" stuff which isn't actually UTF8 (for whatever reason), but the way it will do this decoding is:
replace any invalid UTF-8 sequences with U+FFFD REPLACEMENT CHARACTER, which looks like this: �
And so that is what the odd replacement you get is: byte sequences which can not be decoded as UTF8, and are replaced by the "replacement character".
And this is because hash functions generally return random-looking binary data, meaning bytes across the full range (0 to 255) and with no structure. UTF8 is structured and absolutely does not allow such arbitrary data so while it's possible that a hash will be valid UTF8 (though that's not very useful) the odds are very very low.
That's why hashes (and binary data in general) are usually displayed in alternative representations e.g. hex, base32 or base64.
You could convert the hash to hex before printing it to prevent this
Neither of the other answers so far have covered what caused the Hash generated part of the answer to get overwritten.
Presumably you were running your program in a terminal. Terminals support various "terminal control codes" that give the terminal information such as which formatting they should use to output the text they're showing, and where the text should be output on the screen. These codes are made out of characters, just like strings are, and Unicode and UTF-8 are capable of representing the characters in question – the only difference from "regular" text is that the codes start with a "control character" rather than a more normal sort of character, but control characters have UTF-8 encodings of their own. So if you try to print some randomly generated UTF-8, there's a chance that you'll print something that causes the terminal to do something weird.
There's more than one terminal control code that could produce this particular output, but the most likely possibility is that the hash contained the byte b'\x0D', which UTF-8 decodes as the Unicode character U+000D. This is the terminal control code "CR", which means "print subsequent output at the start of the current line, overwriting anything currently there". (I use this one fairly frequently for printing progress bars, getting the new version of the progress bar to overwrite the old version of the progress bar.) The output that you posted is consistent with accidentally outputting CR, because some random Unicode full of replacement characters ended up overwriting the start of the line you were outputting – and because the code in question is only one byte long (most terminal control codes are much longer), the odds that it might appear in randomly generated UTF-8 are fairly high.
The easiest way to prevent this sort of thing happening when outputting arbitrary UTF-8 in Rust is to use the Debug implementation for str/String rather than the Display implementation – it will output control codes in escaped form rather than outputting them literally. (As the other answers say, though, in the case of hashes, it's usual to print them as hex rather than trying to interpret them as UTF-8, as they're likely to contain many byte sequences that aren't valid UTF-8.)
I'm writing a serialize method that converts a Tree into a string for storage. I was looking for a delimiter to use in the serialization and wasn't sure what to use.
I can't use , because that might exist as a data value in a node. e.g.
A
/ \
B ,
would serialize to A, B, ,, and break my deserialization method. Can I use non-printable ASCII characters, or should I just guess what character(s) are unlikely to show up as input and use those as my delimiters?
Here's what my serialize method looks like, if you're curious:
def serialize(root)
if root.nil?
""
else
root.val + DELIMITER +
serialize(root.left) + DELIMITER +
serialize(root.right)
end
end
There are several common methods I can think of:
Escaping: you define an escape symbol that "escapes" from the "special" interpretation. Think about how \ acts as an escape character in Ruby string literals.
Fixed Fields / Length Encoding: you know in advance where a field begins and ends. (Fixed fields are basically a special-case of length encoding where you can leave out the length because it is always the same.)
Example for escaping:
def serialize(root)
if root.nil?
""
else
"#{escape(root.val)},#{serialize(root.left)},#{serialize(root.right)}" # using ,
end
end
private def escape(str) str.gsub('\', '\\').gsub(',', '\,') end
Example for length encoding:
def serialize(root)
if root.nil?
"0,"
else
"#{root.val.size},#{root.val}#{serialize(root.left)}#{serialize(root.right)}" # using length encoding
end
end
Any , you find within size characters belongs to the value. Fixed fields would basically just concatenate the values and assume that they are all the same fixed length.
You might want to look at how existing serialization formats handle it, like OGDL: Ordered Graph Data Language, YAML: YAML Ain't Markup Language, JSON, CSV (Character-Separated Values), XML (eXtensible Markup Language).
If you want to look at binary formats, you can check out Ruby's Marshal format or ASN.1.
Your idea of finding a seldom-used character is good, even if you use escaping, you will still need less escaping with a less used character. Just imaginee what it would look likee if 'ee' was the eescapee characteer. However, I think using a non-printable character goes too far: unless you specifically want to design a binary format (such as Ruby's Marshal, Python's Pickle, or Java's Serialization), "less debuggability" (i.e. debugging by simply inspecting the output with less) is a nice property to have and one that you should not give up easily.
In Ruby is there any way that data added to a string with interpolation can terminate the string? For example something like:
"This remains#{\somekindofmagic} and this does not" # => "This remains"
I'm assuming not but I want to be sure that doing something like
something.send("#{untrusted_input}=", more_untrusted_input)
doesn't actually leave some way that the interpolated string could be terminated and used to send eval.
Not possible with input string data AFAIK. Ruby Strings can contain arbitrary binary data, there should be no magic combination of bytes that terminates a String early.
If you are worried about "injection" style attacks on Ruby strings, then this is generally not easy to achieve if input is in the form of external data that has been converted to a string (and your specific concern about having an eval triggered cannot occur). This style of attack relies on code that passes an input string into some other interpreter (e.g. SQL or JavaScript) without properly escaping language constructs.
However, if String parameters are coming in the form of Ruby objects from untrusted Ruby code in the same process, it is possible to add side-effects to them:
class BadString
def to_s
puts "Payload"
"I am innocent"
end
end
b = BadString.new
c = "Hello #{b}"
Payload
=> "Hello I am innocent"
Edit: Your example
something.send("#{untrusted_input}=", more_untrusted_input)
would still worry me slightly, if untrusted_input really is untrusted, you are relying heavily on the fact that there are no methods ending in = that you would be unhappy to have called. Sometimes new methods can be defined on core classes due to use of a framework or gem, and you may not know about them, or they may appear in later versions of a gem. Personally I would whitelist allowed method names for that reason, or use some other validation scheme on the incoming data, irrespective of how secure you feel against open-ended evals.
Strings in ruby are internally handled as an array of bytes on the heap and an integer that holds the length of the string. So while in C a NUL byte (\0) terminates a string, this can not happen in ruby.
More info on ruby string internals here: http://patshaughnessy.net/2012/1/4/never-create-ruby-strings-longer-than-23-characters (also includes why ruby strings longer than 23 bytes were slower in ruby 1.9).
I am doing some localization testing and I have to test for strings in both English and Japaneses. The English string might be 'Waiting time is {0} minutes.' while the Japanese string might be '待ち時間は{0}分です。' where {0} is a number that can change over the course of a test. Both of these strings are coming from there respective property files. How would I be able to check for the presence of the string as well as the number that can change depending on the test that's running.
I should have added the fact that I'm checking these strings on a web page which will display in the relevant language depending on the location of where they are been viewed. And I'm using watir to verify the text.
You can read elsewhere about various theories of the best way to do testing for proper language conversion.
One typical approach is to replace all hard-coded text matches in your code with constants, and then have a file that sets the constants which can be updated based on the language in use. (I've seen that done by wrapping the require of that file in a case statement based on the language being tested. Another approach is an array or hash for each value, enumerated by a variable with a name like 'language', which lets the tests change the language on the fly. So validations would look something like this
b.div(:id => "wait-time-message).text.should == WAIT_TIME_MESSAGE[language]
To match text where part is expected to change but fall within a predictable pattern, use a regular expression. I'd recommend a little reading about regular expressions in ruby, especially using unicode regular expressions in ruby, as well as some experimenting with a tool like Rubular to test regexes
In the case above a regex such as:
/Waiting time is \d+ minutes./ or /待ち時間は\d+分です。/
would match the messages above and expect one or more digits in the middle (note that it would fail if no digits appear, if you want zero or more digits, then you would need a * in place of the +
Don't check for the literal string. Check for some kind of intermediate form that can be used to render the final string.
Sometimes this is done by specifying a message and any placeholder data, like:
[ :waiting_time_in_minutes, 10 ]
Where that would render out as the appropriate localized text.
An alternative is to treat one of the languages as a template, something that's more limited in flexibility but works most of the time. In that case you could use the English version as the string that's returned and use a helper to render it to the final page.
I have read this entry (http://stackoverflow.com/questions/8513185/vbscript-to-correctly-re-format-a-delimited-text-file) many times and still do not understand the .Execute section.
WScript.Echo oTDb.Execute(Replace("SELECT * FROM [#T]", "#T", sTbl1)) _
.GetString( adClipString, , "|", vbCrLf, "" )
The pieces I am having trouble with are the [#T] and "#T".
I know it is the "#T" that is reading the filename in the schema file and and the [#T] must be using the "#T" as a substitute. What I cannot find out is where this is mentioned/spoken about.
Some addition questions I have are:
1. If the filename can be substituted with a variable then what else can?
2. What are the rules for maintaining variables
Do they have to start with the # symbol
Are there any reserved words
If they have to start with the # symbol, does the next character have to be a letter
As I am responsible for #Milton's worry/puzzlement:
There is no variable interpolation/substitution in VBScript. Other languages - e.g. Perl - will splice variables or even expression results into string literals when you mark the replacements with special symbols. No such funny letters in VBScript.
SQL dialects allow parameterized commands in which parts to be replaced are marked by ? and/or names prefixed by symbols like #. But here ADO never sees the #T - VBScript's Replace() function has interpolated the table name before the resulting strings is send to .Execute().
Building complex strings from parts (SQL statements, commandlines for .Run or .Exec, ...) by concatenation is cumbersome. The most important drawback is that you can't (proof) read the string anymore for all those " and &.
A simple workaround is to use Replace(), as in
[sResult = ] Replace("SELECT * FROM [#T]", "#T", sTbl1)
I used the # just for letting the placeholder stand out. As you would have to stack/nest the Replace() calls when you need more substitutions on the template, other strategies are worth considering:
writing a function that takes a template string and a dictionary of replacements to apply Regexp.Replace() to the string
using .NET's System.Text.StringBuilder and its .AppendFormat to do the slicing in a sprintf like style