If I have a string consisting of these characters: I'm rich! and I look at this string in the pyCharm debugger, I'm shown 'I\\'m rich!'
Is it possible to see it without the leading, trailing, and escaped single quotes (ie as it would be stored in memory)?
In more complex strings all the escaping shown by the debugger can make it hard to really understand what characters really are, or are not, in the string.
Michael
Related
I found this commit from facebook infer, and I have no idea what \027[0K and \027[%iA means.
What does these special string mean? And (I think) if there are more strings like this, where can I find the full documentation about this?
Those are escape sequences to tell your terminal what to do.
For example, the sequence of characters represented by \027[0K (where \027 is ASCII decimal value for Esc character) tells the terminal to "clear line from cursor to the end."
One helpful document/guide on this subject can be found at https://shiroyasha.svbtle.com/escape-sequences-a-quick-guide-1
The facebook code is copied from another source here, which uses hard-coded formatters imitating termcap (this page gives some background). The original has comments indicating where its information came from.
The formatter uses "%i" for integers. That's a repeat-count for the cursor movement "cursor-up" \033[A
In most languages, \033 (octal) is used for the ASCII escape character. But this source (according to the github analysis) is written in OCaml, and is using the decimal value for the ASCII escape character. According to the OCaml syntax, you could use an octal value like this: \o033
Once you see that the formatting parts (how the escape character is represented, the use of %i to format a number), the rest of this is documented in several places.
The relevant standard is ECMA-48
the termcap (or analogous terminfo) information is in the terminal database.
In my terminal, when I'm typing over the end of a line, rather than start a new line, my new characters overwrite the beginning of the same line.
I have seen many StackOverflow questions on this topic, but none of them have helped me. Most have something to do with improperly bracketed colors, but as far as I can tell, my PS1 looks fine.
Here it is below, generated using bash -x:
PS1='\[\033[01;32m\]\w \[\033[1;36m\]☔︎ \[\033[00m\] '
Yes, that is in fact an umbrella with rain; I have my Bash prompt update with the weather using a script I wrote.
EDIT:
My BashWeather script actually can put any one of a few weather characters, so it would be great if we could solve for all of these, or come up with some other solution:
☂☃☽☀︎☔︎
If the umbrella with rain is particularly problematic, I can change that to the regular umbrella without issue.
The symbol being printed ☔︎ consists of two Unicode codepoints: U+2614 (UMBRELLA WITH RAIN DROPS) and U+FE0E (VARIATION SELECTOR-15). The second of these is a zero-length qualifier, which is intended to enforce "text style", as opposed to "emoji style", on the preceding symbol. If you're viewing this with a font can distinguish the two styles, the following might be the emoji version: ☔︉ Otherwise, you can see a table of text and emoji variants in Working Group document N4182 (the umbrella is near the top of page 3).
In theory, U+FE0E should be recognized as a zero-length codepoint, like any other combining character. However, it will not hurt to surround the variant selector in PS1 with the "non-printing" escape sequence \[…\].
It's a bit awkward to paste an isolated variant selector directly into a file, so I'd recommend using bash's unicode-escape feature:
WEATHERCHAR=$'\u2614\[\ufe0e\]'
#...
PS1=...${WEATHERCHAR}...
Note that \[ and \] are interpreted before parameter expansion, so WEATHERCHAR as defined above cannot be dynamically inserted into the prompt. An alternative would be to make the dynamically-inserted character just the $'\u2614' umbrella (or whatever), and insert the $'\[\ufe0e\]' in the prompt template along with the terminal color codes, etc.
Of course, it is entirely possible that the variant indicator isn't needed at all. It certainly makes no useful difference on my Ubuntu system, where the terminal font I use (Deja Vu Sans Mono) renders both variants with a box around the umbrella, which is simply distracting, while the fonts used in my browser seem to render the umbrella identically with and without variants. But YMMV.
This almost works for me, so should probably not be considered a complete solution. This is a stripped down prompt that consists of only an umbrella and a space:
PS1='\342\230\[\224\357\270\] '
I use the octal escapes for the UTF-8 encoding of the umbrella character, putting the last three bytes inside \[...\] so that bash doesn't think they take up space on the screen. I initially put the last four bytes in, but at least in my terminal, there is a display error where the umbrella is followed by an extra character (the question-mark-in-a-diamond glyph for missing characters), so the umbrella really does occupy two spaces.
This could be an issue with bash and 5-byte UTF-8 sequences; using a character with a 4-byte UTF-encoding poses no problem:
# U+10400 DESERET CAPITAL LETTER LONG I
# (looks like a lowercase delta)
PS1='\360\220\220\200 '
How can I match a balanced pair of delimiters not escaped by backslash (that is itself not escaped by a backslash) (without the need to consider nesting)? For example with backticks, I tried this, but the escaped backtick is not working as escaped.
regex = /(?!<\\)`(.*?)(?!<\\)`/
"hello `how\` are` you"
# => $1: "how\\"
# expected "how\\` are"
And the regex above does not consider a backslash that is escaped by a backslash and is in front of a backtick, but I would like to.
How does StackOverflow do this?
The purpose of this is not much complicated. I have documentation texts, which include the backtick notation for inline code just like StackOverflow, and I want to display that in an HTML file with the inline code decorated with some span material. There would be no nesting, but escaped backticks or escaped backslashes may appear anywhere.
Lookbehind is the first thing everyone thinks of for this kind of problem, but it's the wrong tool, even in flavors like .NET that support unrestricted lookbehinds. You can hack something up, but it's going to be ugly, even in .NET. Here's a better way:
`[^`\\]*(\\.[^`\\]*)*`
The first part starts from the opening delimiter and gobbles up anything that's not the delimiter or a backslash. If the next character is a backslash, it consumes that and the character following it, whatever it may be. It could be the delimiter character, another backslash, or anything else, it doesn't matter.
It repeats those steps as many times as necessary, and when neither [^`\\] nor \\. can match, the next character must be the closing delimiter. Or the end of the string, but I'm assuming the input is well formed. But if it's not well formed, this regex will fail very quickly. I mention that because of this other approach I see a lot:
`(?:[^`\\]+|\\.)*`
This works fine on well-formed input, but what happens if you remove the last backtick from your sample input?
"hello `how\` are you"
According to RegexBuddy, after encountering the first backtick, this regex performed 9,252 distinct operations (or steps) before it could give up and report failure; mine failed in ten steps.
EDIT To extract just the par inside the delimiters, wrap that part in a capturing group. You'll still have to remove the backslashes manually.
`([^`\\]*(?:\\.[^`\\]*)*)`
I also changed the other group to non-capturing, which I should have done from the start. I don't avoid capturing religiously, but if you are using them to capture stuff, any other groups you use should be non-capturing.
EDIT I think I've been reading too much into the question. On StackOverflow, if you want to include literal backticks in an inline-code segment or a comment, you use three backticks as the the delimiter, not just one. Since there's no need to escape backticks, you can ignore backslashes as well. Your regex could turn out to be as simple as this:
```(.*?)```
Dealing with the possibility of false delimiters, you use the same basic technique:
```([^`]*(?:`(?!``)[^`]*)*)```
Is this what you're after?
By the way, this answer doesn't contradict #nneonneo's comment above. This answer doesn't consider the context in which the match is taking place. Is it in the source code of a program or web page? If it is, did the match occur inside a comment or a string literal? How do I even know the first backtick I found wasn't escaped? Regexes don't know anything about the context in which they operate; that's what parsers are for.
If you don't need nesting, regexes can indeed be a proper tool. Lexers of programming languages, for instance, use regexes to tokenize strings, and strings usually allow their own delimiters as an escaped content. Anything more complicated than that will probably need a full-blown parser though.
The "general formula" is to match an escaped character (\\.) or any character that's valid as content but don't need to be escaped ([^{list of invalid chars}]). A "naïve" solution would be joining them with or (|), but for a more efficient variant see #AlanMoore's answer.
The complete example is shown below, in two variants: the first assumes than backslashes should only be used for escaping inside the string, the second assumes that a backslash anywhere in the text escapes the next character.
`((?:\\.|[^`\\])*)`
(?:\\.|[^`\\])*`((?:\\.|[^`\\])*)`
Working examples here and here. However, as #nneonneo commented (and I endorsed), regexes are not meant to do a complete parse, so you'd better keep things simple if you want them to work out right (do you want to find a token in the text, or do you want to delimit it already knowing where it starts? The answer to that question is important to decide which strategy works best for your case).
My OS is Windows 7 and I am building a wireless LAN chat VS wired LAN chat, and I have been wondering if space is considered a character? And if not why? And if yes why?
Yes, space is a character. Usually, programmers use "character" to refer to "some individual and indivisible unit of text." When you're writing out a string or some sequence of text, then you somehow need to mark where in that text the spaces occur. For this reason, character types usually include some encoding of various types of whitespace - spaces, tabs, newlines, etc. The main rationale behind this is that it greatly simplifies the implementation of strings. If your string implementation had to make a distinction between actual text characters and whitespace, it would be much harder to reason about the structure of a string - is it an array? If so, of what? If not, then what is it? Having spaces treated as characters makes the contents of a string uniform and makes it much easier to write programs to manipulate text.
Yes. In programming languages, a character is a value from 0 to 255 that includes letters, spaces and even control characters.
Since it has ASCII Value, it is a character.
Which style of Ruby string quoting do you favour? Up until now I've always used 'single quotes' unless the string contains certain escape sequences or interpolation, in which case I obviously have to use "double quotes".
However, is there really any reason not to just use double quoted strings everywhere?
Don't use double quotes if you have to escape them. And don't fall in "single vs double quotes" trap. Ruby has excellent support for arbitrary delimiters for string literals:
Mirror of Site - https://web.archive.org/web/20160310224440/http://rors.org/2008/10/26/dont-escape-in-strings
Original Site -
http://rors.org/2008/10/26/dont-escape-in-strings
I always use single quotes unless I need interpolation.
Why? It looks nicer. When you have a ton of stuff on the screen, lots of single quotes give you less "visual clutter" than lots of double quotes.
I'd like to note that this isn't something I deliberately decided to do, just something that I've 'evolved' over time in trying to achieve nicer looking code.
Occasionally I'll use %q or %Q if I need in-line quotes. I've only ever used heredocs maybe once or twice.
Like many programmers, I try to be as specific as is practical. This means that I try to make the compiler do as little work as possible by having my code as simple as possible. So for strings, I use the simplest method that suffices for my needs for that string.
<<END
For strings containing multiple newlines,
particularly when the string is going to
be output to the screen (and thus formatting
matters), I use heredocs.
END
%q[Because I strongly dislike backslash quoting when unnecessary, I use %Q or %q
for strings containing ' or " characters (usually with square braces, because they
happen to be the easiest to type and least likely to appear in the text inside).]
"For strings needing interpretation, I use %s."%['double quotes']
'For the most common case, needing none of the above, I use single quotes.'
My first simple test of the quality of syntax highlighting provided by a program is to see how well it handles all methods of quoting.
I use single quotes unless I need interpolation. The argument about it being troublesome to change later when you need interpolation swings in the other direction, too: You have to change from double to single when you found that there was a # or a \ in your string that caused an escape you didn't intend.
The advantage of defaulting to single quotes is that, in a codebase which adopts this convention, the quote type acts as a visual cue as to whether to expect interpolated expressions or not. This is even more pronounced when your editor or IDE highlights the two string types differently.
I use %{.....} syntax for multi-line strings.
I usually use double quotes unless I specifically need to disable escaping/interpolation.
I see arguments for both:
For using mostly double quotes:
The github ruby style guideline advocates always using double quotes:
It's easier to search for a string foobar by searching for "foobar" if you were consistent with quoting. However, I'm not. So I search for ['"]foobar['"] turning on regexps.
For using some combination of single double quotes:
Know if you need to look for string interpolation.
Might be slightly faster (although so slight it wasn't enough to affect the github style guide).
I used to use single quotes until I knew I needed interpolation. Then I found that I was wasting a lot of time when I'd go back and have to change some single-quotes to double-quotes. Performance testing showed no measurable speed impact of using double-quotes, so I advocate always using double-quotes.
The only exception is when using sub/gsub with back-references in the replacement string. Then you should use single quotes, since it's simpler.
mystring.gsub( /(fo+)bar/, '\1baz' )
mystring.gsub( /(fo+)bar/, "\\1baz" )
I use single quotes unless I need interpolation, or the string contains single quotes.
However, I just learned the arbitrary delimiter trick from Dejan's answer, and I think it's great. =)
Single quote preserve the characters inside them. But double quotes evaluate and parse them. See the following example:
"Welcome #{#user.name} to App!"
Results:
Welcome Bhojendra to App!
But,
'Welcome #{#user.name} to App!'
Results:
Welcome #{#user.name} to App!