Find what settings influence a given Pylint message - pylint

When Pylint warns me about something, how can I know what settings it's applying? And what section of .pylintrc this setting goes in?
For example:
Used builtin function 'map'. Using a list comprehension can be clearer. (bad-builtin) is because bad-functions in [BASIC] contains map.
Missing function docstring (missing-docstring) is conditional on docstring-min-length in [BASIC].
invalid-name obeys several [BASIC] settings: variable-rgx, method-rgx, module-rgx, etc.
I expected to find this information in the Pylint documentation but it generally doesn't provide this information. For example, there's no link from the documentation of missing-docstring or invalid-names to the applicable options. With bad-builtin, the information is easy to find, but only because it's from a specific extension. The Pylint message wiki has explanations about what the message means, but not what brought it up. The comments in pylint --generate-rcfile don't relate the settings to warning codenames either (nothing there explains that e.g. bad-functions influences the bad-builtin message).
The reason I'm asking this is that in many cases, Pylint's conventions don't match my project's needs and conventions, and I want to tune it rather than ignore specific instances or ignore messages wholesale (so I'm not looking for disable=!). So how do I find what to tune?

How do I find what to tune?
So it seems like you're asking about how you can determine the combination of factors that lead to a pylint message being raised. Using the bad-builtin example, a warning message that is not listed at all in the .pylintrc, a helpful verbose message would be:
bad-builtin: Used builtin function 'map'. Using a list comprehension can be clearer.
---> (Hypothetical) VERBOSE OUTPUT:
bad-builtin was raised because the bad-functions setting in your .pylintrc under the [BASIC] section contains the function map. If you want to "tune" this behavior look here.
Short answer: I am not sure this exists.
When you run pylint my_file.py the first thing it should tell you is the configuration file it is using. From there, you want to find the .pylintrc in use or else you want to create it. Otherwise it will be using the default configuration. If pylint does not tell you what config you are using you can figure out your current pylintrc configuration by running something like:
pylint --generate-rcfile &> output.txt
When you inspect output.txt it should look like this example pylintrc. It is organized into different categories. You can get more info on this without generating an rcfile by running pylint --long-help. The sections include:
[MASTER]
[MESSAGES CONTROL]
[REPORTS]
[BASIC]
[ELIF]
[TYPECHECK]
[FORMAT]
[MISCELLANEOUS]
[VARIABLES]
[LOGGING]
[SIMILARITIES]
[SPELLING]
[IMPORTS]
[DESIGN]
[CLASSES]
[EXCEPTIONS]
I am not sure what you mean by, "The Pylint message wiki has explanations about what the message means, but not what brought it up." When I run pylint on a file or in a code base it tells me exactly where the offending line is.
Then I usually either:
Fix the warning
Disable the warning
Modify the .pylintrc to meet my needs
If you want to tune a specific message type but you are not sure how to do so, then I think you are right that is something that could be better documented. The question: Is XYZ warning message 'tunable'? Or is the only option to fix OR disable? This page lists the pylint features but I do not see anything about bad built-in, for example, let alone the options that are available to apply to a message type.
UPDATE: I am not sure what version of pylint you are on, but it looks like the bad-builtin was specifically addressed in pylint version 1.6. I was going to say we should look at what is going on in the actual source code to generate this message, but I think that requires figuring out what version of pylint you are on first, then digging into the code, then figuring out how to tune it? I agree that the docs can be more clear on how to tune specific warnings:
The bad-builtin check was moved into an extension.
The check was complaining about used builtin functions which were
supposed to not be used. For instance, map and filter were falling
into this category, since better alternatives can be used, such as
list comprehensions. But the check was annoying, since using map or
filter can have its use cases and as such, we decided to move it to an
extension check instead. It can now be enabled through
--load-plugins=pylint.extensions.bad_builtin.

Related

How do I effectively identify an unknown file format

I want to write a program that parses yum config files. These files look like this:
[google-chrome]
name=google-chrome - 64-bit
baseurl=http://dl.google.com/linux/chrome/rpm/stable/x86_64
enabled=1
gpgcheck=1
gpgkey=https://dl-ssl.google.com/linux/linux_signing_key.pub
This format looks like it is very easy to parse, but I do not want to reinvent the wheel. If there is an existing library that can generically parse this format, I want to use it.
But how to find a library for something you can not name?
The file extension is no help here. The term ".repo" does not yield any general results besieds yum itself.
So, please teach me how to fish:
How do I effectively find the name of a file format that is unknown to me?
Identifying an unknown file format can be a pain.
But you have some options. I will start with a very obvious one.
Ask
Showing other people the format is maybe the best way to find out its name.
Someone will likely recognize it. And if no one does, chances are good that
you have a proprietary file format in front of you.
In case of your yum repository file, I would say it is a plain old INI file.
But let's do some more research on this.
Reverse Engineering
Reverse Engineering maybe your best bet if nobody recognizes your format.
Take the reference implementation and find out what they are using to parse the format.
Luckily, yum is open source. So it is easy to look up.
Let's see, what the yum authors use to parse their repo file:
try:
ini = INIConfig(open(repo.repofile))
except:
return None
https://github.com/rpm-software-management/yum/blob/master/yum/config.py#L1304
Now the import of this function can be found here:
from iniparse import INIConfig
https://github.com/rpm-software-management/yum/blob/master/yum/config.py#L32
This leads us to a library called iniparse (https://pypi.org/project/iniparse/).
So yum uses an INI parser for its config files.
I will show you how to quickly navigate to those kind of code passages
since navigating in somewhat large projects can be intimidating.
I use a tool called ripgrep (https://github.com/BurntSushi/ripgrep).
My initial anchors are usually well known filepaths. In case of yum, I took /etc/yum.repos.d for my initial search:
# assuming you are in the root directory of yum's source code
rg /etc/yum.repos.d yum
yum/config.py
769: reposdir = ListOption(['/etc/yum/repos.d', '/etc/yum.repos.d'])
yum/__init__.py
556: # (typically /etc/yum/repos.d)
This narrows it down to two files. If you go on further with terms like read or parse,
you will quickly find the results you want.
What if you do not have the reference source?
Well, sometimes, you have no access to the source code of a reference implementation. E.g: The reference implementation is closed source.
Try to break the format. Insert some garbage and observe the log files afterwards. If you are lucky, you may find
a helpful error message which might give you hints about the format.
If you feel very brave, you can try to use an actual decompiler as well. This may or may not be illegal and may or may not be a waste of time.
I personally would only do this as a last resort.

What is spifno1stsp really doing as a rsyslog property?

I was reading the template documentation of rsyslog to find better properties and I stumble upon this one:
spifno1stsp - expert options for RFC3164 template processing
However, as you can see, the documentation is quite vague. Moreover, I have not been able to find a longer explanation anywhere. The only mentions found with Google are always about the same snippet or the same very short description.
Indeed, there is no explanation of this property:
on the entire rsyslog.com website,
or in the RFC3164,
or anywhere else actually.
It is like everybody copy & paste the same snippet here and there but it is very difficult to understand what it is actually doing.
Any idea ?
Think of it as somewhat like an if statement. If a space is present, don't do anything. Otherwise, if a space is not present, add a space.
It is useful for ensuring that just one space is added to the output, often between two strings.
For any cases like this that you find where the docs can be improved please feel free to open an issue with a request for clarification in the official GitHub rsyslog documentation project. The documentation team is understaffed, but team members will assist where they can.
If you're looking for general help, the rsyslog-users mailing list is also a good resource. I've learned a lot over the years by going over the archives and reading prior threads.
Back to your question about the spifno1stsp option:
While you will get a few hits on that option, what you'll probably find more results on is searching for the older string template option, sp-if-no-1st-sp. Here is an example of its use from the documentation page you linked to:
template(name="forwardFormat" type="string"
string="<%PRI%>%TIMESTAMP:::date-rfc3339% %HOSTNAME% %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%"
)
Here is the specific portion that is relevant here:
`%msg:::sp-if-no-1st-sp%%msg%`
From the Property Replacer documentation:
sp-if-no-1st-sp
This option looks scary and should probably not be used by a user. For
any field given, it returns either a single space character or no
character at all. Field content is never returned. A space is returned
if (and only if) the first character of the field’s content is NOT a
space. This option is kind of a hack to solve a problem rooted in RFC
3164: 3164 specifies no delimiter between the syslog tag sequence and
the actual message text. Almost all implementation in fact delimit the
two by a space. As of RFC 3164, this space is part of the message text
itself. This leads to a problem when building the message (e.g. when
writing to disk or forwarding). Should a delimiting space be included
if the message does not start with one? If not, the tag is immediately
followed by another non-space character, which can lead some log
parsers to misinterpret what is the tag and what the message. The
problem finally surfaced when the klog module was restructured and the
tag correctly written. It exists with other message sources, too. The
solution was the introduction of this special property replacer
option. Now, the default template can contain a conditional space,
which exists only if the message does not start with one. While this
does not solve all issues, it should work good enough in the far
majority of all cases. If you read this text and have no idea of what
it is talking about - relax: this is a good indication you will never
need this option. Simply forget about it ;)
In short, sp-if-no-1st-sp (string template option) is analogous to spifno1stsp (standard template option).
Hope that helps.

Less beautifier - format code

Is there is code beautifier for less such as http://www.lonniebest.com/formatcss/ for css? I need sort properties in less code by alphabet.
I use CSSComb http://csscomb.com/. This one is a npm module but there are plugins for it. Especially I use it with Sublime Text.
It works with less too although there might me some edge case not (yet) properly handled. But it's good for me.
You can order rules however you want. Just read the docs ;)
You can also use cssbrush. It is based and uses the csscomb under the hood, but include a fix for this bug and also has the ability to remember the files that were previously beautified, so it will only beautify changed files on each run.
Full disclosure, I wrote it.

Structured debug log

I am writing a complex application (a compiler analysis). To debug it I need to examine the application's execution trace to determine how its values and data structures evolve during its execution. It is quite common for me to generate megabytes of text output for a single run and sifting my way through all that is very labor-intensive. To help me manage these logs I've written my own library that formats them in HTML and makes it easy to color text from different code regions and indent code in called functions. An example of the output is here.
My question is: is there any better solution than my own home-spun library? I need some way to emit debug logs that may include arbitrary text and images and visually structure them and if possible, index them so that I can easily find the region of the output I'm most interested. Is there anything like this out there?
Regardless you didn't mentioned a language applied, I'd like to propose apache Log4XXX family: http://logging.apache.org/
It offers customizable details level as well as tag-driven loggers. GUI tool (chainsaw) can be combined with "old good" GREP approach (so you see only what you're interested in at the moment).
Colorizing, search and filtering using an expression syntax is available in the latest developer snapshot of Chainsaw. The expression syntax also supports regular expressions (using the 'like' keyword).
Chainsaw can parse any regular text log file, not just log files generated by log4j.
The latest developer snapshot of Chainsaw is available here:
http://people.apache.org/~sdeboy
The File, load Chainsaw configuration menu item is where you define the 'format' and location of the log file you want to process, and the expression syntax can be found in the tutorial, available from the help menu.
Feel free to email the log4j users list if you have additional questions.
I created a framework that might help you, https://github.com/pablito900/VisualLogs

WiX: Invalid Language Id

Some executable files have resources marked "language neutral".
Is it possible to package these binaries using WiX?
The error I receive is: error LGHT0204 : ICE03: Invalid Language Id; Table: File, Column: Language, Key(s)
Setting the Product.Language attribute to '0' does not fix the problem.
You can suppress the retrieval of file information by passing this option to light.exe:
-sh
Suppress file information: do not get hash, version, language and
other file properties
To fix the original cause of the error, compare the incorrect language ID to the ones here.
Alternatively, if you verify that the Language Id is truly valid (I've never seen a case where the ICE was wrong though) then I would suppress the ICE before suppressing the hash information (-sh). The hash information is very important for your install. Instead use "light.exe -sice ICE03" to skip that ICE.
Also note that the ICE03 checks are pretty important so make sure you solve all the other issues befor supressing this it. There is a feature request out there to suppress only specific errors instead of the whole ICE.

Resources