There is this potentially useful but also pesky Pylint message which we would prefer to keep enabled and written to the console (and especially to the CI log), but only if Pylint can be told to return a 0 exit code if it finds nothing more severe (so we do not have to keep suppressing/unsuppressing it whenever adding/removing breakpoints or changing our workflow which requires 0 exit code from Pylint at a very early stage)
W1515: Leaving functions creating breakpoints in production code is not recommended (forgotten-debug-statement)
I checked the FAQ and the command-line help. There is the opposite switch --fail-on but no obvious way to reduce the severity. Also checked the changelog up to 2.13.0.
For now our poor people's workaround is to use grep instead of the one simple Pylint warning which we now disabled.
grep 'set_trace()\|breakpoint()' \
--dereference-recursive \
--include='*.py' \
$PYTHON_CODE_DIRECTORY \
||true
But the question is how to demote (reduce severity of) any specific Pylint warning.
Related
I have been trying to get a w3wp crash dump to see the crash callstack. I got two dumps but both of them have a single thread in them - seems almost like AppDomain has been recycled already and there is nothing useful left in the process when the dump was saved.
Command used: "procdump -mm -e -n 1 -l pt <PID>"
Also tried -ma for full dump but result is the same:
0:000> ~
. 0 Id: fa4.1dc8 Suspend: -1 Teb: 000000b1`77a78000 Unfrozen
I am not sure if I am missing something in the command, or IIS does not provide usable managed dumps when capturing them with procdump - any inputs are highly appreciated!
Additional detail: I was seeing STACK_OVERFLOW exception being logged by the procdump, which - apparently needs different method to capture useful dump. See my own answer below for details.
It only took few hours - hopefully this will save some time for others like me.
Found a way to do this:
procdump -mm -e 1 -l -f C00000FD.STACK_OVERFLOW -g <PID>
It works! Thanks to the unknown fellow member whose hint lead me to this. I was reading too many pages and missed saving the page link to post acknowledgement here.
What is a difference between ansible-playbook switch/flag -v and -vvv. The documentation just says -v, --verbose verbose mode (-vvv for more, -vvvv to enable connection debugging
what does more verbose exactly mean? is -vv going to be different than -vvv? I don't see -vv anywhere in documetnation.
It is quite common that debug messages have a verbosity level. Knowing how this is handled internally by ansible requires to dig inside ansible sources (which I didn't do in this specific case)
Meanwhile the ansible debug module follows this logic with the verbosity parameter
So setting there verbosity: 0 (default) will always show the debug, verbosity: 1 only if you use at least -v, verbosity: 2 only if you use at least -vv, etc.
The number of v determines the verbosity level. The more vs, the more details, depending on how the authors decided to implement the debugging details for each function/module/...
When troubleshooting git issues of users, I keep running into people not noticing error/warning messages from git, and then burning their fingers. Is there any way to colorify errors and warnings git outputs?
Since I didn't find a suitable way to color error messages, my solution is to add an additional warning when git returns an error code (!=0).
To do it, add this to your ~/.bashrc or ~/.bash_profile
# Wrap git. On errors, print an additional line in red.
git(){
command git "$#"
local exitCode=$?
if [ $exitCode -ne 0 ]; then
printf "\033[0;31mERROR: git exited with code $exitCode\033[0m\n"
return $exitCode
fi
}
Here is the result:
Note that coloring stderr in red does not work very well because git logs many things in stderr, not only errors. And some errors are output in standard output.
With Git 2.18 (Q2 2018), you now have a better documentation of the settings to colorize push errors/hints.
See commit 79f62e7 (21 Apr 2018) by Johannes Schindelin (dscho).
(Merged by Johannes Schindelin -- dscho -- in commit 79f62e7, 24 Apr 2018)
push: colorize errors
This is an attempt to resolve an issue I experience with people that are
new to Git -- especially colleagues in a team setting -- where they miss
that their push to a remote location failed because the failure and
success both return a block of white text.
An example is if I push something to a remote repository and then a
colleague attempts to push to the same remote repository and the push
fails because it requires them to pull first, but they don't notice
because a success and failure both return a block of white text. They
then continue about their business, thinking it has been successfully
pushed.
This patch colorizes the errors and hints (in red and yellow,
respectively) so whenever there is a failure when pushing to a remote
repository that fails, it is more noticeable.
With Git 2.19 (Q3 2018), The sideband code learned to optionally paint selected keywords at the beginning of incoming lines on the receiving end.
See commit bf1a11f (07 Aug 2018) by Han-Wen Nienhuys (hanwen).
Helped-by: Jonathan Nieder (artagnon).
(Merged by Junio C Hamano -- gitster -- in commit d280170, 20 Aug 2018)
sideband: highlight keywords in remote sideband output
The colorization is controlled with the config setting "color.remote".
Supported keywords are "error", "warning", "hint" and "success".
They are highlighted if they appear at the start of the line, which is
common in error messages, eg.
ERROR: commit is missing Change-Id
The Git push process itself prints lots of non-actionable messages (eg. bandwidth statistics, object counters for different phases of the process).
This obscures actionable error messages that servers may send back.
Highlighting keywords in the sideband draws more attention to those messages.
The background for this change is that Gerrit does server-side processing to create or update code reviews, and actionable error messages (eg. missing Change-Id) must be communicated back to the user during the push.
User research has shown that new users have trouble seeing these messages.
The highlighting is done on the client rather than server side, so servers don't have to grow capabilities to understand terminal escape codes and terminal state.
It also consistent with the current state where Git is control of the local display (eg. prefixing messages with "remote: ").
The highlighting can be configured using color.remote.<KEYWORD> configuration settings.
Since the keys are matched case insensitively, we match the keywords case insensitively too.
Finally, this solution is backwards compatible: many servers already
prefix their messages with "error", and they will benefit from this
change without requiring a server update.
By contrast, a server-side solution would likely require plumbing the TERM variable through the git protocol, so it would require changes to both server and client.
Note: Use Git 2.21 (Q1 2019): Lines that begin with a certain keyword that come over the wire, as well as lines that consist only of one of these keywords, ought to be painted in color for easier eyeballing, but the latter was broken ever since the feature was introduced in 2.19, which has been corrected.
See commit 1f67290 (03 Dec 2018) by Stefan Beller (stefanbeller).
(Merged by Junio C Hamano -- gitster -- in commit 20b3bc1, 14 Jan 2019)
sideband: color lines with keyword only
When bf1a11f (sideband: highlight keywords in remote sideband output,
2018-08-07) was introduced, it was carefully considered which strings would be highlighted.
However 59a255a (sideband: do not read beyond the end of input, 2018-08-18) brought in a regression that the original did not test for.
A line containing only the keyword and nothing else ("SUCCESS") should still be colored.
There is no git buit-in way to do that. Git just prints errors to STDERR and doesn’t care about the fatality of the error or anything. What you can do is color STDERR red. How to do this has been asked on on ServerFault: https://serverfault.com/questions/59262/bash-print-stderr-in-red-color
There are three basic options:
Run your command like this:
*git-command* 2> >(while read line; do echo -e "\e[01;31m$line\e[0m" >&2; done)
Use a wrapper script (See ServeFault for those), and run commands like
mywrapper *git-command*
Install stderred. This will allow you to make the effect permanent, without modifying your command line. Not sure whether this will work on windows, though.
You can use the color config section of git.
For more information and examples see http://git-scm.com/book/en/Customizing-Git-Git-Configuration#Colors-in-Git or the second part of http://blog.philippmetzler.com/?p=15
example: (add to your .gitconfig)
[color]
interactive = always
[color "interactive"]
error = red bold
Hive query output that is using UDFs consists of these 2 warnings at the end. How do I suppress these 2 warnings. Please note that the 2 warnings come right after the output as part of output.
WARN: The method class org.apache.commons.logging.impl.SLF4JLogFactory#release() was invoked.
WARN: Please see http://www.slf4j.org/codes.html#release for an explanation.
hadoop version
Hadoop 2.6.0-cdh5.4.0
hive --version
Hive 1.1.0-cdh5.4.0
If you use beeline instead of Hive the error goes away. Not the best solution, but I'm planning to post to the CDH user group asking the same question to see if it's a bug that can be fixed.
This error occurs due to adding of assembly jar which which contains classes from icl-over-slf4j.jar (which is causing the stdout messages) and slf4j-log4j12.jar.
You can try couple of things to begin with:
Try removing the assembly jar, in case if using.
Look at the following link: https://issues.apache.org/jira/browse/HIVE-12179
This suggest that we can trigger a flag in Hive where spark-assembly is loaded only if HIVE_ADD_SPARK_ASSEMBLY = "true".
https://community.hortonworks.com/questions/34311/warning-message-in-hive-output-after-upgrading-to.html :
Although there is a workaround if to avoid any end time changes and that is to manually remove the 2 lines from the end of the files using shell script.
Have tried to set HIVE_ADD_SPARK_ASSEMBLY=false, but it didn't work.
Finally, I found a post question at Cloudera community. See: https://community.cloudera.com/t5/Support-Questions/Warning-message-in-Hive-output-after-upgrading-to-hive/td-p/157141
You could try the follow command, it works for me!
hive -S -d ns=$hiveDB -d tab=$t -d dunsCol=$c1 -d phase="$ph1" -d error=$c2 -d ts=$eColumnArray -d reporting_window=$rDate -f $dir'select_count.hsql' | grep -v "^WARN" > $gOutPut 2> /dev/null
We have bash script (job wrapper) that writes to a file, launches a job, then at job completion it appends to the file information about the job. The wrapper is run on one of several thousand batch nodes, but has only cropped up with several batch machines (I believe RHEL6) accessing one NFS server, and at least one known instance of a different batch job on a different batch node using a different NFS server. In all cases, only one client host is writing to the files in question. Some jobs take hours to run, others take minutes.
In the same time period that this has occurred, there seems to be 10-50 issues out of 100,000+ jobs.
Here is what I believe to effectively be the (distilled) version of the job wrapper:
#!/bin/bash
## cwd is /nfs/path/to/jobwd
## This file is /nfs/path/to/jobwd/job_wrapper
gotEXIT()
{
## end of script, however gotEXIT is called because we trap EXIT
END="EndTime: `date`\nStatus: Ended”
echo -e "${END}" >> job_info
cat job_info | sendmail jobtracker#example.com
}
trap gotEXIT EXIT
function jobSetVar { echo "job.$1: $2" >> job_info; }
export -f jobSetVar
MSG=“${email_metadata}\n${job_metadata}”
echo -e "${MSG}\nStatus: Started" | sendmail jobtracker#example.com
echo -e "${MSG}" > job_info
## At the job’s end, the output from `time` command is the first non-corrupt data in job_info
/usr/bin/time -f "Elapsed: %e\nUser: %U\nSystem: %S" -a -o job_info job_command
## 10-360 minutes later…
RC=$?
echo -e "ExitCode: ${RC}" >> job_info
So I think there are two possibilities:
echo -e "${MSG}" > job_info
This command throws out corrupt data.
/usr/bin/time -f "Elapsed: %e\nUser: %U\nSystem: %S" -a -o job_info job_command
This corrupts the existing data, then outputs it’s data correctly.
However, some job, but not all, call jobSetVar, which doesn't end up being corrupt.
So, I dig into time.c (from GNU time 1.7) to see when the file is open. To summarize, time.c is effectively this:
FILE *outfp;
void main (int argc, char** argv) {
const char **command_line;
RESUSE res;
/* internally, getargs opens “job_info”, so outfp = fopen ("job_info", "a”) */
command_line = getargs (argc, argv);
/* run_command doesn't care about outfp */
run_command (command_line, &res);
/* internally, summarize calls fprintf and putc on outfp FILE pointer */
summarize (outfp, output_format, command_line, &res); /
fflush (outfp);
}
So, time has FILE *outfp (job_info handle) open the entire time of the job. It then writes the summary at the end of the job, and then doesn’t actually appear to close the file (not sure if this is necessary with fflush?) I've no clue if bash also has the file handle open concurrently as well.
EDIT:
A corrupted file will typically end consist of the corrupted part, followed with the non-corrupted part, which may look like this:
The corrupted section, which would occur before the non-corrupted section, is typically largely a bunch of 0x0000, with maybe some cyclic garbage mixed in:
Here's an example hexdump:
40000000 00000000 00000000 00000000
00000000 00000000 C8B450AC 772B0000
01000000 00000000 C8B450AC 772B0000
[ 361 x 0x00]
Then, at the 409th byte, it continues with the non-corrupted section:
Elapsed: 879.07
User: 0.71
System: 31.49
ExitCode: 0
EndTime: Fri Dec 6 15:29:27 PST 2013
Status: Ended
Another file looks like this:
01000000 04000000 805443FC 9D2B0000 E04144FC 9D2B0000 E04144FC 9D2B0000
[96 x 0x00]
[Repeat above 3 times ]
01000000 04000000 805443FC 9D2B0000 E04144FC 9D2B0000 E04144FC 9D2B0000
Followed by the non corrupted section:
Elapsed: 12621.27
User: 12472.32
System: 40.37
ExitCode: 0
EndTime: Thu Nov 14 08:01:14 PST 2013
Status: Ended
There are other files that have much more random corruption sections, but more than a few were cyclical similar to above.
EDIT 2: The first email, sent from the echo -e statement goes through fine. The last email is never sent due to no email metadata from corruption. So MSG isn't corrupted at that point. It's assumed that job_info probably isn't corrupt at that point either, but we haven't been able to verify that yet. This is a production system which hasn't had major code modifications and I have verified through audit that no jobs have been ran concurrently which would touch this file. The problem seems to be somewhat recent (last 2 months), but it's possible it's happened before and slipped through. This error does prevent reporting which means jobs are considered failed, so they are typically resubmitted, but one user in specific has ~9 hour jobs in which this error is particularly frustrating. I wish I could come up with more info or a way of reproducing this at will, but I was hoping somebody has maybe seen a similar problem, especially recently. I don't manage the NFS servers, but I'll try to talk to the admins to see what updates the NFS servers at the time of these issues (RHEL6 I believe) were running.
Well, the emails corresponding to the corrupt job_info files should tell you what was in MSG (which will probably be business as usual). You may want to check how NFS is being run: there's a remote possibility that you are running NFS over UDP without checksums. That could explain some corrupt data. I also hear that UDP/TCP checksums are not strong enough and the data can still end up corrupt -- maybe you are hitting such a problem (I have seen corrupt packets slipping through a network stack at least once before, and I'm quite sure some checksumming was going on). Presumably the MSG goes out as a single packet and there might be something about it that makes checksum conflicts with the garbage you see more likely. Of course it could also be an NFS bug (client or server), a server-side filesystem bug, busted piece of RAM... possibilities are almost endless here (although I see how the fact that it's always MSG that gets corrupted makes some of those quite unlikely). The problem might be related to seeking (which happens during the append). You could also have a bug somewhere else in the system, causing multiple clients to open the same job_info file, making it a jumble.
You can also try using different file for 'time' output and then merge them together with job_info at the end of script. That may help to isolate problem further.
Shell opens 'job_info' file for writing, outputs MSG and then shall close its file descriptor before launching main job. 'time' program opens same file for append as stream and I suspect the seek over NFS is not done correctly which may cause that garbage. Can't explain why, but normally this shall not happen (and is not happening). Such rare occasions may point to some race condition somewhere, can be caused by out of sequence packet delivery (due to network latency spike) or retransmits which causes duplicate data, or a bug somewhere. At first look I would suspect some bug, but that bug may be triggered by some network behavior, e.g. unusually large delay or spike of packet loss.
File access between different processes are serialized by kernel, but for additional safeguard may be worth adding some artificial delays - sleep timers between outputs for example.
Network is not transparent, especially a large one. There can be WAN optimization devices which are known to cause application issues sometimes. CIFS and NFS are good candidates for optimization over WAN with local caching of filesystem operations. Might be worth looking for recent changes with network admins..
Another thing to try, although can be difficult due to rare occurrences is capture of interesting NFS sessions via tcpdump or wireshark. In really tough cases we do simultaneous capturing on both client and server side and then compare the protocol logic to prove that network is or is not working correctly. That's a whole topic in itself, requires thorough preparation and luck but usually a last resort of desperate troubleshooting :)
It turns out this was actually another issue altogether, apparently to do with an out-of-date page being written to disk.
A bug fix was supplied to the linux-nfs implementation:
http://www.spinics.net/lists/linux-nfs/msg41357.html