This seems to me to be a novel idea (since i haven't found any solutions or anyone having implemented it)...
A shell script that automatically runs whenever you git commit or whatever that will let you know if you forgot to delete any debugging or development env specific lines of code in your project.
For example:
Often times (in my Ruby projects) I'll leave lines of code to output variables like
puts params.inspect
or
raise params.inspect
Also, sometimes I'll use different methods so I can easily see the effects such as in cases like using delayed_job where I'd rather call the method without a delay during development.
The problem is sometimes I forget to change those methods back or forget to delete a call to raise params.inspect and I'll inadvertently push that code.
So I thought maybe the simplest solution was to add a comment to any such debugging line such as
raise params.inspect #debug
In essence flagging that line as a development only/debug line. Then in a shell script that runs before some other command like git commit it can use awk or grep to search through all the latest modified files for that #debug comment and stop execution and alert you. However i don't know much about shell scripting so I thought I'd ask for help :)
Although I whole-heartedly recommend following cdeszaq'a advice and discourage doing this sort of thing, it is pretty easy to write a git hook that will prevent you from committing any lines with a particular string. For simplicity, I'm not showing the git rev-parse --verify HEAD that you should use to make this hook work on an initial commit, but if you simply put the following in .git/hooks/pre-commit (and make it executable), you will not be able to commit any lines of code that contain the string '#debug':
#!/bin/sh
if git diff-index -p -M --cached HEAD | grep '#debug' > /dev/null; then
echo 'debug lines found in commit. Aborting' >&2
exit 1
fi
Rather than having to remember to do additional work (removing lines of code) only to have to do more work later when things break again (re-adding that code), why not put in sensible debugging statements from the beginning?
Most languages have fairly expressive and often cheap logging libraries that will allow you to write out various levels of information (error, info, debug, trace) to a number of different locations (a file, a database). Many of these libraries will even let you adjust the logging level for a specific chunk of the code at runtime or even while the program is running.
So, rather than try to bandage up brute-force debugging by scripting away the problem, why not do yourself, and the rest of the world that has to use what you produce, a favor and use an actual logging framework for logging?
As I said in my comment, you can use any programming language you feel comfortable with.
Anyway, searching for other commit hooks, I think this one could be a good one to start with. It basically looks for some words in your files and can be customized just changing the checks array in the top of the file.
#cdeszaq is correct about the logging part.
For behaviour that differs depending on environment, the common way to achieve this is to make the behaviour configurable. delayed_job should read a value from the config file to decide how long to delay. For production environments the config would have one value and for development environments the config would have a different value.
Related
I have a Rails project where we're migrating towards using an image proxy to serve our ActiveStorage images. As part of this migration, we need to change all existing usages of url_for() to ix_image_url().
url_for will continue to exist, but in order to prevent other developers from accidentally using the wrong function in the future and accidentally serving an unproxied image, I'd like to "ban" use of url_for within our project.
Is this possible?
Just Grep for the Method Call
While you could certainly write your own cop, the use case you're defining seems like it would be a better fit for a simple grep done on your continuous integration server. For example:
if egrep -Rn '\burl_for' '/path/to/source/tree'; then
echo 'error: use ix_image_url instead of url_for' >&2
exit 1
fi
can be run as a build or test step in your CI pipeline. It will print the offending lines, along with filenames and line numbers, and send your "how to fix it" custom message to standard error for reporting the build failure.
Note that shell commands like this can be run in most CI implementations, but you can certainly embody it in a cop or inline it into your pipeline (e.g. Groovy code for Jenkins jobs) if that works better for you. However, the KISS principle argues for keeping the implementation as simple as possible, so I'd avoid the necessity of mucking around with ASTs or the maintenance overhead of custom cops unless you are getting a lot more bang for your buck than the use case described in your original question.
I need to do refactoring in a big legacy Python code base.
Often I think "these lines don't get executed in production any more".
But I am unsure.
There are some tests which touch these lines. But I can't tell for sure if really no usage happens in production.
What can I do in this situation?
This question is about coverage on a production system. This question is not about coverage during testing/CI.
I don't want to comment out that lines, since I don't want to produce an error in the production system.
Common practice is to use logging inside that lines of code. e.g. you have a block of code you think is not in use. You add try catch block in the beginning of that block of code. Inside trycatch you add line to a specific json named same as your suspicious block of code.
try:
with open("block1.dat", "rb") as file:
activity = pickle.load(file)
curtime = datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S')
currentact = "dt = {}; code done that: var1 = {},
var2 = {}".format(curdate, var1, var2)
activity.append(currentact)
file = open("block1.dat", "ab")
pickle.dump(activity, file)
file.close()
except Exception: pass
You can use telegram api to log code to. After a while You'll get info how often your code works and what does it do.
Then you monitor for a while and if nothing happens in a month, You can comment the block.
Is the production system deterministic?
Is it interactive?
Does control flow depend on input data?
Do you have access to all possible inputs?
Do the tests exist for a reason or just because?
I'd be careful removing code based on what is needed based on logging unless I knew there are no exceptional situations that occur rarely.
I would follow the common code paths to try to understand the codebase piece by piece in order to figure out what can be simplified. It's hard to give more specific advice without knowing more about the system you're dealing with.
We use a simple pattern to handle this: looks_like_dead_code(my_string)
This is a method which logs the string "my_string".
Example usage:
if ext == '.jpe':
looks_like_dead_code('2018-11-30 tguettler: looks fixed in mime_type_to_extension')
Using the date and the developer login is not enforced, it is just best practice.
If the line gets executed the one who is responsible for checking the logs will talk to the developer.
Since our production environments get updated roughly once in two weeks, you can be sure that this line was not executed during the last months.
I like this solution since in most cases it is like this:
you want to fix a bug or implement a new feature
you look at the code and see some lines which look like dead code. I mean code which is useless, since it won't get executed any more.
You don't have hours of time to investigate. You can dive into your vague guess that this is dead code. You want to do your actual work (fix a bug or implement a new feature. See Step1)
The method looks_like_dead_code() gives you a way to actually do something and leave a note for other developers. It only costs some seconds improve the current situation.
If you have a Tickler file System you can remind yourself to check this code in six months. At least in my context I can be very sure that this is dead code if this line was not executed for several months.
I have a lot of old Perl code that gets called frequently, I have been writing a new module and all of a sudden I'm getting a lot of warnings in my error_log for Apache, they are for every module currently being used. e.g,
"my" variable $variable masks earlier declaration in same statement at
/path/to/module.pm line 40 (#1)
Useless use of hash element in void context at
/path/to/another/module.pm line 212 (#2)
The main layout of the codebase is one giant script that includes the modules and directs the requests to them needed to create certain pages for the website and the main script then handles static elements like menus.
My current project is separated from this main script and doesn't use it however any time I call my code using ajax, there are some other ajax calls that will use the main script and the warnings only seem to appear from those request but only when I'm calling my project.
I have grepped every module and none of them have use warnings (or -w) in them, I have also tried using no warnings 'all' in the main script and my own project but it's not doing anything.
At this point I'm out of ideas on what to do next so all help is appreciated, I'd just like to suppress the warnings, the codebase is quite old and poorly written so going and correcting each issue that causes the warns in the first place isn't do-able.
The Apache server is running mod_perl as well, if that might make a difference I have a feeling it might be something to do with CGI, but I can't seem to find any evidence.
I take it that the code gets called by running certain top-level Perl script(s).
Then use the __WARN__ hook in those script(s) to stop printing of warnings
BEGIN { $SIG{__WARN__} = sub {} };
Place this BEGIN block before the use statements so to affect modules as well.
An empty subroutine is the way to mute warnings since __WARN__ doesn't support 'IGNORE'.
See warn and %SIG in perlvar.
See this post and this post for comments and some examples.
To investigate further and track the warnings you can use Carp
BEGIN {
$SIG{__WARN__} = \&Carp::cluck; # or Carp::confess; to also die
}
which will make it print full stack traces. This can be fine-tuned as you please since we can write our own sub to be called. Or use Carp::Always.
See this post
for some more drastic measures (like overriding CORE::GLOBAL::warn)
Once you find a more precise level at which to suppress warnings then local $SIG{__WARN__} is the way to go, if possible. This is used in a post linked above, and here is another example. It is of course far better to suppress warnings only where needed instead of everywhere.
More detail
Getting stack traces in Perl?
How can I get a call stack listing in Perl?
Note that longmess is unfortunately no longer so standard and well supported.
I have a sizable vim script (a .vim file, in viml syntax). I'd like to check (but not execute!) the file for simple syntax errors.
How do I accomplish this?
I just want a very rough syntax check. Something along the lines of perl -c or pyflakes.
Here is a syntax checker for VimL.
https://github.com/syngan/vim-vimlint/
I don't think (I'm relatively sure, as much as one can be) one exists. VimL is an internal language of Vim (and only Vim), and there aren't many tools developed for it.
I tried searching on vim.org and several other places, with no luck. Not suprising, because I've never heard of one either.
So you're either stuck with running the script, or switching to an outside language like Python, Perl or Ruby.
https://github.com/osyo-manga/vim-watchdogs
vim-watchdogs, apparently, is a syntax checker for vim, it says that it supports many languages, including vimL
if you use vundle, you can just drop this into your vimrc:
Plugin 'git://github.com/osyo-manga/vim-watchdogs.git'
..and then run:
:PluginInstall
..to set it up (vundle is a very nifty plugin manager) If you have syntastic, you might want to be careful and disable it first, and then see if it is an adequate replacement (since it says it supports all those languages anyway).
It is a safe bet that when you have multiple syntax checkers going, you will need to put your "dogs on a leash", so to speak; by configuring one to check languages that the other one does not, and vice-versa. If you do not, there will be at best collisions, duplications, or misdirections. At worst, you will have all of the above and more.
Make sure that you always backup your ~/.vim directory (or your VIMRUNTIME directory if you install things on a global level), you will be glad you did. Hope that helped you or someone else out, good luck! Sorry you had to wait 7.5 months for a response, heh :)
There's now a second option: vim-lint (as opposed to vimlint)
Let me pose a bit of background information before asking my question:
I recently joined a new software development group that uses Rational tools for configuration management, including a source control and change management system.
In addition to these tools, the team has a standard practice of noting any code changes as a comment in the code, such as:
///<history>
[mt] 3/15/2009 Made abc changes to fix xyz
///</history>
Their official purpose for the commenting standard is that "the comments provide traceability from requirement to code modification".
I am preparing to pose an argument that this practice is unnecessary and redundant; that the team should get rid of this standard immediately.
To wit - the change management system is the place to build traceability from requirement to code modification, and source control can provide detailed history of changes by performing a Diff between versions. When source code is checked in, the corresponding change management ticket is noted. When a CM ticket is resolved, we note which source code files were modified. I believe this provides a sufficient cross-reference for the desired traceability.
I would like to know if anyone disagrees with my argument. Am I missing some benefit of commented source code history that change management and source control systems cannot provide?
For myself, I have always found such comments to be more trouble than they're worth: they can cause merge conflicts, can appear as 'false positives' when you're trying to isolate the diffs between two versions, and may reference code changes that have since been obsoleted by later changes.
It's often (not always, but often) possible to change version-control systems without losing metadata. If you were to move your code to a system that doesn't support this, it would not be hard to write a script to convert the change history into comments before the cutover.
A comment allows you to find all the changes and their reasons in the code right where they are relevant without having to dig into diffs and version control system intricacies. Furthermore, should you decide to change of version control system, the comments will stay.
I worked on a large project with similar practice that had changed of source control system twice. There wasn't a day when I wasn't glad to have these comments.
Is it redundant? Yes.
Is it unnecessary? No.
I've always thought that code should be, of course, under version control, and that the current source code (the one that you can open and read today) should be valid only in present tense.
It doesn't matter if a report could have up to 3 axis in the past and last month you updated it to support up to 6 axis. It doesn't matter if you expanded some function or fixed some bug, as long as the current version can be easily understood. When you fix a bug, just leave the fixed code.
There's an exception, though. If (and only if) the fixed code looks less intuitive to you than the previous, incorrect one; if you feel that someone might come tomorrow and, just by reading the code, be tempted to change it back to what "seems more correct", then it's good to add a comment: "This is done this way to avoid... blah blah blah." Also, if the problem behind is an infamous war story inside the team's culture, or if for some reason the bug report database contains very interesting information about this part of the code, I wouldn't find it incorrect to add "(see Bug Id 10005)" to the explaining comment.
The one that jumps to mind to me is vendor lockin. If you ever moved away from Rational, you'd need to make sure that the full change history was maintained during the migration - not just the version of the artifacts.
When you're in the code you need to know why it's structured like that, hence in code commenting. Tools that sit outside the code, good though they may be, require far too much of a context shift in your brain to be useful. As well as that, trying to reverse engineer the code intent from documentation and a diff is pretty damn hard, I'd much rather read a line of comment any day.
There was a phase in the code I work on, back in the 1994-96 time frame, where there was a tendency to insert change history comments at the top of the file. Those comments are now meaningless and useless, and one of the many standard cleanups I perform when editing files containing such comments is to remove them.
In contrast, there are also some comments with a bug number at the location where the change is made, typically explaining why the ridiculous code is as it is. These can be very helpful. The bug number gives you somewhere else to look for information, and fingers the culprit (or victim - it varies).
On the other hand, items like this one - genuine; cleaned up last week - make me grit my teeth.
if (ctab->tarray && ctab->tarray[i])
#ifndef NT
prt_theargs(*(ctab->tarray[i]));
#else
/* Correct the parameter type mismatch in the line above */
prt_theargs(ctab->tarray[i]);
#endif /* NT */
The NT team got the call correct; why they thought it was a platform-specific fix is beyond me. Of course, if the code had used prototypes instead of just parameterless declarations before now, then the Unix team would have had to fix the code too. The comment was a help - assuring me that the bug was genuine - but exasperating.