I have a server on which we execute multiple bash scripts to automate tasks (like copying files to other servers, kicking off backups, etc). It has been working for some months, but today it started to get erratic.
What is happening, is that the script gets 'stuck' for a while, and after that, it runs with no problem. If I copy and paste the commands one by one on the terminal, it works, so is not something on the script itself, but it seems something that is preventing the bash interpreter (if that makes sense).
Another weird behavior is that the same script will run with no issues eventually. However, as we use Jenkins for automation, the scripts are re-created every time a new job starts.
For example, I created a new script, tst.sh, which only contains an echo. If I try to run it directly, it gets stuck for a while. I tried to debug it with bash -xeav but it does not print my script code, which means that it is not reading it. After a while, the script ran, with no changes. However, creating one script, with the same content and a different name, resurfaces the issue.
My hypothesis is that something prevents the script to be read, and just waits until whatever is blocking it to finish. However, I did not see any process holding the file, which means that it may not the case.
Is there any other thing I should try? My knowledge in bash is pretty basic, so I don't know if there is a flag that may help me on debugging this internally.
I am working on RHEL 8.85, the bash version is GNU bash, version 4.4.20(1)-release (x86_64-redhat-linux-gnu)
UPDATES BASED ON THE COMMENTS
Server resources are OK, no usage for them.
Hardware for the server also works fine, the ops team has not reached out with any known issue at least
Reboot makes the issue disappear, however, it reappears after 5 minutes or so
The issue seems that is not related to bash profiles and such.
Issue solved, posting this as an answer so people can find it quicker.
Turns out, as multiple users suggested in the comments (thanks to all!!) the problem was caused by a security monitor, which analyzed each of the scripts that were executed. The team changed some settings on that end to prevent it from happening, and so far is working.
Related
I'm currently using Guard along with the guard-pytest plugin to run the appropriate tests when files change. I was using sniffer, but it doesn't seem to have as much control as I would like, I want to run specific tests, not all of them (and I am a long-time guard user). I am on Mac OSX.
The problem is, Guard or Ruby or Python seem to be buffering the output, so that if a test takes a while to run, I have to wait a while with no feedback in the terminal on the tests in progress. Once the tests have completed, all the output comes at once. Anyone have any experience with getting "live" output from Guard? I messed around with $stdout = sync, PYTHONUNBUFFERED=1 and various attempts at getting readline support, but here I am, asking on StackOverflow.
Any help would be appreciated.
guard-pytest doesn't work too well if the commands take time:
https://github.com/kazufusa/guard-pytest/blob/master/lib/guard/pytest.rb#L20
Basically, it collects the output, then outputs it as a whole.
I created a PR to fix this: https://github.com/kazufusa/guard-pytest/pull/2
I've run into an issue with a program I created using Livecode, and now I'm looking for some help from the experts. My Google-fu is strong, yet the answer eludes me...
So here's the deal. I made a very basic note-taking program for Windows only, using very noob skills. I've been using it daily for work for the past month at least. It functions exactly as it should, except for these few things that don't make sense:
When loading the program, it takes 25-30 seconds to load. Not convenient considering it's pretty basic; one button and 5-6 text input fields, with the same number of label fields. And one background. The button just clears the text input fields.
I started to notice a problem when I went to create a batch file to load all my work programs. When the program is loaded thru Command Prompt, if I close CMD it will close the program too. I tried the same using Powershell, and it still closes as soon as the Powershell window closes. Really really strange.
I managed to find another standalone program made with Livecode, downloaded it and tested the CMD command to see if the same thing happened. It didn't, that program loaded instantly and it is ENORMOUS. It also didn't close when I closed the CMD window. I even tried this: opened my program and the downloaded program using the same CMD session, and when I closed CMD, my program closed but the downloaded one did not.
Then, I downloaded the source code for the program that was working correctly. I created a standalone for it, and tried to open that. It acts the same way as my note program does.
I don't get it. It's got to be something in my Standalone Application Settings considering what happened in the last step I mentioned, and I've been over and over those settings for hours, but I just don't have the knowledge of LC to know what to look for. I've scoured the web looking for answers to this, but it seems to be just me having this issue (story of my life, lol).
I'll be happy to post any codes, scripts, or files needed, please let me know. I just don't know which things to post =P
Any suggestions are very much appreciated!! Thank you. =)
If you call the program from the prompt directly, e.g. using
C:\program files (x86)\your_standalone.exe
the app is treated as a command line app. I have also noticed that a LiveCode app can sometimes close if the invoking command line prompt is closed, while it may sometimes continue to run. Perhaps the handling of the relaunch message has to do with it, since this message basically handles commands from the command line.
If you want the command line process to finish independently from the invoked LiveCode application, you can use the start command:
start "" "C:\program files (x86)\your_standalone.exe"
don't have enough info to be able to tell what your issue is. But that much of a delay is not usual for LiveCode apps so something is definitely wrong. How long does your app take to load if you open the stackfile in the IDE?
Not sure what StackOverflow allows but if you could upload your scripts and if possible stackfile that will give us more to go on
This has nothing to do with Windows, Batchfiles, or CMD.
CMD can workaround your problem, if it's a GUI program. See Start command, and read the help as it explains the starting behaviour of CMD and CMD's Start.
Explorer has different rules. Anything else that starts programs call CreateProcessExW which has it's own rules.
I'm actually interested as the question is stated, but what I'm after is a mechanism for temporarily disable the scheduled run (on multiple servers, whith shared disk), while still being able to run it interactively (if the script can be made to tell if it's started from task-scheduler or not). An external flag (the existence of a file) seemed like a simple approach to do that.
First option was using "%SESSIONNAME%" which seems to be empty when started from the scheduler, although that's also the case when started from powershell (with Start-Process, which might be an option for allowed usage while disabled).
EDIT: A second option, using the whoami cmd-command seems more promising, it reports the user as belonging to the group NT AUTHORITY\BATCH when run from task scheduler (but not when run interactively). Could I shoot myself in the foot using this - can a user be added to that group "permanently" behind my back?
We're on Windows Server 2012 R2. PowerShell solution would be interesting, as a wrapper could probably be used (or even modification/rewrite from .bat to .ps1).
Per #Alex K.: passing a separate argument to signal that we're running in a scheduled context works fine. Simple, reliable techniques that don't require you to know anything about third-party components -- in my neck of the woods we call those "wins" rather than "cheats".
If you really want, you can check for what you're literally asking -- if we're running from Task Scheduler -- by checking the properties of our parent process. From PowerShell this is quite doable, don't know about plain old batch files:
$parentpid = (gwmi "win32_process where (ProcessId = $pid)").parentprocessid
$parentname = (gwmi "win32_process where (ProcessId = $parentpid)").name
if ($parentname -eq 'taskeng.exe') {
"I think this is Task Scheduler, or maybe something else a hacker named taskeng.exe, who knows?"
} else {
"This is not Task Scheduler. Probably."
}
Of course, this might break if one day MS renames the process, or indeed if you want to use something else than TS to run the script automatically, or even if the script invokes another process which in turn invokes a script.
Personally, I would definitely go with the explicit flag. If you forget to apply it, it will be obvious, it can't suddenly break one day, and best of all, you can test if your script works correctly without actually having to schedule it first.
Whenever I start terminal on my Macbook Pro it is running a process. I have to use ctrl+C to kill it. If I close the window directly it warns me that following processs are running: login, bash, bash, perl5.12.
Any idea what might be going on here and how I get back to the normal state?
I personally had this issue a while ago. First check to see if it is from one of your profiles. Assuming you are using bash, we will look at your bash profile.
First, make sure the problem is actually stemming from your bash profile. Source the scrips as follows.
source ~/.bash_profile
source ~/.profile
If after running those you observe the same hanging problem where you have to cntrl+c, then you know what script has the problem.
The best way to remedy the situation is to comment out different parts of your script to figure out where the problem is. Backup your profile and then comment out half of your profile and do a
source ~/.bash_profile
and if it hangs or not will tell you what half the problem is on. Keep repeating this until you find the problem. It sounds longer than it actually is.
I've got these absolutely delicious bash scripts in my .bash_profile that makes working with git on the command line genuinely pleasant.
source ~/dev/git-completion.bash
source ~/dev/git-flow-completion.bash
Only problem is that they require a lot of disk IO (and some CPU) work to work. Every time I cd into a git repo directory (on an uncached disk) there's an annoying delay that can sometimes last for several seconds.
9/10 times I don't need the info in the prompt immediately. Often I just want to start a terminal, do some stuff and close it.
Would it be possible to make it run as a background task? I.e. asynchronously. That way the heavy blocking IO work could be done whilst I'm doing something else. If I need it immediately after opening a terminal I'm happy to wait. Like I have to do today.
A dream would be something like this:
source --async ~/dev/git-completion.bash
source --async ~/dev/git-flow-completion.bash
What do the scripts do? Do they set environment variables up or do they just do some on-disk stuff that's environment independent?
If the former, then your luck is probably out: I don't believe it is possible to run a script asynchronously and have it affect the current environment. If the latter, then have you just tried doing ~/dev/git-completion.bash & ?