Is it possible to write to a 3rd output stream? My situation is that I have an a number of scripts that execute various commands, remotely across a grid of machines. Those commands result in stdout and stderr. I would however like to feedback progress to the central controlling machine, without cluttering it with the interlaced stdout and stderr of the various machines in the grid. I was thinking that if it is possible to write to a 3rd output stream, that I could use it for specific status events from the grid, that the controlling script can report on, meanwhile stdout and stderr can remain redirected to log files for debugging should something go wrong.
For what it is worth I will probably be implementing this in ruby, and the machines involved will be a mixture of windows and unix machines.
I don't think how you architect your logging is constrained by the language you're using, but log4r and syslog come to mind if you're set on ruby. If you need a truly multiplatform solution maybe you might consider some kind of message bus or ØMQ although this will incur an extra layer of complexity.
It sounds like common logfiles for info and errors that all your scripts write to might be the simplest solution. Seeing as you're managing lots of small processes rather than one big monolithic app, using a tool like Splunk might help to aggregate and analyse all the logged events.
Related
So after reading these questions here, and here, I don't really have any other ideas of how I want to ask this question without explaining my situation.
Note:
I am very new to proprietary platforms in general, coming from a background in Free/Open software.
Essentially, I have a program at work for our in-house proprietary motion control platform that uses a script with macros to communicate between the program on the UI side and the firmware on the motion controller side. The documentation and development of all the involved software is handled by a sister company, and the documentation is lacking/not complete, we do have support from them but they are stretched thin, have their own products, are 2,700 mi away, and can't be in-house on call for us, I do not have the source/am not allowed to have the source for either the main application, nor the firmware, and to REALLY kill it, our last, only real programmer left. We are alone with this script and a grip of new products that all depend on this script working well with in the software. The script is in need of a serious, regular bug check solution; for each configured machine that we use this motion controller system with.
So I am going to start debugging this script. That's what's happening here.
I have tried to write a mock-up bash implementation but the degrees of recursion, arrays pulled in from .ini files, system defined command set, and parsing of the script in the firmware; have made this bash debug implementation difficult and I'm not sure I can pull it off fully without hacking away important stuff.
I looked at other options like ANTLR, which is a bit over my head too, but might work.
Now, the controller communicates over some kind of static-IPv4 cross-over Ethernet setup (Telnet?) but has a RS-232 serial that will output formatted strings that are parsed from a 'sout' command. It's been my intent to mod the script to output as much as I can get from formatted strings with predefined system commands and variables, but I'm afraid that it wont give me the big picture.
The script itself defines global, system, and local variables and functions that, (because it's on the motion controller side with hardware limitations), can be nested 25 subroutines deep. The real gotchas seem to be where the recursive side of script enters and exits these functions as they are called from the UI and other functions. Nothing jumps out, but without in depth docs I can only see so much and pretty much have just been learning all this with another engineer who asks questions to the sister company.
Can anyone give me some advice as to how I should proceed in my endeavors? I know it's probably a lot to ask but I am kinda stuck in a rut and need more cognitive resources than my skill, and coffee allow for.
Thank you for your time.
This is question is about the general architecture, I do not require anyone to solve this little hack for me, although I won't be angry if someone does ;).
Suppose I have a web app that spawns standard unix processes (like Travis CI). While it seems simple enough to pick the stdout of such a process, I'd rather like to make the whole thing asynchronous (like e.g. Travis). So I thought of passing the whole output through a websocket and into some web-based terminal emulator.
However, the only emulators I could find were fully interactive (i.e. they allow for user input and thus have some custom server-side component). My goal would be to have a piece of client side code and just stuff the output into it.
So what is necessary to create a websocket, attach it to the stdout of a server-side process (preferably emulating a tty for colors and fancyness) and display a terminal client-side? I recon there are control codes to distinguish a tty from a text file and these control codes need to be encoded on the websocket somehow, but is there some documentation on this?
I have done this for .NET applications. I think this may be worth for you as example.
I have a small .NET project named NLog.Contrib.Targets.WebSocketServer that is a log watcher with WebSocket and AngularJS. Basically, it broadcasts the data that is being logged through a WebSocket, and there is an AngularJS directive that shows the data. How to highlight data is more a presentation stuff, so it will depend on the framework you use. Basically, this component attaches to whatever .NET application that uses NLog as logging framework, so you can try to find some extensibility point in Travis yourself and attach your thing there.
About attaching to stdout, I have a proof of concept about a web interactive CMD.exe also in .NET, although you can disregard the stdin part. If you use Mono, probably is the same thing than in Windows.
I think this is very similar to what you are looking for. If you have a more specific question let me know.
You can use STDWebsocket in order to achieve this. For examples, simply read the index.html script tag. It should solve your problem (or anyone that go through this question)
Is it possible to have certain code executed whenever a file of a certain type is opened? In my case, I want to "listen" for when video files (".avi, mp4, etc.") are opened (either via the windows file explorer shell, or maybe directly from a video player?), so that I can store a history of played videos.
An hour's worth of googling turned up nothing, so I turn to you stackoverflow. Please point me in the right direction.
Thanks.
The best (and only reasonable way) to capture file system events (open/read/write) from arbitrary processes is by writing a File System MiniFilter
If you're developing a commercial product, please refrain from "hooking" Usermode APIs like CreateFile. Doing so requires numerous, platform-specific hacks, and is a compatibility nightmare.
I wouldn't hook CreateFile for this job. Windows has mechanisms built-in to handle jobs like this much more cleanly.
The easy way to handle this would be with ReadDirectoryChangesW with the FILE_NOTIFY_CHANGE_LAST_ACCESS flag. Any time a file is opened, its last-access time will be updated, so this tells you any time the file was opened.
Although it's pretty rare, that can "miss" changes under rare circumstances1. If you must have 100% accuracy (instead of, say, 99.9%), you can read change journals instead, but it's a fair amount of extra work for an advantage you may not care about.
1. There is one circumstance that isn't (necessarily) rare that you might care about though: ReadDirectoryChangesW will only work when/if your program is running. Change journals will let you know about things that happened even when your code isn't running at all.
I recently stumbled across something called pipe (all small letters) and fork (also all small letters). Apparently pipe "is a method of connecting the standard output of one process to the standard input of another". What I do not understand is what does standard input output of a process mean here. I already know that functions can call other functions and use the values returned by them so what is special about pipe, why do we need pipes? I have never come across these in my C/C++ books, what mystery is this? A simple way to communicate between two applications (I am not using the word process here) is that one application creates a file, calls another application and let it open this file and process its data and create a new result file and than terminate itself. Than the original application can continue processing and read from the result file and delete the file it created first. This is a simple way for two applications to communicate, I think that in the age of .Net framworks and complex operating systems, this must be even easier right?
Also, what is a fork? is it something specific to C++? I remember reading somewhere on internet that by using fork we can open another application from out C++ application. However, I do not know of the limitations and implications of this approach and any drawbacks that it may have. Why do we need fork? What does it do?
I do not wish that anyone has to write several pages of information. I just wish to understand what these things are, what do they do, why do we need them, and how come my C/C++ book did not cover these two?
These are unix system calls. They are not part of the C++ language or standard libraries, but are specific to unix-like operating systems.
fork creates a new process, and pipe creates a one-way communication channel. pipe and fork are often combined and used for inter-process communication.
I'm looking for writing a GUI client for a existing application in my job, this application is CLI and because this is not widely used.
This is the first time I'm writing something similar, the I ask you for recommendations, books, techniques, methodologies, advices. My first approach is to create the interface and to make calls to the original CLI client, is this a congruent approach?
Though it's not ideal, I don't think it's a bad approach, creating a GUI shell for your CLI app. In this design, the GUI acts as the CLI program's user. You have to consider things like:
Can the GUI anticipate or understand
all possible CLI program output? How about errors? How
complex will that be? Consider
parsing Unix "ls" output. Simple enough. How
about Windows command prompt "dir" output? A
bit more funky.
The CLI program may take time to
execute, this must be presented in
the GUI. The GUI may have to prevent
the user from running another instance of the CLI.
You might want to consider tcl/tk. I've written several successful commercial GUIs that work in exactly this manner.
I'll admit that it maybe takes a little skill to craft a stunning GUI but it's not impossible, and not even that hard. You won't be able to reproduce the eye candy of flash or silverlight, and if that is important this might not be the right solution for you.
If, on the other hand, you are more concerned with Getting The Job Done, tcl is a very viable choice. It's easy to learn and easy to I integrate with command line tools.