I am writing e2e tests for a command line app where I have to do file manipulation (such as cp, mv, rm, touch, and mkdir). The tests can execute just fine in my local environment. The problem occurs when they are executed on the server across platforms, where the file manipulation gets interfered with each other. Questions are:
It seems wrong to have shell command in test code to begin with, should I just code the commands programmatically?
If above is yes, is there something that would work as a "temporary file system" that is only visible for the process? So that when the tests run on other platforms, the files would not get messed up?
It seems like mutex lock can work as well but it would slow down the entire build.
Sorry this is more of a general and specific question at the same time. Doubt there will be a perfect answer but would love to hear some suggestions and opinions as I am new in both Go and testing. Appreciate the help!
There is nothing wrong in using OS commands in your code otherwise these will not be available to be used, although it may be incompatible depending on the target environment and as you are facing now may have some restrictions.
One tool that can work as a layer to the file commands is Afero, where you can use it even to simulate in-memory operations and S3 resources.
Related
Instead of recording my own keystrokes to create an asciinema tutorial I do want to script them, so I can easily improve my tutorial over time without being being stressed about redoing it each time I want to make a change, or spending a lot of time trying to correct typos in the recording.
Mainly I want to be able to do somethign like:
type: "df"
wait: 2s
type: "echo foo"
This would enable me to easily rebuild the recording from scratch when I change the script.
How can I do this?
I've searched for a lot of solutions for my own projects, and this is the solution that I've come up with:
My solution:
What I'm planning on doing for a project I'm in is to use both asciinema as well as demo-magic.sh.
Specifically, I'll be using asciinema for the recording with the -c flag (see here), and demo-magic.sh for the automated typing, since it supports waiting for commands to finish executing (or not) and custom wait times on top of that. The command would look something like asciinema rec -c "./mydemoscript.sh" myrecording. See the projects for proper usage.
Other things I've come across:
doitlive -- I'm not using this because it's more of a fake-typing automated thing where you actually have to "type like a madman", which could be useful if you're doing a talk in front of an audience but you don't want to mess up your keystrokes, or if you don't want to forget to do certain commands.
asciiscript -- it's written in Go and it works, but you need to compile it yourself, and it also doesn't support waiting for the previous command to finish.
spielbash -- a Ruby project designed specifically to automate asciinema recordings using tmux. I'm not using this, though, because it's not as portable since you need an existing Ruby installation, and more importantly, it keeps corrupting the active console I'm in, and there's also unpredictable corruption in the recordings. Also, the project hasn't been committed to since 2019.
termscript -- it just doesn't work for me, and it freezes my console. It runs on python 2.
UPDATE: I have recently come across terminalizer. It seems to provide support for an window-like border around the actual terminal, and it has a built-in GIF renderer, but I have not tried it.
There is a project svg-term which allows to record a screencast from a command, and I've seen a script that simulates commands being typed letter after letter.
For the moment I ended up using AppleScript to automate iTerm but this approach is buggy and locks you to MacOS and iTerm... and if you happen do to something during recording, you are doomed.
I am still looking for a better approach.
TL;DR: asciinema-automation should do what OP wants. Its dependencies are asciinema and pexpect.
(disclaimer: I developed it for my own usage)
Some time ago, I was also looking for a way to automate asciinema recordings and I saw the very nice answer of a-la-linuques. I chose asciiscript because it used asciinema, but it is not maintained anymore. I tried to keep on using it via a fork adding new small features, but being not very familiar with go, I finally decided to rewrite everything in python in this repo.
It reads bash files, where comments can give special instruction like adding time between command or key stroke, or wait for an expected output (see examples). Of course, this is very much in the spirit of asciiscript.
I need to build a file/folder tree with associated file icons and special locations like network computers.
Currently I'm using Shell API to achieve it: SHGetFileInfo, IShellFolder.EnumObjects and other functions.
It works fine most of the time, but occasionally, on customer's machines it causes various errors like random access violations deep in system libraries. Analyzing bug reports, some of those seem to be a result of 3rd party shell extensions which are loaded to my app's address space when the Shell API is used.
I'm thinking to somehow avoid using Shell API and do the job another way. What are the other good approaches to build a folder tree?
If the problem really is due to faulty shell extensions then the only sensible approach, in my view, is to remove those shell extensions. Trying to work with the shell, but avoid using the shell API won't lead anywhere useful. In fact I think that the likely outcome is that your alternative code will be less functional. All for the sake of one user that won't fix their broken machine. That's a terrible trade off.
If explorer is also crashing then that is a clear indication that the problem is indeed due to shell extensions.
Having said all of that, you post makes me suspect that you have had bug reports from multiple clients. That makes your diagnosis much less plausible. The shell API is a complex beast and it is very plausible that your code is defective in some way. I suspect that you may be guilty of a case of diagnosis by wishful thinking. It's very easy, when facing a fault that is hard to reproduce and diagnose, to believe that your code is not to blame. If multiple clients are reporting problems then my bet is that the defect can be found in your code.
I've got a bash script that runs on OSX.
It needs to manipulate some files on a network-share (AFP share on a Synology NAS).
Unfortunately those files are sometimes still being written when the script runs.
How do I determine if the file is in use or not ?
The normal method is by using "lsof", but that doesn't seem to work on network files if the other user is coming from another client on the LAN.
I could just attempt to rename the file. I suppose that will fail if the file is in use, but that is far from elegant.
Anybody have a better solution ?
This is not a generally solvable problem. The typical solution is to write the file to a temporary location and then move it to the final processing directory (since move within a filesystem is generally atomic). If you cannot control how or where the file is written, then you are left with heuristics, particularly doing things like looking at the file and seeing if it hasn't grown in "awhile," but none of these are particularly good compared to separating the writing from the enqueuing.
Are the other potential accesses being done by arbitrary programs or can it be assumed that it's being done by other instances of your program running on other clients?
If the file is private to your program, then all instances of your program can participate in a cooperative locking scheme. You might use the lockfile command, for example. Be very sure to clean up your lock files even in the face of signals/exceptions. You can use the trap built-in command to help with that. See here for an explanation.
I would like some kind of delete/copy/move/etc Windows commands that completely ignores if a file is "in use" or not and will do its job anyway.
my specific case:
So at the company I'm working at, we have GUI test scripts. The GUI program we're testing is one that is supposed to protect other "testprograms" (as we call them) by modifying them in certain ways. So, setup/teardown for these tests involve making a copy of the archived, un-tampered testprograms so that the GUI program can perform destructive operations (while the un-tampered copies still exists).
However, numerous times there's been some glitch and some process is still using the copied testprograms, thereby preventing teardown from overwriting the testprogram with another un-tampered one for the next round of testing. Thus, every single test "fails" because teardown fails.
Unfortunately I can not provide any specific code.
use the command-line version of Unlocker
My objective is to write a program which will call another executable on a separate computer(all with win xp) with parameters determined at run-time, then repeat for several more computers, and then collect the results. In short, I'm working on a grid-computing project. The algorithm itself being used is already coded in FORTRAN, but we are looking for an efficient way to run it on many computers at once.
I suppose one way to accomplish this would be to upload a script to each computer and then run said script on each computer, all automatically and dependent on my own parameters. But how can I write a program which will write to, upload, and run a script on a separate computer?
I had considered GridGain, but the algorithm is already coded and in a different language, so that is ruled out.
My current guess at accomplishing this task is using Expect (wiki/Expect), but I have no knowledge of the tool.
Any advice appreciated.
You can use PsExec for this:
http://technet.microsoft.com/en-us/sysinternals/bb897553.aspx
You could also look at the open source alternative RemCom:
http://rce.sourceforge.net/
It's actually pretty simple to write your own as well but RCE will show you how to do it if you want. Although, using PsExec may just suffice your needs.
Have a look into PVM, it was made for the type of situation you're describing, but you may have to annotate your existing codebase and/or implement a wrapper application.