I have a bash script that works at the moment. It gets an image and JDK 8 from a link and then runs a installer for the JDK 8 to move on to setting up another piece of software.
As I was debugging the script, I kept finding myself having to delete directories and even the java installation because when I introduce a fix and rerun the script, I have to wait for everything to download again and I have to worry about duplicate files messing up my current logic -which can probably be improved, but I'll go to the StackExchange Code Review site later.
At the moment, I would like to know what approaches there are to prevent commands -like downloading the JDK and running the JDK installer script all over again and others- from running again.
What kind of general approaches are out there for cases such as these?
For the JDK download and running the installer, I did think of simply checking for the existing of java on the system and if there is then bash would not not to run those commands.
However, there are other commands I do not want run and I do want to simply check, for example, the existence of certain files to prevent wget-ing them all over again and moving them -causing duplicates. (Should I maybe suck it up and do that anyway as that might be best practice?)
I did also think of perhaps, at each successful command, outputting like a 1 to a text file and mapping each line in that text file to the commands run in the script (like using an if statement to see if that command had a 1 or not in the text file) and if it was a 0, then the script would know only to run that command and never the 1s.
That sounded clunky to me and I am pretty sure that is not a good approach.
Related
I have a situation where I need to deploy ruby code to lab devices. These machines are slow and very locked down (hard to get files to). They get re-imaged fairly often, but I am not allowed to bake the install into the image.
I've solved the problem by using the same open ports for Git code distribution to distribute my install files.
I have a long install process boiled down to 3 batch files, but if I could reduce it to one, it would make life a lot easier (not to babysit a lot of installs via VNC). I'm only mentioning all this for reference.
The problem:
I can't install git and then do a pull from command line without opening a new cmd prompt - I think it pertains to environment variables, but am not 100% sure.
I get "git is not recognized" blah blah blah if I don't break out at this point and start the next batch. Same deal when I install ruby and don't break out before starting the DevKit install.
Powershell is not an option (I'm only allowed to install Git, Ruby, and the support (DevKit, C++ redistributables, .Net Client 4) and some of the machines do not have it.
I did a version where I scripted reboots and moving the batches into startup progressively - it works, but some of the other machines are tied together in a way that makes rebooting an issue (please don't make me explain - it's complicated lol).
Is there a way to merge my 3 batch files and execute all steps without reboots?
Edit: I have tried starting the next batch from the first (Start... Call...), and even creating a scheduled task to execute the next step. Can't swear I haven't made any mistakes, but they all seem to inherit the initial conditions and don't recognize the "git" command unless a new cmd prompt is opened.
I am developing a gameserver shell script that has multiple smaller scripts for saving content, restarting, stopping, installing, updating, etc. Since this a script for the community, I need a way to allow me to work on it, and then push those changes to the users who use the script.
This is a basic concept of how I think it should work:
User runs the script
script downloads a version file off github
script reads the version string from github and tries to match it with the version listed in the script itself
if it doesn't match, download new script file (url never changes), replace and re-run the script
If this is something that seems possible (I assume it would), please inform me of the process.
This was demoed at docker con 2015 yesterday. But this might help. https://github.com/docker/notary
I have some Issues regarding Jenkins and running a Powershell Script within. Long Story short: the Script takes 8x longe execution time then running it manually (takes just a few minutes) on the Server(Slave).
Im wondering why?
In the script are functions which which invoke commands like & msbuild.exe or & svn commit. I found out that the script hangs up in those Lines where before metioned commands are executed. The result is, that Jenkins time out because the Script take that long. I could alter the Timeout threshold in the Jenkins Job Configuration but i dont think this is the solution for the problem
There are no error ouputs or any information why it takes that long and i do not have any further Idea for the reason. Maybe one of you could tell me, how Jenkins invokes internaly those commands.
This is what Jenkins does (Windows batch plugin):
powershell -File %WORKSPACE%\ScriptHead\DeployOrRelease.ps1
I've created my own Powershell CI Service before I found that Jenkins supports it's own such plugin. But in my implementation and in my current jobs configs we follow sample segregation principle rule: more is better better. I found that my CI Service works better when is separated in different steps (also in case of error it's a lot easy for a root cause analyse). The Single responsibility principle is also helpful here. So as in Jenkins we have pre- & post-, build and email steps as separate script. About
msbuild.exe
As far as I remember in my case there were issues related with the operations in FileSystem paths. So when script was divided/separated in different functions we had better performance (additional checks of params).
Use "divide and conquer" technique. You have two choices: modify your script so that will display what is doing and how much it takes for every step. Second option is to make smaller scripts to perform actions like:
get the code source,
compile/build the application,
run the test,
create a package,
send the package,
archive the logs
send notification.
The most problematic is usually the first step: To get the source code from GIT or SVN or Mercurial or whatever you have as version control system. Make sure this step is not embeded into your script.
During the job run, Jenkins capture the output and use AJAX to display the result in your browser. In the script make sure you flush standard output for every step or several steps. Some languages cache standard output so you can see the results only at the end.
Also you can create log files that can be helpful to archive and verify activity status for older runs. From my experience using Jenkins with more then 10 steps requires you to create a specialized application that can run multiple steps like "robot framework".
I've run into the problem described in this question, where an old package was Obsoleted, and its %preun script is run with $1 = 0, resulting in undesirable behavior. I know this could be worked around by using -e + -i, as suggested in that answer, or the --nopreun flag, but it's difficult to get that information out to users who are accustomed to simply using -U.
I can't modify the existing %preun scripts in the wild. I don't see any way to run additional code from the new package after the old one's preun. I can't find any way to have my new package programmatically prevent the old %preun script from executing.
Is there any safe way to reach into the RPM database and remove a scriptlet for an existing package?
Jeff Johnson is absolutely correct that it should not be done. However it certainly can be done.
I have done this in an RPM at work, for distribution, but note, this was a contained semi-structured environment with no remote hands to all systems.
If you have remote hands access, take the "remove, install" path, and script that.
If you really feel you should be doing this, then these are the pointers. I'm not going to show you exactly how I did it because it was "work" and not mine to share. The concepts are mine :-)
First, back up the /var/lib/rpm/Packages file (cp /var/lib/rpm/Packages /var/tmp/Packages.bkp). Put it somehwere safe. Update your backup if any one else changes the system whilst you are working on your solution. MAke regular checks on the count of RPMs and test every which way from Sunday, after each change or step.
You will need to use the db_unload and db_load commands. For speed, you will need to use "s2p" to convert any shell sed patterns to perl. Then build a pipe which looks like this:
db_unload /var/tmp/Packages.bkp |perl -i -e "s2p converted string" |db_load /var/tmp/Packages.new
You can then attempt to test the Packages.new by copying ot over the original. Always run rpm --rebuilddb after manual changes. If you see any errors, restore the back up and rebuild the db again.
If you need to put it in an RPM, then convert it to Lua, and put it in the pretrans or posttrans scriptlet (%pretrans -p <lua>). The selection depends on the ordering you are trying to achieve. The Lua interpreter is built in to rpm, and so it will run OK during a new system install even if your RPM gets called somehow. I wrapped my "pipe" in a lua long string, and made it only execute if the system already exists. It does nothing otherwise. If you are thinking "that will never happen" then check out "Never say Never".
BTW you can completely stiff your RPM base and thus future administration of the system if you mess this up. If you do that, and have no backup or way out, it would be a hard way to learn that you are responsible for your own actions. Just saying that you have been warned!
No you cannot edit an rpmdb: the headers are protected
from change by a SHA1 or a digital signature.
Instead upgrade to a fixed version of the package using --nopreun
to prevent running the buggy script let.
Does anyone know where I would begin to write an automation script to install a CMS to my wwwroot folder and create the database for it? For this round it would be for DNN on in a windows environment.
I am looking for methods to speed up our development process in the office.
Many thanks!
First write a (textual) script of what you need to do.
Then convert these steps to automated steps.
The exact steps depend on the database and webserver you are using and on how you want everything to be configured, so get these steps clear before you start scripting.
Queries you need to execute can be put in separate files (usually with .sql extension). Most databases allow running those files from the command line.
Most other commands, like creating folders and such can be run from the command line as well. If you can run it in the command window, you can put it in a batch file as well to run everything at once, so you can translate all possible steps to a line (or group of lines) in a .bat script.
Identify the prerequisites, and generally they windows patches, db/sql server etc.
Determine how to install all the prerequisites from the command line.
Create your database manually and then script it out using something like sql server management studio or a third party tool such as the RedGate Sql Tool Belt.
Read up on how to install the CMS from the command line
Automate the installation of the above using a script language, such batch, Powershell, VB script etc.
Once you've got to step 5 you can ask more specific questions on particular aspects of this.