I'm trying to take advantage of AWS Elastic Beanstalk's facility to customize the EC2 instances it creates. This requires creating a .config file in the .ebextensions directory.
You can specify a number of commands which should be executed when the application is deployed to an instance. I'm using that to install some msi files, and also to configure EC2 to assign the instance a unique name. This then requires a reboot.
My problem is that I only want these commands to be run when an instance is first deployed. When I deploy a code-only change to existing instances they shouldn't be run.
I've tried using the "test" parameter, which should prevent the command running. I create a file as the last command, and then I check for the presence of that file in the "test" parameter. But it doesn't seem to work.
My config file is like this:
# File structure documented at http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/customize-containers-windows-ec2.html
files:
"C:\\Users\\Public\\EnableEc2SetComputerName.ps1":
source: "[File Source]"
commands:
init-01-ec2setcomputername-enable:
test: cmd /c "if exist C:\\Users\\Public\\initialised (exit 1) else (exit 0)"
command: powershell.exe -ExecutionPolicy Bypass -File "C:\\Users\\Public\\EnableEc2SetComputerName.ps1"
waitAfterCompletion: 0
init-05-reboot-instance:
test: cmd /c "if exist C:\\Users\\Public\\initialised (exit 1) else (exit 0)"
command: shutdown -r # restart to enable EC2 to set the computer name
waitAfterCompletion: forever
init-06-mark-initialised:
test: cmd /c "if exist C:\\Users\\Public\\initialised (exit 1) else (exit 0)"
command: echo initialised > C:\\Users\\Public\\initialised
waitAfterCompletion: 0
Is there an alternative way to accomplish this? Or am I doing something stupid?
On Unix-based systems, there are the touch and test commands (referred to in this answer asking the equivalent question for Unix systems). What's the equivalent in Windows which will work best in this situation?
I think the problem lies in the fact that you are rebooting the machine before you can write the initialized file. You should be able use a bat file which first writes the semaphore, then reboots the instance, and run that .bat file contingently on the existence of semaphore.
You can either download the .bat file with a files:source: directive or compose it in the .config with a files:content: directive.
Otherwise, your test: lines look good (I tested them locally, without a reboot).
Essentially, no. Elastic Beanstalk is an abstraction and looks after the underlying infrastructure for you. You give up a lot of environment control and gain easier deployment. If you research into CloudFormation - in particular the meta-data and cfn-init / cfn-hup, you'll see a very similar construct around the beanstalk files and commands
http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-init.html
If you need to do instance customization beyond application customization - then you're possibly using the wrong tool, and having to put clumsy workarounds (until touch/test arrive from AWS) Cloud Formation scripts would probably be a better fit.
I wrote about how to configure windows instances via cloudformation and there's also extensive documentation on Amazon itself.
Given you've done all the hard work around the commands, I think it would be pretty easy to shift to a Cloud Formation script, and plonk the one time startup code into userdata.
**edit - I think you could do it like this though if you went with elastic beanstalk
command: dir initialised || powershell.exe -ExecutionPolicy Bypass -File "C:\\Users\\Public\\EnableEc2SetComputerName.ps1"
I recently ran into a very similiar problem, and utilized the answer from Jim Flanagan above and created a PowerShell script to do this.
# restarts the server if this is not the first deployment
param (
)
$strFileName="C:\Users\Public\fwinitialised.txt"
If (Test-Path $strFileName){
#This file was created on a previous deployment, reboot now.
Restart-Computer -Force
}Else{
# this is a new instance, no need to reboot.
New-Item $strFileName -type file
}
And in the .ebextensions file...
6-reboot-instance:
command: powershell.exe -ExecutionPolicy Bypass -File "C:\\PERQ\\Deployment\\RestartServerOnRedeployment.ps1"
waitAfterCompletion: forever
Related
I am trying to install confluent kafka on my databrick drivers and using init scripts there.
I am using below command to write an script to DBFS like below:
%python
dbutils.fs.put("dbfs:/databricks/tmp/sample_n8.sh",
"""
#!/bin/bash
wget -P /dbfs/databricks/tmp/tmp1 http://packages.confluent.io/archive/1.0/confluent-1.0.1-2.10.4.zip
cd /dbfs/databricks/tmp/tmp1
unzip confluent-1.0.1-2.10.4.zip
cd confluent-1.0.1
./bin/zookeeper-server-start ./etc/kafka/zookeeper.properties &
exit 0
""")
The I edit my intiscripts and add en entry there to denote to above location
[![init scripts entry adding][1]][1]
However, when I try to run my cluster it nevers starts and it always halts. If I go to event log, it shows that it is stuck at 'Starting init scripts execution.'
I know there should be tweak in my script to run it on the background but even I am using & at the end of the start command for zookeper.
Can someone give me any hint how to resolve above?
[1]: https://i.stack.imgur.com/CncIL.png
EDIT: I guess this question could be the same if I ask how I can run my script in a %sh databricks cell while the cell can finish the running of above bash script, but at the moment it always telling me that the command is running
I need to set up a PHP script as a windows service.
I need it to run regardless of which user is logged in, and on system start up - so it sounds like a windows service is best, but am happy to hear other suggestions.
(This script runs continuously, it's not a "run every 5 mins" thing I could use the Scheduled Task Manager for.)
http://support.microsoft.com/kb/251192 covers using the sc.exe program to install your service.
But from what I've read, I need to have a wrapper round the PHP script to accept the special commands from the windows service manager. Can anyone help with this?
Maybe the Resource Kit Tools (specifically srvany.exe) can help you here. MSDN: How To Create A User-Defined Service and possibly this hint for 2008 Server should help you setup any executable as a service. (I've successfully used this on Windows 2003 Server, Windows 2008 Server and on Windows XP Professional [other Resource Kit, though])
You'd create a bat containing php your-script.php, wrap that with srvany.exe and voila, the script is started once the machine loads the services.
srvany.exe should handle those start/stop/restart calls you'd expect a daemon to execute. It would load your executable on start, kill the process on stop, do both on restart. So you don't have to worry about this part. You might want to check if a register_shutdown_function() can help identify when your service process is killed.
You can even define dependencies to other services (say some database or some such).
Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\THENAMEOFYOURSERVICE]
"DependOnService"="DEPENDONTHIS"
replace THENAMEOFYOURSERVICE with the name you gave your service and DEPENDONTHIS with the name of the service to depend on (say "Postgres9.0" or something). Save that file to dependency.reg and load it with regedit /s dependency.reg. (Or doubleclick it in explorer…)
We used FireDaemon for this task, it doesn't require wrapper scripts etc. Unfortunately it's not freeware.
You can run php on command line by giving different parameters and also the script-file as parameter. If you add the whole line you need into the service configuration it should run. So you are also able to try the device before creating the service.
If the php-script is outside your web-route perhaps you should add the folder to the PATH-Variable of windows.
I found this but haven't tried it myself. PHP actually comes with some functions to do this:
http://uk.php.net/manual/en/book.win32service.php
Here are some examples:
http://uk.php.net/manual/en/win32service.examples.php
<?php
if ($argv[1] == 'run') {
win32_start_service_ctrl_dispatcher('dummyphp');
while (WIN32_SERVICE_CONTROL_STOP != win32_get_last_control_message()) {
# do your work here.
# try not to take up more than 30 seconds before going around the loop
# again
}
}
?>
After a few days... i found this magnific option!
He build an .exe that recive the service options and works fine!
https://superuser.com/questions/628176/php-cgi-exe-as-a-windows-service/643676#643676
the command correct:
sc create FOO binPath= "service.exe \"C:\php\php-cgi.exe -b
127.0.0.1:9000 -c C:\php\php.ini"\" type= own start= auto error= ignore DisplayName= "FOO php"
NSSM - the Non-Sucking Service Manager is also a solution, and then
nssm install PHP php-cgi.exe -b 127.0.0.1:9000 -c C:\php\php.ini
Loop in shell.
In php loop add loop counter and exit after hour for restarting process.
memory usage controll
reconect to db each 100 seconds
Shell script do simple loop and each for iteration create new logfile
PHP and shell script:
ini_set('memory_limit', '300M');
$loopCnt = 0;
while(true) {
/**
* Maximal time limit for loop execution
*/
set_time_limit(10);
$loopCnt ++;
/**
* each hour finishing
*/
if($loopCnt > 60 * 60){
exit;
}
usleep(self::SLEEP_MICROSECONDS);
if ($loopCnt % 60 === 0) { //log every 60 seconds memory usage
$this->out('memory usage: '.memory_get_usage());
//reconnect DB to avoid timeouts and server gone away errors
Yii::$app->db->close();
Yii::$app->db->open();
}
if (memory_get_usage() > self::MEMORY_LIMIT) {
$this->out('memory limit reached: '.self::MEMORY_LIMIT . ' actual: ' . memory_get_usage() . ' exit');
exit;
}
/**
* do work
*/
}
}
// bat file
set loopcount=1000000
:loop
echo Loop %DATE% %TIME% %loopcount%
set t=%TIME: =0%
php cwbouncer.php > C:\logs\cwbouncer_%DATE:~2,2%%DATE:~5,2%%DATE:~8,2%_%t:~0,2%%t:~3,2%%t:~6,2%.log
set /a loopcount=loopcount-1
if %loopcount%==0 goto exitloop
goto loop
:exitloop
I'm currently trying to run script that will run in the background when my AWS instance boots for the duration of the instance life. I'm testing it with a simple script to see if it works, before I test with my more complicated one:
#!/bin/bash
while [true]; do
sleep 1
echo "Hello World" >> "tempStorage.json"
done
And my sudo crontab -l returns:
# All the comment stuff
#reboot sh /home/ubuntu/test/testScript/test.sh
Which is the path to the script. I've also obviously run chmod +x test.sh to make sure its an executable.
The problem is when I stop and then start the AWS instance there's nothing in the tempStorage.json file. I've checked other threads and they all suggest this is what I should be doing, so I'm very confused and advice would be appreciated. Thanks.
As Mark B mentioned, the issue is the execution directory of the cron script. There are two solutions then.
A) Change the path to file as Mark B recommended so the script would look something like:
#!/bin/bash
while [true]; do
sleep 1
echo "Hello World" >> "/home/ubuntu/test/testScript/tempStorage.json"
done
B) Change the directory of the cron execution and keep the script as it was. This works better if you need to put the script in any directory. It would look like this for the crontab:
# All the comment stuff
#reboot cd /home/ubuntu/test/testScript && sh test.sh
That should work fine. I think the issue is that you aren't giving the full path to the tempSTorage.json file within your script. So it is being written to in a different folder than the one you are looking in, specifically whatever folder cron starts processes in by default. Try changing it to something like /tmp/tempSTorage.json and then rebooting the server again.
Note that if you are wanting something that starts on boot and runs forever, this probably isn't the best method. In that case I would look into running your process as a service.
I have a problem trying to run shell script via Chef (with docker-provisioning).
This is how I try to execute my script:
bash 'shell_try' do
user "root"
run = "#{some_path_to_script}/my_script.sh some_params"
code " #{run} > stdout.txt 2> stderr.txt"
end
(note that this script should run another scripts, processes and write logs)
Here's no errors in the output, but when I log into machine and run ps aux process isn't running.
I guess something wrong with permissions (or env variables), because when I try the same command manually - it works.
A bash resource just runs the provided script text directly, if you wanted to run a long-running process generally you would set up an Upstart or systemd service and use the service resource to start it.
Finally find a solution (thanks to #coderanger) -
Install supervisor:
Download supervisor cookbook
Add:
include_recipe 'supervisor::default'
Add my service to supervisor:
supervisor_service "name" do
action :enable
#action :start
command '/path/script.sh start'
end
Run supervisor service
All done!
Please see the Chef documentation for your resource: https://docs.chef.io/resource_bash.html. The bash resource does not support a run attribute. Text of the code attribute is run as a bash script. The default action is to run the script unless told otherwise by the resource.
bash 'shell_try' do
user "root"
code " #{run} > stdout.txt 2> stderr.txt"
action :run
end
The code attribute is written to a temporary file where it is then run using the attributes specified in the resource.
The line run = "#{some_path_to_script}/my_script.sh some_params" at this point does nothing.
I'm trying to create a job in Jenkins that will execute a simple shell script. However, it seems that Jenkins isn't actually doing anything with my script. No matter what value I put into the Execute Shell Command section, Jenkins always says that it passes. Even if I put in a bogus filename like "RandomBogusFilename.sh" it'll say the job was a success.
Can anyone point out what I'm doing wrong and how I can get Jenkins to actually use my shell script?
The shell script, the job config, and the console output are all shown below. I'm currently trying to do this on a Windows Server 2008 R2 Standard machine.
Thanks.
My .sh file
File Name: surveyToolRequest.sh
File Location: /jobs/Jeff Shell Script Test/workspace
Description:
Hit a web address and retrieve the HTTP Response. Then print out the HTTP Response.
#!/bin/bash
response_code=$(curl -s -o /dev/null -w "%{http_code}" http://SOME-WEBSITE.COM)
echo "The response code is " $response_code
My Jenkins Job Config
Jenkins Console Output
I played with this and found that it worked if I specified the path to the script. If the script is in your job's workspace directory,
./surveyToolRequest.sh
should work as Jenkins looks for files relative to the root of the workspace.
It's more common to just put the contents of the script file directory into the job configuration; that way you can see what the job is doing and you'll avoid problems like this one.
You should use run "Execute windows batch command" and not "Execute Shell"