Where is the documentation to accomplish an orderly shutdown of IBM Cloud Private? - ibm-cloud-private

IBM Cloud Private-CE (Community Edition) 2.1.0.2 is up and running on my repurposed 32-gigabyte laptop.
VMware ESXi 6.5 (vSphere Hypervisor) is the bare metal hypervisor. The environment is up and running.
http://icpmster:8443/console
is serving beautifully constructed pages.
The Knowledge Center guided me this far on my IBM Cloud Private journey. It is silent on how to accomplish an orderly shutdown of an ICP cluster (1).
Google Search results provide no guidance.
My motivation is a home base to return to from my ICP application deployment adventures.
That is,
stop the environment.
take ESXi virtual machine snapshots to provide a baseline state of the cluster
restart the virtual machines
install software packages and deploy applications
return to the baseline after completing business valued use cases
Where is the documentation to accomplish an orderly shutdown of IBM Cloud Private?
Thank you for your assistance,
regards, Chris
Footnotes
(1) What I have read again, and again, and again …
“IBM Cloud Private v2.1.0.2” > “Release Notes” > “Known issues and limitations” > “Docker 17.12-ce is not supported”
“IBM Cloud Private v2.1.0.2” > “Release Notes” > “Known issues and limitations” > “IPv6 is not supported”
* “IPv6 is not supported”
* “Docker 17.12-ce is not supported”, “Docker version 17.09-ce is the latest stable version of Docker CE that is supported by IBM Cloud Private.”
“IBM Cloud Private v2.1.0.2” > “Overview” > “IBM Cloud Private overview”
“IBM Cloud Private v2.1.0.2” > “Overview” > “IBM Cloud Private bundles”
“IBM Cloud Private v2.1.0.2” > “Overview” > “Architecture”
“IBM Cloud Private v2.1.0.2” > “Overview” > “Components”
“IBM Cloud Private v2.1.0.2” > “Installing” > “System Requirements” > “Hardware requirements and recommendations”
“IBM Cloud Private v2.1.0.2” > “Installing” > “System Requirements” > “Supported operating systems and platforms”
“IBM Cloud Private v2.1.0.2” > “Installing” > “System Requirements” > “Supported Docker versions”
IBM Cloud Private is not supported on Docker 17.12-ce, see
“IBM Cloud Private v2.1.0.2” > “Release notes” > “Docker 17.12-ce is not supported”
“IBM Cloud Private v2.1.0.2” > “Installing” > “Configuring options during installation” > “Setting up Docker for IBM Cloud Private”
“IBM Cloud Private v2.1.0.2” > “Installing” > “Configuring options during installation” > “Setting up Docker for IBM Cloud Private” > “Installing from the Docker website”
“IBM Cloud Private v2.1.0.2” > “Installing” > “Configuring options during installation” > “Setting up Docker for IBM Cloud Private” > “Configuring your Docker engine”
“IBM Cloud Private v2.1.0.2” > “Installing” > “Configuring options during installation” > “Sharing SSH keys among cluster nodes”
“IBM Cloud Private v2.1.0.2” > “Installing” > “Configuring options during installation” > “Setting the node roles in the hosts file”
“IBM Cloud Private v2.1.0.2” > “Installing” > “Configuring options during installation” > “Customizing your installation”
* Vulnerability Advisor is not available for IBM Cloud Private-CE (Community Edition)
* Multiple Clusters (Federation) Technical Preview not available for IBM Cloud Private-CE (Community Edition)
* High Availability is supported for IBM Cloud Private Cloud Native and Enterprise editions only.
“IBM Cloud Private v2.1.0.2” > “Installing” > “Configuring options during installation” > “Configuring the monitoring service”
“IBM Cloud Private v2.1.0.2” > “Installing” > “Configuring options during installation” > “Customizing the cluster with the config.yaml file”
“IBM Cloud Private v2.1.0.2” > “Installing” > “Prepare your cluster for installation”
“IBM Cloud Private v2.1.0.2” > “Installing” > “Prepare your cluster for installation” > “Configuring your cluster”
“IBM Cloud Private v2.1.0.2” > “Installing” > “Installing IBM Cloud Private” > “IBM Cloud Private CE”
“IBM Cloud Private v2.1.0.2” > “Troubleshooting and support” > “Installation” > “Installation hangs or fails”
“IBM Cloud Private v2.1.0.2” > “Uninstalling” > “Uninstall IBM Cloud Private-CE”


to shutdown ICP you have to stop docker on each node.
systemctl stop docker
I would say you should stop the master nodes the first ones to avoid, in case you have several workers, that it moves loads from a worker being stopped to a running one.


oficial restart documentation https://www.ibm.com/support/knowledgecenter/SSBS6K_2.1.0/manage_cluster/restart_cluster.html

Related

How to prevent TeamCity from spinning up an AWS cloud agent instance

When using TeamCity with the AWS Cloud Profile, it spins up the agent every 10 minutes for no reason.
I am using the agent to build a Yocto distribution, which takes a lot of time and resources. For that reason the agent is running on a large (expensive) instance which I only want to start when needed.
What I am trying to accomplish:
Have a cheap AWS instance running TeamCity server, always on.
Have a big, expensive AWS instance running a TeamCity Agent, but only turned on when needed.
Only spin up the agent machine when there is a build to run.
Place a tag in git when the build passes.
Preferably: Check for new git commits on the server, and start the agent when there is a commit.
Please note that I want to do tagging on the Agent, so server-side checkout will not work (according to the TeamCity documentation).
I created a setup in AWS where I have a small instance running TeamCity server, and a large (expensive) instance running the agent. In TeamCity, I configured a cloud profile in the Root project to automatically start the machine running the agent, and stop it when the build is finished or the machine has been idle for 5 minutes. Starting and stopping the AWS machine works, and triggering a build als works.
I created a project with a build configuration without any Triggers added to it. I also tried to increase the minimum checking interval for the VCS root for our git repository to 90000000 seconds (so that it will practically never spin up the agent machine).
For some reason, TeamCity spins up the AWS instance running the Agent every 10 minutes and it is not clear to me why it is doing this. The TeamCity log shows:
[2019-03-19 10:51:15,894] INFO - ldServer.AGENT.PollingProtocol - Polling Agent session 'f1nqO09...' was created, agent: "yocto-build-agent1" {id=3}
[2019-03-19 10:56:33,250] INFO - ldServer.AGENT.PollingProtocol - Polling Agent session 'f1nqO09...' was closed, agent: "yocto-build-agent1" {id=3}
[2019-03-19 10:56:33,257] INFO - jetbrains.buildServer.CLEANUP - Removing agent Unregistered agent "yocto-build-agent1" {id=3}
[2019-03-19 10:56:33,262] INFO - jetbrains.buildServer.AGENT - Agent "yocto-build-agent1" {id=3} has been unregistered. Reason: Agent has unregistered
[2019-03-19 11:02:30,612] INFO - jetbrains.buildServer.AGENT - Agent has been registered: "yocto-build-agent1" {id=4, protocol=unidirectional, host=10.55.20.127:9090, agentTypeId=3, pool=Default, registered since 2019-03-19 11:02:30.590}, not running a build
[2019-03-19 11:02:30,615] INFO - ldServer.AGENT.PollingProtocol - Polling Agent session 'FcY5MVV...' was created, agent: "yocto-build-agent1" {id=4}
[2019-03-19 11:02:30,666] WARN - jetbrains.buildServer.AGENT - Agent "yocto-build-agent1" {id=4} is unauthorized on registration
[2019-03-19 11:07:09,821] INFO - ldServer.AGENT.PollingProtocol - Polling Agent session 'FcY5MVV...' was closed, agent: "yocto-build-agent1" {id=4}
[2019-03-19 11:07:09,823] INFO - jetbrains.buildServer.AGENT - Agent "yocto-build-agent1" {id=4} has been unregistered. Reason: Agent has unregistered
[2019-03-19 11:07:22,455] INFO - jetbrains.buildServer.AGENT - Agent has been registered: "yocto-build-agent1" {id=4, protocol=unidirectional, host=10.55.20.127:9090, agentTypeId=3, pool=Default
The teamcity-clouds.log shows:
[2019-03-19 10:50:38,910] INFO [uled executor 1] - .instances.StoppedInstanceTask - Instance has changed status from stopped to Starting: Amazon Instance{instanceId=i-071eff7574b93d12f, imageId=i-071eff7574b93d12f, status: Starting}, profile 'AWS Cloud Profile'{id=amazon-2, projectId=_Root}
[2019-03-19 10:50:43,910] INFO [uled executor 5] - .server.impl.CloudEventsLogger - Cloud instance entered 'starting' state, profile 'AWS Cloud Profile'{id=amazon-2, projectId=_Root}, Amazon Instance{instanceId=i-071eff7574b93d12f, imageId=i-071eff7574b93d12f, status: Starting}
[2019-03-19 10:51:15,883] INFO [nio-8111-exec-2] - r.impl.DBCloudStateManagerImpl - Image: AmazonImageInstance{id=i-071eff7574b93d12f, amazonId=i-071eff7574b93d12f}, profile: profile 'AWS Cloud Profile'{id=amazon-2, projectId=_Root} was marked to CONTAIN agent
[2019-03-19 10:51:15,883] INFO [nio-8111-exec-2] - .server.impl.CloudEventsLogger - Detected cloud agent "yocto-build-agent1" {id=3}, profile 'AWS Cloud Profile'{id=amazon-2, projectId=_Root}, Amazon Instance{instanceId=i-071eff7574b93d12f, imageId=i-071eff7574b93d12f, status: Starting}
[2019-03-19 10:51:43,912] INFO [uled executor 2] - .server.impl.CloudEventsLogger - Cloud instance entered 'running' state, profile 'AWS Cloud Profile'{id=amazon-2, projectId=_Root}, Amazon Instance{instanceId=i-071eff7574b93d12f, imageId=i-071eff7574b93d12f, status: Running}
[2019-03-19 10:56:32,807] INFO [uled executor 2] - te.IdleTimeoutTerminateFactory - Will stop instance Amazon Instance{instanceId=i-071eff7574b93d12f, imageId=i-071eff7574b93d12f, status: Running}, profile 'AWS Cloud Profile'{id=amazon-2, projectId=_Root} with idle time 5 minutes (idle time is 5 minutes)
[2019-03-19 10:56:32,808] INFO [uled executor 2] - l.instances.StopInstanceAction - Terminating instance: Amazon Instance{instanceId=i-071eff7574b93d12f, imageId=i-071eff7574b93d12f, status: Running}, profile 'AWS Cloud Profile'{id=amazon-2, projectId=_Root}, reason: "After 5 minutes idle"
[2019-03-19 10:56:32,808] INFO [uled executor 2] - r.impl.DBCloudStateManagerImpl - Image: AmazonImageInstance{id=i-071eff7574b93d12f, amazonId=i-071eff7574b93d12f}, profile: profile 'AWS Cloud Profile'{id=amazon-2, projectId=_Root} was marked to CONTAIN agent
[2019-03-19 10:56:33,004] INFO [2 Stop Instance] - r.impl.DBCloudStateManagerImpl - Image: i-071eff7574b93d12f, Instance: i-071eff7574b93d12f, profile=amazon-2 is marked with state: stopped.
[2019-03-19 10:56:33,253] INFO [nio-8111-exec-7] - ces.terminate.AgentRemoverImpl - Removing cloud agent "yocto-build-agent1" {id=3} because: Instance has been terminated. Amazon Instance{instanceId=i-071eff7574b93d12f, imageId=i-071eff7574b93d12f, status: Running}, profile 'AWS Cloud Profile'{id=amazon-2, projectId=_Root}
[2019-03-19 10:56:33,260] INFO [nio-8111-exec-7] - .server.impl.CloudEventsLogger - Cloud agent Unregistered agent "yocto-build-agent1" {id=3} is no longer recognized as cloud agent because instance has gone: profileId=amazon-2, imageId=i-071eff7574b93d12f, instanceId=i-071eff7574b93d12f
[2019-03-19 10:56:33,260] INFO [nio-8111-exec-7] - ces.terminate.AgentRemoverImpl - Removing cloud agent Unregistered agent "yocto-build-agent1" {id=3} because: Cloud instance (id=i-071eff7574b93d12f, imageId=i-071eff7574b93d12f, profileId=amazon-2) has gone (is not reported by cloud profile)
[2019-03-19 10:56:33,922] INFO [uled executor 2] - .server.impl.CloudEventsLogger - Cloud instance entered 'scheduled to stop' state, profile 'AWS Cloud Profile'{id=amazon-2, projectId=_Root}, Amazon Instance{instanceId=i-071eff7574b93d12f, imageId=i-071eff7574b93d12f, status: Scheduled to stop}
[2019-03-19 10:56:43,923] INFO [uled executor 5] - .server.impl.CloudEventsLogger - Cloud instance entered 'stopping' state, profile 'AWS Cloud Profile'{id=amazon-2, projectId=_Root}, Amazon Instance{instanceId=i-071eff7574b93d12f, imageId=i-071eff7574b93d12f, status: Stopping}
[2019-03-19 10:57:43,925] INFO [uled executor 1] - .server.impl.CloudEventsLogger - Cloud instance entered 'stopped' state, profile 'AWS Cloud Profile'{id=amazon-2, projectId=_Root}, Amazon Instance{instanceId=i-071eff7574b93d12f, imageId=i-071eff7574b93d12f, status: Stopped}
[2019-03-19 11:02:30,613] WARN [io-8111-exec-11] - .instances.StoppedInstanceTask - Agent "yocto-build-agent1" {id=4} was not matched to stopped instance Amazon Instance{instanceId=i-071eff7574b93d12f, imageId=i-071eff7574b93d12f, status: Stopped}, profile 'AWS Cloud Profile'{id=amazon-2, projectId=_Root}
[2019-03-19 11:02:32,705] WARN [nio-8111-exec-4] - .instances.StoppedInstanceTask - Agent "yocto-build-agent1" {id=4} was not matched to stopped instance Amazon Instance{instanceId=i-071eff7574b93d12f, imageId=i-071eff7574b93d12f, status: Stopped}, profile 'AWS Cloud Profile'{id=amazon-2, projectId=_Root}
[2019-03-19 11:02:43,939] INFO [uled executor 4] - .instances.StoppedInstanceTask - Instance has changed status from stopped to Running: Amazon Instance{instanceId=i-071eff7574b93d12f, imageId=i-071eff7574b93d12f, status: Running}, profile 'AWS Cloud Profile'{id=amazon-2, projectId=_Root}
[2019-03-19 11:02:48,939] INFO [uled executor 3] - .server.impl.CloudEventsLogger - Cloud instance entered 'running' state, profile 'AWS Cloud Profile'{id=amazon-2, projectId=_Root}, Amazon Instance{instanceId=i-071eff7574b93d12f, imageId=i-071eff7574b93d12f, status: Running}
[2019-03-19 11:07:22,457] INFO [nio-8111-exec-1] - r.impl.DBCloudStateManagerImpl - Image: AmazonImageInstance{id=i-071eff7574b93d12f, amazonId=i-071eff7574b93d12f}, profile: profile 'AWS Cloud Profile'{id=amazon-2, projectId=_Root} was marked to CONTAIN agent
[2019-03-19 11:07:22,458] INFO [nio-8111-exec-1] - .server.impl.CloudEventsLogger - Detected cloud agent "yocto-build-agent1" {id=4}, profile 'AWS Cloud Profile'{id=amazon-2, projectId=_Root}, Amazon Instance{instanceId=i-071eff7574b93d12f, imageId=i-071eff7574b93d12f, status: Running}
[2019-03-19 11:12:32,818] INFO [uled executor 5] - te.IdleTimeoutTerminateFactory - Will stop instance Amazon Instance{instanceId=i-071eff7574b93d12f, imageId=i-071eff7574b93d12f, status: Running}, profile 'AWS Cloud Profile'{id=amazon-2, projectId=_Root} with idle time 5 minutes (idle time is 5 minutes)
[2019-03-19 11:12:32,818] INFO [uled executor 5] - l.instances.StopInstanceAction - Terminating instance: Amazon Instance{instanceId=i-071eff7574b93d12f, imageId=i-071eff7574b93d12f, status: Running}, profile 'AWS Cloud Profile'{id=amazon-2, projectId=_Root}, reason: "After 5 minutes idle"
[2019-03-19 11:12:32,818] INFO [uled executor 5] - r.impl.DBCloudStateManagerImpl - Image: AmazonImageInstance{id=i-071eff7574b93d12f, amazonId=i-071eff7574b93d12f}, profile: profile 'AWS Cloud Profile'{id=amazon-2, projectId=_Root} was marked to CONTAIN agent
[2019-03-19 11:12:33,050] INFO [2 Stop Instance] - r.impl.DBCloudStateManagerImpl - Image: i-071eff7574b93d12f, Instance: i-071eff7574b93d12f, profile=amazon-2 is marked with state: stopped.
[2019-03-19 11:12:33,403] INFO [nio-8111-exec-6] - ces.terminate.AgentRemoverImpl - Removing cloud agent "yocto-build-agent1" {id=4} because: Instance has been terminated. Amazon Instance{instanceId=i-071eff7574b93d12f, imageId=i-071eff7574b93d12f, status: Running}, profile 'AWS Cloud Profile'{id=amazon-2, projectId=_Root}
[2019-03-19 11:12:33,415] INFO [nio-8111-exec-6] - .server.impl.CloudEventsLogger - Cloud agent Unregistered agent "yocto-build-agent1" {id=4} is no longer recognized as cloud agent because instance has gone: profileId=amazon-2, imageId=i-071eff7574b93d12f, instanceId=i-071eff7574b93d12f
[2019-03-19 11:12:33,415] INFO [nio-8111-exec-6] - ces.terminate.AgentRemoverImpl - Removing cloud agent Unregistered agent "yocto-build-agent1" {id=4} because: Cloud instance (id=i-071eff7574b93d12f, imageId=i-071eff7574b93d12f, profileId=amazon-2) has gone (is not reported by cloud profile)
[2019-03-19 11:12:33,960] INFO [uled executor 5] - .server.impl.CloudEventsLogger - Cloud instance entered 'scheduled to stop' state, profile 'AWS Cloud Profile'{id=amazon-2, projectId=_Root}, Amazon Instance{instanceId=i-071eff7574b93d12f, imageId=i-071eff7574b93d12f, status: Scheduled to stop}
[2019-03-19 11:12:43,960] INFO [uled executor 5] - .server.impl.CloudEventsLogger - Cloud instance entered 'stopping' state, profile 'AWS Cloud Profile'{id=amazon-2, projectId=_Root}, Amazon Instance{instanceId=i-071eff7574b93d12f, imageId=i-071eff7574b93d12f, status: Stopping}
[2019-03-19 11:15:43,967] INFO [uled executor 3] - .server.impl.CloudEventsLogger - Cloud instance entered 'stopped' state, profile 'AWS Cloud Profile'{id=amazon-2, projectId=_Root}, Amazon Instance{instanceId=i-071eff7574b93d12f, imageId=i-071eff7574b93d12f, status: Stopped}
(instance id and agent name have been changed to protect the innocent)
How can I prevent TeamCity Server from spinning up the agent machine, even if there are no triggers configured?
What is the best way to start the agent machine when there is a git commit? I was thinking of a separate build on a smaller agent which only checks commits, and make the larger build depend on that smaller build.

This is not normal behavior. Logic is: Teamcity will start a cloud build agent if there are no on-premise agents available. Then it will terminate if when idle, depending on the options you provide.
I created several cloud profiles, in the Root project. I use below options:
Additional terminate conditions / If idle, stop 3 minutes before the full hour
Maximum instances count: x
Not sure if that's helpful but I don't have any "was not matched to stopped instance" entries in the teamcity-clouds.log, which I can see in your log.
Also - did you try enabling debug log level for cloud profiles? You can do it in Administration / Diagnostics (/admin/admin.html?item=diagnostics). Then check log files again.

The logging in TeamCity says "State changed from Stopped to Starting", which made me think it was TeamCity that started this agent. As it turned out, the AWS Instance which is running the build agent was started by OpsWorks.
Amazon OpsWorks was configured to monitor the state of the machine and keep it "up". So when TeamCity correctly shuts the server down, Amazon OpsWorks detects the machine going down, and tries to start it up again. After a while TeamCity detects it being "up" and "idle" and stops it again. This goes on indefinetely.
It would be nice to have an OpsWorks feature in TeamCity Cloud configuration in addition to the EC2 and VMWare features. This way TeamCity can tell Opsworks that the machine is supposed to go down.
For now, we have fixed the problem by disabeling OpsWorks health monitoring for the agents.

How to connect spring application on docker to a cluster of activemq on docker

I have a simple REST application running on Tomcat8 in a docker container. I also have 3 docker containers running ActiveMQ 5.11.1 broker in master slave (with mysql as persistence) on ports 61616, 61617 and 61618 respectively.
docker run --name myRESTApp -t -i --rm -p 8080:8080 -p 9090:9090 -v $(pwd)/webapps:/usr/local/tomcat/webapps -v $(pwd)/logs:/usr/local/tomcat/logs --link activemq1:activemq myRESTApp
In myRESTApp's activeMQConfiguration.xml:
<property name="brokerURL" value="failover:(#{systemEnvironment['ACTIVEMQ_PORT_61616_TCP']})?randomize=false"/>
The 3 ActiveMQ brokers has this in their activemq.xml:
<transportConnectors>
<transportConnector name="openwire" uri="tcp://0.0.0.0:0" updateClusterClients="true" rebalanceClusterClients="true" updateClusterClientsOnRemove="true" />
</transportConnectors>
The master initially was running on 61616, I stopped the master, slave (port 61617) became the new master.
Even before the master on 61616 was stopped, it threw
Connect fail to: tcp://172.17.0.84:61616, reason: java.net.ConnectException: Connection refused
Why after linking, it still unable to connect to the broker?
After the master on 61616 is stopped, it is failing with:
Connect fail to: tcp://172.17.0.84:61616, reason: java.net.NoRouteToHostException: No route to host
Shouldn't it auto update the client of the new master like what being mentioned here?
Can anyone please advise what is the best practise for connecting the ActiveMQ cluster of broker to my application? Thanks

JMX connection to tomcat on VirtualBox

I have enabled JMX on my tomcat server with
-Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=localhost and I have a spring app that exposes JMX beans with a JmxRemoteLifecycleListener bean:
<Listener className="org.apache.catalina.mbeans.JmxRemoteLifecycleListener" rmiRegistryPortPlatform="10000" rmiServerPortPlatform="10001" />
When I run this tomcat instance on virtualbox (using vagrant) I forward ports 10000 and 10001, but when I try to connect to the JMX service (tried with VisualVM and JRockit Mission Control), I am unable to connect. Is there special configuration that needs to be done to connect since it is running on VirtualBox?

You need to do the Port forwarding with IPTABLES. Just check whether port is enabled in Iptables.

Teamcity error - Image does not exist in Amazon EC2

I am using teamcity server 8.0.3 (build 27540) and have set up several git projects. I am using Amazon EC2 images which are set up as cloud agents.
My setup starts up the agents and runs the tests on them when a new job is put to the queue.
I got it working fine for weeks and have not touched the server, and suddenly it stopped working.
In the cloud tabs I now get "Image error" with the message: "Image does not exist in Amazon EC2". Even though the image exists there!
If I open and save or re-create my cloud agent settings then one of the agents will be awoken by teamcity and do builds just fine, but the warning is still there and when the agent shuts down after my time delay I have to re-save the cloud agent settings to force a start on the agent.
I have tried restarting Teamcity with no avail.
This is the errors from teamcity-cloud.log
[2013-12-12 06:11:00,850] WARN [rome'{id=cp3} 1] - .clouds.amazon.image.ImageBase - Failed to fetch data of image AmazonImageInstance{id=i-d123cd29, amazonId=ami-463edb31} from EC2. Image does not exist in Amazon EC2
[2013-12-12 06:11:00,958] INFO [ue; Flush Queue] - .instances.StartInstanceAction - Starting cloud instance: profile 'EC2 TC Win Chrome'{id=cp3}, AmazonImageInstance{id=i-d123cd29, amazonId=ami-463edb31}, hash=EZObYYzWwxDOZ4o9svYSvGbdLqf5e7KQ, reason=Automatic start: Compatible with queued build Booking Log Client :: Win Chrome(promotionId=12008)
[2013-12-12 06:11:01,565] INFO [ue; Flush Queue] - .server.impl.CloudEventsLogger - Cloud instance start succeeded: profile 'EC2 TC Win Chrome'{id=cp3}, Amazon Instance{instanceId=i-d123cd29, imageId=i-d123cd29, amazonImageId=ami-463edb31, status: Scheduled to start}
[2013-12-12 06:11:03,135] INFO [rome'{id=cp3} 1] - r.impl.DBCloudStateManagerImpl - Image: i-d123cd29, Instance: i-d123cd29 is marked with state: running.
[2013-12-12 06:12:18,441] WARN [uled executor 4] - .instances.StoppedInstanceTask - Instance has changed status from stopped to Running: Amazon Instance{instanceId=i-d123cd29, imageId=i-d123cd29, amazonImageId=ami-463edb31, status: Running}, profile 'EC2 TC Win Chrome'{id=cp3}
[2013-12-12 06:12:20,759] WARN [rome'{id=cp3} 1] - .clouds.amazon.image.ImageBase - Failed to fetch data of image AmazonImageInstance{id=i-d123cd29, amazonId=ami-463edb31} from EC2. Image does not exist in Amazon EC2
[2013-12-12 06:13:04,668] INFO [nio-8111-exec-9] - r.impl.DBCloudStateManagerImpl - Image: AmazonImageInstance{id=i-d123cd29, amazonId=ami-463edb31}, profile: profile 'EC2 TC Win Chrome'{id=cp3} was marked to CONTAIN agent
[2013-12-12 06:13:04,672] INFO [nio-8111-exec-9] - .server.impl.CloudEventsLogger - Detected cloud agent EC2-i-d123cd29 {id=24, host=10.254.1.23:9090, agentTypeId=122, pool=Win Chrome}, profile 'EC2 TC Win Chrome'{id=cp3}, Amazon Instance{instanceId=i-d123cd29, imageId=i-d123cd29, amazonImageId=ami-463edb31, status: Running}
and from teamcity-server.log
[2013-12-12 06:13:04,648] WARN - jetbrains.buildServer.AGENT - Agent EC2-i-dd123cd29 is unauthorized with comment: Cloud instance (id=i-dd123cd29, imageId=i-dd123cd29, profileId=cp3) has gone (is not reported by cloud profile)
[2013-12-12 06:13:04,661] INFO - jetbrains.buildServer.SERVER - Agent has been registered: EC2-i-dd123cd29 {id=24, host=10.254.1.23:9090, agentTypeId=122, pool=Win Chrome}, not running a build
[2013-12-12 06:13:04,690] INFO - tbrains.buildServer.ACTIVITIES - Agent "EC2-i-dd123cd29 {id=24, host=10.254.1.23:9090, agentTypeId=122, pool=Win Chrome}" was authorised with comment "Virtual agent is authorized automatically."
[2013-12-12 06:13:05,022] INFO - tbrains.buildServer.ACTIVITIES - Build started; Booking Log Client :: Win Chrome {id=booking_log_client, internal id=bt18} #34 {build id=6213, promotion id=12008, branch=<default>, history=false, agent=EC2-i-dd123cd29 {id=24, host=10.254.1.23:9090, agentTypeId=122, pool=Win Chrome}, triggered by "Nisse Nilsson" (##userId='10'). Started Thu Dec 12 06:13:04 UTC 2013}
[2013-12-12 06:13:57,586] INFO - jetbrains.buildServer.SERVER - Agent description updated for agent: EC2-i-dd123cd29 {id=24, host=10.254.1.23:9090, agentTypeId=122, pool=Win Chrome}
[2013-12-12 06:23:54,323] INFO - jetbrains.buildServer.SERVER - Agent description updated for agent: EC2-i-dd123cd29 {id=24, host=10.254.1.23:9090, agentTypeId=122, pool=Win Chrome}

The error appears due to:
The AMI 'ami-463edb31' is no longer available in EC2's 'EU (Ireland)' region.
source:https://aws.amazon.com/amis/amazon-public-images-basic-microsoft-windows-server-2008-64-bit

Spring XD integration with hortonworks sandbox

I am trying to do the Hortonworks tutorial "Using Spring XD to stream Tweets to Hadoop for Sentiment Analysis"
As part of the step I am using the command:
xd>xd-shell --hadoopDistro hadoop20
xd> hadoop config fs --namenode hdfs://10.0.0.27:8020
xd> hadoop fs ls /
At this point I get a timeout exception.
Thus, I used the next step of creating a stream
xd>stream create --name ticktockhdfs --definition "Time | HDFS"
At this point I get the following error:
Caused by: java.net.ConnectException: Call to localhost/127.0.0.1:8020
failed on connection exception: java.net.ConnectException: Connection
refused: no further information
at org.apache.hadoop.ipc.Client.wrapException(Client.java:1142)
at org.apache.hadoop.ipc.Client.call(Client.java:1118)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)
at com.sun.proxy.$Proxy52.getProtocolVersion(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
I am using a Windows 7 laptop where I have the Hortonworks Sandbox.
In the Hortonworks sandbox setting I have:
1) Network --> Disabled the NAT driver
2) USing the Bridged Adapter --> Intel(R) Contrino(R) Ultimate-N 6300 AGN.
I am new to this, so pardon me if I have done small mistake

I wonder if you have the correct IP address for the Hortonworks Sandbox - did you see that address as the Sandbox started up? See if you can connect to the Sandbox using http://10.0.0.27:8888 in a browser.
What version of Spring XD and Hortonworks Sandbox are you using? It should work with Hortonworks Sandbox 1.3. The latest Sandbox 2.0 isn't supported yet in an XD milestone release (should be in the next release).
-Thomas

Resources