I have a Mesos cluster and was running a Spark shell connected to it. I shut down the client, but Mesos still believes the framework should be active.
I am trying to have Mesos drop the framework by using DELETE with curl
(https://issues.apache.org/jira/browse/MESOS-1390)
but I am getting no response from the server. Also, I am not sure how exactly to connect to the master: I have a multi-master setup managed by ZooKeeper, and I was trying to connect just to the active master:
curl -X DELETE http://<active master url>:5050/framworks/<framework id>
Can anyone verify if the above is the correct request?
I am using mesos-0.20.0.
Thanks
There is a restfull option calling by post the url http://your_mesos:5050/master/teardown passing frameworkId parameter
curl -d#/tmp/post.txt -X POST http://your_mesos:5050/master/teardown
/tmp/post.txt is a file with the follow content:
frameworkId=23423-23423-234234-234234
I know is late but for future askers
EDIT: The endpoint is now called teardown.
Example (thanks #Jeff): curl -X POST http://your_mesos:5050/master/teardown -d 'frameworkId=23423-23423-234234-234234'
Just to keep this up to date: The master endpoint was renamed to teardown i.e. http://localhost:5050/master/teardown is the new way to go.
TEARDOWN Request (JSON):
POST /master/teardown HTTP/1.1
Host: masterhost:5050
Content-Type: application/json
frameworkId=12220-3440-12532-2345
TEARDOWN Response:
HTTP/1.1 200 Ok
Riffing on #montells work, a one-liner would be
echo "frameworkId= 23423-23423-234234-234234" | curl -d#- -X POST http://localhost:5050/master/shutdown
Even though that JIRA issue mentions DELETE (in comments) it's not how framework shutdown is implemented. You need to do a POST request to /shutdown endpoint.
Examples: https://github.com/apache/mesos/blob/master/src/tests/teardown_tests.cpp
Regarding why the spark framework is not removed after you shutdown the client, I'm guessing it is because spark uses a high failover timeout? Nonetheless, I'm surprised that Mesos UI shows it as active instead of inactive.
Add this in your .bashrc:
#Mesos
killtask(){ curl -XPOST http://mesos_url:5050/master/teardown -d 'frameworkId='$#''; } ;
Sample usage:
killtask 123
Related
I have installed knox server and done all the steps mentioned on hortonworks site.
When i ran the below command on the sandbox , it gives me the proper output.
curl http://sandbox:50070/webhdfs/v1?op=GETHOMEDIRECTORY
Now i have another VM running fedora . I am assuming it as external client and trying to do external access but getting no output:-
curl -k https://<sandbox-ip>:8443/gateway/sandbox/webhdfs/v1?op=GETHOMEDIRECTORY
Can someone point me whats wrong with my settings.
Not sure about your topology but if you are using the default one (sandbox) you probably need to add basic auth e.g.
curl -k -u guest:guest-password -X GET https://<sandbox- ip>:8443/gateway/sandbox/webhdfs/v1?op=GETHOMEDIRECTORY
Also check the logs at
<knox_install>/logs/gateway.log
They should tell you more about what went wrong.
Good luck !
I'm running Elasticsearch 2.3.3, and am looking to set up a cluster so that I can have a simulation of a production ready set up. This is set up on two Azure VMs with docker.
I'm looking at the /_cluster/settings api, to allow myself to update settings. According to the elasticserch documentation, it should be possible to update settings on clusters.
I've run on each machine the command:
docker run -d --name elastic -p 9200:9200 -p 9300:9300 elasticsearch --cluster.name=api-update-test
so now each machine sees itself as the one master and data node in a 1 machine cluster. I have then made a put request to one of these to tell it where to find the discovery.zen.ping.unicast.hosts, and to update the discovery.zen.minimum_master_nodes, with the following command (in powershell)
curl
-Method PUT
-Body '{"persistent":
{"discovery.zen.minimum_master_nodes":2,
"discovery.zen.ping.unicast.hosts":["<machine-one-ip>:9300"]}
}'
-ContentType application/json
-Uri http://<machine-two-ip>:9200/_cluster/settings
The response comes back invariably with a 200 response, but a confirmation of the original settings: {"acknowledged":true,"persistent":{},"transient":{}}
Why won't elasticsearch respect this request and update these settings? It should be noted, this also happens when I use the precise content of the sample request in the documentation.
I always used this approach:
curl -XPUT "http://localhost:9200/_cluster/settings" -d'
{
"persistent": {
"discovery.zen.minimum_master_nodes": 2
}
}'
And, also, only discovery.zen.minimum_master_nodes is dynamically update-able. The other one is not.
A Spring Boot service running at localhost:7090/someurl accepts parameters Var1 and Var2 in POST requests. How can I test this from the CentOS 7 terminal?
When I type localhost:7090/someurl?Var1=something&Var2=anotherthing into the web browser, the logs show that an unsuported GET request was made. How do I mimmick a POST request to test this service? What do I type into the CentOS 7 terminal?
if it is a normal bash shell, you can always use curl (you maybe need to install it), for example:
curl -X POST -d "Var1=VALUE1&Var2=VALUE2" localhost:7090/someurl
I have a host configured into Ambari which no longer exists. Ambari still thinks it's there. When I try to delete it through the UI I get:
400 status code received on DELETE method for API:
/api/v1/clusters/handy091015/hosts/r-hadoopeco-celeryworker-07ac46a4.hbinternal.com/host_components/ZOOKEEPER_CLIENT
Error message: Bad Request
When I try to delete it via the api, with the command below, I get the same host information as with a GET:
curl -H "X-Requested-By: ambari" -DELETE http://admin:admin#ambari.handy-internal.com//api/v1/clusters/handy091015/hosts/r-hadoopeco-celeryworker-07ac46a4.hbinternal.com
I have tried the instructions here to no avail:
https://cwiki.apache.org/confluence/display/AMBARI/Using+APIs+to+delete+a+service+or+all+host+components+on+a+host
My question is: how do I get Ambari to no longer know about/try to do things with this host.
I am not able to reproduce your behaviour with Ambari 2.1.2 and HDP 2.3 stack.
Limitation:
Note that host removing is supported only for hosts with no master components, so if they are present, then deleting is not possible.
Options:
Try to do ambari-server restart, sometimes it have intermittent issues
If this is an option, I recomend you to do ambari-server reset and install it from scratch. If you don't have much setup, it will save your time probably.
If not, you may want to post ambari-server.log file additionally. This may help to debug the core issue
Another option - just ignore that host, it will not do much harm to you. You can move it to maintenance mode, that will ease cluster operation.
I have a simple Sinatra app. It takes a URL via a POST request, processes that URL and returns a code.
It works fine on my local machine through a Curl call:
curl --data-urlencode "path=PATH_HERE" localhost:4567/process
And it returns a JSON response.
However, once it's on production (EC2) I do a similar POST request:
curl --insecure --data-urlencode "path=PATH_HERE" https://faxattach.staging.myaidin.com/process
However, it doesn't return anything. The traffic is definitely going to the EC2 machine, which tcpdump confirms, but I'm not sure if it's actually hitting the Sinatra app.
How would I check if it is hitting the Sinatra app? The log files remain unchanged, checked using tail -f. And, if it's not hitting the app, how would I start investigating the reason?
Probably the problem is the port. By default HTTP requests go to port 80. You can run your app on that port to fix the problem with:
rackup -p 80
Your need to change the security policy and allow traffic to port 4567.
Or, as #ismael has suggested, you can run the Rack server on port 80. This would require root privileges and you might need to use sudo rackup -p 80 or rvmsudo rackup -p 80.