How can I check which platform my app runs, AWS EC2 instance, Azure Role instance and non-cloud system?
now I do that like this:
if(isAzure())
{
//run in Azure role instance
}
else if(isAWS())
{
//run in AWS EC2 instance
}
else
{
//run in the non-cloud system
}
//checked whether it runs in AWS EC2 instance or not.
bool isAWS()
{
string url = "http://instance-data";
try
{
WebRequest req = WebRequest.Create(url);
req.GetResponse();
return true;
}
catch
{
return false;
}
}
but I have one problem when my apps runs in the non-cloud system, like local windows system. It got very slowly while executing isAWS() method. the code 'req.GetResponse()' takes a long time. so I want to know how can I to deal with it? please help me! thanks in advance.
The better way to do this would be to make a request to get instance metadata.
From the AWS Documentation:
To view all categories of instance metadata from within a running
instance, use the following URI:
http://169.254.169.254/latest/meta-data/
On a Linux instance, you can use a tool such as cURL, or use the GET
command, for example:
PROMPT> GET http://169.254.169.254/latest/meta-data/
Here's an example using the Python Boto wrapper:
from boto.utils import get_instance_metadata
m = get_instance_metadata()
if len(m.keys()) > 0:
print "Running on EC2"
else:
print "Not running on EC2"
I think your original idea is pretty good, but no need to make the web request. Simply try to see if the name resolves (in python):
def is_ec2():
import socket
try:
socket.gethostbyname('instance-data.ec2.internal.')
return True
except socket.gaierror:
return False
As you said the WebRequest.Create() call is slow on your desktop so you really need to check the network traffic (using Netmon) to actually determine what took long time. This request, opens connection, connects to target server, downloads the content and then close the connection so it is good to know where this time is taken.
Also if you just want to know if any URL (on Azure, on EC2 or any other web server is live and working fine you can just request to only download headers by using
string URI = "http://www.microsoft.com";
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(URI);
req.Method = WebRequestMethods.Http.Head;
var response = req.GetResponse();
int TotalSize = Int32.Parse(response.Headers["Content-Length"]);
// Now you can parse the headers for 200 OK and know that it is working.
You can also use GET only a range of the data instead of full data to expedite to call:
HttpWebRequest myHttpWebReq =(HttpWebRequest)WebRequest.Create("http://www.contoso.com");
myHttpWebReq.AddRange(-200, ContentLength); // return first 0-200 bytes
//Now you can send the request and then parse date for headers for 200 OK
Any of the above method will be faster to get where your site is running.
On ec2 Ubuntu instances, the file /sys/hypervisor/uuid exists and its first three characters are 'ec2'. I like using this because it doesn't rely on external servers.
Related
In Laravel we have an api endpoint that may take a few minutes. It's processing an input in batches and giving a response when all batches are processed. Pseudo-code below.
Sometimes it takes too long for the user, so the user navigates away and the connection is killed client-side. However, the backend processing still continues until the backend tries to return the response with a broken pipe error.
To save ressources, we're looking for a way to check after each batch if the client is still connected with a function like check_if_client_is_still_connected() below. If not, an error is raised and processing is stopped. Is there a way to achieve this ?
function myAPIEndpoint($all_batches){
$result = [];
for ($batch in $all_batches) {
$batch_result = do_something_long($batch);
$result = $result + $batch_result;
check_if_client_is_still_connected();
}
return result;
}
PS: I know async tasks or web sockets could be more appropriate for long requests, but we have good reasons to use a standard http endpoint for this.
I am using finagle as rest client. In ClientBuilder I specify a range of hosts, but the request requires setting a url with host.
How can I avoid specifying host in the request and let finagle choose one ?
Thanks.
val client = ClientBuilder().hosts("host1:81,host2:82").codec(Http()).build()
val request = RequestBuilder()
// .url("http://host1/get") // dont want to specify host
// .url("/get") // MalformedURLException: no protocol
.buildGet()
var resp = client(request) // sent to host specified by url
It looks like you're using finagle-http module. It's not possible to build request without host in URL using RequestBuilder. Still, you can construct Request manually (or create your own RequestBuilder for further use)
I'd recommend however switching to finagle-httpx module (https://github.com/twitter/finagle/tree/develop/finagle-httpx). It's not compatible with finagle-http, but it has lots of API improvements and the ability to create requests without host in URL in among them, for example:
val client = Httpx.client.withTls("my.api")
.newService("host1.my.api:443,host2.my.api:443")
val req = Request("/get")
val rep = client(req)
We just upgraded our Heroku postgres database using the follower changeover method. We have over 50 dataclips attached to the old database, and now we need to move them over to the new database. However, doing them one by one will take a lot of time.
Is there a programatic way to update the database a dataclip is attached to, perhaps with the CLI tools?
At least once the old database has been deprovisioned, you can now (as of March 2016) reattach them to another database:
Go to https://dataclips.heroku.com/clips/recoverable. It will display your old database and a set of 'orphaned' dataclips and you can choose to transfer them to another database (in my case the promoted follower from the changeover).
Note that this only affects the dataclips that you created, it does not affect the dataclips one of your team members created and that you only had access to. So they will have to go through this process as well.
Official devcenter article: https://devcenter.heroku.com/articles/dataclips#dataclip-recovery
Thanks to Heroku CSRF measures, programmatically updating data clips is much more difficult than you might expect. You'll need to suck it up and start clicking buttons by hand, or beg their support team to do it for you, which is just as difficult.
There is no official support for programmatically moving the dataclips. That being said, you can script it out against their HTTP API.
The base URL is https://dataclips.heroku.com/api/v1/. There are three relevant endpoints:
clips /clips
resources (databases) /heroku_resources
move clip /clips/:slug/move
Find the slug of the clip you want to move, find the resource id of the new database, and make a post to the move clip endpoint:
POST /api/v1/clips/fjhwieufysdufnjqqueyuiewsr/move
Content-Type: application/json
{"heroku_resource_id":"resource123456789#heroku.com"}
I had over 300 dataclips to move. I used the following technique to update them all (essentially reverse engineering the dataclips API).
Open Chrome with Web Developer tools, Network tab.
Log into Heroku Dataclips
Observe the network call which returns all the dataclips, in JSON (https://dataclips.heroku.com/api/v1/clips). Take this response and extract out all dataclip slugs.
Update the database for one dataclip. Observe the network call which does this (https://dataclips.heroku.com/api/v1/clips/:slug/move). Right click, Copy as cURL. This is the easiest way to get all the correct parameters, since the API uses cookies for authentication.
Write a script that loops through each dataclip slug, and shells out to curl. In Ruby, this looks like:
slugs = <paste ids here>.split("\n")
slugs.each do |slug|
command = %Q(curl -v 'https://dataclips.heroku.com/api/v1/clips/#{slug}/move' -H 'Cookie: ...' --data '{"heroku_resource_id":"resource1234567#heroku.com"}')
puts command
system(command)
end
You can contact Heroku support, and they will bulk transfer the dataclips to your new database for you.
Batch working on dataclips
I've finally found a solution to work on my Dataclips as a batch using the javascript console and some scraping technique. I needed it to retrieve every dataclips. But it guess It can be updated as such:
// Go to the dataclip listing (https://data.heroku.com/dataclips).
// Then execute this script in your console.
// Be careful, this will focus a new window every 4 seconds, preventing
// you from working 4 seconds times the number of dataclips you have.
// Retrieve urls and titles
let dataclips = Array.
from(document.querySelectorAll('.rt-td:first-child a')).
map(el => ({ url: el.href, title: el.innerText }))
/**
* Allows waiting for a given timeout before execution.
* #param {number} seconds
*/
const timeout = function(seconds) {
return new Promise(resolve => {
setTimeout(() => {
resolve()
}, seconds);
})
}
/**
* Here are all the changes you want to apply to every single
* dataclip.
* #param {object} window
*/
const applyChanges = function(window) {
}
// With a fast connection, 4 seconds is OK. Dial it down if you
// have errors.
const expectedLoadTime = 4000 // ms
// This is the main loop, windows are opened one by one to ensure focus and a
// correct loading time.
for (const dataclip of dataclips) {
// This opens another window from the script, having access to its DOM.
// See https://github.com/buonomo/kazoo for a funnier example usage!
// And don't be shy to star and share :D
const externWindow = window.open(dataclip.url)
// A hack to wait for loading, this could be improved for sure.
await timeout(expectedLoadTime)
applyChanges(externWindow)
externWindow.close()
}
You'd still have to implement applyChanges yourself which I conceed is a bit tedious and I don't have time to do it know (if one does, please share!). But at least it can be done on all of your dataclips in a single function.
For an example usage of this script, you can take a look at the gist I made to scrape every dataclips and related errors.
I have an express app running with Sequelize.js as an ORM. My express app receives requests from my main Rails app, and because of the cross-domain policy, these requests are performed with getJSON.
On the client, the request is fired when the user hits a key.
Everything goes fine and express logs the queries being performed (and json being served) each time the user hits the key. Even trying to hit quickly it performs ok. But, whenever I leave the key pressed (or maybe several clients hitting the key very quickly), as it starts firing lots of requests, at some moment the server just hangs, all the requests from that point on are left pending (I see that in the Network tab of Chrome Dev Tools), and they slowly start to timeout. I have to reboot the server to make it respond again.
The server code for my request is:
models.Comment.findAllPublic(req.params.pId, req.params.sId, function(comments){
var json = comments.map(function(comment){
var com = {};
['user_id','user_avatar', 'user_slug', 'user_name', 'created_at', 'text', 'private', 'is_speaker_note'].forEach(function(key){
com[key]=comment[key];
});
return com;
});
res.json({comments: json});
});
And the findAllPublic method from the Comment model (this is a Sequelize model) is:
findAllPublicAndMyNotes: function(current_user, presentationId, slideId, cb){
db.query("SELECT * FROM `comments` WHERE commentable_type='Slide' AND commentable_id=(SELECT id from `slides` where `order_in_presentation`="+slideId+" AND `presentation_id`="+presentationId+") AND (`private` IS FALSE OR (`private` IS TRUE AND `user_id`="+current_user+" AND `is_speaker_note` IS FALSE))",self.Comment).on('success', cb).on('failure',function(err){console.log(err);});
}
How to avoid the server from getting stuck? Am I leaving some blocking code in the request that may slowly hang the server as new requests are made?
At first I thought it could be a problem because of the "forEach" when composing the json object from the Sequelize model, but I also tried leaving the callback for the mysql query empty, just responding empty json and it also got frozen.
Maybe it is a problem of the mysql connector? When the server gets stuck I can normally run the mysql console and perform queries on my database and it also responds, so I don't know if that's the problem.
I know I could just control the key event to prevent it from firing too many requests when the key gets pressed for a long time, but the problem seems to appear also when several clients hit the key repeatedly and concurrently.
Any thoughts? Thanks in advance for the help :D
Two things:
It seems like you have some path where res.render is not being called. It could be that the database you're connecting to is dropping the connection to your Express server after the absurd number of requests and the callback is never fired (and there's no database.on('close', function() { // Handle disconnect from DB, perhaps auto-restarting }) code to catch it.
Your client-side code should detect when an AJAX request on keypress is still pending while a new one is being started, and cancel the old one. I'm guessing getJSON is a jQuery method? Assuming it's jQuery's, then you need something like the following
.
var currKeyRequest = null;
function callOnKeyUp() {
var searchText = $('#myInputBox').value;
if(currKeyRequest) {
currKeyRequest.reject();
currKeyRequest = null;
}
currKeyRequest = $.getJSON('path/to/server', function(json) {
currKeyRequest = null;
// Use JSON code
});
}
This way, you reduce the load on the client, the latency of the autocomplete functionality (but why not use the jQuery UI autocomplete if that's what you're after?), and you can save the server from some of the load as well if the keypresses are faster than handshaking with the server (possible with a good touch-typist a few hours flight away).
I'm trying to simulate a ping operation to find out if a remote host is reachable. I could not find any conclusive code samples to do this for WP7 so I figured I'd try out the following.
What I'm looking for is confirmation of whether this is the appropriate way to do this.
Socket socket =
new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
SocketAsyncEventArgs args = new SocketAsyncEventArgs();
args.RemoteEndPoint = new DnsEndPoint (someIP, 80); // use HTTP port 80
args.Completed += (obj, eva) =>
{
if (eva.SocketError != SocketError.Success)
{
//raise an error or set a view model property indicating error
}
socket.Close();
};
socket.ConnectAsync(args);
Unless things have changed in Mango, you can only really reach other hosts with HTTP or HTTPS anyway - so "reachable" really means "is listening for web requests on a known URL"... so the simplest approach would be to make some harmless web request to the relevant server. WebClient is probably the simplest way of doing that, although in my experience it does more work on the UI thread than you'd really expect, so I've ended up using the lower-level HttpWebRequest. For a single ping-like request, you may be okay to use WebClient.