I am new to phantomjs. I need to get the data from an external site that will loaded through ajax calls that is when I try to load the external site the data will first load in json format in the ajax response then it will be parsed and then loading in the form of tables(i dont know how this will done because it is external website ) now my task is to get the json data which will be load through ajax.This json data I can see from the firebug in firefox then when i click the open response in a new tab then another tab will open and I will get the json data. Then I will copy the response url and use in phantomjs then i will get the data but the data i am getting from the response url is different when the data in the external website changes and again i need get the response url and replace it with the previous url I think this is wrong process i am doing .Even I tried of using the location by copy location in the firefox and tried to use that but it is showing nothing. Is there any way that I can load data with out updating every time.
Related
According to this website: I am trying to make a request, send data, and hope to get the result like json format, but not have to be.
In the sandbox page, I've checked the javascript, (I am using Chrome browser, you can go to "resources" -> Frame folder -> sandbox.ala.org.au), it uses AJAX to send and get data.
For example, there is a function (parseColumns), which is using ajax for sending data and getting result.
Is there any other way instead of ajax that could make a request to the server and get the result data? Because I am trying to create an application work similar with the sandbox.
One alternative is to make regular http requests. The problem with this approach is that you have to decipher the data format for input and output.
You can see all the request/response traffic in details, from and to your browser, using firebug (https://getfirebug.com/) and enabling the "net" tab
when you mouse over the "POST" line in the URL column, it shows you the complete URL. Click on the (+) to expand the request item to see details on the request.
Using a TWebBrowser in Delphi, I'm showing a website that uses JSON to update its content. To get the source code of the newly updated content and load it into a Memo, I believe I have to get the URLs of the GET requests. Unfortunately, these are always different and generated with an encrypted Javascript. Is there any way to list the URLs the GET requests go to in a similar way like FireBug does in its console view?
Thanks a bunch!!!
I need load localStorage data right on my homepage. But when the page loads, the PHP/SQL instructions are already processed.
There's a way to process it before the server interation and use the localStorage data in my SQL instructions?
ps. I tried AJAX, loading my homepage in a DIV inside an index page... but it didn't work.
The answer is no - you can't read the LocalStorage data on the first load of your web site. However, you can use Ajax to hit your server and send a copy of any LocalStorage value to your server. You mentioned using ajax to load your homepage in a div. That should work, if you pass along the LocalStorage information in the url. So for example, imagine you have a localstorage key called name. If you want to load the contents of foo.php into a div and have PHP get access to name, you would pass it in the URL: "foo.php?name="+escape(localstorage.name)". You can also use a Form post too.
I have been trying to scrape information from a site where the relevant information is continuously updated using Ajax calls . Inspite of repeated attempts, I haven't been able to determine the link from which the ajax calls are receiving data . So , I decided to change track and use firebug to get the data since it is showing me the data got from the Ajax call. So, does Firebug save the page source anywhere and can we access that by any chance?
This is the link and I am trying to get the song + movie name from the bottom of the page which is updated using Ajax calls. I have tried going through the firebug source code to determine the save but that has also not yielded any result.
Actually I did find the solution to my problem . Looking under Net tab in firebug, I was able to find out the ajax call that was editing the particular site. Using the response I finally managed to use urllib and urllib2 to get the required data :).
As for how firebug gets the source code , there were a few links that I read which said that Firebug gets the exact source code at that instant because it is a browser extension and so has more access to the browser variables and the server response. But not sure how correct the information is.
Cheers
In my page I am making an ajax request & in success I am building tabs of page.
And user can change the tabs & save the layout.
So after refreshing again this ajax has to make call & from database it needs to bring saved tabs list. on which the tabs need to be built.
But in my case, even i have written all my code in success, it is entering success & building tabs on old list only..
not with the new list.
even the response is coming as old list only..
& in fidler I am not able to see the url call.
But when i delete the cache at that time url call also goes & it is able to load with new tabs list.
If the response is still the old list your data isn't being saved in your backend, verify your database that the records are in fact saved.
If you are sending over POST client side caching shouldn't be the problem, if it is a GET request check firebug if the cachebuster parameter is being sent along. This is an extra GET parameter _dc that is being sent with each request to prevent GET responses from being cached by the browser.
Are you perhaps using some kind of reverse proxy by any chance like Varnish or Squid?