I am developing a tool to extract people's web visits on Mac and Windows (history on Chrome, Firefox, and Safari). I have successfully retrieved history generated by Chrome and Firefox. However, I am having trouble retrieve the complete browsing history on Safari.
I have done some searches and know that, on Mac, Safari stores history records in these two files:
/Users//Library/Safari/History.plist (storing the "last visit" time of a page and visit count)
/Users//Library/Caches/com.apple.Safari/Cache.db (storing blob cache data, URL, timestamp).
Reference:
http://www.appleexaminer.com/MacsAndOS/Analysis/HowTo/SafariBrowserAnalysis/SafariBrowserAnalysis.html
Originally, I was planning to use the information from these two files (or just the Cache.db) to re-construct the complete browsing history on Safari. However, after examining the cache.db for a few times, I realize that it might not store all the records.
For example, I have tried to visit Google through 1) typing in the URL in the address bar and 2) using bookmarks to access the page. Both of them leave records with no entries like "http://www.google.com/". *The closest one is "www.google-analytics.com", with a long list of parameters following after*, but I assume that this could potentially also be records generated when I visit other web sites that also use Google Analytics, so I cannot use this to say that it means someone visits Google.
Did I miss something?
Are there other files that I should know to reconstruct the complete browsing history for Safari?
Do people have similar experience or know whether it is possible to reconstruct the complete browsing history for Safari?
There're also Files stored in /Users/user/Library/Caches/MetaData/Safari/History/.
I got it. Please follow following path then you will retrieve Browser history for iOS7 iPhone.
/private/var/mobile/Applications//Library/Safari/History.plist
In iOS7 safari app not a system app. When you open Applications folder then you can find history and other plist files.
Related
So I am developing a WebExtensions based add-on for Firefox. My add-on successfully stores and later retrieves some cruft (using browser.storage.local.set()/get(). But how can I view and edit the contents interactively (using keyboard and mouse), and not using WebExtension code? For example, to quickly modify some stored object properties and/or values by hand, for quick testing?
I figured I'd try, "Storage Inspector", SHIFT-F9 or Menu -> Web Developer -> Storage Inspector. But there's nothing there. I have one tab open which is on about:addons page, and Storage Inspector just has "about:addons" in each section (Cache Storage, Cookies, Indexed DB, Local Storage [I want this] and Session Storage). But they seem tied to the about:addons page, tied to whatever tab the Storage Inspector was opened from.
Hmm, so I click on my Options page, which I configure to open in a tab, and every storage area has moz-extension://random-uuid-etc, which is not the ID of my Addon, presumably because I started the browser session with the web-ext which injects the addon into a profile, for live updates on file changes.
I know, you're saying, but, that will create a new profile each time, so you won't have any old values saved anyways. Sure, ok, that might be an excuse if I hadn't already set stuff with the add-on before trying to find a way to view and edit. So really that's not an excuse for the local storage to be totally blank in storage inspector.
But additionally, I am using web-ext with -p C:\path\to\profile --keep-profile-changes, which stores profile changes in a persistent manner. I have verified that values previously saved from the add-on remain in local storage, even after commenting out the call to set, exiting, saving file, restarting, etc.
So how do I view and edit local storage used by a WebExtension based Firefox addon?
A "Web Developer" tool that has a caveat of not working for web development, is a prized dunsel in any toolkit.
Currently this isn't possible in Firefox. Vote for https://bugzilla.mozilla.org/show_bug.cgi?id=1292234 to make it happen one day.
In the mean time, I use Chrome for Firefox WebExtensions development where the Storage Area Explorer extension offers the feature you're looking for.
In FF 60esr I can view extension local storage data in the page rendered by navigating to about:debugging and selecting debug on the extension in question. The URL is of the form moz-extension://{Extension UUID}/_generated_background_page.html and it is possible to navigate directly to it. As you would expect, local storage data appears in Web Developer/Storage Inspector - example below.
As for editing it, The GUI needs some improvement, but this is possible by double-clicking in either a key or value field, as per the docs.
yesterday I created a simple "quote of the day" script for my google site. The script reads the quote from a google sheet. I also wanted each quote to go
with a companion image, which I host in my google drive (the image Ids also are stored in the sheet).
I followed the instructions explained here, and everything seemed to work fine. The permissions for the images was "everyone with the link within my institution" (this is supposed to work in a website on the google apps for education).
It was a success, because the images did display correctly.
Today I opened that page again, and the images have disappeared.
I know that image hosting was discontinued in 2016, but I think it was a different way of hosting (no permalink). Indeed, as I mention, yesterday I could see the images in the website.
Also, when I write the "permalink" in chrome address bar I get "403. That’s an error. We're sorry, but you do not have access to this page. That’s all we know." This happens despite I'm the owner of the file, and I'm logged in the correct google account. Also, it happens irrespective of the browser and/or machine I use (I tried with my phone too).
I also tried what explained here, although I'm not sure I understand that completely. There should be a problem though, because instead of the image I get the alt text (NA).
Is the "permalink" method still supposed to work?
Could this be a problem of excessive traffic? I haven't even published the page yet, I only did some testing...
What is the best way of displaying images from google drive into google sites, without resorting to third party services?
Thanks a lot
Francesco
You already answered your question. Hosting on Google Drive has been stopped.
However, you can try the workaround from this SO post where you'll use the URL:
https://drive.google.com/uc?export=view&id={IMAGE_FILE_ID}
It worked fine when use with the <img src="{IMAGE_URL}"> HTML tag. That's the closest thing to what you're trying to do.
You can also try Firebase Storage where you host your images on the cloud and display them on your web page.
This is at least a partial answer to the problem I tried to explain, which I'm summarizing below:
I was hosting images for a page in my google site on my google drive, via the syntax
https://drive.google.com/uc?export=view&id={IMAGE_FILE_ID}
the sharing settings of the individual images was "anyone with the link within [my institution]";
this worked for one single day and then stopped working: the images stopped showing, and I could not figure out why. All this happenened a few days ago (oct 21-22 2017), and hence cannot possibly have to do with google discontinuing a previous service, which happened in 2016;
this happened despite I was logged into the account in my institution (and hence the owner of the images).
I found out that the problem appears to be in the sharing settings. Apparently, the images only display if their sharing settings are "public on the web" or "anyone with the link". Instead "anyone with the link within [my institution]", or more restrictive settings won't work. The first settings were mentioned in the webpage I linked, but I thought it would also work with "anyone with the link within [my institution]" (and, for one day, it did).
I also tried by setting the permissions of the parent folder. In that case only "public on the web" would work. With anything more restrictive than that the images won't display.
Despite I carefully double checked all this, I think this permissions thing seems to be a bit erratic though.
As I mention, on the first day (oct 21) the images would display even with "anyone with the link within [my institution]. I could swear on my life that, before the systematic tests I just finished carrying out, all was working with the folder settings to "anyone with the link [within my institution]". Now that setting won't work.
I know, I said "it appears" a bunch of times. This is because all of this is just from my point of view, so far. I still have to share the link of the page I'm preparing. As soon as I can I'm going to ask a colleague to take a look at it.
What I anyway want is that the page displaying the images is visibile only to people within my institution, for the moment being.
However, now I'm at least able to see the images hosted in drive, in my website.
Yeah,I use FireFox and IE9.0+,sometimes,Chrome.
Chrome always takes much RAM usage but it is faster.
IE10 which i am using is much faster than before,but because of my win7 os,I can't sync nothing simply.
So,I like FireFox best.
Well,I want to implement the function in a app like this:
When i use IE to browse pages, I wanna watch the favorites's folder in real time,if i add a page to favorites, the app will create a online bookmark immediately into firefox's bookmarks which has been logined .
How to implement?Have any sapiential ideas in your mind?
Best in javascript or c# language!
Pls share for me ,thanks in advance!
For the other's answer in Stack Overflow ,I get the correct skill to operate firefox bookmarks.
Firefox online bookmarks are stored in the sqlite which named "places.sqlite" as a backup file.
In my computer,It is under "C:\Users\xx\AppData\Roaming\Mozilla\Firefox\Profiles\gyhdax0e.default",xx is represented your windows current username,and the latest folder(gyhdax0e.default) will be another name in your computer(such as abcdef.default).
And then you could operate table "moz_places" and "moz_bookmarks" for your function.But please keep signing in.
Sorry ,I forgot the original url.
So I know this might sound crazy, as it is technically a security concern which I understand. So I'm just trying to find out if there's any ideas on how to handle something like this.
Anyways, long story short, I was told to look into figuring out a possible way to scrape information from another browser window/tab. I have been asked to do this because, and I know this sounds crazy too, but the users of our website are incompetent enough to not be able to copy/paste and or type correctly something from a different website. I know it's tough for some to have to have several things in their workflow, but this is basically what they do: Go to their first website (after logging in) and bring up a record with information on it...including an identification number. Then, the user should take that number and go to the second website, our website (after logging in), and type it that number in a textbox (and eventually do some other stuff). But we have found that getting that identification number from the first website to ours is difficult for them. Some copy/paste correctly, some copy/paste too much text from the page, some write it down on paper then type it in our website, and some just seem to have trouble visually "copying" the number from site to site.
What I was thinking was that this could happen: the user would have already brought up the record on the first site, then they would come to ours. They could click a button, and that would run whatever I/we here come up with, that goes and finds the other browser window, finds the specific text needed, and puts it in our textbox. Sounds simple, right? HA.
The first website is not owned or managed by us in any way, otherwise this might be a little easier.
A little bit of background information: unfortunately, I'm technically targeting IE >= 10 through 9, so if there's a solution just for this (why I tagged vbscript), then that's great. If there's a broader solution (like with an applet or browser extensions... http://crossrider.com/ ), then that's even better, but not important. If it helps, we already have a hidden applet on the page that accesses the OS (yes, it has the mayscript attribute on the element so it is able to), so I thought that could be something to incorporate with. Also, the way I expect to know which window/tab to access is by URL and/or document title - either will be very specific.
We cannot install stuff on the users' computers, at least something outside of the browser (like extensions). I'm not sure how browser extensions work, so I'm wondering if they'd need to be "installed".
I know of HTML5's postMessage, but it only has partial support in IE (and none in IE <= 7)...and the partial support refers to not including exactly what I might need. It also requires that the other website be listening (which we don't have control over, but technically might be possible to include). So it doesn't count :)
The things I found with Java are to possibly find the list of processes currently running, but I don't know how to access/control one. Especially how to access the browser's Document.
And vbscript...I just don't know. I don't know if it's just me, but I can't seem to find good documentation on it, so I'm not sure what can be done with it.
Even if I could get control of the other browser window, I don't know how I would get information from it (like the DOM).
I'm not looking for code, just ideas...I'll do the research. And although it may sound impossible, don't just brush it off because Javascript can't do it - I haven't.
UPDATE:
I ended up developing a browser extension with http://www.crossrider.com/ which wasn't ideal, but works.
You could use a bookmarklet for this ... the user would have to drag the bookmarklet into their bookmarks bar on their browser, but if doing that wasn't beyond your user's abilities/the technical restrictions you've mentioned, then you'd definitely be able to send the information you need back to your site that way.
You'd just need to give your users instructions to:
i) drag the bookmarklet into their bookmarks bar on their browser
ii) go to the website in question and click the bookmarklet
you could code the bookmarklet so that it would grab the info you need, and redirect the browser to your website. All done in one click.
I think you may be thinking about it in the wrong way when you talk about posting from one 'window' to another. You could write the bookmarklet so that it would do a http post of whatever information you wanted into your site from the other site, and it could also redirect the window that they were looking at when they clicked it (the other site) to your site. Or if, for some reason, you didn't want to redirect the the window that they had the 'other' site in to your site, then you could add a listener to your site so that once the bookmarklet had posted the info you require then the window with your displaying could automatically update. The first option would make more sense and be easier though.
Maybe to open the other site from button/link resided in your site using window.open() method?
I use Firefox to develop a web site and at the same time to browse the web, read my gmail, etc.
The problem is every now and then I need to delete the cache and or remove the cookies of the web app, but I want to stayed logged in in the other web pages I am visiting.
Do you know a Firefox plugin (or Firefox trick) that can help with this issue?
Preferences → Privacy → remove individual cookies
They are grouped by domain, and you can eliminate all cookies from selected domain with one click.
Update:
Option → Privacy → remove individual cookies
There are two add-ons for FF that every web developer needs and they are Web Developer Toolbar and Firebug, the Firebug extension YSlow comes in very handy too.
Web Developer Toolbar has great cache and cookie control down to individual cookies. Firebug lets you mess with the DOM and CSS directly for a page and YSlow is good for page weight and response times.
Be careful having Firebug enabled for JavaScript heavy sites as they really slow down. GMail will warn you about this and tell you to turn it off. I have had a problem trying to disbable sites with the menus when the tool is embedded in the browser at the bottom but opening it in its own windows the sites menu works fine.
I personnaly also like IE Tab which means I can quickly view a page in IE without leaving FF and I can also get it to load anything that only works in IE with it.
Cookie Monster might work?. Also, have you considered just using two profiles in Firefox? You could setup two profiles with two icons and use one profile for the website and the other for the non-development. The only problem would be you can only have one open at once.
Also, two browsers (Chrome, IE, Firefox, etc) might work well too.
WebDeveloper extension allows to delete cookies for domain. It also can clear cache, although for me Ctrl-F5 is enough usually.
I had the same problem and found a firefox addon to remove the cookies for the current site that you are displaying:
https://addons.mozilla.org/en-US/firefox/addon/remove-cookies-for-site/
Using that I don't have to navigate the firefox menus to find my site in the list (which was too time consuming). This does not cover the cache-issue. For that there is the Ctrl+F5 other people have mentioned. It is covered in good detail here:
What requests do browsers' "F5" and "Ctrl + F5" refreshes generate?
Best solution:
1) Just Click CTRL+H and search the site.
2) right click on it and choose Forget About this site
(source: https://superuser.com/a/733154/249349 )
If you would like to use firefox, clear cache and cookies for one domain and stay logged in (and retain cache) at other sites, you could run a second firefox profile concurrently using the '-no-remote' parameter. I use this all the time to view sites I am building from an anonymous and administrative perspective at the same time.
One way to do this (in windows xp) is: make two profiles, then startmenu->run type 'firefox -profilemanager', untick the 'don't ask next time' box and load profile 1, then startmenu->run type 'firefox -no-remote' and load profile 2. You should now have two seperate firefox profiles running.
Clearing cookies for a single domain is easy with any one of a number of tools. Clearing the cache for a single domain or at the individual page level is sometimes useful, but hasn't turned out so easy for me thus far.
One specific situation is changing a meta redirect, where you can't just ctrl-F5 (you're being redirected, after all). Using a different profile isn't an acceptable general solution. Web Developer toolbar can disable the cache to test that the change is working but it's only a temporary fix, and not one that can readily be suggested to a client in order for them to view the change.
I use SQLite Manager for firefox to remove single sites from cache. All you have to do is open places.sqlite in your user profile folder using this plugin and remove the site. It's not exactly an easy solution but it does work. Make sure you backup your profile 1st!
Ross, I have the perfect answer for you. Get on the current tab that you want cookies cleared. Click tools > Page Info. Or install Page Info Forms & links for a shortcut. Then click the Security Tab. Then View Cookies button. It will bring up all the cookies for the relevant domain. Click the ones you want to delete, and click remove.
Seems we have cookies covered here.
As for cache, the trick is to make your development server force a refresh.
In httpd, you probably have some some cache directives to swap out for these:
Header set Cache-Control "no-cache, must-revalidate"
ExpiresActive On
ExpiresDefault "now"
In header:
<meta http-equiv="cache-control" content="no-cache, no store"/>
The benefit of doing it the httpd way is that your scripts/images/css get refreshed.
In that way you are not restricting yourself to just browser with web developer toolbar, you can use other browsers such as chrome that does not do the no-cache thing so easily.
Another extension that handles the cookie part of the question with a little more ease than the rest is Close'n forget
https://addons.mozilla.org/en-US/firefox/addon/closen-forget
You can use it by toolbar button/context menu/keyboard shortcut in order to close the current page, discarding any cookies it held.
Optionally, the site can be removed from browser history and (again optionally) all the pages you visited in the current page can be removed from browser history too.
Simple solution is simple - use two different browsers.
Say chrome for your email and Firefox for dev.
Also, you have Cookie Swap. but it doesn't really work quite as smoothly as you'd like.