Pretty easy question I hope: does anyone know of a tool that will effectively scrape sites built with Microsoft Matrix? I could write the code in python, but it will take me way longer than I think I want to dedicate to the task, namely because of the really bad and ugly HTML generated by Matrix.
I have tried Web Harvey, Helium Scraper, and I tried the Web Scraper plugin for Chrome. WebHarvey choked on the HTML and couldn't load subsequent pages. Helium Scraper was able to move from one details page to another (the Next links were followed) but content from within the details pages was not lifted out. The Chrome plugin web scraper was not able to navigate links, with the popup window displaying an error page. My gut is telling me that this has to do with uniquely ASP.net things, but I could be wrong.
Any pointers or suggestions appreciated.
You know there are two completely different versions of Microsoft Web Matrix right? There's the one from 2003; i have no idea what its html looks like. There's the one from 2011 to current which uses razor cshtml source files to produce its html. In the 2011+ one, you write the html by hand; there's no drag and drop, and so it's unlikely you'll get consistent html from site to site.
I am just trying to streamline my MVC3 development approach. I use Razor in the View and was wondering what the recommendation is for easily creating more fancy UI widgets such as Menus. I would hope I could leverage some good open source components and plug them in. I do subscribe to a good 3rd party component library, but wonder whether this type of feature is best implemented via JQuery UI or old style HTML/CSS?
Many thanks.
In general, Stack Overflow isn't here to recommend tools for you. However, I will say that whether to choose a javascript tool or straight html/css depends on the needs.
It's best to work with the simplest solution that solves your problem. CSS can do menus quite well, and straight CSS menus are often the best choice. However, if you need things straight css can't do, then you need to add javascript. For instance, one problem with CSS menus is that it's very easy to "roll off" the menu and have it disappear from under your pointer. Most javascript menus add a delay that allows you to roll off for a second and get back before the menu disappears. This improves usability.
Even lots of fancy effects can be done strictly in CSS, but as with many such things.. it won't be compatible across all browsers, which is again where javascript comes into play. Sometimes the only way to be compatible across the board is with js (at least without losing functionality).
I wouldn't recreate the wheel on this one, since so many web sites already do this. If you are trying to streamline your development, I'd say use your 3rd party library or find a good jQuery plugin. You could probably easily create your own in HTML/CSS/Jquery, but it will add on to your testing later. An existing 3rd party/jQuery type of component would take significantly less time and have a lot of features you want already built in.
I'd recommend using MVC SiteMap Provider in conjecuyion with a JavaScript menu of choice e.g. SuperFish http://users.tpg.com.au/j_birch/plugins/superfish/. The menu fancy JS part part is the least important.
From the MVC point of view what I think really matters is how you manage and maintain your menu. MVC SiteMap Provider makes it really easy to build an maintain menus (can even use Attributes to make Actions appear on the menu). It can also do things like Security Trimming so if a user doesn't have permission to get to the action etc it isn't displayed on the menu.
I'd check that out.
I'm going to create a website with lots of business logic, connected to a background data model. For these reasons I chose ASP.NET MVC3 as development platform.
Unfortunately, I left web programming at the time of the old ASP and JSP and lately I worked with windows applications and C#.
Now I'm wondering which is the best(easiest, fastest, most reliable, most compatible with browsers) technique to create user views?
I explored a little Razor, but it is unclear for me, is it a so good choice? Is it supported by forums or still too fresh?
I'm very tempted of using webcontrols since I'm now addicted to them. Would this be a good choice? Can I use webcontrols just in aspx or in razor as well?
What about Ajax controls? Would it be a better choice?
Thanks!
Yes, Razor is a good choice. See here for a pretty good summary.
As Robert mentioned, "controls" go against the MVC pattern and will not even work in Razor. You want html helpers for small bits of markup (for example, to render a text box) and partial views for more complicated things (like a shopping cart widget)
Use of AJAX depends on the UI needs of your application. Initially it would be simpler to start without AJAX. Also, some clients might have JavaScript disabled and then AJAX would not work.
"web controls" are not appropriate for MVC at all - they go against the MVC pattern. Instead, look into "partial views" for creating common bits of UI that get reused across multiple pages
Happy Friday All,
I have an ASP.net 2.0 website that I'm updating to .Net 3.5. The site uses a master page and I've placed asp:UpdatePanel in it.
One of the pages has a 3-step wizard control that I'd like to "Ajaxify" in some way. Are there better controls in the AJAX control toolkit that provide an improved user experience over the asp:Wizard control?
Please share with me your efforts to improve upon the asp:Wizard control. Did you move to a different control? Stay with asp:Wizard and add effects to it?
Thanks,
Sid
Kind of late, but I'll answer. I used the Telerik RadAjaxManager control to ajaxify my Wizard control, and I regretted it because of performance issues.
If I were you, I'd pursue setting up divs and use JQuery to selectively show/hide them to emulate a wizard.
What Firefox add-ons do you use that are useful for programmers?
I guess it's silly to mention Firebug -- doubt any of us could live without it. Other than that I use the following (only listing dev-related):
Console2: next-generation error console
DOM inspector: as the title might indicate, allows you to browse the DOM
Edit Cookies: change cookies on the fly
Execute JS: ad-hoc Javascript execution
IE Tab: render a page in IE
Inspect This: brings the selected object into the DOM inspector
JSView: display linked javascript and CSS
LORI (Life of Request Info): shows how long it takes to render a page
Measure IT: a popup ruler.
URL Params: shows GET and POST variables
Web Developer: a myriad of tools for the web developer
Here are mine (developer centric):
FireBug - a myriad of productivity enhancing tools, includes javascript debugger, DOM inspector, allows you to edit the CSS/HTML on the fly which is highly valuable for troubleshooing layout and display problems.
Web Developer - again another great developer productivity tool. I mostly use it for quickly validating pages, disabling javascript (yes I disable javascript sometimes, don't you?), viewing cookies, etc.
Tamper Data - lets you tamper with http headers, form values, cookies, etc. prior to posting back to a page, or getting a page. Incredibly valuable for poking and prodding your pages, and seeing how your web app responds when used with slightly malicious intent.
JavaScript Debugger - has a few more features than javascript debugger provided by firebug. Although I must admit, I sparingly use this one since firebug has largely won me over.
Live HTTP Headers - invaluable for troubleshooting, use it frequently. Lets you spy on all HTTP headers communicated back and forth between client and server. It has helped me track down nefarious problems, especially when debugging issues when deploying your web app between environments.
Header Spy - nice addon for the geeky types, shows you the web server and platform a web site runs on in the status bar.
MeasureIt - I don't use this all too frequently, but I've still found it valuable from time to time.
ColorZilla - again, not something I use all that frequently, but when I need it, I need it. Valuable when you want to know a color and you don't want to dig through a CSS file, or open up a graphics editing app to get a color embedded in some image.
Add N Edit Cookies - this has been a great debugging tool in web farms where the load balancer writes a cookie, and uses the cookie value to keep your session "sticky". It allowed me to switch at will between servers to track down problems on specific machine. Also a good tool if you want to try to mess with a site that uses cookies to track your login status/account, and you want to see how your code responds to malformed or hacked info.
Yellowpipe Lynx Viewer Tool - yeah I know what your thinking, lynx, who needs it, its so 1994. But if you are developing a site that needs to take web accessibility into account (meaning accessible to users with visual impairments who use screen readers), or if you need to get a sense of how a web spider/indexer "sees" your site, this tool is invaluable. Granted, you could always just go out and grab Lynx for yourselfhere's the windows xp port that I use.
I've got a handful of other addons that I've used from time to time that I'll just quickly mention: FireFTP (one I installed wasn't stable and I've not tried a newer release), Html Validator (also found this one unstable, least back when I installed like a year ago), IE Tab (I usually just have both IE and FireFox open concurrently, but that is just me, I know many others that find this addon useful).
I'd also recommend the Web Developer extension by Chris Pederick.
As far as web development, especially for javascript, I find Firebug to be invaluable. Web developer toolbar is also very useful.
The ones I have are...
Y-SLow
Live Headers
Firebug
Dom Inspector
One that wasn't mentioned yet is this HTML Validator extension that I found very useful.
#Flávio Amieiro
MeasureIt is an unnecessary extension to have if you install the Web Developer Toolbar. Web Developer Toolbar includes a ruler as one of its features. Under the "Miscellaneous" category for Web Developer click the option "Display Ruler" to use a ruler identical to the MeasureIt one.
That will allow you reduce the number of extensions needed by at least one.
Firefox addons:
FireBug:helps web developers and designers test and inspect front-end code. It provides us with many useful features such as a console panel for logging information, a DOM inspector, detailed information about page elements, and much, much more.
Web Developer-gives you the power disable CSS, edit CSS on the fly, measure certain areas of a page and much more.
ColorZilla
Just click on the icon, hover over the area you'd like to know the hex color for, and click.
Window Resizer
to make sure the layout is displayed properly in the standard resolutions of today.
Total Validator
validating websites much easier by checking HTML, links, CSS and doing a lot more.
Web Developer for web development. Scribefire if you're a blogger-progammer
For web developing I use the Web Developer Toolbar, CSS Viewer and MeasureIt.
But I'm really not one of those who has a thousand of extensions to do everything. I like to keep things simple.
EDIT: Thanks to Dan's answer I don't need MeasureIt anymore. Can't believe I've never seen that! I guess I'll just have to pay more atention to this WebDeveloper toolbar.
Adding to everyones lists, Tamper Data is quite useful, lets you intercept requests and change the data in them.
It can be used to bypass javascript validation and check whether the server side is doing its thing.
I use Web Developer, it's a real time saver.
+1 for LORI ("life-of-request-info"). It's a very convenient alternative for rough measurements of the load time of a particular web page -- the kind of thing that you might otherwise use an external stopwatch for.
New Tab Homepage. Combined with a "speed dial"-type homepage (a personal, fast-loading page of links that you use frequently), helps you get where you're going faster when you open a new browser tab.
LastTab. Changes the behavior of Ctrl+Tab to let you navigate back and forth between your most-recently-used tabs with repeated presses of Ctrl+Tab, the same way that Alt+Tab works in Windows. Also provides a nice view of all open tabs while Ctrl is still being held down for easy navigation. (The resultant behavior is very similar to the Ctrl+Tab behavior in recent releases of Visual Studio.)
FireFTP is good for grabbing/uploading any necessary files.
I find Hackbar to be quite useful. Very useful if you want to edit the querystring part of the url, to test for vulnerabilities, or just general other types of testing where you might end up with complicated query string values.
I was learning DOM inspector, but I've switched to Firebug.
Some of which has been missed above are here
Load Time Analyzer – View detailed graphs of the loading time of web pages in firefox. The graphs display events like page requests, image loading times etc.
Poster – A must have tool for web developers enabling them to interact with web services and other web resources.
Aardvark – A cool extension for web developers and designers, allows them to view CSS attributes, id, class by highlighting page element individually.
Fiddler is a really great debugging proxy. Think of it as a more powerful version of the "Net" panel in Firebug or the Live HTTP headers.
It used to be an IE-only extension, now it also has hooks into Firefox.
Groundspeed, is useful for testing server side code. It was created for input validation tests during pentest, but can be useful for any test that require manipulating input (similar to TamperData).
It lets you control the form elements in the page, you can change their type and other attributes (size, lenght, javascript event handlers, etc). So for example you can change a hidden field or a select to a textbox and then enter any value to test the server response and stuff like that.