I am debugging an API request in Firefox and I am looking to filter multiple domains in the developer tools' Network panel. I can filter by one domain with the domain:domainname.com, but how do I add additional domains?
Filtering by two domains is not directly possible because the filters in the Network panel is always accumulative, though one workaround is to use a regular expression filter.
This allows you to provide several domains separating them by pipes like this:
regexp:domain1.com|domain2.com
That should work in most cases, but note that this is not just filtering by domain but searching within all the data inside the requests. That means that when the domain name appears in one of the other columns, the request will also be listed.
Another way to achieve this is to use negative filtering by prepending the filter expression with a minus.
So in order to get the requests of two domains you have to write several -domain: expressions for all domains you want to exclude.
There's also the regexp keyword to use Regular Expressions for URL filtering.
https://developer.mozilla.org/en-US/docs/Tools/Network_Monitor/request_list#Filtering_by_properties
If you want to filter out domains that are cluttering up your requests pane, you simply use the - prefix and separate them with spaces.
-domain:domain.com -domain:domain2.com
The important thing here is that in Firefox you cannot use wildcards like you can in Chrome. So this won't work.
-domain:email.*.com
This is what kept tripping me up. Also, for awhile I thought you separated them with commas. Nope, use spaces.
Related
I am trying to crawl different websites (e-commerce websites) and extract specific information from the pages of each website (i.e. product price, quantity, date of publication, etc.).
My question is: how to configure the parsing since each website has a different HTML layout which means I need different Xpaths for the same item depending on the website? Can we add multiple parser bolts in the topology for each website? If yes, how can we assign different parsefilters.json files to each parser bolt?
You need #586. At the moment there is no way to do it but to put all your XPATH expressions regardless of the site you want to use them on in the parsefilters.json.
You can't assign different parsefilters.json to the various instances of a bolt.
UPDATE however you could have multiple XpathFilters sections within the parseFilters.json. Each could cover a specific source, however, there is currently no way of constraining which source a parse filter gets applied to. You could extend XPathFilter so that it takes some extra config e.g. regular expression a URL must match in order to be applied. That would work quite nicely I think.
I've recently added JsoupFilters which will be in the next release. These should be useful for your use case but that still doesn't solve the issue that you need an implementation of the filter that organizes the resources per host. It shouldn't be too hard to implement taking the URL filter one as a example and would also make a very nice contribution to the project.
We have an ASP.NET Web API controller that accepts a comma-delimited string of database column names from the query string. This string of column names is checked to ensure only alpha characters, underscores, and commas are present in the string; no spaces, special characters or numbers allowed. This string is eventually added to a SQL statement via string interpolation exactly as passed to the API in the query string.
Any other query string parameters are added to the SQL statement as parameters. We encountered an issue with parameterizing a list a columns not be interpreted correctly by Oracle so we ended up with this solution.
Although not ideal, are there any additional steps to prevent SQL injection attacks via this vector?
We are currently using Dapper for data access but frequently use plain ADO.NET.
If that's the only thing you can do, as long as you make sure you don't allow numbers and special characters like quotes, equal and different you will be fine. I would even say you should be good with numbers, I have seen cases where numbers are actually used as part of column names so you could relax that restriction a little.
You could basically create a white list of characters allowed and when the request comes in, if any character is not part of that list then you can throw Bad Request and stop right there. A white list is much better than a black list as it can be much shorter and it's easier to maintain.
Another layer of protection can come from the API itself. Make sure you :
authenticate the access to your controller, no anonymous users allowed, maybe something like OAUTH2 if you can.
turn that API call into a POST and send the list of columns in the body of the request not in the query string.
issue the request over HTTPs.
All these things together should keep you well protected as no one can now watch and see what you are sending through and they can't issue their requests either if you protect your endpoint properly.
Some of web api endpoints have strings as input parameters and some times I have to pass special characters like /,\,( and ). But, it is not allowing special characters because those has special functionalities. Is there any way solve this problem.
Is there any way solve this problem.
No.
(Please) Stop Using Unsafe Characters in URLs. URLs are by definition machine readable. There are rules to follow about what can and cannot be part of a URL.
Although technically using unsafe characters is supposed to be possible through encoding, all bets are off that all browsers, web servers, and firewalls will treat them as "plain text" when encoded instead of assigning them special meaning. Some may just reject the URL entirely, considering it SPAM or attempted hacking.
We're using RavenDB on the backend and so all the DB keys are strings that contain forward slashes e.g. users/1 in /api/users/1/foo.
This question superseeds this question for the previous version of the Web API.
The solution I came up with v1 works fine but it's a little messy setting up the tables. v2 introduces attribute routing which would be a very welcome change.
However, I could find no way to match parts of the URL that contain forward slashes. I tried creating a custom constraint but it seems that the pipeline will split the URL segments before trying to match it.
What's the simplest way to extend the URL => attribute matcher such that it can accept constraints that contain full-blown regexes (i.e. not regexes limited to one segment)?
I am trying to build a simple RSS filter in Yahoo Pipes for the first time.I would like to filter by date.
I have a bunch of RSS feeds from a "fetch feed" data source and I connect them to a filter. I am now supposed to get a item.y:published.utime field in the Rules dropdown. However, it is not there: I do not have any of the item.y fields at all.
If I start by editing an example filter, these fields are there. What is going on?
Dropdown lists include only the basic options as these are most commonly used by novices making simple pipes. Yahoo developers also left many options off the dropdown lists because some of them are quite long and would spoil the layout such as this example.. *item.y:published.day_ordinal_suffix*
If you want to use the special 'y' fields as sources of data then simply type them by hand into the rules box of the filter module using the keyboard device attached to your computer.