My Project Client wants all the application to be Veracode Compliant....After giving it for Veracode Staic Scan found that there are 326 flaws which are needed to be fixed as a part of Cross Site Scripting Error.
The application for which Scan was given is basically built using ASP Classic (VBScript).
some of the lines reported are:
<td <%=feltfarve%> nowrap><%=tabeltekst_start%><%=rsDBox("filnavn")%><%=tabeltekst_slut%></td>
<a onmouseover="EnterContent('ToolTip','<%=f(33)%>','<% if sagerRS("dkrerefnr") <> "" then Response.Write (replace(f(34),"%1", sagerRS("dkrerefnr"))) else Response.Write(replace(f(34),"%1", f(35))) end if%>'); Activate();" onmouseout="deActivate()" href="javascript:void(0)"><img src="/secure/images/rolloverknap/gray_e.gif" border="0" WIDTH="12" HEIGHT="12"></a>
Response.Write " <a onmouseover=""EnterContent('ToolTip','"& f(36) &"','" & kretxt & "'); Activate();"" onmouseout=""deActivate()"" href=""javascript:void(0)""><img src=""/secure/images/rolloverknap/gray_i.gif"" border=""0"" WIDTH=""12"" HEIGHT=""12""></a> "
What type of fix can be applied to solve these Cross Scripting Flaws??
Looking at your code there are several variables that could create problems. I'm not exactly sure how Veracode checks for problems, but if any of these variables come from a user supplied value, they pose a risk.
First the basics, if you allow > and < in any variable, especially combined with allowing " and ', everything is a thread. They can insert pretty much anything.
In your case allowing " and ' is enough to create problems, because a lot of variables are already inserted inside Javascript code like onmouseover and onmouseout
Besides that feltfarve could always cause a problem, even if you don't allow these charactors.
feltfarve in <td <%=feltfarve%> nowrap> is inside a tag without qoutes or anything. If somebody could modify the value to onmouseover=doBadStuff() your in trouble. If they combine it with style they could even make it full screen, so almost anybody would hover it.
Keep in mind that if you directly insert values from your database, what %=rsDBox("filid")%> is doing I think, then you must be 100% sure that you properly check them on insert.
Since DB best practise is to insert data as provided (not encoded), you should always check these values before you put them in your HTML.
I think we should use Server.HTMLEncode to fix XSS (cross site scripting) threat.
I am using Ember.js to build a website for my company.
The problem I am having is that the initial load time of the page is around 10 seconds.
I cant give you the profiling data from chrome because I can't get them out of work.
However what I noticed when looking at them is that there is a function called "Get" which takes in total around 8.5 seconds. I realize this is probably just many uses of Ember.Get(), but still this is just the initial page load.
I don't know if this is normal or not but it's extremely unpleasant. Is there something I can do about this?
Thanks, Jason
try using a production release (the minified version of ember.js), it uses a significantly faster get.
Are you rendering some very large lists? If so look into using List View.
If you have a ton of fields being bound that don't ever change modify them to be unbound.
{{unbound someField}}
If you are having some weird issue where a template is taking a long time, yet you aren't sure which one it is, you can add some timestamp logging to the beginning of your templates to track down the culprit. At the bottom I whipped up a quick helper. In your template you could use it like so. It will print out a timestamp with that data point passed in.
{{logTime this}}
{{logTime name}}
Ember.Handlebars.helper('logTime', function(someField){
var d = new Date,
timestamp = d.toTimeString().replace(/.*(\d{2}:\d{2}:\d{2}).*/, "$1") + "." + d.getMilliseconds();
console.log(timestamp + " - " + text);
return "";
});
I'm using openReports that uses freeMarker formats as a template.
The following:
<#display.table name="results" class="displayTag" sort="list" export=true pagesize=10 requestURI="queryReportResult.action">
<#display.column property="first_name" title="First Name" sortable=true headerClass="sortable" />
<#display.column property="last_name" title="Last Name" sortable=true headerClass="sortable"/>
</#display.table>
The data is automatically grabbed using a stored procedure.
This will create a sortable table, does anyone know how I could access just the first row of data. I intend to save it into a variable and output it in some part of the page.
The reason I want to do this is we have a basic report and what would make it perfect is if I could print some from it toward the top of the page above the report.
I know a lot of people aren't familiar with OpenReports, but I figured freeMarker does have a pretty good following. I understand if this is pretty obscure
From what I can see from here, the #display.table call prints the whole table at once, so there's nowhere to insert the FreeMarker code to catch the first row. But of course you should check the documentation of #display.table to see if it offers any helpful options. But, I suppose you have already done that. So as a last resort, you can capture the whole table into a variable with <#assign tableHTML><#display.table ...>...</#display.table></#assign> and then extract the first row with a regular expression (or something like that) from the value of the tableHTML variable.
I'm running Opencart 1.5.2 on my server and after importing a large number of products I got a massive speed down. I tried installing a vq mod which had to speed up the site... it didn't.
Store
I know I have some elements on the site which are relatively big but before the import it was running fine.
The product count in the categories is a significant source of slow page loads in opencart. There are a few places this could be calculated and you will have to get rid of all of them to notice an improvement in page load time due to this factor.
If you edit the Catalog module you can disable the product count for the navigation menu (displayed in the left column by default) by setting Product Count: to Disabled.
The default theme also has a product count displayed in the main menu of the site. You can find this code in catalog/controller/common/header.php:
$product_total = $this->model_catalog_product->getTotalProducts($data);
$children_data[] = array(
'name' => $child['name'] . ' (' . $product_total . ')',
'href' => $this->url->link('product/category', 'path=' . $category['category_id'] . '_' . $child['category_id'])
);
Remove or comment out any references to $product_total:
//$product_total = $this->model_catalog_product->getTotalProducts($data);
$children_data[] = array(
'name' => $child['name'],
'href' => $this->url->link('product/category', 'path=' . $category['category_id'] . '_' . $child['category_id'])
);
That should take care of all the references in a default opencart install, but if you are using a custom theme or modules there could be more. More generally, you can search the entire catalog/ directory for references to model_catalog_product->getTotalProducts().
If you search for other references to getTotalProducts() Make sure you don't remove the references that use the product count for pagination, otherwise the pagination will not work properly. Here is an example from catalog/controller/product/search.php, a file that needs the product count to function properly.
$pagination->total = $product_total;
Removing these references has led to almost a 10x speedup in page load time on my development servers in an opencart install with ~2,000 products.
If you have seo urls enabled in your opencart, it can be a significant source of slowness.
I have oc v1.5.5.1 with about 3K categories and 40K products. I run it on shared hosting, and at first my loading times were about 40 seconds and higher. Then i got a little bit more into opencart, and realized that it is making a huge number of requests on oc_url_alias table in oc database. So my advice is:
1.Add index to query column in oc_url_alias table
With this my load times went down to about 7 seconds per page.
2.Add index to keyword column in oc_url_alias table
After both steps I got it down to about 1-3 seconds per page.
Now my oc is running in about 1 second per page, I achieved that by adding some more indexes in my mysql table, according to this guy's post http://bloke.org/php/opencart-is-slow-with-many-categories/, which are:
Index on parent_id column in category table
Index on category_id column in product_to_category table
Index on language_id column in category_description table
Index on store_id column in category_to_store table
Index on attribute_id AND language_id columns in product_attribute table
Index on manufacturer_id column in product table
Index on language_id column in product_description table
Index on store_id column in product_to_store table
Also, as aforementioned in other answers, I also removed product count in: catalog/controller/product/category.php and catalog/controller/module/categories.php.
If still experiencing slowness you can completely disable categories module from sidebar.
At last, you can try using this extension: http://www.opencart.com/index.php?route=extension/extension/info&extension_id=10464, but it does not support seo urls as it is, so I had to tweak it a little (by decoding $_REQUEST['_route_'] parameter, which contains seo url request, in function run).
Got it sorted out. There seems to be a "new" approach in SQL queries discovered by OC team and it should be called "DDoS yourself to death".
Product base has now grown to 37k products in 129 categories(from 18k over night, lol... shouldn't have writed that automated import scripts and give it to a lamer...) and load times have grown from 6-12 seconds to 20-25 seconds, hitting the SQL server hard:
[quote]Вопросов начиная с запуска: 514,064,911(ques since startup)
ø в час: 1,301,788(ques avg for hour)
ø в минуту: 21,696(ques avg for minute)
ø в секунду: 362 (ques avg for second)
[/quote]
This is NOT normal, since there are 2 users on the system -the automated script(limited - 30 ques per second, then sleep(1)) and me(362-30 = 332 queries per second? by a human? WTF devs?). Based on this statistics OC in this way will need a serious server farm to serve 500+ users at the same time. Not gonna happen. Not in this life.
I do maintain various websites and have rewritten almost all of them. My most visited site(200k visits per day) is generating "only" 2.5Mil ques per day. And it is has heavy(content), believe me. If OC was loaded equally(200k views) this would mean there were going to be 100-120Mil ques per day.
ALSO the queries are not-so-wise, giving the server hard times with ORDER BY(as i suspected) and SELECT DISTINCT(pain!!!).
ALSO there are numerous options set to each query, no matter if they are set by the user or not(sort, order, etc). This makes ques like 4-5x longer than expected, even if the user don`t want any sorting order(ASC, DESC, etc.)
ALSO there are ques written in such a bad way, that amuses me. How could you pull total numbers for anything by using 5 phps and 3 queries, as long as you can do simple 1 line "SELECT COUNT(*) FROM ..."? OC team seems to not care about execution times and server loads.
I would like to apologize if somebody is offended by what I've written, but in my case I am right: the whole approach is wrong for achieving the goals(fast execution on 37k products/129 cats). OC could be good for someone with 2 categories and 50 products(lol?). Dunno. And i probably won't find out.
As an INFO - caching is not a solution. Server side caching is pretty enough. Anything beyond this means you have serious coding problems. So don't... I will repeat DON'T BUY caching modules. The are hiding the problems, not solving them. If a caching module can hide the problem on 40k products, it won't be able to do it in 140k products. You will need caching module for the caching module, lol.
Now, to the solution. Easy way. We will modify only the main problems. I will not explain the que modifications I made on my version, because they are in many files and are critical if you don't understand what are you doing(you might loose OC options you would like to keep, while I don`t care about options as long as the site is loading for half a minute). So - minor modifications ONLY.
Wil say - explained for version 1.5.5.1 stock, stock theme. Means - no mods. After modification you will loose left side block with categories, but your site will load REALLY FAST(37k products/129 cats -> 0.137 seconds avg in sumulated 5 loads, server distance is ~200mi)
0) BACKUP your site. We will be modifying files. You could make a horrible mess. And cry afterwards.
1) Get /catalog/controller/product/category.php
Find line: 184
Must contain: $product_total = $this->model_catalog_product->getTotalProducts($data);
Replace with: //$product_total = $this->model_catalog_product->getTotalProducts($data);
Description: Commenting out categories count as it takes pretty damd much to count products in 129 categories.(129 queries? WTF?)
2) Get /catalog/controller/product/category.php
Find line: 187
Must contain: 'name' => $result['name'] . ($this->config->get('config_product_count') ? ' (' . $product_total . ')' : ''),
Replace with: 'name' => $result['name'],
Description: Cleaning up - there is no count to be shown in categories, as we don`t count them anymore.
3) Get /catalog/controller/product/category.php
Find line: 388
Must contain: 'common/column_left',
Replace with: // 'common/column_left',
Description: Skippng generation of the left column in the category view.
4) Get /catalog/controller/product/product.php
Find line: 463
Must contain: 'common/column_left',
Replace with: // 'common/column_left',
Description: Skippng generation of the left column in product view.
5) Get /store/catalog/view/theme/default/template/product/product.tpl
Find line: 1
Must contain: <?php echo $header; ?><?php echo $column_left; ?><?php echo $column_right; ?>
Replace with: <?php echo $header; ?><?php echo $column_right; ?>
Description: Removing left column from theme - products view.
6) Get /store/catalog/view/theme/default/template/product/category.tpl
Find line: 1
Must contain: <?php echo $header; ?><?php echo $column_left; ?><?php echo $column_right; ?>
Replace with: <?php echo $header; ?><?php echo $column_right; ?>
Description: Removing left column from theme - catalog view.
DONE. Test your load speed. Should be pretty amazing, if your problem was like mine.
NOTE: Please note, that I am not familiar with nay version of OC and have never used it before. As we have solved part of the problem, it is not fully resolved. This is a temporary fix. Deleting the parts that cause slow load is a solution until you write them again, this time hopefully better. I am willing to rewrite it, if someone wants to outbid my boss. I can take vacation and work for you : ) My payment is currently 4700€ per week. Understanding and rewriting this left column in the right way shouldn`t take more than 1-2 working days.
PP. Will post this on several places, because I don't think OC dev team will like what they've read, no matter I don`t mean to offend them - just to point the critical mistakes they've made(25.31 avg load time for each page in tests - no customer will wait more than 3-4 seconds before going away to another site! Dafuq?). And by not letting me post this info, people don't know how to work their way out of the problem and go buy a "caching module" that is actually crapping files around on the HDD like wild. Waste of money, waste of hard drive resources, waste of electricity... and all this - for creating an illusion everything runs fine, while it doesn't.
In a store I work on which is Version 1.5.6.4_rc (i am pretty sure applies ot your version too) the issue was with the following:
foreach ($categories as $category) in catalog/controller/module/category.php around line 33
foreach ($categories as $category in catalog/controller/common/header.php around line 107
Because of that code we had over 900 db queries many from $this->url->link() along with various others.
We created a vqmod to address this issue by caching this category data so atleast it will only happen when the cache regenerates:
<modification>
<id>Cache category data to speed up page load for store with many categories and sub categories.</id>
<version>1.0.0</version>
<vqmver>2.3.2</vqmver>
<author>Weismann Web</author>
<file name="catalog/controller/module/category.php">
<operation>
<search position="after"><![CDATA[
foreach ($categories as $category) {
]]></search>
<add><![CDATA[
$category_data = $this->cache->get('vqmod_category_data_controller_module_category');
if ($category_data) {
$this->data['categories'] = $category_data;
break;
}
]]></add>
</operation>
<operation>
<search position="before"><![CDATA[
if (file_exists(DIR_TEMPLATE . $this->config->get('config_template') . '/template/module/category.tpl')) {
]]></search>
<add><![CDATA[
if (!$category_data) {
$this->cache->set('vqmod_category_data_controller_module_category', $this->data['categories']);
}
]]></add>
</operation>
</file>
<file name="catalog/controller/common/header.php">
<operation>
<search position="after"><![CDATA[
foreach ($categories as $category) {
]]></search>
<add><![CDATA[
$category_data = $this->cache->get('vqmod_category_data_controller_common_header');
if ($category_data) {
$this->data['categories'] = $category_data;
break;
}
]]></add>
</operation>
<operation>
<search position="before"><![CDATA[
$this->children = array(
]]></search>
<add><![CDATA[
if (!$category_data) {
$this->cache->set('vqmod_category_data_controller_common_header', $this->data['categories']);
}
]]></add>
</operation>
</file>
</modification>
Here is my post about it on the Opencart forums: Issue With Slow Opencart When Having Lots of Categories
Have a look at the .htaccess file, as explained in this tutorial:
`## EXPIRES CACHING ##
<IfModule mod_expires.c>
ExpiresActive On
ExpiresByType image/jpg "access plus 1 week"
ExpiresByType image/jpeg "access plus 1 week"
ExpiresByType image/gif "access plus 1 week"
ExpiresByType image/png "access plus 1 week"
ExpiresByType text/css "access plus 1 week"
ExpiresByType application/pdf "access plus 1 week"
ExpiresByType text/x-javascript "access plus 1 week"
ExpiresByType application/x-shockwave-flash "access plus 1 week"
ExpiresByType image/x-icon "access plus 1 week"
ExpiresDefault "access plus 1 week"
</IfModule>
## EXPIRES CACHING ##`
Take a look in catalog/controller/module/categories.php. By default, the category module displays the product count next to each item in the menu. This results in a fair bit of query overhead for a very small ux gain (In my opinion).
The line
$product_total = $this->model_catalog_product->getTotalProducts($data);
appears twice, if you comment it out (and where $product_total is used below it) you should see pretty significant gains.
Maybe you need Full Page Cache aka FPC is made to speed up your shop by creating simple file to load instead of the all opencart framework. This extension will cache all your catalog pages but will keep the user specific information dynamic.
Link : Full Page Cache
I've found another solution to the problem. Basically, MySQL is choking when it tries to filter query results from both the Product_category and Product_Tag tables simultaneously. I wrote a vqMod for OC 1.5.2.1 that replaces the getTotalProducts() function and constructs individual SQL queries to be executed against the 2 tables. It then uses a SQL UNION to tie the results of both queries together. This solution improves performance and allows you to keep using product counters on your site.
This took my page load time from 45 to 50 seconds down to less than 1 second!!!
I just posted the vqMod file here: http://www.opencart.com/index.php?route=extension/extension/info&token=7bc7d0149c7101c3d336b2e0b29e3f03&extension_id=13155
Let me know if you have any questions and I'll help where I can.
Here's what fixed it for me. As previous posters have mentioned a major cause of the slowdown is in :
catalog/model/catalog/product.php -> public function getTotalProducts($data = array()) {
...
Put this at the start of the function:
public function getTotalProducts($data = array()) {
if (isset($_SESSION['totalproducts'.$_SERVER['QUERY_STRING']])){
return $_SESSION['totalproducts'.$_SERVER['QUERY_STRING']];
}
Put this at the end
$_SESSION['totalproducts'.$_SERVER['QUERY_STRING']] = $query->row['total'];
return $query->row['total'];
}
By doing this you don't have to comment anything out, but the total products would be wrong if it got updated whilst a session was active.
Hope that helps.
You should create 02 indexes for keyword and query columns in OC_URL_ALIAS table. You will be impressed with speed.
I used PageSpeed Insight of Google to measure speed of my website.
At the first time, the result is "In our test, your server responded in 12.2 seconds". After creating two indexes, the response time is 7.8 seconds :)
I have a scenario where I need to generate 4 digit confirmation codes for individual orders. I don't want to just do random codes due to the off chance that two exact codes would be generated near the same time. Is there a way to use the id of each order and generate a 4 digit code from that? I know I am going to eventually have repetitive codes with this but it will be ok because they will not be generated around the same time.
Do you really need to base the code on the ID? Four digits only gives you ten thousand possible values so you could generate them all with a script and toss them in a database table. Then just pull a random one out of the database when you need it and put it back in when you're done with it.
Your code table would look like this:
code: The code
uuid: A UUID, a NULL value here indicates that this code is free.
Then, to grab a code, first generate a UUID, uuid, and do this:
update code_table
set uuid = ?
where code = (
select code
from code_table
where uuid is null
order by random()
limit 1
)
-- Depending on how your database handles transactions
-- you might want to add "and uuid is null" to the outer
-- WHERE clause and loop until it works
(where ? would be your uuid) to reserve the code in a safe manner and then this:
select code
from code_table
where uuid = ?
(where ? is again your uuid) to pull the code out of the database.
Later on, someone will use the code for something and then you just:
update code_table
set uuid = null
where code = ?
(where code is the code) to release the code back into the pool.
You only have ten thousand possible codes, that's pretty small for a database even if you are using order by random().
A nice advantage of this approach is that you can easily see how many codes are free; this lets you automatically check the code pool every day/week/month/... and complain if the number of free codes fall below, say, 20% of the entire code space.
You have to track the in-use codes anyway if you want to avoid duplicates so why not manage it all in one place?
If your order id has more than 4 digits, it is theoreticly impossible without checking the generated value in a array of already generated values, you can do something like this:
require 'mutex'
$confirmation_code_mutex = Mutex.new
$confirmation_codes_in_use = []
def generate_confirmation_code
$confirmation_code_mutex.synchronize do
nil while $confirmation_codes_in_use.include?(code = rand(8999) + 1000)
$confirmation_codes_in_use << code
return code
end
end
Remember to clean up $confirmation_codes_in_use after using the code.