Recently I encounter an issue while profiling my JS code and found out a very small function in the code which took 3-10ms. After investigation, I found out the function Element.innerHTML is the one who took almost all the CPU time when called 1-2 times inside this function.
For example: element1.innerHTML = "<span>text</span>".
I am searching for an alternative to reduce CPU time.
If you can ignore using Element.innerHTML please do. it is super expensive. Instead, use document.createElement and Element.appendChild instead it.
I converted this code element1.innerHTML = "<span>text</span>" to this code:
var element2 = doc.createElement('span');
element2.innerText = "text";
element1.appendChild(element2);
When I did this change in both places I dropped the CPU from each innerHTML usage from 1.5ms to 0.01ms, and even 0ms in most cases.
Then I have a need to add NO_BREAK sign (" ") in different locations that existed in the old code but all the suggestions I got were to use innerHTML for this sign, which will add the CPU time I want to prevent.
The solution to this is using:
element.appendChild(document.createTextNode('\u00A0'))
which will do the same effect without innerHTML. Even using Element.insertAdjacentHTML is parsing the HTML and will take more CPU time, so I tried to ignore the use of it as well.
I hope it helps.
Related
I couldn't work out if Stack Overflow is an appropriate site to ask this and if not then what the appropriate site would be! There are so many stack exchange sites now and I didn't want to go through 200 sites :S
When I try and test whether my functions run within X seconds using behave (ie gherkin feature files and behave test steps), the code takes longer to run with behave testing than it would on its own. Especially at the beginning of the test but also in other parts.
Has anybody tested time constraints with behave before and know a workaround to adjust for the extra time that behave adds?
Is this even possible?
EDIT: To show how I'm timing the tests:
#when("the Python script provides images to the model")
def step_impl(context):
context.x_second_requirement = 5
# TODO: Investigate why this takes so long, when I'm not using behave I can use a 0.8 second timing constraint
context.start_time = time.time()
context.car_brain.tick()
context.end_time = time.time()
#then("the model must not take more than X seconds to produce output")
def step_impl(context):
assert context.end_time - context.start_time < context.x_second_requirement
Cheers,
Milan
I'm working on a script to set to uppercase a selected text in a Google document. The script works ok, but it is extremely slow in a large document. So I stripped down almost everything to perform some speed test and I realised that this simple line:
var document = DocumentApp.getActiveDocument();
already takes 9-10 seconds to perform for a 150 pages document.
Is there a way to speed up the process, i.e. precaching "document" so that when I invoke my script I can use my getCursor() or getSelection() in a reasonable time?
You can't precache this.
The services provided for caching are:
CacheService : only store String values
ScriptDB : can't store Document on this.
So I'm afraid that you will need to cope with it.
I had an implementation of a min heap in ruby that I wanted to test against more professional code but I cannot get Kanwei's MinHeap to work properly.
This:
mh = Containers::MinHeap.new # Min_Binary_Heap.new
for i in 0..99999
mh.push(rand(9572943))
end
t = Time.now
for i in 0..99999
mh.pop
end
t = Time.now - t
print "#{t}s"
The version I have performs the same popping operations on 100,000 values in ~2.2s, which I thought was extremely slow, but this won't even finish running. Is that expected or am I doing something wrong?
I don't think you are doing something wrong.
Looking at the source (https://github.com/kanwei/algorithms/blob/master/lib/containers/heap.rb), put a puts statement for when you finish setting up the heap. It looks like a very memory intensive operation to put the elements in (potentially resorting each time), so it might help you working through it.
I'm also not sure about him creating a node class for each actual node. Since they won't get cleaned up, there's going to be around 100,000 objects in memory by the time you are done.
Not sure how much help that is, maybe see how the source differs from your attempt?
The main problem I'm having is pulling data from tables, but any other general tips would be welcome too. The tables I'm dealing with have roughly 25 columns and varying numbers of rows (anywhere from 5-50).
Currently I am grabbing the table and converting it to an array:
require "watir-webdriver"
b = Watir::Browser.new :chrome
b.goto "http://someurl"
# The following operation takes way too long
table = b.table(:index, 1).to_a
# The rest is fast enough
table.each do |row|
# Code for pulling data from about 15 of the columns goes here
# ...
end
b.close
The operation table = b.table(:index, 5).to_a takes over a minute when the table has 20 rows. It seems like it should be very fast to put the cells of a 20 X 25 table into an array. I need to do this for over 80 tables, so it ends up taking 1-2 hours to run. Why is it taking so long and how can I improve the speed?
I have tried iterating over the table rows without first converting to an array as well, but there was no improvement in performance:
b.table(:index, 1).rows.each do |row|
# ...
Same results using Windows 7 and Ubuntu. I've also tried Firefox instead of Chrome without a noticeable difference.
A quick workaround would be to use Nokogiri if you're just reading data from a big page:
require 'nokogiri'
doc = Nokogiri::HTML.parse(b.table(:index, 1).html))
I'd love to see more detail though. If you can provide a code + HTML example that demonstrates the issue, please file it in the issue tracker.
The #1 thing you can do to improve the performance of a script that uses watir is to reduce the number of remote calls into the browser. Each time you locate or operate on a DOM element, that's a call into the browser and can take 5ms or more.
In your case, you can reduce the number of remote calls by doing the work on the browser side via execute_script() and checking the result on the ruby side.
When attempting to improve the speed of your code it's vital to have some means of testing execution times (e.g. ruby benchmark). You might also like to look at ruby-prof to get a detailled breakdown of the time spent in each method.
I would start by trying to establish if it's not the to_a method rather than the table that's causing the delays on that line of code. Watir's internals (or nokogiri as per jarib's answer) may be quicker.
There are two large text files (Millions of lines) that my program uses. These files are parsed and loaded into hashes so that the data can be accessed quickly. The problem I face is that, currently, the parsing and loading is the slowest part of the program. Below is the code where this is done.
database = extractDatabase(#type).chomp("fasta") + "yml"
revDatabase = extractDatabase(#type + "-r").chomp("fasta.reverse") + "yml"
#proteins = Hash.new
#decoyProteins = Hash.new
File.open(database, "r").each_line do |line|
parts = line.split(": ")
#proteins[parts[0]] = parts[1]
end
File.open(revDatabase, "r").each_line do |line|
parts = line.split(": ")
#decoyProteins[parts[0]] = parts[1]
end
And the files look like the example below. It started off as a YAML file, but the format was modified to increase parsing speed.
MTMDK: P31946 Q14624 Q14624-2 B5BU24 B7ZKJ8 B7Z545 Q4VY19 B2RMS9 B7Z544 Q4VY20
MTMDKSELVQK: P31946 B5BU24 Q4VY19 Q4VY20
....
I've messed around with different ways of setting up the file and parsing them, and so far this is the fastest way, but it's still awfully slow.
Is there a way to improve the speed of this, or is there a whole other approach I can take?
List of things that don't work:
YAML.
Standard Ruby threads.
Forking off processes and then retrieving the hash through a pipe.
In my usage, reading all or part the file into memory before parsing usually goes faster. If the database sizes are small enough this could be as simple as
buffer = File.readlines(database)
buffer.each do |line|
...
end
If they're too big to fit into memory, it gets more complicated, you have to setup block reads of data followed by parse, or threaded with separate read and parse threads.
Why not use the solution devised through decades of experience: a database, say SQLlite3?
(To be different, although I'd first recommend looking at (Ruby) BDB and other "NoSQL" backend-engines, if they fit your need.)
If fixed-sized records with a deterministic index are used then you can perform a lazy-load of each item through a proxy object. This would be a suitable candidate for a mmap. However, this will not speed up the total access time, but will merely amortize the loading throughout the life-cycle of the program (at least until first use and if some data is never used then you get the benefit of never loading it). Without fixed-sized records or deterministic index values this problem is more complex and starts to look more like a traditional "index" store (eg. a B-tree in an SQL back-end or whatever BDB uses :-).
The general problems with threading here are:
The IO will likely be your bottleneck around Ruby "green" threads
You still need all the data before use
You may be interested in the Widefinder Project, just in general "trying to get faster IO processing".
I don't know too much about Ruby but I have had to deal with the problem before. I found the best way was to split the file up into chunks or separate files then spawn threads to read each chunk in at a single time. Once the partitioned files are in memory combining the results should be fast. Here is some information on Threads in Ruby:
http://rubylearning.com/satishtalim/ruby_threads.html
Hope that helps.