I've have been wondering about the time complexity of the programs that I run and I would like to know how I can compare their speeds. I usually like do questions on platforms like CodeAbbey and when I'm done I get the chance to see other peoples attempts of the problem as well. With the diversity, people have many different methods and my hope is to pick up hacks from what they have done to improve my skills. So is there some platform/website/app/program etc that I can use to to check which gets done first by running them simultaneously?
One other curious question iv been wondering about is:
does each program take exactly the same time to run everytime? like if I code something up and run it once, if I run it a second time, will it execute in the same time as before or will it be slightly different and the best I can hope for is an average?
I have tried to Google it but i don't really understand the results or how to use the sites. I just want to be able to see how much time it take to run.
Related
Please forgive me if the terms are inaccurate or this is the wrong forum.
Essentially I write sketches in Processing and I am struggling to find out why my code runs slow.
Sometimes a sketch runs fast and I have no idea why other than there are less lines of code. Sometimes a different sketch runs slow and I have no idea why.
I am curious if there is a way within the Processing IDE, or maybe a general tool, to determine or analyze which lines of code are causing the sketch to run slow?
Such as a way to isolate "Oh, these lines are causing it to run the slowest. Looks like it's a section of this loop. Maybe I should concentrate on improving this function rather than searching for a needle in a haystack."
Similar to how when my computer is running slow I can use task manager to take a look at which programs are running slow and then adjust. I don't just guess. Or develop an unfounded penchant for quitting one program over another.
I of course could upload my sketch but this is a example independent problem I am trying to get better at solving. I would like to find a way to analyze all sketches. Or even code in different languages, etc.
If there is no general tool for analyzing a Processing sketch how do people go about this? Surely there must be a better method than trial and error, brute force, intuition, or otherwise. Of course those methods could yield better running code but there must be some formal analysis.
Even if you didn't have anything specific to share for Processing any suggestions on search terms, subjects, or topics would be appreciated as I have no idea how to proceed other than the brute force/trial and error method.
Thank you,
Without an example code snippet I can only provide a couple of general approaches.
The simplest thing you could do is use time sections of code and add print statements (poor man's profiler). The idea is to take a time snapshot before running a function/block of code you suspect is slow, take another one after then print the difference. The chunks of code with the largest time difference is what you want to focus on. Here's a minimal example, assuming doSomethingInteresting() is a function that you suspect is slow
// get the current millis since the sketch started
int before = millis();
// call the function here
doSomethingInteresting();
// get the millis again, after the function and calculate the time difference
int diff = millis() - before;
println("executed in ~" + diff + "ms");
If you use Processing's default Java mode you can use VisualVM to sample your PApplet. In your case you will want to sample CPU usage and the nice thing about VisualVM is that you will the results sorted by the slowest function calls (the first you should improve) and you can drill down to see what is your code versus what is part of the runtime or other native parts of code.
I have been doing sport programming for a while and still improving day by day. But one thing I have always wondered is that it would be really nice if I could automate the test-case generation process and cross-validation of my program. Definitely it would be a brute force approach as some test cases would be algorithm specific.
Doing a google search gives me a nice link on Quora : How do programming contest problem setters make test cases ? and the popular testlib used by problem setters.
But isn't this a chicken-egg problem?
Assume I generated 1 million input test cases, but what would I check them against? How will I generate the outputs? Because I am still in the process of validating the program... If my script generates the correct outputs as well, then whats the point of writing the program in the first place. I can submit the script itself. Also, its not possible to write 1 million outputs for generated test cases manually. Can anyone please clarify this confusion.
I hope i have clarified the problem correctly.
It's common to generate the answer by a slow but obviously correct solution (like an exhaustive search). It can't be used as a main solution as it's too slow for large test cases, but you can check the output of your fast (but possibly incorrect) program using it.
Well the thing is it is not as broad as you think it to be. Test generation in competitive programming is guided by the algorithm of the problem and it's correctness proof.
So when you are thinking that there are million of test cases if you analyze the different situations the program can be then you will likely to get all the test cases. Maybe in certain algorithm you are some times processing the even index elements or the odd index elements of an array. Now what you will do? Divide it in 2 cases even or odd. Consider the smallest case for even ones . Same for odd ones. This way you are basically visiting all the control flow path of the program.
In competitive programming as we first determine the algorithm then we decide on a proper input sizes and then all this test cases and validations, it is often easy to think the corner points. Test case for 1000000 elements or when input is 0 or 1 ...test cases like this.
Another is most of the time we write a brute force solution much more slower than the original one. Now what we do? we just generate random medium size test cases and then run it again the slow program and we can check with our checker solution etc.
correctness is guided by some mathematical proof also.(Heuristics, Induction, Box principle, Number Theory etc ) That way we are sure about the correctness of the solution.
I faced the same issue earlier this year and saw some of my colleagues also figuring out a way to deal with this. Because there are sometimes when I just couldn’t think any more test cases and that’s when I decided to make a test case generator tool of my own, It’s free and open-source so anyone can use it.
You can easily generate a lot of test cases using this tool and validate the result using the output given from the correct but slower approach(in terms of time complexity and space complexity). You can either run them parallelly and check for outputs or write a simple script to compare the output of both programs (the slower but correct one and the better but unsure one) to validate.
I believe good coders won’t be needing it anyway, but for the middle level (div2, div3) coders and newbies, it can prove to be a lot helpful.
You can access it from GitHub : Test Case generator.
Both python source code and .exe files are present there with instructions,
If you want to make some changes of your own you can work with the python file.
If you just directly want it to generate some test cases you should prefer the .exe file(inside the zip).
It’ll help a lot if you’re beginning your Competitive programming journey.
Also, any suggestions or improvements are always welcome. Additionally you can also contribute to this project by adding some feature that you think the project is lacking or by adding some new test case formats by yourselves or making by request for the same.
When I'm writing tests using lettuce, I want to create a huge scenario that contains an user making every possible action on website. But testing tools are making me aim separating them. What is the benefit of it?
You put "BDD" into the title of this question, and tag it with both "BDD" and "TDD" tags. So you're interested in Behavior Driven Development and Test Driven Development.
Why should you drive development a little bit at a time instead of driving the entire application all at once? That's what your question amounts to in the context of BDD and TDD.
You are going to write one method, one additional bit of functionality, next. Of course that bit will contribute to the overall behavior, and it's good to have an understanding of the overall behavior you're trying to develop, but you need focus. You need to know when that next bit is working and complete, so you can move on to the next bit. A full-blown test of the entire app will fail at the beginning; it will fail after your first bit of functionality is implemented; it will fail when you are half-done, and it will fail when you are 99% done. Unfortunately, it will probably also fail when you are 100% done - and now you'll have to find where you went wrong, and fix that bit (or those bits).
But if you write a test of just that new bit of functionality, it will fail right now, and it will pass in five minutes or ten minutes or twenty minutes or maybe an hour. Then you'll know it's time for the next bit. And it will keep passing as you write more tests and implement more functionality. When you're 99% done, you'll have 99% of your ultimate tests passing - and 100%, minus one, of your current tests. You can see your real progress and know that what you have written up to date is really working.
That's why you should write small tests, one at a time, and make them pass one at a time.
Three important things come to my mind:
readability: when a scenario fails, it is easier to understand from a first glance at the scenario's name what went wrong and significantly easier to fix when it's small and focused
maintainability: it is easier to modify/update small scenario
independence: large scenarios tend to make steps dependent on each other. This way, the further an action in a scenario is, the more it is depending on previous actions in more complicated ways that are hard to comprehend. This directly influences two previous reasons.
I have a feature with one scenario outline that takes too long to run because of the number of examples I use. I would like to take advantage of the multiple threads available and run these outlines in parallel. One way is to split up the feature/scenario outline into multiple features. This works but then it leaves me with multiple copies of the same scenario with only a difference in examples. My question is if there is a way to split this scenario outline to run in parallel without having to create multiple features?
Jay, I doubt there is a way to do exactly what are asking (splitting a single scenario outline into separate parallel runs). I would say if your primary concern is 'speed' regarding this test then it would be worth having duplicate features and separating the examples out so that the coverage would complete faster. I generally like to keep my scenarios below a minute unless I am doing something that is 'expansive' like the one you are describing sounds, but for those tests I run overnight and don't expect quick results.
If the feature as you say 'takes too long to run" BUT you want it to be quicker then i would suggest to simply cut into 'quicker' bite size tests that you can then run in parallel.
Even if there was a way to do what you asked, I believe you would be better off splitting up the test anyways.
I'm working on some pet projects and generally I sit around my personal computer about 22:30 or 23:00 to code. But since I try to sleep about 24:00 I don't start coding and ending up reading articles, playing some games etc.
I don't feel like I can write decent code in an hour, because the project is quite big and I don't want to randomly or carelessly hack it. Even though I use TDD, most of the time stuff I'm doing is not straight forward which requires lots of testing before getting it right.
What's your approach to these kind of issues? Do you just code later when you got enough time or do you have a different approach which allows you to code just for 30 minutes and continue later?
I generally don't write lots of code until i have the time to do it. The reason is that for me to get effective takes focus and that takes a bit of time to be correctly focused. That said those 30min slots are great for
Writing more tests: nothing like trying to get to 100% code coverage, and it's not a big waste since you are investing
Research: I spend lots of time reading blogs, looking for frameworks I can use or tools. Spending 30min finding a framework that does 80% of a feature you need is much better than spending hours trying to code it. The other factor to this is that if you implement the framework and you find it is a bad fit you are better educated in the needs which means your development will be smoother.
Well my first thought was "use unit testing", but then I read you are already using this. But I still think it's the solution to your program.
Try to make your tests as small as possible and use the "1 assert per unit test" rule to create small atomic tests. You should be able to fix several of these small tests in a 30-minute session.
Here are some things you could try:
Don't sit near the computer. Instead, take a large piece of paper and go somewhere quiet. Think about what you want to accomplish. Write down interface ideas, detailed implementation. Make a list of questions you need to solve before you can go on.
Take off a week and code away. The ratio of getting-into-flow over flow-time is just too bad for 30 minutes.
Keep a log about what you do instead of coding. Observe your emotional state.
Go to bed early and try to have your pet coding session very early in the morning.
A small tip (that I use at work too) is to stop coding in the middle of something, with an obvious big red compile error waiting.
The next time you start working, the error will actually help you to remember what on earth you were doing.
While you are working on the small problem, the big picture clears up and then you can continue designing.
Developing with such short times is difficult, but you can still get something of that time. Unit testing is one. Writing down the interface of a class is another. While coding the real stuff will take much more time, these tasks are essentially a no-brainer, and they are just an exercise in typing.
So, my suggestion is: focus on small tasks that do not require thinking and concentration, and can be completed in the timespan you have.
Never worked for me, pet projects are usually too interesting and I end up working to late hours of the night or through a weekend.
I would suggest reconsidering your priorities - if all the time you have available is one hour late at night, maybe it's better to spend it on games, articles etc. Or just hanging out. When you have a bit more time, say a lazy Sunday, spend all the time at once and actually get the sense of accomplishing something that pet projects are supposed to give.
Here's what I do with my personal projects outside of work:
1) I try to give myself a good map of my project by planning it out on paper. I diagram all of my objects, data structures, and/or SQL tables and determine what basic functions and interactions between those components are necessary. I may write some actual code during this phase if the solution is obvious, but typically not.
2) Once the big picture is in place, I prioritize the most basic and critical elements. I also try to figure out which parts will be easier to write than others.
3) After priorities have been set I start working on the easiest and most critical parts first and work gradually towards the more complex and less important components. Breaking each task down into smaller parts tends to help. For example, I may design a database table first and the next day create a data interface class that controls interactions with that table.
4) Unit testing really helps me feel a sense of accomplishment, even if 30 minutes of effort only results in a few quality lines code.
5) Keep a change log, even if it isn't very detailed. I've found my change logs to be invaluable if I work on a big project in many short spurts over an extended period of time.
Those steps right there help me the most. In the end I am able to identify small chuncks of the project that can typically be completed in about 30-60 minutes. Of course, as a project develops, I usually have to re-evaluate something and go back to the beginning for a while when I discover I left something out of the planning phases. Sometimes I need to go a bit farther and give myself a time line with some deadlines and make sure I celebrate milestones with a personal reward. If you have a tendency of staying up till the wee hours of the morning, something I struggle with, I also suggest giving yourself a coding curfew as well. I also try to make sure my "coding computer" doesn't have many distractions on it, such as games.
There's always one more bug. If not, there's one more neat feature you can add, and THAT will add more bugs. Which is one reason why I think that the use of the phrase "All you have to do..." by anyone in (or using) IT should be a hanging offense.
I can cut down on how long my coding sessions last by doing trivial things, thinking things out while away from the keyboard (the shower is best - or while in bed at 4 a.m.) and by using lightweight environments such as script languages, but "quick" coding sessions are something I long ago gave up hope on.
Just shifting mental gears into the coding mode takes time, picking up the threads of where I was before takes time, discovering that my "quick and easy solution" was neither of the above takes time. Fixing my "quick and easy solution takes time, debugging - more time, and so forth.