I'm writing a Spring application which scrapes 50-100 links at the beginning and does some operations. So every time I modify and restart the application it scrapes all these links again.
What can I do to avoid it? May be store the scraped links in a H2 database, backup it and restore it when I start the application? Or are there any other possibilities?
Related
I want to Load Dropdown data from Database at once and set inside java object and tie to my view (JSP page ) and available all the time for that particular controller or functionality using spring mvc AND jsp pages.
I dont want to load on application start up as ours is big one and and each functionality is independent.
It takes a lot of time to start the application if i load on application start up
Is there a way to it using spring mvc pattern and using JSP
Could someone please let me know how to do it
As you have not mentioned how frequently you are doing the database operation or how frequently you are fetching the data. Considering the average user.
Approach: Create your own local cache/ program cache implementation.
Instead of loading all the data from the database during startup, load only master data which will be common for all. If master data is also high then you can perform the lazy loading approach.
Load the data of a specific feature when it is requested for the first time. Keep the data in the local cache.
Whenever someone is making the changes then add the data in the cache and save the same to the database. so you will always have latest data in the cache.
Advantage:
Very useful for common or static master data
-If you need good business logic for some common data. This way only once you are processing the data and keeping cache.
-Fetching the data is very fast as it doesn't involve database request except for the first time
Disadvantage:
If you have a very high number of users and a very high update operation then the updating cache will delay the update process as you need to update it sequentially.
I suggest you can use a combination of approaches to improve the code quality and processing.
This sounds like a typical cache functionality.
Spring supports caching out of the box by #EnableCaching on application level and #Cacheable(“cachename”) on the repository method retrieving your dropdown data. In your simple use case you not even need an additional framework as there is a CacheManager based on ConcurrentHashMap which simply caches for ever.
With caching in place your controller can simply fetch the dropdown data from the repository. Caching will ensure only the first call will really fetch from database and keeps the result in memory for all upcoming calls.
If you ever need more sophisticated caching you only have to exchange the cache manager and configure the cache for your needs.
I´m developing a Jokenpo Game using React with Spring Rest, but I can´t have a database to store all the information needed(create and delete moves, create and delete players).
I don´t know the best practice of development, or if there is some design pattern on how to store that kind of information. I know there is the folder src/main/resources where maybe I can store a text file there and thought about on the startup of the api it loads that file with the begin of the game, maybe, and after changing it during the game.
Trying to be more clear: I just would like to know the simplest way of storing information without being a database inside of a Spring Rest application. I really appreciate any helps. Thanks.
Take a look at SQLite. It's a very light database library that you can include as a dependency of your Spring application, It doesn't require a separate database server to run, and the entire database is stored in a single file, that you can choose where to store in the connection string.
It offers the flexibility of a standard database, so you can use Spring Data / JPA to access the data. It has some limitations compared with robust databases like MySQL, specially related with concurrent writes that you should investigate and be aware of. Usually it works very well for small applications or embedded applications.
My app makes requests to
http://www.brewerydb.com/api/breweries/?apikey=<key>&<parameters>
but I'm only given 100 requests per hour. I considered recording request instances in the app's database but during development I am often regenerating the database using code the Entity Framework 4.1 so that doesn't seem like it would work. Any ideas?
Some kind of database would be best.
If your regenerating the DB as you say, then store it in a different DB, or if you simply want to store basic info (date/time of hit), then nothing wrong with using XML or even a CSV file on your web server.
I am developing a web service that will be invoked (using JSON) from client side each time the selection of a drop down changes.
The goal is to register each "intermediate" change (on client side) using the "OnSelectedIndexChanged" event and before submitting the form to the Server.
Each new selected value will be written to a shared txt file calling a relative web method via Ajax/JSON.
Would it be better to write these changes to a txt file (having to implement a lock/unlock policy to assure exclusive access) or rather define a DB table and save the changes there?
Everyday the web app will have around 10 to 20 active users that might potentially changes the DropDownLists and usually the right value will be selected at first, hence generally no more than one "intermediate" entry would be registered.
Thanks.
Don't use the filesystem. It's slow. Use mongodb via a node.js webserver.
http://howtonode.org/express-mongodb
Good Luck!
This sounds exactly like what you would want to use a database for, since ACID is already implemented there.
If you want a real headache (and a programming challenge!) trying to debug overlapping writes, resource starvation and deadlocks, by all means, go with a shared text file!
I have a big rich-internet-application file (qooxdoo,js,html). The users use their browser to point to the web server and run it. The problem is that it takes a long time for the users to load the application every time they visit the site.
Is there a way to somehow "bundle" and save the application locally and have the user refer to it locally? So, the url would be like [c:/]/home/myfiles/application/index.html instead of http://site/path-to-app?
I was thinking something like java's jar files to bundle the application and make it runnable locally in browsers, yet the application reaches the external website to get data.
Any ideas?!
Thanks in advance.
The browser should cache all the files so the second load of the app should be quite fast. If thats not the case, maybe you are not using the qooxdoo build version of your application or you disabled the optimizations of the build process.
But there are two ways to get a desktop like application:
You can offer the files you upload to the server as zip and let the user unzip it. If you don't need a web server to run the files, that should work.
If you want to build a real desktop application, you should have a look at titanium [1] which can bring a webapp to the desktop.
[1] http://www.appcelerator.com/products/titanium-desktop/
Running the qooxdoo application from the file system, like Martin sad, should not be a problem. But you have to ensure that "crossDomain" property for example "qx.io.remote.Request" [1] is set to "true", otherwise the same origin policy (SOP) from the Browser blocks the requests to the server.
[1] http://demo.qooxdoo.org/current/apiviewer/#qx.io.remote.Request~crossDomain