I'm trying to have a caching system for my project, the idea is to save the last information (as JSON data) for every user in a text file which exists in a special folder for this user, and instead of hitting the database to fetch the whole required info every time this user logged in or updated the page I call a small field in Users table which called Uptodate to decide whether should I update the text file or simply calling the existed one .
I'm tring to avoid using the memory for chaching data, is this is a good approach to do the job ? should I save data as a text file ? how can I add data to the top of existed file ?
thanx
HTML5 localStorage is perfect for what you're trying to do.
localStorage.setItem("json_data", JSONdata);
alert(localStorage.getItem("json_data"));
Related
I have a basic spring batch app which is trying to load the data from a csv file to mysql. the program does load the file into db during the first run. However when I accidently re-run the job/app again, it had thrown the primary key violation (for the right reasons).
What is the best way to avoid reloading the data that is present on the target system? when the batch job is scheduled, if for any good reason, the source file has not changed since the previous run, I want to see 0 record processed message rather than a primary key violation error. hope it makes sense.
more information:
Thanks. I have probably not understood the answer. Let me explain my requirement in a better way. I have a file contains the data from external data source (say new hire data) with a fixed name of hire.csv. The file should be updated with the delta changes for every run. As there is a possibility of manual error of not removing all loaded rows, some new hires from previous run would also be present on current run. Is there a mechanism available within itemreader or itemprocessor to skip those records that are already present on the target db? I can do "insert into tb where not in (select from tb)" but this run for every row which I dont want to use. Hope it is clear now. thanks again.
However when I accidently re-run the job/app again, it had thrown the primary key violation (for the right reasons). What is the best way to avoid reloading the data that is present on the target system?
The file you are ingesting should be a (identifying) job parameter. This way, when the first run succeeds, the job instance is complete and cannot be run again. This is by design in Spring Batch for this very use case: preventing accidental job execution twice by error.
Edit: Add further options based on comments
If deleting the file is an option, then you can use a job listener or a final step to delete the file after ingesting it. With this option, you need to add a second identifying paramter (since the file name is always hire.csv) to make sure you have a different job instance for each run. This option does not require having a different file name for each run.
If the file can be renamed to hire-${timestamp}.csv and will be unique, then deleting the file after ingesting it and using a single job parameter with the filename is enough
Side note: I have seen people using a business key to identify records in the input file and using an item processor to query the database and filter items that have been already ingested. This works for small datasets but performs poorly with large datasets due to the additional query for each item.
I have a logic app which polls for files does some things with them, succeeds or fails then ends. It will run every 5 minutes and poll for a file.
If it finds a file I can create a blob storage with a date time suffix eg LogutcNow('s').txt
I want to append to this file various messages generated from the logic app eg if steps succeed or fail.
Is Blob storage the best way to put a file in my Azure storage account?
Since the name of the blob depends on the date time, how do I append to it?
It may be that the logic app does not write anything to the log file. In that case I want to delete it.
I want to create the blob at the beginning of my logic app then update it. If there are no updates then I want to delete it. In the update action it seems to require me to specify the name of the blob. Since I haven't create the blob yet this is impossible. One thing I also tried was initialising a string variable to the current date and time and putting that variable into the filename.
Suppose your main problem is after you create a blob with dynamic name and could not get the blob name to do other actions. If that you could just set the blob name with dynamic content Path, if it doesn't shown the dynamic content just set the expression body('Create_blob')?['Path'].
I have a financial system with all its business logic located in the database and i have to code an automated workflow for transactions batch processing, which consists of steps listed below:
A user or an external system inserts some data in a table
Before further processing a snapshot of this data in the form of CSV file with a digital signature has to be made. The CSV snapshot itself and its signature have to be saved in the same input table. Program updates successfully signed rows to make them available for further steps of code
...further steps of code
Obvious trouble is step#2: I don't know, how to assign results of a query as a BLOB, that represents a CSV file, to a variable. It seems like some basic stuff, but I couldn't find it. The CSV format was chosen by users, because it is human-readable. Signing itself can be made with a request to external system, so it's not an issue.
Restrictions:
there is no application server, which could process the data, so i have to do it with plsql
there is no way to save a local file, everything must be done on the fly
I know that normally one would do all the work on the application layer or with some local files, but unfortunately this is not the case.
Any help would be highly appreciated, thanks in advance
I agree with #william-robertson. you just need to create a comma delimited values string (assuming header and data row) and write that to a CLOB. I recommend an "insert" trigger. There are lots of SQL tricks you can do to make that easier). On usage of that CSV string will need to be owned by the part of the application that reads it in and needs to do something with it.
I understand yo stated you need to create a CVS, but see if you could do XML instead. Then you could use DBMS_XMLGEN to generate the necessary snapshot into a database column directly from the query for it.
I do not accept the concept that a CVS is human-readable (actually try it sometime as straight text). What is valid is that Excel displays it in human-readable form. But is should also be able to display the XML as human-readable. Further, if needed the data in it can be directly back-ported into the original columns.
Just a alternate idea.
I'm creating an application that will take a number of user inputs, store the data for a while, and eventually (at the end of the day) export it to an excel file.
An example might be that a user would input what they did throughout the day. Breakfast/At Home/for 10 minutes. Then later on they would input Coding/At Work/8 hours. Then later on Commuting/Subway/15 minutes. Etc.
I can handle the user interface, and the exporting to excel.
I'm just wondering what might be the best way to store that data and display it back to the user while the program is running. I'm used to working with macros in Excel itself, where I could simply store each row of data in another row on the excel spreadsheet itself.
I would still like a spreadsheet-like display, so that the user can go in to each data point and correct any mistakes. But I am making this as a standalone application using visual basic. Fortunately, I think the ListView or DataGridView tools will let me do this.
At the moment the method I'm thinking of using is simply to store all the user inputs in an array. But I would have to ReDim the array and increase its size each time the user created a new entry.
I can already see a problem with this, however, and that is that an array would have to be constantly stored in active memory. If the user's computer were to crash then all the data would be lost for good.
I'm really a rookie here, so I could use some guidance on how to store a bunch of user inputs like this.
You can use a database file. A local Sql Server Compact Editon database (a single file) that will store your data. You can use Entity Framework to interact with this database.
If you want to use Code First (generate your database from your code) use this:
https://www.codeproject.com/Articles/680116/Code-First-with-SQL-CE
If you want to use Database First (generate your entities from your database) use this:
http://erikej.blogspot.com/2013/11/entity-framework-6-sql-server-compact-4_25.html
You can also use SQLite or other kind database file, but i like SQL Server CE
I am working on a WP7 app that contains
CategoryGroups
Categories
Products
The rows for each of these entities are populated on first run of the application.
The issues is that when the app gets published, the rows in each of the entities will change (added, deleted, modified). I would like some suggestions on how I should handle this? Any pointers to existing code samples will be great?
I am using an object oriented database to store my entities. The app also allows the user to add their own entities (which get added to the database as personalized (flagged) entities). One solution I was thinking was to read an xml file from the server and then loop through the database entries and make the necessary modifications in the database. So, on the first run, all the entities will just get inserted. On subsequent runs, if the version number attribute in xml is different, then the system populated data is reloaded from xml but the user data is preserved.
Also, maybe only check for the new xml file on the server when internet connection is available and only periodically (like every 2 weeks).
Any other suggestions are welcome. If there is a simpler, cleaner way - please share.
Pratik
I think it's fair to say that this question has nothing to do with WP7 and everything to do with finding an efficient way to to compute and deliver update deltas.
Timestamp your items. When requesting an update, specify the time of last update. You server can trivially query for items newer than this and return a delta. At the client (ie in the phone) it is not necessary to store a last update time because you can simply add one second to the most recent timestamp in the items present on the phone.