Blob into Oracle: about 15% are filled with \00 - oracle

Under Weblogic 10, I am using Hibernate to store data into several tables with BLOBs. It's always worked fine but the customer found specific circumstances where 15% of the BLOBs have the correct size but only contain null characters. I can't figure out what makes it good or full of emptiness.
The BLOB type I am using does a:
public void nullSafeSet(PreparedStatement st, Object value, int index) throws HibernateException, SQLException {
if (value == null) {
st.setNull(index, sqlTypes()[0]);
return;
}
try {
Connection conn = st.getConnection();
if (conn instanceof org.apache.commons.dbcp.DelegatingConnection) {
log.debug("Delegating connection, digging for actual driver");
conn = ((org.apache.commons.dbcp.DelegatingConnection)st.getConnection()).getInnermostDelegate();
}
OutputStream tempBlobWriter = null;
BLOB tempBlob = BLOB.createTemporary(conn, true, BLOB.DURATION_SESSION);
try {
tempBlob.open(BLOB.MODE_READWRITE);
tempBlobWriter = tempBlob.setBinaryStream(1L);
tempBlobWriter.write((byte[])value);
tempBlobWriter.flush();
} finally {
if (tempBlobWriter != null)
tempBlobWriter.close();
tempBlob.close();
}
st.setBlob(index, (Blob) tempBlob);
} catch (IOException e) {
throw new HibernateException(e);
}
}
I put a log in there and can confirm that the value (byte[]) is good. I tried to change the createTemporary parameters, no success.
I am running this under Weblogic 10.0 (can't upgrade that) with the bundled Oracle Thin driver.
A clue is that the working calls come from the standard web service deployed and managed by WLS. But the problematic calls are done from a thread started along with the component that interfaces with some legacy system with JNI. This thread works like a charm for everything except these BLOBs. I am getting a new Session just before inserting the data and closing it a bit after. (The Session does NOT remain open for the lifetime of the thread)
I have set the Hibernate log level to DEBUG but it does not give me any clue. I'm starting to run out of ideas...

Problem solved.
In fact, I was doing:
open session
open transaction
get first item from legacy system
write first item to database (blob)
close transaction
open transaction
get second item from legacy system
write second item to database (blob)
close transaction
... until the legacy system has nothing more to process
close session
This would typically process between 1 and 5 items per round.
But because the Oracle driver does not use the standard way of handling blobs in JDBC, our custom type has to create a temporary blob that is stored in the session. And apparently when you're inserting blobs in differents transactions within the same session, they tend to interfere and cause my problem.
I solved it by closing the session after each commit. I do not like it but I consider it being the Oracle driver's fault.

Related

The system hang while calling sqlSesssion.flushStatement()

I'm using some SQL query with MyBatis to update data to my Oracle db, but the system stop half way and not throwing any error.
Some forum say that it might because of connection pool size, time out problem or might be SqlSessionFactory configuration problem.
I changed all the timeout to 30 sec but still have the same problem
//my hikari datasource setup
#Bean
public HikariDataSource dataSource() {
HikariDataSource db = new HikariDataSource();
db.setDriverClassName(driverClassName);
db.setJdbcUrl(url);
db.setUsername(username);
db.setPassword(pwd);
db.setReadOnly(false);
db.setMaximunPoolSize(80);
db.setConnectionTimeout(30000);
db.setIdleTimeout(30000);
db.setMaxLifetime(30000);
db.setMinimunIdle(5);
db.setValidationTimeout(500);
return db;
}
The section I'm calling update to my Oracle db
try (SqlSession sqlSession = sqlSessionFactory.openSession(ExecutorType.BATCH)) {
ItemMapper mapper = sqlSession.getMapper(ItemMapper.class);
for (Object obj: lists) {
//It only works fine when I'm calling insert function
mapper.update(obj);
}
//After running this statement then system got stuck
sqlSession.flushStatements();
sqlSession.clearCache();
sqlSession.commit();
} catch (Exception ex) {
ex.printStackTrace();
}
This is MyBatis .xml file
<update id="update">
UPDATE <include refid="tableName" />
SET
item_price = #{price},
update_time = #{updateTime}
WHERE id = #{id}
</update>
But the weird things is everything works fine while I'm calling insert sql statement, the system will hang or stuck at sqlSession.flushStatements(); and not throwing any errors only when I change to update sql statement.
UPDATE
Even I use normal MyBatis update query also make the whole system hang but nothing happen if I use insert query..
The normal update query i execute
Obj obj = new Obj();
obj.setprice("1");
obj.setupdateTime(new Date());
mapper.update(obj);
This problem has been solve weirdly, seems like there is some uncommited session in somewhere although I actually restarted my server.
Just manually one click of commit button in Oracle database, then my program able to update data in database.
Please feel free to share any relevant information about some weird uncommited session of Oracle database here

JDBC statement pooling with DB2 does not have significant time difference

I'm using JDBC db2 driver, a.k.a. JT400 to connect to db2 server on Application System/400, a midrange computer system.
My goal is to insert into three Tables, from outside of IBM mainframe, which would be cloud instance(eg. Amazon WS).
To make the performance better
1) I am using already established connections to connect to db2.
(using org.apache.commons.dbcp.BasicDataSource or com.ibm.as400.access.AS400JDBCManagedConnectionPoolDataSource, both are working fine.)
public class AS400JDBCManagedConnectionPoolDataSource extends AS400JDBCManagedDataSource implements ConnectionPoolDataSource, Referenceable, Serializable {
}
public class AS400JDBCManagedDataSource extends ToolboxWrapper implements DataSource, Referenceable, Serializable, Cloneable {
}
2) I want to cache the insert into statements for all three tables, so that I don't have to send query every time and compile every time, which is expensive. I would instead just pass the parameters only. (Obviously I am doing this using JDBC prepared statements)
Based on an official IBM document Optimize Access to DB2 for i5/OS
from Java and WebSphere, page 17-20 - Enabling Extended Dynamic Support, it's possible to cache the statement with AS400JDBCManagedConnectionPoolDataSource.
BUT, the problem is the insert into queries are being compiled each time, which is taking 200ms * 3 queries = 600ms each time.
Example I'm using,
public class CustomerOrderEventHandler extends MultiEventHandler {
private static Logger logger = LogManager.getLogger(CustomerOrderEventHandler.class);
//private BasicDataSource establishedConnections = new BasicDataSource();
//private DB2SimpleDataSource nativeEstablishedConnections = new DB2SimpleDataSource();
private AS400JDBCManagedConnectionPoolDataSource dynamicEstablishedConnections =
new AS400JDBCManagedConnectionPoolDataSource();
private State3 orderState3;
private State2 orderState2;
private State1 orderState1;
public CustomerOrderEventHandler() throws SQLException {
dynamicEstablishedConnections.setServerName(State.server);
dynamicEstablishedConnections.setDatabaseName(State.DATABASE);
dynamicEstablishedConnections.setUser(State.user);
dynamicEstablishedConnections.setPassword(State.password);
dynamicEstablishedConnections.setSavePasswordWhenSerialized(true);
dynamicEstablishedConnections.setPrompt(false);
dynamicEstablishedConnections.setMinPoolSize(3);
dynamicEstablishedConnections.setInitialPoolSize(5);
dynamicEstablishedConnections.setMaxPoolSize(50);
dynamicEstablishedConnections.setExtendedDynamic(true);
Connection connection = dynamicEstablishedConnections.getConnection();
connection.close();
}
public void onEvent(CustomerOrder orderEvent){
long start = System.currentTimeMillis();
Connection dbConnection = null;
try {
dbConnection = dynamicEstablishedConnections.getConnection();
long connectionSetupTime = System.currentTimeMillis() - start;
state3 = new State3(dbConnection);
state2 = new State2(dbConnection);
state1 = new State1(dbConnection);
long initialisation = System.currentTimeMillis() - start - connectionSetupTime;
int[] state3Result = state3.apply(orderEvent);
int[] state2Result = state2.apply(orderEvent);
long state1Result = state1.apply(orderEvent);
dbConnection.commit();
logger.info("eventId="+ getEventId(orderEvent) +
",connectionSetupTime=" + connectionSetupTime +
",queryPreCompilation=" + initialisation +
",insertionOnlyTimeTaken=" +
(System.currentTimeMillis() - (start + connectionSetupTime + initialisation)) +
",insertionTotalTimeTaken=" + (System.currentTimeMillis() - start));
} catch (SQLException e) {
logger.error("Error updating the order states.", e);
if(dbConnection != null) {
try {
dbConnection.rollback();
} catch (SQLException e1) {
logger.error("Error rolling back the state.", e1);
}
}
throw new CustomerOrderEventHandlerRuntimeException("Error updating the customer order states.", e);
}
}
private Long getEventId(CustomerOrder order) {
return Long.valueOf(order.getMessageHeader().getCorrelationId());
}
}
And the States with insert commands look like below,
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import java.sql.Connection;
import java.sql.PreparedStatement;
import java.sql.SQLException;
public class State2 extends State {
private static Logger logger = LogManager.getLogger(DetailState.class);
Connection connection;
PreparedStatement preparedStatement;
String detailsCompiledQuery = "INSERT INTO " + DATABASE + "." + getStateName() +
"(" + DetailState.EVENT_ID + ", " +
State2.ORDER_NUMBER + ", " +
State2.SKU_ID + ", " +
State2.SKU_ORDERED_QTY + ") VALUES(?, ?, ?, ?)";
public State2(Connection connection) throws SQLException {
this.connection = connection;
this.preparedStatement = this.connection.prepareStatement(detailsCompiledQuery); // this is taking ~200ms each time
this.preparedStatement.setPoolable(true); //might not be required, not sure
}
public int[] apply(CustomerOrder event) throws StateException {
event.getMessageBody().getDetails().forEach(detail -> {
try {
preparedStatement.setLong(1, getEventId(event));
preparedStatement.setString(2, getOrderNo(event));
preparedStatement.setInt(3, detail.getSkuId());
preparedStatement.setInt(4, detail.getQty());
preparedStatement.addBatch();
} catch (SQLException e) {
logger.error(e);
throw new StateException("Error setting up data", e);
}
});
long startedTime = System.currentTimeMillis();
int[] inserted = new int[0];
try {
inserted = preparedStatement.executeBatch();
} catch (SQLException e) {
throw new StateException("Error updating allocations data", e);
}
logger.info("eventId="+ getEventId(event) +
",state=details,insertionTimeTaken=" + (System.currentTimeMillis() - startedTime));
return inserted;
}
#Override
protected String getStateName() {
return properties.getProperty("state.order.details.name");
}
}
So the flow is each time an event is received(eg. CustomerOrder), it gets the establishedConnection and then asks the states to initialise their statements.
The metrics for timing look as below,
for the first event, it takes 580ms to create the preparedStatements for 3 tables.
{"timeMillis":1489982655836,"thread":"ScalaTest-run-running-CustomerOrderEventHandlerSpecs","level":"INFO","loggerName":"com.xyz.customerorder.events.handler.CustomerOrderEventHandler",
"message":"eventId=1489982654314,connectionSetupTime=1,queryPreCompilation=580,insertionOnlyTimeTaken=938,insertionTotalTimeTaken=1519","endOfBatch":false,"loggerFqcn":"org.apache.logging.log4j.spi.AbstractLogger","threadId":1,"threadPriority":5}
for the second event, takes 470ms to prepare the statements for 3 tables, which is less than the first event but just < 100ms, I assume it to be drastically less, as it should not even make it to compilation.
{"timeMillis":1489982667243,"thread":"ScalaTest-run-running-PurchaseOrderEventHandlerSpecs","level":"INFO","loggerName":"com.xyz.customerorder.events.handler.CustomerOrderEventHandler",
"message":"eventId=1489982665456,connectionSetupTime=0,queryPreCompilation=417,insertionOnlyTimeTaken=1363,insertionTotalTimeTaken=1780","endOfBatch":false,"loggerFqcn":"org.apache.logging.log4j.spi.AbstractLogger","threadId":1,"threadPriority":5}
What I'm thinking is since I'm closing preparedStatement for that particular connection, it does not even exist for new connection. If thats the case whats the point of having statement caching at all in multi-threaded environment.
The documentation has similar example, where its making transactions inside the same connection which is not the case for me, as I need to have multiple connections at the same time.
Questions
Primary
Q1) Is DB2 JDBC driver caching the statements at all, between multiple connections? Because I don't see much difference while preparing the statement. (see example, first one takes ~600ms, second one takes ~500ms)
References
ODP = Open Data Path
SQL packages
SQL packages are permanent objects used to store information related
to prepared SQL statements. They can be used by the IBM iSeries Access
for the IBM Toolbox for
Java JDBC driver. They are also used by applications which use the
QSQPRCED (SQL Process Extended Dynamic) API interface.
In the case JDBC, the existence of the SQL package is
checked when the client application issues the first prepare of a SQL
Statement. If the package does not exist, it is created at that time
(even though it may not yet contain any SQL statements)
Tomcat jdbc connection pool configuration - DB2 on iSeries(AS400)
IBM Data Server Driver for JDBC and SQLJ statement caching
A couple of important things to note regarding statement caching:
Because Statement objects are child objects of a given Connection, once the Connection is closed all child objects (e.g. all Statement objects) must also be closed.
It is not possible to associate a statement from one connection with a different connection.
Statement pooling may or may not be done be by a given JDBC driver. Statement pooling may also be performed by a connection management layer (i.e. application server)
Per JDBC spec, default value for Statement.isPoolable() == false and PreparedStatement.isPoolable() == true, however this flag is only a hint to the JDBC driver. There is no guarantee from the spec that statement pooling will occur.
First off, I am not sure if the JT400 driver does statement caching. The document you referenced in your question comment, Optimize Access to DB2 for i5/OS from Java and WebSphere, is specific to using the JT400 JDBC driver with WebSphere application server, and on slide #3 it indicates that statement caching comes from the WebSphere connection management layer, not the native JDBC driver layer. Given that, I'm going to assume that the JT400 JDBC driver doesn't support statement caching on its own.
So at this point you are probably going to want to plug into some sort of app server (unless you want to implement statement caching on your own, which is sort of re-inventing the wheel). I know for sure that both WebSphere Application Server products (traditional and Liberty) support statement caching for any JDBC driver.
For WebSphere Liberty (the newer product), the data source config is the following:
<dataSource jndiName="jdbc/myDS" statementCacheSize="10">
<jdbcDriver libraryRef="DB2iToolboxLib"/>
<properties.db2.i.toolbox databaseName="YOURDB" serverName="localhost"/>
</dataSource>
<library id="DB2iToolboxLib">
<fileset dir="/path/to/jdbc/driver/dir" includes="jt400.jar"/>
</library>
The key bit being the statementCacheSize attribute of <dataSource>, which has a default value of 10.
(Disclaimer, I'm a WebSphere dev, so I'm going to talk about what I know)
First off, the IBM i Java documentation is here: IBM Toolbox for Java
Secondly, I don't see where you are setting the "extended dynamic" property to true which provides
a mechanism for caching dynamic SQL statements on the server. The first
time a particular SQL statement is prepared, it is stored in a SQL
package on the server. If the package does not exist, it is
automatically created. On subsequent prepares of the same SQL
statement, the server can skip a significant part of the processing by
using information stored in the SQL package. If this is set to "true",
then a package name must be set using the "package" property.
I think you're missing some steps in using the managed pool...here's the first example in the IBM docs
import javax.naming.Context;
import javax.naming.InitialContext;
import javax.sql.DataSource;
import com.ibm.as400.access.AS400JDBCManagedConnectionPoolDataSource;
import com.ibm.as400.access.AS400JDBCManagedDataSource;
public class TestJDBCConnPoolSnippet
{
void test()
{
AS400JDBCManagedConnectionPoolDataSource cpds0 = new AS400JDBCManagedConnectionPoolDataSource();
// Set general datasource properties. Note that both connection pool datasource (CPDS) and managed
// datasource (MDS) have these properties, and they might have different values.
cpds0.setServerName(host);
cpds0.setDatabaseName(host);//iasp can be here
cpds0.setUser(userid);
cpds0.setPassword(password);
cpds0.setSavePasswordWhenSerialized(true);
// Set connection pooling-specific properties.
cpds0.setInitialPoolSize(initialPoolSize_);
cpds0.setMinPoolSize(minPoolSize_);
cpds0.setMaxPoolSize(maxPoolSize_);
cpds0.setMaxLifetime((int)(maxLifetime_/1000)); // convert to seconds
cpds0.setMaxIdleTime((int)(maxIdleTime_/1000)); // convert to seconds
cpds0.setPropertyCycle((int)(propertyCycle_/1000)); // convert to seconds
//cpds0.setReuseConnections(false); // do not re-use connections
// Set the initial context factory to use.
System.setProperty(Context.INITIAL_CONTEXT_FACTORY, "com.sun.jndi.fscontext.RefFSContextFactory");
// Get the JNDI Initial Context.
Context ctx = new InitialContext();
// Note: The following is an alternative way to set context properties locally:
// Properties env = new Properties();
// env.put(Context.INITIAL_CONTEXT_FACTORY, "com.sun.jndi.fscontext.RefFSContextFactory");
// Context ctx = new InitialContext(env);
ctx.rebind("mydatasource", cpds0); // We can now do lookups on cpds, by the name "mydatasource".
// Create a standard DataSource object that references it.
AS400JDBCManagedDataSource mds0 = new AS400JDBCManagedDataSource();
mds0.setDescription("DataSource supporting connection pooling");
mds0.setDataSourceName("mydatasource");
ctx.rebind("ConnectionPoolingDataSource", mds0);
DataSource dataSource_ = (DataSource)ctx.lookup("ConnectionPoolingDataSource");
AS400JDBCManagedDataSource mds_ = (AS400JDBCManagedDataSource)dataSource_;
boolean isHealthy = mds_.checkPoolHealth(false); //check pool health
Connection c = dataSource_.getConnection();
}
}

Google App Engine: Object with id “” is managed by a different Object Manager - Revisited

I'm getting the following error using GAE, JPA, and Spring
Object with id “” is managed by a different Object Manager
When I first create an account, I put the User object in the session. Then when I update the user profile during that initial session, I merge the detached User. All works great.
I then logout and later create a new session. This time, I load the User object and place into the session. Still OK, but problem is when I update the user profile, the merge fails with the above error.
public boolean loadProfile(String openId, String email) {
User user = null;
try {
user = userDao.findByOpenId(openId);
} catch (NoResultException e) {
}
if (user != null) {
logger.error(JDOHelper.getPersistenceManager(user));
getSessionBean().setUser(user);
return true;
} else {
user = createNewAccount(openId, email);
getSessionBean().setUser(user);
return false;
}
}
#Transactional(propagation=Propagation.REQUIRES_NEW)
private User createNewAccount(String openId, String email) {
User user = new User();
user.setDisplayName(Long.toString(System.currentTimeMillis()));
OpenIdentifier oid = new OpenIdentifier();
oid.setOpenId(openId);
oid.setEmail(email);
oid.setUser(user);
Set<OpenIdentifier> openIds = new HashSet<OpenIdentifier>();
openIds.add(oid);
user.setOpenIds(openIds);
user = userDao.merge(user);
return user;
}
#Transactional(propagation=Propagation.REQUIRED)
public void createOrUpdate(ActionEvent e) {
logger.error(JDOHelper.getPersistenceManager(userFacade.getDelegate()));
User user = userDao.merge(userFacade.getDelegate());
sessionBean.setUser(user);
}
I found these related questions, but I'm still not able to fix.
AppEngine datastore: "Object with id ... is managed by a different Object Manager"
Google App Engine - Object with id "" is managed by a different - JPA
Datanucleus: moving from #Transactional to non-transactional
http://www.atentia.net/2010/03/object-with-id-is-managed-by-a-different-object-manager/
WRT closing the PM (as per 1 & 2), I'm not able to explicitly close the PM since I'm using Spring
org.springframework.orm.jpa.support.OpenEntityManagerInViewFilter. From logs, it appears to be opening and closing on each page request.
WRT making the entity detachable (as per 3 & 4), first of all, I'm using JPA and it seems wrong to use a JDO-related annotation. Secondly, it didn't work when I tried.
For extra credit, how do you debug with JDOHelper.getPersistenceManager(obj)? I am getting null in this case, as the User was detached between page requests. That seems normal to me so I'm not clear how to debug with it.
You don't have a PM, you have an EM. No idea what you're referring to there.
Detachable : with JPA all classes are (enhanced as) detachable
You're using some ancient GAE JPA plugin there (v1.x?), and that uses old versions of DataNucleus that are not supported. Use GAE JPA v2.x. "ObjectManager" hasn't existed in DataNucleus for years.
You (or the software you're using) have to close the EM or you get resources leaked all over the place.
NucleusJPAHelper.getEntityManager(obj); is how you get the EntityManager that manages an object (in DataNucleus v3.x, used by GAE JPA v2.x)

Can I switch use of 'entities.SingleOrDefault' ==> 'entities.Find' without hazards?

In my WCF service's business logic, most of the places when I need to locate an entity, I use this syntax:
public void UpdateUser(Guid userId, String notes)
{
using (ProjEntities entities = new ProjEntities())
{
User currUser = entities.SingleOrDefault(us => us.Id == userId);
if (currUser == null)
throw new Exception("User with ID " + userId + " was not found");
}
}
I have recentely discovered that the DbContext has the Find method, and I understand I can now do this:
public void UpdateUser(Guid userId, String notes)
{
using (ProjEntities entities = new ProjEntities())
{
User currUser = entities.Find(userId);
if (currUser == null)
throw new Exception("User with ID " + userId + " was not found");
}
}
Note : the 'userId' property is the primary key for the table.
I read that when using Find method entity framework checks first to see if the entity is already in the local memory, and if so - brings it from there. Otherwise - a trip is made to the database (vs. SingleOrDefault which always makes a trip to the database).
I was wondering if I now will convert all my uses of SingleOrDefault to Find is there any potential of danger?
Is there a chance I could get some old data that has not been updated if I use Find and it fetches the data from memory instead of the database?
What happens if I have the user in memory, and someone changed the user in the database - won't it be a problem if I always use now this 'memory' replica instead of always fetching the latest updated one from the database?
Is there a chance I could get some old data that has not been updated
if I use Find and it fetches the data from memory instead of the
database?
I think you have sort of answered your own question here. Yes, there is a chance that using Find you could end up having an entity returned that is out of sync with your database because your context has a local copy.
There isn't much more anyone can tell you without knowing more about your specific application; do you keep a context alive for a long time or do you open it, do your updates and close it? obviously, the longer you keep your context around the more susceptible you are to retrieving an up to date entity.
I can think of two strategies for dealing with this. The first is outlined above; open your context, do what you need and then dispose of it:
using (var ctx = new MyContext())
{
var entity = ctx.EntitySet.Find(123);
// Do something with your entity here...
ctx.SaveChanges();
}
Secondly, you could retrieve the DbEntityEntry for your entity and use the GetDatabaseValues method to update it with the values from the database. Something like this:
var entity = ctx.EntitySet.Find(123);
// This could be a cached version so ensure it is up to date.
var entry = ctx.Entry(entity);
entry.OriginalValues.SetValues(entry.GetDatabaseValues());

Saving Data Locally and Remotely (Syncing)

When data is entered, it ultimately needs to be saved remotely on a server. I do want the app to work if there is no data connection at the time also, so I need to save everything locally on the phone too. The app can then sync with the server when it gets a connection.
This brings up a little issue. I'm used to saving everything on the server and then getting the records back with id's generated from the server for them. If there is no connection, the app will save locally to the phone but not the server. When syncing with the server, I don't see a way for the phone to know when a record comes back which locally record it's associated with. There isn't enough unique data to figure this out.
What is the best way to handle this?
One way I've been thinking is to change the id of the records to a GUID and let the phone set the id. This way, all records will have an id locally, and when saving to the server, it should still be a unique id.
I'd like to know what other people have been doing, and what works and what doesn't from experience.
This is how we done with a first windows phone 7 app finished few days ago with my friend.
It might not be the best solution but 'till additional refactoring it works just fine.
It's an application for a web app like a mint.com called slamarica.
If we have feature like save transaction, we first check if we have connection to internet.
// Check if application is in online or in offline mode
if (NetworkDetector.IsOnline)
{
// Save through REST API
_transactionBl.AddTransaction(_currentTransaction);
}
else
{
// Save to phone database
SaveTransactionToPhone(_currentTransaction);
}
If transaction is successfully saved via REST, it responses with transaction object and than we save it to local database. If REST save failed we save data to local database.
private void OnTransactionSaveCompleted(bool isSuccessful, string message, Transaction savedTransaction)
{
MessageBox.Show(message);
if(isSuccessful)
{
// save new transaction to local database
DatabaseBl.Save(savedTransaction);
// save to observable collection Transactions in MainViewModel
App.ViewModel.Transactions.Add(App.ViewModel.TransactionToTransactionViewModel(savedTransaction));
App.ViewModel.SortTransactionList();
// Go back to Transaction List
NavigationService.GoBack();
}
else
{
// if REST is failed save unsent transaction to Phone database
SaveTransactionToPhone(_currentTransaction);
// save to observable collection Transactions in MainViewModel
App.ViewModel.Transactions.Add(App.ViewModel.TransactionToTransactionViewModel(_currentTransaction));
App.ViewModel.SortTransactionList();
}
}
Every Transaction object has IsInSync property. It is set to false by default until we got confirmation from REST API that it's saved successful on the server.
User has ability to refresh transactions. User can click on a button Refresh to fetch new data from the server. We do the syncing in the background like this:
private void RefreshTransactions(object sender, RoutedEventArgs e)
{
if (NetworkDetector.IsOnline)
{
var notSyncTransactions = DatabaseBl.GetData<Transaction>().Where(x => x.IsInSync == false).ToList();
if(notSyncTransactions.Count > 0)
{
// we must Sync all transactions
_isAllInSync = true;
_transactionSyncCount = notSyncTransactions.Count;
_transactionBl.AddTransactionCompleted += OnSyncTransactionCompleted;
if (_progress == null)
{
_progress = new ProgressIndicator();
}
foreach (var notSyncTransaction in notSyncTransactions)
{
_transactionBl.AddTransaction(notSyncTransaction);
}
_progress.Show();
}
else
{
// just refresh transactions
DoTransactionRefresh();
}
}
else
{
MessageBox.Show(ApplicationStrings.NETWORK_OFFLINE);
}
}
private void DoTransactionRefresh()
{
if (_progress == null)
{
_progress = new ProgressIndicator();
}
// after all data is sent do full reload
App.ViewModel.LoadMore = true;
App.ViewModel.ShowButton = false;
ApplicationBl<Transaction>.GetDataLoadingCompleted += OnTransactionsRefreshCompleted;
ApplicationBl<Transaction>.GetData(0, 10);
_progress.Show();
}
OnTransactionRefreshCompleted we delete all transaction data in local database and get the latest 10 transactions. We don't need all the data, and this way user have synced data. He can always load more data by taping load more at the end of transaction list. It's something similar like those twitter apps.
private void OnTransactionsRefreshCompleted(object entities)
{
if (entities is IList<Transaction>)
{
// save transactions
var transactions = (IList<Transaction>)entities;
DatabaseBl.TruncateTable<Transaction>();
DatabaseBl.Save(transactions);
((MainViewModel) DataContext).Transactions.Clear();
//reset offset
_offset = 1;
//update list with new transactions
App.ViewModel.LoadDataForTransactions(transactions);
App.ViewModel.LoadMore = false;
App.ViewModel.ShowButton = true;
}
if (entities == null)
{
App.ViewModel.ShowButton = false;
App.ViewModel.LoadMore = false;
}
// hide progress
_progress.Hide();
// remove event handler
ApplicationBl<Transaction>.GetDataLoadingCompleted -= OnTransactionsRefreshCompleted;
}
Caveat - I haven't tried this with windows phone development but use of GUID identities is something I usually do when faced with similar situations - eg creating records when I only have a one-way connection to the database - such as via a message bus or queue.
It works fine, albeit with a minor penalty in record sizes, and can also cause less performant indexes. I suggest you just give it a shot.

Resources