What are some of the performance, transactional and crash recovery issues to consider for transactional and non-transactional DataStores?

Abstract: What are some of the performance, transactional and crash recovery issues to consider for transactional and non-transactional DataStores?

When transaction support is enabled I have found the JDBC driver and DataExpress to
be of similar performance. DataExpress is slightly faster for simple insert/update/delete operations

When transaction support is disabled (DataStore.TxManager property is null or DataStore.TxManager.Enabled
property is false), only DataExpress can be used (we may support non transactional JDBC in the future). This
is usually at least twice as fast as using a Transactional DataStore. Note that to
disable a DataStore that is already transactional, you must either 1) Use
DataStoreExplorer or 2) Instantiate a TxManager component, set the TxManager.Enabled
property to false and then set this TxManager to your DataStore.TxManager property.
You can't just set the DataStore.TxManager property to null to disable transaction

For changes to be persistent there are two cases to consider:

1) Transactional DataStore. For JDBC, java.sql.Connection.commit() must be called
or the connection needs to be in autoCommit mode. For DataExpress, the
DataStoreConnection.commit() method must be called. Note that closing a connection
will also cause an outstanding transaction to be committed.

2) Non-transactional DataStore. Again, only DataExpress components can access
these at this time. There is a DataStore.SaveMode property that controls the saving of dirty
cache blocks. There is also a Daemon thread (also used for transactional
DataStores) that saves dirty pages every 100 milliseconds by default. There is a
DataStore.SaveInterval property that can be used to adjust this frequency.
DataStore.save() and DataStore.close() force all dirty cache pages to disk. So by
default we get dirty pages to disk pretty quickly.

Whether or not a DataStore is transactional, we always know if the DataStore was
shutdown properly. There is a Response event that can be wired via the
DataStore.addResponseListener, that will tell you lots of interesting things
including the fact that a DataStore was not shutdown properly (i.e. system or
application crash). With a transactional DataStore, crash recovery will proceed to
repair the DataStore and rollback uncommitted transactions. With a non-transactional
DataStore, you can run StreamVerifier.verify() on the DataStore. If
there are errors, you could use DataStoreConnection.copyStreams() to repair the

Use transactional DataStore for for JDBC SQL access, crash recovery, and
transactional semantics (commit/rollback).

Use non transactional DataStore for higher performance with a small chance of data
loss (no crash recovery). You may want to primarily run with transaction support
enabled, but disable transaction support to quickly perform a time consuming batch
operation. The drawback to such an optimization is that a failure during the long running batch
operation could leave your DataStore file in a damaged state.