light weight transaction and multi-dc with cassandra - cassandra-2.0

Question

We have a problem with our Cassandra 2.0.6. cluster. Our setup is the following:
2 data centers, named: DC1, DC2
Two nodes in each DC
Using the NetworkTopologyStrategy for replication
Client is connecting with the Datastax Java Driver v. 1.0.3
First, I made the keyspace containing one table.
CREATE KEYSPACE test
WITH replication = {
'class': 'NetworkTopologyStrategy',
'DC1': '1',
'DC2': '1'
};
CREATE TABLE account (
id text,
code text,
alias text,
PRIMARY KEY (id, code)
);
And then I shutdown DC2 before running this statement:
INSERT INTO test.account (id, code, alias) VALUES ( '1', '2', '3') if not exists;
which resulted in the error message:
>>>> Unable to complete request: one or more nodes were unavailable.
Using the same environment, running this statement was OK:
INSERT INTO test.account (id, code, alias) VALUES ( '1', '2', '3')
I found the Cassandra ticket for DC-local CAS, so I thought the CQL in this situation must be processed only in the local data center, but it wasn't.
What's wrong with my understanding of light-weight transactions?

Answer 1

The DataStax documentation explains (emphasis added):
Cassandra 2.0 uses the Paxos consensus protocol, which resembles 2-phase commit, to support linearizable consistency. All operations are quorum-based and updates will incur a performance hit ...
By default, your CAS operation (IF NOT EXISTS) uses a SERIAL isolation level for Paxos and needs to contact a quorum of all replicas. When you have only two replicas, as in your case (one in each data center), quorum requires both replicas. If you try your non-CAS insert with QUORUM consistency level, it too will fail.
CASSANDRA-5797 introduces the concept of LOCAL_SERIAL isolation level but it is not the default and must be explicitly specified in order to be used. How to do that will depend on how you are interfacing with Cassandra, e.g. the cqlsh client vs the DataStax Java Driver.