Testing Cassandra ApplicationsChristopher Batey@chbatey

@chbatey

Overview

● Cassandra overview● Cassandra failure scenarios e.g ReadTimeout,

WriteTimeout, Unavailable● Existing technologies:

○ CCM○ Cassandra Unit○ Integration tests

● Stubbed Cassandra

@chbatey

Sleeping is good

@chbatey

Single node

S

@chbatey

More realistic

@chbatey

Tunable consistency

● Each write/read is done at a consistency● ANY, ONE, TWO, THREE, QUORUM,

LOCAL_QUORUM, EACH_QUORUM, ALL● Any more than ANY and ONE require a

cluster of more than one node to make sense

@chbatey

Read and write timeouts

@chbatey

Unavailable

@chbatey

Unit & Integration testing● Requirements:

○ Quick to run○ Deterministic○ Not brittle!!

● Integration tests against an embedded Cassandra? E.g. cassandra-unit

● Integration tests against an external Cassandra running on developer / CI machines

● Mocking the driver

S

@chbatey

Acceptance testing

@chbatey

How do this if it was a HTTP service?

● Release it - great book● Wiremock - mocking HTTP services● Saboteur - adding network latency

@chbatey

Stubbed Cassandra

@chbatey

Some code that needs testingpublic List<Person> retrieveNames() {

ResultSet result;

try {

Statement statement = new SimpleStatement("select * from person");

statement.setConsistencyLevel(ConsistencyLevel.QUORUM);

result = session.execute(statement);

} catch (ReadTimeoutException e) {

throw new UnableToRetrievePeopleException();

}

List<Person> people = new ArrayList<>();

for (Row row : result) {

people.add(new Person(row.getString("first_name"), row.getString("last_name")));

}

return people;

}

@chbatey

The Java API

● Wraps Stubbed Cassandra ● With a method call can:

○ Create and start an instance○ Stop an instance○ Prime to behave in a certain way○ Verify interactions

@chbatey

Starting Scassandra@BeforeClass

public static void startScassandraServer() throws Exception {

// can override port, listen address

scassandra = ScassandraFactory.createServer();

scassandra.start();

primingClient = scassandra.primingClient();

activityClient = scassandra.activityClient();

}

@chbatey

Priming Client public void primeQuery(PrimingRequest primeRequest)

public void primePreparedStatement(PrimingRequest primeRequest)

public List<PrimingRequest> retrievePreparedPrimes()

public List<PrimingRequest> retrieveQueryPrimes()

● Priming Request:○ Query○ Consistency(s)○ Rows○ Column Types○ Result e.g. Success, Readtimeout, Writetimeout, Unavailable○ Variable types (for prepared statements)

@chbatey

Activity clientpublic List<Query> retrieveQueries()

public List<Connection> retrieveConnections()

public List<PreparedStatementExecution> retrievePreparedStatementExecutions()

public void clearConnections()

public void clearQueries()

public void clearPreparedStatementExecutions()

public void clearAllRecordedActivity()

● Query has text and consistency● Prepared statement has text, consistency and the

bound variables

@chbatey

Verifying query consistencypublic void testQueryIssuedWithCorrectConsistency() {

Query expectedQuery = Query.builder()

.withQuery("select * from person")

.withConsistency("QUORUM").build();

underTest.retrieveNames();

List<Query> queries = activityClient.retrieveQueries();

assertTrue("Expected query with consistency QUORUM, found following queries: "

+ queries,

queries.contains(expectedQuery));

}

@chbatey

Testing behaviourpublic void testRetrievingOfNames() throws Exception {

Map<String, ? extends Object> row = ImmutableMap.of(

"first_name", "Chris",

"last_name", "Batey");

PrimingRequest singleRowPrime = queryBuilder()

.withQuery("select * from person")

.withRows(row)

.build();

primingClient.primeQuery(singleRowPrime);

List<Person> names = underTest.retrieveNames();

assertEquals(1, names.size());

assertEquals("Chris", names.get(0).getFirstName());

}

@chbatey

Testing errors@Test(expected = UnableToRetrievePeopleException.class)

public void testHandlingOfReadRequestTimeout() throws Exception {

PrimingRequest primeReadRequestTimeout = queryBuilder()

.withQuery("select * from person")

.withResult(Result.read_request_timeout)

.build();

primingClient.primeQuery(primeReadRequestTimeout);

underTest.retrieveNames();

}

@chbatey

Testing retry policies public void testRetriesConfiguredNumberOfTimes() throws Exception {

PrimingRequest readtimeoutPrime = PrimingRequest.queryBuilder()

.withQuery("select * from person")

.withResult(Result.read_request_timeout)

.build();

primingClient.primeQuery(readtimeoutPrime);

activityClient.clearAllRecordedActivity();

try {

underTest.retrieveNames();

} catch (UnableToRetrievePeopleException e) {}

assertEquals(CONFIGURED_RETRIES + 1, activityClient.retrieveQueries().size());

}

@chbatey

But.. I don’t like Java

@chbatey

The REST API - priming{

"when": {

"query" :"select * from person", "consistency" : ["ONE", "TWO"]

},

"then": {

"rows" :[{"first_name": "Chris", "last_name": "Batey"},

{"first_name": "Mansoor", "last_name": "Panda"},

{"first_name" : "Wayne", "last_name": "Rooney"}] ,

"column_types" : { "last_name" : "text"},

"result" : "success"

}

}

@chbatey

The REST API - Activity[{ "query": "SELECT * FROM system.schema_columns", "consistency": "ONE" }, { "query": "SELECT * FROM system.peers", "consistency": "ONE" }, { "query": "SELECT * FROM system.local WHERE key='local'", "consistency": "ONE" }, { "query": "use people", "consistency": "ONE" }, { "query": "select * from people", "consistency": "TWO" }]

@chbatey

Priming prepared statements{ "when":{ "query":"insert into people(first_name, last_name) values (?,?)" }, "then":{ "variable_types":[ "ascii", "text" ],

"result" : "success" }}

@chbatey

Priming errors{ "when": { "query" :"select * from people where name = ?" }, "then": { "result" : "read_request_timeout" }}

@chbatey

Priming cluster name{ "when": { "query": "SELECT * FROM system.local WHERE key='local'" }, "then": { "rows": [ { "cluster_name": "custom cluster name", "partitioner": "org.apache.cassandra.dht.Murmur3Partitioner", "data_center": "dc1", "rack": "rc1", "tokens": [ "1743244960790844724" ], "release_version": "2.0.1" } ], "result": "success", "column_types": { "tokens": "set" } }}

@chbatey

What else can I do?

● Start as many instances as you like in the same JVM

● Prime all primitive types● Prime sets and lists of type text● Prime prepared statements● Verify number of connections your

application makes

@chbatey

Future features

● Regular expressions for the query● Loading of schema for priming prepared

statements● Pretending to be multiple nodes

@chbatey

How does this work?

● Server implemented in Scala● Binary port for your Cassandra application to

connect to● Admin port for the REST API● Thin Java wrapper to make it easy to be

used in JUnit tests

@chbatey

How do I get it?

● It is on maven central<dependency> <groupId>org.scassandra</groupId> <artifactId>java-client</artifactId> <version>0.2.1</version> <scope>test</scope></dependency>

● Or go to www.scassandra.org○ Executable jar + REST API

@chbatey

I want to help

● Server: ○ https://github.com/scassandra/scassandra-server

● Client:○ https://github.com/chbatey/scassandra-java-client

● Tests:○ https://github.com/scassandra/scassandra-it-java-

driver-1

@chbatey

Summary

● Testing is good - I want more tools to help me

● Existing tools for Cassandra are great at happy path scenarios

● Stubbed Cassandra allows testing of edge cases

@chbatey

The end - Questions?

www.scassandra.orghttp://christopher-batey.blogspot.co.uk/

@chbatey