I wanted to clarify where the actual shards are… in Redis or Cassandra?
5 Likes
Have the same question, the article doesn’t explain this well enough. It seems nobody is checking the messages since the original question was asked more than 8 months ago.
2 Likes
Probably Redis since it’s an in memory database and has good performance for rapidly changing data. Each shard can be key-value pair in a redis dictionary. Furthermore, we can probably place several 100s of shards for same counter on the same redis database while the counter redis database would store a list of pointers to these shards.
Hi Brigette.
We use Redis (or Memcached) for storing sharded counters because it’s in-memory storage and, hence, fast for random reads and writes.
For storing accumulated counters (for example, for all shards), we use Cassandra. Cassandra can handle more complicated data than key values.
In summary, we use Redis as a cache for reading and writing shards of counters. We use Cassandra for storing more complicated derivative data, such as accumulated counters.
I hope that clears up the confusion. Feel free to reach out to us if you have any further queries.
Thank you.