educative.io

Sharding By Epoch Timestamp

Courses
,
#1

Hi,

I am little confuse here!

We tried different strategy for Sharding.

  1. Tweet Id :
    Pro’s : Hot Users problem get resolved
    Con’s : User’s Follower’s tweet will take more latency since we have to calculate from all Application server.

Understand!

  1. Tweet creation time :
    Pros : all the top tweets quickly and we only have to query a very small set of servers
    cons :
    1. Not distributed in good manner ,
    2. User’s Follower’s tweet will take more latency since we have to calculate from all Application server.

Understand too!

  1. Merge Solution

    HashKey : tweetId (New tweet will distributed uniformly)

    Tweet Id : Epoch Time + Auto-Incremental
    TweetID : universally unique

Not Understand!

NO Column added of CreationTime, Handling from tweetId ? // is it good practice

PROS :

  1. Read & write will be fast :slight_smile: (Because No secondary index using) // OK

Cons :

  1. Still we have to calculate all Hot tweets from All App Server.
  2. For get all tweets from user, he also have to take same approach by calculation from all App Servers.

–> Confusion
I am calculating from All App server for My Tweets/Follower tweets & I am not using epoch time while calculating ?
then what is the use of epoch time ?

Or am I missing something here?

1 Like
#2

@Design_Gurus
Please answer few queries related to same:

  1. what is the advantage that we get in reading when we are appending epochtime with Tweet_id ?
  2. Are we talking about the perfomance benifits we are getting because of the non-requirement of secondary index ( on timestamp) ? This too would only help the writes. For reads it still not clear what advantage this id gives…