educative.io

Where would videos be stored?

Courses
,
#1

Where would videos be stored? Videos can be stored in a distributed file storage system like HDFS or GlusterFS.

why don’t we use an object store such as s3? what are pros and cons of object store vs. distributed file system?

#2

Hi @Dewey_Munoz

We preferred distributed file storage because we would have huge traffic. We can expect our read:write ratio to be 200:1, which means for every video upload, there are 200 video views. Thus distributed file storage is more preferable.
Object-storage systems provide eventual consistency, whereas distributed-file systems can support strong consistency or eventual consistency (both, depending on the vendor).

Object-storage exposes a REST API, and is therefore limited to applications that are specially designed to support this type of storage. Whereas, distributed file systems expose a traditional filesystem API, i.e., they are suitable for any application, including legacy applications which were designed to work over a hierarchical filesystem.

#3

I totally disagree with the reply posted. Object storage services like S3 are preferred for video uploads because of support for multi-part uploads. In fact, you would always want to solution this using a vendor managed service that provides a) quick throughput and b) quick recovery from transient failures. Traditional file system APIs aren’t designed to support any of this for video uploads.

1 Like