SD Occasions Open-Supply Challenge of the Week: SeaweedFS
SeaweedFS is a straightforward and highly-scalable distributed file system that’s an impartial Apache-licensed open supply venture.
SeaweedFS began as an Object Retailer to deal with small recordsdata effectively. As a substitute of managing all file metadata in a central grasp, the central grasp solely manages file volumes, and it lets these quantity servers handle recordsdata and their metadata. The venture can transparently combine with the cloud.
“With scorching information on native cluster, and heat information on the cloud with O(1) entry time, SeaweedFS can obtain each quick native entry time and elastic cloud storage capability, with none consumer aspect modifications,” the working group behind the venture wrote in a put up.
SeaweedFS began by implementing Fb’s Haystack design paper and implements erasure coding with concepts from f4: Fb’s Heat BLOB Storage System.
Additionally, the Filer can help directories and POSIX attributes. Filer is a separate linearly-scalable stateless server with customizable metadata shops, MySQL, Postgres, MongoDB, Redis, Cassandra, Elastic Search, LevelDB, MemSQL, TiDB, etcd, CockroachDB, and extra.
Further options embrace the flexibility to decide on no replication or totally different replication ranges, rack and information heart conscious, in addition to computerized grasp server failover, computerized GZip compression relying on file mime sort, and different automated options and help.