This is an open-source, MIT-licensed implementation of Uber's Schemaless (immutable BigTable-style sharded RDBMS).
This is only a learning project and not production ready.
The github issues list describes what has been intentionally left unimplemented and what differences there are between this implementation and Uber's (based on the materials linked at the end.)
All code is in Go.
API SUPPORTED
Get(ctx context.Context, tableName, rowKey, columnKey string, refKey int64) (cell models.Cell, found bool, err error)
GetLatest(ctx context.Context, tableName, rowKey, columnKey string) (cell models.Cell, found bool, err error)
PartitionRead(ctx context.Context, tableName string, partitionNumber int, location string, value int64, limit int) (cells []models.Cell, found bool, err error)
FindPartition(tblName, rowKey string) (int, error)
Put(ctx context.Context, tableName, rowKey, columnKey string, refKey int64, jsonBody string) (err error)
ResetConnection(ctx context.Context, key string) error
Destroy(ctx context.Context) error
DATABASE SUPPORT
For learning or other:
* SQLite
For more serious testing and usage:
* MySQL
* Postgres
DISCLAIMER
I do not work for Uber Technologies.
VIDEOS
"Taking Storage for a Ride With Uber", https://www.youtube.com/watch?v=Dg76cNaeB4s (30 mins)
"GOTO 2016 • Taking Storage for a Ride", https://www.youtube.com/watch?v=kq4gp90QUcs (1 hour)
ARTICLES
"Designing Schemaless, Uber Engineering’s Scalable Datastore Using MySQL"
https://eng.uber.com/schemaless-part-one/ - "Part One"
https://eng.uber.com/schemaless-part-two/ - "Part Two"
https://eng.uber.com/schemaless-part-three/ - "Part Three"
https://eng.uber.com/schemaless-rewrite/ - "Code Migration in Production: Rewriting the Sharding Layer of Uber’s Schemaless Datastore"
https://eng.uber.com/mezzanine-codebase-data-migration/ - "Project Mezzanine: The Great Migration"
OTHER RESOURCES
https://backchannel.org/blog/friendfeed-schemaless-mysql - FriendFeed's original design
https://engineering.pinterest.com/blog/sharding-pinterest-how-we-scaled-our-mysql-fleet - Pinterest's original design
https://martinfowler.com/articles/schemaless/ - Martin Fowler's slides on Schemaless Data Structures
SIMILAR OPEN-SOURCE WORK
https://github.com/hoteltonight/shameless - A similar append-only data store in Ruby influenced by Schemaless.
https://github.com/dgryski/go-shardedkv - Much of the implementation is a derivative of this work.
THANKS
To Damian Gryski for releasing https://github.com/dgryski/go-shardedkv
To Uber Technologies for releasing numerous materials on the design and implementation of Mezzanine, their Schemaless store.
To John Rinehart for NixOS support.
To Simon Dassow for Postgres debugging and support.
And to many others :)