shreyansh

High Level Design

Theory

  1. Database Indexes
    1. How hash indexes work?
    2. How B-Tree indexes work?
    3. How LSM Tree & SSTable indexes work?
    4. Indexes conclusion
  2. Transactions
    1. ACID database transactions
    2. Read committed isolation
    3. Snapshot isolation
    4. Write Skew and Phantom writes
    5. Achieving ACID: Serial execution
  3. Database Internals
    1. Two Phase Locking
    2. Serializable Snapshot Isolation
    3. Column Oriented Storage
    4. Data Serialization Frameworks
  4. Intro to Replication - Need to knows 1. replication
    1. Dealing with Stale Reads

      Monotonic reads are particularly relevant in distributed systems with data replication, where data is copied and maintai

    2. Single Leader Replication

      Last write: 80, was written to the replication log.

    3. MultiLeader Replication - chaos

      As an alpha sigma Male myself, not a big fan of having multiple leaders. Clearly, such a setup is destined to fail and h

    4. Write Conflicts

      Concurrent writes: Conflicting writes that are unaware of each other at the time of writing.

    5. Conflict-Free Replicated Data Types - CRDTs

      Conflict-Free Replicated Data Types (CRDTs) are designed to enable concurrent updates and ensure eventual consistency ac

    6. Leaderless Replication

      In leaderless replication, there is no single leader node that coordinates write operations. Instead, write operations c

    7. Quorums

      If there are n replicas, every write must be confirmed by w nodes to be considered successful, and we must query at leas

    8. Conclude Replication

      Replication is a key strategy in distributed systems to enhance availability, fault tolerance, and scalability. There ar

  5. Into to Partitioning or Sharding 2. sharding-partitioning

    For very large datasets, or very high query throughput, replication is not sufficient: we need to break the data into pa

    1. Atomic Commit and Two-Phase Commit (2PC)

      Atomicity prevents failed transactions from littering the database with half-finished results and half-updated state. Th

    2. Rebalancing Partitions

      Rebalancing is the process of redistributing data and requests across nodes in a cluster to handle changes such as incre

  6. Consistency and Consensus
    1. Linearizability

      In a database with multiple nodes, inconsistencies occur because write requests arrive at different times on different n

    2. Distributed Consensus
      1. The Raft Consensus Algorithm - Leader Election

        https://web.stanford.edu/ouster/cgi-bin/cs190-winter20/lecture.php?topic=raft

      2. Raft Writes

        Once a leader is elected, it is responsible for handling all client write requests. Here’s how a typical write operation

    3. Coordination Services

      Coordination services are just thin layer that're built on distributed consensus algorithm like Raft (The Raft Consensus

  7. Databases Duel Arena
    1. Relational Vs Non-Relational DBs

      SQL vs NoSQL (not preferred terms)

    2. MySQL vs PostgreSQL

      SQL databases are particularly well-suited for handling normalized, relational data. They excel at managing data in a st

    3. MongoDB vs Cassandra

      MongoDB is a NoSQL database that is document-oriented, meaning it stores data in flexible, JSON-like documents. Unlike t

  8. Streams
    1. Stream Processing

      Stream processing is basically real-time processing of a continuous data streams as they're generated. Unlike, batch pro

    2. Kafka vs RabbitMQ

      Here is a comparison table highlighting the key differences between Kafka (a log-based message broker) and RabbitMQ (an