MongoDB Transactions, Replication and Sharding Questions

Yes. A write to a single document has always been atomic, including its embedded documents and arrays. Multi-document ACID transactions arrived in MongoDB 4.0 for replica sets and 4.2 for sharded clusters. They cost more than single-document writes, so a good schema keeps data that changes together in one document and saves transactions for writes that span several, such as moving money between two accounts.

Start a session and pass it to every operation. withTransaction commits when the callback finishes, aborts if it throws, and retries the whole callback on transient errors such as write conflicts. Transactions need a replica set or a sharded cluster, not a standalone server.

const session = client.startSession();
try {
  await session.withTransaction(async () => {
    const accounts = client.db('bank').collection('accounts');
    const debit = await accounts.updateOne(
      { _id: 'asha', balance: { $gte: 100 } },
      { $inc: { balance: -100 } },
      { session }
    );
    if (debit.modifiedCount !== 1) throw new Error('insufficient funds');
    await accounts.updateOne({ _id: 'ravi' }, { $inc: { balance: 100 } }, { session });
  });
} finally {
  await session.endSession();
}

By default a transaction must finish within 60 seconds (transactionLifetimeLimitSeconds) or the server aborts it. Two transactions writing the same document cause a write conflict, and one has to retry. Uncommitted changes are held in the WiredTiger cache, so MongoDB advises modifying no more than about 1,000 documents per transaction and splitting larger jobs. Use writeConcern: 'majority' when the result must survive a failover.

A group of mongod servers holding the same data: one primary that takes all writes and secondaries that copy its operations. If the primary fails, the members elect a new one automatically, so a replica set gives redundancy and high availability. A typical production set has three data-bearing members; a set can have up to 50 members, at most 7 of them voting.

Members send each other heartbeats every 2 seconds. If the primary does not respond within electionTimeoutMillis (10 seconds by default), a secondary calls an election and becomes primary if a majority of voting members vote for it; members with stale data cannot win. A new primary is usually in place within about 12 seconds. Meanwhile writes fail, and drivers retry a failed write once (retryable writes).

An arbiter votes in elections but stores no data, a cheap way to get an odd number of voters. A majority is required so that two sides of a network split can never both elect a primary: only the side with more than half the votes can. With two data members and an arbiter, losing one data member leaves nobody to acknowledge w: 'majority' writes, so three data-bearing members are preferred.

The operations log: a capped collection (local.oplog.rs) on every member, where the primary records each write as an idempotent operation (an $inc is recorded as the resulting value). Secondaries copy it and replay it. Its size sets the replication window: a secondary that is down longer than the oplog covers must resync from scratch. Change streams read from it too.

Sharding splits one collection's data across several servers (shards) by a shard key, so each holds part of the data; it scales writes and storage horizontally. Replication copies the same data to several servers for availability and extra read capacity. Production clusters use both: every shard is itself a replica set.

Shards hold the data, each one a replica set. mongos routers take queries from the application and send them to the right shards. Config servers, also a replica set, store the cluster metadata: which ranges of the shard key live on which shard. The application connects only to mongos, never to a shard directly.

Look for high cardinality (many distinct values), low frequency (no value in a large share of documents), and a value that does not only increase, because a key such as a timestamp or a default ObjectId sends every insert to the same shard. The key should also appear in most queries, so the router can target one shard. A compound key such as { customerId: 1, orderDate: 1 } often meets all three.

Ranged sharding splits data into contiguous ranges of the shard key, so range queries touch few shards, but an increasing key makes one shard hot. Hashed sharding splits by a hash of the key, which spreads increasing keys such as ObjectIds evenly, but a range query on the key must go to every shard. Use hashed for write-heavy keys you look up by equality, ranged when queries ask for ranges.

Yes, since MongoDB 5.0: reshardCollection rewrites the data under a new key while the cluster stays online, which is slow and heavy on a large collection. MongoDB 4.4 added refineCollectionShardKey, which appends fields to the existing key, a cheaper fix for low cardinality. Since 4.2 you can also update a document's shard key value. Before that, the only way out was to dump and reload the data.

A query without the shard key: mongos cannot tell which shards hold the data, so it sends the query to all of them and merges the results. It works, but the slowest shard sets the latency and every shard does the work. A query that includes the shard key, or a prefix of it, is targeted at one shard or a few. explain lists the shards a query touched.