Bajaj Data Engineer Interview Preparation

The guarantee has a defined boundary. Coordinating Kafka processing and Kafka output is different from committing an unrelated database or API side effect. Design an idempotent sink or transactional coordination at the external boundary, and document what happens after a crash between those operations.

Check your understanding: Crash after the sink write but before progress is committed, then replay.

Common trap: Extending a broker guarantee to every downstream service.

Concept reference: Apache Kafka: design and delivery semantics

Keep stable event identity, transformation version and a deliberate output key. Make replayed writes replace or deduplicate the intended record, and isolate destructive backfills from live traffic. Validate counts and representative records before promoting the rebuilt dataset.

Check your understanding: Replay the same batch twice and compare output cardinality and values.

Common trap: Appending every replay as new business activity.

Concept reference: Apache Kafka: design and delivery semantics

Define the atomic unit: the entire batch or an idempotent record. Store progress only after its corresponding durable write, and use uniqueness or conditional updates to make reprocessing safe. Surface rejected rows separately rather than advancing a checkpoint over uncommitted data.

Check your understanding: Terminate the importer mid-batch and reconcile source and destination records.

Common trap: Using an in-memory row counter as durable progress.

Concept reference: PostgreSQL: transaction isolation

First, identify technical constraints and define measurable service objectives. Next, trace data from producer contract through transformation to a verified consumer result. Contrast architectural trade-offs across simplicity, correctness, maintainability and scale, explicitly mitigate the risk of logs show symptoms but not the triggering request path, and confirm system stability using a trace, a minimal reproduction and a regression test.

Common trap: Reaching for a specific library or framework before defining constraints, failure envelopes, and automated verification criteria.

Prioritise the failure mode exhibiting the highest user blast radius and lowest observability. Formulate an explicit containment boundary, implement idempotent retries with jitter, and establish an automated rollback threshold. Verify resilience through semantic validation results, quarantine records and a replay verification.

Common trap: Relying on passive monitoring dashboards without defining explicit error-budget alerts, rollback triggers, and verified recovery procedures.

Structure the response using Context-Decision-Tradeoff-Result: articulate the business and technical constraints, compare viable alternatives, explain the implementation (trace data from producer contract through transformation to a verified consumer result), and document the accepted trade-off. Provide concrete proof: a project example, measured result and repeatable verification step.

Common trap: Speaking only in high-level abstractions or team accomplishments without detailing your direct implementation decisions, trade-offs, and measured results.

Formulate a falsifiable hypothesis from observable telemetry before altering configurations. Then inspect lineage, event time, partition skew, query plans and replay checkpoints. Isolate the defect to the smallest reproducible boundary, validate root cause with evidence, and confirm full resolution using shadow-traffic comparison and an error-budget based cutover rule.

Common trap: Applying speculative fixes or restarting services blindly without establishing an observable signal connected to a falsifiable hypothesis.

Produce a schema-compatibility test with lineage evidence and an idempotent replay result. Document baseline assumptions, technical mechanism (trace data from producer contract through transformation to a verified consumer result), rejected alternatives, bounded failure envelopes, and deterministic pass criteria. Supply reproducible verification via load-test results, retry counts and a measured cost estimate.

Common trap: Presenting architecture diagrams or slides lacking automated unit/integration tests, observable metrics, or automated rollback configurations.