It was a quick call where they asked about my working experience and expectations.
Sr Data Engineer Interview Questions
2,563 sr data engineer interview questions shared by candidates
More around Java threading , oops concept and cluster related questions
SQL questions involving simple aggregations and joins, but my interviewer got caught up on the intricacies of date formatting (that wasn’t part of the question). Since I code primarily in SparkSQL and PySpark he got bent out of shape about how my answer wasn’t correct. Huge waste of my time. There was also a question about calculating rolling averages where the “solution” was a cross join (an antipattern that almost certainly wouldn’t work in a prod environment) so this gives you a sense of what we’re dealing with. The whole thing felt like it was something dreamed up by someone with 2-3 years of data engineering experience who is fully on the Dunning Kruger curve and thinks they’re a lot smarter than they actually are.
Tell me about yourself in detail
Whats the similarity on map join in hive and broadcast join in spark
Can we run spark without hadoop and yarn.
Tech interview consisted of two blocks. The first one was about reviewing some code written in Python; pointing out potential flaws in the design and reasoning about how to fix them. The code snippet looked like a module that a Data Scientist that lacks understanding of software engineering would write. Second part was about writing a sql query that would filter and join a couple of tables to compute a metric with multiple levels of aggregation.
An example of a project where you worked in with bringing up many dataset on board.
Toughest work that you did?
Got asked which operations in Spark trigger shuffle, as well as when would I develop streaming vs batching pipelines.
Viewing 1341 - 1350 interview questions