Explain how a data point is processed in a transformer. Tell about a paper you studied and enjoyed.
Nlp Research Intern Interview Questions
125 nlp research intern interview questions shared by candidates
He: what is IDCNN? Me: it is a CNN based architecture that is used in NER, it is based on dilated convolutions. He: why for sequence labelling use a CNN architecture? Me: because a CNN based architecture is parallelizable so the processing will be faster in contrast to RNN based architecture which is sequential. But I’ve seen in the littérature that generally RNN based+ CRF give better performance. He: why? Me: I’ve not seen formal explanations, all what I ve seen is reported empirical results that show that better performance is given by RNN+ CRFHe: No because RNN catch the long sequence dependencyMe: the IDCNN I just explained catch also long term dependency He: how ? Me: by dilated convolutions as I explained before. He: ok.
He: explain transformer Me: the architecture is an encoder- decoder, in the encoder there are two layers one for self attention and one a feedforward and after each there is a residual connexion + layer normalization. For the encoder, the input is the addition of the embedding of the tokens and their corresponding positional encodings.Then the interviewer interrupted me. He: what is the shape of the input?Me: every token has a shape of the embedding shape and in the original paper it is 512.He: No, it is batch size, sequence length, etc Me: do you want me to explain conceptually or the code ?!!He: ok.
He: I am not understanding what is the difference between image segmentation and object detection? Me: object detection, you re trying to detect objects in an image by having a x,y,width, height and a label for detected objects whereas for image segmentation you want to segment the image as it is the world map catch the borders and give a label for each segment.He: ok. I am not saying that it is bad to ask a question but for a “researcher scientist” you need at least to be aware of tasks in machine and deep learning otherwise it doesn’t hurt to google them before the interview.
He: why we can use lstm and cannot use transformer for long sequences ? Me: With the architecture and formally speaking I am not seeing why you re saying that. Tell me the reason in your opinion. He: because in a transformer we need to fix the size of the sequence and in lstm we can have variable sequence length. Me: but in anyways we need to always fix the length at least batch wise for either by padding or truncating. He: No, because it is a tensor, In transformer it is fixed and in lstm in tensorflow we can vary the size dynamically. Me: as you said it is a tensor that you are giving as input, so for both and at least batch wise it needs to have the same length because it is by definition of the tensor and to alleviate the padding effect, in pytorch for eg you can use the pack padding sequence He: ok.
Technical questions on computing, tell me about yourself.
Given an API endpoint to query, design a caching system in Python.
Describe regularization
Questions about what was introduced in the latest version of Python, Machine learning (overfitting, PCA, neural networks), experience in NLP, experience with REST apis and Django. They also look at your github and ask questions about your projects.
Entity extraction, general ideas to solve
Viewing 61 - 70 interview questions
See Interview Questions for Similar Jobs
Research InternshipGraduate Research InternResearch InternGraduate Research InternshipMachine Learning InternshipMachine Learning InternQuantitative Research InternshipQuantitative Research InternSummer Research InternData Science Internship FallMachine Learning Researcher InternMarket Research InternMarket Research InternshipUser Experience Research InternUser Experience Research InternshipData Science Intern FallApplied ResearchData Science Internship