OpenAI Interview Question

Design a web crawler at scale