Generative AI & LLMs ইন্টারভিউ প্রশ্ন

Generative AI refers to models that create new content, and in the context of LLMs, this content is text. When you give an LLM a prompt, it does not search for a stored answer. Instead, it predicts the next word from context and builds the response step by step. That is why it can write articles, answer questions, or even hold a conversation. The model uses patterns learned during training to generate a response in real time. This is one of the most common Gen AI interview questions in fresher and screening rounds.

LLMs generate responses by predicting one token at a time. First, the input prompt is converted into tokens, which are smaller pieces of text. The model evaluates the prompt and estimates the probability of the next tokens. It selects the token with the highest probability or uses sampling methods to choose. It then repeats the process for the next token until the response is complete. This comes up in Gen AI interview questions to check whether you understand tokens and next token prediction.

Zero-shot learning refers to asking a model to perform a task without providing examples. The model relies entirely on prior training knowledge. Interview line: "Zero-shot learning evaluates a model's ability to generalize to unseen tasks based solely on pre-trained knowledge."

By 2025, generative AI has moved from experiment to production in virtually every major industry. In software development , GitHub Copilot and similar tools write and complete code, with GitHub reporting 55% of developers using AI-assisted coding as of 2024. In pharmaceuticals , companies like Insilico Medicine use generative models to design novel drug molecules, compressing discovery timelines from years to months. In media and entertainment , Runway and Sora generate video content; Adobe Firefly integrates image generation into professional creative workflows. In finance , JPMorgan's LLM Suite assists analysts with research summarization and report drafting. In customer service , AI agents handle tier-1 support at scale, with enterprises like Klarna reporting their AI assistant resolved 2.3 million conversations in its first month.

Prompt engineering | Fine-tuning Changes only the input prompt (instructions, examples, constraints) | Changes the model by training on additional data Faster and cheaper to iterate | Slower and more expensive Best to start with for most use cases | Use when you need consistent performance at scale or for a specific task

RAG is important because it improves output accuracy and timeliness. Many LLMs have a fixed knowledge cutoff, meaning they cannot know events or updates after that date. RAG addresses this by using external documents that can be updated at any time. It also reduces hallucinations by basing its answers on retrieved information. This is useful for customer support, knowledge bases, and internal tools where accurate data is critical.

Latent space is the compressed, lower-dimensional representation a generative model learns to encode its training data. Each coordinate in latent space corresponds to a potential output. A well-formed latent space is continuous and semantically structured, nearby points generate similar outputs, and interpolating between two points produces smooth transitions. In a face generation model, a single latent axis might control age, another controls gender expression, another controls lighting. This structure enables controllable generation: you navigate latent space to specify exactly what you want to generate. Latent space quality is a direct proxy for model quality, a fragmented or discontinuous latent space produces incoherent outputs.

Tokenization is the process of splitting raw text into discrete units (tokens) that a model can numerically represent and process. Modern LLMs use subword tokenization algorithms, Byte-Pair Encoding (BPE, used by GPT), WordPiece (BERT), or SentencePiece (LLaMA), which balance vocabulary size against out-of-vocabulary handling. A single word like "unbelievable" might be split into ["un", "believ", "able"], three tokens, not one. Poor tokenization inflates sequence length, increases context window consumption, and can degrade model performance on specialized domains. Interviewers for AI engineer roles often ask how tokenization choices affect inference cost and latency.

A vector database stores embeddings and enables rapid similarity-based searches. Instead of keyword matching, it performs semantic search. Interview explanation: "A vector database indexes high-dimensional vectors and retrieves semantically similar entries based on distance metrics."

Embeddings are dense, low-dimensional representations of high-dimensional data, serving as a fundamental concept in machine learning to efficiently capture the essence of data entities (such as words, sentences, or images) in a form that computational models can process. Unlike raw input data, which might be sparse and high-dimensional (e.g., one-hot encoded vectors for words), embeddings map these entities to continuous vectors, preserving semantic relationships while significantly reducing dimensionality. This compact representation enables models to perform operations and learn patterns more effectively, capturing similarities and differences in the underlying data. For instance, in natural language processing, word embeddings place semantically similar words closer in the embedding space, facilitating a more nuanced understanding of language by machine learning models.

Fine-tuning involves training a pre-existing model on additional domain-specific data for specific tasks. For example, if you are building a model for legal document summarization, you would fine-tune the model using legal text. Fine-tuning updates the model’s weights to learn new patterns and improve performance on that specific task. It helps the model generate responses that match the required tone, style, or domain knowledge.

Prompt engineering is the process of writing prompts to guide the model to produce the desired response. It involves adding clear instructions, examples, constraints, or specific formats. Because LLMs respond based on how the prompt is structured, even small changes can lead to very different outputs. Prompt engineering is important because it helps you achieve consistent, accurate results without modifying the model.

In real systems, these methods are used together based on the use case.

• Prompting is used first to guide the model’s output format and tone.
• RAG is added when the system needs access to specific or updated information.
• Fine-tuning is used when the model must follow strict style or domain requirements. Together, they help create a system that is accurate, scalable, and easy to maintain.

In an r/learnmachinelearning thread, learners swap GenAI interview prep tips and keep coming back to one point: focus on clear answer structure, not buzzwords. They also note that many interviews quickly shift from definitions to practical system design tradeoffs. Read the full Reddit conversation .

Aspect | Traditional AI | Generative AI Primary goal | Predict or classify | Create new content Output type | Label, score, or numeric value | Text, images, code, audio, etc. Example task | “Spam or not spam?” | “Write a polite follow-up email.” Output variability | Usually consistent for the same input | Can vary based on prompt and context Control | More bounded and predictable | More flexible, harder to contro

Few-shot learning provides sample examples within the prompt to guide output behavior. Interview explanation: "Few-shot learning enhances task performance by supplying contextual examples within the input, enabling the model to mimic demonstrated patterns."

Generative AI is a class of machine learning models that learn statistical patterns from large datasets and generate new content, text, images, audio, or code, that matches those patterns. The three dominant architectures are Transformers, GANs (Generative Adversarial Networks), and diffusion models. Transformers use self-attention to process entire sequences in parallel, making them the backbone of GPT, LLaMA, and Claude. GANs pit a generator against a discriminator in an adversarial training loop. Diffusion models progressively add noise to training data, then learn to reverse that process to produce clean outputs, the approach behind Stable Diffusion and DALL·E 3.

RAG enables models to create responses by accessing external documents and knowledge bases. The system first identifies relevant documents, then uses them to complete the user query. The model generates a response by using the retrieved information. RAG helps the model answer questions based on real data, rather than relying only on what it learned during training.

A large language model is a deep learning model trained on a huge dataset of text. It learns language structure, grammar, and relationships between words and phrases. The model uses a transformer architecture, which helps it understand long-range context. That means it can connect ideas from different parts of a paragraph, not just the immediate words. LLMs are used in applications like chatbots , content generation, and virtual assistants.

Predictive AI answers: "What will happen?" Generative AI answers: "What can be created?" For example:

• Predictive AI forecasts stock trends.
• Generative AI drafts financial analysis reports.

A strong interview framing: "Predictive systems estimate outcomes, whereas generative systems construct new data instances that resemble learned distributions."

• Generative models, such as Variational Autoencoders (VAEs) and Generative Adversarial Networks (GANs), are designed to generate new data samples by understanding and capturing the underlying data distribution. Discriminative models, on the other hand, focus on distinguishing between different classes or categories within the data.