From A to N in 691 words

This week, I present you with one term from AI and Generative AI, starting with each letter of the alphabet from A to N.

I also mention its definition and end with one interview question for each.

Activation Function: In deep neural networks, an activation function is a mathematical function that determines whether a neuron is activated based on its input.

List various activation functions used in deep neural networks. Which one of the functions creates a probability distribution? 

Bag-of-Words: It is an NLP technique that converts text into numerical data for machine learning algorithms to process. It represents text as an unordered collection of words, disregarding grammar, word order, and syntax, while capturing the frequency of each word’s occurrence. What does the bag of words miss? 

Categorical Variable: A categorical variable is a type of variable that can take on one of a limited, usually fixed number of possible values, which represent qualitative characteristics rather than numerical measurements. List examples of four categorical variables. 

Data Modality: It refers to the different types of data an AI system can process. Different modalities represent distinct forms of information, such as text, images, audio, and video. How would you integrate multiple modalities into a machine learning model? 

Embedding Size: The embedding size is the number of dimensions in the vector space in which words, phrases, or documents are represented, with each dimension capturing distinct semantic or syntactic features of the language. What does an embedding vector length of 1536 floating-point numbers mean? 

Feature Dimension: Feature dimension is the number of individual measurable variables used to describe data points in computational models.

This concept encompasses both the structural complexity of data representation and the computational challenges that arise in high-dimensional feature spaces.

Does a high feature dimension cause underfitting or overfitting in ML models?

Gemma: It is a family of lightweight, open-source language models built from the same research and technology that powers the proprietary Gemini models. Gemma models are multimodal. Which one is the latest model in the Gemma series? What is its context window length?

Hallucination: It occurs when LLMs generate content that appears coherent and grammatically correct but contains factual inaccuracies, fabricated information, or nonsensical elements presented as authoritative facts. How do you evaluate and detect hallucinations in LLM outputs, and what methods or tools can be used to mitigate factual inaccuracies generated by these models?

Image Embedding: It transforms static images into numerical vector representations that capture semantic meaning, allowing multimodal AI systems to perform complex tasks that require both visual and linguistic comprehension. Give me a practical scenario where image embedding is necessary.

Joint Embedding: It is a technique that combines multiple data types or modalities into a single embedding space, allowing semantically similar content across modalities to be mapped to nearby points in that shared latent space. Do multimodal LLMs utilise joint embedding?

Knowledge distillation for LLMs: It is a compression technique in which a smaller language model learns to replicate the behaviour of a larger one, retaining performance while reducing computational requirements. What role does the teacher-student framework play in the Knowledge Distillation process?

LangChain: It is an open-source framework designed to simplify the development of Generative AI applications powered by large language models (LLMs). It offers a standardised interface and modular components, such as prompt chains, retrieval modules, memory management, and agent orchestration. Which other frameworks are alternatives to using LangChain? 

Moving Average: The Moving Average method computes the average of a fixed-size window of past values and “moves” this window along the data sequence. It is used for smoothing volatile data, stabilising training dynamics, and forecasting time series. What role does MA play in ARIMA?

Named Entity Recognition: NER is a technique for identifying, extracting, and classifying key pieces of information, called named entities, from unstructured text. Named entities typically include names of people, organisations, locations, dates, monetary values, percentages, and other specific data points. How would you approach building a Named Entity Recognition (NER) model for a new domain, such as medical or legal texts?

So, you know what’s coming next week. I will cover the letters of the alphabet from O to Z.

END OF ARTICLE