A Mid-Sized Language Model (MLM) is a generative language model is an advanced AI system comprising of maximum 10 billion (miliarden!) parameters, organized into multiple layers with attention mechanisms. These layers process and interpret vast amounts of text data, while the attention mechanisms allow the model to focus on relevant parts of the input. This architecture enables the model to understand and generate human-like language, perform nuanced tasks like answering complex questions, writing detailed texts, and engaging in sophisticated conversations, leveraging its deep learning capabilities.

On a machine amsel.udk.ai (running somewhere in this room), there are many nice Generative AI tools installed, including:

text-generation-webui web-based interface for work with language models

Training PRO extension for training LoRas for Mistral-architecture models

superbooga-v2 for easy RAG prototyping

We define an Extended Educational Environment (EEE or E3) as an immersive and interactive XR learning environment enriched with AI-driven artifacts and avatars. These avatars and artifacts, developed using tools like UnrealEngine and MetaHuman, each possess distinct "personalities" or "characters" reflecting their underlying knowledge bases and machine learning models.

Essentially, You have three options:

fine-tuning (in every training step, training process updates billions and billions of parameters)

training a LoRa (instead of updating billions of parameters, You update just few millions)

do Retrieval Augmented Generation