GenAI & LLM Breadcrumbs from the Front Lines: Deep-Diving into the AI Revolution

Over the weekend, I delved deep into the world of AI/LLMs/GenAI, soaking in insights from some of the brightest minds in the field. The recent VivaTechnology event in Europe showcased AI luminaries like Yann LeCun, Yoshua Bengio, and Elon Musk. In addition, I spent hours listening to and reading about the thoughts of Eric Schmidt, the former CEO of Google, Sam Altman of OpenAI, Satya Nadella of Microsoft, Sal Khan of Khan Academy, Jeff Hinton's methods for selecting PhD students and research topics, and Illya Sutskever, Greg Brockman, and many more. I gleaned a lot, and here are a few concepts I learned, which I hope will serve as breadcrumbs to help you on your own journey of exploration and understanding of this fascinating revolution happening before our very eyes.

TL;DR: The models you use today are the worst they will ever be. We're still in the primitive stone age of model capabilities. The topics and links provided here is just the tip of the iceberg, but even a quick exploration of these concepts will give you a significant advantage in understanding the rapidly evolving AI landscape. You'll be ahead of the curve compared to most people.

AI Models and Techniques

• Frontier Models Forum: Cutting-edge AI models representing the latest advancements in the field.

• Chain of Thought Reasoning: Method where AI processes problems step-by-step, improving its decision-making process.

• Quantization vs. Pruning vs. Distillation

  • Quantization: Reducing the precision of the model’s parameters (e.g., from 32-bit to 8-bit), which decreases the model's size and computational requirements, making it faster and more efficient without significantly sacrificing accuracy.

  • Pruning: Removing less important parts of the model, such as redundant neurons or weights, to streamline the model. This reduces the complexity and size of the model, improving efficiency and sometimes even enhancing performance by eliminating noise.

  • Distillation: Training a smaller, simpler model (the "student") to replicate the behavior of a larger, more complex model (the "teacher"). The student model learns to mimic the teacher model's predictions, achieving comparable performance with reduced size and computational load.

• GPTQ 5-bit Quantization: Technique to reduce the size and computational load of AI models by compressing their parameters.

• Zero Point Quantization: Method to improve model efficiency by scaling and reducing the precision of numerical values.

• The Hessian: Mathematical concept used in optimization algorithms to fine-tune AI models.

AI in Healthcare and Science

• Alphafold Server: Google DeepMind’s tool for predicting protein structures, revolutionizing biological and medical research.

• Alphafold 3: Future iteration of Alphafold aiming to go beyond protein structure prediction to other biological applications.

• Synthetic DNA and Evolution: AI applications in creating synthetic DNA, with comparisons to natural evolution and considerations of ethical and regulatory restrictions.

AI Infrastructure and Hardware

• Nvidia H100: High-performance hardware designed for AI training and inference, known for its powerful computational capabilities.

• Project Groot: Nvidia’s initiative focused on large-scale AI model training. NVIDIA Announces Project GR00T Foundation Model for Humanoid Robots and Major Isaac Robotics Platform Update

• Microsoft Phi3: A project or hardware initiative by Microsoft related to AI.

• CPU, GPU, NPU:

  • CPU (Central Processing Unit): General-purpose processor for various computing tasks.

  • GPU (Graphics Processing Unit): Specialized for parallel processing, ideal for AI and graphics tasks.

  • NPU (Neural Processing Unit): Designed specifically for accelerating AI computations. Pushed heavily by Microsoft

• VASA-1: Lifelike Audio-Driven Talking Faces Generated in Real Time

• Autogen: Tool or framework focused on automating the generation of AI models or content.

• Wifi 7: Next-generation wireless technology that will enhance connectivity, impacting AI and IoT applications.

AI for Software Development

• Codeium: AI tool for code generation and assistance, similar to GitHub Copilot.

• Project Stargate: Microsoft’s project focused on integrating AI with future technologies.

• Open AI Sora: Creating realistic and imaginative video scenes from text instructions. 

• GPT Next: The next iteration of GPT models, promising improved capabilities and performance.

• Future of Agents: Exploration of how AI agents will evolve and impact various fields.

• Microsoft Copilot+PC: Integration of AI copilots with personal computing, featuring:

  • Total Recall: Advanced feature for retrieving and managing data.

  • File Explorer Version Control: AI-driven file management system.

AI Tools and Platforms

• Leonardo AI: AI tool for generating creative content, including art and designs.

• Midjourney: AI platform for creating artwork based on textual descriptions, popular among digital artists.

• Adobe Firefly: Adobe’s suite of AI tools designed to assist creative professionals with tasks like image and video editing.

• DeepFaceLive: Real-time face swapping software used in live video and streaming applications.

• FaceVision: AI software for facial recognition and analysis, often used in security and identity verification.

• SynthFlow: Human-like conversational AI voice assistants

• Unsloth: AI-powered automation tool that simplifies and automates repetitive tasks.

• Langraph: Tool for visualizing and interacting with the outputs of language models.

• AnythingLLM: Versatile platform leveraging large language models for various text generation and comprehension tasks. 

• AgentsOps.ai: Platform for deploying and managing AI agents to automate and optimize complex workflows.

• Ollama Web UI: User interface for interacting with AI models through a web application.

• OpenVoice / Whisper: OpenAI’s tools for speech recognition and generation, enabling voice interaction with AI systems.

• LM Studio: Web-based platform for developing and deploying large language models.

• Textgen WebUI Oobabooga: User-friendly interface for generating text using AI models, designed for accessibility.

• MemGPT: Advanced version of GPT with memory capabilities, allowing it to retain and recall past interactions.

• Devin AI / OpenDevin: An AI development platform (“first AI software Engineer”). OpenDevin - the open-source version of Devin AI.

• Zapier Agents Central: Platform for automating workflows by integrating different applications using AI agents.

AI for Creative Arts

• Google Music LM: AI model for generating music based on user input.

• Google Project Astra: AI project by Google focused on creative applications.

• UPix: AI tool for generating and editing pictures.

• Vidu: AI tool for creating and editing videos.

• UViT (Universal Vision Transformer): Transformer model designed for various vision tasks.

• Diffusion vs. Transformer Models:

  • Diffusion Models: Generate data by iteratively refining noise, often used in image creation.

  • Transformer Models: Good for Language i.e. LLMs. Excel at understanding and generating sequences, like text and images.

Model Training and Optimization

• Llama: Large language model developed by Meta, known for its robust performance in various tasks. Currently at LLama3

• Chameleon Model: Flexible AI model that can adapt to different tasks and environments.

• LoRA (Low-Rank Adaptation): Technique to fine-tune large models efficiently by adapting a small number of parameters.

• Alpaca: Lightweight model derived from Llama, optimized for specific applications.

• QLoRA: Quantized version of LoRA, designed for enhanced efficiency and performance.

AI in Corporate and Industry

• Cisco, Qualcomm, Apple: Major technology companies involved in developing AI hardware and software solutions.

• IBM Open Sources Granite Code Models: IBM’s initiative to release AI models to the open-source community.

• Codenet Dataset: Dataset released by IBM for training AI models on coding tasks.

Security and Threats

• Morris II Gen AI Virus Worm: Hypothetical or historical example of AI-based malware, highlighting security concerns.

Ethical and Social Implications

• AI as an Enabler: We don’t compare human’s ability to run with cars for a reason. AI’s role in augmenting human capabilities rather than replacing them.

• Laws of Training & LLM Scaling Laws: Principle that larger compute resources and more data lead to more accurate models, with no clear limit in sight. BUT …
  [We can’t expect more intelligence “for free” by scaling]

• AI Replacement Myth: The idea that AI will complement human abilities, not replace humans entirely, similar to how cars coexist with human runners.

• AI and Native African Languages: This is a fascinating question near and dear to my heart. Exploring the impact of AI on preserving and enhancing lesser-known languages and promoting linguistic diversity.

Recommended Watch

• Geoffrey Hinton | On working with Ilya, choosing problems, and the power of intuition

• How AI Creates Art - Diffusion, Auto-regression, Creativity and Curve-fitting. 

• The Future Of AI, According To Former Google CEO Eric Schmidt