Adrien Treuille

Check out our latest post on Clappy.ai, "Leverage Layout-Aware RAG for More Accurate Retrieval of Knowledge from Documents." Discover how to harness the power of Retrieval-Augmented Generation (RAG) for documents in PDF, Doc, and Markdown formats. Dive into the specifics of implementation and explore our reference repository to get started with advanced RAG for your knowledge base. Don't miss out on this essential read! Continue reading here for all the details: The Power of Hierarchical Indexing with RAG. (https://lnkd.in/gbiVHfcH)

2

New Mindset Needed for Large Language Models - https://buff.ly/3XbN4vp #AI #ML #LLMs #genAI

𝐀𝐠𝐞𝐧𝐭𝐢𝐜 𝐖𝐨𝐫𝐤𝐟𝐥𝐨𝐰: 𝐏𝐢𝐨𝐧𝐞𝐞𝐫𝐢𝐧𝐠 𝐭𝐡𝐞 𝐅𝐮𝐭𝐮𝐫𝐞 Recently I happened to see this nice video ( 𝘩𝘵𝘵𝘱𝘴://𝘸𝘸𝘸.𝘺𝘰𝘶𝘵𝘶𝘣𝘦.𝘤𝘰𝘮/𝘸𝘢𝘵𝘤𝘩?𝘷=𝘴𝘢𝘭78𝘈𝘊𝘵𝘎𝘛𝘤 ) by Andrew Ng in which he talks about Agentic workflows. In Language Model (LLM) development, we provide a zero-shot prompt and receive an answer. We may review it and add more prompts to get it to a desired shape. This is exactly what the agentic workflow does. Basically, 𝘢 𝒃𝒖𝒏𝒄𝒉 𝒐𝒇 𝑨𝑰 𝒂𝒈𝒆𝒏𝒕𝒔 𝒘𝒐𝒓𝒌 𝒕𝒐𝒈𝒆𝒕𝒉𝒆𝒓 𝒊𝒏 𝒂𝒏 𝒊𝒕𝒆𝒓𝒂𝒕𝒊𝒗𝒆 𝒘𝒂𝒚 𝒕𝒐 𝒑𝒓𝒐𝒅𝒖𝒄𝒆 𝒂 𝒅𝒆𝒔𝒊𝒓𝒆𝒅 𝒐𝒖𝒕𝒑𝒖𝒕. Moreover, he even claims the output produced by agentic workflow is much better than the output produced single LLM which might sound obvious but what surprised me was, as per the “𝑯𝒖𝒎𝒂𝒏𝑬𝒗𝒂𝒍” coding benchmark, agents-based approach beating plain LLM by quite a margin (see the picture below). The plain 𝐆𝐏𝐓 3.5 was at 48% but 𝑳𝑫𝑩+𝑹𝒆𝒇𝒍𝒆𝒄𝒕𝒊𝒐𝒏 (𝘈 𝘓𝘢𝘳𝘨𝘦 𝘓𝘢𝘯𝘨𝘶𝘢𝘨𝘦 𝘔𝘰𝘥𝘦𝘭 𝘋𝘦𝘣𝘶𝘨𝘨𝘦𝘳 𝘷𝘪𝘢 𝘝𝘦𝘳𝘪𝘧𝘺𝘪𝘯𝘨 𝘙𝘶𝘯𝘵𝘪𝘮𝘦 𝘌𝘹𝘦𝘤𝘶𝘵𝘪𝘰𝘯 𝘚𝘵𝘦𝘱-𝘣𝘺-𝘴𝘵𝘦𝘱) fared at impressive 95.1% with the same GPT 3.5. The plain 𝐆𝐏𝐓 4 was at 67% but 𝑨𝒈𝒆𝒏𝒕𝑪𝒐𝒅𝒆𝒓 (𝘔𝘶𝘭𝘵𝘪-𝘈𝘨𝘦𝘯𝘵-𝘣𝘢𝘴𝘦𝘥 𝘊𝘰𝘥𝘦 𝘎𝘦𝘯𝘦𝘳𝘢𝘵𝘪𝘰𝘯 𝘸𝘪𝘵𝘩 𝘐𝘵𝘦𝘳𝘢𝘵𝘪𝘷𝘦 𝘛𝘦𝘴𝘵𝘪𝘯𝘨 𝘢𝘯𝘥 𝘖𝘱𝘵𝘪𝘮𝘪𝘴𝘢𝘵𝘪𝘰𝘯) fared at whooping 96.3% with the same GPT 4.  You can get more information about the evaluation and the models at 𝘩𝘵𝘵𝘱𝘴://𝘱𝘢𝘱𝘦𝘳𝘴𝘸𝘪𝘵𝘩𝘤𝘰𝘥𝘦.𝘤𝘰𝘮/𝘴𝘰𝘵𝘢/𝘤𝘰𝘥𝘦-𝘨𝘦𝘯𝘦𝘳𝘢𝘵𝘪𝘰𝘯-𝘰𝘯-𝘩𝘶𝘮𝘢𝘯𝘦𝘷𝘢𝘭. He also mentioned the following are the 4 design patterns that are emerging in the agentic workflow. 1.    Reflection 2.    Tool use 3.    Planning 4.    Multi-agent collaboration If you are interested in learning more, instead of me explaining each of these patterns, I would encourage watching the video :) #ai #agent #agenticworkflow #gpt

22

Quantization is a game changer for deploying large language models like Mistral 7B on consumer hardware. This course offers invaluable techniques to harness powerful AI models even with limited resources.

2

#RAG DOES NOT REDUCE HALLUCINATIONS IN #LLMS: Think I've called this out a few times, but the validation is nice - https://lnkd.in/eAi6zxW2. The article makes two well established points: 1) "Hallucination is a problem of reasoning and not relevance! Any amount of relevant text fed through RAG to a machine will retain the original perplexity and entropy in the system to hallucinate independent of the text"; and 2) "The initial challenge arises from the retrieval phase’s reliance on semantic similarity, which is quantified by inner product spaces or cosine similarities. This metric, while capturing semantic closeness, does not differentiate between factually accurate and inaccurate information." This should be clear to everyone, but the industry still shills flimsy RAG scaffolding anyway (shovels to gold miners).

104

34 Comments

Another great paper on In-Context Learning (ICL) for LLMs: https://lnkd.in/e4SP4K5d More and more, we're hearing from builders that ICL is an improvement over fine-tuning for dialing in domain/task specificity. Fine tuning is slow, expensive, and improves specialization at the cost of generalizability. On the other hand, an efficient ICL pipeline is quick and cheap to set up, boosts performance on specialized tasks without sacrificing on general ones, and can be iterated quickly as the project context/use case shifts. If any builders are still choosing fine-tuning over ICL, let's chat. I'd love to learn more!

15

*Mixture of Agents (MoA)* is an innovative way to boost Large Language Models’ performance by leveraging the strengths of multiple agents as a team. Layered Architecture: MoA arranges many LLMs in layered architecture, each layer composed of numerous individual agents(LLMs). Collaboration: These agents generate responses that are based on outputs from the previous layer’s agents. They repeatedly refine and enhance the last output. Diverse Insights: By integrating different capabilities and insights from various models, MoA achieves a more powerful combined model that is secure against diverse risks. Performance Boost: MoA outperforms standalone LLMs by a wide margin. For instance, it set a new benchmark with 65.1% on AlpacaEval 2.0 as against GPT-4 Omni at 57.5%, using only open-source models. In short, if we combine language modelers’ strengths then it results in increased efficiency and versatility. Here is a nice implementation of MOA by Together AI Github source: https://lnkd.in/dMCVRS3D #AI #LLM #LargeLanguageModels #MachineLearning

57

3 Comments

🚀 Unlocking the Power of Text Embedding with Cohere and Hugging Face 📚 Text embedding lies at the heart of the remarkable capabilities of Large Language Models (LLMs). It transforms textual data into numerical vectors, enabling machines to understand and process language effectively. In my latest notebook, we delve into the world of text embedding using Cohere's Embed endpoint and the rich email intent classification dataset from HuggingFace. Join me on this exciting journey as we harness the potential of text embedding to unlock deeper insights and drive innovation in natural language processing. Notebook link: https://lnkd.in/dXbFcFdJ _Love and AI_ 🤍🤖 #TextEmbedding #Cohere #HuggingFace #NLP #AI

1

1 Comment

This new little product demo is awesome! Our new Rev AI Transcript Assistant will save you TONS of time. Easily pull out nuggets of info from your transcripts, making valuable content more accessible than ever. https://lnkd.in/gzzFpcDa #transcription #asr #accessibility #transcript #summarization #ai #assistant

31

2 Comments

OpenPipe Introduces a New Family of ‘Mixture of Agents’ MoA Models Optimized for Generating Synthetic Training Data: Outperform GPT-4 at 1/25th the Cost Quick read: https://lnkd.in/gfxdiDT5 OpenPipe’s MoA models have excelled in rigorous benchmarking tests, achieving notable scores on LMSYS’s Arena Hard Auto and AlpacaEval 2.0. The MoA model scored 84.8 on Arena Hard Auto and 68.4 on AlpacaEval 2.0, indicating its superior performance in generating high-quality synthetic data. These benchmarks are critical as they represent challenging user queries that test the robustness and adaptability of AI models. The MoA model has been benchmarked against various GPT-4 variants in real-world scenarios. Results showed that OpenPipe’s MoA model was preferred over GPT-4 in 59.5% of the tasks evaluated by Claude 3 Opus. This is a significant achievement, highlighting the model’s effectiveness and practical applicability in diverse tasks encountered by OpenPipe’s customers...... OpenPipe

43

3 Comments

This is a great first step in understanding how scaling dynamics extend beyond language to other types of sequential data. Time is a universal axis, understanding how models learn optimally over data collected over time is a crucial step in the broader AI research agenda.

26

Make #GenAI work: The landscape, SLMs vs. #LLMs, cost and more - https://buff.ly/4ddjwTr #AI #ML #IT #technology

3

RAG fine-tuning combines code retrieval & model training, addressing outdated knowledge & hallucinations in LLMs. Our experiments show RAG fine-tuned models using the Together AI Fine-tuning API and Morph Labs Code API achieve up to 16% better accuracy, 3.7x faster speed & 150x cost reduction vs Claude 3 Opus, and up to 19% quality improvement, 1.1x faster speed & 37.5x cost reduction vs GPT-4o. #RAG #Finetuning #TogetherAI #MorphLabs

17

Solitary LLMs are impressive, but several can work together as a Multi-Agent System to astounding effect and overcome solo LLM (GPT-4o, Claude 3, Gemini) limitations. Hear all about it in today's "Five-Minute Friday" episode. In a bit more detail: • In recent weeks, OpenAI's GPT-4o and Google's Project Astra showcased A.I. agents that engage in humanlike conversations and analyze real-time video. • Despite such impressive capabilities, single A.I. agents still struggle with many categories of complex tasks. • Thoughtfully linking individual LLMs together into a Multi-Agent System (MAS) enables them to work together, assign tasks, build upon each other's work, and deliberate to find solutions, in aggregate exceeding the capabilities of a single LLM on its own... often without any decrease in compute efficiency. • As an example, a DARPA experiment demonstrated 3 agents (named Alpha, Bravo, Charlie) collaborating to defuse (virtual!) bombs efficiently. • In an Massachusetts Institute of Technology study, two chatbots solving math problems outperformed a single agent through dialogue and updating each other's answers. • Microsoft developed a software-writing MAS with specialized roles (Commander, Writer and Safeguard; see the image for this post) wrote code 3x faster without sacrificing accuracy. • As always, there are potential risks, e.g.: illogical solutions cascading through the team, agents getting stuck in loops, and malicious exploitation. • Commercial interest in MAS is accelerating, with frameworks like AutoGen and Camel making it straightforward for you to get started with MAS today yourself. What in your industry could be automated or improved by MAS where single-LLM approaches aren’t sufficiently effective? 🤔 For more detail on all of the above, check out today's episode (#788)! The Super Data Science Podcast is available on all major podcasting platforms and a video version is on YouTube. #superdatascience #machinelearning #ai #llms #multiagentsystems

52

4 Comments

There is so much research going into bringing the power of LLMs into the Enterprise context. The main techniques are k-shot learning or prompt engineering, RAG (Retrieval Augmented Generation) & Model Fine tuning (not to mention many variations of these e.g. Agentic RAG, GraphRAG etc.) However, I came across an interesting take on this from IBM and they are calling it "InstructLab" - where the claim is that they are able to teach the LLM "new skills" grounded in an Enterprise context. On the surface it looks like a k-shot learning + fine turning on steroids. Approach seems to be to utilize a small "skills training dataset" in order to generate a large synthetic dataset (perhaps utilizing a specialized LLM) that is then incorporated into the LLM weights natively. This could be a very interesting concept as long as the cost of adding new skills is not very high (when compared to fine tuning). Exciting times! #genAI #enterpriseAI

40

2 Comments

That's a good summary Manny Bernabe - Reflecting on an insightful article from O'Reilly about LLMs, I was struck by this point: "LLMs are also broadly accessible, allowing everyone, not just ML engineers and scientists, to build intelligence into their products. While the barrier to entry for building AI products has been lowered, creating those effective beyond a demo remains a deceptively difficult endeavor." This democratization reminds me of the printing press's impact on biblical texts, which expanded access but also introduced new interpretive challenges. As we navigate these advancements, it's crucial to balance accessibility with responsible use. At Ushur, we've been at the forefront of leveraging LLMs in our backend/knowledge work automation as well as carefully introducing them on the conversational side via our Chatbots and other modes. More will be coming out on that. Read more here: O'Reilly Article #AI #LLM #Technology #Innovation #History #Democratization #ResponsibleAI #Ushur

10

1 Comment

#ObsessivelyEnthusiasticInvestor #UnreasonablyEnthusiasticInvestor #IrrationallyEnthusiasticInvestor at REFASHIOND Ventures: The Industrial Transformation Fund. We look for the people we believe are doing the best work in Industrial Transformation & Supply Chain Technology; Attempting the impossible. Then we go all-in on them. With obsessive and unreasonable enthusiasm. Join us. #IndustrialTransformation #SupplyChain #Innovation #Technology #Startups #VentureCapital #DemandTheImpossible #UnreasonableEnthusiasm #ObsessiveEnthusiasm #IrrationalEnthusiasm 

6

🏅 Large Language Model Leaderboard 🏆 When it comes to evaluating different Large Language Models (LLMs) for your use case, understanding their relative strengths is key. 🏋‍♂️ Here's a concise breakdown: 1️⃣ Multitasking Understanding: Claude 3 Opus from Anthropic and GPT-4 from OpenAI lead in the MMLU benchmark for general knowledge. This makes them strong candidates for projects requiring a wide-ranging understanding. 2️⃣ Arithmetic Reasoning: In arithmetic, Claude 3 Opus scores highest, ideal for tasks that involve math or logical problem-solving. 3️⃣ Coding Proficiency: For coding tasks, Claude 3 Opus and Meta Llama 3 70B show top performance, perfect for software development support. 4️⃣ Context Window: Google's Gemini 1.5 Pro offers the largest context window, able to process extensive data at once, beneficial for complex tasks involving large volumes of text. 5️⃣ Cost-Efficiency: Meta Llama 3 8B stands out for its low input cost per token, offering a budget-friendly option without compromising too much on capability. Choosing the right LLM involves balancing these attributes against your specific requirements. Subscribe to Generative AI with Varun for more insights on LLMs: https://lnkd.in/dNZ4b7ix #GenerativeAI #LLM #OpenAI #Anthropic #Google #Meta

23

3 Comments

📊 phospho first technical paper is out! We are releasing Intent Embed, an intention embedding model for text messages! 🤖💬 Intent Embed is designed to capture and represent user intention within dense embedding vectors. Unlike traditional text embedding models that focus on semantic or syntactic aspects, Intent Embed zeroes in on the underlying user intent, making it a powerful tool for developers and machine learning engineers working on LLM-based applications. Key highlights from our report: ✅ Demonstrated effectiveness in capturing user intent from complex inputs 📈 Superior performance compared to industry-standard models like OpenAI's text-embedding-3-small 🌐 Generates 1536-dimensional vectors, ensuring compatibility with existing vector infrastructure 🛡 Robust to noise, typing errors and adversarial prompts 🔍 Potential use cases include user request classification, out-of-topic exclusion, and user message analysis 🌟 We believe Intent Embed has the potential to improve how developers and ML engineers approach user-centric applications, contributing to more effective and intuitive LLM interactions. 🤫 We already use it extensively for our user intent clustering in the phospho platform. Of course, you can access it via the phospho API (link in the comments). 📰 Read the full technical report below!

145

18 Comments