Refuel

“[𝑞𝑢𝑎𝑙𝑖𝑡𝑦 𝑑𝑎𝑡𝑎] 𝑖𝑠 𝑡ℎ𝑒 𝑏𝑖𝑔𝑔𝑒𝑠𝑡 𝑖𝑛ℎ𝑖𝑏𝑖𝑡𝑜𝑟 𝑓𝑜𝑟 𝑐𝑜𝑚𝑝𝑎𝑛𝑖𝑒𝑠 𝑡ℎ𝑎𝑡 ℎ𝑎𝑣𝑒 𝑎𝑙𝑟𝑒𝑎𝑑𝑦 𝑖𝑛𝑣𝑒𝑠𝑡𝑒𝑑 𝑛𝑜𝑤 𝑖𝑛 𝐿𝐿𝑀𝑠, 𝑎𝑟𝑐ℎ𝑖𝑡𝑒𝑐𝑡𝑢𝑟𝑒 𝑎𝑛𝑑 𝑝𝑒𝑜𝑝𝑙𝑒”. This was the quote that stood out the most in CB Insights “Enterprise AI Report” released last week. A few interesting insights and takeaways: 🚀 𝟏. 𝐓𝐡𝐞 𝐩𝐞𝐫𝐟𝐨𝐫𝐦𝐚𝐧𝐜𝐞 𝐠𝐚𝐩 𝐛𝐞𝐭𝐰𝐞𝐞𝐧 𝐨𝐩𝐞𝐧 𝐬𝐨𝐮𝐫𝐜𝐞 𝐚𝐧𝐝 𝐜𝐥𝐨𝐬𝐞𝐝 𝐬𝐨𝐮𝐫𝐜𝐞 𝐢𝐬 𝐜𝐥𝐨𝐬𝐢𝐧𝐠 𝐟𝐚𝐬𝐭: Meta’s open-source Llama-3-70B recently outperformed Anthropic’s Claude-3-Sonnet according to the MMLU benchmark (although Claude-3.5-Sonnet is back to being stronger than the Llama models). As business leaders grapple with financial constraints, they will have to find the sweet spot between performance, cost, and flexibility while considering the ROI of open source models. ⭐️ 𝟐. 𝐁𝐢𝐠𝐠𝐞𝐫 𝐢𝐬𝐧’𝐭 𝐚𝐥𝐰𝐚𝐲𝐬 𝐛𝐞𝐭𝐭𝐞𝐫: Smaller language models (SLMs) built for specific use cases are not only often faster and cheaper, but can also outperform LLMs For example, Microsoft Phi-3 with 7B parameters outperformed ChatGPT 3.5 trained on 20B parameters, as measured by MMLU. And of course, Refuel-LLM-2, our purpose-built model, outperforms GPT-4-Turbo on data labeling, cleaning and enrichment benchmarks. Domain-specific-models are not an opportunity enterprise buyers should shy away from, and should be explored for task specific applications. 📈 𝟑. 𝐏𝐫𝐨𝐩𝐫𝐢𝐞𝐭𝐚𝐫𝐲 𝐚𝐧𝐝 𝐜𝐥𝐞𝐚𝐧 𝐝𝐚𝐭𝐚 𝐚𝐫𝐞 𝐞𝐯𝐞𝐫𝐲𝐭𝐡𝐢𝐧𝐠: Clean data minimizes downstream AI effects and proprietary data drives differentiated business outcomes. As the quote below aptly alludes to, curating quality data and developing the supporting infrastructure will become the lifeblood of product development and the determinant of success in the era of Gen AI. We’re lucky to see this in action every day with our customers and partners — good data strategy, the curiosity and bravery to try task-specific models and focus on ROI — 𝐭𝐡𝐞𝐬𝐞 𝐚𝐫𝐞 𝐭𝐡𝐞 𝐢𝐧𝐠𝐫𝐞𝐝𝐢𝐞𝐧𝐭𝐬 𝐭𝐨 𝐬𝐮𝐜𝐜𝐞𝐬𝐬 𝐰𝐢𝐭𝐡 𝐀𝐈 𝐭𝐨𝐝𝐚𝐲. Which takeaway stood out to you the most?

OpenAI just announced the acquisition of Rockset. Super interesting move — OpenAI already has a few products where retrieval is important (ChatGPT and GPT-builder). A few interesting implications/questions: 1. What does this mean for companies focused on RAG tooling if some of these capabilities are going to be housed much closer to the model layer? 2. Which use cases are going to demand custom retrieval approaches that a one-size fits all from OpenAI won’t satisfy? 3. OpenAI gets a host of new enterprise sources from which they can pull data and make ChatGPT even more effective for them. 4. What is the next suite of products that we might see from OpenAI that build on retrieval capabilities, and not simply the improvement of the model layer? Also interesting to see this non-model-related announcement come through in the same week as Anthropic’s Claude 3.5 Sonnet — which looks like a very strong contender.

The team at Refuel just released their latest #LLM for data labeling, enrichment and cleaning, trained on our Databricks Mosaic AI Training infrastructure! Get the details here: https://lnkd.in/gVcarizZ

Better data = Better AI. In this episode of Software Engineering Daily I dive into why this is true, what makes it hard and how we're solving this at scale at Refuel. Thank you Sean Falconer for hosting and having me on! 🚀

🚀 TeachFX + Refuel: Leveraging Custom LLMs to Enhance Classroom Interactions 🎓 92% Agreement with human experts, in a complex domain ⏱ Reduced AI feature development time from 2 months to 2 weeks 📚 TeachFX, an ed-tech company focused on elevating classroom dialogue, teamed up with Refuel to revolutionize their product with new AI capabilities, enabling the detection of pivotal educational moments in classroom sessions. ✅ Leveraging Refuel's platform, TeachFX achieved a 92% agreement with expert annotators to create training datasets, on a complex, domain-specific task. ⚡ This streamlined the feature development process from two months to just two weeks, enabling a dramatic acceleration of TeachFX’s product roadmap. 💡 This partnership not only exemplifies the power of custom LLMs in enhancing data labeling efficiency and output quality, but also marks a significant stride towards improving educational outcomes. 👉 If you're interested to learn about how custom LLMs are changing the game with respect to data quality, check out the full case study in the comments below. For more insights into leveraging AI for educational excellence, follow TeachFX and Refuel on LinkedIn or sign up for a Refuel demo here: https://lnkd.in/gtKqbXix.

🚀 Retail AI success story for Beni + Refuel: Data normalization with LLMs for product catalog data 🎯 2x Accuracy Improvement 💨 < 1 day of Engineering Effort 📈 245% Increase In GMV for a major partner 👕👖 With a massive catalog of over 200 million items, Beni faced a daunting task: improving the accuracy of their product size attribute from 46% to over 80%. The solution? A partnership with Refuel and our custom LLMs. 🚀 In just one day of effort (compared to the weeks or months such a task would typically require), Beni improved the accuracy to an astounding 87%. This led to a 99% reduction in data quality issues for reseller partners and a 245% increase in Gross Merchandise Value for a major partner. 💡 If you're intrigued by how AI can improve product catalog data and drive significant business impact for marketplaces, check out this case study (click on the comments for the full story) 👉 For more stories like this, follow Refuel on LinkedIn or sign up for Refuel here: https://lnkd.in/gtKqbXix. #datascience #machinelearning #aiinnovation #retailtech #customervalue  #speed  #revenueboost #ai  #ml  #llms

Labeling with Confidence: Confidence estimation is an effective tool to mitigate hallucinations when leveraging LLMs for data labeling and enrichment: If we are able to estimate the model’s inherent confidence in its response, we can automatically reject low confidence labels, chain and ensemble LLMs. Excited to share a bit more about what we've been exploring and building at Refuel in this direction: https://lnkd.in/gyg54vfZ. You can access all of these features in Autolabel (https://lnkd.in/g7dX8Awi) with a one line config change to your labeling task!