Manjunath Kudlur

At Corgnit Research Lab, India, we are excited to share our latest findings on the convergence of Reinforcement Learning and Federated Learning and their implications for advancing AGI. Key Insights: - 𝐒𝐜𝐚𝐥𝐚𝐛𝐢𝐥𝐢𝐭𝐲 𝐚𝐧𝐝 𝐄𝐟𝐟𝐢𝐜𝐢𝐞𝐧𝐜𝐲: The distributed nature of FL enables scalable and efficient training of RL models, significantly reducing computational overhead and accelerating convergence. - 𝐏𝐫𝐢𝐯𝐚𝐜𝐲-𝐏𝐫𝐞𝐬𝐞𝐫𝐯𝐢𝐧𝐠 𝐋𝐞𝐚𝐫𝐧𝐢𝐧𝐠: By ensuring that sensitive data remains local, FL facilitates the development of AGI in privacy-sensitve domains, such as healthcare, without compromising data security. - 𝐂𝐨𝐥𝐥𝐚𝐛𝐨𝐫𝐚𝐭𝐢𝐯𝐞 𝐋𝐞𝐚𝐫𝐧𝐢𝐧𝐠: Multiple RL agents can share knowledge through FL, enhancing overall learning efficiency and robustness without the need to share raw data. - 𝐏𝐞𝐫𝐬𝐨𝐧𝐚𝐥𝐢𝐳𝐞𝐝 𝐀𝐝𝐚𝐩𝐭𝐚𝐭𝐢𝐨𝐧: The integration of FL allows AGI models to be tailored to individual users or environments, combining global intelligence with local customization. - 𝐑𝐨𝐛𝐮𝐬𝐭𝐧𝐞𝐬𝐬 𝐚𝐧𝐝 𝐑𝐞𝐬𝐢𝐥𝐢𝐞𝐧𝐜𝐞: Aggregating diverse experiences through FL results in more generalized and resilient AGI models, mitigating the risk of overfitting. Benefits for Researchers: - 𝐑𝐞𝐬𝐨𝐮𝐫𝐜𝐞 𝐎𝐩𝐭𝐢𝐦𝐢𝐳𝐚𝐭𝐢𝐨𝐧: Leveraging distributed computational resources lowers costs and enhances the feasibility of large-scale AGI experiments. - 𝐄𝐧𝐡𝐚𝐧𝐜𝐞𝐝 𝐂𝐨𝐥𝐥𝐚𝐛𝐨𝐫𝐚𝐭𝐢𝐨𝐧: Model updates can be shared instead of raw data, promoting cross-institutional and interdisciplinary research. - 𝐀𝐜𝐜𝐞𝐥𝐞𝐫𝐚𝐭𝐞𝐝 𝐃𝐞𝐯𝐞𝐥𝐨𝐩𝐦𝐞𝐧𝐭: Faster training times and rapid iteration lead to quicker advancements in AGI. - 𝐄𝐭𝐡𝐢𝐜𝐚𝐥 𝐀𝐈 𝐃𝐞𝐯𝐞𝐥𝐨𝐩𝐦𝐞𝐧𝐭: Prioritizing user privacy and data security aligns with ethical standards and regulations, fostering responsible AI research. - 𝐑𝐞𝐚𝐥-𝐖𝐨𝐫𝐥𝐝 𝐀𝐩𝐩𝐥𝐢𝐜𝐚𝐭𝐢𝐨𝐧𝐬: Immediate feedback and improvements from real-world data enable practical deployment in sectors like smart cities, healthcare, and IoT. Our research at Corgnit underscores the transformative potential of merging RL and FL in the pursuit of AGI. We believe these insights will catalyze further innovations and collaborations in the AI community. #ArtificialGeneralIntelligence #ReinforcementLearning #FederatedLearning #AIResearch #CorgnitResearchLab #PrivacyPreservingAI #ScalableAI #EthicalAI Corgnit, Corgnit Research India, Reinforcement Learning, Federated Learning Consortium

20

Large Language Models (LLMs) are absolute powerhouses when it comes to generating top-notch #GenAI responses. Yet, #LLMs lack that in-depth, domain-specific knowledge that can make all the difference in certain use cases. Additionally, let's not forget the hefty computing resources and expertise required to develop and deploy them. In my latest article, I delve into why Small Language Models (SLMs) are becoming the go-to choice for specific tasks. With their more focused approach, #SLMs are more efficient than LLMs, and have a smaller compute footprint. Thus, they are more suitable to run locally on workstations or on-prem servers. Check out my article 👇 and ‘d love to hear your thoughts about it https://lnkd.in/diRYaNXF #DellTechEMEA #iWork4Dell #AMD, #Intel, #NVIDIA 

143

7 Comments

AI workflows are driving demand for memory and storage to prepare data, train the model and create useful inference engines.  This is driving announcements of storage and memory products that support the various elements of these AI workflows.

51

4 Comments

Widespread deployment of AI may be in trouble. John Loeffler, writing on the web site Techradar (https://lnkd.in/gDHWKEjU) published one of the more thoughtful assessments of Nvidia CEO Jensen Huang's keynote speech June 3 at Taipei's Computex 2024. But I think John need not be overly concerned about the speed of deployment of AI. I attended Computex this year and the hype surrounding the coming of AI seems significantly out of touch with a simple reality: there's not enough electrical generating or grid transmission capacity to run all the proposed deployments of new data centers, or "AI factories". Virtually lost among all the hype over Nvidia's new Blackwell GPU is the massive amount of power required to run it. Can someone who was there help fact check this for me? Marc Hamilton from Nvidia while showing an example of a dual Blackwell server blade said with a straight face that a server rack of 72 Blackwell processors required 100KW, along with liquid cooling. I thought I heard wrong at first but then started doing some back of the envelope math. If every Blackwell GPU was running at full tilt in a rack the peak energy consumption needed for the rack plus cooling system just might approach 100KW. Now how many racks will be in a "AI factory"? It will need its own multi-megawatt power plant out back just to run the place. Some other fun facts about data center power consumption today: 30% of the electricity generated in Ireland goes to run data centers there. In 2022 data centers worldwide consumed 460 TeraWatt/hrs. By 2026 data centers at current deployment rates will need 1050 TeraWatt/hrs annually. That's more energy than the entire country of Japan uses today. Where is all this power going to come from? Jensen Huang has proposed building a large AI factory here in Taiwan, where we barely have enough energy in the summer to run all our air conditioners. How is that going to work?

7

4 Comments

Thought provoking research paper- Enhancing the ubiquitous FPGA BRAMs with in-memory compute-capabilities. Specifically note, the "proposed architectural change results in 1.6× and 2.3× increase in the peak multiply-accumulate throughput of a large Stratix 10 FPGA, at a minimal cost of only 1.8% increase in the FPGA die size and no change to the BRAM's interface to the programmable routing." In practical terms, this work proposes enhancing the capabilities of FPGA (Field-Programmable Gate Array) on-chip memory, specifically BRAMs (Block RAMs), by enabling them to perform computational tasks in addition to storing data. This enhancement aims to significantly increase the efficiency and performance of data-intensive applications, such as deep learning inference. For example, in the field of deep learning, the proposed architecture, RIMA, could enable faster and more efficient execution of inference tasks on FPGA devices compared to traditional methods or even state-of-the-art solutions like GPU-based processing. This advancement could have significant implications across various industries where deep learning inference is applied, such as healthcare for medical imaging analysis, autonomous vehicles for real-time object detection, or finance for fraud detection in large datasets. Essentially, it allows for faster and more efficient processing of complex tasks, which can lead to improved performance and potentially new capabilities in a wide range of verticals.

1

Level up Chip Design Innovation with MATLAB and Silicon Catalyst! #MathWorks News & Stories brings you an exciting collaboration between MathWorks and Silicon Catalyst. This article explores how MATLAB an Simulink empowers chip-based semiconductor startups to develop cutting-edge solutions in medicine and wireless technology. Key Takeaways: >> Richard Curtin, Managing Partner at Silicon Catalyst, highlights the growing demand for "specialized design tools to overcome the complexities of modern chip design." >> Raphael Guimond, Antenna Designer at SPARK Microsystems, emphasizes how MATLAB's capabilities streamline the antenna design process, allowing engineers to "focus on innovation rather than repetitive tasks." ➡️ Dive deeper and discover how MATLAB can: >> Shorten design cycles through its powerful simulation and modeling tools. ⏱️➡️ >> Optimize performance with efficient algorithms and code generation capabilities. >> Facilitate collaboration with a robust platform for data sharing and version control. Don't miss out! Read the full story here: https://lnkd.in/eAv7JhYV #engineering #semiconductor #wireless #innovation #MATLAB #medicine #Simulink #Probius #SPARK #Microsystems #SiliconCatalyst

4

Checkout our latest report on how just eight SN40L chips run 1 Trillion parameter model at blazing fast speed https://lnkd.in/grwdFVAi. It was great to collaborate with some of the most talented folks in this area. Here is the link to the demo operating on similar principles https://fast.snova.ai/

28

I'll be presenting two talks at SID 2024 in San Jose on the topic of digital chemistry for display technology. Let's connect and discuss the exciting developments and opportunities. #DigitalChemistry #DisplayTechnology #Innovation #machinelearning #materialsscience #oled #optoelectronics #datascience #vr #ar #optics #displays #microled #lcd

36

12 techniques to reduce your LLM API bill and launch blazingly fast products Cost-efficiency and minimal latency are critical for deploying production-ready LLM applications.   Sahar Mor's article sheds light on 12 innovative techniques to slash your LLM API bills, with gems like Semantic Caching and Summarizing Conversations leading the charge. This isn't just about saving tokens; it's a strategic playbook for scaling smartly in an AI-dominated landscape.  #MachineLearning #AI #NLP #CostOptimization #Innovation #TechTrends https://lnkd.in/ec3qst-M

19

1 Comment

Chip Level Performance and Security Intel Corporation decision to phase out Hyper-Threading in their upcoming 15th Gen Arrow Lake processors indeed marks a significant performance and security shift. This smart move is primarily driven by Intel's findings that traditional Hyper-Threading didn't deliver the expected performance improvements with their new architectures. Instead, they are exploring new approaches like "Rentable Units" for future generations, such as the 16th Gen Panther Lake. Chip level Security concerns have also played a crucial role in this decision. Hyper-Threading has been associated with vulnerabilities like MDS attacks (Microarchitectural Data Sampling), which include exploits like ZombieLoad, RIDL, Meltdown and Spectrem, and Fallout, etc.. These vulnerabilities arise from the sharing of resources between threads on the same CPU core, potentially exposing systems to data inference attacks. The proactive steps by operating systems like OpenBSD to disable Hyper-Threading highlight the seriousness of these security risks. Despite potential performance benefits, the decision underscores Intel's commitment to enhancing both performance and security in their processors moving forward. We look forward to seeing better innovations and technology from Intel as they lead in this new era of CPU innovation and generation. Resource: Hardware Times: https://lnkd.in/gCvB9_Sr Hyper-Threading and the Processor Pipeline: https://lnkd.in/gGeTirPt

36

When you focus on synthesizable-only Verilog like I do, it's easy to forget that you can do arbitrary computations during elaboration to compute constants for synthesis. Here's an example: a helper function to compute the Greatest Common Divisor of two integers, which is a useful building block for larger calculations. It needs the absolute value helper function, but that's trivial. https://lnkd.in/gvq7a6pG

29

I appreciate the insight from Prof. Shashua to highlight the pos and cons of E2E and abstraction. I think that Tesla took the route of Data Center + Data Engine + E2E on the edge while minimizing engineers' intervention. It is not necessarily the best route for the other competitors. In other market, engineers' intervention in the design could be greater or better than the data engine at that time.

2

This is also the XPU using scenario In my mind too. “Whisper on NPU, LLMs on iGPU, document embeddings on CPU! Full use of #AIPC through #OpenVINO to implement scenarios like chat with documents and voice guided assistant that generates code. Everything on our new Lunar Lake platform that was just announced.”

9

One of the best things about being in San Francisco 🌉 is the ability to drop in on great talks. Last week I had a chance to see NVIDIA's CEO Jensen Huang (JH) being interviewed by Stripe's CEO Patrick Collison. Jensen is among the longest-serving founder-CEOs in Silicon Valley at the moment (30+ years). With that comes a special clarity of mind and some hot takes. 🌶👇 🗣 Jensen doesn't believe in 'praise in public and criticism in private'. Instead, he reasons that most information in the organization should not be privy to certain people. By giving feedback in public, you allow everyone to learn. 'What is better than learning from your own mistakes? It's learning from someone else's mistakes!!' 🔥 JH also doesn't like to fire people. He would rather "torture them into greatness". He invests in people and he doesn't like to write off his investments. 👨‍💻 Asked whether he likes his job every day, he answers "no". The perfect job doesn't give you happiness 100% of the time. "No great things were done, that were easy all the way." ... "Even if I come home from a day where I did things I didn't enjoy, it could still be have been a great day." 👩‍👧‍👦 Jensen famously has 60 direct reports. "This allowed us to cut out probably 7 levels of management". His subordinates don't necessarily have such large teams. "It doesn't scale down, less senior people need more guidance". Looking at the numbers, this approach allowed NVidia to reach ~2T market cap with ~28.000 people vs. Microsoft's 3T with ~220.000 people and Apple's 2.6T with 160.000 people. ❔ Jensen answers almost every question in the same way: "Let's break it down, let's reason about it". This is a great response on so many levels. First, it allows him to not be put on the spot. Second, it allows him to teach his workforce how he reasons through a problem so that they can emulate it and create alignment. Third, it gives people a chance to point out flaws in his reasoning. He admits he is not right all the time. ON AI 👨‍🔧 "You will not lose your job to AI, your company will go out of business because another company used AI." "If you're not using AI you're doing it wrong." ⏱ The extent to which AI will enter many processes is largely underestimated. Right now we are focused on language - sounds in a time sequence. "Any time sequence can be tokenized." This will include robotic movement. 🥛 JH also expects that we won't be satisfied with foundation models "You're going to want to tune it into perfection because you care so deeply. Say if it's already at 99% you'll want to get it to say 99.3%" This last piece is also why we created Query Vary (YC W22). AI will never feel truly intelligent until it completely understands your data and context. It needs to be taught, regardless of how powerful the foundation model is. Finally, besides insights, it was remarkable how funny Jensen was, cracking a joke at roughly a 1 per 3 min rate. It was a very entertaining talk. #ai #sf

61

8 Comments

As part of POLITICO’s “Future in Five Questions” series, Sr. Director, Product Management for AI & ML Dr. Vinesh Sukumar discussed Qualcomm’s AI leadership, his passion for machine learning, and how anyone — even his own mother — can use #AI tools. Take a look:

87

3 Comments

At 16:00 tomorrow at TPC, don't miss a panel with experts like Tobias Becker, VP of R&D at Groq, to hear how systems can be designed for the training and inference of the largest next generation models. Explore hardware/software approaches designed specifically to tackle the challenge of scaling.

8

We put extra energy into creating this blog post. Single-View 3D Tracking (SV3DT) is a simple yet effective solution for mitigating partial-occlusion problems in real IVA use cases. A GitHub sample allows users to try this with NVIDIA DeepStream SDK quickly. https://lnkd.in/gSy5B5Xs

21

1 Comment