“Transforming Enterprise Intelligence: The Power of Computer Vision and Gen AI at the Edge with OpenVINO,” a Presentation from Intel
•
0 likes•31 views
For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2024/06/transforming-enterprise-intelligence-the-power-of-computer-vision-and-gen-ai-at-the-edge-with-openvino-a-presentation-from-intel/ Leila Sabeti, Americas AI Technical Sales Lead at Intel, presents the “Transforming Enterprise Intelligence: The Power of Computer Vision and Gen AI at the Edge with OpenVINO” tutorial at the May 2024 Embedded Vision Summit. In this talk, Sabeti focuses on the transformative impact of AI at the edge, highlighting the role of the OpenVINO tool kit in streamlining the AI solution life cycle on Intel hardware. This includes the development of energy-efficient computer vision and generative AI models suitable for edge computing. Sabeti showcases cutting-edge AI applications, such as multimodal LLMs for document understanding and YOLO object detection for smart retail solutions. She addresses the entire edge compute ecosystem, discussing how to optimize AI processes from training to inference across various computing platforms, including Intel GPUs. Additionally, she explores how businesses can seamlessly transition between edge and cloud environments and how Intel’s portfolio of solutions unlock the advantages of edge computing, such as data protection and AI acceleration.
Report
Share
More Related Content
Similar to “Transforming Enterprise Intelligence: The Power of Computer Vision and Gen AI at the Edge with OpenVINO,” a Presentation from Intel
Similar to “Transforming Enterprise Intelligence: The Power of Computer Vision and Gen AI at the Edge with OpenVINO,” a Presentation from Intel (20)
More from Edge AI and Vision Alliance
More from Edge AI and Vision Alliance (20)
Recently uploaded
Recently uploaded (20)
“Transforming Enterprise Intelligence: The Power of Computer Vision and Gen AI at the Edge with OpenVINO,” a Presentation from Intel
- 1. Transforming Enterprise Intelligence: The Power of Computer Vision and Gen AI at the Edge with OpenVINO Leila Sabeti Americas AI Technical Sales Lead Intel
- 2. • Edge AI applications • The OpenVINO toolkit: An open-standard for building AI at the edge, in the cloud, or locally • Deep-dive: Enterprise intelligence and Intel®’s portfolio • Flexible edge and cloud computing paradigms • Q&A Agenda 2 © 2024 Intel
- 3. ▪ Real time data processing ▪ Wider reach ▪ Data sovereignty ▪ Cost efficiency Edge IP Camera Depth Camera NVR Sensor Frictionless Retail Traffic Monitoring Defect Detection AI is Everywhere – from Edge to Cloud © 2024 Intel 3
- 4. Enterprise Intelligence at the Edge with Intel® Node Fine-tuning, Inference Cluster Light Training, Tuning, Peak Inf. © 2024 Intel 4
- 5. Medium AI Light AI Heavy AI Efficiency for sub-100 W designs CPU AI, built-in GPU, built-in NPU Scale up perf/W for diverse system designs Discrete GPU & built-in CPU AI Optimize for peak perf and density Optimize with high-end Discrete GPU Choose Your Compute © 2024 Intel Small Gen AI 5
- 6. OpenVINO: An Open Standard for Building AI at the Edge © 2024 Intel Optimized Performance FPGA CPU NPU GPU 6
- 7. Supports myriad AI use cases Wide range of AI performance Real-time and offline execution Flexible video streams Ranges from few to many streams, and TOPS in low- end NVRs to TOPS for on- prem servers Temperature range Ranges from -5 to +105 degree C TDP/Power Ranges from sub 10 W to over 300 W Software Benefits with OpenVINO Hardware Benefits with Intel®’s Edge AI Portfolio Unlocking Software Optimizations with Hardware Using the Intel® Edge AI Portfolio Open-source for AI, DL, Inference Performance-optimized Cross-platform Support OpenVINO Toolkit Value Differentiators © 2024 Intel 7
- 8. Compute for AI: Intel®’s Platforms ▪ Single NN Pipeline ▪ Multi NN Pipeline ▪ Data Fusion Pipeline Seamlessly Integrated into Existing Camera and Video Deployments Intel® Edge AI Box Edge AI Platforms Partner edge platforms using Intel® Arc GPU © 2024 Intel 8
- 9. 9 © 2024 Intel Challenges and Opportunities with AI at the Edge
- 10. Objective: Optimize the queuing process and reduce wait times via object detection. Challenges: • Real-time scalability • Device setup and calibration • Model performance • Low-power Intel’s solution: • Fast and efficient inference with optimized YOLOv8 models using OpenVINO Intelligent Queue Management at the Edge with OpenVINO and YOLOv8 © 2024 Intel 10
- 11. Mobile Multi-modal Assistant with MobileVLM and OpenVINO © 2024 Intel Objective: Use a mobile chatbot to answer questions about images Challenges: • Fast token generation • Memory-efficiency • Model size Intel’s solution: • Compress and quantize LLM models for faster, efficient local inference GPU 11
- 12. Snapshot of MobileVLM Output © 2024 Intel 12
- 13. Document Visual Question Answering with Pix2Struct and OpenVINO on CPU © 2024 Intel Objective: Use a low-power chatbot to answer questions about documents on the fly. Challenges: • Visual understanding • Memory-efficiency • Model size Intel’s solution: • Compress and quantize multi- modal models for faster, efficient local inference Intel® Core Ultra 9 processor CPU 13
- 14. Snapshot of Pix2Struct Output © 2024 Intel 14
- 15. Enterprise Intelligence with LLMs using RAG 15 © 2024 Intel Connect knowledge bases to LLMs with Retrieval Augmented Generation (RAG)
- 16. Running LLM + RAG with OpenVINO and LangChain on iGPU for the edge Enable enterprise intelligence through knowledge-based search © 2024 Intel 16
- 17. Enterprise Data Protection at the Edge 17 Intel® Software Guard Extensions (Intel® SGX) Secure Access Service Edge (SASE) Intel® QuickAssist Technology (Intel® QAT) © 2024 Intel
- 18. 18 © 2024 Intel Edge to Cloud Paradigms
- 19. Edge to Cloud: Flexibly Using Compute 19 Edge Cloud Network Pros Cost efficiency, increased control, etc. Cons Low-power constraints Pros Large amount of data, limitless compute on demand Cons Privacy risks, high latency, etc. © 2024 Intel
- 20. Cloud Edge Edge to Cloud with OpenVINO Model Server Move workloads across the edge and cloud Powered by OpenVINO Runtime © 2024 Intel 20
- 21. Deploying a Quantized Tiny-llama model across client and server LLM Assistants: OpenVINO Model Server with INT8 Compression © 2024 Intel 21
- 22. Intel®’s AI Hardware Portfolio 22 © 2024 Intel Cloud Platforms Edge AI Platforms Client Platforms Desktops Laptops Partner edge platforms using Intel® Arc GPU
- 23. • AI at the edge is transforming enterprise intelligence • But not without several challenges: scalability, setup, AI performance, etc. • At Intel®, we see the full end-to-end stack as key for optimizing AI at the edge, and across the cloud to edge • OpenVINO is an open standard, ready-to-use for building AI and Gen AI • Try It Yourself: openvino.ai Conclusion © 2024 Intel 23
- 24. Resources Resources • openvino.ai • intel.com/edgeai • Demos: intel.com/openvinonotebooks • Enterprise Security Solutions at the Edge with Intel 2024 Embedded Vision Summit May 23rd (12:00 pm – 12:30 pm) “Identifying and Mitigating Bias in AI” May 23rd (1:30 pm – 2:00 pm) “Intel’s Approach to Operationalizing AI in the Manufacturing Sector” 24 © 2024 Intel
- 25. AI: The New Age Solving the World’s Toughest Challenges, Together. Opt-In for Early Access When Registration Opens! Calling All Developers & Technologists! From front-end, web, app devs to back-end, full-stack, database & DevOps to data scientists & researchers, and more: Learn, collaborate, and solve at Intel Innovation – an event for developers by developers. www.intel.com/innovation Save the Date: September 24-25, 2024 San Jose Convention Center, CA Hear from leading industry luminaries, technologists & start-up entrepreneurs in the field of AI. Learn the breadth of future technology advancements in AI through keynotes, sessions, birds of a feathers, and hands-on labs. Get the latest AI development tools, hands-on experience & join on-site Hackathons to optimize your AI code & workflows. Share unique ideas and perspectives and collaborate with your peers.
- 26. Join us at CVPR! Hackster OpenVINO Challenge Ends June 1st https://www.hackster.io/contests/ope nvino2024/ OpenVINO at CVPR Tutorial Date: June 17th https://paularamo.github.io/cvp r-2024/