Industrialize your AI with Cisco Data Intelligence Platform

MLOps with Cisco Data Intelligence Platform is industrializing your AI, similar to assembly line in industrial revolution

Overview

Cisco Data Intelligence Platform (CDIP) is a cloud scale architecture which brings together big data, AI/compute farm, and storage tiers to work together as a single entity while also being able to scale independently to address the IT issues in the modern data center. This architecture allows for:

• Extremely fast data ingest, data engineering done at the data lake.
• AI compute farm allowing for different types of AI frameworks and compute types (GPU, CPU, FPGA) to work on this data for further analytics.
• A storage tier, allowing to gradually retire data which has already been worked on to a storage dense system with a lower $/TB providing a better TCO.
• Seamlessly scale the architecture to thousands of nodes with a single pane of glass management via Cisco Intersight.

Today AI initiatives within enterprises are gaining tremendous traction whether they are in incubation, test, or pilot, or ready for production. However, it comes with complexity of how to quickly, securely, and efficiently move it to production

If we look at traditional DevOps processes, they mostly deal with code changes. Whereas ML projects are dynamic in nature with respect to code, model, and frequent data changes, that require retraining the model, then package it and release it on a continuous basis. Hence, mounting lot of operational challenges for enterprises and affecting the speed of innovation. Machine Learning projects introduced 3Ps. i.e. Personas, Processes, and Platform.

Personas

Data platform are transforming, and different roles or personas has different set of priorities. For example, data scientist is more focused with model development, use of latest and great AI software. They are not much interested in dealing with compatibility and upgrades issues, favors cloud like experience. They are more involved in fine tuning the model, tweaking layers or hyper parameters, and deciding the use of right activation function than how to package, deploy, and monitor it in production environment. As a result, there is a clear and significant gap between pilot and production environment. Further, production environment has other strict requirements such as security, governance, and compliance. Same is true with other personas or roles such data engineer responsible for data pipelines, data analyst, or AI/ML architects.

Organization not having DevOps culture, eventually struggles in catering the needs of these distinct roles resulting operational silos.

Processes

It is imperative that implementation of automated processes is critical for business agility. For example, DevOps which emphasizes culture with collaboration among those roles can eliminate silos, remove duplication of efforts, and delivers value-driven approach. Existing business models are disrupted by more agile and reduced time-to-market methods and become the new source of competitive advantage. Every business is becoming technology savvy resulting DevOps to be a key tool to compete.

Platform

Third component is platform that should align all processes with business outcomes and allow every role that we just talked about to contribute. Cisco Data Intelligence Platform solves it by democratization via common data lake accessible from compute intensive farm along with advanced compute resources such as GPU. With Cloudera Machine Learning(CML) MLOps functionality fully integrated within CDIP compute farm, entire repeatable model training pipeline to model release can be implemented out of the box, giving full end to end machine learning lifecycle management. Not only that, with Cloudera Secure Data Exchange(SDX), we can enable holistic security, governance, and compliance across full data lifecycle. Using CML model cataloging and ML lifecycle lineage, enterprises can unlock their data’s full potential at scale. Enterprises can detect model performance and drift overtime, measure prediction accuracy, fine tune and redeploy with ease. All of it is integrated in CDIP.

Conclusion

Cisco Data Intelligence Platform is a robust platform that lays out the foundation for all the exciting new architectural and technological innovation happening in the data lake world. It sets the stage for Cloudera Data Platform Private Cloud and for all of the upcoming enhancements for increased flexibility. By design, Cisco Data Intelligence Platform is a disaggregated architecture, which makes the big data journey easy and removes complexity out of refresh or upgrade cycles. With Cisco Data Intelligence Platform, each component can not only scale independently but also be refreshed or upgraded.