June 5, 10:15am
Location: Santa Clara I

MLOps Automation for the Capture and Analysis of Responsible AI Metrics

co-presented with Bruce Goldfeder

The United States Defense Intelligence Agency Chief Technology Office (DIA/CTO) is piloting Responsible Artificial Intelligence in Machine Learning (AI/ML) by exercising Department of Defense (DoD) and United States policy upon a reference AI/ML architecture.  This architecture – consistent with U.S. values – will be used to facilitate the creation of processes, tools, and training to ensure ethical, reduced bias, and data secure AI/ML that are essential for fostering a positive effect on society. Our team will introduce this architecture to perform natural language classification with the goal of identifying areas to increase trust and acceptance of AI/ML generated outputs in the context of Human-AI teaming. A key element will be to promote explainability of the inferencing of the AI/ML allowing introspection into metrics from data bias analysis, model analysis, and performance metrics as these relate to AI/ML Strategy, Policies, and constraints for the Responsible use of AI/ML. 

Mr. Ramesh Menon was appointed Chief Technology Officer (CTO), Office of the Chief Information Officer (CIO), Defense Intelligence Agency (DIA) on March 1, 2021. Mr. Menon is responsible for technology and strategy planning, experimentation, emerging technology solutions architecture supporting DIA and IC priorities including FVEY partners. Mr. Menon assumed his current position upon his appointment to the SES / DISL-T1. Mr. Menon serves as the CTO and authoritative information technology expert for DIA. He develops a cohesive technological strategy for CIO’s, business, and mission partner’s capabilities on the Joint Worldwide Intelligence Communications System and in the Department of Defense Intelligence Information Systems enterprise. Mr. Menon serves as the senior DIA technologist and interface to the Intelligence Community, Department of Defense, Military Service’s Intelligence Centers, Combatant Commands, international partners, national labs, industry and academia. He conducts feasibility assessments of current and future technology trends within the private and public sectors and makes recommendations on adoption that will increase DIA’s decision advantage. As the CTO, Mr. Menon will serve as the principal advisor for artificial intelligence to the Director and collaborate with other IC and DoD AI leaders. Born in India, Mr. Menon earned a Bachelor’s degree in Electrical Engineering and a Master’s degree in Business Administration (MBA) from California State University. He has co-authored three books and has a patent pending for supply chain assurance using Block chain technology. Mr. Menon was a CTO in the private sector at IBM and a Chief Architect at Johns Hopkins University Applied Physics Laboratory (JHU-APL) leading Artificial Intelligence, cloud, cyber and strategic national security Initiatives. Mr. Menon is a member of IEEE congressional delegation on science and technology and an invited frequent guest lecturer on emerging technologies at Johns Hopkins University. He was a member of Brookings Institution AI expert panel on autonomous weapon systems and served on space innovation council workgroup along with NASA, SMC and DARPA. He is also a mentor to AFRL catalyst campus, NASA Frontier  Development Labs supporting OSINT and EdgeAI to operate in contested spaces