Qizheng Zhang (Stanford)- Practical Online Learning of In-Network ML Models via a Generative Labeling Agent
Abstract: Recent work on in-network machine learning (ML) assumes offline models will perform well in modern networks. However, these models often struggle with fluctuating traffic and network conditions, necessitating frequent online validation and updates. In this talk, I will present Caravan, a practical online learning system for in-network ML. Caravan addresses two key challenges: (a) automatic labeling of evolving traffic and (b) efficient model quality monitoring. By repurposing existing systems like heuristics and foundation models, Caravan generates high-quality labeled data and introduces an “accuracy proxy” metric to track and mitigate model drift. As a case study, I will demonstrate how foundation models (e.g. GPT-4) can be adapted for near real-time labeling of high-volume network traffic.
Speakers
Qizheng Zhang
Qizheng Zhang is a 3rd-year PhD student at Stanford University. His recent research focus on scaling AI models, including large language models (LLM) serving, high-speed traffic analysis, and video analytics, while addressing key system challenges such as performance optimization, resource management, and reliability. He is also exploring the integration of LLM agents into large-scale systems.
