Much progress has been made in the last two years on efficient object detection networks (e.g., YOLO9000, SqueezeDet and MobileNet). In this talk, we will address the unique challenges of autonomous driving applications that go beyond the traditional object detection methods. First, we will introduce a unified network that jointly performs various autonomous driving tasks in real-time on mobile to protect drivers on the road. Then, we will address the challenges that emerge when training a single mobile network for multiple tasks such as object detection, object attributes recognition, classification, and tracking. Next, we will describe a scalable pipeline for continuous training of mobile networks through hard negative mining. Finally, we will go over some of our advanced driver assistance applications that aim to make driving safer worldwide.