I think we're not far away from that.
LineageOS et al already show that building AOSP is generally feasible, though it does require at least 64GB of RAM, some terabyte(s) of storage and best lots of processing cores/threads (last time I checked the AOSP docs).
Plus, some vendor kernels are really messy; the Amlogic one for Android TV (in Google's own repos) had a few hundred #if AMLOGIC instances, even in mm/ when I checked about two years ago. 😅
Maybe upstream is complete enough now.