“And, to be abundantly clear, I am not sure there is enough training data in existence to get these models past the next generation. Even if generative AI companies were able to legally and freely download every single piece of text and visual media from the internet, it doesn’t appear to be enough to train these models, with some model developers potentially turning to model-generated "synthetic” data — a process that could introduce “model collapse,” a form of inbreeding that Jathan Sadowski called “Habsburg AI” that destroys the models over time.“
Personally I think that the AI bubble is going to be the largest tech collapse since the dot com crash.