The way it’s been explained to me by prominent older actresses (like Meryl Streep) and in articles regarding same, most of the producers and studio heads were older white males who in their fantasies saw themselves with younger women. Perhaps, too, the screenplay reflected this ridiculous age disparity, but again most screen writers in the past were males also. Those movies gave them a chance to sort of live out their fantasies regarding younger women- vicariously.
Yes it begs credulity, and it’s pathetic in my opinion because you do not see this in the real world that often. Yes, there are women who have Daddy issues that really are drawn to older men, but I would say they’re in the minority!
Perhaps one of the most ridiculous of all folks: