You just don’t know those people. Also they don’t all brag about it. I just learned my BIL ownes 5 rentals now and is looking at more. He also has a good day job, but he wants to transition into pandlordong full time. I know many many people like this in So cal as a construction worker.
The issue is the government letting the cheap affordable single family homes (not apartments or condos) go extinct.