This is on my mind too, TMH. Factored into my choices. I would have that exact concern about top-tier FBS teams.
My frame of reference: my dad attended the Naval academy in the ’60s, and he tells stories right in line with your concern. While Navy has always taken pride in playing with the big boys, and could usually be competitive for a good chunk of the game, by the end they were just worn down. The difference in size and strength just eventually takes its toll. Late in those games, Navy often lost kids to serious injury. That’s the last thing I’d want to see.