In my opinion, impossible. At least under the current NCAA climate/setup. If for some reason the NCAA Div. 1 setup changed radically.
Lehigh, Bucknell, Lafayette, and since the 80’s Colgate, are basically joined at the hip. For a multitude of reason too long to list, those four aren’t doing anything unless all four go along. The only thing that could change that is if two or more of the four dropped to Div. 3.