One year of Trump is all it took. https://www.nytimes.com/2018/04/11/us/politics/paul-ryan-speaker.html
I realize "he wanted to spend more time with his family" is usually a euphemism, but judging by his longstanding statements about the time pressures of Congress, in Ryan's case it's actually probably true, especially since he's in a safe seat.