Historically, this is true. But I believe over the last decade this has flipped the other way around. I do not have a source though, so don’t take this as truth. I’m actually interested in looking at that data.

Edit: I found this, which is pretty informative. Basically, the last 6 decades, regardless elections more often flipped against the party that holds the White House. So maybe the perception that conservatives get out and vote more often in special elections is actually incorrect. It’s just a weird perception we have. https://centerforpolitics.org/crystalball/the-mini-midterms-five-takeaways-from-six-decades-of-house-special-elections/