Because you don't want to talk about it. I don't really get it, but it has to be important to you.

... Also you get nervous. You're not a good liar.

[It's a good thing Shouto doesn't want to push too hard about this. Because he feels like if he pressed for long enough Midoriya might give information he didn't want to.]