Jim is visibly upset, but not by much. He's not mad at Hunter, particularly, especially since it was his own idea to try to trigger him that way. He just feels off-kilter still, adjusting back to his usual lingering issues from the traumas in his past instead of insane live-action flashbacks.

He folds his arms across his chest in a sort of subconscious, self-protective gesture. "You're right, I didn't."