Justin Baldoni opened up about his experience filming It Ends With Us and a “near breakdown” moment after filming a scene with Blake Lively.

The Jane the Virgin alum directed and starred in the film based on Colleen Hoover’s 2016 novel and recalled how it was acting and directing simultaneously.

“Directing is a very lonely job, I’ll just be very candid. Because you are kind of at the top of this totem pole,” Baldoni said on the How To Fail with Elizabeth Day podcast. “In your moments of quiet, everybody has a thousand questions for you and also nobody wants to disturb you and you don’t really have many people to talk to and you can’t necessarily share your anxiety or your nervousness about something because you’re also the leader.”

Baldoni noted that it was “a very strange place to be” in “trying to play a character who does the things that Ryle does in the movie,” adding that “there were moments in the filming of this where I would just have to leave.”

The actor played the role of Ryle who starts a romantic relationship with Lively’s Lily and begins to abuse her. Baldoni recalled a scene with Lively in which he had to remove himself due to how emotionally intense it was to film.

“I’d have to remove myself and go shake it out,” he continued. “I mean, I’ve done a lot of somatic therapy so there were times when I was actually just shaking. There’s a moment in the movie where Ryle finds Lily’s phone and he finds a phone number and he’s very jealous and he’s heartbroken and he’s angry and he doesn’t harm her but you can see in his eyes how dangerous he is. After that scene, I had a near breakdown. And I had to leave and just cry and shake because there was so much pain.”

Baldoni said that “what’s hard about a character like that isn’t necessarily what he does. What he does is a result of what he has kept in and so what’s hard about having that in your body is having the trauma live in your body of what he’s experienced or creating that trauma in your body and creating that insecurity and the pain and the feeling that you shouldn’t actually be alive. Your brother should be. It’s all your fault. Nobody really loves you. That was very hard and that took a few months. I had dreams as him for a while, and it lived in my body but I think for the most part, he’s out.”