Drew and Jonathan Scott Get Candid About Parenting Challenges Ahead of Father's Day (Exclusive)

Jonathan Scott and Drew Scott are opening up about fatherhood.

In a snippet of a special Father's Day interview on their website shared with PEOPLE, the Property Brothers stars, 45, talk about some of the challenges they each face in parenting.

Drew and wife Linda Phan, 38, share son Parker, 13 months, while Jonathan co-parents longtime girlfriend Zooey Deschanel's two children — 6-year-old son Charlie and 7-year-old daughter Elsie — with the actress, 43, and ex-husband Jacob Pechenik.

Jonathan says one experience that was a "struggle" was determining what his stepkids would call him.

"They have their dad and they have a great relationship with him, so he’s daddy. I couldn’t say papa because that’s what they call their grandpa — though at one point we tried to change the narrative and get them to call me papa instead, but they weren’t having it," he explains.

Never miss a story — sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human interest stories. 

"Eventually, they came up with 'my Jonathan,' So, whenever they tell a story they say, 'There’s daddy, there’s mommy, and he’s my Jonathan.'"

"One thing that was a struggle early on was that the kids didn't necessarily run up and jump in my arms and give me a big hug," Jonathan recalls. "I also remember several times where they'd be out with other kids and someone would say, 'Oh your daddy's so funny,' and they'd be like, 'That's not my daddy, he's my Jonathan.' So that would hurt a little bit."

Jonathan explains he had to realize "that in their mind, that's not a mean thing to say — it's how they know me."

"Over time, I worked on expressing my feelings to them and the things that make me sad and happy. I think it's good for kids to know that grownups have feelings, too," he continues. "Having that open dialogue with them is so important because as they get older, they'll continue to keep those lines open."

The kids have a much stronger reaction to seeing him these days, with him noting "they scream my name and they come running and want a big hug."

"So, you go through these different phases and adapt, but I think as long as that communication and sincerity is there, they'll know they can trust you and that you have feelings too."

For Drew, the "early stages" of his son's first year were the most challenging.

"I was back in school finishing my Executive MBA and I had to fly out to Boston two months after Parker was born for two weeks," he recalls. "The early stages of being born is a tough time to leave a parent solo."

"[Linda] was good at letting me know what she needed to tackle everything, but after doing that it made me understand how easy it could be to just let Linda take the reins on everything Parker — I'm at work all day and she's with him, so it's easy for me to forget that I'm a parent here too and I have certain responsibilities," he shares.

"The real, open dialogue between Linda and me is a huge part of me being a better dad because she'll give me a slight reminder if I'm not doing something I promised to do. It's not just about the baby, it's also about your communication with your partner."

"Part of being a great dad is being a really great partner," Jonathan agrees, adding, "Doing thoughtful things ... it doesn't matter if it's big or small, but spending time connecting and finding out what their needs are, even if those duties shift back and forth."

"For example, Zooey does all the cooking—she’s an incredible cook and loves food, and I couldn’t even attempt to do one percent of what she can in the kitchen," Jonathan shares. "That being said, sometimes it’s helpful when I can come in and take over the meal duties for one day. Or I always do the cleaning. And even with the kids, too, making sure they take the dishes away."

For more insights on fatherhood from Drew and Jonathan, read the full story at DrewandJonathan.com and check out the brothers on Instagram, where they'll be sharing their favorite dad jokes, gift ideas, family photos, and more.