Sunday’s ‘Sister Wives’ episode saw Robyn and Kody keeping their distance as they spoke with Meri about her Utah move, reminding Kody of a PDA rule he previously established with all four wives

Robyn Brown’s continued advocacy for Kody Brown’s reconciliation with estranged wife Meri Brown is beginning to irk him.

On Sunday’s episode of Sister Wives, Robyn and Kody met up with Meri to discuss her plans to move her LuLaRoe clothing business to Utah, where she will be able to work on that and her bed and breakfast in one place. The move would require her to spend larger chunks of time in Utah, though she still planned to keep a smaller home in Flagstaff, Arizona.

As Meri predicted, Kody didn’t seem to care about the shakeup. He, in fact, was “still confused” by her decision to remain in Arizona as their marriage is non-existent.

“We talked about this before and she’s like, ‘Kody’s not going to care’ and ‘He’s going to be good with it’ and stuff,” Robyn, 44, then said in a confessional. “And I’m like, ‘Please, say the right thing. Please, please give her something to hang on to.'”

 From Kody’s perspective, he sees Meri’s Utah B&B as her “safe and happy place” and believes it’s a “good thing.” But he wasn’t sure if he should “be reading between the lines.”

“I can’t tell if she’s saying, ‘Hey, I’m going to move my business to the B&B’ so I go, ‘No, no, Meri, please don’t, stay here,'” he told the cameras. “I don’t see her that often. But I’m not trying to act like we’re getting back into this marriage full fledge. I’m not trying to do that. I think I see the writing on the wall. I’m literally waiting for her to catch up.”

Even though Kody, 54, has insisted he no longer wants to be in a relationship, Robyn still wondered if there was “something there” between the estranged pair because Kody recalled something related to Meri’s business that occurred many years back.

Robyn then suggested that she and Kody make sure they’re “scheduling” time with Meri, 52, as she begins to spend more time away. But that suggestion didn’t sit well with the Brown family patriarch.

“Robyn and Mary are still talking about this relationship that the three of us have like we’re a family,” he said to the cameras. “And what I’m gathering from what Robyn is saying is like, ‘Well, you got to be around because this is your wife.’ And I’m like, I just don’t need that kind of pressure from you.”

As for how Robyn felt, she said in a confessional: “I just feel like there’s just so little left and I’m like trying to show [Meri] that there’s still some hope and Kody is not helping me with that. It’s very frustrating. And some days, it feels like a deal breaker.”

At one point, Robyn pushed Meri to reiterate to Kody that her move didn’t mean she was leaving the family or him. But from Kody’s view, he told the cameras that her decision “actually has no effect on me.”

“I’m not complaining about her B&B, I’m just questioning what she’s doing,” he added. “What are you doing, Meri? In my head, I’m really wondering why she lives in Flagstaff.”

Meri, however, expressed that she still has “hope” about her future with Kody as she declared her interest in permanently remaining in Flagstaff. But this conversation left her with less hope, saying in a confessional, “Conversations like this basically just tell me, ‘Go ahead, Meri, and just live your life on your own like you’ve been doing because nothing else is going to change.'”

During the trio’s entire conversation, Robyn stood beside Meri as Kody sat on the opposite side of Meri’s kitchen island. Kody noted to the cameras that Robyn tended to “keep her distance” and didn’t show affection when Meri “or any one of the wives” were around them.

Although Meri noted that it wouldn’t “bother” her if they did, the trio, alongside Kody’s now-ex-wives Janelle Brown and Christine Brown, previously made a rule choosing to be “a little bit more careful and a little bit more reserved” instead of showing PDA in front of each other.

But the distance between Robyn and Kody as she continued to advocate for Meri made him “very uncomfortable.”

“What I’m seeing here is Robyn being an advocate of me and Meri in our reconciliation, in our marriage, and they’re making me feel very uncomfortable,” he said in a solo interview. “I got one wife that I’ve got a great relationship with and I’m afraid if I abandon these women that she’ll lose respect for me.”

When it comes to his marriage with Meri, Kody felt “the writing is on the wall. It’s been obvious to me. When I try to go into that discussion with Meri, I get so much resistance. It’s not really fair to either of us. But as long as she wants to live in this denial, I guess I can live in it with her.”

Despite this, Robyn still aimed to serve as a source of encouragement for Meri to keep going. “She can really be in this, like, dark place, and she’ll go, ‘Like, why am I trying?'” Robyn recalled to the cameras. “I said, ‘Meri, if you don’t want to try anymore, then like, I want you to be happy. And if this is like, you don’t want to do this anymore and you don’t see that there’s any hope.’  I went, ‘I don’t want to be selfish and keep you here.'”

Robyn added, “And she said, ‘Don’t ever say that to me again.’ And she says, ‘I need you to always tell me there’s hope.’ She says, ‘I need you to always be fighting for our relationship because there’s days when I am too weak to continue to fight. I need someone else to fight.’ And I said, ‘Okay.'”

Never miss a story — sign up for PEOPLE’s free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human interest stories.