The UVF and UDA fear shadowy figures are manipulating young loyalists into violently opposing the Northern Ireland Protocol.
Leaders of fringe groups such as the Orange Volunteers, which carried out sectarian murders and attacks in the late 1990s, have re-emerged in recent months amid unionist anger over the Irish Sea border.
Among their members is a convicted Co Antrim paedophile who has been spreading hate in online loyalist forums.
UVF and UDA leaders accuse these individuals of pipebomb attacks on the East Belfast GAA club last summer.
They are also believed to have been behind failed plans to hold an Ulster Day of Action, which was cancelled because of the Covid pandemic, last September.
A senior UVF source told Sunday Life: "Ex-members of the Orange Volunteers are very active online poisoning the minds of young loyalists and encouraging them to use violence to oppose the Northern Ireland Protocol.
"The question needs to be asked, whose interest does this serve? It definitely does not advance loyalism.
"It is no coincidence that any time there is a crisis in loyalism, these same people emerge, only now they are using social media rather than pamphlets to spread misinformation."
In an interview in yesterday's Times newspaper, the chairman of the Loyalist Communities Council (LCC), David Campbell, touched on how the UVF and UDA were coming under pressure from "radical elements" in their own ranks.
He said: "Young people are absolutely incensed by the protocol. The danger here is that any radical person who wants to start a movement that isn't based on peace and constitutional means of protest, they will have a ready and receptive audience.
"The leaderships of the loyalist groups, particularly since the ceasefires, could well be undone."
Sunday Life understands these internal concerns were, in part, the catalyst for the LCC announcing its temporary withdrawal of support for the Good Friday Agreement on Wednesday.
In a letter to Prime Minister Boris Johnson, the loyalist umbrella group, which includes UDA and UVF members, said that fears over the Northern Ireland Protocol had left it with no other option.
"Withdrawing support for the Good Friday Agreement was a way for the UVF and UDA to keep the hardliners in check," added a UVF source.
"The row over the Irish Sea border leaves some loyalists open to manipulation and we are trying desperately to avoid that."
Under the terms of the protocol agreed after Brexit, goods travelling from Great Britain to Northern Ireland have to cross a regulatory border in the Irish Sea.
Because the European Union has very strict rules on what can enter its market, some products, such as foodstuffs, have to be checked to ensure they meet the necessary standards.
This has required the creation of control posts, which unionists oppose because they say they cut Northern Ireland off from the rest of the UK.
The European Commission is taking legal action against the Government, accusing it of breaching aspects of the protocol. It came after the Government said it would be ignoring rules banning the export of plants between Great Britain and Northern Ireland.
Welcoming Westminster's stance, DUP First Minister Arlene Foster said: "The number of checks occurring between Great Britain and Northern Ireland is so disproportionate to the risk to the EU single market that it has become completely out of step with what the protocol was meant to do."