It is now being forensically tested to see if it matches the 11 stab wounds inflicted on Ogle by a UVF gang last month.
A PSNI insider said: "It is too early to say whether it was the actual knife used to kill Ian Ogle, but it's a significant enough development.
"What we don't want is the public to assume this was the murder weapon and to maybe see another knife lying somewhere and not to report it."
Referring to the Connswater River knife find, Detective Superintendent Jason Murphy said: "Detectives have recovered a number of items, including a knife, following searches in relation to the murder of Ian Ogle. Enquiries are ongoing to determine if any of these items are connected to this murder."
The river flows through the Sydenham and the Newtownards Road areas into the sea.
It is close to Pitt Place, where Ian Ogle's killers fled in a car after stabbing and beating the leading loyalist outside his Cluan Place home on January 27.
Although five people were involved in the horrific attack on the dad-of-two, only one man has been charged with his murder. Johnny Brown (33) was remanded in custody two weeks ago after returning to Northern Ireland from Thailand, which he had flown out to the day after the killing. He denies any involvement.
Joining him on the trip, but remaining abroad, was close friend Glenn Rainey, who is being treated by police as a suspect who they want to question.
Another leading loyalist who cops want to speak to, Mark Sewell (below), has so far evaded detectives amid unconfirmed reports that he is in Spain.
The Ogle family have accused UVF members of being involved in the murder, a view shared by the PSNI, which says it is a major line of inquiry.
The terror gang has admitted that some of its men were involved.
However, it insists that the killing was unsanctioned at a leadership level and the Newtownards Road commander who okayed it at a local level has been stood down. This individual was the gunman who murdered Jameson Lockhart in 2005 during a UVF/LVF feud. PSNI sources told Sunday Life that although they accept the UVF leadership did not give the order for Ogle to be murdered, senior figures have since been involved in aiding his killers.
One insider said: "Within hours of the murder, a number of suspects fled Northern Ireland, at least two of them remain outside the jurisdiction.
"That required a good deal of planning."
Security sources doubt if the Ogle suspects will remain outside Northern Ireland indefinitely. Another added: "We expect them to come home at some point, probably when their cashflow runs dry."
Ogle (45) was a low-level UVF member who fell out with the gang's Newtownards Road unit after his children, Ryan and Toni, were assaulted during a bar room brawl.
He was forced to pay for the damage to the premises and told that both he and son Ryan were to be victims of a pre-arranged punishment attack.
They refused to attend the meeting at which their arms and legs were to be broken, leading to further threats.
In the hours before his murder, Ian Ogle and Ryan were involved in a clash with a UVF relative on the Beersbridge Road. The paramilitary later summoned a UVF gang who had spent the day drinking and taking drugs in an east Belfast pub.
They showed up at the Ogle family home in Cluan Place and attacked Ian, who was praying with his local pastor in the street, stabbing him 11 times and fracturing his skull.
The cop leading the murder inquiry, DS Murphy, added: "It is important that anyone with information contacts the investigation team."