The court ruled against Ashers, run by the McArthur family, for refusing to fulfil an order to make a £36.50 cake with the slogan because it conflicted with their Christian beliefs.
The McArthur family insisted they did not know the sexual orientation of Mr Lee, an LGBT activist, when declining his order.
The family insist their problem was with the cake, not the customer. But Mr Lee claimed it made him feel a lesser person.
However, on Monday morning the Court of Appeal upheld the original decision that Ashers Bakery had "directly discriminated".
In delivering the appeal judgement, Northern Ireland's Lord Chief Justice Sir Declan Morgan rejected the argument that the bakery would be endorsing the slogan by baking the cake.
He said: "The fact that a baker provides a cake for a particular team or portrays witches on a Halloween cake does not indicate any support for either."
Lord Chief Justice said bakers couldn't provide a service only to customers who agreed with their religious beliefs.
In a ruling which lasted 30 minutes, Sir Declan said the original judgment had been correct in finding that "as a matter of law" Ashers had "discriminated against the respondent directly on the grounds of sexual orientation contrary to the Equality Act (Sexual Orientation) Regulations 2006".
The judge also included criticism of the NI Equality Commission.
He said the publicly funded body should also have offered the McArthur family advice during the case, as the bakers believed their rights as people of faith within the commercial sphere were being undermined.
After the ruling, speaking publicly for the first time, LGBT activist Gareth Lee said he was "grateful" for the outcome of the appeal.
He said: "The only thing that I would like to say is I'm relieved and very grateful to the Court of Appeal for the judgment."
The Equality Commission have requested that Ashers Bakery pay their legal costs which have totalled £88,000.
Speaking after the ruling Mr McArthur said they were "extremely disappointed" with the outcome and that they would be taking advice from their lawyers on whether they can further appeal the judgement.
Mr McArthur said Ashers Bakery has served Mr Lee before and would be happy to do so again in future.
He said: "We’re extremely disappointed with today’s ruling. If equality law means people can be punished for politely refusing to support other people’s causes, then equality law needs to change.
"This ruling undermines democratic freedom. It undermines religious freedom. It undermines free speech.
"We had served Mr Lee before and would be happy to serve him again. The judges accepted that we did not know Mr Lee was gay and that was not the reason we declined the order. We have always said it was never about the customer, it was about the message. The court accepted that. But now we are being told we have to promote the message even though it's against our conscience.
"What we refused to do, was to be involved with promoting a political campaign to change marriage law.
"Because we’re Christians we support the current law. And we felt that making this cake would have made us responsible for its message.
"We wouldn’t decorate a cake with a pornographic picture or with swear words. We wouldn’t decorate a cake with a spiteful message about gay people. Because to do so would be to endorse and promote what was said.
"The court said the commission gave the impression it was not interested in assisting the faith community in issues like this. I think a lot of people will agree with that. That’s certainly how we have felt.
"We’ll have to take advice from our lawyers about whether there is a way to appeal this ruling. In the meantime, other businesses will have to take advice about whether they can refuse orders that conflict with their consciences. Or whether they too may be coerced into promoting other people’s views.
"We’d like to thank the huge numbers of people who have been supporting us and praying for us through all this. We’re very grateful to you all.
"It’s been a trying time. But we’re thankful to God for His faithfulness to us through everything."
DUP MP Gregory Campbell called the appeal a "defeat for freedom of expression and freedom of conscience"
Mr Campbell said: “Freedom of expression and freedom of conscience are vital to any democracy and most people support attempts to defend these fundamental rights. Unfortunately, today those rights have been undermined.
"People do not need to share the McArthur family’s Christian faith to support the stance they have taken on this issue. No one needs to share their Christian faith to realise the ruling is a defeat for freedom of expression. As a result of this ruling will other commercial firms be forced to supply produce with political or religious slogans attached with which they profoundly disagree?
"The ruling does not advance tolerance, freedom or a more liberal society. Neither does this do anything to tackle discrimination, which cannot be justified and is rightly outlawed. It moves beyond discrimination, with implications that are not yet known.
"It remains to be seen whether all of those who are celebrating today’s decision would be as content if the ruling is used to enforce a message with which they disagree. There are those who profoundly disagree with the McArthur family’s beliefs, but who have recognised the dangerous precedent that could be set.
"This was a case pursued with some vigour by the Equality Commission yet, as the Lord Chief Justice noted, the other side of this case was ignored. The pursuit of this case by the Commission was not only one-sided but stands in contrast to their foot dragging in other cases such as the shameful McCreesh Park incident.”
TUV leader Jim Allister called the ruling a "black day for justice".
He said: "This is a black day for justice and religious liberty.
“A small family owned business has been dragged before the courts by a publicly funded quango and subjected to two years of torture. While this is a disappointing verdict the McArthur family are to be congratulated for the grace and dignity they have displayed through the process.
“This is an assault on freedom of conscience and as such it should worry Christians and non-Christians alike. The aggressive “gay rights” lobby will see this as the green light to demand that everyone not only accepts but celebrates their lifestyle.
“I trust that every appeal opportunity will be taken”.
Sinn Fein MLA Sean Lynch called it a "victory for common sense and equality".
The party's equality spokesman said: "Today's decision from the Court of Appeal is welcome and makes sense.
"The original decision was the correct one and I'm glad it has been upheld.
"This is the right decision and represents another step forward for equality.
"Equality must be paramount and all forms of discrimination must be challenged."
The Evangelical Alliance called it a "sad day" for the McArthur family and for "freedom of conscience and religion".
Peter Lynas, former barrister and EA Northern Ireland director said: “Ashers have lost the case, but even more importantly we have all lost some our freedom. Forcing someone to promote a view that they fundamentally disagree with is the antithesis of a free and fair society.”
“This was a very complex case about a very simple cake. Today’s outcome is disappointing. Ashers discriminated against an idea, not a person. They would not have made a cake with that slogan for anyone - gay or straight. As they would have treated everyone the same, there can be no discrimination. That is why the majority of people in Northern Ireland support the bakers and will struggle to understand this ruling.”
“While the Chief Justice noted it was important that a ‘chill factor’ was not created, we are concerned that that is exactly what will happen.
“This case shows how far the state can go in forcing someone to act against their fundamental beliefs. We will have to wait and see if there will be further appeals to the Supreme Court. We will also have to review whether the current law is fit for purpose given how it has been interpreted. It now appears not just to protect people, but also particular public messages.”
“The McArthur family have been dragged through the courts for standing up for their beliefs. We hope this ruling will not discourage others from boldly and graciously standing firm in their faith.”
“The court noted that the Commission did not offer any assistance to the McArthurs and that in future the Commission should give advice in such cases. We have long sought a relational approach from the Commission and it is clear that bridges will have to be built with the faith community.”
John O’Doherty, director of The Rainbow Project welcomed the judgement.
He said: "Ashers Baking Company entered into a contractual agreement to make this cake and then changed their mind. While sympathetic as some may be to the position in which the company finds itself; this does not change the facts of the case. The judgement clearly articulated that this is direct discrimination for which there can be no justification
"We do not believe that this matter should have been brought to court. We believe that Ashers bakery should have accepted the Equality Commission’s invitation to engage in mediation, where a remedy could have been found without the expense and division surrounding this court case.
"We once again extend the hand of friendship to all people of faith, churches and families. We would encourage faith leaders to engage with our community to ensure better relations and to develop trust and respect between our overlapping communities for the betterment of our society."
Last year Belfast County Court held that the bakery had discriminated against him on grounds of sexual orientation and religious belief or political opinion. The firm was ordered to pay Mr Lee £500 compensation. At an appeal in May, lawyers for the McArthurs challenged the finding by insisting it would have been sinful for them to complete the order.