Rev Ewen Gilchrist, of the Church of Scotland Cults Parish, led the service and encouraged people to write a message on boards around the church, or to leave a note of condolence and support for Bailey's family and hang it on a "prayer tree".
Unlike traditional services, tea, coffee and cake was served and chairs were laid out to encourage people to sit and talk in groups.
Current and former pupils, parents, members of the local Cults community and church elders moved around the room adding their messages of support throughout the vigil.
The church was full to capacity with some people sitting on the carpeted floor as all the seats were taken.
Mr Gilchrist said: "We had no idea what to expect tonight.
"We thought something, something must be offered to bring people together.
"Bailey Gwynne's family will hear about this and the number of people here and hopefully it will give them a bit of light in the darkness.
"We don't want to fill the vigil time with words. We don't tell people what to feel or what to think. But we do want to provide a safe and healing place where people can bring their hurt, their bewilderment, their questions, their sadness and even their anger.
"People are free to move around, to light candles, to write on the message boards for Bailey and add a prayer to the prayer tree."
Messages left included "heal the hole in our hearts" and "you will be missed so badly, Bailey".
One message dedicated to the 16-year-old who had talked of his wish to join the Royal Marines, read simply "soldier on soldier."
Addressing those gathered, Mr Gilchrist said: "How can people be so alive and then not be alive? How can people who so easily fill their space and place like Bailey suddenly be a terrible absence?
"Bailey went to school yesterday and would never come home. Some time soon, some days from now, Bailey's parents will have to bury their son.
"It's not meant to be like that, we know it's not meant to be like that.
"Sometimes life seems so unfair and sometimes life can be so uncertain and fragile and so easily and suddenly lost."
Mr Gilchrist, dressed in a shirt, tie and tweed jacket rather than religious clothing, lit a candle on the altar and invited young people up to do the same.
He told the gathering: "Please talk to one another, help each other, and if you can light a candle."
A piano was played in the background as people moved around the church.
Minister Paul Watson of St Devenick's Episcopal Church led a short period of prayers.
He told Bailey's year group not to be defined by grief but to come together to help each other and be known as the "closest year group ever".
Cults Academy chaplain John Ferguson, whose two sons went to the school, read a poem about love.
He ended by saying: "My hope is that the love in this community will help Bailey's family and get us all through the coming days.
"Bailey in his life knew love and for that we can all be thankful."