Borra caves are located at a distance of above 90-95 km from Vishakhapatnam. Situated at 1400 metres above sea-level, they spread over the Eastern Ghats and occupy an area of 2 sq km. William King George of the Geological Survey of India discovered these caves in the year 1807. Tribals, for whom the caves are religiously significant, inhabit the whole surrounding area.

Deep in the caves, there is a "Shivalingam", over which there is an idol of a cow "Kamdhenu", and the river Gosthani takes its origin from the udder of this cow. The mercury and halogen lamps lit inside, make the interior of the caves colourful and spectacular. There are mica mines in and around the village and it is believed that mining of precious stones like rubies is possible. The natural formations of the caves is unique and breathtakingly beautiful. Due to the pressure of the flow of the Gosthani river, the limestones in the area dissolved and gradually caves were formed. The dissolved stone trickled down drop by drop on the floor of the cave and solidified to form irregularly shaped stalagmites and stalactites. There are several interesting structures like a mushroom formation, a temple, a mosque, a church and others inside the caves.