It used to be a pirate's cave where the pirates buried their treasure there. The only way to enter it is through a vertical hole tunnel on a rocky outcrop just off the swampland coast of Devil's Bayou. Madame Medusa and her henchman, Mr. Snoops discovered the cave and correctly deduced it to be the burial place of the Devil's Eye, the worlds largest diamond. However, to their disappointment and dismay, the entrance was too small and narrow for any adult to fit through. They then decided to kidnap the orphan Penny to enter the cavern, search for and retrieve the diamond for them.
For three months after her abduction, Mr. Snoops had successfully forced Penny down the "Black Hole" as she calls it, but had been unable to persuade her to search thoroughly, partially because she isn't afraid of him (due to his bumbling personality), her fear of her Teddy Bear getting damaged, but mostly because of a genuine threat of drowning, as the cave being submerged under the ocean, completely fills with water at each high tide.
Medusa eventually learns of this delay and out of anger and frustration over Snoops incompetance in controlling a 'little girl', flew down to Devil's Bayou herself to take charge. She then concocted a threat that would ensure Penny would find the diamond. She would personally see that Penny would enter the cave at the next low tide and then leave her inside until the diamond is found, regardless of the danger. Finally for good measure, she snatches Penny's Teddy as a bargaining chip, threatening to never give him back, unless Penny finds the diamond.
After the diamond is found, Penny is lifted out, but is almost permenantly dropped back in, bucket, rope and all by Mr. Snoops due to his excitement over the diamond.
The cave isnt seen for the rest of the film, but due to the large amount of treasure still in the cave, it could still have been utilized by other treasure hunters.