Turn to your right at spawn and you will see a big ocean opening up. Swim or fly out into the ocean and you will soon see the extreme mountain islands on your left. In total there are two islands. One of the islands is made only out of rock while the other one got some patches of grass.
If you walk through the waterfall as seen in the image below you will enter into a cave which leads down quite a bit down underground.