The Internet of Things (IoT) is a transformative technology marking the beginning of a new era where physical and digital worlds are integrated by connecting a plethora of uniquely identifiable smart objects. Although the Internet of terrestrial things (IoTT) has been at the center of our IoT perception, it has been recently extended to different environments, such as the Internet of underWater things (IoWT), the Internet of Biomedical things (IoBT), and Internet of underGround things (IoGT). Even though radio frequency (RF) based wireless networks are regarded as the default means of connectivity, they are not always the best option due to the limited spectrum, interference limitations caused by the ever-increasing number of devices, and severe propagation loss in transmission mediums other than air. As a remedy, optical wireless communication (OWC) technologies can complement, replace, or co-exist with audio and radio wave-based wireless systems to improve overall network performance. To this aim, this paper reveals the full potential of OWC-based IoT networks by providing a top-down survey of four main IoT domains: IoTT, IoBT, and IoGT. Each domain is covered by a dedicated and self-contained section that starts with a comparative analysis, explains how OWC can be hybridized with existing wireless technologies, points out potential OWC applications fitting best the related IoT domain, and discusses open communication and networking research problems. More importantly, instead of presenting a visionary OWC-IoT framework, the survey discloses that OWC-IoT has become a reality by emphasizing ongoing proof-of-concept prototyping efforts and available commercial off-the-shelf (COTS) OWC-IoT products.