Recent years have seen a rise of techniques based on artificial intelligence (AI). With that have also come initiatives for guidance on how to develop “responsible AI” aligned with human and ethical values. Compared to sectors like energy, healthcare, or transportation, the use of AI-based techniques in the water domain is relatively modest. This paper presents a review of current AI applications in the water domain and develops some tentative insights as to what “responsible AI” could mean there. Building on the reviewed literature, four categories of application are identified: modeling, prediction and forecasting, decision support and operational management, and optimization. We also identify three insights pertaining to the water sector in particular: the use of AI techniques in general, and many-objective optimization in particular, that allow for a pluralism of values and changing values; the use of theory-guided data science, which can avoid some of the pitfalls of strictly data-driven models; and the ability to build on experiences with participatory decision-making in the water sector. These insights suggest that the development and application of responsible AI techniques for the water sector should not be left to data scientists alone, but requires concerted effort by water professionals and data scientists working together, complemented with expertise from the social sciences and humanities.