Concern with the potential harmful effects of the implementation of multimodal AI in robots and of AI in general is rising. There is more and more recognition that roboticists should not be left alone with the development of ethical technologies and that they need the support of experts from other disciplines. Interdisciplinary design methods are developed to make sure that the new technologies serve and are respectful of their target users. This paper argues that while methods like value-based design are essential to ensure that new social robots serve their target population, there are challenges with their application. Risks connected to the unintended use of the technology are likely to be overlooked and some risk mitigation strategies might not be accessible to robot designers. Lastly, the current system does not offer enough support or incentives to engage in a time-consuming although ethical process. However, there are measures that can be introduced to overcome those challenges and make progress towards responsible technology development.