The project “Human in the AI loop” is targeted to shop floor personnel (e.g. machine operators) as well as students and its main goal is to create the right competencies to work and collaborate with data-based artificial intelligence (DB-AI) systems. We define DB-AI systems as software agents that can automatically create knowledge from data (e.g. automatic machine failure classification) and provide (actionable) recommendations to humans or direct system control. The technology is believed to yield significant disruption and improvement potential for production processes, however shop floor personnel has not been sufficiently up-skilled to interact with this new technology yet. In order to do so, the project intends to create competency-based learning using deep knowledge of AI use-cases and the skills needed during different AI life-cycle phases.