76 3 CHAPTER 3 Feedback between interpretation and observation stage throughout the lifespan of the system is necessary: continuous observation informs us on which requirements are consistently unfulfilled, which may prompt changes in the implementation or in the chosen requirements. This approach therefore transparently monitors and exposes possible malfunctions or misuse of the system. Ensuring accountability and adherence to values in the context of drone or Autonomous Weapon System deployment is inextricably tied to the notion of human oversight and human accountability. For this reason, we propose to consider drone and Autonomous Weapon System deployment a “process within a socio-technical system”, the monitoring of which includes not only examining the behaviour of the drone or Autonomous Weapon System itself but also examining human-led procedures in pre- and post-deployment. A specific adaptation of the Glass Box approach to this context is therefore the explicit inclusion of the operator(s) as an entity to which norms can apply. A significant choice in this framework is the decision to consider the drone and Autonomous Weapon Systems a “black box”, the internal logic of which is not accessible. This responds to two motivations. Firstly, relying on access and monitoring capabilities on the internal workings of drones and Autonomous Weapon Systems would be a strong assumption, since the proprietary nature of this technology often precludes observation of its software. Second, for auditability purposes, the users of this framework should be able to transparently follow the monitoring process. However, such users, who will respond to the monitoring process, do not necessarily possess the technical background required to understand or check constraints on the internal logic of a drone. Thus, our framework is based on monitoring adherence to norms constraining purely observable elements of pre-, and post-deployment of the drone or Autonomous Weapon System. Another choice is that we purposely designed a technology-agnostic approach so that it can be used on many different systems independent from the AI techniques and algorithms that are used as internal workings of the drone or Autonomous Weapon System. We consider these as part of the black box. In what follows, we present an adaptation of the Glass Box approach for the inclusion of human oversight in autonomous drone or weapon deployment. The proposed framework includes an interpretation and an observation stage, each discussed in detail. Interpretation stage The interpretation stage entails turning values into concrete norms constraining observable elements and actions within the socio-technical system in a similar way as is done in the value hierarchy (Van de Poel, 2013) which is described in section 2.7. This hierarchical structure of values, norms and design requirements makes the value
RkJQdWJsaXNoZXIy MjY0ODMw