Have you considered the system purpose, including critical objectives and intended uses or applications?
Do you think the use of AI is in line with your organization’s/community's values?
Did you take specific account of ethical considerations relating to the use of AI before developing an AI solution?
Have you taken a specific account of algorithmic bias for the envisioned AI solutions?
Have you envisioned stakeholders, including demographic groups (i.e., race, gender, age, disability status, skin tone, etc.), that will directly or indirectly affect the system and the expected benefits and consequences?
Have you envisioned whether the system/activity will impact human rights?
Have you considered how to track and mitigate the effects of your system/activity on the environment?
Have you envisioned sensitive, premature, or adversarial uses or applications?
Did you identify subject matter experts and engage them during the system envisioning phase and AI solution development process?
Did you consider inclusivity by putting people in the center from the beginning of the process (use-case definition, data collection, and system development)?
Have you identified sustainability strategies for the envisioned AI solution?
Is data collection guidelines, specifications, and protocol in place?
Do you have proper data governance and control over personal and or other sensitive data?
Is the source of data used for the development of AI solutions known?
Is the dataset publicly available?
Does the data contain any serious issues such as (missing values, label-imbalances, invariance, etc.) that could impact the AI system developed?
Does the dataset follow acceptable standards best practices and specifications for data development like datasheets for datasets?
Is the dataset self-contained?
Does the data link to or otherwise rely on external resources (e.g., websites, tweets, other datasets)?
Is the data anonymised?
Do you have guidelines and specifications in place to facilitate reproducible model development?
Have traditional models been evaluated for reference?
Has the model overfitted to a sample dataset?
Is the model dependent on specific features?
Is most of the model an original idea?
Is the model based on published research?
Has the base paper been appropriately cited?
Does the base paper have 10+ citations?
Is the model tuned for optimal hyper-parameters?
Is the trained model publicly shared?
Is the model architecture publicly shared?
Are the results of the hyperparameter tuning reported?
Does the model allow use of Explainable AI (XAI) methods?
Does the model employ Explainable AI (XAI) methods?
Does the model use an active learning framework?
Is the model published in a conference or journal?
Is the AI Lab/Organisation capable of reliably deploying AI solutions or applications at scale?
Is there a human in the decision making process?
Are the decision made by the model analysed at regular intervals?
Does the model learn from all the production data via active learning?
Are infographics/graphs used to convey the analyses of the production data?
Is the production data anonymised?
Is the inference data stored for future review, analyses or modelling?
Has the inference data been analysed?
Is the inference data publicly shared?
Are the predictions averaged using ensemble methods?
Is the confidence score considered in the decision of the model?
Does the model make a prediction for every production datapoint regardless of the decision workflow?
For Explainable AI (XAI) methods, is the explanation for the prediction shared in a decision report?
Is the prediction confidence shared in a decision report?