Organisations using personal data in artificial intelligence applications need to manage it carefully to ensure they remain compliant with POPIA
Rapid evolution in artificial intelligence (AI) applications, as well as improvements in computing power and the increasing availability of data, have led to significant growth in AI across most industries. The key developments in AI over the past few years have been driven by machine learning which, in turn, is fuelled by data. As more and more data is being gathered, so AI enables more sophisticated analysis of large data volumes. As the importance of data rises, so do the associated legal issues.
In some cases businesses are free to use the data they hold for whatever purpose they want, including developing AI algorithms. However, there are many instances where they are not free to use the data. For example, if personal data is used to develop, train or test AI algorithms, that processing will need to be fair and lawful and comply with data protection laws. In addition, if the data relates to a third party, it might be confidential or provided under a limited licence. Also, if third parties will have access to the data, that may complicate the data protection and confidentiality issues.
It is of the utmost importance that AI is used responsibly. On a global scale, an important step to strengthen trust in AI has been the development of principles for the responsible development and deployment of AI, including accountability and transparency. One important aspect of ensuring responsible deployment of AI is legal and regulatory compliance.
From a compliance perspective, businesses in South Africa will firstly need to ensure that their AI system is compliant with the Protection of Personal Information Act, 2013 (POPIA). Almost all of the remaining provisions of POPIA came into effect on 1 July 2020. Various aspects of POPIA must be considered when creating an AI system.
Of key importance for AI systems, given their inherent problem-solving ability, is section 71(1) of POPIA, which governs automated decision-making. This section protects data subjects from being subjected to a decision which is based solely on automated decision-making, which results in legal consequences for the data subject and the data subject being profiled.
For instance, an AI system would have the ability to profile customers seeking a bank loan, and determine their creditworthiness based on previous loan repayments, income, indebtedness etc. Section 71(1) prohibits the bank from making a decision to grant or reject the loan application based solely on the profile created by the AI system. In this particular example, however, the bank which is receiving the customer's loan application would need to determine whether it can rely on one of the exceptions to the prohibition on automated decision-making, which are set out in section 71(2) of POPIA.
Businesses implementing an AI system should also be mindful of section 57(1)(a) of POPIA, which requires a responsible party to obtain prior authorisation from the Information Regulator if it intends to process any unique identifiers of data subjects (i) for another purpose than intended at collection, and (ii) with the aim of linking the information with information processed by other responsible parties. In this instance, "unique identifier" can be any identifier that uniquely identifies a data subject in relation to the responsible party such as, for example, an identity number or employee number.
Section 57(1) will be relevant when, for example, an AI system deployed by Business A intends to combine the identity number of an employee with data collected by Business B to determine whether the employee is more susceptible to a certain work-related risk based on his or her age. In this instance, Business A would have to approach the Information Regulator before it could utilise the AI system. The responsible party must consider not only what information will be processed by the AI system but also how the AI system will use it, to ensure that all data protection compliance requirements have been met.
In many instances, AI systems learn and become more intuitive by acquiring vast quantities of data. However, as mentioned above, organisations must proceed with caution when the data inadvertently contains personal information. Before feeding the data into the system, the organisation should consider whether the information can be input in de-identified form, which would exclude it from the application of POPIA. If the information cannot be de-identified, it would be incumbent on the responsible party to ensure that data subjects are aware that their personal information is being used to test an AI system. If the data subjects originally provided their personal information to the organisation for the purposes of procuring a product or service, for example, then the use of the personal information for AI testing must be compatible with that purpose. If not, before the organisation can use the data subjects’ personal information for AI testing purposes, it would need to obtain their consent.
While the deployment of AI systems creates great opportunities for organisations, it is important for them to understand the laws that apply to the data being input into the system to ensure that use of data in the AI system is not in breach of any laws.
