Conference Agenda

Gender-responsive public procurement (GRPP) represents an innovative tool adopted by the European Commission as part of the European Strategy for Gender Equality 2020-2025, aimed at promoting gender mainstreaming through public contracts, thereby empowering women and fostering sustainable and inclusive growth across Europe.

In the Italian legal system, GRPP was introduced through the National Recovery and Resilience Plan and, more specifically, with the enactment of the new Public Contracts Code (Legislative Decree No. 36 of 2023).

In particular, Articles 57 and 61 of the Code introduced social clauses that contracting authorities must incorporate in order to ensure equal opportunities, establishing mandatory requirements or granting additional points in the procurement evaluation process.

Moreover, Article 46-bis of the Equal Opportunities Code (Legislative Decree No. 198 of 2006) established the Gender Equality Certification System, which plays a crucial role in the context of public procurement, by encouraging participating companies to adopt gender-sensitive practices through a reward-based mechanism.

This certification is recognized as one of the conditions for reducing the amount of the provisional guarantee that economic operators must provide, according to Article 106 of the Code, to participate in the procurement process. Additionally, under Article 108, paragraph 7, contracting authorities can award additional points to companies holding the certification, granting them a competitive advantage over the other ones in the public procedures they participate in.

The widespread adoption of these measures can be facilitated through the effective implementation of digital tools, distributed ledger technologies and artificial intelligence techniques (AI). For this reason, the paper seeks to analyze how digitalization and automatization of the public procurement life cycle, as promoted by Articles 11-36 of the Legislative Decree No.36 of 2023, may contribute to advancing the objectives of GRPP.

First of all, the application of digital tools to public procurement can help direct public funds towards policy priorities, such as gender equality. In this context, the effective implementation of the «virtual file of economic operators» (Art. 24) holds strategic significance, enabling contracting authorities to verify, more transparently, the existence of certifications, including that related to gender equality.

Furthermore, GRPP can be further enhanced through the automatization of public contracts procedures, utilizing advanced technologies such as blockchain or artificial intelligence (Art. 30).

Article 106 is the only regulation that explicitly refers to blockchain and it relates to the management of provisional guarantees. Since gender certification allows for a reduction in the amount, distributed ledgers technologies can support, albeit indirectly, the achievement of GRPP objectives.

In conclusion, it is important to recognize the potential of AI, even though machine learning-based algorithms are not yet employed in the Italian public contracts. Indeed, AI could serve as a valuable tool for officials in administrations who draft notices with social clauses aimed at promoting gender equality. It could also assist them in automating the verification of certifications and the evaluation of criteria for awarding bonus points, both related to gender equality certification.

*Introduction*

The state today has transformed into one of the biggest financial services providers (MoF, 2024). With BHIM, Rupay, public sector banks, and other state-led financial entities the government is providing financial services to millions of Indians both within and outside the country (Union Budget, 2024-25). In this context, we explore the risks that application of GenAI may pose if it is integrated in the system and offer potential solutions to mitigating these risks.

*Background*

Generative AI (hereon referred to as GenAI) is a large language model based on the transformer architecture (Vaswani et al. 2017). It differs from current AI models since it focuses on generating new data, rather than simply analyzing or classifying existing data (Banh and Strobel 2023). It employs machine learning algorithms to learn patterns and structures from a given dataset and uses this knowledge to create new, original content (Nie et al. 2024).

*Methodology*

We undertake a mixed methods approach to get an overview of the risks associated with GenAI in public financial services. First, we interview five financial managers and public servants working in the financial domain by reaching out to our network. Second, we peruse related literature to see relevant studies that highlight the risks of adopting GenAI in augmenting financial services. We use the methodology adopted by (Gioia, Corley, and Hamilton 2013) to draw themes of risks from our interviews and juxtapose these with the literature.

*Findings*

Our analysis of interviews and literature revealed the following twelve themes: accountability, compliance, data privacy, security, bias, fairness, transparency, interpretability, operational failures, reputational damage, quality control, and third-party dependency. These themes collectively highlight the complex challenges and risks associated with using generative AI in public financial services. To bind these GenAI risks within a theory, we utilize the Enterprise Risk Management (ERM) framework (Prewett and Terry 2018) which is a holistic approach to managing risks across an organization. These include four categories of risk:

1. Strategic risks: Accountability, Fairness, and Third-party dependency.

2. Operational Risks: Data privacy, Security, Operational failure, and Quality control.

3. Reporting Risks: Transparency and Interpretability.

4. Compliance Risks: Compliance and Reputational damage.

*Future Directions*

We offer three potential solutions for each of the four GenAI risks we enumerated above.

1. To manage strategic risks, public financial firms should focus on maintaining clear ownership of data, ensuring data equity, and build guidelines for ensuring vendor reliability.

2. To address operational risks, the state and public financial firms should prioritize data protection, fraud prevention, and reliability check on generated data.

3. To mitigate reporting risks, public financial organizations should focus on data transparency and data accuracy through the adoption of Explainable AI (XAI) methods.

4. To address compliance risks, financial institutions should ensure regular audits of model training data and use of XAI to ensure transparency in decisions based on GenAI models.

We believe by adopting these solutions public financial institutions would be able to offset GenAI risks and will be able to harness its potential in full measure.

While Artificial Intelligence’s impact can bring considerable benefits, especially for the private sector, there are undeniable risks for the public sector.

The main concerns are related to security, prejudice and discrimination, and privacy issues.

Artificial intelligence can drop a veil over decades of achievements in terms of democracy and participatory processes, drawing citizens and their underlying interests away from the public decision-making circuit to which they had been painstakingly drawn.

The challenge to democracy posed by artificial intelligence passes through information as a data medium for democratic participation. It has already been accused of creating uncontrollable news on the web, where content is presented based on what the user has interacted with in the past, instead of creating an open environment for multi-voice, inclusive and accessible debate. It can also be used to create fake but extremely realistic images, videos and audio, known as deepfakes, which can be used to defraud, ruin reputations and undermine trust in decision-making processes. All these risks leading to the biasing of public debate and the manipulation of elections.

Adoption of AI in democratic participation naturally raises important ethical and transparency challenges. Algorithm opacity and a lack of understanding of how AI makes decisions could undermine public trust. It is crucial to ensure that AI-based decision-making processes are transparent, inclusive and include accountability mechanisms. Furthermore, it is necessary to prevent discrimination and misuse of data in the creation of AI algorithms and models.

AI technology may pose large-scale risks to democracy, including acute harm to individuals, large-scale harm to society. Problematically, there may not be a single responsible party or institution that primarily qualifies as the culprit even when there is a single responsible institution, there are different types of misunderstandings and intentions that could lead to harmful outcomes. These types of risks include larger-than-expected, worse-than-expected AI impacts, deliberately accepted side effects of other objectives.

Possible ways forward go in different directions and must concern a connection between the principle of participation and the new technological challenges through the realization that the institutions as conceived decades ago are obsolete, but the principle of law guiding them is not.