Motivation for the eProcurement Ontology Methodology
Introduction
Public procurement represents around the 20% of GDP in Europe. This big buying volume offers a high economic potential to enhance efficiency of European procurement.
The EU is investing significantly in the digitalisation of the public procurement process (referred to as eProcurement). Digitising public procurement goes beyond simply moving to electronic tools; it rethinks various pre-award and post-award phases with the aim of making them simpler for businesses to participate in, and for the public sector to manage. It also allows for the integration of data-based approaches at various stages of the procurement process.
With eProcurement, public spending should become more transparent, evidence-oriented, optimised, streamlined and integrated with market conditions. More specifically, eProcurement offers a range of benefits such as:
-
significant savings for all parties, both businesses and the public sector;
-
simplified and shortened processes;
-
reductions in red-tape and administrative burdens;
-
increased transparency;
-
greater innovation;
-
new business opportunities by improving the access of all enterprises, including small and medium-sized enterprises (SMEs), to public procurement markets.
To deliver the aforementioned benefits, eProcurement relies heavily on the exchange of data between the different systems supporting the procurement processes i.e., achieving end-to-end interoperability of public procurement processes and underlying systems, and on the availability and dissemination of procurement data to the wider public i.e., improving transparency and stimulating innovation and new business opportunities.
As stated in the Commission Regulation 2015/1986 (EU), contracting authorities in the EU are legally required to publish notices above certain capital thresholds. Article 6 of that Regulation states that either the online eNotices application or the TED eSender systems should be used for electronic transmition of notices to the Publications Office of the European Union. From a different angle, the implementation of the/ a revised PSI directive across the EU is calling for open, unobstructed access to public data to improve transparency, and to boost innovation, via the reuse of public data. Procurement data has been identified as data with a high-reuse potential. Making this data available in machine-readable formats, following on from the "data as a service" paradigm, is required to maximise its reuse.
Data exchange, access, and reuse are key requirements for efficient and transparent end-to-end public procurement. Because of this, a shift in focus is taking place from the definition of procurement process standards for system-to-system exchange, to the development of data standards for publishing e-Procurement data in open, machine-readable formats.
The Challenges
The main challenge is that procurement data exists in different systems across the European Union:
-
the relations between the different concepts in the procurement chain and data flow are not fully documented, therefore data and data relationships cannot be reused directly in a flexible and comparable manner;
-
some data has inherited formats from its paper origins leading to illogical business processes and incorrect conceptual models;
-
different systems use different data formats therefore reuse of information is not always efficient; and
-
taxonomies like the CPV are often not used correctly which creates serious problems e.g., making it very difficult for SMEs to find suitable business opportunities.
Given the increasing importance of data standards for e-Procurement, a number of initiatives driven by the public sector, industry, and academia have been initiated in recent years. Some have grown organically, while others are the result of standardisation work. The vocabularies and the semantics that they introduce, the phases of public procurement that they cover, and the technologies that they use all differ. These differences hamper data interoperability and reuse. This creates the need for a common data standard for publishing procurement data, hence allowing data from different sources to be easily accessed and linked, and consequently reused. The e-Procurement ontology (henceforth referred to as the ePO) introduced by this study attempts to address this.
The Solution
The ultimate objective of the ePO is to deliver a common and agreed (machine readable) OWL ontology that will conceptualise, formally encode, and make available in an open, structured, and machine-readable format, data about public procurement. The ontology will cover the public procurement process from end to end, i.e. from notification, through tendering to awarding, ordering, invoicing and finally payment.
The ePO does not reinvent or redefine existing terms or processes, but uses other existing ontologies and standards, thus facilitating the seamless exchange, access, and reuse of data.
Process, Methodology and Technology discuss in detail the open process and methodology that will be followed for developing the ePO.
The Target Audience and Use Cases
The Target Audience
The target audience of the ePO is made up of the following groups of stakeholders:
-
Contracting authorities and entities, i.e. buyers, such as public administrations in the EU Member States or EU institutions;
-
Economic operators, i.e. suppliers of goods and services such as businesses, entrepreneurs and financial institutions;
-
Academia and researchers;
-
Media and journalists;
-
Auditors and regulators;
-
Members of parliaments at regional, national and EU level;
-
Standardisation organisations;
-
NGOs; and
-
Citizens
Use Cases
The ePO is designed to meet specific needs of the aforementioned stakeholders. These needs are described in the use cases below. The use cases are organised around the following categories:
-
Transparency and monitoring
-
Innovation & value added services
-
Interconnection of public procurement systems
| 1 | Transparency and monitoring |
|---|---|
1.1 |
Public Understandability To facilitate the understandability of the public procurement process, the parties involved in procurement processes, as well as citizens, journalists, and regulators, should be able to access procurement data easily in a structured and machine-readable format. Many stakeholders aim at gaining a quick understanding of the information provided rather than performing an in-depth analysis of the published documentation. Currently, two main challenges exists. Firstly, data coming from different e-Procurement systems are often fragmented, reflecting the compatibility challenges between source systems. Second, the data is available in different formats and representations, which are not always consistent and interoperable, and are therefore hard to connect and interlink. By providing a common view over e-Procurement data, the ePO will allow providers of procurement data to link their data and make it available in ways which will be easier for the non-technical consumer to interpret and reuse, in order to create a complete view of the public procurement process. Example: A watchdog would like to understand how a public administration purchases goods and services. Their main goal is to understand the procedure and gain visibility of all the procedural steps. Procurement procedures often consist of complicated documents and processes, which are scattered on different platforms and websites, and are not always understood by the wide public. As all procurement data is now represented and made available using the ePO, the watchdog can easily combine data from different sources, thereby providing the context for understanding the information. Information requirements: In this case it is required that: * the ePO can model all documents that result from any phase of the procurement process; * the ePO can model all metadata about elements of the procurement process, such as participating entities. |
1.2 |
Data journalism The ever increasing amount of digitised information leads to new ways of producing and disseminating knowledge in society. Data journalism helps journalists to: * identify information; * understand complex information; * identify complex data deriving from different sources; and * create compelling stories (e.g. through data visualisation techniques) which can be easily communicated and understood by the wider public. By providing a common way to describe e-Procurement resources and data, the ePO will enable data journalists to identify, extract integrate and analyse relevant information coming from different sources. Example: A journalist in France is writing an article about the total number and volume (in Euro) of tenders in the domain of transportation by looking at different data sources in the country, and also by comparing the French data with data from neighbouring countries, such as Belgium and Spain. As all data has been modelled using the ePO, it is easy for the journalist to identify all the data that is related to procurement procedures and the resulting invoices. The journalist is then able to integrate and analyse the data related to transportation, and produce data visualisations based on the organisation and location data of the tenders. Information requirements: In this case, it is required that: * the ePO can model data about economic operators, such as businesses (names, locations, contact details etc.); * The ePO can model calls for tenders; * The ePO can model invoices, moreover, it requires core, not private or sensitive data, about invoices to be available as open data; * data from the ePO can be linked with procurement data from other countries' procurement systems. |
1.3 |
Monitor the money flow To obtain an exhaustive and unified view of the flow of public money, from tax collection and budget through to procurement and spending, e-Procurement data should be integrated with other datasets such as budget, spending and location data. A common ontology such as the ePO is necessary in order to interlink such datasets, and help with the creation of a unified view of the flow of public money. Example: A procurement watchdog is analysing the flow of public money over an interval of two years. Using the ePO as the common model for representing data allows the watchdog to find their way through the different sources that have to be consulted, e.g. budget dataset, calls for tender and procurement notices, and to interlink the data in order to identify the trails. Examples of the data to be interlinked by the watchdog, in order to discover the flow of money could be: * the value of the contract; * the name of the awarded tender; * the location of the awarded tender; and * the department of the public administration that awarded the tender. Information requirements: In this case it would be required that: * the ePO can model all procurement process data e.g. calls for tenders, notices etc.; * the ePO can model economic operator data e.g. name, location etc.; * the ePO can model contract data e.g. contract value; * the ePO can model exclusion criteria etc.; * the ePO can link to other datasets e.g. budget datasets, spending datasets, tax information datasets. |
1.4 |
Detect fraud and compliance with procurement criteria For assuring efficiency and transparency, and for detecting fraud and corruption in public administrations, EU institutions, and contracting authorities, rigorous audits of procurement need to take place. In order to improve and further automate the audit process, different data should be made available in structured, machine-readable formats so that different data sources can be referenced and integrated. The creation of the ePO will be a first step towards achieving such integration. Example: While auditing the evidence submitted by the tenderer who was awarded the contract, the auditor noticed that the supplier did not comply with the location criteria that were agreed during the signing of the contract. The collated payment evidence proved that by disregarding the initial agreement, the supplier had leased services from outside of the European Union to reduce the cost of the works. Publishing e-Procurement data in a structured, linked, and machine-readable format, allows the interconnection of data on transactions, criteria, contracts, and evidences from different sources, e.g. including BRIS and ECRIS, thus facilitating cross-checking and automated fraud detection. Information requirements: In this case it would be required that: * the ePO can model the evidence, the contract, the procurement criteria, including the location criteria; * the ePO can link its data to data in other datasets, such as procurement systems of different countries or the BRIS or ECRIS. |
1.5 |
Audit procurement process To monitor the correct use of funds it is necessary to cross-check data from different sources. In the case of public procurement, when the payment and invoice data is represented as linked data through the ePO, it is possible to link it with budget data. In this way one can check if the amounts resulting from the invoices do correspond to the initially budgeted amounts. Example: A governing body wants to make sure that no payment through public procurement on any specific category exceeds the agreed amount. For this, the government body can easily organise all the invoice data of all procurements by category, combine it with budget data, and cross-check if the numbers add up correctly. Information requirements: In this case it would be required that: * the ePO can model payments, contract terms; * the ePO can link this data with budget data. |
1.6 |
Cross-validate data from different parts of the procurement process Representing all phases of procurement in a linked data format can allow for better cross-validation of the data of any part of the process. Example: After a contract has been awarded to a specific tenderer a watchdog would like to check if the criteria for the awarding of the contract have been met. By having all parts of the process linked, the watchdog can by identifying the specific contract and immediately identify the tenderer and the criteria of the contract. Through linking this data with data about the tenderer from other sources, such as their financial data, they can double check if the tenderer does actually fulfil the requirements. Information requirements: In this example it would be required that: * the ePO can model the contract awarded, the criteria of the contract, the details of the supplier; * the ePO can link is data to data in other databases such as those containing financial data about businesses. |
ID |
2. Innovation & value added services |
2.1 |
Automated matchmaking of procured services and products with businesses Automated matchmaking of procured services and products with businesses Example: An economic operator requires more information in order to find and decide on a trade partner. The economic operator is able to identify the ideal candidates by displaying the names of winners in different products or services against the value/cost of said products or services. Representing e-Procurement data following an ontology and making it available in a machine-readable format facilitates the automated mapping between the provided data about the economic operators and that about the economic activities. Information requirements: In this case it would be required that: * the ePO can model economic operator’s details such as names, locations, contact details etc.; * the ePO can model procurement criteria; * the ePO can link the data of the ePO to data of other sources including material costs, labour costs etc. |
2.1 |
Automated validation of procurement criteria Economic operators that submit a tender are required to fulfil several criteria. In order for a contracting authority to automatically validate whether the criteria are met by an economic operator, data, both from the contracting authority’s and the economic operator’s side, should be cross-checked. In order to automate this process, both the data and the evaluation criteria should be made available in machine-readable formats. Example: An economic operator submits a tender to DG Informatics of the European Commission. The offer is written based on the criteria defined by the contracting authority in the tender specifications. Through the semi-automated validation of the tender, the economic operator is notified whether the tender meets the procurement requirements in terms of evidence required to check against financial and other exclusion criteria. if not, the tenderer is provided with a list of further evidence required to fulfil said criteria, and only after this submission does the process move on to the manual evaluation of technical requirements. Such preliminary automation allows for gains in speed and efficiency. Information requirements: In this example it would be required that: * the ePO can model tenders, notices, offers by tenderers, procurement criteria, evidence; * the ePO can model the relationship between offers and procurement criteria. |
2.3 |
Alerting services Contracting authorities announce and publish calls for tender to economic operators, citizens, and third parties. Through the use of alerting services, economic operators can be informed about published calls for tenders that match their profile. In order to automate alerting services, e-Procurement data such as tenders and information about economic operators should be machine processable, so they can be integrated, matched, and the right data delivered to the right person (depending on their subscription to the alerting services). Example: A Spanish public administration procures stationery and textbooks for the forthcoming year. The public administration publishes the call for tenders on an online platform. Since the call for tenders is published in a machine-readable format, following the structure of the ePO, third-party applications can process the call for tender and send alerts to interested parties in their client bases. Usually, such third party applications offer their clients the ability to define criteria they want to be automatically alerted on. Information requirements: In this example it would be required that: * the ePO can model the calls for tenders and the tender details. |
2.4 |
Data analytics on public procurement data Although data is available in vast amounts, businesses and public administrations often fail to manage these data efficiently and extract useful and qualitative information from them. Applying e-Procurement data analytics could be advantageous for economic operators, contacting authorities, and external parties such as journalists and watchdogs. Applying data analysis techniques to e-Procurement data allows stakeholders not only to understand public procurement better, but also to take better informed, evidence-based decisions. In order to fully exploit the potential data analytics in e-Procurement, data should be published in machine-readable formats, in which the ePO plays a major role, and (preferably) linked open data. Linked Data allows for flexible data integration over the Web; this helps to increase data quality and fosters the development of new services. Example: The European Commission aims to leverage its decision-making capability during a call for tenders in telecommunications by analysing all the data available about the potential suppliers and forecasting a fair market price. The European Commission aims at ensuring that the contract will be awarded to the supplier that provides the best services at the best price. In order for the European Commission to conduct its analysis, e-Procurement data should be integrated with a large amount of data coming from different sources, such as data about fees and pricing, qualifications, technical specifications, and cost of materials. Information requirements: In this example it would be required that: * the ePO can model economic operators and procurement criteria; * the ePO can link its data with that of other sources that provide data on fees, pricing, cost of materials etc. |
ID |
3. Interconnection of public procurement systems |
3.1 |
Increase cross-domain interoperability among Member States The European Union aims at providing a competitive economic environment for economic operators from different Member States. In order to achieve such a competitive environment, economic operators, public administrations, researchers, and academia should be able to access and exchange procurement information coming from different sources around Europe, allowing them to participate in calls for tenders from procurers from different Member States. Similarly, contracting authorities should be able to access information about economic operators, which are based in different Member States, and submit tenders for procured services. Making e-Procurement data available in common well-structured and machine-readable formats enhances cross-domain and trans-European competiveness by allowing economic operators from any Member State to participate in public procurement in any other Member State. Example: The VAT authority of a Member state wants to monitor the activity of a certain economic operator. By having all procurement data in all Member States published in a common and machine readable format, this data can be integrated into the systems of the VAT authority. This way it can instantly gain access to all data about any business conducted for public administrations by that economic operator in any other Member State. Information requirements: In this case it would be required that: * the ePO can model the whole procurement process and the details of each phase; * the ePO uses unique identifiers for the economic operators and contracting authorities and uses common reference data wherever required, such as NALs, NACE codes, CPV, common codes for products etc.; *the ePO can link its data to a dataset containing information about economic operators. In this example the VAT authority would simply have to gain access to the systems hosting procurement data of each Member State and it will instantly acquire all needed data. |
3.2 |
Introduce automated classification systems in public procurement systems During the procurement procedure, especially upon the receipt of offers, procurers receive many documents from different sources. Improved and automated classification of these documents would facilitate, and make more efficient, their processing and archiving. The ePO will set the grounds for common ways and rules for classifying such documents. Example: A contracting authority procuring agricultural products is receiving different types of documents and evidences from potential suppliers via its electronic submission platform. When uploading documents, suppliers are asked to complete core metadata coming from the ePO. For example, implementing the ePO facilitates the provision of the specifications of their products, the financial state and the contact details of the suppliers in a commonly agreed and structured way. The platform of the procurer can then automatically classify all received documentation, using machine learning techniques, based on different dimensions including, among others, the following: * The price of the tender; * The category of the tenderer’s business; and * The extent to which the tenderer complies with specific criteria. Information requirements: In this case it would be required: * Of the ePO to model all documents and evidences regarding tender offers; * Of the ePO to model procurement criteria; * Of the ePO to model details about the economic operators; * Of the ePO to model product categories. |
Table 5, Relevant actors for each use case, below summarises the relationships between the identified actors and the uses cases.
Use cases/Actors |
Contracting authorities |
Economic operators |
Academia |
Media/ journalists |
Auditors/ regulators |
Parliament |
Standardisation organisations |
NGOs |
Citizens |
1.1: Increase transparency and public understandability |
x |
x |
x |
x |
x |
x |
x |
x |
x |
1.2: Data journalism |
x |
x |
x |
x |
x |
x |
x |
x |
|
1.3: Monitor the money flow |
x |
x |
x |
x |
x |
x |
x |
x |
|
1.4: Detect fraud and compliance with procurement criteria |
x |
x |
x |
x |
x |
x |
|||
1.5: Audit procurement process |
x |
x |
x |
x |
x |
x |
|||
1.6: Cross-validate data from different parts of the procurement process |
x |
x |
x |
x |
x |
x |
|||
2.1: Automated matchmaking of procured services, products and businesses |
x |
x |
|||||||
2.2: Automated validation of procurement criteria |
x |
x |
|||||||
2.3: Alerting services |
x |
x |
x |
x |
x |
x |
x |
x |
x |
2.4: Data analytics on public procurement data |
x |
x |
x |
x |
x |
x |
x |
x |
|
3.1: Increase cross-domain interoperability among Member States |
x |
x |
x |
x |
x |
x |
x |
x |
|
3.2: Introduce automated classification systems in public procurement systems |
x |
x |
