Representative sample data selection

This section describes TED notice data samples and methods used to generate them. At first a sampling is performed on the notices from 2021, and then on a wider set.

Sample TED notices from 2021

This data sample (test_data/sampling_2021) contains carefully selected TED notices based on the following criteria: maximise representativeness, minimise the number of selected documents. The selected notices are guaranteed to cover all possible XPath configurations available in the data. The sampling was performed automatically using a custom algorithm available in the TED-RDF Conversion Pipeline repository.

  • The current sampling exercise was performed on all records collected in 2021;

  • The data space was partitioned twice based on two criteria: the notice form number (e.g. F03, F06) and the eForms Subtype (e.g. cn-standard, can-utilities);

  • The minimal set of the most representative notices have been selected for each form_number and eforms_subtype .

Sampling result statistics

Below we present two tables with statistics indicating the total number for each selection criteria, the percentage relative to the total number of considered notices (i.e. total in 2021), and the number of selected most representative notices.

Sampling based on form_number:

form_number nr. of notices percentage of total number of notices nr. of the most representative notices

F03

249618

37,35

20

F02

223430

33,43

18

F14

77564

11,6

11

F20

30244

4,52

7

F06

27746

4,15

21

F05

22452

3,36

29

F01

11483

1,72

23

F15

7788

1,17

43

F21

5844

0,87

20

F18

2035

0,3

25

F08

2034

0,3

10

F24

1764

0,26

19

F17

1680

0,25

27

F12

1546

0,23

16

F25

1115

0,17

19

F13

957

0,14

13

F04

932

0,14

20

F22

93

0,01

13

F16

56

0,01

11

F23

0

0

0

F07

0

0

0

F19

0

0

0

Total

668381

100%

365

Sampling based on eforms_subtype:

eforms_subtype nr. of notices percentage of total number of notices nr. of most representative notices

29

249618

37,35

19

16

223430

33,43

19

41

77564

11,6

11

30

27746

4,15

21

17

22452

3,36

27

38

19151

2,87

8

39

10797

1,62

7

4

7177

1,07

10

25

7006

1,05

17

20

5771

0,86

18

7

3872

0,58

21

31

2035

0,3

24

1

1839

0,28

8

19

1764

0,26

19

18

1680

0,25

26

23

1502

0,22

15

32

1115

0,17

19

36

930

0,14

13

26

499

0,07

15

5

460

0,07

11

10

434

0,06

17

8

411

0,06

14

40

296

0,04

6

27

235

0,04

17

2

157

0,02

4

21

93

0,01

13

12

73

0,01

5

11

61

0,01

12

6

56

0,01

11

28

48

0,01

8

24

44

0,01

10

3

38

0,01

5

37

27

0

4

13

0

0

0

15

0

0

0

22

0

0

0

33

0

0

0

34

0

0

0

35

0

0

0

Total

668381

100%

454