COMMISSION STAFF WORKING DOCUMENT EVALUATION Accompanying the document Proposal for a DECISION OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL amending Decision No 573/2014/EU on enhanced cooperation between Public Employment Services (PES) - Hoofdinhoud

Council of the European Union

Brussels, 16 September 2019 (OR. en)

12137/19

Interinstitutional File: ADD 2

2019/0188(COD) i

SOC 603 EMPL 458 MI 641 CODEC 1381

COVER NOTE

From: Secretary-General of the European Commission, signed by Mr Jordi AYET PUIGARNAU, Director

date of receipt: 11 September 2019

To: Mr Jeppe TRANHOLM-MIKKELSEN, Secretary-General of the Council of the European Union

No. Cion doc.: SWD(2019) 1350 final

Subject: COMMISSION STAFF WORKING DOCUMENT EVALUATION

Accompanying the document Proposal for a DECISION OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL amending Decision No 573/2014/EU on enhanced cooperation between Public Employment Services (PES)

Delegations will find attached document SWD(2019) 1350 final.

Encl.: SWD(2019) 1350 final

EUROPEAN COMMISSION

Brussels, 11.9.2019 SWD(2019) 1350 final

COMMISSION STAFF WORKING DOCUMENT

EVALUATION

Accompanying the document

Proposal for a

DECISION OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL

amending Decision No 573/2014/EU on enhanced cooperation between Public Employment Services (PES)

{COM(2019) 620 final i} - {SWD(2019) 319 final}

Table of contents

• 1. 

  INTRODUCTION...................................................................................................................... 5

1.1. Purpose ............................................................................................................................ 5

1.2. Scope ............................................................................................................................... 5

• 2. 

  BACKGROUND TO THE INTERVENTION .......................................................................... 6

2.1. Description of the intervention and its objectives ........................................................... 6

2.2. Baseline and points of comparison ................................................................................ 10

2.2.1. Pre-2014 cooperation model showed limitations .................................................. 10

2.2.2. Enhanced cooperation between PES from 2014.................................................... 11

• 3. 

  IMPLEMENTATION / STATE OF PLAY ............................................................................. 12

3.1. Governance of the Network .......................................................................................... 12

3.2. Benchlearning ................................................................................................................ 12

3.3. Mutual assistance .......................................................................................................... 14

3.4. Modernising PES in line with the objectives of Europe 2020 ....................................... 15

3.5. Analytical reports .......................................................................................................... 17

3.6. Implementation of relevant policy initiatives ................................................................ 17

3.7. Implementation of the European Social Fund ............................................................... 19

3.8. Annual work programme............................................................................................... 20

3.9. Promote and share best practices for people not in education, employment or training (NEETs) ............................................................................................................................ 20

3.10. Participation in learning events ..................................................................................... 20

• 4. 

  METHOD ................................................................................................................................. 21

4.1. Short description of methodology and sources ............................................................. 21

4.2. Limitations and robustness of findings ......................................................................... 24

• 5. 

  ANALYSIS AND ANSWERS TO THE EVALUATION QUESTIONS ............................... 25

5.1. Relevance ...................................................................................................................... 25

5.1.1. The objectives have been relevant for the Network .............................................. 25

5.1.2. The relevance of each objective ............................................................................. 25

5.1.3. The objectives have been resilient to the changing labour market and policy developments ......................................................................................................................... 27

5.1.4. The initiatives of the PES Network have been relevant ........................................ 28

5.1.5. Potential for improving relevance .......................................................................... 28

5.2. Effectiveness ................................................................................................................. 29

5.2.1. Quantitative effects of the enhanced cooperation between PES ........................... 29

5.2.2. Modernisation promoted by benchlearning ........................................................... 30

5.2.3. Mutual learning between PES has improved ......................................................... 31 5.2.4. Effectiveness of mutual assistance ........................................................................ 32

5.2.5. PES have taken up the results of Network cooperation......................................... 32

5.2.6. Network contributions to effective policy implementation ................................... 34

5.2.7. Potential for improving effectiveness .................................................................... 34

5.3. Efficiency ...................................................................................................................... 35

5.3.1. The Network has been efficient in delivering initiatives and reaching objectives 35

5.3.2. Costs ...................................................................................................................... 35

5.3.3. Benefits .................................................................................................................. 37

5.3.4. Balance of costs and benefits ................................................................................ 37

5.3.5. Potential to increase efficiency .............................................................................. 38

5.4. Coherence ...................................................................................................................... 39

5.4.1. Consistency of the Decision with EU policies and initiatives ............................... 39

5.4.2. Contributions of the Network to relevant EU policies and initiatives .................... 41

5.4.3. Large degree of complementarity with the EU policy framework ......................... 41

5.4.4. Potential to increase consistency ............................................................................ 42

5.5. EU added value ............................................................................................................. 42

5.5.1. Impacts of the Decision ......................................................................................... 42

5.5.2. The role of the Network in cooperation between PES .......................................... 43

5.5.3. Potential for increased added value........................................................................ 44

• 6. 

  CONCLUSIONS AND LESSONS LEARNED ...................................................................... 44

Annex 1: Procedural information .................................................................................................. 48

Annex 2: Stakeholder consultation — Synopsis report ................................................................. 50

• 1. 

  Consultation strategy ................................................................................................................ 50

1.1. Identification of stakeholders ............................................................................................ 50

1.2. Consultation activities ...................................................................................................... 51

• 2. 

  Consultation activities .............................................................................................................. 51

2.1. Public consultation ............................................................................................................ 51

2.2. Targeted consultations ...................................................................................................... 52

2.2.1. Targeted semi-structured interviews ........................................................................... 52

2.2.2. Written questionnaires ................................................................................................. 53

2.2.3. Workshop .................................................................................................................... 53

2.2.4. Case studies ................................................................................................................. 54

2.2.5. Summary and reflections on challenges ...................................................................... 54

• 3. 

  Information on stakeholder groups .......................................................................................... 54

3.1. Public consultation ............................................................................................................ 54

3.1.1. Country of respondents ............................................................................................... 54 3.1.2. Sector of activity ......................................................................................................... 55

3.2. Targeted consultations ...................................................................................................... 55

3.2.1. Targeted semi-structured interviews ........................................................................... 56

3.2.2. Workshop .................................................................................................................... 56

3.2.3. Written questionnaires ................................................................................................. 56

3.2.4. Case studies ................................................................................................................. 56

• 4. 

  Methodology for data processing ............................................................................................. 57

4.1. Public consultation ............................................................................................................ 57

4.2. Targeted consultations ...................................................................................................... 57

4.3. Case studies ....................................................................................................................... 57

• 5. 

  Dissemination strategy implemented for the public consultation ............................................ 57

• 6. 

  Overview of the results of the consultations ............................................................................ 58

Annex 3: More detailed intervention logic of the evaluation ........................................................ 62

Figures, Boxes and Table

Figure 1: Summary of the intervention logic of the Decision

Figure 2: Increase in benchmarking scores from the first to the second cycle

Figure 3: PES participation across all mutual learning activities, 2011-2014 and 2015- 2018

Figure 4: Summary of the evaluation process

Figure 5: Composition of the PES population

Figure 6: Number of dashboard sessions per country

Box 1: The five pillars of benchlearning

Box 2: Mutual assistance — Cyprus

Box 3: Mutual assistance — Romania

Box 4: Examples of reform initiatives resulting from the benchlearning initiative

Table 1: Selected examples of contributions of the Network to relevant policies and initiatives

Glossary

Term or acronym Meaning or definition

AFEPA Adviser for European PES affairs

ALMP Active labour market policies

CSR Country-specific recommendation

EaSI Employment and social innovation programme

ECE European Centre of Expertise

EMCO Employment Committee

ESF European Social Fund

ESPN European Social Policy Network

EURES European Network of Employment Services

FTE Full-time equivalent

HoPES Heads of public employment services

ILO International Labour Organization

LTU Long-term unemployed

MISSOC Mutual Information Systems on Social Protection

ML Mutual learning

NEET Not in employment, education or training

NGO Non-governmental organisation

OECD Organisation for Economic Cooperation and Development

PARES Partnership between Employment Services Initiative

PC Public consultation

PDCA Plan-do-check-act

PES Public employment services

Network European Network of Public Employment Services

PrES Private employment services

TFEU Treaty on the Functioning of the European Union

WAPES World Association of Public Employment Services

• 1. 

  I NTRODUCTION

  1.1. Purpose

Decision No 573/2014/EU on enhanced cooperation between Public Employment Services (PES 1 ) (the ‘Decision’) establishes the European Network of Public Employment Services (the ‘Network’) from 17 June 2014 until 31 December 2020.

Although not required by the Decision, this evaluation has been undertaken in the interest of good administration and in line with the ‘evaluate first’ principle to take into account the lessons of past EU action and to better design future interventions. This evaluation assesses the state of play and degree of implementation of the Decision, and whether the Network has fulfilled its legal obligations and achieved its objectives, further described in Section 2. The evaluation covers all five criteria set out by the Better Regulation requirements:

• Relevance: the extent to which the Decision, in particular the Network’s objectives and initiatives as outlined in Articles 3 and 4 respectively, have been relevant and remain so in the current labour market context and the social situation in the EU.

• Effectiveness: the extent to which the Network has been effective in delivering the initiatives mentioned in the Decision (Article 4) and reaching its objectives (Article 3). This includes the extent to which the Decision’s objectives have been effectively implemented, assessing indicators of relative success, and their contribution to achieving the Decision’s objectives.

• Efficiency: the extent to which the Network has been efficient in delivering the initiatives mentioned in the Decision and reaching its objectives, and whether the costs were proportionate to the benefits achieved.

• Coherence: the extent to which the Decision and the initiatives under its aegis are complementary with other EU or national policies and initiatives with similar objectives.

• EU value added: the extent to which the Decision has generated added value for the EU as a whole compared to the situation prior to or in the absence of the Decision’s entry into force.

Based on the assessments of each criteria, the evaluation draws some overall conclusions, including lessons learned. The results of the evaluation will feed into reflections on the possible future cooperation between European public employment services post-2020.

1.2. Scope

The geographical scope of the Network is the EU Member States and the EEA countries Iceland and Norway. Inclusion of the EEA countries is supported by their participation in

1 https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=uriserv%3AOJ.L_.2014.159.01.0032.01.ENG

the Employment and social innovation (EaSI) programme. This has also contributed to extending the range of members in the Network.

The timeframe for the evaluation covers the period from the entry into force of the Decision establishing the Network, which is 17 June 2014, until 31 July 2018. The cutoff date was set to maximise the availability of data in the evaluation. Secondary sources used in the evaluation can have different cut-off dates. For example, the 2018 annual

report of the European Network of Public Employment Services (PES) 2 covers the full

calendar year. The evaluation focuses on the extent to which the Decision has been implemented, and the extent to which this has so far led to the expected outputs.

• 2. 

  B ACKGROUND TO THE INTERVENTION

  2.1. Description of the intervention and its objectives

Public employment services are the main agencies carrying out employment policies directly accountable to governments. They are set up to facilitate the labour market integration of jobseekers and the payment of unemployment and/or welfare benefits. Although structured differently in each country, all PES help match supply and demand in the labour market through information, placement and active support services at local, national and European levels. PES are also the main actors implementing activation policies in the Member States, and they play an important role in facilitating successful labour market transitions and integration. The quality of their services has direct consequences for the impact of employment policies on the ground. All of the above make PES key actors in fighting unemployment in Europe and in ensuring the success of the EU 2020 strategy for jobs, and smart, sustainable and inclusive growth.

Performance differs considerably across PES, depending on business models, capacity in terms of staff and resources, and management approaches. Additionally, PES operate in different labour market contexts. They face challenges due to changes in the way the labour market functions and experience fluctuating demand from citizens, for instance due to the increased unemployment rates following the economic crisis. Shrinking public budgets and the need for increased cost-effectiveness of PES have prompted several Network members to undertake reforms. The PES need to respond to these challenges, to changes on the labour market and to customers’ expectations of service delivery.

European-level collaboration between PES started in 1997, when the Commission set up an informal advisory group of the heads of PES (the European Network of Heads of PES (HoPES 3 )). The aim was to promote cooperation, exchange and mutual learning between the member organisations and to receive feedback on employment policy initiatives.

Building on this, in 2013 the Commission proposed to formalise this cooperation to support innovation, benchmarking and mutual learning at European level. This was achieved through the Decision of 15 May 2014 establishing the European Network of

2 https://ec.europa.eu/social/BlobServlet?docId=20613&langId=en

3 https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=uriserv%3AOJ.L_.2014.159.01.0032.01.ENG

Public Employment Services. The Network’s objective is to strengthen the capacity, effectiveness and efficiency of public employment services by providing a platform for comparing their performance at European level, identifying good practices and establishing a mutual learning system. It also aims to give PES more opportunities to help develop innovative, evidence-based policies in line with the Europe 2020 objectives.

Figure 1 below, summarising the intervention logic of this evaluation, gives an overview of needs for stronger and more formalised cooperation. The intervention logic includes the objectives, the problems that the interventions under the Decision intended to solve and how it was expected to work.

The Network contributes to the European employment strategy to create more and better jobs throughout the EU, an integral part of the ‘Europe 2020’ strategy for jobs, and smart, sustainable and inclusive growth.

The objective of the Decision, as stated in Article 3, is to encourage cooperation between PES through a formalised network, in order to contribute to the ‘Europe 2020’ strategy and to the implementation of relevant EU policies. In this way the Network will support:

(a) the most vulnerable social groups with high unemployment rates, especially older

workers and young people not in employment, education or training (NEETs);

(b) decent and sustainable work;

(c) the better functioning of labour markets in the EU;

(d) the identification of skills shortages and the provision of information on their extent

and location, as well as the better matching of jobseekers’ skills with employers' needs;

(e) the better integration of labour markets;

(f) increased voluntary geographical and occupational mobility on a fair basis to meet specific labour market needs;

(g) the integration of people excluded from the labour market as part of the fight against

social exclusion;

(h) the evaluation and assessment of active labour market initiatives and their effective

and efficient implementation.

Figure 1: Summary of the intervention logic of the Decision

NEEDS

• Strengthen PES service capacity and efficiency, as efficient PES are an essential factor for the

successful implementation of employment policies and measures

• Increase the PES potential to contribute to the development of policy implementation measures in line with the Europe 2020 objectives.

OBJECTIVE

Strengthen cooperation between the PES of the Member States + Iceland and Norway with a view to improving PES performance and contributing to the implementation of policy initiatives in the field of

employment

INPUTS

Financial resources: ca. €3.5 million/year over 2014-2020 Staffing of PES Network Secretariat, liaisons (AFEPAs) at national level

ACTIVITIES

• Benchmarking self-assessments, site visits, external assessments • Provision of mutual assistance among PES via mutual learning events • PES Network governance meetings

OUTPUTS

• Benchlearning reports and recommendations among PES to compare their activities in given areas (e.g. contribution to reducing unemployment)

• Studies, toolkits, PES practices, PES Knowledge Centre • Learning dialogues starting in 2019 •

RESULTS

• Increased cooperation and exchange among PES • Follow-up and implementation of the benchlearning recommendations

IMPACTS

• Strengthened and modernised PES • Better performing PES (efficiency and effectiveness) • Better support to the unemployed and the inactive not registered with PES • Better functioning labour markets • Better matching and integration of labour markets • Increased voluntary geographical and occupational mobility • Lower unemployment, including for vulnerable groups • Contribution to meeting the EU2020 target of a 75% employment rate

EXOGENOUS FACTORS

Source: Study supporting the evaluation of Decision 573/2014/EU on enhanced cooperation between

Public Employment Services, Ecorys for the European Commission, 2019.

Article 4 of the Decision sets out the initiatives and activities to be implemented by the Network:

‘(a) the development and implementation of Union-wide, evidence-based benchlearning among PES to compare, with appropriate methodology, the

performance of their activities in the following areas:

(i) contribution to reducing unemployment for all age groups and for vulnerable groups; (ii) contribution to reducing the duration of unemployment and reducing inactivity, so as to address long-term and structural unemployment, as well as social exclusion; (iii) filling of vacancies (including through voluntary labour mobility); (iv) customer satisfaction with PES services;

(b) the provision of mutual assistance, either in the form of peer-to-peer or group activities, through cooperation, exchanges of information, experiences and staff between the members of the Network, including support for the implementation of PES-related country-specific recommendations issued by the Council upon request

by the Member State or the PES concerned;

(c) contribute to modernising and strengthening PES in key areas, in line with the employment and social objectives of Europe 2020;

(d) prepare reports at the request of the European Parliament, the Council or the

Commission, or on its own initiative;

(e) contribute to the implementation of relevant policy initiatives;

(f) adopt and implement its annual work programme setting out its working methods,

deliverables and the details related to the implementation of benchlearning;

(g) promote and share best practices on the identification of NEETs and on the development of initiatives to ensure those young people gain the skills necessary to

enter and remain in the labour market.’

The aim of the Decision is to: (i) increase cooperation among PES and the implementation of learning and recommendations achieved through this cooperation; (ii) contribute to better performing PES and better functioning labour markets. Contribution to better implementation of EU policy initiatives such as the EU2020 targets is an overarching goal. Due to the broad range of themes and responsibilities covered by the Decision, the outputs take, among others, the form of studies and analysis, learning events, and visits. Dissemination and sharing of knowledge is of key importance.

2.2. Baseline and points of comparison

2.2.1. Pre-2014 cooperation model showed limitations

As outlined in the explanatory memorandum for the 2013 proposal for the Decision 4 , experience had shown that PES were not engaging sufficiently in mutual learning and

benchmarking 5 activities by themselves (see further details in Figure 3). In summary, four

main forms of PES cooperation were in operation.

• Voluntary mutual learning opportunities via the ‘PES to PES Dialogue’. This EU-funded programme was established to financially support learning across PES and the implementation of efficient policies.

• Strategic dialogue between different employment services (public, private and non-profit) in the context of the Partnership between Employment Services (PARES) initiative.

• Informal, voluntary cooperation of the heads of employment services (HoPES) via the HoPES Network.

• Bilateral or multilateral cooperation between staff at different PES levels in the context of self-funded exchanges or EU-funded projects.

Available evidence from the PES to PES Dialogue programme (annual reports, dissemination of conference papers) indicates that the programme focused on exchanges based on specific guidelines from the European employment strategy. Guidelines addressed for example in 2014 focused on: (i) increasing labour market participation of men and women, reducing structural unemployment and promoting job quality (Guideline 7); and (ii) developing a skilled workforce responding to labour market needs and promoting lifelong learning (Guideline 8).

The activities of the PES to PES Dialogue programme included: (i) conferences; (ii) peer reviews intended to identify good practices, explore them and disseminate their core elements; and (iii) follow-up study visits providing ad hoc support provided when needed and requested by PES. High-level advice and expertise was delivered by peer PES experts to increase the operational capacity of PES, to improve service offer and delivery or to implement good practices highlighted by the peer review. In addition, ad hoc analytical

6

papers were published on various themes .

Despite progress over the years, this cooperation model showed considerable limitations:

• Since participation of national PES in these activities was voluntary, this made it harder to quickly identify low performance by PES and potential structural labour market challenges deriving from this.

• There was no reporting mechanism, meaning that policy designers at national and EU level were not systematically informed about the results of the

4 https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52013PC0430&from=EN

5 Definition of benchmarking and benchlearning are given under point 3.2 on p.12.

6 The PES Knowledge Centre, established as an initiative under the Decision, includes all papers and reports back to 2008 https://ec.europa.eu/social/main.jsp?catId=1163&langId=en

cooperation. • The linkages between benchmarking and mutual learning activities were weak

and inconsistent, and the evidence base for the activities of the mutual learning

programme were not scientifically robust.

• Not all PES participated, and often the least mature PES and those with the

greatest labour market challenges participated the least.

• PES developed their own tools and approaches (e.g. for digitalisation) at national

level, rather than saving time and money by learning from the more advanced

PES.

As a result of the above, efforts to make PES more comparable by clustering them according to business models had not succeeded by 2013.

2.2.2. Enhanced cooperation between PES from 2014

The aim of the Decision was to fill the gap between the pre-2014 situation and the new requirements for PES to contribute effectively to the Europe 2020 strategy. To contribute to the work of the Employment Committee (EMCO), which supports the coordination of the Member States' employment policies, PES were required to adapt their organisation models, business strategies and processes faster than what had been achieved by the previous, informal network. Formalising the Network was identified as an effective way to provide a platform for comparing PES performance across countries, identify good practices and foster mutual learning.

While Member States would remain responsible for organising, staffing and running their PES, a formal PES Network would expand, reinforce and consolidate benchmarking and learning initiatives for the benefit of all PES. The Network was expected to provide a platform for comparing their performance at European level, identifying good practices and fostering mutual learning to foster learning organisations to strengthen service capacity and efficiency. It constituted a concerted action to modernise and empower PES to successfully act in unison in the face of the then predominant economic crisis.

A European PES Network established on a solid legal foundation was expected to be able to ramp up coordination between PES and provide the Network with legitimacy to act. A formalised structure was also considered a pre-condition for increasing the Network’s potential to help develop innovative, evidence-based policy implementation measures in line with the Europe 2020 objectives. The Network was also expected to feed into the European Semester surveillance process by identifying low-performing PES at an early stage before their problems became structural. Formalising the Network was also expected to facilitate implementation of labour market projects financed by the European Social Fund (ESF), and the proposed initiative would contribute to improved cost-efficiency. The Network established under the Decision would carry out initiatives in the form of ‘incentive measures’ provided for under Article 149 of the Treaty on the Functioning of the European Union (TFEU) designed to improve cooperation between Member States in the employment field. The Network’s initiatives would be in addition to cooperation among PES within the European Network of Employment Services (EURES) on the basis on Articles 45 and 46 TFEU.

• 3. 

  I MPLEMENTATION / STATE OF PLAY

This section describes the Network’s governance structure and the implementation of each of the initiatives provided for under Article 4 of the Decision. Where thematically appropriate, some initiatives have been linked together.

3.1. Governance of the Network

The Network has a governing board made up of the heads of PES from each EU Member State and from Norway and Iceland, plus a representative from the Commission. EMCO has observer status. The Board appoints a chair and a vice chair, while the second vice chair is held by the Board member from the PES of the country holding the EU Presidency. The chair represents the Network. The Board meets twice a year in the Member State that holds the EU Presidency. It sets the strategic direction of the Network, debates labour market developments and monitors the work programme’s implementation. Each Board member appoints an Advisor for European PES Affairs (AFEPA) to help run the Network. AFEPAs meet twice a year in Brussels.

The Network’s Board is assisted by a Secretariat provided by the Commission (specifically by the Directorate-General for Employment, Social Affairs and Inclusion (DG EMPL)). The Secretariat, in cooperation with the chair and vice-chairs, prepares the Board meetings and helps implement the annual work programme. It also organises and chairs the meetings of the AFEPAs.

3.2. Benchlearning

Benchlearning is defined in the Decision as the ‘process of creating a systematic and integrated link between benchmarking and mutual learning activities that consists of identifying good performances through indicator-based benchmarking systems […] and of using findings for tangible and evidence-informed mutual learning activities, including good or best practice models’. The aim is to support each PES in improving their performance through comparisons and institutional learning from peers.

Box 1. The five pillars of benchlearning

• 1. 

  Quantitative assessment creating transparency on PES performance by collecting, validating and analysing PES data;

• 2. 

  Qualitative assessment of PES performance against performance enablers, achieved through self-assessment followed by a peer/Commission/external expert assessment on the basis of a site visit;

• 3. 

  Systematic identification of good practices collated online, such as in the PES Knowledge Centre;

• 4. 

  A mutual learning programme which builds on the results of and supports

better PES performance by focusing on identified strengths and weaknesses;

• 5. 

  PES follow-up of the benchlearning assessment, by drawing up and implementing action plans to improve performance. Peer-to-PES coaching and learning through learning dialogues.

The benchlearning concept is based on five pillars, as explained in Box 1 above. Quantitative and qualitative assessments of PES performance build the benchmarking side. This is the basis for performance transparency. Knowing about PES structures, processes and services, as well as their outcomes, makes it possible to identify good practices and mutual learning activities. This leads to PES action plans to improve performance, benefiting from peer PES support.

In the context of the benchmarking (quantitative and qualitative assessments), a structured and systematic analysis has been conducted of PES results and its drivers (‘performance enablers’). In 2015, a thorough analysis of performance data (quantitative benchmarking) was carried out and a process of PES self-review, peer review and expert review (qualitative benchmarking) was implemented. The quantitative and qualitative performance indicators are listed in the Annex to the Decision.

On quantitative assessment, each year PES administrative data is collected, validated and analysed to provide the eight mandatory benchmarking indicators listed in the Annex to the Decision:

• 1. 

  Contribution to reducing unemployment for all age groups and for vulnerable groups:

  • a) 

    Transition from unemployment into employment per age group, gender and qualification level, as a share of the stock of registered unemployed persons;

  • b) 

    Number of people leaving the PES unemployment records, as a share of registered unemployed persons.

• 2. 

  Contribution to reducing the duration of unemployment and reducing inactivity, so as to address long-term and structural unemployment, as well as social exclusion:

  • a. 

    Transition into employment within, for example, 6 and 12 months of unemployment per age group, gender and qualification level, as a share of all PES register transitions into employment;

  • b. 

    Entries into a PES register of previously inactive persons, as a share of all entries into that PES register per age group and gender.

• 3. 

  Filling of vacancies (including through voluntary labour mobility): a. Job vacancies filled;

  • b. 

    Answers to Eurostat’s Labour Force Survey on the contribution of PES to the finding of the respondent’s current job.

• 4. 

  Customer satisfaction with PES services: a. Overall satisfaction of jobseekers; b. Overall satisfaction of employers.

Areas of benchmarking through qualitative internal and external assessments of performance are as listed in the Annex to the Decision:

• 1. 

  Strategic performance management; 2. Design of operational processes such as effective channelling and profiling of jobseekers and tailored use of active labour market instruments; 3. Sustainable activation and management of transitions; 4. Relations to employers; 5. Evidence-based design and implementation of PES services; 6. Effective management of partnerships with stakeholders; 7. Allocation of PES resources.

The qualitative assessment was conducted in two cycles: 2015/2016 and 2017/2018. In the second cycle, the assessment was enriched by a focus on changes implemented since the first cycle and by introducing change management as an additional performance area.

The exercise is based on the ‘PDCA cycle’ (plan, do, check, act). This means that the PES are assessed on: (i) their planning and implementation of activities within each performance area; (ii) how they check implementation; and (iii) how they react to needs for revisions or other actions.

By the end of 2018, all PES participating in the Network had been visited and assessed

twice 7 . At the end of each visit, each PES received a feedback report containing

qualitative assessments and performance scores; with 6 as the best score and 1 as the lowest. This provides PES with insight into their strengths and weaknesses in the various performance areas, provides practical suggestions for further improvements, and identifies peer PES as potential partners for exchange (for the first cycle). The feedback report also provides insight into the relevance, coherence and consistency of the reform agenda and an assessment of the PES’s change management (for the second cycle).

As follow-up, reports on change are submitted by the PES one year after the visit. These reports give an overview of the recommendations to which the PES has given priority, and of the actions taken to bring about improvement in the specific area.

The results of the benchlearning assessments have over the past four years been increasingly used as an evidence base to develop and steer the Network’s mutual learning programme, as reflected in its annual work programmes 8 . Thematic clusters such as digitalisation, services to employers and performance management were central to the 2018 programme. Finally, a collection of potentially transferable PES practices identified during the assessments and the mutual learning activities were shared within the Network for learning purposes. In the PES Practice Repository, more than 170 PES practices had been presented by the end of 2018.

3.3. Mutual assistance

Results and assessments from the benchlearning programme have helped PES identify performance areas or tasks where support from other PES could be an appropriate method for initiating or implementing reform. The benchlearning recommendations and the country-specific recommendations coming out of the European Semester process form the basis for selecting performance areas or tasks.

7 With the exception of the UK, which was only visited in the first cycle and notified the Commission that it was not participating in the second cycle.

8 Annual reports on the Network’s activities are available for 2015-2018: http://ec.europa.eu/social/main.jsp?catId=1100&langId=en

Through mutual assistance, the selected PES receive technical support from peer PES to develop specified performance areas or tasks upon request. This includes support for the implementation of European Semester country-specific recommendations 9 . Between 2015 and 2018, mutual assistance projects lasting between 9 and 18 months were carried out in Bulgaria, Cyprus, Greece, Italy, Romania and Spain. PES colleagues from Austria, from VDAB (the PES for Belgium’s Flanders Region), from Denmark, Estonia, France, Germany, Ireland, Latvia, Slovenia, Spain, Sweden and the United Kingdom participated in the projects as peer experts.

The projects covered a broad range of tasks. For example, the project in Spain focused on three main topics: a common profiling tool, customer satisfaction and partnership with employers. The aim was to achieve better coordination between the country’s 17 regional employment services, social services and employers so as to better respond to customers’ needs. At this stage, it is too early to assess the impacts on coordination and the role of the central PES. The Cypriot PES requested support to enhance its capacity to serve the long-term unemployed and to improve outreach to non-registered unemployed people.

Box 2. Mutual assistance — Cyprus

The mutual assistance project in Cyprus took place in 2017. A follow-up meeting was held in 2018. The project focused on the following thematic areas:

• Better customer segmentation; • Expanded use of IT for self-service provision; • Evidence-based design and enhanced delivery of active labour market policies (ALMPs) / improved activation services (including outreach) for jobseekers especially from the most disadvantaged groups, e.g. the young, the disabled, the long-term unemployed; • Services for employers; • Performance management/organisational culture/quality management.

The benchmarking site visit identified the following progress as a consequence of the mutual assistance: improvement in digital services, segmentation, better monitoring of ALMPs and implementation of a new service offer to employers. Success factors included: help in identifying key reform priorities, developing a clear action plan and committing to implementing it, and achieving high staff motivation.

This is further explained in Box 2.

Source: PES Secretariat/European Commission

9 https://ec.europa.eu/info/strategy/european-semester/european-semester-timeline/eu-country-specific href="https://ec.europa.eu/info/strategy/european-semester/european-semester-timeline/eu-country-specific-recommendations_en">recommendations_en

3.4. Modernising PES in line with the objectives of Europe 2020

Modernising a PES involves increasing its level of professionalism, which in turn increases its maturity as an organisation. As explained in Section 3.2 above, PES performance is scored on a scale from 1 to 6 as part of the benchlearning exercise. The following rule is used to approximate maturity in the benchlearning exercise:

• A mature organisation with respect to a specific performance area is achieved when all four scores in the PDCA cycle are 5 or higher.

• A well-developed organisation with respect to a specific performance area is achieved when at least three of the four scores are 4 or higher.

• A developing organisation with respect to a performance area is achieved when at least three of the four scores are 3 or higher.

• In all other cases, the maturity of the organisation is considered ‘developable’.

Low scores indicate performance areas where a PES can improve, while high scores indicate that PES is strong in these areas. Among the 29 PES that had received their second cycle scores by the end of 2018, 9 PES were at the overall level assessed as ‘mature’ or ‘well developed 10 ’.

The comparison between the first and second benchlearning cycle shows that on average PES maturity gradually increased (see Figure 2). Out of 28 countries included in Figure 2, 6 improved their scores by at least 20% from the first cycle to the second, and 10 countries improved by at least 10%. The remaining PES show moderate or no progress at all. Figure 2 shows that Cyprus, Bulgaria and Greece, which have received mutual assistance, (see Section 3.2 above), all belong to the group of PES with a high increase in scores from the first to the second cycle.

Figure 2: Increase in benchmarking scores from the first to the second cycle

10 https://ec.europa.eu/social/BlobServlet?docId=20613&langId=en

Source: PES Secretariat/European Commission

Some PES had a high maturity level already in the first cycle. Their room for further improvements could be more restricted than for PES assessed as less mature in the first cycle.

3.5. Analytical reports

Research and studies on PES topics related to priorities of the annual work programme are published each year at the request of the European Parliament, the Council or the Commission, or on the initiative of the Network. Since 2015, about 60 papers and reports have been published in the PES Knowledge Centre 11 . The reports enable members and practitioners to learn about initiatives across the EU and access their impact. The topics of the reports include PES practices and toolkits as well as analytical papers. Examples of reports from 2018 include a study report on PES’ role in outreach to the inactive population 12 and an analytical paper on early activation and employment promotion 13 .

3.6. Implementation of relevant policy initiatives

The Network delivered a common response to the consultation on the European Pillar of Social Rights after assessing its impact on the Network Decision and reflecting on new trends in work patterns and society. In line with the Pillar’s principles, the Network updated its strategy document ‘EU Network of Public Employment Services Strategy to 2020 and beyond’, demonstrating the Network’s ability respond to new EU policies and initiatives. Some examples of its contributions are set out in Table 1 below.

The overview in Table 1 demonstrates that the Network has contributed extensively to the implementation of EU relevant policies and initiatives. Its contributions have been more

11 https://ec.europa.eu/social/main.jsp?catId=1163&langId=en

12 https://ec.europa.eu/social/BlobServlet?docId=19219&langId=en

13 https://ec.europa.eu/social/BlobServlet?docId=19912&langId=en

extensive in areas such as implementation of the Youth Guarantee and the integration of the long-term unemployed into the labour market. The Network has also provided contributions to other relevant policy areas such as the European Pillar of Social Rights.

Table 1: Selected examples of the Network’s contributions to relevant policies and initiatives

Policies/initiatives Examples of contributions by the Network

• Formal role in monitoring as enshrined in the Recommendation Council • PES Network self-assessment report on PES capacities to implement the Recommendation of Youth Guarantee, 2014

22 April 2013 on • PES Network Catalogue of Measures for implementation of the Youth establishing a Youth Guarantee, 2014

Guarantee • Summary Report: PES Network Conference: Implementation of the Youth Guarantee — Challenges and Success Factors, 2016

• Annual reports on PES Implementation of the Youth Guarantee

• Formal role in monitoring as enshrined in the Recommendation 14

• Peer review on PES approaches for sustainable activation of LTU

• PES LTU Working Group, 2016 15 Council • PES Network contribution to the consultation on long-term unemployment, Recommendation of 2015

15 February 2016 • Proposal to EMCO for quality standards to monitor implementation of the on the integration of Recommendation, 2016 the long-term • Summary report: PES Network Conference: on the integration of the longunemployed into the term unemployed, 2016 labour market • Thematic review workshop on ‘Sustainable integration into the labour market’,

November 2016 • Analytical paper: Sustainable integration into the labour market, 2017 • PES Network study on assessment and early intervention to prevent long-term

unemployment, 2017 • Ad hoc module to the 2018 PES capacity questionnaire survey report 2018

European Pillar of • Network Board contribution to the Commission’s consultation on the

Social Rights European Pillar of Social Rights 16 , December 2016

Integration of • Thematic paper on ‘Integration of refugees into the labour market’

refugees • Follow-up visit on integration of refugees into the labour market, 2017 • Key considerations on labour market integration of refugees, 2016 and 2019

• In some countries (e.g. Bulgaria, Croatia, France, Luxembourg, Hungary, ESF Lithuania and Romania) the PES actively cooperates with the ESF managing

authorities and receives funds for their operational programmes

• Cooperation with the European Network of Employment Services (EURES)

re-established under Regulation (EU) 2016/589 17

EURES • Closer cooperation between the PES Network and EURES at European level agreed at the December 2016 meeting of the PES Network Board

• A member of the PES Network acts as liaison point with the EURES Coordination Group

Source: Study supporting the evaluation of Decision on enhanced cooperation between Public Employment Services, Ecorys for the European Commission, 2019.

14 Official Journal of the European Union, Council Recommendation of 15 February 2016 on the integration of the long-term unemployed into the labour market (2016/C 67/01), paragraph (9).

15 PES Network (2017), European Public Employment Services (PES) Network study on assessment and early intervention to prevent long term unemployment.

16 COM(2017) 287 final i, p. 7.

17 Regulation (EU) 2016/589 of the European Parliament and of the Council of 13 April 2016, OJ L 107, 22.4.2016, p. 1-28.

3.7. Implementation of the European Social Fund

PES are key actors in implementing the European Social Fund (ESF) in some Member States, in particular for measures promoting sustainable and quality employment. Total funding in this area amounts to €40.7 billion (2014-2020, EU and national funding 18 ). Out of this amount, the ESF provided direct support of €2.1 billion for reforms to modernise PES and more than €25 billion to develop active labour market policies that are mainly implemented by the PES.

One of the ESF’s priorities is the funding of active labour market policies aimed at disadvantaged people in the labour market, such as young people and the long-term unemployed. ESF funding is therefore important for the implementation of policy initiatives for many national PES. In some countries, Bulgaria and Romania for example, ESF funding has played an important role in reforming PES in order to ensure they have the institutional capacity and their staff have the necessary skills to help disadvantaged groups to find jobs. In Romania, the ESF project was implemented by assistance from the Network (see Box 3).

Box 3. Mutual assistance — Romania

Romania approached the Network with a request for mutual assistance after receiving country-specific recommendations in 2016 and 2017 on the capacity of its PES, specifically related to improving services to employers and jobseekers. PES colleagues from Belgium (VDAB), Germany and Sweden participated in the project, which took place in 2017 and 2018.

Understanding labour market needs and close collaboration with employers are crucial for PES. The project included a human resource component, with 400 employees of Romania’s national employment service receiving training aimed at improving the skills needed for an effective relationship with employers. In addition, particular action was taken to: (i) develop a new tool to support jobseekers’ profiles and channel them towards appropriate active measures; (ii) develop the IT system; and (iii) develop the case management approach for the registered unemployed. Funding from the ESF had an important contribution to financing the project. Work is still in progress and measures could lead to a positive impact if they are embedded in a comprehensive implementation strategy.

Source: PES Secretariat/European Commission

Member States are responsible for appointing a designated managing authority that provides information on the ESF programme, selects projects and monitors implementation. Out of the 31 PES covered by the 2018 PES capacity report, 4 PES acted as managing authority for the ESF and 5 PES acted as managing authority in partnership with other institutions 1920 . Other PES are involved because the managing authority has delegated certain responsibilities to them.

18 ESF synthesis report of annual implementation reports for 2017.

19 https://ec.europa.eu/social/BlobServlet?docId=20575&langId=en

3.8. Annual work programme

The Network adopts an annual work programme, which is based on exchange of views and a learning needs assessment, agreed upon with all its members. The work programme is designed to assist the PES in: (i) delivering the Network’s mission to promote modernisation; and (ii) helping individual PES to contribute more to the implementation of the EU2020 strategy. Benchlearning is central to the Network’s activities and is key to identifying and addressing mutual learning activities. In recent years, the work programme has been grouped under three headings: benchmarking, mutual learning and network governance.

3.9. Promote and share best practices for people not in education, employment or training (NEETs)

Each year, a number of practices to improve the situation of vulnerable groups in the labour market are published and shared on the ‘PES Practices’ page 21 on the European Commission website. Some practices have a broader scope, not targeted at specific groups of vulnerable people. In addition, practices are presented which focus on NEETs as a group with particular needs, for example the ‘Toolkit on Sustainable Activation of NEETs’ and ‘PES Practices for the outreach and activation of NEETs’.

3.10. Participation in learning events

Since 2015, the range of learning events has been broadened to better support the learning needs of individual PES. Specifically, the focus has changed towards smaller, more targeted learning formats, with events now organised with specific learning groups in mind. This methodology means that benchlearning site visit reports (conducted in the context of the qualitative assessments) and other documents are used to identify PES with specific learning needs in a particular topic area.

PES that are advanced in certain areas are in general invited to host or chair thematic review workshops and follow-up visits, so that they can share their practices with other countries, helping them further develop in that same area. Likewise, advanced PES are invited to participate as external experts in mutual assistance projects. Working groups chaired by, and with participation from, Network members, have a key role in developing knowledge that is relevant to PES, and in preparing (for example) key considerations to be endorsed by the Network Board. The range of learning events has included webinars since 2017 and learning dialogues since 2019. Learning dialogues are small groups of PES, where a PES that is advanced in a certain area shares experience with two to four PES that have learning needs in that area. The coaching approach includes follow-up support aimed at PES implementing what they have learnt for that specific area.

As illustrated in Figure 3, the number of learning events more than doubled between the 2011-2014 and 2015-2018 periods. Similarly, the number of participants more than doubled, and the number of learning days increased by 75%.

20 Norway and Iceland do not participate in the ESF.

Figure 3: PES participation across all mutual learning activities. 2011-2014 and 2015-2018

Source: PES Secretariat/European Commission

• 4. 

  M ETHOD

  4.1. Short description of methodology and sources

Following the evaluation roadmap 22 published in October 2017, the evaluation presented in this staff working document was carried out on the basis of information and data collected from different sources in accordance with the Better Regulation guidelines and Toolbox. The information aims to cover all the elements in the intervention logic presented in Figure 1, with a focus on results and impacts of the activities.

This evaluation draws to a large extent on information from an external study

commissioned from a contractor (the ‘external study’) and the 2017 interim review 23 . The

2017 interim review is the Commission report published (in accordance with Article 10 of the Decision) on 6 June 2017 on the Network’s activities up until that point.

Furthermore, desk research was carried out to map changes resulting from the Decision and the Network’s activities; the research covered a range of written sources such as annual reports, data collections and satisfaction surveys. The sources are complementary and mutually reinforcing. The main sources are summarised in Figure 4.

22 https://ec.europa.eu/info/law/better-regulation/initiatives/ares-2017-5258712_en

Figure 4: Summary of the evaluation process

Staff working document

External study Desk research

Annual reports/surveys

Targeted Benchlearning results

consultations Case studies

Public

consultation Annual data collections Satisfaction surveys

Source: PES Secretariat/European Commission

The external study was conducted between June 2018 and March 2019. Its methodological approach combined quantitative and qualitative data, which have been systematically cross-checked to answer the evaluation questions. The study includes a mapping of policy changes implemented by PES following the creation of the Network in June 2014.

A range of actors were consulted in the external study (see details in Annex 2). Interviews were conducted with:

• members of each of the 32 PES 24 involved in the Network (when possible both

with the Network Board member and the AFEPA);

• selected representatives of relevant EU-level organisations and bodies (e.g.

EMCO, EU-level private employment services and temporary work agencies, the European Lifelong Guidance Policy Network);

• selected representatives of relevant international organisations (for example ILO,

OECD, World Bank, World Association of Public Employment Services (WAPES));

• other stakeholders closely involved in the running of the Network (e.g. the PES

Secretariat, contractors);

• a sample of individuals who were former members of the Network or involved

with PES cooperation pre-2014.

In addition, a workshop was conducted with AFEPAs, as well as case studies in five countries (Estonia, France, the Netherlands, Spain and Romania) to further elaborate the

24 27 Member States PES + 3 Belgian regional PES + Iceland and Norway.

evaluation questions. These countries were selected on the basis of several criteria, namely geographic location, population size, economic situation (measured by real GDP growth, employment and unemployment rates), and on the basis of PES performance indicators (maturity levels, increase/decline in PES expenditure, and increase/decline in new vacancies reported). ESF managing authorities and labour and social affairs ministries received a targeted questionnaire, which 16 labour ministries and 7 ESF managing authorities filled out.

An open internet-based public consultation ran from 17 September 2018 to 13 December 2018. An online questionnaire was available in English, French and German, and replies could be made in any of the 24 official EU languages. In particular, the consultation sought to gather views from: (i) individual citizens in their personal or professional capacity; (ii) researchers and academia; (iii) civil society organisations working with jobseekers or on the issue of unemployment/employment services; (iv) social partners; (v) employment and social services providers; and (vi) public authorities at national, regional and local levels. 126 replies from 18 Member States and Pan-EU organisations were received.

The evaluation also draws on various other secondary sources, cited throughout in the text. Benchlearning results, the evaluation of mutual learning events and the annual reports are among the main sources, a list of which is set out below:

• Annual reports of the Network, which were sent to the European Parliament and

to the Council and published online 25 .

• The outputs produced by the Network as part of its annual work programme,

published on the PES Knowledge Centre and PES Practices site 26 .

• Satisfaction surveys and other monitoring activities with regard to the Network’s activities, carried out by the contractors who supported the implementation of the Network’s work programme. In particular, in 2018, the contractor in charge of mutual learning activities carried out an enhanced

evaluation to understand the potential impact of mutual learning activities 27 . The

enhanced evaluation included: (i) third-stage qualitative interviews with PES who attended a thematic review workshop plus a follow-up visit in 2015 (thus building a continuum with interviews undertaken with 2015 cohorts in 2017); (ii) an online mutual learning participant survey, implemented between May and June 2018; and (iii) follow-up interviews with the respondents to the mutual

learning participant survey who agreed to be interviewed in summer 2018.

• Yearly survey and report on PES capacity available for 2014-2018, published online. In addition, in 2014-2017 an annual report on PES implementation of the

25 https://ec.europa.eu/social/BlobServlet?docId=20613&langId=en

26 https://ec.europa.eu/social/main.jsp?catId=1206&langId=en

27 See Annex 3 to the 2019 annual report, https://ec.europa.eu/social/BlobServlet?docId=20613&langId=en

Youth Guarantee was published, followed in 2018 by a report on PES

implementation of the Recommendation on the long-term unemployed 28 .

• The annual quantitative data collection and analysis gathered via the benchmarking exercise is available for the years 2015-2018. Biennial qualitative assessment site visits to the PES, which involve self-assessment reports, and overall assessment reports are available for the years 2015/2016 and 2017/2018

for the whole Network 29 . For the first cycle, change reports from all 28 Member

States, Norway and Iceland had been produced one year after the site visit.

• The EMCO thematic reviews of PES/ALMPs and the European Semester.

4.2. Limitations and robustness of findings

As per the Decision, the Network should contribute to the improved functioning of the PES themselves, as well as to the policy objectives described in Article 3. Evidence from this evaluation shows that for various reasons many of the envisaged reforms and changes take time to be properly implemented. It is important to point to the contextual factors, such as political decisions, allocation of budgetary resources and changing labour market contexts due to economic and social developments. The period covered by this evaluation may therefore be too short to achieve major changes and not least to measure the impacts, and thus it is not possible at this time to make a full assessment of Network efficiency. This may also mean that there are unrealised gains from plans or initiatives that will only appear in the years to come.

Secondly, it is sometimes difficult to assess whether an initiative was implemented only because of the Decision. This in turn makes it hard to make a full assessment of the added value. Estimating the impact of the Network’s activities and of PES initiatives as a result of the former is in some cases difficult and may hide the real impact. This is because of data deficiencies due to different definitions between countries or the different design of measures or activities. Differences in PES responsibility from country to country is furthermore a reason why data comparability could be limited. For example, the supply of active labour market measures is in some countries a responsibility for the PES, while in other countries it is a responsibility of the municipalities. Furthermore, some PES produce most of their services in-house, while other PES have extensive use of contractors.

An important information source is the qualitative assessments made by PES representatives. However, since it could be argued that people who are deeply involved in a task could be biased in their assessments, this evaluation draws on a broad range of sources. When possible, reliability is checked by comparing information from different sources such as similar information from the external study and the final evaluation report for the mutual learning event, or by comparing assessments by PES representatives

28 https://ec.europa.eu/social/BlobServlet?docId=20186&langId=en

29 The second cycle does not include the qualitative assessment from the UK.

with similar assessments by other stakeholders as the labour ministries. In this way, triangulation or cross-checking of sources mitigates the risk of biased conclusions.

• 5. 

  A NALYSIS AND ANSWERS TO THE EVALUATION QUESTIONS

  5.1. Relevance

  5.1.1. The objectives have been relevant for the Network

As described in Section 3, the objectives set in the Decision have been a major guideline for Network initiatives implemented through the annual work programmes (see Chapter 2.1 for overview). There was strong consensus among stakeholders interviewed that the eight objectives from 2014-2018 had a high overall degree of relevance. The relevance of the Decision’s content was also strongly confirmed by the findings from the targeted consultations with labour ministries and ESF managing authorities. Finally, the relevance of the Decision was confirmed by the public consultation.

One of the main missions for PES is to help disadvantaged groups such as NEETs or the long-term unemployed to find jobs. This has been and still is a major challenge for PES due to high unemployment in the wake of the 2008 economic crisis and the high number of refugees in many countries. The economic recovery has facilitated labour market access for disadvantaged groups, but the long-term unemployed, young people and older jobseekers are still overrepresented among those seeking assistance from PES. Inclusion of inactive people, who are often further away from the labour market, is becoming a priority task for some PES.

The Decision’s objectives encompass key areas of responsibility for PES in the field of employment. By providing a broad framework for Network activities, the objectives enable a flexible approach to addressing priorities for action depending on labour market developments. The PES Network’s 2020 Strategy 2020 and Beyond 30 is updated when necessary to be as relevant as possible to the goals of the Europe 2020 strategy. The range of areas and themes is very broad. At the same time, the external study highlights that there is some variation in the degree of relevance for each of the objectives.

5.1.2. The relevance of each objective

Section 2.1 of the external study gives an overview of assessments of the relevance of each of the objectives set in the Decision. The ‘inclusion of the most vulnerable social groups’ (objective a) is regarded by consultees in the external study as one of the most relevant Decision objectives as it is a long-standing issue in different European countries, particularly focusing on the inclusion of the long-term unemployed and NEETs. The consultations and the 2017 interim review of the Decision reveal that objective a) is closely related to objective g) on the ‘integration of people excluded from the labour market as part of the fight against social exclusion’. This demonstrates that some of the target groups for PES meet multiple and even interlinked barriers to integration in the labour market. Social exclusion in itself, for example because of poverty, can reduce a

30 https://ec.europa.eu/social/BlobServlet?docId=20662&langId=en

person’s chances in the labour market. Achievements on objective g) can therefore contribute to progress on objective a) and vice versa.

‘Supporting decent and sustainable work’ (objective b) is a relevant objective. Nonstandard work (i.e. employment contracts that are neither permanent nor full-time) represent nearly 20% of part-time employment and over 10% of temporary employment in the EU in 2017 (see The Future of Work: Implications and Responses by the PES

Network, Section 3.1.3) 31 . These types of work can provide opportunities for people who

want to work less than full working hours or want to have flexibility. At the same time, this work often comes with a greater unpredictability of working conditions and income flows, especially for workers in low-skilled occupations. Due to the changing labour market characterised by new forms of work, work relationships and forms of work organisation, this objective may become increasingly relevant in the future.

A ‘better functioning of the labour markets in the EU’ (objective c) is a prerequisite for optimal use of the labour force and is at the heart of the mission of European PES. The relevance of this objective is confirmed in the external study. A well-functioning labour market depends on a number of factors such as economic growth, the general regulatory framework, access to a skilled labour force and professional PES services. The Network contributes to professionalisation of PES by facilitating cooperation and helping them to learn from each other, as well as cooperation with for example EURES and the private employment services.

Facilitating the matching of jobseekers with jobs, facilitating labour market integration and effective active labour market policies are all key to a well-functioning labour market. Objective c) is therefore closely linked to objectives d), e), f) and h).

Objective d), ‘identification of skills shortages and the provision of information on their extent and location and the better matching of the skills of jobseekers with the needs of employers’, is assessed as relevant by experts and other stakeholders, for example to fill bottleneck vacancies. ‘Better integration of the EU labour markets’ (objective e) also has close links to objective d), and is clearly a relevant objective for the Network, since it directly concerns cooperation at EU level or bilateral cooperation. Objective f), ‘increased voluntary and geographical and occupation mobility on a fair basis to meet specific labour market needs’ is also considered as relevant. There is no duplication and, according to the external study, there is a good level of coordination between the Network and EURES. Finally, objective h), ‘evaluation and the assessment of active labour market initiatives and their effective and efficient implementation’, has been and remains relevant.

The relevance of the objectives could depend on the labour market situation at national level or on a where an economy is in its business cycle. At country level, objective d) is seen as most relevant in countries facing skills shortages. Some consultees in the external study raised the question whether objective f) is as relevant for the Network as some of the other objectives. In Greece, Italy and Spain, unemployment was still above 10% at the

31 https://ec.europa.eu/social/BlobServlet?docId=20520&langId=en

end of 2018. In other countries, unemployment had dropped to below 4%, and tackling skill shortages is becoming a more prominent task for PES. This illustrates how important it is that the objectives are designed to be relevant for PES facing different labour market conditions. The feedback from PES in the external study confirms that the objectives as they stand fulfil this purpose.

To sum up, there was a strong consensus among stakeholders interviewed that the Decision’s objectives are relevant to the current challenges of different national labour markets. Together, they provide a common strategic framework for PES cooperation to tackle the increasing common global challenges that PES are facing, challenges that require concerted action.

5.1.3. The objectives have been resilient to the changing labour market and policy developments

The external study, Section 2.1, has shown that overall the objectives have been relevant to the challenges of the European labour market and to the social situation. The flexibility in their wording has made it possible to address new and unexpected challenges (e.g.

adaption of the Network strategy and work programme to the 2015 32 migrant crisis),

thereby showing their continuous relevance.

Stakeholders’ assessment of the relevance of the objectives matches well with statistical information. The 2018 assessment report on PES capacity 33 highlights that PES are still dominated by difficult-to-place groups of unemployed people such as young people, the long-term unemployed and older workers. These three groups account for a fairly constant share of the registered jobseekers (see Figure 5).

Figure 5: Breakdown of registered jobseekers

32 https://ec.europa.eu/social/BlobServlet?docId=20662&langId=en

33 https://ec.europa.eu/social/BlobServlet?docId=20575&langId=en

Source: PES Secretariat/European Commission

5.1.4. The initiatives of the PES Network have been relevant

Article 4 of the PES Decision sets out seven initiatives (labelled a-g) which the Network should carry out (see Chapter 2.1 for overview). These initiatives have been the major guideline for the activities of the annual work programmes.

Overall, the initiatives set out in Article 4 of the Decision are relevant to the needs of PES and are consistent with the Decision’s objectives. This is confirmed by the vast majority of consultees in the external study, Section 2.2, both from PES and from other EU and international institutions. The initiatives provide relevant tools to: (i) enhance cooperation between national PES and a well-constructed mix of approaches to suit the needs of different PES; and (ii) ensure synergies and complementarities between the different initiatives. Consultees all assessed ‘benchlearning’ (initiative a), ‘mutual assistance’ (initiative b), ‘reports’ (initiative d), ‘implementation of relevant policy initiatives’ (initiative e) and ‘promoting and sharing’ (initiative g) as very relevant.

Consultees regarded initiative c), ‘contribute to modernising and strengthening PES’ as more overarching although still highly relevant, and implemented by a range of other initiatives such as benchlearning and mutual assistance. In addition, while initiative f), ‘adopt and implement its annual work programme’ was seen as clearly relevant, it could be considered as a requirement of the Network’s function rather than an initiative with the same status as benchlearning and mutual assistance.

As evidenced in the Network’s annual work programmes and confirmed by the external study, Section 2.2, the initiatives set in Article 4 of the Decision are well linked to the objectives set in Article 3. The Network has demonstrated flexibility by focusing on activities related to new upcoming topics such as the integration of migrants and refugees and the prevention of unemployment.

5.1.5. Potential for improving relevance

The remaining difficulties that these groups (young people, long-term unemployed and older workers) face when trying to (re-)enter the labour market indicate that the objective of supporting vulnerable group with high unemployment rates remains relevant.

How PES can contribute to better functioning of the labour market in the changing world of work has been addressed more prominently in the Network’s work programme over the past few years. The challenges posed by the changing world of work are also having an impact on how the objective to ensure decent and sustainable work can be achieved. New working conditions are resulting in employment opportunities in new technology sectors, change or loss of jobs due to the large-scale digitalisation of many professions, more short-term or flexible working and a growing trend towards entrepreneurialism and platform jobs.

To achieve the objectives of the Decision, jobseekers need the necessary skills, knowledge and competences. PES can: (i) support jobseekers and employers by identifying future skills needs; (ii) deliver suitable training, career guidance and support to enable jobseekers to manage their own careers; and (iii) forge partnerships to improve their offer in this area. A working group established by the Network in 2018 concluded that the Network needs to evolve as a learning organisation, while continuing with benchlearning and promoting the principles of knowledge-based development 34 . A continuous learning environment is key to PES transforming their services, and to focusing on empowerment of jobseekers.

5.2. Effectiveness

5.2.1. Quantitative effects of the enhanced cooperation between PES

The Network is considered generally very effective in enhancing cooperation and improving the functioning of PES, in particular through the benchlearning and mutual assistance initiatives. The vast majority of the stakeholders consulted emphasised benchlearning and mutual assistance as the most effective initiatives because of their concrete results. Stakeholders interviewed in the external study believe that the success of the two benchlearning cycles derives not so much from their value as a tool for ranking performance, but rather as a tool for PES to help them learn and improve.

The quantitative performance indicators set in the Annex to the Decision are grouped under four headings: ‘labour market flows’, ‘relations with employers’, ‘customer satisfaction’ and ‘long-term unemployment’. The indicators show that almost all PES can be found at least once among the over-performers, thanks to their achieving better results than what statistically could be expected based on their context, e.g. given the demographics, education level, macroeconomic context, etc. in the country or region that they serve. At the same time: there is still room for improvement.

PES access the quantitative benchlearning indicators via the PES Dashboard. The dashboard is the major tool used to promote transparency between all members of the

34 https://ec.europa.eu/social/BlobServlet?docId=20520&langId=en

Network. The dashboard does not appear to be consistently used by the Network members (see Figure 6 below). Some of the reasons mentioned by PES stakeholders during the external consultation concerned technical aspects, mentioning that IT system security problems limited access to the dashboard, while others referred to the comparability and presentation of the indicators.

The fact that so many different PES have agreed on a set of quantitative and qualitative common indicators for the benchlearning process is in itself an important achievement. Since quantitative information can play a role in the planning of initiatives and reforms, one aim should be to achieve more consistent use of the dashboard among PES.

Figure 6: Number of PES Dashboard sessions per country. March 2018 to February 2019

Source: PES Secretariat/European Commission

5.2.2. Modernisation promoted by benchlearning

The quantitative and qualitative benchlearning results indicate that PES performance is

related to how PES are organised and how services are offered to customers 35 . Evidence

on this relationship has improved the foundation for knowledge-based development, both in national PES and across countries in the Network.

Comparing results from the first to the second cycle of benchlearning demonstrates that the exercise has given guidance to PES on their path towards modernisation and change 36 . Modernisation is being pushed further by the strong focus of the second cycle on change processes and management of these changes. Decentralisation and involving PES, jobseekers and employers and other stakeholders in design and implementation of services are now playing a more important role in setting up services that are suited to the needs of jobseekers and employers. Staff commitment to delivering quality customer services in all PES continued to be high, although unfavourable caseloads persist in many PES, and many PES continue to work under difficult conditions.

Based on the finding, some trends can be summarised:

• Strategic performance management: On the basis of the recommendations made during the first cycle, many PES have critically reviewed their performance management system and especially the structure of their key performance indicators, with the aim of achieving outcome-oriented business development.

• Design of operational processes: the second cycle gave evidence that customer needs and expectations are at the heart of effective modernisation of processes.

35 https://ec.europa.eu/social/BlobServlet?docId=20613&langId=en

36 https://ec.europa.eu/social/BlobServlet?docId=20613&langId=en

Efficient customer segmentation makes it possible to shift resources to those

customers who need in-depth support.

• Sustainable activation and management of transitions: Nearly all PES have

introduced different channels for communication with customers, and digital

tools especially have helped to reduce administrative burden in some PES,

freeing up resources for in-depth counselling of jobseekers with special needs

(long term unemployed, NEETS, etc).

• Relations with employers: The PES are currently developing different concepts

on how to satisfy employers’ needs given an increasing share of difficult-toplace

customers. The Network European Employers’ Day supports PES

visibility in participating countries.

• Evidence-based design and implementation of PES services: The use of pilot

projects has become more prominent since the first cycle, especially for testing

new organisational set-ups, service models and ALMPs for specific groups A

systematic evaluation of practices and communication of results has, however,

only been implemented by a few PES.

• Management of partnerships and stakeholders: The discussion has

intensified as to which PES services should be outsourced and which should be

kept in-house. One-stop shop solutions providing combined social and labour

market services have been introduced in some PES, and others will follow.

Even if it is not possible to reach conclusions on the direct impacts of the various actions, PES maturity has increased overall from the first to the second cycle. The observed trends in findings demonstrate that the systematic benchlearning approach, including the follow-up reports, has resulted in more concerted actions across PES. A clearly defined process starting with the first cycle assessments and recommendations, followed by a change plan, follow-up reports and the renewed assessments in the second cycle, seems to have facilitated commitment to plans and implementation of them.

5.2.3. Mutual learning between PES has improved

As an inherent part of benchlearning, mutual learning has been a key pillar of Network activity, together with benchmarking activities. Mutual learning involves the use of evidence to identify and address PES learning needs. The aim of the Network’s activities in this area has been to help PES learn from employment services with particular expertise or insights in a specific area. Mutual learning offers a key mechanism to support the other broader initiatives, activities and objectives specified in the Decision. For instance, it includes the types of enhanced activity intended to modernise and strengthen PES, and supports the implementation of relevant policy initiatives.

Quantitative and qualitative benchmarking have led to the creation of an evidence base with information on learning needs as well as expertise and good practices to share. This feeds into the development and implementation of the Network’s mutual learning programme. Mutual learning activities focused on several priority areas in 2014-2018, addressing various Decision objectives. In 2015-2016, the focus was on thematic clusters such as young people, performance management and measurement of customer satisfaction, while in 2017-2018 clusters such as digitalisation and services to employers attracted more attention. Activities related to sustainable integration were relatively important throughout the whole period.

2018 saw the introduction of an enhanced approach to monitoring and evaluation of mutual learning activities. The findings, reported in the 2018 annual report 37 , demonstrate that participants are generally very satisfied and that the available resources are useful in the development of changes, initiatives and reforms in PES. Overall, participants gave higher scores to face-to-face learning events than to reports and papers. This can be a reason to explore new channels for dissemination, for example whether videoconferencing can be a useful tool. However, the higher scoring of face-to-face meetings also highlights the importance of networking and discussions with peers and experts in the specific field.

5.2.4. Effectiveness of mutual assistance

Benchlearning is often accompanied by direct mutual assistance (technical assistance through peer PES support to selected PES) offered through the Network. Mutual assistance offers a needs-based learning format responding to individual PES needs. For example, following previous activity linked to the benchlearning exercises, throughout 2015-2018 the Network and its members provided mutual assistance for PES modernisation in Spain, Bulgaria, Romania (Box 3) and in Italy and Cyprus (Box 2), again highlighting the interrelated nature of PES activities supporting the Decision objectives.

Mutual assistance is seen as a highly effective initiative of the Network. At the request of individual PES, two to three mutual assistance projects take place each year. In the external study, representatives both of PES receiving assistance and of those providing assistance emphasised the role this activity plays in raising awareness about PES strengths and weaknesses. The respondents cited the importance of comparing systems and of providing concrete opportunities to learn and design change strategies (see Boxes 2 and 3 for descriptions of outcomes).

Stakeholders consulted for the French case study (the external study Section 3.1) highlighted that mutual assistance activities, as judged by Pôle emploi (France’s PES), are very interesting not only for the PES receiving the assistance but also for the PES providing it. Similarly, the Estonian case study highlighted the exchange of best practice and mutual assistance opportunities as particularly valuable. This illustrates that mature PES (see Annex 1 in the 2018 Annual Report 38 ) can also get inspiration for new reforms by acting as mentors and advisers. The mutual assistance projects should therefore not only be considered as assistance, but also as a learning arena.

37 https://ec.europa.eu/social/BlobServlet?docId=20613&langId=en

38 https://ec.europa.eu/social/BlobServlet?docId=20613&langId=en

5.2.5. PES have taken up the results of Network cooperation

Assessments from peer experts during site visits and their recommendations to PES are an important input and provide inspiration for change in PES. This is borne out by the change reports from the PES one year after the first site visit and by the reports on the follow-up of benchlearning recommendations made in response to the annual reports. The assessors can give advice about other PES that could provide assistance, and some countries have approached peer PES for learning and exchange (see the example of Austria in Box 4 below), demonstrating the effectiveness of the benchlearning initiative.

Box 4. Examples of reform initiatives resulting from the benchlearning initiative 39

The Austrian PES is setting up a system of causal impact evaluations of ALMP measures in order to identify the efficiency of all measures on a regular basis. The Austrian PES organised an exchange at expert level with the German PES to get inspiration from the German ‘Treatment Effect and Prediction’ system.

The Slovenian PES has introduced a ‘Randomized Control Trials’ approach (RCTs) to piloting and evaluation of new services and products. This approach will reduce the need for more extensive empirical methods when evaluating interventions. Through the benchlearning initiative, the Slovenian PES received information about other PES with experience in implementing RCTs.

As a result of activities from the second benchlearning cycle, the Spanish PES is identifying and disseminating best practices. An internal benchlearning and mutual learning project for the regional PES in Spain is the main framework for this sharing of practices.

Source: PES Network Annual Report 2018

In addition to benchlearning, mutual learning activities are part of the Network cooperation. Evidence from internal evaluations of the impact of mutual learning indicates that 70% of respondents used the knowledge they accumulated during the activities and one third declared that the activities have contributed to changes in their PES 40 .

The types of change that have occurred often relate to concrete and specific operational changes. These include:

• development of new measures for young people and the long-term unemployed; • development and launch of evaluations of ALMPs; • redesign of approaches to measuring customer satisfaction; and • redevelopment of web-based resources.

39 Further examples can be found in the European Network of Public Employment Services, Annual Report 2018: https://ec.europa.eu/social/BlobServlet?docId=20613&langId=en

40 ICF — enhanced approach to the monitoring and evaluation of mutual learning activities (see Annex 3 of Annual Report 2018).

Other types of change include:

• an increased knowledge base; • changes to a strategic approach; • changes in priorities among senior or middle management; and • raised awareness of a specific topic or approach.

This overview demonstrates that changes are often directly targeted at services for young people and the long-term unemployed. Other changes could have more indirect impacts on services by increasing the capacity and professionalism of the PES. Change can be initiated not only by mutual learning, but also through other channels such as political decisions at the national level, for example introducing new measures targeted at young people or the long-term unemployed. To implement such changes, learning from other PES can still be important. Even if there are differences, the main conclusion is that PES use mutual learning results extensively.

5.2.6. Network contributions to effective policy implementation

The external study, Section 3.2, demonstrates that Network activities have been most useful in supporting the most vulnerable groups with high unemployment rates and, to a certain extent, the integration of people excluded from the labour market. A large majority of national labour ministries indicated that the Network has been very successful or successful in supporting the most vulnerable groups, while only about one third found that the Network had been very successful or successful in supporting the integration of people excluded from the labour market.

The Network prepares reports at the request of the European Parliament, the Council and the Commission. As stated in the annual reports from the Network, a number of reports and analytical papers are published, for example on themes related to the long-term unemployed, young people and support to vulnerable groups. Labour ministries were asked by the external contractor (see Section 3.1 in the external study) how they evaluated the Network’s reports. A total of 5 ministries found this activity very successful, 7 found it successful, and 2 somewhat successful, while 2 ministries did not offer any assessment. Overall, the responses from the labour ministries confirm that reports prepared by the Network have proved useful.

The evidence gathered that the Network is providing evidence-based information to the European Semester process, indicating PES’ progress and fostering well-informed dialogue with the Employment Committee. This dialogue is important in developing EU employment policy which is applicable to Member States’ needs, for example to address the increased focus on policies to integrate people excluded from the labour market.

5.2.7. Potential for improving effectiveness

Limited internal PES capacity, high staff workloads and infrastructure problems were seen as problematic by some PES consulted in the external study (Section 3.1). Despite technical and financial support at EU level, some PES have limited financial and human capacity to balance national and international commitments, particularly those PES with less resources from national governments. Moreover, many consultees stressed the influence of political decision-making as a potential barrier in cases where the political will is not present to encourage sustainable change. Some PES have experienced difficulties in demonstrating the benefits of the Network activities to their national governments in quantifiable terms.

Changing national priorities influence whether change can take place at the national level. This is demonstrated by a combination of findings from the external consultation and the reports from PES on follow-up of benchlearning recommendations. Thus, for a mutual learning activity to be effective (meaning that change is achieved) it is important that it comes at the right time. Effectiveness also depends on the ability to find a balance between launching promising proposals and keeping new ideas within what could realistically be realised. This balance differs across PES, and can change over time. The benchlearning exercise has been important to improve evidence for designing better measures and recommendations at EU level that are also targeted at each PES. The 2017 interim review of the Decision highlights the benchlearning project as a positive innovation in that national institutions can have external references, receive constructive feedback from expert practitioners and engage in tailor-made learning with peers.

The evidence gathered by the various consultation activities for this evaluation and reports from PES 41 demonstrate that benchlearning and mutual learning activities have contributed to concrete changes in PES organisational processes and service delivery, thus rendering them more effective. These changes have the potential of contributing further to: (a) improving PES performance; and (b) effectively implementing employment policies at the national and local levels.

5.3. Efficiency

5.3.1. The Network has been efficient in delivering initiatives and reaching objectives

The external study, Section 4.1, confirmed that the Network is considered both to be efficient and a good use of national (or regional) and European public resources by the vast majority of actors consulted both at EU level and in national PES (almost 100% of those who expressed an opinion). This view was also held by representatives of international organisations consulted, who felt that the Network provided a key forum where practical issues for PES performance and service delivery can be discussed and ensure efficiency gains by learning from others. However, the highest proportion of respondents in the public consultation was unable to determine the cost-effectiveness.

The PES Secretariat has proven to be an important link between the Commission and the Network. The PES Secretariat consists of Commission staff and Seconded National Experts from European PES. This legitimately supports the Commission by being a source of information from and contacts with the Network. This close link is also contributing to the efficiency of the Network .

41 https://ec.europa.eu/social/BlobServlet?docId=20613&langId=en

5.3.2. Costs

Examining efficiency requires an assessment of the degree to which the objectives and activities of the Network have been delivered at an appropriate cost. This implies a need to assess the costs and benefits associated with the Network. However, many of the benefits in particular do not lend themselves to monetary valuation. Moreover, a four-year period is in some cases too short to realise the full gains of organisational or cultural changes that have been set in motion. A full cost-benefit analysis can therefore not be carried out.

The external contractor asked the national members of the PES Network to indicate time and other costs used in participating in Network activities (external study Section 4.2). Estimates were often approximate and some consultees did not want to share them or were not able to obtain information on costs. Furthermore, some respondents found it difficult or impossible to separate time used in Network activities from other activities, particularly when related to European or international tasks. The estimates can therefore only give indications of the use of resources.

Moreover, estimations of time investments by PES differed quite widely by PES. There are also real differences across countries, depending on the involvement in the Network that should be considered when analysing the average.

On average, the PES estimated that around 200 days of staff time per year were committed to Network activities. This approximates to one full-time equivalent per PES per year. Concerning costs such as travel, accommodation, translation and expert inputs, PES considered them hard to estimate as some of them were reimbursed by the Commission, while others could be merged into other budget items.

At European level, the average annual cost is around €3.45 million per year, incurred by the European Commission and funded by the Employment and Social Innovation (EaSI) programme. This amount includes: (i) the annual budget to implement the Network’s work programme (€3.12 million); (ii) Commission staff devoted to the PES Secretariat, allowances paid to Seconded National Experts 42 , mission costs for the Secretariat and for Commission representatives to the Board (about €200,000); and (iii) costs for the organisation of meetings (Board and AFEPA meetings) and reimbursing the participants (about €130,000).

These resources are in accordance with Article 7 of the Decision. In addition, the Network Secretariat liaises with the AFEPAs and other experts at national level with a view to hosting and participating in learning events, data collections, etc. Compared, for example, to the total direct support from the ESF to reforms to modernise PES (€2.1 billion over the period 2014-2020), the support for this Network is small.

Other networks such as the EURES Network, the European Social Policy Network (ESPN) and the European Labour Law Network are also receiving allocations from EaSI.

42 Much of the cost of Seconded National Experts is borne by the national PES and in principle included in the national PES cost estimates.

This gives a scale of the financial support involved. In 2018, EaSI allocations to the EURES Network’s tasks related to network meetings, training and support activities was €4.45 million. EaSI allocations in the field of labour law and employment and labour market policies amounted to €2.2 million to ESPN, while the European Centre of Expertise (ECE) received €2.0 million. However, it remains difficult to properly compare the costs of these networks due to the difference in their tasks and responsibilities. For example, the EURES Network organises training for advisers and line managers. As seen above, a substantial task for the Network covered by this evaluation is mutual learning across national PES. ESPN provides the Commission with an overview of policies on social protection and social inclusion, and acts as the secretariat to the Mutual Information Systems on Social Protection (MISSOC). The ECE is a network of labour law experts and provides an ad hoc capacity in labour market policies analysis through a flexible pool of experts. It assists the Commission in ensuring a correct application of EU law across all Member States and monitoring labour legislation reforms in the EU.

5.3.3. Benefits

PES performance and impacts for the users of PES services depend on many different factors. Impacts on PES performance can be indirect, for example through changes in organisational culture, which is normally also affected by a number of other measures and channels. Furthermore, the causality of improvements, or how objectives have been achieved, cannot always be established. For instance, reduced unemployment can be a consequence of better PES performance, but also of changes in demand for labour.

However, many PES identified in the external study cases where they felt that their PES and the national labour market had benefited from participation in the Network, for example:

• new initiatives or processes introduced following Network activities; • participation in the Network facilitated bilateral relationships between PES; • Network participation stimulated internal learning, reflection and policy debate; • Network participation enhanced PES staff development, motivation and engagement.

Many consultees highlighted in the external study that benchlearning and associated visits had a significant impact on helping individual PES understand how they compare with other PES, how they can plan for the future, and how they can evaluate their own activities. It is true that some of the examples mentioned in the consultations, such as new tools and development of staff, could to some extent have been implemented by the use of private sector consultants. This can, however, be costly. Moreover, such support is unlikely to be as relevant as the learning that can be obtained from experienced peers from other PES. Less advanced PES have benefited the most from participating in the Network in terms of progress in performance (see Figure 2). However, comparing maturity levels between the first and second cycles of benchlearning demonstrates that advanced PES have also progressed.

5.3.4. Balance of costs and benefits

The benefits of the Network’s activities cannot be directly estimated, neither at the EU level nor the national level. How the Network members themselves view the benefits, however, is borne out by the willingness of PES to participate in benchlearning and mutual learning events and by the PES responses in the external consultation.

In the external consultation (Section 4.3), 26 PES responded to the question about the costs and benefits of participation in the Network. Of these, 14 PES stated that benefits significantly exceeded costs. A further 5 stated that benefits slightly exceeded costs, while 3 reported that costs were equal to benefits. Only 1 PES felt that costs exceeded benefits. 3 PES responded that they were not sure.

One of the respondents stated:

‘To our knowledge, the EU PES Network absorbs only a very limited amount of EU public resources, but coordinates more than one million public employees who are helping vulnerable and jobless across the Europe. With its activities aimed at better provision of quality services and efforts to improve internal efficiency, we believe that it is good value for money and HR spent.’

These results suggest that the ‘leverage’ of the Network is high, in that it yields benefits which considerably exceed the costs of participation, even if not equally distributed. The impact of scale was mentioned in the consultation as contributing to leverage. Participation in the Network means joining a vast network of PES employees across Europe.

Less mature PES are more likely to report net benefits of the Network participation than the more advanced ones. In some cases, where national budgets would be a barrier to Network participation, the financial investment from the EU was mentioned as a key contributor to the Network’s leverage. This indicates that the contributions from the EU are important both to ensure that the mature PES do not see Network participation as a burden and to allow the less mature PES to participate in activities which are crucial for their further development.

5.3.5. Potential to increase efficiency

Some potential means to further improve efficiency were mentioned in the external study, Section 4.1. These relate mainly to better targeting of learning events and potential to further use digital solutions.

Some consultees mentioned that study visits could be organised in smaller and more targeted groups, and to increase the focus on training of PES employees so that they can have a greater impact on Network activities on the ground. Finding solutions for the translation of materials, tools and guidelines is mentioned as a means to increase take-up of outputs by staff in local PES who may not speak English.

A result of two cycles of benchlearning is the accumulation of an evidence base on the maturity and priorities of PES. This evidence base has made it easier to target learning events. This has paved the way for learning dialogues, a new format of learning event starting in 2019. This peer-coaching approach related to specific PES learning needs is designed to support national PES as they bring about change in an identified area for improvement.

Greater efficiency could be obtained by reducing language barriers. Some consultees mentioned that many PES staff cannot read outputs in English. Linked to this, it can be difficult to find PES staff proficient in English who are experts on certain topics and who can attend Network activities. Lack of internal capacities as well as language barriers between experts can restrict the possibility to draw benefits from mutual learning. A higher focus on smaller, targeted events could therefore increase efficiency. Calls for experts to attend mutual assistance projects could, for example, put the emphasis on their knowing the native language in the receiving country.

It was also mentioned that there could be a greater focus on dissemination, which might increase impacts, by targeting and distributing outputs more strategically. Efficiency can be increased by making better use of modern e-solutions and online interactive tools for communication, and through the exchange of best practices and dissemination of outputs. Two webinars were organised as part of the 2018 work programme and three webinars will be organised in 2019. However, as pointed out by several consultees during the consultations, personal contact between PES staff is the key to the success of the cooperation. It is therefore important to find the right balance between face-to-face and virtual meetings to achieve the efficiency goals.

5.4. Coherence

5.4.1. Consistency of the Decision with EU policies and initiatives

The preamble of the Decision refers to the links between the Network’s responsibilities and other EU policies. Specifically, the Network should:

• contribute to the implementation of policy initiatives in the field of employment; • support initiatives aimed at skills matching, decent and sustainable work,

voluntary labour mobility and transition from education and training to work; • address the evaluation of active labour market policies;

• ensure that it complements and does not replace or duplicate other EU initiatives. Consistency between legal frameworks as well as implementation is thus important.

The Europe 2020 strategy is the EU’s agenda for growth and jobs for the current decade. One of the goals is to increase work participation to a rate of 75% for people aged 20-64. In the Council Decision on guidelines for the employment policies of the Member States, guideline 7 declares that the Member States should aim for better and more effective public employment services to reduce and shorten unemployment by providing tailored services to support jobseekers, supporting labour-market demand and implementing performance- measurement systems. The PES Decision clearly states in Article 3 that it aims to “encourage cooperation between Member States [...] in order to contribute to Europe 2020” and, in Article 4, that it should “contribute to modernising and strengthening PES in key areas, in line with the employment and social objectives of Europe 2020”.

The new legal basis for EURES laid down in 2016 43 reinforces the obligations to share information and cooperate across Member States in areas such as apprenticeships. It also lays down a minimum set of support services that PES have to provide. EURES focuses on the pooling of job vacancies and applications and the provision of mobility services in view of facilitating free movement of workers and the further integration of labour markets. The external study explains that whereas the Network focuses primarily on cooperation to improve PES performance and contribute to implementation of employment policies, EURES directly targets employers, workers and jobseekers. To avoid duplication and ensure synergies, the Network Board decided in 2016 to promote closer cooperation between the Network and EURES.

The Network has demonstrated the ability to adapt to new legal frameworks for EU policies. The dedicated and relevant chapters of the European Pillar of Social Rights, which was proclaimed by the European Parliament, the Council and the Commission in 2017 (equal opportunities and access to the labour market, fair working conditions and social protection and inclusion), which are aligned with the guidelines from the European employment strategy, have become guiding in setting the Network’s initiatives and actions. An example of this is the increased focus on the role of PES in outreach to the inactive population 44 .

The tasks of the Network also include monitoring implementation and carrying out analytical activities such as preparing reports and studies. The Network has established its identity and presence at European level and helps national PES to perform better and deliver employment policies more successfully, taking into account the impact of labour market trends. The Network’s activities have also contributed to better analysis as part of the European Semester and the country-specific recommendations (CSRs) it produces. In 2018, 11 Member States received CSRs related to their PES and active labour market policies (ALMPs). This includes needs for more personalised tailored services for the unemployed, a more comprehensive performance measurement system, better targeting and prioritisation, and better coordination between central and regional administrations. The CSRs focus for some Member States on measures to make ALMPs more effective and accessible to those furthest away from labour market.

To address these challenges identified in the European Semester, the PES Network uses its benchlearning approach, striving for continuous performance improvement. One of its unique features is the attention given to organisational factors, drivers and practices that are likely to improve PES performance.

43 Regulation (EU) 2016/589 of the European Parliament and of the Council of 13 April 2016, OJ L 107, 22.4.2016, p. 1-28

Overall, the external study, Section 5.2, revealed that most stakeholders consider that there is a high degree of consistency between the Decision and the EU policy framework. Although improvements could be made in future to improve synergies, no duplication of action was observed. There is some cooperation with other relevant labour market stakeholders (as set out in Article 5 of the Decision) at EU level, but there is scope for improvement in future.

In particular, a large majority of the PES consultees pointed out that the Decision reflects the relevant priorities set out in the Europe 2020 strategy. This complementarity was also emphasised in relation to the shared goals and commitments included in the Decision and other European initiatives such as the Youth Guarantee, the Youth Employment Initiative, the Council Recommendation on the integration of the long-term unemployed into the labour market, and ESF-funded programmes. Synergies were highlighted in relation to the mandate of other EU institutional actors such as the EURES Network and EMCO. The fact that different institutional actors and networks share similar core objectives — e.g. reducing unemployment, improving labour market policies’ effectiveness and the inclusion of vulnerable groups — was acknowledged and seen in very positive terms by the majority of those consulted. The facilitation of information exchange and best practices across institutions and networks at the EU and national level was likewise deemed as particularly important in facilitating complementarity and coherence.

5.4.2. Contributions of the Network to relevant EU policies and initiatives

The Network has contributed to implementation of a broad range of EU policy initiatives. Table 1 in Section 3.6 gives an overview of selected examples of the Network’s contributions to policy implementation. The Network’s contributions have been most extensive in implementation of the Youth Guarantee and the integration of the long-term unemployed into the labour market. The Network has also demonstrated flexibility to target new topics such as the integration of migrants and refugees, the prevention of unemployment and addressing skills shortages. The Network has also provided contributions to implementation of the European Pillar of Social Rights.

As regards the contributions of the Network to the implementation of the CSRs under the European Semester, PES are continuing to innovate in their development of ALMPs, while general ALMP expenditure is increasing at many PES. Moreover, PES are adapting their staff deployment to the changing environment. This includes specialised teams offering tailored approaches and intensive support to target groups such as the long-term unemployed and customers with disabilities. For example, in 2017 Spain received a CSR to reinforce coordination between regional employment services, social services and employers so as to better respond to jobseekers’ and employers’ needs. In this context, the Spanish PES requested a mutual assistance project from the PES Network to help it work towards addressing the CSR.

5.4.3. Large degree of complementarity with the EU policy framework

Other stakeholders confirmed the high degree of complementarity with the EU policy framework. There is well-functioning cooperation between the Network and other bodies within the EU. As an illustration, there are various forms of contact with EMCO through the Committee’s role as an observer in the Network, through presentations by the Network Chair at EMCO meetings and through contacts between the PES Secretariat and the EMCO Secretariat.

However, some concerns were also raised in the consultations about the potentially overlapping mandates of the Network, the EURES Network, EMCO and — potentially — the European Labour Authority. Overall, there was agreement about the need for effective cooperation between all relevant bodies at EU and national level to ensure that common objectives are achieved and that outcomes are sustainable. However, it was felt that while further upfront coordination could avoid some overlaps, most overlaps of topics tend to be mutually reinforcing and can provide synergies, thus addressing key topics (e.g. integration of migrants) from different perspectives and with different types of actors. There was no evidence found of duplication of actions with other EU policies or initiatives.

PES in all Member States are in one way or another involved in implementing the ESF. The strength of the links between national PES and national ESF programming and funding varies between countries. For instance, in several countries (e.g. Bulgaria, Croatia, France, Luxembourg, Hungary, Lithuania and Romania) the PES actively cooperates with the ESF managing authorities (Ministry for Social Affairs, Ministry of Labour or other public authorities) and receives funds for their operational programmes. In a few other countries (e.g. Austria and Greece), the influence of the PES on ESF funding and programming is more limited, due for example to the different parameters for cooperation with ESF managing authorities or because PES do not use ESF funds to implement their initiatives, measures or programmes.

5.4.4. Potential to increase consistency

To reinforce the complementarity of activities of the Network with relevant policies and initiatives, the Network could reach out more widely to other relevant labour market stakeholders as envisaged under Article 5 of the Decision. These could include other providers of employment services, social partners and NGOs working in the field of employment. While such cooperation has been observed in certain cases at national level, and established sometimes formally at EU level, it does not yet appear to be extensive across all countries or at EU level. Building close links between PES and focusing discussions on issues of particular relevance to PES has most likely been beneficial in this relatively early phase of inter-PES cooperation. However, it could be beneficial, as the cooperation deepens, to explore whether other actors such as relevant EU-level NGOs, social partners and networks should have more structural links with and roles within the Network. While the Chair already has a role representing the Network to other stakeholders, the external role of the Chair could be further developed.

5.5. EU added value

5.5.1. Impacts of the Decision

The findings in the external study, Section 6.2, demonstrated a number of significant impacts, outputs and results of the Decision which did not occur before the Network was set up in 2014 and would not have been achieved by voluntary cooperation of national PES, confirming the added value of the PES Network:

• a structured framework, via the benchlearning process, for assessing PES performance and capacity and for facilitating comparisons and improvements;

• increased and more structured cooperation and peer learning between PES to tackle labour market needs across the EU;

• greater ownership, collegiality and buy-in by all national PES, in particular via their role on the PES Network’s Board and in approving annual work programme;

• more effective assessment and identification of the needs and areas of development for individual PES capacity and performance via the benchlearning process;

• the provision of appropriate individual support and collective mutual learning for PES based on precise needs identified through the benchlearning assessments;

• a greater understanding among EU policymakers of the needs and challenges of

national (and regional) PES, leading to more effective policy solutions;

• ensuring a collective voice and formal platform for PES in policymaking at EU level in relevant fields through, for example, the monitoring of the Council recommendations establishing the Youth Guarantee and on the integration of the long-term unemployed, and through contributions to relevant policy initiatives such as the European Pillar of Social Rights;

• contributions to the EU2020 objectives by providing targeted support to PES and cooperating with EMCO (and the labour ministries) to identify those which require additional support;

• a common strategic framework for PES cooperation to tackle the increasing common global challenges that PES are facing, which require concerted action.

As regards the impact at national level, the size and scale of the added value differs across countries, but the vast majority of PES reported that participation in mutual learning events has been useful for them. The evaluation of the mutual learning events 45 indicated that for 20% of the respondents, their participation in the mutual learning activities was critical to initiate and support changes. Over half of the respondents considered that the changes would have happened anyway, but less efficiently. 27% of the respondents considered that the changes would have happened in the same way without their participation in the mutual learning activities.

The benchlearning approach has attracted interest outside of the Network, and contributed to value added also beyond the EU level. A simplified version of benchlearning is being implemented in Western Balkan countries. In addition, the World

45 Service contract for measures to enhance cooperation between Public Employment Services (PES), in particular services to implement a ‘benchlearning’ concept within the PES Network (VC/2015/0062) - Final Evaluation Report – Year 4 (annex available on request).

Association of Public Employment Services (WAPES) is considering developing a version to be used on a global basis.

5.5.2. The role of the Network in cooperation between PES

PES representatives in particular confirmed in the external study that information sharing about PES practices would not have been as effective and systematic in the absence of the Decision (see Section 6.3 in the external study). Furthermore, without the Network’s existence, Network activities such as the self-assessments and external assessments carried out within the benchlearning process, mutual assistance visits and other opportunities to compare different practices and models are unlikely to have been implemented. The Network is a mechanism that ensures the continuity of collaboration in a way that includes all its members. While many consultees considered that some PES would continue to cooperate in the absence of a formal Network, they also stressed that cooperation was unlikely to be maintained with the same frequency, depth, structure and commitment from all participating PES. Representatives from PES with less financial and human resources, and those facing the greatest challenges in the development of their PES, underlined that in the absence of the Network they would not be able to participate to the same degree in cooperation with other PES. Failing to continue the initiative was also seen as a loss of the resources invested to date.

Due to the Decision, the Network obtained a formal role in organising cooperation and contacts with other stakeholders. This commitment and support of resources provided at EU level has contributed to ensuring the effective implementation of the Decision, in terms of both financing (funding for Network initiatives including meetings, events, benchlearning expertise) and human resources (the PES Secretariat, contractors) invested in the running of the Network. Comparison with the situation before 2014 as well as responses from PES in the external study are strong indications that a formal commitment to continuing to provide resources for the Network is important to maintain the scope, extent and quality of PES cooperation and the Network’s ability to implement strategic EU employment goals.

5.5.3. Potential for increased added value

The 2017 interim review of the Network activities 46 emphasised that PES are stronger

when they speak with one voice, and that the Network has been productive in its output. Many of these outputs have a wider benefit for researchers and decision-makers. The review confirmed that greater visibility of the Network and its outputs would ensure that these resources are used effectively. The external study also highlights the potential for improved visibility of the added value of PES (Section 7.2). An agreement from the Board to allow benchlearning data to be published, would strengthen the business case for reform, and further increase the added value of the Network for achieving EU policy objectives.

• 6. 

  C ONCLUSIONS AND LESSONS LEARNED

Relevance

The Decision remains highly relevant for PES. The objectives of the Decision, as set out in Article 3, and the initiatives set out in Article 4, encompass the key areas of PES responsibility and provide a robust framework for the policy and concrete activities of the Network.

How PES can contribute to the better functioning of the labour market in the changing world of work has been addressed more prominently in the Network’s work programme over the past years. These challenges are also having an impact on how the objective of decent and sustainable work can be achieved, as new working conditions are resulting in employment opportunities in new technology sectors, the large-scale digitalisation of many professions, more short-term flexible working and a growing trend towards selfemployment and platform jobs. By providing a broad framework for Network activities, the objectives enable a flexible approach to addressing priorities for action depending on labour market developments.

Effectiveness

The evaluation reveals that the Network has been effective in delivering its initiatives and objectives. PES have taken up the results of PES cooperation, and the PES have increased their level of maturity as organisations. The Network’s effectiveness is illustrated out by the strong endorsement and implementation of the successful benchlearning concept, as well as by the success of knowledge sharing through mutual learning events, reports and PES practices.

Benchlearning, including the learning dialogues, has proven to be a successful example of systematic evidence-based learning, which could be transferable to other policy fields at EU level. The fact that so many different PES have agreed on a set of quantitative and qualitative common indicators for the benchlearning process is in itself a significant achievement.

The benchlearning project has been a key success factor in the establishment of a structured process for learning and sharing of practices. The Network has served as a mechanism ensuring the commitment and participation of all its members. This extensive cooperation, which includes the benchlearning process, mutual assistance visits, and mutual learning activities to share and explore different practices, are unlikely to have been implemented without the Network.

Efficiency

The Network has been efficient in delivering initiatives and reaching objectives. Some Network activities are difficult to quantify, and four years is also a short period to realise certain gains and make them visible, such as changes in organisational culture. However, the willingness of PES to participate in benchlearning and mutual learning events, as well as positive feedback from PES demonstrate that the Network outputs are highly effective and appreciated by all Network members. Overall, less advanced PES have benefited more from participating in the Network in terms of the progress they have made on performance, but advanced PES have also progressed.

Coherence

The evaluation reveals that there is a good degree of consistency between the Decision and the EU policy framework (for instance with concrete policy initiatives, such as the recommendations on the Youth Guarantee and the integration of the long-term unemployed into the labour market). No duplication of action is observed. As an example, whereas the Network focuses primarily on cooperation to improve PES performance and contribute to implementation of employment policies, EURES directly targets employers, workers and jobseekers.

The Network has contributed extensively to the implementation of EU relevant policies and initiatives in various areas and by different means, and has established a collective voice and a formal platform in policymaking at EU level, while making contributions to the EU 2020 strategy for jobs, and smart, sustainable and inclusive growth.

The Network’s contributions have been most extensive in the long-standing issues of implementation of the Youth Guarantee and the integration of the long-term unemployed into the labour market. The Network has also demonstrated flexibility to target new topics such as the integration of migrants and refugees, the prevention of unemployment and addressing skills shortages. Moreover, the Network supports national PES as they implement country-specific recommendations addressed to the Member States on active labour market policies (ALMPs) and PES issues in the context of the European Semester.

EU value added

An added value of the PES Network has been a structured framework for assessing PES performance and capacity and for facilitating comparisons, peer learning and improvements. The evaluation demonstrates several other results of the Decision that would not have been achieved by voluntary cooperation of national PES. Some important achievements are greater ownership, the provision of individual PES learning and collective mutual learning, ensuring a collective voice and a formal platform in policymaking at EU level, and contributions to the EU2020 objectives. Some activities, such as reports or conferences, are able to reach stakeholders beyond the Network, for example researchers, decision-makers, social partners, NGOs and international organisations. The Network also plays an important role in representing the unique perspective of PES in the design, implementation and monitoring of the success of policies relevant to labour market integration and retention. Nevertheless, there is potential to develop stronger relationships with social partners, NGOs and ESF managing authorities, and at the international level in order to increase synergies and mutual benefits.

To date, there is no information available on how cooperation across PES could be organised if the Decision is not extended. It is, however, likely that information sharing would have been less effective and systematic in the absence of the Decision. Although some cooperation is likely to continue without the Network, the formal framework for cooperation that the Decision supports, as well as the Commission’s technical and financial support, are considered by stakeholders as crucial.

Lessons learnt

Overall, the evaluation shows that the Decision has been successfully implemented and some lessons learnt could be identified. The persisting difficulties that vulnerable groups face when trying to (re-)enter the labour market demonstrate the continued relevance of the objective of supporting vulnerable groups with high unemployment rates.

Less advanced PES have benefitted more from participating in the Network in terms of the progress they have made on performance, but advanced PES have also progressed. The evaluation clearly underlines the importance of continued focusing on learning and sharing experiences. The recently launched learning dialogue is a promising new learning format which actively supports the national reform agenda via peer PES coaching. It demonstrates the evolution of individual peer PES learning and the growth of the Network as a learning organisation itself. Potentials to further improve efficiency of the Network include focusing on smaller and more targeted learning events and the potential for further use of digital solutions.

Since quantitative information can form part of the planning of initiatives and reforms, one aim should be to ensure that PES make more consistent use of the PES Dashboard, where the quantitative benchlearning indicators can be accessed. This could be achieved by allowing benchlearning data to be published, as it would strengthen the business case for reform and further increases the added value of the Network for achieving EU policy objectives.

The PES are also key actors implementing the European Social Fund, and play an active role in the development of a comprehensive strategy for a policy framework on ALMPs, thereby contributing to the effective and efficient spending of EU funds.

Financial, organisational and expert support from the EU level is essential to ensure the ongoing participation of all PES. Without such support, some countries could face difficulties participating, while commitments to long-term activities could become difficult, and cooperation could be more fragmented in terms of themes.

Annex 1: Procedural information

Organisation

DG Employment, Social Affairs and Inclusion (DG EMPL) is the lead for the evaluation of Decision No 573/2014/EU on enhanced cooperation between Public Employment Services (PES).

The evaluation started with the publication of the evaluation roadmap 47 on 27 October

2017. The evaluation was carried out with the support of the interservice group chaired by DG EMPL, to which the following Directorates-General were invited: ECFIN, HOME, RTD and SG.

Evidence, sources and quality

Both internal and external expertise were deployed to ensure the quality of the evaluation and the staff working document, applying the triangulation method to cross-check the quality of the findings (see the methodology section below). The main sources are listed below (other secondary sources are cited throughout in the text):

• External study carried out by Ecorys 48 . The quality of the deliverables was endorsed by the inter service group.

• Commission report on the PES Network Decision to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions adopted on 6 June 2017 49 .

• Annual reports of the Network for the years 2014-2018 50 .

• The outputs produced by the Network (thematic and analytical papers, PES practices, success stories, manuals, toolkits, working papers, study visits, etc.)

published on the Knowledge Centre 51 .

• Satisfaction surveys and other monitoring activities conducted by the contractors supporting the implementation of the Network’s work programme.

• Yearly survey and report on PES capacity for the years 2014-2018 52 .

• Yearly reports on PES implementation of the Youth Guarantee available for

2014-2017 53 .

47 https://ec.europa.eu/info/law/better-regulation/initiatives/ares-2017-5258712_en

48 https://publications.europa.eu/en/home

49 http://ec.europa.eu/social/BlobServlet?docId=17834&langId=en

50 https://ec.europa.eu/social/main.jsp?catId=1163&langId=en

51 https://ec.europa.eu/social/main.jsp?catId=1163&langId=en and https://ec.europa.eu/social/main.jsp?catId=1206&langId=en

52 https://ec.europa.eu/social/main.jsp?catId=1163&langId=en

53 https://ec.europa.eu/social/main.jsp?catId=1163&langId=en

• Annual data collection, validation and analysis gathered via the benchmarking exercise for the years 2015-2018. Overall assessment reports from the two cycles of benchlearning visits to the PES and their change reports one year after the first site visit.

• EMCO thematic reviews of PES/ALMPs and the European Semester 54 .

• ESF synthesis report of the annual implementation report for 2017 55 .

54 https://ec.europa.eu/social/main.jsp?catId=115

55 http://ec.europa.eu/esf/home.jsp?langId=en

Annex 2: Stakeholder consultation — Synopsis report

The main goal of the stakeholders’ consultation was to collect data, experiences and opinions on the relevance, effectiveness, efficiency, coherence and EU added value of Decision 573/2014/EU on enhanced cooperation between Public Employment Services (PES) from relevant stakeholders groups and the general public. This was done in line

with the Better Regulation guidelines 56 , which mandate transparent and inclusive

policymaking by offering interested citizens and stakeholders the opportunity to provide feedback on the design of polices and their evaluation methods.

This annex presents the consultation strategy, activities undertaken, overview of the consulted stakeholders, the analysis process, the dissemination strategy and overview of the results of the consultation. Activities and analysis were mainly conducted under the

external study 57 .

• 1. 

  C ONSULTATION STRATEGY

The Commission published an evaluation roadmap 58 to provide information on the content and the strategy of the evaluation of Decision 573/2014/EU.

The roadmap was available for public feedback from 27 October until 24 November

2017. It yielded two responses 59 from NGOs. This feedback, which was rather brief,

supported the initiative and expressed a desire for the initiative to continue post-2020. Therefore it is not commented on further.

The strategy in the roadmap made provision for the consultation activities and the stakeholders to be targeted. These are detailed below.

1.1. Identification of stakeholders

The main stakeholders identified were the members of the Network: the PES of the 28 Member States and PES from EEA countries. Both Board members and Advisers for European PES Affairs (AFEPAs) were consulted, as well as the Employment Committee (EMCO) as an observer to the Network. Other main stakeholders included the Network of Member State Labour Ministries, EU-level organisations representing private employment services and temporary work agencies, the EURES Network and the World Association of Public Employment Services (WAPES).

Stakeholders not covered by any targeted consultation were encouraged to take part in the public consultation. Examples of stakeholders falling into this category include social partners, local and regional authorities in Member States, civil society organisations, job seekers and citizens, and researchers and academia.

56 https://ec.europa.eu/info/law/law-making-process/planning-and-proposing-law/better-regulation-why-and-how_en 57 Study supporting the evaluation of Decision on enhanced cooperation between Public Employment Services, Ecorys

for the European Commission, 2019 58 https://ec.europa.eu/info/law/better-regulation/initiatives/ares-2017-5258712_en 59 https://ec.europa.eu/info/law/better-regulation/initiatives/ares-2017-5258712/feedback/add_en

The detailed description of the different groups of stakeholders for each consultation activity is outlined in Section 3.

1.2. Consultation activities

The strategy in the roadmap indicated that the evaluation of the Decision would involve targeted consultations of the main stakeholders and a 12-week public consultation in German, French and English, accessible via the Commission’s central public

consultations page 60 . Replies could be made in any of the 24 official EU languages.

Detailed activities included the following:

• public consultation;

• targeted consultations including:

o targeted semi-structured interviews with a broad range of Network members and other stakeholders at the national, EU and international levels;

o targeted questionnaires for ESF managing authorities, labour and social affairs ministries; and

o an evaluation workshop with the AFEPAs

• case studies (in five countries — Estonia, France, Italy, Netherlands and

Romania) which also included consultations with key stakeholders in the form of

in-depth interviews and focus groups.

The consultation approach was suitable for the scope of the consultation and implemented as planned. The purpose of these consultation activities and the range of participants are described in the next section.

• 2. 

  C ONSULTATION ACTIVITIES

This section presents the consultation activities carried out for the purpose of this evaluation under the external study.

2.1. Public consultation

The public consultation was launched on 20 September 2018 as an online questionnaire

on the Commission’s public consultation website 61 . It remained open until 13 December

2018, by which time the consultation had accumulated a total of 126 respondents.

60 https://ec.europa.eu/info/consultations/public-consultation-enhanced-cooperation-between-public-employment href="https://ec.europa.eu/info/consultations/public-consultation-enhanced-cooperation-between-public-employment-services-1_en">services-1_en

61 https://ec.europa.eu/info/consultations/public-consultation-enhanced-cooperation-between-public-employment href="https://ec.europa.eu/info/consultations/public-consultation-enhanced-cooperation-between-public-employment-services-1_en">services-1_en

The consultation was open to any interested party and to the general public. Care was taken to ensure that a range of key viewpoints were represented, going beyond the close circle of those familiar with the 2014 Network set-up, and to encourage participation by key stakeholders not expressly targeted by the targeted consultations. For that purpose, the public consultation was further disseminated and promoted by sending targeted

emails to key representatives of identified organisations 62 and asking them to share the

notification with the members of their organisations or networks through their email distribution lists, websites and social media channels.

2.2.Targeted consultations

Targeted consultations were held with key stakeholders, with the aim of producing more detailed inputs on issues closest to the stakeholders’ experience and involvement with the Network. The targeted consultations consisted of four main activities: (i) interviews with key stakeholders; (ii) written questionnaires; (iii) workshops; and (iv) consultation activities in the context of five country case studies (focus groups and interviews) conducted as part of the external supporting study.

2.2.1. Targeted semi-structured interviews

Interviews were conducted with:

• members of each of the 32 PES involved in the Network 63 (when possible both

with the AFEPAs and the Board Member);

• selected representatives of relevant EU-level organisations and bodies (e.g.

EMCO, EU-level private employment services and temporary work agencies, the

European Lifelong Guidance Policy Network);

• selected representatives of relevant international organisations (e.g. ILO, OECD,

World Bank, WAPES);

• other stakeholders closely involved in the running of the Network (e.g. the PES

Secretariat, contractors);

• a sample of individuals who were former members of the Network or involved

with PES cooperation pre-2014.

62 These were: social partners at EU and national level, employers, local/regional authorities, organisations or networks representing

the interests of different groups of jobseekers, members of academic/research networks, citizens.

63 The PES Network includes PES from each of the EU-28 Member States, as well as Iceland and Norway (30 countries in total).

However, since in the case of Belgium there are three participating PES, the total number of PES participating in the Network is 32.

The targeted semi-structured interviews were conducted from September 2018 until January 2019 to best accommodate stakeholders’ availability and ensure a large and representative pool of responses across the Network as well as from external stakeholders. One topic guide was developed for interviews with PES Network Board members and AFEPAs, while a separate topic guide was prepared for other stakeholders (e.g. EU-level and international organisations), which could be adapted to each interviewee. A third topic guide was subsequently created for Network interviewees who asked to do the interview in writing.

In most cases, the conversations were digitally recorded, with respondents’ prior consent, as an aide mémoire for the interviewer and to facilitate detailed write-ups. The interviewees were informed that their feedback was confidential, and interview conversations were recorded only with the interviewees’ explicit approval. In some instances, stakeholders preferred not to be recorded; in these cases, notes were taken.

The subject fields covered by the topic guide aimed to encompass all five evaluation criteria (relevance, effectiveness, efficiency, coherence and EU added value). Within this context, retrieving information for the efficiency criteria was challenging as very little information about labour (days invested in the Network activities) and non-labour costs (other costs such as travel, translation, event organisation) was readily available to the PES stakeholders interviewed. Consequently, following the interim meeting, a short follow-up with each PES was conducted, asking three cost-efficiency related questions. The questions were sent via email in early December; after follow-up, the response rate was very high, with 30 PES out of 32 submitting their answers.

2.2.2. Written questionnaires

Two separate questionnaires were designed, with one targeting ESF managing authorities and the other labour ministries. The questionnaires were sent on 9 October 2018, followed by a reminder sent on 7 November 2018 and a final reminder sent at the end of December 2018.

2.2.3. Workshop

A workshop was held at the meeting of the AFEPAs on 11 October 2018. The workshop commenced with a presentation of the evaluation aims, approach and timetable of the evaluation, including the AFEPAs’ role. Two rounds of breakout sessions (with four subgroups) then took place: part I focused on the main positive impacts of the Network to date, while part II focused on identifying the Network’s future aims and activities, and possible improvements. A seven-page outcome document presenting lessons from the

workshop activities was circulated shortly after the workshop to relevant stakeholders 64 .

The AFEPA workshop write-up also served as a key reference document providing background knowledge and context for the researchers conducting the semi-structured interviews.

64 See Annex 6 to the Study supporting the evaluation of Decision on enhanced cooperation between Public Employment Services,

Ecorys for the European Commission, 2019

2.2.4. Case studies

Case studies were carried out in five countries (Estonia, France, Italy, the Netherlands and Romania) to gain deeper insight into the actions taken in response to the PES Decision at national/regional/local level(s) and to assess impact on individual PES. The purpose of the case studies was to explore in depth the qualitative evidence and to look into actions taken in response to the PES Decision at national/regional/local level(s) to assess impact on individual PES and their influence on programming and funding, where applicable.

The four main tasks in the context of the case studies included secondary data analysis activities (desk-based research), two primary data collection methods (interviews and, in certain countries, a focus group), and analysis and reporting. The focus groups set out, where feasible, to explore the changes triggered by the PES Decision through collective discussions with key stakeholders in each selected case study country. Such stakeholders included employment service providers and representatives of PES ‘users’ (e.g. jobseekers, employers, other labour market stakeholders).

2.2.5. Summary and reflections on challenges

The consultation methods were overall implemented as planned and in line with the agreed consultation strategy. The interviews were conducted as planned, both for the targeted consultations and as part of the case studies. The response rate for the in-depth interviews was very high (only one individual included in the consultation strategy neither replied nor delegated the interview to a colleague). The response rate for the targeted questionnaires was, however, lower: only 16 labour ministries and 7 ESF managing authorities filled out the questionnaires sent by the evaluation team. The rate of response to the consultations conducted in the five case studies fluctuated. In some case study countries, it was necessary to replace the focus groups with additional interviews as for different reasons 65 it was not possible to organise the interviews within the evaluation timeframe.

• 3. 

  I NFORMATION ON STAKEHOLDER GROUPS

3.1. Public consultation

A total of 126 consultation responses were received from public consultation stakeholders by the cut-off date of 13 December. Their profile is presented in the following subsections.

65 For the French study, the PES was unable to find a suitable date in November 2018, December 2018 or January 2019 for the focus

group; for the Estonian case study, there were not sufficiently large numbers of individuals and organisations with sufficient knowledge of the PES Network to warrant a focus group; in the Dutch case study, the Dutch PES did not wish for a focus group to be organised because there were not sufficiently large numbers of individuals and organisations with sufficient knowledge of the PES Network, so instead the Dutch PES facilitated the organisation of interviews with several representatives involved in the

PES Network in different capacities.

3.1.1. Country of respondents

Out of the 126 responses, the highest numbers were from Italy (23) and France (22). Table 1 below sets out the number of responses received from each country, as well as the percentage of the overall number of responses.

Table 1: Distribution of respondents by country

Country Respondents

Italy (n= 23) 18.3%

France (n= 22) 17.5% Spain (n= 19) 15.1% Belgium (n= 15) 11.9% Germany (n= 13) 10.3% Finland (n= 3) 2.4% Portugal (n= 3) 2.4% Greece (n= 2) 1.6% Netherlands (n= 2) 1.6% Slovenia (n= 2) 1.6% Sweden (n= 2) 1.6% Bulgaria (n= 1) 0.8% Croatia (n= 1) 0.8% Czech Republic (n= 1) 0.8% Hungary (n= 1) 0.8% Latvia (n= 1) 0.8% Liechtenstein (n= 1) 0.8% United Kingdom (n= 1) 0.8% Pan-EU/European umbrella organisation (n = 6) 4.8% Other (n= 7) 5.6% Total (n = 126) 100%

3.1.2. Sector of activity

The highest number of respondents mentioned ‘public administration’ as their sector of activity (54.8%), 7.1% mentioned ‘education’ and 5.6% ‘other community, social and personal services. Table A1.2 in Annex 1 to the external study gives a full overview of sectors of activity of the public consultation respondents.

3.2. Targeted consultations

In line with the consultation strategy of the evaluation, targeted consultations were conducted with a variety of key stakeholders at both national and EU levels, as detailed above. The full list of stakeholders participating in the targeted consultations are set out in Table A1.3 in Annex 1 to the external study.

3.2.1. Targeted semi-structured interviews

By the closing date for conducting interviews in January 2019, interviews with all 32 PES included in the consultation strategy had been conducted. This included interviews with the Network Board members (usually the heads of PES) and interviews with AFEPAs. Some 21 interviews with other stakeholders were also conducted, including interviews with the Commission PES Secretariat, the Council’s Employment Committee, the World Employment Confederation, EU-level networks/institutions (DG EMPL, DG REGIO, the European Lifelong Guidance Policy Network), WAPES, other international organisations (ILO, OECD, World Bank), and other stakeholders with knowledge of preand post-2014 activities, as well as the contractors that supported the implementation of Network activities (ICON Institut and ICF International).

3.2.2. Workshop

PES stakeholders in EU Member States, Norway and Island (AFEPAs and some Board members) were further consulted during the AFEPA workshop on 11 October 2018. The evaluation workshop was organised during one of the bi-annual AFEPA meetings, which are organised under the aegis of the Network activities. A total of 35 AFEPAs (or their alternative representative 66 ) took part in the evaluation workshop and substantially contributed to increasing the evidence base.

3.2.3. Written questionnaires

The two written questionnaires derived from the consultation topic guide framework primarily targeted two key stakeholder groups: ESF managing authorities and labour and social affairs ministries. By 14 January 2019, 16 contributions had been collected from

labour ministries 67 and 7 contributions from ESF managing authorities 68 .

3.2.4. Case studies

Interviews and focus groups were also conducted as part of the case studies carried out in five selected countries (Estonia, France, Italy, the Netherlands and Romania). In Estonia, interviews were conducted with representatives from the Estonian Unemployment Insurance Fund and the Ministry of Social Affairs. In the French case study, interviews were held with a range of different representatives of Pôle emploi, the French employment service and the ministry of labour. Interviews were conducted with the Italian Agency for Labour Policies (ANPAL) at the national level, and a focus group was organised in which national and regional representatives of the Agency participated. This was also the case in Romania, where national and regional representatives of the National Agency for Employment participated in the consultation process. Several interviews were organised in the Netherlands as well.

66 In some cases, more than one representative from each PES was present at the meeting (e.g. Ireland).

67 Austria, Belgium (Wallonia), Croatia, Czech Republic, Denmark, Estonia, Germany, Hungary, Ireland, Latvia, Malta, the

Netherlands, Romania, Slovakia, Spain and Sweden. 68 Bulgaria, Cyprus, Czech Republic, Hungary, Latvia, Lithuania, Slovakia and Sweden.

• 4. 

  M ETHODOLOGY FOR DATA PROCESSING

4.1. Public consultation

The responses to the public consultation were examined through a combination of closed and open-ended questions. Closed questions provided respondents with different categorical answers (on the challenges faced by PES and the most important areas for improvement) or with a rating of the Network’s importance, strength of complementarity, success, efficiency and necessity.

The analysis of results was carried out using both quantitative methods (to analyse the frequencies of the closed answers) and qualitative methods (for the open questions, to analyse complex concepts and to substantiate and interpret the quantitative data with relevant insights). In addition, the responses were categorised using a range of relevant typologies (i.e. of respondents, of type of information provided, of country, etc.)

Quantitative analysis includes: (i) analysis of frequency distribution for each of the variables related to the closed questions; (ii) cross-tabulations between specific variables and characteristics of respondents ; and (iii) analysis of variability, calculating averages 69 and measuring distances from the average to allow for comparison. Qualitative data analysis was used to contextualise the analysis carried out on quantitative data, including inclusion of any quotes where relevant, and to identify major trends or changes.

4.2. Targeted consultations

The write-ups from the targeted consultations (interviews and written questionnaires) were collected and exported into analytical grids, broken down by the different questions and by the respective evaluation criteria. The analytical grids were used to carry out an in-depth analysis of the data collected to feed into the relevant sections of the external study and the analysis carried out for the staff working document. The write-up of the AFEPA workshop was similarly used to feed into the evaluation.

4.3. Case studies

Information gathered from the case studies was written up into extensive case study templates. The data provided in the different subsections of the templates was used in the analysis carried out for the staff working document.

• 5. 

  D ISSEMINATION STRATEGY IMPLEMENTED FOR THE PUBLIC CONSULTATION

The dissemination strategy aimed to ensure that the consultation: (i) reached stakeholders not specifically targeted under the targeted consultations, i.e. social partners at EU and national level, employers, local/regional authorities, organisations or networks representing the interests of different groups of jobseekers, members of academic/research networks, citizens; and (ii) went beyond the close circle of those who

69 The data was disaggregated by the capacity in which the respondent is answering (individual or professional), the respondents’

involvement with the PES network and the type of organisation that respondents answering in their professional capacity come from (government body, PES or NGO and others).

are very familiar with the post-2014 Network set-up. Table 2 includes a detailed list of stakeholders targeted for the public consultation.

Table 2: List of stakeholders targeted for the online public consultation

Stakeholder Stakeholders Multipliers for survey distribution categories

Social partners Business • European Trade Union Confederation (ETUC) organisations • European Trade Union Federations

• Business Europe

Sectoral • European Association of Craft, Small, and Medium-Sized organisations Enterprises (UEAPME)

• European Centre of Enterprises with Public Participation and

Trade unions of Enterprises of General Interest (CEEP)

•

Chambers of The Association of European Chambers of Commerce and commerce Industry (Eurochambers)

National, National and • National PES (AFEPA) and regional/local offices local/regional local/regional • National ESF authorities stakeholders PES • National labour ministry representatives

• European Lifelong Guidance Policy Network (ELGPN)

Other national/ • EQF national coordination points local actors • The Council of European Municipalities and Regions

(CEMR)

Other actors Private • World Employment Confederation-Europe employment • Eurocities

services Associations representing local actors

Research and other Think tanks in • European Student Union networks relevant policy • Social Platform

areas • European Anti-Poverty Network (EAPN)

• European Disability Forum (EDF)

Academic • European Youth Forum (YJF)

networks • European Women’s Lobby

•

NGOs European Network against Racism (ENAR) representing • European Confederation of cooperatives active in industry

jobseekers or and services (CECOP)

specific groups • Civil Society Europe • Network of Experts on Social aspects of Education and

International Training (NESET) organisations • Lifelong Learning Platform

• Euroguidance • ILO • Europass • EURES Network

Citizens Citizen voice • European citizens’ initiative (ECI) channels • European Citizen Action Service (ECAS)

Social media Twitter: @EU_Social, @EU_Commission, @EURESjob, @EuropeanYouthEU, etc.

• 6. 

  Overview of the results of the consultations

Relevance: There was a strong consensus among consultees about the overall high degree of relevance of the objectives of the Decision, albeit to different degrees. These findings were confirmed by the interviews with PES representatives, other stakeholders (EU and international level stakeholders, as well as participants with knowledge of the cooperation between PES before, as well as after 2014) and those interviewed for case studies. All interviewees emphasised that the PES Network itself remains very relevant to needs and should continue beyond 2020. The majority of consultees, as well as participants in the AFEPA workshop, underlined the capacity which the Network has shown to adapt to changing policy priorities (e.g. the refugee crisis). The relevance of Decision objectives was strongly confirmed by the findings from the written consultations of Labour Ministries and ESF Managing Authorities. The results of the public consultation also endorsed the Decision’s relevance, with 53% of public consultation respondents answering in their personal capacity and 59% answering in their professional or institutional capacity considering that PES cooperation at EU level is ‘very important’ for developing national PES effectiveness in addressing social and labour market challenges.

Effectiveness: All stakeholders who could express an informed opinion (based on frequent participation in Network activities) reported that, overall, Network initiatives have been effective in supporting the achievement of the objectives. This is equally valid for the interviewed PES Network members, as well as EU, international and national level stakeholders that have participated in or have knowledge about the Network. The vast majority of stakeholders – including PES representatives who were interviewed, those who participated in the AFEPA workshop and in the case study research, EU and international level stakeholders, Labour Ministries, and ESF Managing Authorities - emphasised benchlearning as the most effective PES Network initiative. Mutual assistance are also perceived as very effective by the PES representatives that have participated in them as learners or tutors alike. Other Network initiatives (such as contributing to the implementation of policy initiatives, the promotion of best practices and the adoption of the annual work programmes) are also perceived as effective, but to different degrees. Another initiative often emphasised by the consulted PES representatives was the contribution of the Network to the implementation of relevant policy initiatives at national level (e.g. the Youth Guarantee). The vast majority of all the consulted PES representatives and other stakeholders indicated that the Network has contributed to supporting the most vulnerable groups with high unemployment rates and the integration of persons excluded from the labour market. The majority of responses received from the national Labour Ministries (12 out of 16 ministries) indicated that Network has been successful or very successful in supporting the most vulnerable groups with high unemployment rates. However, only a minority (5 out of 16) indicated that the Network has been successful or very successful in supporting the integration of persons excluded from the labour market. Stakeholders generally considered that the Network activities have contributed less, although to some degree, to supporting decent and sustainable work, increased geographical and occupational mobility, the evaluation and assessment of ALMPs and their implementation.

The public consultation results show that one third of the respondents who have been involved in Network consider that it has been very successful in supporting enhanced cooperation. Respondents who have not been directly involved with the Network consider that it has somewhat successful in this area (25%), while 19% consider it somewhat unsuccessful, and 29% don’t know.. Most of the respondents answering in their professional or institutional capacity reported that the Network is somewhat successful in supporting enhanced cooperation (38%), while 17% of them reported that they do not know. 40% of respondents answering in their individual capacity reported that they do not know to what extent the Network has been successful in this area.

Efficiency : The interviews and written consultations, as well as evidence from the case studies, confirmed that the PES Network is considered both to be efficient and a good use of resources by the vast majority of actors consulted both at EU level and in national PES (almost 100% of those who expressed an opinion). This view was also held by representatives of international organisations consulted. Given the relatively modest resources (compared to the scale of PES activity) which most consultees consider are devoted to the Network and its activities at EU level, the vast majority of consultees felt that the benefits of the Network are very important.

Coherence : Most stakeholders consider that there is a good degree of coherence between the Decision and the EU policy framework. Most stakeholders also reported that there is some cooperation between the Network and other relevant labour market stakeholders, but that there is scope for improvement, for example by developing stronger relationships: at EU level with the EU social partners, EU level NGOs, EMCO and EURES; at national level with the social partners, NGOs supporting jobseekers, Managing Authorities of EU Funds (in particular the ESF) and training providers; and at international level with the OECD, ILO, WAPES and the World Employment Confederation.

In the public consultation, almost half (45%) of the respondents answering in their professional or institutional capacity reported that a ‘strong complementarity’ between the objectives of the Network and other EU and/or national policies aimed at improving PES, while 15% reported a very strong complementarity. Almost a third of respondents (32%) answering in their personal capacity reported a strong complementarity, while 8% reported a very strong complementarity. 25% of respondents answering in their personal capacity reported that they do not know whether there is complementarity between the Network objectives and other EU and/or national policies, compared to 15% of those answering in their professional capacity.

The targeted consultations (interviews, case studies and workshop) with PES representatives show that the vast majority consider that there is a high degree of coherence of the Decision with the European policy framework. In particular, a large majority of PES consultees emphasised that the Decision reflects the priorities set out in the Europe 2020 strategy. Complementarity was also emphasised in particular with the Youth Guarantee, the Youth Employment Initiative, the Council Recommendation 2016/C 67/01 on the integration of the long-term unemployed into the labour market, and ESF-funded programmes. Synergies were also highlighted in relation to the mandate of other EU institutional actors, such as the EURES Network and EMCO.

Added value: PES representatives, Labour Ministries and ESF Managing Authorities, external stakeholders at EU and international level have overwhelmingly reported that from their perspective, the learning process and cooperation would not have occurred to the same extent in the absence of the Decision. According to all consultees, the Network has an added value in driving improvements in PES organisation and performance, increasing effectiveness and stimulating cooperation. The consultations strongly emphasised the crucial nature of the commitment and support provided at EU level (European Commission) to ensuring the effective implementation of the Decision, in terms of both financial and human resources invested in the running of the Network. It was reported that the absence of a formal commitment to continue to provide resources for the Network to continue to function would have a negative impact in reducing the scope, extent and quality of PES cooperation and its ability to implement strategic EU employment goals.

The importance of the EU support for PES cooperation is reinforced by the public consultation results. More than for any other question in the public consultation, there was a strong consensus amongst respondents about the necessity of EU action for strengthening the cooperation between PES. A large majority (80%) of respondents answering in their professional or institutional capacity reported that EU action for strengthening the cooperation between PES is ‘very necessary’ (62%) or ‘somewhat necessary’ (20%); similarly, 82% answering in an individual capacity felt that it was ‘very’ (57%) or ‘somewhat’ (25%) necessary. The share of those considering that EU action is ‘very necessary’ was highest among respondents from PES (67%) and NGOs/other stakeholders (65%).

Annex 3: More detailed intervention logic of the evaluation

Needs

• Strengthen PES service capacity, effectiveness and efficiency. • Increase PES potential to contribute to Europe 2020 objectives, including support for lowering unemployment (including for the most vulnerable social groups);

decent and sustainable work; better functioning of EU labour markets; identification of skills shortages and better matching; better integration of labour markets; increased voluntary mobility to meet specific labour market needs; integration of persons excluded from the labour market; evaluation and assessment of active labour market initiatives and their implementation.

Objectives Effectiveness

n ce Strengthen cooperation

Activities

• va between PES in order to:

Benchmarking: self-assessments,

• Improve PES site visits, external assessments R e

le •

performance Mutual learning events Results

• • Contribute to the PES Network governance • Increased cooperation and

implementation of meetings exchange amongst PES • policy initiatives in Mutual assistance • Follow up and Impacts •

the field of Supporting policy implementation of • Strengthened and modernised

employment implementation benchlearning PES •

Additional activities e.g. recommendations • Better performing PES (efficiency

monitoring of pan-EU initiatives, • Changes undertaken by PES and effectiveness)

ad-hoc studies and research, etc. (organisational process and • Better support to the

service delivery) unemployed and inactive not

• Changes in ESF registered with PES programming and/or • Better functioning labour

Inputs

• funding EC contribution markets •

Better matching of skills and of €3.5m/yr Outputs

(2014-2020) • Benchlearning reports and integration of labour markets

• • Member State recommendations, PES Capacity Increased mobility (geographical

and occupational)

contribution Reports, PES Implementation of e •

• Staffing of PES the Youth Guarantee, etc. Lower unemployment, including a lu

for vulnerable groups

Network • Studies, toolkits, PES practices, d v •

Secretariat PES Knowledge centre. d e Contributing to meeting the • Liaison persons at • PES Network meeting minutes, a d EU2020 targets

national level decisions and position papers E U

n ce

re Efficiency

h e C o

External (exogenous) factors

• Labour market functioning and social situation at MS level • Structure and funding of PES at MS level

Met de EU Monitor volgt u alle Europese dossiers die voor u van belang zijn en bent u op de hoogte van alles wat er speelt in die dossiers. Helaas kunnen wij geen nieuwe gebruikers aansluiten, deze dienst zal over enige tijd de werkzaamheden staken.

De EU Monitor is ook beschikbaar in het Engels.