Youth Research & Knowledge

Introduction: The review research methodology emerging from Demokratie & Dialogue’s Policy Reviews

Published on

Through our policy reviews, we take a broader look at policy in relation to youth, analyzing not only specific youth policies, but the wider policy dossiers that can affect young peoples’ lives and rights, from justice to health, and from participation to the environment. The key unique feature of the review process is its research methodology, a matrix specifically developed for this purpose, which we introduce here.

While this is not the only review process…

Our review methodology - the evaluation matrix

The pilot review process we are currently undertaking is not the only mechanism to undertake assessment of policies pertaining to young people. The Council of Europe has a longstanding process of national reviews supported by international teams. The review of a particular country is initiated by invitation from the government of the country concerned, and is not considered an evaluation per se, but rather as an international perspective on what a given country might consider to improve if and when youth policy is up for review.

Various specialized United Nations agencies and programs formulate review instruments and integrate them into their program planning processes. These are generally conducted on the basis of obtaining information for background descriptions or situation analyses for a country program document, and sometimes as was the case in 2007 for UNFPA, these have been conducted for a region (Europe and Central Asia). The World Bank and the International Labor Organization (ILO), the United Nations Children’s Fund (UNICEF), the Youth Unit of the Department of Economic and Social Affairs have all undertaken some form of youth policy review in the last decade.

…it has a unique approach

This project differs from its predecessors in several respects. First, it takes a broader look at policy in relation to youth, analyzing not only specific youth policies, but the wider policy dossiers that can affect young peoples’ lives, from housing to education, from health to participation. Second, it attempts to understand the impact of said policies pertaining to young people on the achievement of their human rights, asking the question in which way do said policies support or hinder young people in becoming fully active and engaged citizens. Third, it acknowledges the role of international exchange and good practice in the development of youth policy knowledge, and tries to assess the extent to which international policy initiatives, legislation and declarations have influenced the national policy field - for better or worse. Finally, and not least importantly, this project has taken the rare approach of ‘not waiting to be asked’, in that it does not rely on government invitation to consider the merits and possible gaps in a country’s policy provisions for young people, thereby making a strong statement as regards the necessity of government to be held to account by citizens.

The key unique feature: the matrix

Our review methodology - the evaluation matrix

Probably the key unique feature of this review process is that it was rolled out on the basis of a specifically developed research methodology, known as the Matrix. This project’s approach to youth policy research can be broadly understood as one of impact assessment. Its contribution to youth research starts from fairly consolidated values and interests that are already based on strong institutional reflections.

The policy matrix was developed to assist in assessing the impact of public policy on the rights of young people in a variety of country contexts, and was first tested in the present pilot review. Given the pilot nature of the initiative the matrix served as a training framework for understanding the policy review process and was the basis on which the country report structure was designed.

First insights on the usage and usefulness of the matrix

According to the evaluation conducted by the International Editorial Board (IEB), the experience of working with the matrix has been mixed. Country teams were initially a little overwhelmed by its scope, but despite initial negative reactions to the matrix’s complexities and ambiguities, teams were able to adapt and contextualize it for their country research processes. The matrix proved useful in the sense that its purpose was clear, and despite a broad scope, it provided the review team with rather concise questions pertaining to youth policy and comprehensively alluded to a detailed account of the different youth arenas. However, the country teams were confronted with a trade-off: the degree of in-country adaptation decided the extent to which the report would be useful for advocacy within a country versus easy international comparability.

Based on at least four of the country reports taken into consideration by the IEB, the country teams appear to have interpreted the matrix as a kind of check-list that would help them to identify and classify the issues relating to youth policy in the country under review. Accordingly, certain issues proposed by the matrix are missing in the individual reviews.

This can imply that local researchers or their international advisors consciously avoided a topic, but it may also indicate that they considered it irrelevant after serious examination. While the scope and breadth of the matrix provided the teams with a useful framework for guiding the youth policy reviews, it also meant that in-country certain choices about what to include and what not to include had to be made. In some countries at least some of these choices were likely also determined by the expertise and interests of the local researchers rather than the actual situation on the ground.

Further, feedback from the country teams indicate that the matrix was useful for framing the youth policy review process and ensuring that it addressed the many issues affecting young people. It also helped ensure a certain degree of consistency in the structure of the country reports.

Next steps: how will the matrix evolve?

The Matrix is currently being re-developed to take into account the experience of the pilot round of reviews. The revised version will be used to orient the teams that will conduct the second round. In the long run, it is hoped that the rights-based approach it proposes can inform other review processes, and consequently, we will make the re-worked matrix available here on youthpolicy.org.

Further information

Updates about the reviews and various follow-up activities will be provided continuously at https://www.youthpolicy.org/reviews.

For additional information about the youth policy review process and the matrix please contact us at reviews@youthpolicy.org.