Prepare an evaluation report on the implementation of the National Urban Policy in selected countries.
Remote
The contract is expected to start in January 2022 and is for a duration of three (3) months.
The evaluation will be conducted over the period of three months spread over five months, including the desk review, from January 2022 to June 2022.
1. Background and Context
The United Nations Human Settlements Programme, UN-Habitat, is the agency for human settlements. It is mandated by the UN General Assembly to promote socially and environmentally sustainable towns and cities with the goal of providing adequate shelter for all.
Within UN-Habitat, the Urban Practices Branch (UPB) is the tools and methodology production and skills centre of the Agency. It develops normative guidance and cutting-edge tools through the following communities of urban practices: (i) policy, legislation and governance; (ii) urban planning, finance and economic development; (iii) urban basic services; and (iv) land, housing and shelter. It also ensures that the cross-cutting areas of resilience, safety and the social inclusion issues of human rights; gender; children, youth and older persons; and disability among others. In addition, it provides technical and policy advice, approaches and methodologies to MCOs/CO, government and partners and mainstreams them throughout the programme portfolio in collaboration with the Programme Development Branch and other organizational units.
The Policy, Legislation and Governance Section (PLGS) coordinates the Agency’s substantive, normative and operational work on various dimensions of Policy, Legislation and Governance, fostering high impact initiatives, programmes and projects to deliver timely, fit-for-purpose and targeted solutions in these areas. The Section aims to develop, document, pilot/test and disseminate norms, standards, tools, methodologies, regulatory frameworks; operates procedures and inspiring practices at global, regional, national and sub-national levels; designs tailored-made solutions on policy, legislative and governance for cities, regions and metropolitan development; enhances urban-rural continuum, with the view to reduce spatial inequality and poverty and increase shared prosperity; and generates and manages cutting-edge knowledge and data (including Indexes and database) on Policy, Legislation and Governance.
One of the section’s important work stream is on National Urban Policy (NUP). NUP facilitates the role of cities in the transformation of the national economy by providing a coordinating framework and action plan to bring coherence across various urban sectors and scales of urban management. The Section works towards increasing awareness of countries on tools and best practices in NUP making and providing platforms for interactions between state and non-state actors as well as between levels of government. In doing so, UN-Habitat supports countries throughout the public policy cycle (feasibility, diagnosis, formulation, implementation, monitoring and evaluation) while promoting capacity development, participation and demonstration projects throughout the policy cycle. Given UN-Habitat’s conceptualization of NUP as an effort to rally multiple stakeholders (governments, professionals, academia and civil society organizations etc.) to collectively define a common urban vision.
These TOR is for the evaluation for the Phase One of the Korea-funded project entitled “National Urban Policy Programme: Developing NUPs and Smart City Strategies in three selected countries (I.R. Iran, Myanmar and Nigeria)” in light of informing the subsequent phases.
2. The project Background and context
National governments are encouraged to develop and implement New Urban Policies (NUPs), which are perceived to be vital to support and enable the implementation of the New Urban Agenda that was adopted at the Habitat III conference that took place in Quito, Ecuador in 2016. Following the Habitat III Conference in Quito, Ecuador, UN-Habitat, OECD, and Cities Alliance launched the National Urban Policy Programme (NUPP). The joint initiative aims to contribute to the successful implementation of the New Urban Agenda (NUA) and to achieving sustainable and prosperous human settlements for all, leaving no one behind, through the development of NUPs. With the signing of a Memorandum of Understanding between the Republic of Korea and UN-Habitat at the Habitat III Conference in October 2016, Korea committed to support the development of the National Urban Policy Programme through providing financial support for Phase One of this Programme.
The launching of the pilot phase of the National Urban Policy Programme: Developing NUPs and Smart City Strategies in Three Selected Countries, supported by the Ministry of Land, Infrastructure and Transport (MOLIT), Republic of Korea, began 2017. The purpose of the project is to develop National Urban Policies and Smart City Strategies in the three pilot countries, Myanmar, Iran, and Niger State (Nigeria). In particular, the project aims to:
1. Enhance capacity of sub-national and national governments in the three pilot countries to develop, implement, and monitor and evaluate national urban policy (NUP and SUP) and develop smart city strategies.
2. Increase centralization of knowledge and tools on the development, implementation, and monitoring and evaluation of urban policy (NUP and SUP) and smart city strategies.
3. Provide augmented opportunity for knowledge sharing and peer learning activities on urban policy (NUP and SUP) and smart city strategies.
Through the processes of developing the policies and strategies the pilot phase of the NUPP has endeavoured to build capacity to develop, implement and monitor and evaluate urban policy in the pilot countries, implement the NUPs through demonstration project identified using a smart city approach, and provide for knowledge creation, knowledge exchange on urban policy through an online platform and in person conferences and workshops.
Since the launching of this project in July 2017, there has been significant progress, both at the global level of the programme, and at the country level. Country level activities, staffing and activities had been undertaken in I.R Iran, Myanmar, and Niger State (Nigeria).
At the global level, the project has facilitated the organisation of International Conferences on NUP, supported the preparation of normative products and international capacity building including annual exchange visits to the Republic of Korea (in partnership with the Korea Land and Housing Cooperation). The project is implemented in partnership, both internally at UN-Habitat, in close coordination between UN-Habitat Headquarters, Regional Offices, and Country Offices, but also with partner organizations, with experts from the Korean Research Institute for Human Settlements (KRIHS), the Korea Land and Housing Corporation (LH), Ministry of Land, Infrastructure and Transport (MOLIT), the Organization for Economic Cooperation and Development (OECD) and Cities Alliance, amongst others.
The project started in 2017 and will end in June 2022. The project had a budget of US$2,600,000 funded by the Republic of Korea through the Ministry of Land, Infrastructure and Transport (MOLIT). In global context, The Global state of National Urban Policy 2018 revealed that out 150 countries studied, 75 countries had adopted National Urban Policies, 92 countries were at the implementation stage of NUPs. This upwards trend had continued as revealed in the 2021 Global Sate of National Urban Policies.
2.2 Project Management
The management of the project is within the Policy, Legislation and Governance Section (PLGS) of the Urban Practices Branch (UPB) with the involvement of country-based focal points.
Supervision:
The consultant will report to the Section Chief of the PLGS and the Chief of the UPB .
3. Mandate, Purpose, Objectives, Scope of the Evaluation
This evaluation is mandated by both the donor, Korea and UN-Habitat, through the cooperation agreement/MOU. It is also in line with the UN-Habitat evaluation policy (2013) and the Revised UN-Habitat Evaluation Framework (2016).
The evaluation is intended to provide NUP partners and UN-Habitat and its governing bodies with an independent and forward-looking appraisal of the project’s operational experience, achievements, opportunities and challenges based on its performance and expected accomplishments. What will be learned from the evaluation findings are expected to be used to inform the development and implementation of future NUP funded projects.
The evaluation seeks to serve the purposes of accountability for results achieved as well as enhancing learning that would improve current and future NUPs development and implementation. The specific objective of the evaluation is to assess the project’s performance during the 2017-2021 period, and to make recommendations for the next steps in the implementation of the national urban policy and New Urban Agenda in the three countries. Specifically, the evaluation will:
• Assess the performance of the programme in terms of the extent to which it achieved planned results at the expected accomplishment (outcomes) and output levels;
• Assess the relevance, efficiency, effectiveness, sustainability, impact and coherence of the projects with other policies and programmes related to NUPs;
• Assess enhance technical and institutional capacities of national, sub-national and local governments to strengthen their national urban policy-making processes and increased awareness of countries in the region of tools, frameworks, procedures and best practices in National Urban Policy making. This will entail analysis of delivery of outputs, achievement of outcomes, and long-term effect;
• Assess the planning and implementation modalities, including working arrangements and how they may have affected the effectiveness of the projects;
• Assess how social inclusion issues of gender equality, youth, human rights and climate change were integrated in the projects;
• Identify lessons and propose recommendations for the implementation of the national urban policy and the New Urban Agenda in the three countries, in terms of what should be done on what needs to be done to effectively implement, promote, develop and monitor UN-Habitat’s support to national and local authorities in formulating and implementing National Urban Policies;
• The evaluation findings when used by UN-Habitat management and project team, the donor, and other key stakeholders, including governing bodies and Member States, on what was achieved and learned from the project.
This evaluation covers the whole period of the project ’s implementation, from its start in 2017 to 2021 and geographically covers three countries where the project was implemented: I.R Iran, Myanmar, and Niger State (Nigeria).
4. Evaluation Criteria and Evaluation questions
The project team together, together with the Independent Evaluation Unit have proposed evaluation questions organized around the evaluation criteria of relevance, effectiveness, efficiency, coherence, sustainability and impact outlook. The evaluation questions will be assessed to supplement the specific objectives of the evaluation. The evaluation team should build on these questions to develop an evaluation matrix with evaluation questions, key stakeholders and data collection sources.
Relevance
• To what extent is the project consistent with relevant national policies and strategies, and national development plans?
• To what extent is the implementation strategy responsive to SDGs, NUA, UN-Habitat Strategic Plans, UN-Habitat’s National urban policies and its strategies on human development priorities on vulnerable groups and poor, human rights, women and youth?
• To what extent are the project’s intended outputs and outcomes relevant to the needs of target beneficiaries?
• What was UN-Habitat’s comparative advantage in implementing the NUPs compared with other UN entities and key partners?
Efficiency
• How well was the project designed and implemented, and what have been the most efficient types of activities implemented?
• How efficiently were the inputs (financial and human resources), partnerships, policies and implementations strategies used to achieve the planned outputs?
• To what extent were the institutional arrangements adequate for achieving the expected accomplishments? What type of (administrative, financial and managerial) obstacles did the project face and to what extent has this affected project delivery of outputs and achievement of the expected accomplishments?
• What types of products and services did the project provide to beneficiaries through activities implemented? What kind of changes to beneficiaries has resulted from products and services delivered?
• To what extent was the project delivered in a cost-effective manner?
Effectiveness
• To what extent did the project achieve its planned results and how did UN-Habitat contribute to towards achieving these results?
• To what extent were the resources used to implement the project justified in terms of delivering on the expected accomplishments?
• What were the major factors that influenced the achievement of the expected accomplishments (outcomes)?
• To what extent have national and local stakeholders been involved in the design and implementation of the project?
• To what extent and in what ways has the ownership by local stakeholders impacted on the effectiveness of the project?
• To what extent the management of the project has learned from and adjusted to changes during implementation?
• How effectively has UN-Habitat and other implementing partners credibly monitored the implementation of the project, using the indicators of achievements to provide evidence on performance and flag any necessary adjustments to improve delivery of the project. How effectively was the project engaging with countries where the project was implemented to achieve desired outcomes of the project
• To what extent were UN-Habitat’s cross-cutting issues of gender, youth, climate change, and human rights integrated into the design, planning and implementation, reporting and monitoring of the project?
Impact Outlook
• To what extent has the project attained or not (or is expected to attain) its goal, and objective and expected accomplishments short, medium and long-term) to the targeted beneficiaries, participants, whether individuals, communities, institutions, partners, etc.?
Sustainability
• To what extent have local stakeholders been able to design, implement and sustain activities implemented during the project?
• To what extent did the project engage the participation of beneficiaries in design, implementation, monitoring and reporting?
• To what extent will the in-country activities be replicable or scaled up at national or local levels or encourage further collaboration between stakeholders?
Coherence
• Was the project coherent and implemented in synergy with other programme of similar objectives
• Was the project coherent or complemented by other donors’ development interventions
5. Evaluation Approach and Methodology
5.1 Approach
The evaluation will be conducted in line with the United Nations Evaluation Group (UNEG) Norms and Standards for evaluation in Nations System. The evaluation team will decide on the concreted approach and methodology, considering the COVID-19 situation. It is anticipated that the evaluation will apply results-based approach (Theory of Change) in assessing performance of the project. The consultant is encouraged to build on the official approved TOC of NUPs and construct one for this particular project.
Using the TOC, the evaluation consultant will then test and verify whether the theoretical chain of results and underlying assumptions hold true during the implementation of the programmeto achieve its planned results. The logical framework could also be reconstructed and could outline the results chain and pathways. The evaluation should also use participatory and utilization focused approach, to enhance the utilization of evaluation results and engagement of the stakeholders in the evaluation process. Systems approach could also be used where the TOC cannot explain issues of coherence, partnership, coordination etc.
5.2 Methods
A variety of methodologies will be applied to collect information during evaluation. These methodologies include the following elements:
a) Review of documents relevant to the project. Documents to be provided by relevant UN-Habitat entities, and documentation available with stakeholders and beneficiaries (such documentation shall be identified and obtained by the evaluation).
Documentation to be reviewed will include:
• Project document, results framework and implementation plans;
• Annual work plan;
• Annual monitoring reports;
• Overall NUPP evaluation report (undertaken in 2021);
• Publications;
• NUP conference reports;
• NUP training reports;
• Reviews of websites and other information available on social media and other platforms (including www.urbanpolicyplatform.org);
• Strategic plans, as deemed relevant, such as UN-Habitat’s Strategic Plans (2014-2019 and 2020-2023), and other relevant policy documents;
• Outreach and communication material.
b) Key informant interviews and consultations, including focus group discussions will be conducted with key international and national stakeholders and others, including UN-Habitat staff. The principles for selection of stakeholders to be interviewed as well as evaluation of their performance shall be clarified in advance (or at the beginning of the evaluation). The informant interviews will be conducted to obtain qualitative information on the evaluation issues, allowing the evaluators to assess project relevance, efficiency and effectiveness.
c) Surveys. In order to obtain quantitative information on stakeholders’ views and perceptions, questionnaires to different target audiences (beneficiaries, local and national authorities, members of the National Habitat Committees, other stakeholders, etc.) will be deployed as deemed relevant to give views on various evaluation issues.
d) Field visits, under strict health considerations, if deemed feasible with resources available to the evaluation, to assess selected activities in one or two countries. Field visits should provide insight into both the scope (time), depth and range of activities.
The evaluator will describe expected data analysis and instruments to be used in the inception report. Presentation of the evaluation findings should follow the standard format of UN-Habitat Evaluation Reports (evaluation purpose and objectives, evaluation methodology and approach, findings (achievements and performance rating assessments), conclusions, lessons learned, recommendations).
Rating of performance and Characteristics
(Source: UN-Habitat Evaluation Unit 2015)
Highly satisfactory (5) - The programme had several significant positive factors with no defaults or weaknesses in terms of relevance/ efficiency/ effectiveness/ sustainability/ impact outlook.
Satisfactory (4) - The programme had positive factors with minor defaults or weaknesses in terms of relevance/ efficiency/ effectiveness/ sustainability/ impact outlook.
Partially satisfactory (3) - The programme had moderate to notable defaults or weaknesses in terms of relevance/ efficiency/ effectiveness/ sustainability/ impact outlook.
Unsatisfactory (2) - The programme had negative factors with major defaults or weaknesses in terms of relevance/ efficiency/ effectiveness/ sustainability/ impact outlook.
Highly unsatisfactory (1) - The programme had negative factors with severe defaults or weaknesses in terms of relevance/ efficiency/ effectiveness/ sustainability/ impact outlook.
6. Stakeholder involvement
It is expected that this evaluation will be participatory and involving key stakeholders. Stakeholders will be kept informed of the evaluation processes including design, information collection, and evaluation reporting and results dissemination to create a positive attitude for the evaluation and enhance its utilization. Relevant UN-Habitat and United Nations entities, national/state governments/ local authorities (particularly the Ministry of Land, Infrastructure and Transport of Korea), beneficiaries, and other stakeholders may participate through a questionnaire, interviews or focus group discussions.
7. Accountability and Responsibilities
The independent Evaluation Unit of UN-Habitat manage the evaluation, with logistical support from the Policy, legislation and Governance Section, the Urban Practices Branch on day-to-day basis and in consultation with the members of the evaluation reference group.
The Evaluation Unit will guide and ensure that the evaluation is contracted to a suitable candidate. The Evaluation Unit will advise on the code of conduct of evaluation and provide technical support as required. The Evaluation Unit will have overall responsibility of ensuring that contractual requirements are met and approve all deliverables (Inception Report with work plan, Draft and Final Evaluation Reports).
An evaluation reference group will be established at the start of the evaluation process with members representing the Policy, Legislation and Governance Section (PLGS), focal points from the Regional Offices or other branches, and the Evaluation Unit. The reference group will be responsible for providing guidance on the process, approving the selection of evaluation, and commenting on the inception report and drafts of the evaluation report.
The evaluation will be conducted by one consultant. The evaluator is responsible for meeting professional and ethical standards in planning and conducting the evaluation, and producing the expected deliverables in accordance with UN-Habitat evaluation policy and norms and standards for evaluation.
The evaluator will receive overall guidance from the reference group, technical support from the Evaluation Unit and logistical support from the Policy Legislation and Governance Section (PLGS).
8. Qualifications and Experience of the Evaluator
The evaluation shall be carried out by one consultant. To ensure complementarity within the evaluation, the consultant should have proven evaluation expertise.
9. Work Schedule
The evaluation will be conducted over the period of three months spread over five months, including the desk review, from January 2022 to June 2022. The evaluation consultant is expected to prepare an inception report with a work plan that will operationalize the evaluation. In the inception report, Theory of Change, understanding of the evaluation questions, methods to be used, limitations or constraints to the evaluation as well as schedule and delivery dates to guide the execution of the evaluation, should be detailed. The provisional timetable is as follows in section 13.
10. Specific Duties
1. Undertake consultations with UN-Habitat personnel and other international and national stakeholders, including focus group discussions
2. Review of background documents relevant to the project
3. Prepare an inception report with evaluation work plan and methodology of work
4. Undertake data collection including document reviews, interviews, consultations and group meetings
5. Undertake surveys to different target audiences
6. Analyse and evaluate findings
7. Draft evaluation report
8. Brief and present preliminary findings to UN-Habitat (Virtual)
9. Undertake consultation with the evaluation reference group for review and comments
10. Revise and prepare final evaluation report
11. Submit final evaluation report including editing and layout
11. Deliverables
The three primary deliverables for this evaluation are:
a) Inception Report with evaluation work plan. Once approved, it will become the key management document for the evaluation, guiding evaluation delivery in accordance with UN-Habitat’s expectations throughout the performance of contract. The draft inception report is reviewed and approved by the evaluation reference group.
b) Draft Evaluation Reports. The evaluation consultant will prepare evaluation report draft(s) to be reviewed by UN-Habitat. The draft should follow UN-Habitat’s standard format for evaluation reports. The draft report is shared with the evaluation reference group for review and comments. The evaluation reference group will review and provide comments on draft reports.
c) Final Evaluation Report (including Executive Summary and Appendices) will be prepared in English and follow the UN-Habitat’s standard format of an evaluation report. The report should not exceed 25 pages (excluding Executive Summary and Appendices). The report should be technically easy to comprehend for non-specialists. The final report will be approved by the reference group.
12. Resources
The funds for the evaluation of the project are made available from the project’s budget.
The remuneration rate of the consultant will be determined by functions performed, qualifications, and experience of the consultant. There are set remuneration rates for consultancies.
Payments will be based on the three main deliverables over the consultancy period. The fees will be paid upon satisfactory delivery of outputs as per agreement. Where applicable, travel costs of the consultant (airplane ticket economy class), transfers, and daily allowance as per the UN rate is payable in addition to the daily fee. Where applicable, daily subsistence allowance will be paid only when working outside the official duty station (home-based) of the consultant.
13. Provisional Time Frame
Task Description
1 Onboarding of consultant - 4 weeks
2 Consultations with UN-Habitat - 8 weeks
3 Review of background documents - 4 weeks
4 Preparation and approval of inception report with work plan and methodology of work - 5 weeks
5 Data collection including document reviews, interviews, consultations and group meetings - 5 weeks
6 Analysis of evaluation findings, commence draft report writing and briefings to UN-Habitat - 4 weeks
7 Presentation of preliminary Findings to UN-Habitat (Virtual) - 1 week
8 Draft Evaluation Report - 2 weeks
9 Review of Evaluation Report - 4 weeks
10 Production delivery of Final Evaluation Report, including editing, and layout - 2 weeks
Competency: COMPETENCIES
Professionalism: Knowledge of concepts and approaches relevant to the urban policy development and the ability to carry out independent policy evaluation and analysis, identify problems, formulate options, and reach conclusions and recommendations.
Communication: Excellent oral and written communication skills, ability to articulate ideas in a clear and precise manner, including the ability to analyse a series of written documents, reports, and summaries.
Planning and organization: Ability to work under pressure, prioritize and plan, coordinate one's own work plan, use time efficiently, and apply common sense in the context of competition deadlines.
Skills:
Good management of information processing tools, Policy analysis, socio-economic data, urban statistics.
Extensive evaluation experience. The consultant should have ability to present credible findings derived from evidence and putting conclusions and recommendations supported by the findings.
Specific knowledge and understanding of National Urban Policies issues and UN-Habitat and its mandate.
It is envisaged that the consultant would have a useful mix of experience and familiarity with public administration in various parts of the world.
Academic Qualifications: Advanced academic degree (Masters) in political sciences, socio-economic, urban policy, urban planning, public policy, land and governance, public administration, other areas related urban studies, or similar relevant fields is required..
Experience:
- Minimum of 10 years of specific experience in positions related to public policy research and / or public policy management, project and Programme evaluation and the like is required.
- Programme management experience in results-based management working with projects/ programmes in the field of policy, urban planning, land, housing and governance is desirable.
- Recent and relevant experience working in developing countries (e.g. Nigeria, I.R. Iran and Myanmar) is desirable.
Language: English and French are the official languages of the UN. For this position, fluency in English (understanding, reading and writing) is a requirement. Knowledge of other UN official languages is an asset.
THE UNITED NATIONS DOES NOT CHARGE A FEE AT ANY STAGE OF THE RECRUITMENT PROCESS (APPLICATION, INTERVIEW MEETING, PROCESSING, OR TRAINING). THE UNITED NATIONS DOES NOT CONCERN ITSELF WITH INFORMATION ON APPLICANTS’ BANK ACCOUNTS.