1
Qualitative methods evaluation in PS


2
Analytical methods for public projects (types)
§Quantitative methods
§One-criteria
•
§Multi-criteria
•
§
§Managerial methods
§Empirical methods
•
§Qualitative methods
•SWOT analysis, causal analysis
§

3
Qualitative methods for evaluation
•
•To determine mutual relationships between gained data
•
•Is used to identify and explain causal relationships and potential benefits
•
•Analyse difficult to quantify phenomena and processes
•
•Identify mutual connections
•
•Provide explanation of causal relationships
•
§

Causal models
§
§Measure impacts of many factors of different projects or programs
§
§E.g. Simultation models, microeconomics models or statistical models
§
§Models are used to evaluate impacts of differente projects / programs / policy
§
§Risks presence => potential risk of misleading results
§
4

§„circle“ problem = project / program / policy and its consequences / effects
§influence of external factors and their share on consequences
§
§Example:
§
§Infrastructure development (eg. highway network) around cities and its impact on cities in the
locality which are influenced
§
§=> Economic development of influenced cities (compare to others) ???
§
§Solution could be detailed descripiton of the project with help of various suitable methods (case
study, interviews, document etc.)
§
§
§
5

Non-causal methods
§
§eg. participants observation, interviews, case study, preference identification etc.
§
§political dimension problem in decision-making process within application on projects / programs /
policy in public sector
§
§primary vs secondary data
§
§
§
§
§
§
§
6

SWOT analysis
§Analysis of strength (S) and weak (W) sides, opportunities (O) and threats (T)
§
§Aim: to get subject / service / product from A to B (positioning)
§
§How to do?
§Identify S and W
§Assess O and T (inside and outside)
§
§Use in public sector (city development, projects and grants etc.)
§
§Pragmaticism empirism
§
7

Disadvantage and limits of SWOT analysis
§Wrong use or execution of the method
§Speed of execution
§Depth of execution
§Wrong interpretation (causal analysis)
§Wrong positioning in the matrix
§Use of analysis for already decided goals (bias)
§Limits of brainstorming
§Organization goals vs community goals
§Width of the working group
8

SWOT analysis
9
Blair (2007)

SWOT analysis (processing)
10


SWOT for more complex projects / subjects
11


CS: SWOT analysis for city Němčice nad Hanou
12


CS Hrušovany n. J.
Zdroj: Cyprich (2014)
13

Focus groups vs roundtable
§Focus groups
§Time: app. 60-90 minutes
§Small groups of people interested in the issue (around 7 people)
§Together discussing about previously chosen issue / topic
§Advantage: lower time consumption
§Advantage: developing debate / discussion and potential space for generating of new opinions and
views
§
§Roundtable
§Time: 90 minutes +
§Large group with potential of representativeness (see rules for representativeness)
§Multidisciplinarity and more topics
§Advantage: connection and confrontation of municipal council (mayor) with residents
§Advantage: potentially higher objectivity => findings objectivisation
§
14

Further methods
§Case study
§For more detalied description and analysis of the project
§Collecting data (qualitative, quantitative)
§Critical analysis, implementation analysis or analysis of project impacts
§See structure and form
§
§Interview
§To find out specific type of information or gain more complex view
§Type: free, semi-structured and structured
§Suitable selection of the sample and interviewer experience
§
§Questionnaire survey
§Segmentation of the sample (representativeness??).
§See rules for questionnaire preparation
§Closed questions vs vs open vs semi-open question
§Test (structure, answers, calibration)
§Data coding for further processing
§
§
§
15

Helpful tips
§If secondary data are not available, it is necessary to collect primary data (expensive,
time-demanding etc…)
§Not everybody is interested in full use of evaluation or asessment – sensitive issues and areas
(politicians, public officers, etc..)
§Fear of different outcomes
§Fear of criticism or feedback
§Therefore it is important to consult with them definition of the problems (why the program is
applied) and also the plan of evaluation / asessment
§Because they are providing information and data
§Because they can help with interpretation of results and realization of recommendations
§Beacuse if they become active adversaries, they can sabotage the project or its evaluation or
asessment
§
16

Causal analysis
§Approach: searching and exploring of connections between two or more phenomena / processes
§Goal: Get deeper into essence of phenomena / processes observed
§Goal: Get through to causal connections
§Cause = existence of phenomena (A) is resulting (consequence) into existence of different
phenomena (B)
§Disadvantage
§Sensible to the level of awareness and subjective approach of individual investigators
17

Causal analysis
§Deductive type
§We know consequences => We dont know their causes (revealing)
§Induktive type
§We know their causes => We dont know consequences (revealing)
§
§Tools:
-Kepner method
-Causal chain
-Tree of causal relations
-Ishikawa
-Diagram of causes and consequences
-Causal diagnosis
-Analysis of the force field
-Pareto analysis
-Causal layers
§
18

Causal chain
§Causal deduction
§Assumption (1 cause => 1 consequence)
§
§Example:
1.High price of the service
2.Low demand
3.Low volume of the production of this service
4.Reduced utilization of production factors
5.Unused production capacity (expensive production)
6.
§Alternatively can be used for the tree of causal realations (assumption: 1 consequence has more
causes)
19

Thank for your attention J
20