Information technology security
evaluation – standards,
assurance
IA169 – System verification and assurance
Vashek Matyáš
Resources used – this lecture
• Common Criteria for Information Technology Security
Evaluation, v 3.1, release 5, April 2017
– http://www.commoncriteriaportal.org/
• Separation Kernel Protection Profile Revisited: Choices
and Rationale, T.E. Levin et al., 4th Annual Layered
Assurance Workshop, 2010
• Common Criteria Certification in the UK – UK IT security
evaluation & certification scheme, CESG
• Understanding the Windows EAL4 evaluation, J.S.
Shapiro, IEEE Computer 03/2003
• Security Requirements for Cryptographic Modules, FIPS
PUB 140-2
– http://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.140-2.pdf
2 IA169
Security threat model
• Two views:
1) Description of security threats considered when
designing a (security) solution/system.
2) Definition of (all) possible threats to consider.
• Usual security notion:
– Assets to be protected
– Vulnerabilities of assets and relevant systems
– Threats exploiting the vulnerabilities
– Countermeasures (aim to) mitigate the threats
3 IA169
Threat modelling – approaches
• Attacker centric
– Popular in the research community (following two slides)
• System centric (a.k.a. design/SW centric)
– Taking over in the past decade or so, e.g. used in the
Microsoft Security Development Lifecycle
• Asset centric
– Business logic
• Defender/owner view getting more prominent
4 IA169
Attacker models – Needham & Schroeder
• attacker can eavesdrop and interfere all
communication
– record/modify/replay/inject messages
• node internal processes are safe
– secret keys, encryption process, …
• Comms security classics, paper from 1978 – paper
Using encryption for authentication in large
networks of computers, ACM Communications
5 IA169
Attacker models – Dolev & Yao
• Network = set of abstract machines exchanging
messages.
• Message = formal terms. Terms reveal some of the
message internal structure to the adversary, but not
all.
• Adversary can overhear, intercept, and synthesize
any message, is limited by the constraints of the
cryptographic methods used.
– Sometimes put as “the attacker carries the message.”
• Paper “On the security of public key protocols”,
IEEE Trans. on Information Theory, 1983
6 IA169
Trusted system/product
• Such one that behaves in a way we expect it to
behave
• Can be trusted to only such a functionality that
adheres to the relevant security policy
• Trust
– Belief that (a system…) satisfies given (security)
requirements and specifications
– Chance that (a system…) can breach the (security) policy
without leaving any trace of evidence 
• Truly appreciating this caveat is important!
7 IA169
Common Criteria
8 IA169
• Interests of users, manufacturers, evaluators
• Target of evaluation (TOE) – what is (to be)
evaluated
• Protection profile (PP) (smartcards, biometrics, etc.)
– Catalogued as a self-standing evaluation document
• Security target (ST) – theoretical concept/aim
• Security Functional Requirements (SFRs) –
individual security functions provided by the TOE
• Evaluation of TOE – is the reality corresponding to
theory (ST)?
Common Criteria model
• TOE: Target of
Evaluation – the
evaluated system
• TSF: TOE Security
Functions – HW, SW,
FW used by the TOE
• TSC: TSF Scope of
Control – interactions
under the TOE
security policy
9 IA169
Study of a particular PP
• PP BSI-PP-0025 – German (BSI) Common Criteria
Protection Profile for USB Storage Media
– Link in the IS
• PP organisation:
– the TOE description,
– the TOE security environment,
– the security objectives,
– the IT security requirements and
– the rationale.
10 IA169
PP BSI-PP-0025 – roles in the TOE
• Authorised user (S1)
– Holds the authentication attribute required to access the
TOE protected memory area, in which the confidential data
is stored.
– Can modify the authentication attribute.
11 IA169
PP BSI-PP-0025 – roles in the TOE, cont’d
• Non-authorised user (S2)
– Wishes to access S1’s confidential data in the USB
storage medium’s memory (examples of confidential data
are given in Section 2.5).
– Does not have the authentication attribute to access the
protected data.
– Can obtain a USB storage medium of the same type. Can
try out both logical and physical attacks on this USB
storage medium.
– Can gain possession of the TOE relatively easily since the
TOE has a compact form.
12 IA169
PP BSI-PP-0025 – threats (countered)
• T.logZugriff – Assuming that S2 gains possession of the
TOE, he/she accesses the confidential data on the TOE.
S2 gains logical access by, for example, connecting the
TOE to the USB interface of a computer system.
• T.phyZugriff – Assuming that S2 gains possession of the
TOE, he/she accesses the TOE’s memory by means of a
physical attack. Such an attack could take the following
form, for example: S2 removes the TOE memory and
places it into another USB storage medium which he/she
uses for the purpose of logical access to the memory.
13 IA169
PP BSI-PP-0025 – threats, cont’d
• T.AuthÄndern – Assuming that S2 gains
possession of the TOE, he/she sets a new
authentication attribute, with the result that the
data becomes unusable for S1.
• T.Störung – A failure (e.g., power failure or
operating system error) stops the TOE operating
correctly. As a result, confidential data remains
unencrypted or the TOE’s file system is damaged.
14 IA169
Common Criteria – two catalogues
• Two catalogues of components for specification of
assurance and functionality requirements, with a
standard terminology.
• Functionality – rules governing access to & use of
TOE resources, and thus information and services
controlled by the TOE
• Assurance
– grounds for confidence that an entity meets its security
objectives (CC v2.3)
– grounds for confidence that a TOE meets the SFRs (CC
v3.1)
15 IA169
Assurance is not robustness
• Assurance
– grounds for confidence that an entity meets its security
objectives (CC v2.3)
– grounds for confidence that a TOE meets the SFRs (CC
v3.1)
• Robustness
– characterization of the strength of a security function,
mechanism, service or solution, and the assurance (or
confidence) that it is implemented and functioning correctly
(US DoD definition)
16 IA169
CC – going for evaluation (in a nutshell)
1. Define the product/system for evaluation
2. Specify its functionality
3. Specify the assurance level claimed
4. See details of evaluation with a certification body
5. Prepare evidence
17 IA169
CC: Functional & Assurance Requirements
Security Functional
Requirements (SFRs)
• The core – CC is in a
major part a catalogue
of security functions
• Same product with
different ST => different
SFRs
• Correctness of one
function can depend on
another function
Security Assurance
Requirements (SARs)
• Measures taken to
assure compliance with
the claimed functionality
• Design, development,
evaluation/verification
• CC provides a
catalogue of SARs
18 IA169
CC functional classes
• FAU: SECURITY AUDIT
• FCO: COMMUNICATION
• FCS: CRYPTOGRAPHIC SUPPORT
• FDP: USER DATA PROTECTION
• FIA: IDENTIFICATION AND AUTHENTICATION
• FMT: SECURITY MANAGEMENT
• FPR: PRIVACY
• FPT: PROTECTION OF THE TSF
• FRU: RESOURCE UTILISATION
• FTA: TOE ACCESS
• FTP: TRUSTED PATH/CHANNELS
19 IA169
CC assurance classes
• APE: PROTECTION PROFILE EVALUATION
• ACE: PROTECTION PROFILE CONFIGURATION
EVALUATION
• ASE: SECURITY TARGET EVALUATION
• ADV: DEVELOPMENT
• AGD: GUIDANCE DOCUMENTS
• ALC: LIFE-CYCLE SUPPORT
• ATE: TESTS
• AVA: VULNERABILITY ASSESSMENT
• ACO: COMPOSITION
20 IA169
Assurance through evaluation I
a) analysis and checking of process(es) and
procedure(s);
b) checking that process(es) and procedure(s) are
being applied;
c) analysis of the correspondence between TOE
design representations;
d) analysis of the TOE design representation against
the requirements;
e) verification of proofs;
21 IA169
Assurance through evaluation II
f) analysis of guidance documents;
g) analysis of functional tests developed and the
results provided;
h) independent functional testing;
i) analysis for vulnerabilities (including flaw
hypothesis);
j) penetration testing.
22 IA169
CC assurance paradigms
• assurance based upon an evaluation (active
investigation)
• measuring the validity of the documentation and of
the resulting IT product by expert evaluators with
increasing emphasis on scope, depth, and rigour
• CC does not exclude, nor does it comment upon,
the relative merits of other means of gaining
assurance
23 IA169
CC evaluation assurance scale
The increasing level of effort is based upon:
a) scope – the effort is greater because a larger
portion of the IT product is included;
b) depth – the effort is greater because it is deployed
to a finer level of design and implementation detail;
c) rigour – the effort is greater because it is applied in
a more structured, formal manner.
24 IA169
CC – assurance hierarchy & component
structure
25 IA169
26 IA169
Assurance elements – 3 exclusive classes
1. Developer action elements: activities that shall be
performed by the developer. Further qualified by
evidential material referenced in the following set of
elements. Req’s marked by “D” at the element No.
2. Content and presentation of evidence elements: the
evidence required, what the evidence demonstrates,
what the evidence shall convey. Marked by “C”.
3. Evaluator action elements: activities that shall be
performed by the evaluator. Marked by “E”.
27 IA169
28 IA169
7 evaluation assurance levels (EALs)
• Hierarchical system – higher or new components
– bold faced text in the description for the added
components
• The following slides present first the EALs in the
language of the CC and then from a practical
perspective.
29 IA169
EAL1 – functionally tested
• some confidence in correct operation is required,
but the threats to security are not viewed as serious
– sufficient to simply state the SFRs that the TOE must
meet, rather than deriving them from threats, etc. through
security objectives;
– analysis is supported by a search for potential
vulnerabilities in the public domain and independent
testing (functional and penetration) of the TSF.
– This EAL provides a meaningful increase in assurance
over unevaluated IT.
30 IA169
EAL2 – structurally tested
• assurance by a full security target (with given SFRs);
• analysis of the SFRs, using functional and interface
specs, guidance documentation and basic TOE
architecture description to understand the security
behaviour;
• configuration management system and evidence of
secure delivery procedures;
• independent confirmation of the developer test
results, vulnerability analysis (based upon the above
in italics) demonstrating resistance to penetration
attackers with a basic attack potential.
31 IA169
EAL3 – methodically tested and checked
• architectural description of the TOE design;
• development environment controls
• improved testing coverage of the security
functionality and mechanisms and/or procedures
that provide some confidence that the TOE was not
tampered with during development.
32 IA169
EAL4 – methodically designed, tested, and
reviewed
• complete interface specification, description of the
basic modular design of the TOE, implementation
representation for the entire TSF;
• demonstrating resistance to penetration attackers
with an Enhanced-Basic attack potential;
• additional TOE configuration mgmt incl. automation.
33 IA169
EAL5 – semiformally designed and tested
• modular TSF design;
• comprehensive TOE configuration management;
• semiformal design descriptions, a more structured
(and hence analysable) architecture.
34 IA169
EAL6 – semiformally verified design and
tested
• formal model of select TOE security policies;
• semiformal presentation of the functional
specification and TOE design;
• modular layered and simple TSF design;
• structured development process, development
environment controls, and comprehensive TOE
configuration mgmt incl. complete automation;
• more comprehensive analysis, more architectural
structure (e.g. layering), more comprehensive
independent vulnerability analysis.
35 IA169
EAL7 – formally verified design and tested
• structured presentation of the implementation;
• implementation representation, complete
independent confirmation of the developer test
results;
• comprehensive analysis using formal
representations and formal correspondence, and
comprehensive testing.
36 IA169
CC certified products by country & EAL
37 IA169
EAL1 – functionally tested
• analysis supported by independent testing of a
sample of the security functions;
• applicable where confidence in correct operation is
required but the security threat assessment is low.
• This EAL is particularly suitable for legacy systems
as it should be achievable without the assistance of
the developer.
38 IA169
EAL2 – structurally tested
• analysis exercises a functional and interface
specification and the high-level design of the
subsystems of the TOE;
• independent testing of the security functions;
• evidence required of developer 'black box' testing
and development search for obvious vulnerabilities.
• EAL2 is applicable where a low to moderate level of
independently assured security is required.
39 IA169
EAL3 – methodically tested and checked
• analysis supported by 'grey box' testing, selective
independent confirmation of the developer test
results and evidence of a developer search for
obvious vulnerabilities;
• development environment controls and TOE
configuration management are also required.
• EAL3 for a moderate level of independently assured
security, with a thorough investigation of the TOE
and its development, without incurring substantial reengineering
costs.
40 IA169
EAL4 – methodically designed, tested, and
reviewed
• analysis supported by the low-level design of TOE
modules and a subset of the implementation;
• testing supported by an independent search for
obvious vulnerabilities;
• development controls supported by a life-cycle
model, identification of tools and automated
configuration management.
• EAL4 for a moderate to high level security, where
some additional security-specific engineering costs
may be incurred.
41 IA169
EAL5 – semiformally designed and tested
• analysis includes all of the implementation;
• supplemented by a formal model, a semiformal
presentation of the functional specification and high
level design and a semiformal demonstration of
correspondence;
• search for vulnerabilities must ensure resistance to
penetration attackers with a moderate attack potential;
• covert channel analysis and modular design required.
• EAL5 for a high level of security in a planned
development coupled with a rigorous development
approach.
42 IA169
EAL6 – semiformally verified design and
tested
• analysis supported by a modular approach to
design and a structured presentation of the
implementation;
• independent search for vulnerabilities must ensure
resistance to penetration attackers with a high
attack potential;
• a systematic search for covert channels;
• EAL6 where a specialised security TOE is required
for high risk situations.
43 IA169
EAL7 – formally verified design and tested
• the formal model is supplemented by a formal
presentation of the functional specification and high
level design, showing correspondence;
• evidence of developer 'white box‘ testing and
complete independent confirmation of developer
test results.
• EAL7 where a specialised security TOE is required
for extremely high risk situations.
44 IA169
CC certified products by country & EAL
45 IA169
Famous issue – Windows 2000
• Windows 2000 operating system was certified
(Common Criteria) at EAL-4 in 2002.
– with SP3 and one patch;
– EAL-4, augmented with ALC_FLR.3 (Systematic Flaw
Remediation);
– Microsoft invested millions of dollars and three years of
effort to gain the certification. (S. Bekker, Redmond
Magazine).
• Controlled Access Protection Profile (CAPP)
46 IA169
CAPP assumption A.PEER
“Any other systems with which the TOE communicates
are assumed to be under the same management
control and operate under the same security policy
constraints.
The TOE is applicable to networked or distributed
environments only if the entire network operates under
the same constraints and resides within a single
management domain.
There are no security requirements that address the
need to trust external systems or the communications
links to such systems.”
47 IA169
Controlled Access Protection Profile
• Level of protection appropriate for an assumed nonhostile
and well-managed user community
– requiring protection against threats of inadvertent or casual
attempts to breach the system security.
• The profile is not intended to be applicable to
circumstances in which protection is required
against determined attempts by hostile and well
funded attackers to breach system security.
• CAPP does not fully address the threats posed by
malicious system development or administrative
personnel.
48 IA169
Windows 2000 EAL-4 certification
• EAL4 rating means that you did a lot of paperwork
related to the software process, but says absolutely
nothing about the quality of the software itself. (J.S.
Shapiro)
• System disconnected from networks (at different
security level), disabled media drives, etc.
• Don't hook this to the internet, don't run email, don't
install software unless you can 100 percent trust the
developer, and if anybody who works for you turns
out to be out to get you, you are toast. (J.S. Shapiro)
49 IA169
And Now for Something Completely
Different… about Assurance viewed by…
• Customer – what level of guarantee do I get that
security has been implemented in the product?
• Developer – what (inputs and cooperation) will my
team have to provide for the evaluation?
• Evaluator – did I get all required inputs and did all
tests run OK to confirm the claim?
• Operator – what assumptions can I build on when
preparing for my actions?
50 IA169
Security Requirements for Cryptographic
Modules
• Federal Information Processing Standard (FIPS)
Publication 140-2 (FIPS PUB 140-2)
• published May 2001 and last updated Dec 2002
– FIPS 140-3 (Draft) – proposed revision, hanging in the air
since 2009 (!)
• 4 levels, hierarchical levelling
• 11 functions (requirements):
– Cryptographic module specification; Cryptographic module
ports and interfaces; Role, services, and authentication;
Physical security; Operational environment; Cryptographic
key management; Mitigation of other attacks; …
51 IA169
FIPS 140-2 Annexes (drafts)
• Annex A: Approved Security Functions (Draft 2011)
• Annex B: Approved Protection Profiles (Draft 2007)
• Annex C: Approved Random Number Generators
(Draft 2010)
• Annex D: Approved Key Establishment Techniques
(Draft 2011)
52 IA169
FIPS 140-2 levels I
Level 1
– basic security requirements (e.g., certified algorithm);
– no specific physical security mechanisms.
Level 2
– features that show evidence of tampering – physical
access to the plaintext cryptographic keys and critical
security parameters (CSPs) within the module
• including tamper-evident coatings or seals that must be
broken to attain,
– or pick-resistant locks on covers or doors to protect
against unauthorized physical access.
53 IA169
FIPS 140-2 levels II
Level 3
– high probability of detecting and responding to attempts at
physical access, use or modification of the cryptographic
module;
– may include the use of strong enclosures and tamper
detection/response circuitry that zeroes all plain text CSPs
when the removable covers/doors of the cryptographic
module are opened.
54 IA169
FIPS 140-2 levels III
Level 4
– physical security mechanisms provide a complete
envelope of protection around the cryptographic module
with the intent of detecting and responding to all
unauthorized attempts at physical access;
– protects a cryptographic module against a security
compromise due to environmental conditions or
fluctuations outside of the module's normal operating
ranges for voltage and temperature;
– for operation in physically unprotected environments.
55 IA169
FIPS 140-2
• Level 2 – operating system at EAL2+
• Level 3 – operating system at EAL3+
– and additional req.: Security Policy Model (ADV_SPM.1)
• Level 4 – operating system at EAL4+
56 IA169
Nice standards and theory, but…
• OpenSSL derivative FIPS-certified, found flawed
– that particular one de-certified, others including the flaw
not
• Dual EC DRBG defective by design mandated for
FIPS 140-2
• IBM 4758 (with CCA API) – level 4
– easy/fast logical attacks on CCA API
• Safenet Luna CA3
– disassembling showed no potting material
– undocumented API functions
– functionality in breach of security policy
57 IA169
Study of another PP development
• Security kernel – used to simulate a distributed
environment, introduced by J Rushby (1981) as a solution
to the difficulties and problems that had arisen in the
development and verification of large, complex security
kernels that were intended to “provide multilevel secure
operation on general-purpose multi-user systems.”
• U.S. Government Protection Profile for Separation Kernels
in Environments Requiring High Robustness, v 1.03
• Study paper “Separation Kernel Protection Profile
Revisited: Choices and Rationale”, T.E. Levin et al., URL:
http://fm.csl.sri.com/LAW/2010/law2010-03-Levin-Nguyen-Irvine.pdf
58 IA169