papers.csv

pdf_name,text
p87-biggio.pdf,|Is Data Clustering in Adversarial Settings Secure?

Battista Biggio

Università di Cagliari

Piazza d’Armi

09123, Cagliari, Italy

battista.biggio@diee.unica.it

Davide Ariu

Università di Cagliari

Piazza d’Armi

09123, Cagliari, Italy

davide.ariu@diee.unica.it

Ignazio Pillai

Università di Cagliari

Piazza d’Armi

09123, Cagliari, Italy
pillai@diee.unica.it

Marcello Pelillo

Università Ca’ Foscari di

Venezia

Via Torino, 155

30172 Venezia-Mestre
pelillo@dais.unive.it

Samuel Rota Bulò

FBK-irst

Via Sommarive, 18
38123, Trento, Italy
rotabulo@fbk.eu

Fabio Roli

Università di Cagliari

Piazza d’Armi

09123, Cagliari, Italy
roli@diee.unica.it

ABSTRACT
Clustering algorithms have been increasingly adopted in se-
curity applications to spot dangerous or illicit activities.
However, they have not been originally devised to deal with
deliberate attack attempts that may aim to subvert the
clustering process itself. Whether clustering can be safely
adopted in such settings remains thus questionable. In this
work we propose a general framework that allows one to
identify potential attacks against clustering algorithms, and
to evaluate their impact, by making specific assumptions on
the adversary’s goal, knowledge of the attacked system, and
capabilities of manipulating the input data. We show that
an attacker may significantly poison the whole clustering
process by adding a relatively small percentage of attack
samples to the input data, and that some attack samples
may be obfuscated to be hidden within some existing clus-
ters. We present a case study on single-linkage hierarchical
clustering, and report experiments on clustering of malware
samples and handwritten digits.

Categories and Subject Descriptors
D.4.6 [Security and Protection]: Invasive software (e.g.,
viruses, worms, Trojan horses); G.3 [Probability and Statis-
tics]: Statistical computing; I.5.1 [Models]: Statistical;
I.5.2 [Design Methodology]: Clustering design and eval-
uation; I.5.3 [Clustering]: Algorithms

General Terms
Security, Clustering.

Keywords
Adversarial learning, Unsupervised Learning, Clustering, Se-
curity Evaluation, Computer Security, Malware Detection.

Permission to make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for profit or commercial advantage and that copies bear this notice and the full cita-
tion on the first page. Copyrights for components of this work owned by others than
ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or re-
publish, to post on servers or to redistribute to lists, requires prior specific permission
and/or a fee. Request permissions from permissions@acm.org.
AISec’13, November 4, 2013, Berlin, Germany.
Copyright 2013 ACM 978-1-4503-2488-5/13/11 ...$15.00.
http://dx.doi.org/10.1145/2517312.2517321

1.

INTRODUCTION

Clustering algorithms are nowadays a fundamental tool
for the data analysts as they allow them to make inference
and gain insights on large sets of unlabeled data. Appli-
cations of clustering span across a large number of different
domains, such as market segmentation [14, 26], classification
of web pages [10], and image segmentation [12]. In the spe-
cific domain of computer security, clustering algorithms have
been recently exploited to solve plenty of different problems,
e.g., spotting fast-flux domains in DNS traffic [24], gaining
useful insights on tools and sources of attacks against Inter-
net websites [25], detecting repackaged Android applications
[16] and (Android) mobile malware [9], and even automati-
cally generating signatures for anti-virus software to enable
detection of HTTP-based malware [23].

In many of the aforementioned scenarios, a large amount
of data is often collected in the wild, in an unsupervised man-
ner. For instance, malware samples are often collected from
the Internet, by means of honeypots, i.e., machines that pur-
posely expose known vulnerabilities to be infected by mal-
ware [28], or other ad hoc services, like VirusTotal.1 Given
that these scenarios are intrinsically adversarial, it may thus
be possible for an attacker to inject carefully crafted samples
into the collected data in order to subvert the clustering pro-
cess, and make the inferred knowledge useless. This raises
the issue of evaluating the security of clustering algorithms
against carefully designed attacks, and proposing suitable
countermeasures, when required. It is worth noting that re-
sults from the literature of clustering stability [29] can not
be directly exploited to this end, since the noise induced
by adversarial manipulations is not generally stochastic but
specifically targeted against the clustering algorithm.

The problem of learning in adversarial environments has
recently gained increasing popularity, and relevant research
has been done especially in the area of supervised learning
algorithms for classification [6, 8, 17, 3], and regression [13].
On the other hand, to the best of our knowledge only few
works have implicitly addressed the issue of security evalua-
tion related to the application of clustering algorithms in ad-
versarial settings through the definition of suitable attacks,

1http://virustotal.com

87while we are not aware of any work that proposes specific
countermeasures to attacks against clustering algorithms.

The problem of devising specific attacks to subvert the
clustering process was first brought to light by Dutrisac and
Skillicorn [11, 27]. They pointed out that some points can be
easily hidden within an existing cluster by forming a fringe
cluster, i.e., by placing such points sufficiently close the bor-
der of the existing cluster. They further devised an attack
that consists of adding points in between two clusters to
merge them, based on the notion of bridging. Despite this
pioneering attempts, a framework for the systematic security
evaluation of clustering algorithms in adversarial settings is
still missing, as well as a more general theory that takes
into account the presence of the adversary to develop more
secure clustering algorithms.

In this work we aim to take a first step to fill in this gap, by
proposing a framework for the security evaluation of cluster-
ing algorithms, which allows us to consider several potential
attack scenarios, and to devise the corresponding attacks, in
a more systematic manner. Our framework, inspired from
previous work on the security evaluation of supervised learn-
ing algorithms [6, 17, 3], is grounded on a model of the
attacker that allows one to make specific assumptions on
the adversary’s goal, knowledge of the attacked system, and
capability of manipulating the input data, and to subse-
quently formalize a corresponding optimal attack strategy.
This work is thus explicitly intended to provide a cornerstone
for the development of an adversarial clustering theory, that
should in turn foster research in this area.

The proposed framework for security evaluation is pre-
sented in Sect. 2.
In Sect. 3 we derive worst-case attacks
in which the attacker has perfect knowledge of the attacked
system.
In particular, we formalize the notion of (worst-
case) poisoning and obfuscation attacks against a clustering
algorithm, respectively in Sects. 3.1 and 3.2.
In the for-
mer case, the adversary aims at maximally compromising
the clustering output by injecting a number of carefully de-
signed attack samples, whereas in the latter one, she tries to
hide some attack samples into an existing cluster by manipu-
lating their feature values, without significantly altering the
clustering output on the rest of the data. As a case study, we
evaluate the security of the single-linkage hierarchical clus-
tering against poisoning and obfuscation attacks, in Sect. 4.
The underlying reason is simply that the single-linkage hier-
archical clustering has been widely used in security-related
applications [4, 16, 23, 24]. To cope with the computa-
tional problem of deriving an optimal attack, in Sects. 4.1
and 4.2 we propose heuristic approaches that serve well our
purposes. Finally, in Sect. 5 we conduct synthetic and real-
world experiments that demonstrate the effectiveness of the
proposed attacks, and subsequently discuss limitations and
future extensions of our work in Sect. 6.

2. ATTACKING CLUSTERING

In this section we present our framework to analyze the
security of clustering approaches from an adversarial pattern
recognition perspective. It is grounded on a model of the ad-
versary that can be exploited to identify and devise attacks
against clustering algorithms. Our framework is inspired by
a previous work focused on attacking (supervised) machine
learning algorithms [6], and it relies on an attack taxonomy
similar to the one proposed in [17, 3]. As in [6], the adver-

sary’s model entails the definition of the adversary’s goal,
knowledge of the attacked system, and capability of manip-
ulating the input data, according to well-defined guidelines.
Before moving into the details of our framework, we intro-
duce some notation. Clustering is the problem of organizing
a set of data points into groups referred to as clusters in a
way that some criteria is satisfied. A clustering algorithm
can thus be formalized in terms of a function f mapping a
given dataset D = {xi}n
i=1 to a clustering result C = f (D).
We do not specify the mathematical structure of C at this
point of our discussion because there exist different types
of clustering requiring different representations, while our
model applies to any of them. Indeed, C might be a hard or
soft partition of D delivered by partitional clusterings algo-
rithms such as k-means, fuzzy c-means or normalized cuts,
or it could be a more general family of subsets of D such as
the one delivered by the dominant sets clustering algorithm
[22], or it can even be a parametrized hierarchy of subsets
(e.g., linkage-type clustering algorithms).

2.1 Adversary’s goal

Similarly to [6, 17, 3], the adversary’s goal can be defined
according to the attack specificity, and the security violation
pursued by the adversary. The attack specificity can be
targeted, if it affects solely the clustering of a given subset
of samples; or indiscriminate, if it potentially affects the
clustering of any sample. Security violations can instead
affect the integrity or the availability of a system, or the
privacy of its users.

Integrity violations amount to performing some malicious
activity without significantly compromising the normal sys-
tem operation.
In the supervised learning setting [17, 3],
they are defined as attacks aiming at camouflaging some
malicious samples (e.g., spam emails) to evade detection,
without affecting the classification of legitimate samples. In
the unsupervised setting, however, this definition can not be
generally applied since the notion of malicious or legitimate
class is not generally available. Therefore, we regard in-
tegrity violations as attacks aiming at deflecting the group-
ing for specific samples, while limiting the changes to the
original clustering. For instance, an attacker may obfuscate
some samples to hide them in a different cluster, without
excessively altering the initial clusters.

Availability violations aim to compromise the functional-
ity of the system by causing a denial of service. In the super-
vised setting, this translates into causing the largest possible
classification error [17, 6, 7]. According to the same ratio-
nale, in the unsupervised setting we can consider attacks
that significantly affect the clustering process by worsening
its result as much as possible.

Finally, privacy violations may allow the adversary to ob-
tain information about the system’s users from the clustered
data by reverse-engineering the clustering process.

2.2 Adversary’s knowledge

The adversary can have different degrees of knowledge of
the attacked system. They can be defined by making specific
assumptions on the points (k.i)-(k.iv) described below.

(k.i) Knowledge of the data D: The adversary might
know the data D or only a portion of it. More realistically,
she may not know D exactly, but she may be able to obtain a
surrogate dataset sampled from the same distribution as D.

88In practice, this can be obtained by collecting samples from
the same source from which samples in D were collected;
e.g., honeypots for malware samples [28].

(k.ii) Knowledge of the feature space: The adversary
could know how features are extracted from each sample.
Similarly to the previous case, she may know how to com-
pute the whole feature set, or only a subset of the features.
(k.iii) Knowledge of the algorithm: The adversary’s
could be aware of the targeted clustering algorithm and how
it organizes the data into clusters; e.g., the criterion used to
determine the cluster set from a hierarchy in hierarchical
clustering.

(k.iv) Knowledge of the algorithm’s parameters:
The attacker may even know how the parameters of the
clustering algorithm have been initialized (if any).

Perfect knowledge. The worst-case scenario in which
the attacker has full knowledge of the attacked system, is
usually referred to as perfect knowledge case [6, 7, 19, 8,
17, 3].
(k.i) the
data, (k.ii) the feature representation, (k.iii) the clustering
algorithm, and (k.iv) its initialization (if any).

In our case, this amounts to knowing:

2.3 Adversary’s capability

The adversary’s capability defines how and to what extent
the attacker can control the clustering process. In the super-
vised setting [17, 6], the attacker can exercise a causative or
exploratory influence, depending on whether she can control
training and test data, or only test data. In the case of clus-
tering, however, there is not a test phase in which some data
has to be classified. Accordingly, the adversary may only ex-
ercise a causative influence by manipulating part of the data
to be clustered.2 This is often the case, though, since this
data is typically collected in an unsupervised manner.

We thus consider a scenario in which the attacker can
add a maximum number of (potentially manipulated) sam-
ples to the dataset D. This is realistic in several practical
cases, e.g., in the case of malware collected through honey-
pots [28], where the adversary may easily send (few) samples
without having access to the rest of the data. This amounts
to controlling a (small) percentage of the input data. An
additional constraint may be given in terms of a maximum
amount of modifications that can be done to the attack sam-
ples. In fact, to preserve their malicious functionality, mali-
cious samples like spam emails or malware code may not be
manipulated in an unconstrained manner. Such a constraint
can be encoded by a suitable distance measure between the
original, non-manipulated attack samples and the manipu-
lated ones, as in [6, 20, 17, 3].

2.4 Attack strategy

Once the adversary’s goal, knowledge and capabilities have
been defined, one can determine an optimal attack strategy
that specifies how to manipulate the data to meet the ad-
versary’s goal, under the restriction given by the adversary’s
knowledge and capabilities. In formal terms, we denote by
Θ the knowledge space of the adversary. Elements of Θ hold
information about the dataset D, the clustering algorithm

2One may however think of an exploratory attack to a clus-
tering algorithm as an attack in which the adversary aims to
gain information on the clustering algorithm itself, although
she may not necessarily manipulate any data to this end.

f , and its parametrization, according to (k.i)-k(.iv). To
model the degree of knowledge of the adversary we con-
sider a probability distribution µ over Θ. The entropy of
µ indicates the level of uncertainty of the attacker. For
example, if we consider a perfect-knowledge scenario like
the one addressed in the next section, we have that µ is a
Dirac measure peaked on an element θ0 ∈ Θ (with null en-
tropy), where θ0 = (D, f, · · · ) holds the information about
the dataset, the algorithm and any other of the informations
listed in Sect.2.2. Further, we assume that the adversary is
given a set of attack samples A that can be manipulated be-
fore being added to the original set D. We model with the
function Ω(A) the family of sample sets that the attacker
can generate according to her capability as a function of the
set of initial attack samples A. The set A can be empty, if
the attack samples are not required to fulfill any constraint
on their malicious functionality, i.e., they can be generated
from scratch (as we will see in the case of poisoning attacks).
Finally, the adversary’s goal given the knowledge θ ∈ Θ is
expressed in terms of an objective function g(A′; θ) ∈ R
that evaluates how close the modified data set integrating
the (potentially manipulated) attack samples A′ is to the ad-
versary’s goal. In summary, the attack strategy boils down
to finding a solution to the following optimization problem:

maximize Eθ∼µ[g(A′; θ)]

s.t. A′ ∈ Ω(A) .

(1)

where Eθ∼µ[·] denotes the expectation with respect to θ be-
ing sampled according to the distribution µ.

3. PERFECT KNOWLEDGE ATTACKS

In this section we provide examples of worst-case integrity
and availability security violations in which the attacker has
perfect knowledge of the system, as described in Sect. 2.2.
We respectively refer to them as poisoning and obfuscation
attacks. Since the attacker has no uncertainty about the sys-
tem, we set µ = δ{θ0}, where δ is the Dirac measure and θ0
represents exact knowledge of the system. The expectation
in (1) thus yields g(A′; θ0).

3.1 Poisoning attacks

Similarly to poisoning attacks against supervised learn-
ing algorithms [7, 19], we define poisoning attacks against
clustering algorithms as attacks in which the data is tainted
to maximally worsen the clustering result. The adversary’s
goal thus amounts to violating the system’s availability by
indiscriminately altering the clustering output on any data
point. To this end, the adversary may aim at maximizing
a given distance measure between the clustering C obtained
from the original data D (in the absence of attack) and the
clustering C′ = fD(D′) obtained by running the clustering
algorithm on the contaminated data D′, and restricting the
result to the samples in D, i.e., fD = πD ◦ f where πD is
a projection operator that restricts the clustering output to
the data samples in D. We regard the tainted data D′ as
the union of the original dataset D with the attack samples
in A′, i.e., D′ = D ∪ A′. The goal can thus be written as
g(A′; θ0) = dc(C, fD(D ∪ A′)), where dc is the chosen dis-
tance measure between clusterings. For instance, if f is a
partitional clustering algorithm, any clustering result can be
represented in terms of a matrix Y ∈ Rn×k, each (i, k)th com-
ponent being the probability that the ith sample is assigned

89to the kth cluster. Under this setting, a possible distance
measure between clusterings is given by:

scalar. Consequently, the function Ω representing the at-
tacker’s capacity is given by

dc(Y, Y′) = kYY⊤ − Y′Y′⊤kF ,

(2)

where k · kF is the Frobenius norm. The components of the
matrix YY⊤ represent the probability of two samples to be-
long to the same cluster. When Y is binary, thus encoding
hard clustering assignments, this distance counts the num-
ber of times two samples have been clustered together in one
clustering and not in the other, or vice versa. In general, de-
pending on the nature of the clustering result, other ad-hoc
distance measures can be adopted.

As mentioned in Sect. 2.3, we assume that the attacker
can inject a maximum of m data points into the original
data D, i.e.  A′  ≤ m. This realistically limits the adversary
to manipulate only a given, potentially small fraction of the
dataset. Clearly, the value of m will be considered as a pa-
rameter in our evaluation to investigate the robustness of
the given clustering algorithm against an increasing control
of the adversary over the data. We further define a box con-
straint on the feature values xlb ≤ x ≤ xub, to restrict the
attack points to lie in some fixed interval (e.g., the smallest
box that includes all the data points). Hence, we define the
function Ω encoding the adversary’s capabilities as follows:

Ωp = n{a′
i}m

i=1 ⊂ R

d : xlb ≤ a′

i ≤ xub for i = 1, · · · , mo .

Note that Ω depends on a set of target samples A in (1), but
since A is empty in this case, we write Ωp instead of Ω(∅).
The reason is simply that, in the case of a poisoning attack,
the attacker aims to find a set of attack samples that do
not have to carry out any specific malicious activity besides
worsening the clustering process.

In summary, the optimal attack strategy under the afore-
mentioned hypothesis amounts to solving the following op-
timization problem derived from (1):

maximize dc(C, fD(D ∪ A′))

s.t. A′ ∈ Ωp .

(3)

3.2 Obfuscation attacks

Obfuscation attacks are violations of the system integrity
through targeted attacks. The adversary’s goal here is to
hide a given set of initial attack samples A within some ex-
isting clusters by obfuscating their content, possibly without
altering the clustering results for the other samples. We de-
note by Ct the target clustering involving samples in D ∪ A′
the attacker is aiming to, being A′ the set of obfuscated
attack samples. With the intent to preserve the cluster-
ing result C on the original data samples, we impose that
πD(Ct) = C, while the cluster assignments for the samples
in A′ are freely determined by the attacker. As opposed to
the poisoning attack, here the attacker is interested in push-
ing the final clustering towards the target clustering and
therefore her intention is to minimize the distance between
Ct and C′ = f (D ∪ A′). Accordingly, the goal function g in
this case is defined as g(A′; θ0) = −d(Ct, f (D ∪ A′)).

As for the adversary’s capability, we assume that the at-
tacker can perturb the target samples in A to some maxi-
mum extent. We model this by imposing that ds(A, A′) ≤
dmax, where ds is a measure of divergence between the two
sets of samples A and A′ and dmax is a nonnegative real

Ωo(A) = n{a′

i=1 : ds(A, A′) ≤ dmaxo .
i} A 

The distance ds can be defined in different ways. For in-
stance, in the next section we define ds(A, A′) as the largest
Euclidean distance among corresponding elements in A and
A′, i.e.,

ds(A, A′) = max

i=1,...,m

kai − a′

ik2

(4)

where we assume A = {ai}m
i=1. This
choice allows us to bound the divergence between the origi-
nal target samples in A and the manipulated ones, as typi-
cally done in adversarial learning [20, 17, 8, 6].

i=1 and A′ = {a′

i}m

In summary, the attack strategy in the case of obfusca-
tion attacks can be obtained as the solution of the following
optimization program derived from (1):

minimize dc(Ct, f (D ∪ A′))

s.t. A′ ∈ Ωo(A) .

(5)

4. A CASE STUDY ON SINGLE-LINKAGE

HIERARCHICAL CLUSTERING

In this section we solve a particular instance of the opti-
mization problems (3) and (5), corresponding respectively to
the poisoning and obfuscation attacks described in Sects. 3.1
and 3.2, against the single-linkage hierarchical clustering.
The motivation behind this specific choice of clustering algo-
rithm is that, as mentioned in Sect. 1, it has been frequently
exploited in security-sensitive tasks [4, 16, 23, 24].

Single-linkage hierarchical clustering is a bottom-up al-
gorithm that produces a hierarchy of clusterings, as any
other hierarchical agglomerative clustering algorithm [18].
The hierarchy is represented by a dendrogram, i.e., a tree-
like data structure showing the sequence of cluster fusion
together with the distance at which each fusion took place.
To obtain a given partitioning of the data into clusters, the
dendrogram has to be cut at a certain height. The leaves
that form a connected sub-graph after the cut are considered
part of the same cluster. Depending on the chosen distance
between clusters (linkage criterion), different variants of hi-
erarchical clustering can be defined.
In the single-linkage
variant, the distance between any two clusters C1, C2 is de-
fined as the minimum Euclidean distance between all pairs
of samples in C1 × C2.

For both poisoning and obfuscation attacks, we will model
the clustering output as a binary matrix Y ∈ {0, 1}n×k, in-
dicating the sample-to-cluster assignments (see Sect. 3.1).
Consequently, we can make use of the distance measure dc
between clusterings defined in Eq. (2). However, to obtain
a given set of clusters from the dendrogram obtained by the
single-linkage clustering algorithm, we will have to specify
an appropriate cut criterion.

4.1 Poisoning attacks

For poisoning attacks against single-linkage hierarchical
clustering, we aim to solve the optimization problem given
by Eq. (3). As already mentioned, since the clustering is ex-
pressed in terms of a hierarchy, we have to determine a suit-
able dendrogram cut in order to model the clustering output

90 ✂✄

 

✁✂✄

✁

☎✂✄

☎

✲☎✂✄

✲✁

✲✁✂✄

✲ 

✲ ✂✄

✁✻

 ✂✄

✁✹

✁ 

✁☎

✽

✻

✹

 

 

✁✂✄

✁

☎✂✄

☎

✲☎✂✄

✲✁

✲✁✂✄

✲ 

✲ ✂✄

✹✂✄

✹

✸✂✄

✸

 ✂✄

 

✁✂✄

✁

☎✂✄

✲  ✲✁✂✄ ✲✁ ✲☎✂✄ ☎ ☎✂✄ ✁ ✁✂✄

✲  ✲✁✂✄ ✲✁ ✲☎✂✄ ☎ ☎✂✄ ✁ ✁✂✄

Figure 1: Poisoning single-linkage hierarchical clustering. In each plot, samples belonging to different clusters
are represented with different markers and colors. The left and middle plot show the initial partitioning of
the given 100 data points into k = 4 clusters. The objective function of Eq. 3 (shown in colors) for our greedy
attack ( A′  = 1) is respectively computed with hard (left plot) and soft assignments (middle plot), i.e., with
binary Y and posterior estimates. The k − 1 = 3 bridges obtained from the dendrogram are highlighted with
red lines. The rightmost plot shows how the partitioning changes after m = 20 attack samples (highlighted
with red circles) have been greedily added.

as a binary matrix Y. In this case, we assume that the clus-
tering algorithm selects the cut, i.e., the number of clusters,
that achieves the minimum distance between the clustering
obtained in the absence of attack C and the one induced by
the cut, i.e., min dc(C, fD(D ∪ A′)). Although this may not
be a realistic cut criterion, as the ideal clustering C is not
known to the clustering algorithm, this worst-case choice for
the adversary gives us the minimum performance degrada-
tion incurred by the clustering algorithm under attack.

Let us now discuss how Problem (3) can be solved. First,
note that it is not possible to predict analytically how the
clustering output Y′ changes as the set of attack samples
A′ is altered, since hierarchical clustering does not have a
tractable, underlying analytical interpretation.3 One possi-
ble answer consists in a stochastic exploration of the solution
space (e.g. by simulated annealing). This is essentially done
by perturbing the input data A′ a number of times, and eval-
uating the corresponding values of the objective function by
running the clustering algorithm (as a black box) on D ∪ A′.
The set A′ that provides the highest objective value is even-
tually retained. However, to find an optimal configuration of
attack samples A′, one should repeat this procedure a very
large number of times. To reduce computational complexity,
one may thus consider efficient search heuristics specifically
tailored to the considered clustering algorithm.

For the above reason, we consider a greedy optimization
approach where the attacker aims at finding a local maxi-
mum of the objective function by adding one attack sample
at a time, i.e.,  A′  = m = 1.
In this case, we can more
easily understand how the objective function changes as the
inserted attack point varies, and define a suitable heuris-
tic approach. An example is shown in the leftmost plot of
Fig. 1. This plot shows that the objective function exhibits a
global maximum when the attack point is added in between
clusters that are sufficiently close to each other. The reason
is that, when added in such a location, the attack point op-

3In general, even if the clustering algorithm has a clearer
mathematical formulation, it is not guaranteed that a good
analytical prediction can be found. For instance, though k-
means clustering is well-understood mathematically, its vari-
ability to different initializations makes it almost impossible
to reliably predict how its output may change due to data
perturbation.

erates as a bridge, causing the two clusters to be merged in
a single cluster, and the objective function to increase.

Bridge-based heuristic search. Based on this observa-
tion, we devised a search heuristic that considers only k − 1
potential attack samples, being k the actual number of clus-
ters found by the single-linkage hierarchical clustering at a
given dendrogram cut.
In particular, we only considered
the k − 1 points lying in between the connections that have
been cut to separate the k given clusters from the top of
the hierarchy, highlighted in our example in the leftmost
plot of Fig. 1. These connections can be directly obtained
from the dendrogram, i.e., we do not have to run any post-
processing algorithm on the clustering result. Thus, one is
only required to evaluate the objective function k − 1 times
for selecting the best attack point. We will refer to this ap-
proach as Bridge (Best) in Sect. 5.1. The rightmost plot in
Fig. 1 shows the effect of our greedy attack after that m = 20
attack points have been inserted. Note how the initial clus-
ters are fragmented into smaller clusters that tend to contain
points which originally belonged to different clusters.

Approximating Y′. To further reduce the computational
complexity of our approach, i.e., to avoid re-computing the
clustering and the corresponding value of the objective func-
tion k − 1 times for each attack point, we consider another
heuristic approach. The underlying idea is simply to select
the attack sample (among the k − 1 bridges suggested by
our bridge-based heuristic search) that lies in between the
largest clusters.
In particular, we assume that the attack
point will effectively merge the two adjacent clusters, and
thus modify Y′ accordingly (without re-estimating its real
value by re-running the clustering algorithm). To this end,
for each point belonging to one of the two clusters, we set
to 1 (0) the value of Y′ corresponding to the first (second)
cluster. Once the estimated Y′ is computed, we evaluate the
objective function using the estimated Y′, and select the at-
tack point that maximizes its value. We will refer to this
approach as Bridge (Hard) in Sect. 5.1.

Approximating Y′ with soft clustering assignments.
Finally, we discuss another variation to the latter discussed
heuristic approach, which we will refer to as Bridge (Soft),
in Sect. 5.1. The problem arises from the fact that our ob-
jective function exhibits really abrupt variations, since it is

91computed on hard cluster assignments (i.e., binary matri-
ces Y′). Accordingly, adding a single attack point at a time
may not reveal connections that can potentially merge large
clusters after few attack iterations, i.e., using more than one
attack sample. To address this issue, we approximate Y′ with
soft clustering assignments. To this end, the element y′
ik of Y′
is estimated as the posterior probability of point xi belong-
ing to cluster ck, i.e., y′
ik = p(ck xi) = p(xi ck)p(ck)/p(xi).
The prior p(ck) is estimated as the number of samples be-
longing to ck divided by the total number of samples, the
likelihood p(xi ck) is estimated with a Gaussian Kernel Den-
sity Estimator (KDE) with bandwidth parameter h:

p(xi ck) =

1

 ck  Xxj ∈ck

exp(cid:18)−

  xi − xj  2

h

(cid:19) ,

(6)

and the evidence p(xi) is obtained by marginalization over
the given set of clusters.

Worth noting, for too small values of h, the posterior es-
timates tend to the same value, i.e., each point is likely to
be assigned to any cluster with the same probability. When
h is too high, instead, each point is assigned to one cluster,
and the objective function thus equals that corresponding to
the original hard assignments. In our experiments we simply
avoid these limit cases by selecting a value of h comparable
to the average distance between all possible pairs of samples
in the dataset, which gave reasonable results.

An example of the smoother approximation of the objec-
tive function provided by this heuristic is shown in the mid-
dle plot of Fig. 1. Besides, this technique also provides a reli-
able approximation of the true objective: although its values
are significantly re-scaled, the global maximum is still found
in the same location. The smooth variations that character-
ize the approximated objective influence the choice of the
best candidate attack point.
In fact, attack points lying
on bridges that may potentially connect larger clusters after
some attack iterations may be sometimes preferred to attack
points that can directly connect smaller and closer clusters.
This may lead to a larger increase in the true objective func-
tion as the number of injected attack points increases.

4.2 Obfuscation attacks

In this section we solve (5) assuming the worst-case (perfect-

knowledge) scenario against the single-linkage clustering al-
gorithm. Recall that the attacker’s goal in this case is to
manipulate a given set of non-obfuscated samples A such
that they are clustered according to a desired configuration,
e.g., together with points in an existing, given cluster, with-
out altering significantly the initial clustering that would be
obtained in the absence of manipulated attacks.

As in the previous case, to represent the output of the clus-
tering algorithm as a binary matrix Y representing clustering
assignments, and thus compute dc as given by Eq. 2, we have
to define a suitable criterion for cutting the dendrogram.
Similarly to poisoning attacks, we define an advantageous
criterion for the clustering algorithm, that gives us the low-
est performance degradation incurred under this attack: we
select the dendrogram cut that minimizes dc(C⋆, f (D ∪ A′)),
where C⋆ represents the optimal clustering that would be ob-
tained including the non-manipulated attack samples, i.e.,
C⋆ = f (D ∪ A). The reason is that, to better contrast an ob-
fuscation attack, the clustering algorithm should try to keep
the attack points corresponding to the non-manipulated set

A into their original clusters. For instance, in the case of
malware clustering, non-obfuscated malware may easily end
up in a well-defined cluster, and, thus, it may be subse-
quently categorized in a well-behaved malware family. While
the adversary tries to manipulate malware to have it clus-
tered differently, the best solution for the clustering algo-
rithm would be to obtain the same clusters that would be
obtained in the absence of attack manipulation.

We derive a simple heuristic to get an approximate solu-
tion of (5) assuming ds to be defined as in (4). We assume
that, for each sample ai ∈ A, the attacker selects the closest
sample di ∈ D belonging to the cluster to which ai should
belong to, according to the attacker’s desired clustering Ct.
To meets the constraint given by Ωo in Eq. 5, the attacker
then determines for each ai ∈ A a new sample a′
i ∈ A along
the line connecting ai and di in a way not to exceed the
maximum distance dmax from ai, i.e., a′
i = ai + α(di − ai),
where α = min(1, dmax/kdi − aik2).

5. EXPERIMENTS

We present here some experiments to evaluate the effec-
tiveness of the poisoning and obfuscation attacks devised
in Sect. 4 against the single-linkage hierarchical clustering
algorithm, under perfect knowledge of the attacked system.

5.1 Experiments on poisoning attacks

For the poisoning attack, we consider three distinct cases:
a two-dimensional artificial data set, a realistic application
example on malware clustering, and a task in which we aim
to cluster together distinct handwritten digits.

5.1.1 Artificial data

We consider here the standard two-dimensional banana-
shaped dataset from PRTools,4 for which a particular in-
stance is shown in Fig. 1 (right and middle plot). We fix the
number of initial clusters to k = 4, which yields our original
clustering C in the absence of attack.

We repeat the experiment five times, each time by ran-
domly sampling 80 data points. In each run, we add up to
m = 20 attack samples, that simulates a scenario in which
the adversary can control up to 20% of the data. As de-
scribed in Sect. 4.1, the attack proceeds greedily by adding
one sample at a time. After adding each attack sample, we
allow the clustering algorithm to change the number of clus-
ters from a minimum of 2 to a maximum of 50. The criterion
used to determine the number of clusters is to minimize the
distance of the current partitioning with the clustering in
the absence of attack, as explained in details in Sect. 4.1.

We consider five attack strategies, described in the follow-

ing.

Random: the attack point is selected at random in the

minimum box that encloses the data.

Random (Best): k − 1 attack points are selected at ran-
dom, being k the actual number of clusters at a given attack
iteration. Then, the objective function is evaluated for each
point, and the best one is chosen.

Bridge (Best): The k−1 bridges suggested by our heuristic

approach are evaluated, and the best one is chosen.

4http://prtools.org

92n
o

 

i
t
c
n
u
F
e
v
i
t
c
e
b
O

j

)
k
(
 
s
r
e
t
s
u
C
m
u
N

 

l

Banana

60
50
40
30
20
10
0
0% 2% 5% 7% 9% 12% 15% 18% 20%
6
4
14
12
10
8
0% 2% 5% 7% 9% 12% 15% 18% 20%
Fraction of samples controlled by the attacker

n
o

 

i
t
c
n
u
F
e
v
i
t
c
e
b
O

j

180
160
140
120
100
80
60
40
20
0
0%

Malware

n
o
i
t
c
n
u
F
 
e
v
i
t
c
e
b
O

j

1%

2%

3%

4%

5%

Digits

 

800
700
600
500
400
300
200
100
0
0.0% 0.2% 0.4% 0.6% 0.8% 1.0%

Random
Random (Best)
Bridge (Best)
Bridge (Soft)
Bridge (Hard)

 

)
k
(
 
s
r
e
t
s
u
C
m
u
N

 

l

30
25
20
15
10
5
0%
5%
Fraction of samples controlled by the attacker

1%

2%

3%

4%

)
k
(
 
s
r
e
t
s
u
C
m
u
N

 

l

100
80
60
40
20
0
0.0% 0.2% 0.4% 0.6% 0.8% 1.0%
Fraction of samples controlled by the attacker

Figure 2: Results for the poisoning attack averaged over five runs on the Banana-shaped dataset (first
column), the Malware dataset (second column), and the Digit dataset (third column). Top plots show the
variation of the objective function dc(f (D), fD(D ∪ A′)) as the fraction of samples controlled by the adversary
increases. Bottom plots report the number of clusters selected after the insertion of each attack sample.

Banana (20%)

Malware (5%)

Digits (1%)

Split

Merge

Split

Merge

Split

Merge

Random
Random (Best)
Bridge (Best)
Bridge (Soft)
Bridge (Hard)

1.15 ± 0.22
1.40 ± 0.34
2.40 ± 0.60
3.85 ± 1.35
3.75 ± 1.43

1.29 ± 0.06
1.54 ± 0.30
1.40 ± 0.23
1.22 ± 0.11
1.21 ± 0.23

1.00 ± 0.00
1.00 ± 0.00
1.49 ± 0.23
2.76 ± 0.84
2.41 ± 0.73

1.00 ± 0.00
1.34 ± 0.39
1.31 ± 0.17
1.12 ± 0.09
1.10 ± 0.10

1.00 ± 0.00
1.00 ± 0.00
33.9 ± 0.15
33.9 ± 0.15
34.0 ± 0.00

1.00 ± 0.00
1.00 ± 0.00
1.02 ± 0.00
1.02 ± 0.00
1.02 ± 0.00

Table 1: Split and Merge averaged values and standard deviations for the Banana-shaped dataset (at 20%
poisoning), the Malware dataset (at 5% poisoning), and the Digit dataset (at 1% poisoning).

Bridge (Hard): The k − 1 bridges are evaluated here by
predicting the clustering output Y′ as discussed in Sect. 4.1
(i.e., assuming that the corresponding clusters will be merged),
using hard clustering assignments.

Bridge (Soft): This is the same strategy as Bridge (Hard),
except for the fact that we consider soft clustering assign-
ments when modifying Y′. To this end, as discussed in
Sect. 4.1, we use a Gaussian KDE. We set the kernel band-
width h as the average distance between each possible pair
of samples in the data. On average, h ≈ 2 in each run.

It is worth remarking that Random (Best) and Bridge
(Best) require the objective function to be evaluated k − 1
times at each iteration to select the best candidate attack
sample. This means that the clustering algorithm has to be
run k − 1 times at each step. Instead, the other methods
do not require us to re-run the clustering algorithm to select
the attack point. Their complexity is therefore significantly
lower than the aforementioned methods.

The results averaged over the five runs are reported in
Fig. 2 (first column). From the top plot one may appre-
ciate how the methods based on the bridge-based heuris-
tics achieve similar values of the objective function, while
clearly outperforming the random-based methods. Further,
as reasonably expected, Random (Best) outperforms Ran-
dom since it considers the best point over k − 1 attempts.
Nevertheless, even selecting a random attack sample, in this
case, turned out to significantly affect the clustering results.
The bottom plot provides us a better understanding of
how the attack effectively works. The main effect is in-

deed to fragment the original clusters into a high number of
smaller clusters. In particular, after the insertion of m = 20
data points, i.e., when 20% of the data is controlled by the
attacker, the selected number of clusters increases from 4 to
about 7-14 clusters depending on the considered method.

To further clarify the effect of the attack on the clustering
algorithm, we consider two measures referred to as Split and
Merge in Table 5.1, which are given as follows. Let C and C′
be the initial and the final clustering restricted to elements
in D, respectively, and let C be a binary matrix, each entry
Ckk′ indicating the co-occurrence of at least one sample in
the kth cluster of C and in the k′th cluster of C ′. Then, the
above measures are given as:

Split = mean

i Xj

Cij , Merge = mean

j Xi

Cij .

Intuitively, split quantifies to what extent the initial clusters
are fragmented across different final clusters, while merge
quantifies to what extent the final clusters contain samples
that originally belonged to different initial clusters.

From Table 5.1, it can be appreciated how, for the most
effective attacks, i.e., Bridge (Soft) and Bridge (Hard), the
initial clusters are split into approximately 3.8 clusters, while
the final clusters merge approximately 1.2 initial clusters, on
average. This clarifies how the proposed attack eventually
compromises the initial clustering: it tends to fragment the
initial clusters into smaller ones, and to merge together final
clusters which originally came from different clusters. Bridge
(Best) tends instead to induce a lower number of final clus-

93❘ ✁✂✄☎

❘ ✁✂✄☎

✭✝✞✟✠✡

✝❇☛✂☞✞

✭✝✞✟✠✡

✝❇☛✂☞✞
✭✌✄✍✠✡

✝❇☛✂☞✞
✭✎ ❇✂✡

✥

✆

✥✶

Figure 3: Attack samples produced by the five
strategies at iterations 1, 2 and 10, for the digit data.

ters, i.e., the clustering algorithm tends to merge more final
clusters than splitting initial ones. However, this is not the
optimal choice according to the attacker’s goal.

5.1.2 Malware clustering

We consider here a more realistic application example in-
volving malware clustering, and in particular a simplified
version of the algorithm for behavioral malware clustering
proposed in [23]. The ultimate goal of this approach is to
obtain malware clusters that can aid the automatic gen-
eration of high quality network signatures, which can be
used in turn to detect botnet command-and-control (C&C)
and other malware-generated communications at the net-
work perimeter. With respect to the original algorithm, we
made the following simplifications:

(a) we consider only the first of the two clustering steps
carried out by the original system. The algorithm pro-
posed in [23] clusters samples through two consecutive
stages, named coarse-grain and fine-grain clustering, re-
spectively. Here, we just focus on the coarse-grain clus-
tering, which is based on a set of numeric features.

(b) We consider a subset of six statistical features (out of
the seven used by the original algorithm). They are: (1)
number of GET requests; (2) number of POST requests;
(3) average length of the URLs; (4) average number of
parameters in the request; (5) average amount of data
sent by POST requests; and (6) average length of the
response. We exclude the seventh feature, i.e., the total
number of HTTP requests, as it is redundant with respect
to the first and the second feature. All feature values are
re-scaled in [0, 1] as in the original work.

(c) We use the single-linkage hierarchical clustering instead
of the BIRCH algorithm [30], since this modification does
not significantly affect the quality of the clustering re-
sults, as the authors demonstrated in [23].

For the purpose of this evaluation, we use a subset of 1,000
samples taken from Dataset 1 of [23]. This dataset consists
of distinct malware samples (no duplicates) collected dur-
ing March 2010 from a number of different malware sources,
including MWCollect [1], Malfease [2], and commercial mal-
ware feeds. As in the previous setting, we repeat the ex-
periments five times, by randomly selecting a subset of 475
samples from the available set of 1, 000 malware data in each
run. The initial set of clusters C, as in [23], is selected as the
partitioning that minimizes the value of the Davies-Bouldin
Index (DBI) [15], a measure that characterizes dispersion

and closeness of clusters. We consider the cuts of the initial
dendrogram that yield from 2 to 25 clusters, and choose the
one corresponding to the minimum DBI. This yields approx-
imately 9 clusters in each run. While the attack proceeds,
the clustering algorithm can choose a number of clusters
ranging from 2 to 50. The attacker can inject up to 25 at-
tack samples, that amounts to controlling up to 5% of the
data. The value of h for the KDE used in Bridge (Soft) is
set as the average distance between pairs of samples, which
turns out to be approximately 0.2 in each run.

Results are shown in Fig. 2 (second column). The effect of
the attack is essentially the same as in the previous experi-
ments on the Banana-shaped data, although here there is a
significant difference among the performances of the bridge-
based methods. In particular, Bridge (Soft) gradually out-
performs the other approaches as the fraction of injected
samples approaches 5%. The reason is that, as qualitatively
discussed in Sect. 4.1, this heuristic approach tends to bridge
clusters which are too far to be bridged with a single attack
point, and are thus disregarded by Bridge (Best) and not al-
ways chosen by Bridge (Hard). It is also worth noting that,
in this case, the Random approach is totally ineffective. In
particular, no change in the objective function is observed
for this method, and the number of clusters increases lin-
early as the attack proceeds. This means simply that the
clustering algorithm produces a new cluster for each newly-
injected attack point, making the attack totally ineffective.
The behavior exhibited by the different attack strategies is
also confirmed by the Split and Merge values reported in Ta-
ble 5.1. Here, the most effective methods, i.e., again Bridge
(Soft) and Bridge (Hard), split the 3 initial clusters each
into 2.7 and 2.4 final clusters, on average, yielding a total
number of clusters of about 20-25 clusters. Similarly to the
previous experiments, Bridge (Best) yields a lower number
of final clusters, as it induces more the clustering algorithm
to cluster together samples that originally belonged to dif-
ferent initial clusters.

5.1.3 Handwritten digits

We finally repeat the experiments described in the pre-
vious sections on the MNIST handwritten digit data [21].5
In this dataset, each digit is size-normalized and centered,
and represented as a grayscale image of 28 × 28 pixels. Each
pixel is raster-scan ordered and its value is directly consid-
ered as a feature. The dimensionality of the feature space is
thus 784, a much higher value than that considered in the
previous cases. We further normalize each feature (pixel) in
[0, 1] by dividing its value by 255.

We focus here on a subset of data consisting of the three
digits ‘0’, ‘1’, and ‘6’. To obtain three initial clusters, each
representing one of the considered digits, we first compute
the average digit for each class (i.e., the average ‘0’, ‘1’,
and ‘6’), and then select 700 samples per class, by retaining
the closest samples to the corresponding average digit. We
repeat the experiments five times, each time by randomly
selecting 330 samples per digit from the corresponding set
of 700 pre-selected samples. While the attack proceeds, the
clustering algorithm can choose a number of clusters ranging
from 2 to 100. We assume that the attacker can inject up to
10 attack samples, that amounts to controlling up to 1% of

5This dataset is publicly available in Matlab format at http:
//cs.nyu.edu/~roweis/data.html.

94the data. The value of h for the KDE used in Bridge (Soft)
is set as in the previous case, based on the average distance
between all pairs of samples. For this dataset, it turns out
that h ≈ 1 in each run.

Results are shown in Fig. 2 (third column). With respect
to the previous experiments on the Banana-shaped data,
and on the Malware data, the results here are significantly
different. In particular, note how the Random and Random
(Best) approaches are totally ineffective here. Similarly to
the previous case in malware clustering, the clustering algo-
rithm essentially defeats the attack influence by creating a
new cluster for each attack sample. The underlying reason
is that, in this case, the feature space has a very high dimen-
sionality, and, thus, sampling only k − 1 points at random is
not enough to find a suitable attack point. In other words,
if an attack sample is not very well crafted, it may be easily
isolated from the rest of the data. Although increasing the
dimensionality may thus seem a suitable countermeasure to
protect clustering against random attacks, this drastically
increases its vulnerability to well designed attack samples.
Note indeed how the clustering is already significantly wors-
ened when the adversary only controls a fraction as small
as of 0.2% of the data. In fact, the number of final clusters
raises immediately to the maximum allowed number of 100.
This is also clarified in Table 5.1, where it can be appreci-
ated how the initial clusters are fragmented into an average
of 33 final clusters for the bridge-based methods. Note how-
ever that, in this case, the final clusters are almost pure, i.e.,
the attack algorithm does not succeed in merging together
samples coming from different initial clusters.

In Fig. 3 we also show some of the attack samples that
are produced by the five attack strategies, at different attack
iterations. The random-based attacks clearly produce very
noisy images which yield a completely ineffective attack, as
already mentioned. Instead, the initial attacks considered
by bridge-based methods (at iteration 1 and 2) resemble
effectively the digits corresponding to the two initial clusters
that they aim to connect (‘0’ and ‘6’, and ‘1’ and‘6’). Since
the attack completely destroys the three initial clusters after
very few attack samples have been added, at later iterations
(e.g., iteration 10), the bridge-based methods tend to enforce
some connection within the cluster belonging to the ‘0’ digit,
probably trying to merge some of the final clusters together.
However, since the maximum number of allowed clusters has
been already reached, no further improvement is observed in
the objective function.

5.2 Experiments on obfuscation attacks

For the obfuscation attack, we present an experiment on
handwritten digits, using again the MNIST digit data de-
scribed in Sect. 5.1.3.

5.2.1 Handwritten digits

We consider the same initial clusters of Sect. 5.1.3, con-
‘0’,
sisting of 330 samples for each of the following digits:
‘1’, and ‘6’. As in the previous case, we average the results
over five runs, each time selecting the initial 330 samples per
cluster from the pre-selected sets of 700 samples per digit.
In this case, however, we consider a further initial cluster of
100 samples corresponding to the digit ‘3’ (which are also
randomly sampled from a pre-selected set of 700 samples of
‘3’, chosen with the same criterion used in Sect. 5.1.3 to end
up in the same cluster, initially). These represent the attack

)
k
(
 
s
r
e

l

t
s
u
C
m
u
N

 

5
4.6
4.2
3.8
3.4
3
0

1

2

3

4

5
d max

6

7

8

9 10

Figure 4: Results for the obfuscation attack aver-
aged over five runs on the Digit dataset. The top
plots shows the variation of the objective function
for the attacker dc(Ct, f (D ∪ A′)) and for the cluster-
ing algorithm dc(f (D ∪ A), f (D ∪ A′)) as the maximum
amount of modifications dmax to the initial attack
samples A increases. The bottom plot reports the
corresponding average number of selected clusters.

✵ ✵

✷ ✵

Figure 5: An example of how a digit ‘3’ is gradually
manipulated to resemble the closest ‘6’, for different
values of dmax.

✸ ✵

✹ ✵

✺ ✵

✼ ✵

samples A that the attacker aims to obfuscate. We remind
the reader that the attacker’s goal in this case is to manip-
ulate some samples to have them clustered according to a
desired criterion, without affecting significantly the normal
system operation.
In particular, we assume here that the
attacker can manipulate samples corresponding to the digit
‘3’ in order to have them clustered together with the cluster
corresponding to the digit ‘6’, while preserving the initial
clusters. In other words, the desired clustering output for
the attacker consists of three clusters: one corresponding to
the ‘0’ digit, one corresponding to the ‘1’ digit, and the latter
corresponding to the digits ‘6’ and ‘3’. These constraints can
be easily encoded as a desired clustering output Ct through
a binary matrix Yt. This reflects exactly Problem 5, where
the attacker aims at minimizing dc(Ct, f (D ∪ A′)).

On the other hand, as explained in Sect. 3.2, the clustering
algorithm attempts to keep the attack points corresponding
to the digit ‘3’ into a well-separated cluster from the re-
maining digits, i.e., it selects the number of clusters that
minimizes dc(C⋆, f (D ∪ A′)), which can thus be regarded as
the objective function for the clustering algorithm. In this
case, C⋆ is the clustering obtained on the initial data and
the non-manipulated attack samples, i.e., C ⋆ = f (D ∪ A).
The results for the above discussed obfuscation attack are
given in Fig. 4, where we report the values of the objective
function for the attacker and for the clustering algorithm,
as well as the number of selected clusters, as a function of
the maximum amount of allowed modifications to the at-
tack samples, given in terms of the maximum Euclidean dis-

95tance dmax (see Eq. 4). The results clearly show that the
objective function of the attacker tends to decrease, while
that of the clustering algorithm generally increases. The
reason is that, initially, the clustering algorithm correctly
separates the four clusters associated to the four distinct
digits, whereas as dmax increases, the attack digits ‘3’ are
more and more altered to resemble the closest ‘6’s, and are
then gradually merged to their cluster. The number of clus-
ters does not decrease immediately to 3 as one would expect
since, while manipulating the attack samples, their cluster
is fragmented into smaller ones (typically, two or three clus-
ters). The reason is that, to remain as close as possible to
the ideal C⋆, the clustering algorithm avoids some of the ‘3’s
to immediately join the cluster of ‘6’s by fragmenting the
cluster of ‘3’s.

When dmax takes on values approximately in [3, 4], the
clustering algorithm creates only three clusters, correspond-
ing effectively to the attacker’s goal Ct (this is witnessed
by the fact that the averaged attacker’s objective is almost
zero). Surprisingly, though, as soon as dmax becomes greater
than 4, the number of clusters raises again to 4, and some
of the attack samples are again separated from the cluster
of ‘6’s, worsening the adversary’s objective. This is due to
the fact that, when dmax ≈ 3 or 4, some of the attack points
work as bridges and successfully connect the remaining ‘3’s
to the cluster of ‘6’s, whereas when these points are further
shifted towards the cluster of ‘6’s, the algorithm can success-
fully split the two clusters again. Based on this observation,
a smarter attacker may even manipulate only a very small
subset of her attack samples to create proper bridges and
connect the remaining non-manipulated samples to the de-
sired cluster. We however left a quantitatively investigation
of this approach to future work.

In Fig. 5 we finally report an example of how a digit ‘3’
is manipulated by our attack to be hidden in the cluster
associated to the digit ‘6’.
It is worth noting how, when
dmax ∈ [2, 4], the original attack sample still significantly re-
sembles the initial ‘3’: this shows that the adversary’s goal
can be achieved without altering too much the initial at-
tack samples, which is clearly a strong desideratum for the
attacker in adversarial settings.

6. CONCLUSIONS AND FUTURE WORK

In this paper, we addressed the problem of evaluating the
security of clustering algorithms in adversarial settings, by
providing a framework for simulating potential attack sce-
narios. We devised two attacks that can significantly com-
promise availability and integrity of the targeted system.
We demonstrated with real-world experiments that single-
linkage clustering may be significantly vulnerable to delib-
erate attacks, either when the adversary can only control a
very small fraction of the input data, or when she slightly
manipulates her attack samples. This shows that attack-
ing clustering algorithms with tailored strategies can signif-
icantly alter their output to meet the adversary’s goal.

Admittedly, one of the causes of the vulnerability of single-
linkage resides in its inter-cluster distance, which solely de-
pends on the closest points between clusters, and thus al-
lowed for an efficient constructing of bridges. It is reason-
able to assume that algorithms based on computing averages
(e.g., k-means) or density estimation might be more robust
to poisoning, although not necessarily robust to obfuscation

attacks. However, the results of our empirical evaluation can
not be directly generalized to different algorithms, and more
investigation should thus be carried out in this respect.

In general, finding the optimal attack strategy given an
arbitrary clustering algorithm is a hard problem and we have
to rely on heuristic algorithms in order to carry out our
analysis. For the sake of efficiency, these heuristics should
be heavily dependent on the targeted clustering algorithm,
as in our case. However, it would be interesting to exploit
more general approaches that ideally treat the clustering
algorithm as a black box and find a solution by performing
a stochastic search on the solution space (e.g. by simulated
annealing), or an educated exhaustive search (e.g. by using
branch-and-bound techniques).

In this work we did not address the problem of countering
attacks by designing more secure clustering algorithms. We
only assumed that the clustering algorithm can select a dif-
ferent number of clusters (optimal according to its goal) after
each attack iteration. More generally, though, one can de-
sign a clustering algorithm that explicitly takes into account
the adversary’s presence, and her optimal attack strategy,
e.g., by modeling clustering in adversarial settings as a game
between the clustering algorithm and the attacker. This has
been done in the case of supervised learning, to improve
the security of learning algorithms against evasion attempts
[8], and similarly, in the regression setting [13]. Other ap-
proaches may more directly encode explicit assumptions on
how the data distribution changes under attack, similarly to
[5]. We left this investigation to future work.

Another possible future extension of our work would be
to consider a more realistic setting in which the attacker has
limited knowledge of the attacked system. To this end, the
upper bound on the performance degradation incurred under
attack provided by our worst-case analysis may be exploited
to evaluate the effectiveness of attacks devised under limited
knowledge (i.e., how close they can get to the worst case).
One limitation of our approach may be the so-called in-
verse feature-mapping problem [17, 6], i.e., the problem of
finding a real attack sample corresponding to a desired fea-
ture vector (as the ones suggested by our attack strategies).
In the reported experiments, this was not a significant prob-
lem since modifications to the given feature values could be
directly mapped to manipulations on the real attack sam-
ples. Although inverting the feature mapping may be a cum-
bersome task for more complicated feature representations,
this remains a common problem of optimal attacks in adver-
sarial learning, and it has to be addressed in an application-
specific manner, depending on the given feature space.

As a further future development, we plan to establish a
link between the evaluation of the security of clustering al-
gorithms and the problem of determining the stability of a
clustering, which has been already addressed in the liter-
ature and used as a device for model selection (see, e.g.,
[29]). Indeed, stable clusterings can be regarded as secure
under specific non-targeted attacks like, e.g., perturbation
of points with Gaussian noise.

Understanding robustness of clustering algorithms against
carefully targeted attacks under a more theoretical perspec-
tive (e.g., by devising theoretical bounds that evaluate the
impact of single attack points on the clustering output) may
also be a promising research direction. Some results from
clustering stability may be also exploited to this end.

967. ACKNOWLEDGMENTS

This work has been partly supported by the Regional Ad-
ministration of Sardinia (RAS), Italy, within the projects
“Security of pattern recognition systems in future internet”
(CRP-18293), and “Advanced and secure sharing of mul-
timedia data over social networks in the future Internet”
(CRP-17555). Both projects are funded within the frame-
work of the regional law L.R. 7/2007, Bando 2009. The
opinions, findings and conclusions expressed in this paper
are solely those of the authors and do not necessarily reflect
the opinions of any sponsor.

8. REFERENCES

[1] Collaborative Malware Collection and Sensing.

https://alliance.mwcollect.org.

[2] Project Malfease. http://malfease.oarci.net.
[3] M. Barreno, B. Nelson, R. Sears, A. D. Joseph, and

J. D. Tygar. Can machine learning be secure? In
ASIACCS ’06: Proc. 2006 ACM Symposium on
Information, Computer and Communications Security,
pages 16–25, NY, USA, 2006. ACM.

[4] U. Bayer, P. M. Comparetti, C. Hlauschek, C. Kr¨ugel,

and E. Kirda. Scalable, behavior-based malware
clustering. In NDSS. The Internet Society, 2009.

[5] B. Biggio, G. Fumera, and F. Roli. Design of robust

classifiers for adversarial environments. In IEEE Int’l
Conf. on Systems, Man, and Cybernetics (SMC),
pages 977–982, 2011.

[6] B. Biggio, G. Fumera, and F. Roli. Security evaluation

of pattern classifiers under attack. IEEE Trans. on
Knowledge and Data Eng., 99(PrePrints):1, 2013.

[7] B. Biggio, B. Nelson, and P. Laskov. Poisoning attacks

against support vector machines. In J. Langford and
J. Pineau, editors, 29th Int’l Conf. on Machine
Learning. Omnipress, 2012.

[8] M. Br¨uckner, C. Kanzow, and T. Scheffer. Static

prediction games for adversarial learning problems. J.
Mach. Learn. Res., 13:2617–2654, 2012.

[9] I. Burguera, U. Zurutuza, and S. Nadjm-Tehrani.

Crowdroid: behavior-based malware detection system
for android. In Proc. 1st ACM workshop on Security
and Privacy in Smartphones and Mobile devices,
SPSM ’11, pages 15–26, NY, USA, 2011. ACM.

[15] M. Halkidi, Y. Batistakis, and M. Vazirgiannis. On

clustering validation techniques. Journal of Intelligent
Information Systems, 17(2-3):107–145, Dec. 2001.
[16] S. Hanna, L. Huang, E. Wu, S. Li, C. Chen, and

D. Song. Juxtapp: a scalable system for detecting
code reuse among android applications. In Proc. 9th
Int’l Conf. on Detection of Intrusions and Malware,
and Vulnerability Assessment, DIMVA’12, pages
62–81, Berlin, Heidelberg, 2013. Springer-Verlag.

[17] L. Huang, A. D. Joseph, B. Nelson, B. Rubinstein,

and J. D. Tygar. Adversarial machine learning. In 4th
ACM Workshop on Artificial Intelligence and Security
(AISec 2011), pages 43–57, Chicago, IL, USA, 2011.

[18] A. K. Jain and R. C. Dubes. Algorithms for clustering

data. Prentice-Hall, Inc., NJ, USA, 1988.

[19] M. Kloft and P. Laskov. Online anomaly detection

under adversarial impact. In Proc. 13th Int’l Conf. on
Artificial Intell. and Statistics, pages 405–412, 2010.

[20] A. Kolcz and C. H. Teo. Feature weighting for

improved classifier robustness. In Sixth Conf. on
Email and Anti-Spam (CEAS), CA, USA, 2009.

[21] Y. LeCun, L. Jackel, L. Bottou, A. Brunot, C. Cortes,

J. Denker, H. Drucker, I. Guyon, U. M¨uller,
E. S¨ackinger, P. Simard, and V. Vapnik. Comparison
of learning algorithms for handwritten digit
recognition. In Int’l Conf. on Artificial Neural
Networks, pages 53–60, 1995.

[22] M. Pavan and M. Pelillo. Dominant sets and pairwise

clustering. IEEE Trans. on Pattern Analysis and
Machine Intelligence, 29(1):167–172, 2007.

[23] R. Perdisci, D. Ariu, and G. Giacinto. Scalable
fine-grained behavioral clustering of http-based
malware. Computer Networks, 57(2):487 – 500, 2013.

[24] R. Perdisci, I. Corona, and G. Giacinto. Early

detection of malicious flux networks via large-scale
passive DNS traffic analysis. IEEE Trans. on
Dependable and Secure Comp., 9(5):714–726, 2012.

[25] F. Pouget, M. Dacier, J. Zimmerman, A. Clark, and

G. Mohay. Internet attack knowledge discovery via
clusters and cliques of attack traces. J. Information
Assurance and Security, Vol. 1, Issue 1, March 2006.

[26] G. Punj and D. W. Stewart. Cluster analysis in
marketing research: Review and suggestions for
application. J. Marketing Res., 20(2):134, May 1983.

[10] C. Castillo and B. D. Davison. Adversarial web search.

[27] D. B. Skillicorn. Adversarial knowledge discovery.

Foundations and Trends in Information Retrieval,
4(5):377–486, May 2011.

[11] J. G. Dutrisac and D. Skillicorn. Hiding clusters in

adversarial settings. In IEEE Int’l Conf. on
Intelligence and Security Informatics (ISI 2008),
pages 185–187, 2008.

[12] D. A. Forsyth and J. Ponce. Computer Vision: A

Modern Approach. Prentice Hall, 2011.

[13] M. Großhans, C. Sawade, M. Br¨uckner, and

T. Scheffer. Bayesian games for adversarial regression
problems. In J. Mach. Learn. Res. - Proc. 30th Int’l
Conf. on Machine Learning (ICML), volume 28, 2013.

[14] P. Haider, L. Chiarandini, and U. Brefeld.

Discriminative clustering for market segmentation. In
Proc. 18th ACM SIGKDD Int’l Conf. Knowledge
Discovery and Data Mining, KDD ’12, pages 417–425,
NY, USA, 2012. ACM.

IEEE Intelligent Systems, 24:54–61, 2009.
[28] L. Spitzner. Honeypots: Tracking Hackers.

Addison-Wesley Professional, 2002.

[29] U. von Luxburg. Clustering stability: An overview.

Foundations and Trends in Machine Learning,
2(3):235–274, 2010.

[30] T. Zhang, R. Ramakrishnan, and M. Livny. Birch: an

efficient data clustering method for very large
databases. In Proc. 1996 ACM SIGMOD Int’l Conf.
on Management of data, SIGMOD ’96, pages 103–114,
NY, USA, 1996. ACM.

97|
p635-wang.pdf,|Unauthorized Origin Crossing on Mobile Platforms:      

Threats and Mitigation 

Rui Wang 

Luyi Xing 

Microsoft Research 

ruiwan@microsoft.com 

Indiana University 

luyixing@indiana.edu 

XiaoFeng Wang 
Indiana University 
xw7@indiana.edu 

 

Shuo Chen 

 

Microsoft Research 

shuochen@microsoft.com 

ABSTRACT 
With  the  progress  in  mobile  computing,  web  services  are 
increasingly delivered to their users through mobile apps, instead 
of  web  browsers.  However,  unlike  the  browser,  which  enforces 
origin-based security policies to mediate the interactions between 
the web content from different sources,  today’s mobile OSes do 
not have a comparable security mechanism to control the cross-
origin communications between apps, as well as those between an 
app  and  the  web.  As  a  result,  a  mobile  user’s  sensitive  web 
resources could be exposed to the harms from a malicious origin.  
In this paper, we report the first systematic study on this mobile 
cross-origin  risk.  Our  study  inspects  the  main  cross-origin 
channels on Android and iOS, including intent, scheme and web-
accessing  utility  classes,  and  further  analyzes  the  ways  popular 
web services (e.g., Facebook, Dropbox, etc.) and their apps utilize 
those channels to serve other apps. The research shows that lack 
of origin-based protection opens the door to a wide spectrum of 
cross-origin attacks. These attacks are unique to mobile platforms, 
and their consequences are serious: for example, using carefully 
designed  techniques  for  mobile  cross-site  scripting  and  request 
forgery,  an  unauthorized  party  can  obtain  a  mobile  user’s 
Facebook/Dropbox authentication credentials and record her text 
input. We report our findings to related software vendors, who all 
acknowledged  their  importance.  To  address  this  threat,  we 
designed an origin-based protection mechanism, called Morbs, for 
mobile  OSes.  Morbs  labels  every  message  with  its  origin 
information,  lets  developers  easily  specify  security  policies, and 
enforce the policies on the mobile channels based on origins. Our 
evaluation demonstrates the effectiveness of our new technique in 
defeating  unauthorized  origin  crossing,  its  efficiency  and  the 
convenience for the developers to use such protection. 

Categories and Subject Descriptors 
D.4.6 [Operating Systems]: Security and Protection – access 
controls, invasive software 
Keywords: Android, iOS, same-origin policy, mobile 
platform. 

1.  INTRODUCTION 
The popularity of smartphones, tablets and other mobile devices 
has  brought  in  a  plethora  of  software  applications  designed  for 
 
Permission to make digital or hard copies of all or part of this work for personal 
or classroom use is granted without fee provided that copies are not made or 
distributed for profit or commercial advantage and that copies bear this notice 
and the full citation on the first page. Copyrights for components of this work 
owned  by  others  than  ACM  must  be  honored.  Abstracting  with  credit  is 
permitted. To copy otherwise, or republish, to post on servers or to redistribute to 
lists,  requires  prior  specific  permission  and/or  a  fee.  Request  permissions  from 
permissions@acm.org. 
CCS’13, November 4–8, 2013, Berlin, Germany. 
Copyright © 2013 ACM  978-1-4503-2477-9/13/11…$15.00. 
http://dx.doi.org/10.1145/2508859.2516727 

systems 

iOS),  whose 

in  developing 

(e.g.,  Android, 

running on these devices. Such applications, commonly known as 
apps,  are  typically  used  to  deliver  web  services  (data  storage, 
social  networking,  web  mails,  etc.)  through  their  compact  user 
interfaces and simple program logic, which are tailored to mobile 
platforms. Moreover, other than interactions with their own web 
services,  many  of  those  apps  are  also  built  to  work  with  other 
apps and services, leveraging the third-party’s resources to enrich 
their  functionalities.  This  is  a  trend  that  echoes  web-API 
integrations  extensively  utilized 
traditional, 
browser-based web applications. Examples include the apps that 
support social login and data sharing through the services offered 
by Facebook, Twitter, Google Plus, etc.  
Mobile  origin-crossing  hazard.  Those  mobile  apps  essentially 
play the same role as traditional web browsers at the client side. 
However,  different  from  conventional  web  applications,  which 
enjoy browse-level protection for their sensitive data and critical 
resources  (e.g.,  cookies),  apps  are  hosted  directly  on  mobile 
operating 
security 
mechanisms  (such  as  Android’s  permission and sandbox model) 
are  mainly  designed  to  safeguard  those  devices’  local  resources 
(GPS  locations,  phone  contacts,  etc.).  This  naturally  calls  into 
question  whether  the  apps’  web  resources  are  also  sufficiently 
protected  under  those  OSes.  More  specifically,  web  browsers 
enforce  the  same  origin  policy  (SOP),  which  prevents  the 
dynamic web content (e.g., scripts) of one domain from directly 
accessing  the  resources  from  a  different  domain.  When  the 
domain boundary has to be crossed, a designated channel needs to 
go  through  to  ensure  proper  mediation.  An  example  is  the 
postMessage channel [5], which a domain uses to send a message 
to another domain by explicitly specifying the recipient’s origin,  
and the browser mediates to ensure that only the content from that 
origin gets the message and the recipient is also well informed of 
the sender’s origin. Such origin-based protection has become a de 
facto security standard for a modern browser. However, it is not 
present  on  any  channels  provided  by  mobile  OSes  for  apps  to 
communicate  with  each  other  or  the  web.  As  a  result,  the  web 
content or apps from an untrusted domain may gain unauthorized 
access to the web resources of other apps/websites through those 
channels, causing serious security consequences.  
As  an  example,  consider  the  scheme  mechanism  [25][26] 
supported  by  Android  and  iOS,  through  which  an  app  on 
phone/tablet can be invoked by a URL once it registers the URL’s 
scheme (e.g., the “youtube” part of “youtube://watch?token=xxx”, 
with “xxx” as the input parameter for the app). What an adversary 
can  do  is  to  post  on  Facebook  a  link  that  points  to  a  malicious 
script  hosted  on  his  website.  Once  this  link  is  clicked  by  the 
victim  through  the  Facebook  app  on  her  iOS  device,  the  script 
starts  to  run  within  the  app’s  WebView  instance,  which  is  then 
redirected  to  a  dynamically  generated  URL  with  the  scheme  of 
another app that the adversary wants to run on the victim’s device 
and the parameters he chooses. As a result, the target app will be 

635invoked  to  blindly  act  on  the  adversary’s  command,  such  as 
logging  into  his  Dropbox  account  to  record  the  victim’s  inputs 
(Section  3.3.2),  since  the  app  is  given  no  clue  the  origin  (the 
adversary’s  site)  of  the  request.  In  another  case,  the  Android 
mobile browser processing a URL with the “fbconnect://” scheme 
from the Facebook server will deliver the secret token on the URL 
to an app from an arbitrary origin, as long as it claims to be able 
to handle that scheme (Section 3.3.1).   
Such  unauthorized  origin  crossing  is  related  to  the  confused 
deputy  problem  [24]  on  mobile  devices.  Prior  research  on  this 
subject, however, focuses on permission redelegation [10], which 
happens when an app with a permission requires sensitive system 
resources (e.g., a phone’s GPS location) on behalf of another app 
without that permission. The interactions between the two apps go 
through an Inter-Process Call (IPC) that delivers a message called 
intent from one app to invoke the other app’s Activity for services 
such as getting GPS coordinates. This intent channel can also be 
used  to  cross  origins:  for  example,  it  allows  an  app  from  one 
origin  to  send  intents  to  another  app  when  the  latter’s  related 
Activity is accidentally made public, a mobile cross-site request 
forgery  (CSRF)  attack  [27].  However,  given  that  those  prior 
studies  primarily  aim  at  protecting  mobile  devices’ 
local 
resources,  the  general  problem  has  not  been  dug  deeper:  for 
example, it is not clear whether an app’s private Activity can still 
be invoked by the intent message from an unauthorized origin, not 
to mention the security implications of other channels (such as the 
URL scheme) that can also be used for crossing domains.  
Our findings. To better understand this critical security issue, we 
conducted  the  first  systematic  study  on  unauthorized  origin-
crossing  over  mobile  OSes,  including  Android  and  iOS.    In  our 
study, we investigated all known channels that allow apps to cross 
domains,  such  as  intent,  scheme  and  utility  classes  for  web 
communications,  by  dissecting  popular  apps  like  Facebook, 
Dropbox, Google Plus, Yelp, and their SDKs, to understand how 
they utilize these channels to serve other apps on different mobile 
OSes. Our study found 5 generic cross-origin weaknesses in those 
high-profile  apps  and  SDKs,  which  can  be  exploited  through 
CSRF, login CSRF and cross-site scripting (XSS). Many of those 
problems affect multiple apps and web services. They are unique 
to  the  communication  channels  on  mobile  OSes,  which  are 
fundamentally different from those within the browsers. The root 
cause  of  the  vulnerabilities  is  the  absence  of  origin-based 
protection. More specifically, due to missing origin information, 
an app or a mobile web service is often left with little clue about 
the  true  origin  of  an  incoming  message,  nor  does  it  have  any 
control on where its outgoing message will be delivered to. 
The  consequences  of  these  cross-origin  attacks  are  dire.  They 
allow  a  malicious  app  to  steal  the  mobile  device  owner’s 
Facebook,  Dropbox  authentication  credentials,  or  even  directly 
perform  arbitrary  operations  on  her  Dropbox  account  (sharing, 
deleting, etc.) on Android.  On iOS, a remote adversary without 
direct access to any app on the victim’s device can stealthily log 
the phone into the adversary’s Dropbox account through Google 
Plus, Facebook apps. As a result, the victim’s text input on iPhone 
and iPad, her contacts and other confidential information are all 
exposed  to  the  adversary  once  she  uses  popular  editing  and 
backup apps (e.g., PlainText, TopNotes, Nocs, Contacts Backup, 
etc.)  that  integrate  Dropbox  iOS  SDK.  We  reported  those 
problems to related parties, who all acknowledged the importance 
of  our  discoveries.  We  received  over  $7000  bounty  for  these 
findings,  most  of  which  were  donated  to  charity.  The  details  of 

such  recognition  together  with  demos  of  our  attacks  are  posted 
online [31].   
Origin-based  defense.  Without  any  OS-level  support,  not  only 
does  app  development  become  highly  error-prone,  but  software 
manufacturers can also have hard time fixing the problems after 
they are discovered.  As examples, both Dropbox and Facebook 
need  to  spend  a  significant  amount  of  effort  to  fix  the  security 
problems  we  discovered,  which  involves  changing  software 
architecture  (Section  3.2.1)  or  deprecating  some  core  features 
within  their  apps  and  SDKs  (Section  3.3.1).  To  address  these 
issues and facilitate convenient development of securer apps, we 
present  in  this  paper  the  design  of  the  first  mobile  origin-based 
defense mechanism, called Morbs. Our approach mediates all the 
cross-origin  channels  on  Android  and  iOS,  including  intent, 
scheme  and  the  utility  classes  for  web  communications,  and 
enables  a  developer  to  specify  to  the  OS  authorized  origins  her 
app/website can receive requests from and/or send responses to. 
We implemented Morbs on Android in a way that fully preserves 
its  compatibility  with  existing  apps.  Moreover,  we  show  that 
through  our  mechanism,  the  developers  can  easily  gain  controls 
on  all  cross-origin  events,  avoiding  the  ordeal  experienced  by 
Facebook, Dropbox, and other companies. Our evaluation on the 
implementation  also  shows  that  it  is  both  effective,  stopping  all 
the  exploits  we  found,  and  efficient,  incurring  only  a  negligible 
impact on performance (< 1% overhead).  
The source code of Morbs is publicly available on GitHub [40].  
Contributions.  We summarize the paper’s contributions here:  
  New  problem.  Up  to  our  knowledge,  the  research  reported 
here  is  the  first  attempt  to  systematically  understand 
unauthorized origin crossing on mobile OSes.  The discovery 
made  by  our  study  brings  to  light  the  presence  of  such 
vulnerabilities in high-profile apps and more importantly, the 
seriousness and pervasiveness of the problem.  

  New techniques.  We developed new origin-based protection 
for existing mobile OSes, which works with apps/websites to 
oversee the communication channels on these systems. 
Implementation and evaluation.  We implemented our design 
on  Android,  and  evaluated  its  effectiveness,  efficiency, 
compatibility, and usability to the app developer.  

 

Roadmap.    The  rest  of  the  paper  is  organized  as  follows:    
Section  2  describes  the  mobile  channels  used  for  apps  to 
communicate with each other or the web; Section 3 elaborates our 
study on mobile cross-origin problems and our findings; Section 4 
presents  our  defense  mechanism  Morbs,  Section  5  reports  the 
evaluation of our techniques; Section 6 compares our work with 
other  related  research;  Section  7  concludes  the  paper  and 
discusses the future research.  
2.  MOBILE CHANNELS 
Today’s  mobile  OSes  (including  Android  and  iOS)  provide 
various channels for apps to communicate with each other or the 
web.  Those  channels  include  intent,  scheme,  and  web-accessing 
utility classes (which we elaborate later in the section). As shown 
in  ,  an  app  communicates  with  other  apps  through  the  intent  or 
scheme channel. It can also invoke the browser to load a webpage 
using an intent and be triggered by the web content rendered in 
the  browser  through  a  URL  scheme.  Moreover,  the  app  can 
simply  acquire  and  display  any  web  content  through  the 
WebView class, which embeds a browser-like widget within the 
app, and directly talks to a web server using the methods provided 

636by the HttpClient classes. Note that the intent channel is Android-
specific,  while  others  are  also  available  on  iOS.  Unlike  the 
domain-crossing  mechanisms  within 
(e.g., 
postMessage),  these  mobile  channels  are  not  under  the  origin-
based  protection:  messages  exchanged  do  not  carry  any  origin 
information 
inspect,  and  are 
completely unmediated with regard to where they come from.  

the  sender/receiver  can 

browser 

that 

a 

Mobile Device

App

callbacks

HttpClient classes

WebView class

mobile 
browser

intents
schemes

App

App
Apps

Web

 

Figure 1 Mobile communication channels 

Here we elaborate how those channels work: 

 

Intent.  An intent is an inter-process message delivered 
through  an  IPC.    It  is  a  channel  only  supported  by  Android. 
Through  intent  messaging,  one  app  on  Android  can  activate  the 
background  Services,  Activities  (application  components  with 
user interfaces) or Broadcast-Receivers of another app, as well as 
the Activities/Services of its own. Intent invocation is conducted 
through  APIs  such  as  startActivity,  startActivityForResult,  and 
startService. An app developer can specify a set of intents the app 
can  receive  from  other  apps  in  its  manifest  file.  However,  the 
intent channel never labels the origin of each message (i.e., who 
created it). This causes the problem we elaborate in Section 3.2.1.  
  URL scheme.  As discussed before, scheme is supported 
by both Android and iOS, which allows an app or website to use a 
URL  to  invoke  another  app  (on  iOS)  or  its  Activity/Service 
components (on Android) that claim the scheme of that URL. For 
example, the URL “youtube://watch?token=xxx” can be  used to 
start the YouTube app to play the video “xxx”. When such a URL 
is loaded in the mobile browser or a WebView instance, the OS 
will launch the target app with this URL as input. In addition, an 
app  can  also  invoke  other  apps  through  the  schemes  they 
registered.  On  Android,  scheme  invocation  is  implemented 
through the intent channel: a scheme URL is wrapped in an intent 
instance, and invoked by an app through the same set of APIs that 
also serve intent messages, such as startActivity.  On iOS, this is 
done through openURL API. Again, the OSes do not mediate the 
scheme-based invocations using origins.  

  Web-accessing utility classes. Mobile platforms provide 
several utility classes for apps to communicate with the web. We 
call  them  web-accessing  utility  classes.  For  example,  both 
Android and iOS support the WebView class (called UIWebView 
on  iOS),  which  an  app  can  embed  for  displaying  webpages.  An 
app  can  interact  with  its  WebView  instance  through  a  set  of 

method  calls  or  callbacks.  For  example,  it  can  call  loadURL  on 
Android (loadRequest on iOS) with a URL to load a page into the 
instance; it can also register events, like URL loading, to inspect 
every  URL  its  WebView  instance  processes  through  a  callback 
shouldStartLoadWithRequest (iOS) or shouldOverrideUrlLoading 
(Android).  In  addition,  the  mobile  platforms  provide  utility 
classes for an app to directly talk to a web server without loading 
its  web  content.  HttpClient  [36]  or  HttpURLConnection  [37] 
(Android)  and  NSURLConnection  [38]  [39]  (iOS)  are  such 
examples.  We  call  those  classes  (for  direct  communication  with 
web servers) HttpClient classes. Origin-based protection is not in 
the picture here: e.g., a WebView/HttpClient instance never labels 
which app is the origin of an HTTP request. 
3.  ATTACKS 
In  this  section,  we  elaborate  our  study  on  unauthorized  origin 
crossing on mobile OSes. What we want to understand here are 
whether the ways real-world apps utilize those channels for cross-
origin  communications  indeed  expose  them  to  attackers,  and 
whether  those  apps have proper means to mitigate such a threat 
and  safeguard  their  operations  over  those  channels.  For  this 
purpose,  we  systematically  analyzed  high-profile  apps  on  both 
Android  and  iOS,  including  the  official  apps  of  Facebook  and 
Dropbox and their SDKs, and the official Google Plus and Yelp 
app.  Note  that  these  SDKs  are  very  popular.  They  have  been 
integrated into many real-world apps. Problems discovered there 
may  have  a  broad  impact.  In  our  research,  we  looked  into  how 
those  apps  use  the  aforementioned  cross-origin  channels  to 
interact  with  other  apps,  or  the  web.  The  study  reveals  the 
pervasive  presence  of 
cross-origin 
vulnerabilities, allowing an unauthorized party to activate an app 
remotely  with  arbitrary 
internal 
Activities, steal user’s authentication credentials and even directly 
manipulate its operations.  
Such discoveries were made through an in-depth analysis on the 
code and operations of those apps. Specifically, for Android apps, 
we  decompiled  the  binary  code  of  their  latest  versions  using 
apktool  [33]  and  AndroChef  Java  Decompiler  [34]  in  order  to 
analyze their program logic related to the mobile channels. When 
it comes to iOS apps, decompiling their executables is often hard. 
Therefore,  we  resorted  to  a  black-box  traffic  analysis  to 
understand those apps’ interactions with other parties (apps, web 
services, etc.). We also studied the SDKs provided by Facebook 
and  Dropbox,  whose  source  code  is  publically  available.  In  the 
rest  of  the  section,  we  report  our  findings.  The  demos  of  our 
exploits  on  those  apps  and  other  supplementary  materials  are 
posted on the web [31]. 

input  parameters,  call 

subtle  yet 

serious 

its 

3.1  Adversary Model 
Our  adversary  model  describes  practical  threats  to  different 
mobile platforms. On Android, we consider an adversary who can 
trick  a  user  into  installing  a  malicious  app  on  her  device.  That 
app,  however,  may  not  have  any  permission  considered  to  be 
dangerous by Android. Also, threats to Android can come directly 
from the web, when the victim uses her mobile app or browser to 
view malicious web content posted by the adversary on a website. 
On  iOS,  we  only  consider  this  remote  threat  (from  a  malicious 
website),  not  the  malicious  app,  given  the  fact  that  Apple’s 
vetting  process  on  iOS  apps  is  more  restrictive  than  that  of 
Android apps, and few malicious apps have been reported so far.  
Note  that  we  treated  Android  and  iOS  differently  to  respect  the 
realistic  threats  those  systems  face:  we  could  have  found  more 

637login  Activity  before  proceeding  with 

issues  had  we  assumed  the  presence  of  malicious  apps  on  iOS. 
Finally, we do not consider an adversary with OS-level controls.  
3.2  Exploiting the Intent Channel 
The  security  implication  of  the  intent  channel  on  Android  has 
been  studied  in  prior  research  [10][27].  All  existing  work, 
however, focuses on how such a channel can be leveraged by a 
malicious  app  to  invoke  a  legitimate  app’s  Activities  that  are 
accidentally made public by the app’s developer.  In our research, 
we  found  that  even  the  private  Activities  not  exposed  to  the 
public, which is meant to be called only by the app itself, can be 
triggered by an app from an unauthorized origin.  This problem 
has  a  serious  consequence,  letting  the  malicious  app  gain  great 
control  of  the  victim  app.  We  discovered  this  vulnerability  on 
both  the  Facebook  app  and  the  Dropbox  app.    Here  we  use  the 
Dropbox app as an example to explain where the problem is. 
3.2.1  Next Intent  (Android) 
An  Android  app  can  have  two  types  of  Activities,  private  or 
public. By default, an Activity is private, meaning that only the 
code inside the app can invoke it. When the app developer sets the 
“exported” property of the Activity to true, or she declares at least 
one intent for the Activity in the manifest of the app, the Activity 
becomes public, in which case other apps can invoke the Activity 
with an intent instance as an argument. 
Our analysis on the Dropbox app reveals that the app exposes a 
few  Activities,  such  as  login,  which  is  meant  to  be  public.    An 
interesting observation is that when any of its public Activities are 
invoked  by  an  intent  instance,  the  Activity  first  needs  to  check 
whether the user is in a logged-in status. If not, it redirects him to 
the 
task. 
Specifically, the Activity creates a new intent instance, in which 
the current intent, the one it receives from another app, is saved 
under 
the  key  “com.dropbox.activity.extra.NEXT_INTENT” 
(called “NEXT_INTENT” here). The new intent instance is then 
issued by the app itself to invoke LoginOrNewAcctActivity (the 
login  Activity).  Once  the  user  completes  her  login,  the  login 
Activity 
from 
“NEXT_INTENT”,  and  uses  it  to  invoke  the  unfinished  public 
Activity to fulfill its task. 
The cross-origin exploit. It turns out that this next-intent feature 
can be exploited by a malicious app to cross origins and invoke 
the  Dropbox  app’s  private  Activity.  Since  the  login  Activity  is 
public, a malicious app can trigger it with an intent instance, in 
which  the  attacker  puts  another  intent  instance  under  the 
“NEXT_INTENT”  key.  The  second  instance  points  to  a  private 
Activity of the Dropbox app.  This login intent will not be noticed 
by the user if she is already in the logged-in status, and cause little 
suspicion if she is not, simply because it is the authentic Dropbox 
app that asks the user to log in. Either way, once the login is done, 
LoginOrNewAcctActivity  retrieves  the  intent  content  under  the 
“NEXT_INTENT”  key  and  use  it  to  call  the  startActivity  API. 
Since startActivity is now invoked by the Dropbox app itself, all 
of  its  Activities,  including  those  private  ones,  can  be  executed, 
even  though  the  next-intent  actually  comes  from  a  different 
origin, the malicious app. The root cause of this problem is that 
the  startActivity  API  never  checks  (and  also  has  no  means  to 
check) the provenance of the intent under the “NEXT_INTENT” 
key, due to the lack of origin-based protection on the mobile OS. 
In the absence of the origin information (here, the app creating the 
intent),  even  an  app’s  private  Activity  can  be  exposed  to 
unauthorized parties.  

its  own 

retrieves 

instance 

original 

intent 

the 

to 

the 

requests 

the  user 

the  origin 

injected  can  make  arbitrary  AJAX 

to  run  LoggedOutWebViewActivity  with 

The problem goes beyond a single app. In the Facebook app, we 
discovered  the  same  problem  in  a  public  Activity  called 
UriAuthHandler. The Facebook app also checks the login status, 
and  directs 
login  Activity,  and  uses 
“CALLING_INTENT”  (equivalent  to  “NEXT_INTENT”)  as  a 
key  to  store  the  current  intent  instance.  This  channel  is  equally 
vulnerable  and  can  be  abused  in  the  same way, as found in our 
study.  We  suspect  that  other  apps  with  this  type  of  next-intent 
feature are also subject to the same exploit. 
Attacks  and  consequences.  Once 
is  crossed 
illegitimately,  the  door  is  open  to  all  kinds  of  abuses.  In  our 
research, we implemented two attacks (one against the Dropbox 
app,  another  one  against  the  Facebook  app)  to  demonstrate  the 
serious security consequences of the problem.  
Our  attack  on  the  Facebook  app  leverages  a  private  Activity 
LoggedOutWebViewActivity.  The  Activity  takes  a  URL  as  an 
input parameter and loads the content pointed by the URL into a 
WebView instance. What can happen here is that a malicious app 
running on the same device can drop a Javascript file onto its SD 
card  (Secure  Digital  memory  card)  and  exploit  the  next-intent 
feature 
the  URL 
pointing to that Javascript file. Since the SD card is viewed as a 
local storage by Android, the script is allowed to access contents 
from  all  Internet  domains  [32].  Specifically  in  our  attack,  the 
script 
to 
facebook.com and read the contents of the responses. Given that 
all  such  requests  carry  the  user’s  Facebook  cookie,  this  cross-
origin  scripting  ends  up  allowing  the  adversary  to  perform 
arbitrary operations on the user’s account, and obtain all private 
data.  
For 
the  Dropbox  app,  we  exploited  a  private  Activity 
VideoPlayerActivity,  which  has  an  input  parameter  “EXTRA_ 
METADATA_URL”  that  specifies  a  URL  from  which  to  fetch 
the  metadata  for  a  video  file.  In  a  normal  situation,  this  URL 
points  to  a  file  kept  by  dropbox.com.  However,  our  next-intent 
exploit enables a malicious app to set the URL to arbitrary web 
domain,  such  as  “http://attacker.com”.  When  the  Dropbox  app 
makes a request with that URL, it always assumes the recipient to 
be  dropbox.com  and  attaches  to  the  request  an  authentication 
header,  as  opposed  to  applying  the  conventional  origin-based 
cookie  policy.  Since  right  now,  EXTRA_METADATA_URL 
points to “http://attacker.com”, the adversary gets the header and 
can use it to gain a full access to the user’s Dropbox account.  
Vendor  responses.  Fixing  this  problem  turns  out  to  be  much 
more complicated than it appears to be. Specifically, the Dropbox 
security  team  told  us  they  were  “working  on  changing  the 
architecture in our Android app to make that API secure”, but the 
next-intent  feature  is  “unfortunately  also  very  useful  for  us”. 
Facebook also said that this problem “will take some time to fix”. 
As an acknowledgement to the importance of our work, Facebook 
awarded us $5000 bounty for finding this vulnerability, which we 
donated to charity. Dropbox also awarded us 100GB free storage 
for  each  author,  and  included  our names on their special thanks 
page.    The  details  of  those  software  vendors’  responses  can  be 
found  here  [31].  From  our  communications  with  the  vendors,  it 
can  be  seen  that  addressing  this  next-intent  problem  from  the 
developer side alone can be hard. In Section 4, we show how a 
well-thought-out  OS-level  support  can  make  this  type  of  flaws 
more convenient to fix.  

6383.3  Abusing the Scheme Channel 
As  discussed  in  Section  2,  scheme  is  an  important  cross-origin 
channel  supported  by  both  Android  and  iOS.  Through  this 
channel, an app on those OSes can be invoked by a URL (with the 
scheme the app claims) from another app or from a webpage in a 
WebView  instance  or  a  browser  (see    in  Section  2).    In  our 
research,  we  found  that  this  channel  can  be  easily  abused  for 
unauthorized origin crossing, enabling a malicious app to acquire 
a  user’s  authentication  token  with  Facebook  or  perform  a  login 
CSRF on iOS, as described below.  
3.3.1  Fbconnect  (Android) 
Facebook provides a Dialog mechanism [35] through its apps and 
SDKs  for  both  Android  and  iOS.  Using  the  mechanism,  an  app 
can  send  through  the  Facebook  official  app  a  Dialog  request  to 
query  the  Facebook  server  for  sensitive  user  data  (e.g.,  access 
token)  or  operate  on  the  user’s  account  (e.g.,  sharing  a  post). 
Among  all  the  arguments  carried  by  the  Dialog  request  are 
client_id,  the  ID  assigned  to  the  sender  app  by  Facebook,  and 
redirect_uri,  which  is  set  to  “fbconnect://success”.    In  the  case 
that  the  user’s  access  token  is  requested,  the  Facebook  server 
displays a dialog within Facebook app’s WebView instance to ask 
for the user’s consent, and upon receiving it, the server redirects 
the  WebView  instance  to  “fbconnect://success#...”,  where  the 
secret  token  is  attached  to  the  “…”  part  of  the  message.  This 
token  is  then  extracted  by  the  Facebook  app,  which  later 
dispatches  it  to  the  target  app  (i.e.,  the  sender  of  the  Dialog 
request) associated with the client_id.   
The URL “fbconnect://success#...” is just used for delivering data 
from the Facebook server to its official app. However, if this URL 
is  loaded  in  the  mobile  browser,  a  serious  attack  can  happen. 
More specifically, a malicious app on the device first registers this 
fbconnect://  scheme,  and  then  invokes  the  browser  to  load  a 
Dialog URL, in an attempt to request the sensitive data of another 
app (e.g., the TexasHoldem app) from the Facebook server. This 
can  be  easily  done  by  setting  client_id  in  the  URL  to  that  of 
TexasHoldem  because  an  app’s  client_id  is  public.  Also,  within 
the browser, the dialog may not even show up to alert the user, if 
it  is  already  in  the  logged-in  status.  As  a  result,  Facebook  will 
redirect  the  browser  to  “fbconnect://success#...”.    Unlike  the 
Facebook  app,  the  browser  treats  this  URL  as  a  scheme 
invocation,  and  therefore  will  trigger  the  scheme’s  handler  (i.e., 
the malicious app) with the URL as an argument. This exposes to 
the  malicious  app  the  victim’s  Facebook  secret  token  for  the 
TexasHoldem  app.  We  tested  the  attack  on  an  Android  device 
(Galaxy Tab 2) and confirmed that the malicious app can get the 
user’s access token, authorization code and other secrets. In this 
case, we can see that although the Facebook server is the sender 
of the scheme message, it cannot control which app to receive the 
message through the mobile browser. This is different from what 
happens  within  a  web  browser:  for  example,  if  a  script  from 
a.com sends a message to b.com through the postMessage API, it 
can specify the recipient domain and the browser then guarantees 
that  only  b.com  gets  the  message.  On  today’s  mobile  OS, 
however, there is no way that the Facebook server can specify the 
authorized  receiver  of  its  scheme  URL,  not  to  mention  any 
mechanism to enforce this security policy.   
Also  note  that  the  fbconnect  problem  here  is  present  on  both 
Android and iOS.  However, given that iOS malware is rare, the 
risk it poses is mainly to Android (see our adversary model).  
Vendor response. Without the OS support, this problem turns out 
to be even harder to fix than the next-intent issue. We reported it 

to  Facebook  on  Sept.  11,  2012.  On  Jan.  22,  2013,  Facebook 
security team told us that they took steps to “ensure that popular 
app stores block apps that attempt to register this URI schema”. 
Moreover, they were “crafting a formal deprecation plan for the 
fbconnect  schema”,  but  this  plan  needs  a  “several  month 
deprecation period” because “all of our embedded SDKs currently 
depend  upon  this  functionality”.  On  March  20,  2013,  Facebook 
informed  us  that  they  “crafted  a  plan  for  the  deprecation  of  the 
fbconnect schema in the next major release”. They expect to “see 
this  disappear  entirely  as  users  continue  to  upgrade”.  They 
awarded us a bounty of $1500 for this finding, which we donated 
to charity.  
3.3.2  Invoking Apps from the Web (Android and iOS) 
In this section, we elaborate a new security threat that comes from 
a malicious website the user visits with a mobile device. The root 
cause of the problem has been confirmed to exist on both Android 
and iOS. For the simplicity of presentation, here we just use iOS 
as an example to explain the problem.  
Mobile apps typically use their WebView instances to render web 
content.  Such  content  could  come  from  less  trustworthy  web 
sources, such as public posts on Facebook and restaurant reviews 
from the strangers on Yelp. In our research, we found that during 
such rendering of web content, whenever the WebView instance 
of  an  app  is  directed  by  the  content  to  a  URL  with  a  scheme 
registered  by  another  app  on  the  same  device,  the  latter  will  be 
automatically invoked, without being noticed by the user, and act 
on the parameters given by the URL. This is dangerous because 
the  app  receiving  the  scheme  message  which  carries  the  URL 
cannot  distinguish  whether  this  message  comes  from  the  sender 
app  itself  or  from  the  web  content  within  the  app’s  WebView 
instance,  which  causes  the  confusion  about  the  message’s  true 
origin.  Here  we  use  two  examples  to  show  the consequences of 
this confusion. 
Login  CSRF  attacks  on  Dropbox  iOS  SDK.  We  studied  the 
latest  version  (v.1.3.3)  of  Dropbox  iOS  SDK,  which  enables  a 
3rd-party  app  on  iOS  to  link  to  the  device  owner’s  Dropbox 
account,  using  Dropbox  as  the  app’s  storage.  Here,  we  use 
PlainText  [41],  a  popular  text-editing  app,  as  an  example  to 
explain what can go wrong, though apps using Dropbox iOS SDK 
are  all  vulnerable.  Specifically,  after  the  mobile  user  authorizes 
this  account  linking,  the  Dropbox  app  delivers  to  the  PlainText 
app  a  scheme  URL,  which  is  in  the  following  format:  db-
<APP_ID>://1/connect?oauth_token&oauth_token_secret&uid. 
The URL includes 3 arguments, oauth_token, oauth_token_secret, 
and uid, which the PlainText app uses to communicate with the 
Dropbox  server  to  complete  the  account  linking.  However,  we 
found that the linking process can be exploited to launch a serious 
login  CSRF  attack,  without  any  malicious  app  running  on  the 
user’s device. Specifically, in our attack, the adversary uses the 3 
URL  arguments  collected  from  his  own  device  to  build  a  URL: 
db-<APP_ID>://1/connect?oauth_token’&oauth_token_ secret’ & 
uid’, where oauth_token’, oauth_token _secret’, and uid’ are the 
adversary’s  Dropbox  credentials,  and  APP_ID  identifies  the 
PlainText  app.  The  attacker  shares  a  malicious  web  URL  (e.g. 
pointing  to  attacker.com)  on  his  Google  Plus  status  updates  or 
comments. Once the victim user clicks it within the Google Plus 
app on her device, attacker.com is loaded in the app’s WebView 
instance,  and  redirects  the  WebView  to  the  scheme  URL.  As  a 
result, the PlainText app is invoked with the URL as input. The 
app  treats  the  URL  as  part  of  the  scheme  message  from  the 
Google Plus app, without knowing that it actually comes from the 

639web  content  of  attacker.com  rendered  in  the  Google  Plus  app’s 
WebView.  It  is  then  unknowingly  linked  to  the  attacker's 
Dropbox  account.  When  this  happens,  the  app  automatically 
sends the user's text input to the attacker's account. A demo of this 
attack is posted online [31]. We also checked a few other popular 
iPad  apps  using  Dropbox  SDK,  including  TopNotes,  Nocs,  and 
Contacts Backup to Dropbox. They are all found vulnerable. 
Bypassing  Facebook’s  app  authentication  mechanism.  Many 
apps  using  Facebook  iOS  SDK,  such  as  Yelp  and  TripAdvisor, 
may  also  render  untrusted  web  contents  within  their  WebView 
instances.  Below we show that an attacker who posts a malicious 
link on Yelp can bypass an important mechanism Facebook uses 
to  authenticate  3rd-party  iOS  apps.  Specifically,  when  app  A 
invokes  the  Facebook  app  through  schemes  such  as  “fbauth://”, 
the  Facebook  app  sends  the  app  ID  specified  by  app  A  and  its 
bundle  ID  retrieved  from  the  OS  to  the  Facebook  server  for 
authentication.  This  prevents  app  A  from  impersonating  another 
app to communicate with the Facebook server because it cannot 
manipulate the bundle ID. However, this protection does not work 
when a malicious page is loaded to the WebView instance of the 
Yelp app because the Facebook app cannot distinguish whether an 
incoming scheme message is from the Yelp app or a webpage in 
its  WebView  (the  bundle  ID  from  the  OS  always  points  to  the 
Yelp app). Therefore, whoever posts a comment on Yelp acquires 
the same privilege as Yelp has on the victim’s Facebook account. 
Vendor  response.  We  reported  these  problems  to  Dropbox, 
Google, and Facebook. For the first problem (login CSRF through 
Dropbox  SDK),  Dropbox  started  its  investigation  immediately 
after receiving our report. They have implemented a fix that needs 
to  change  both  the  SDKs  and  the  Dropbox  official  apps  on  all 
platforms  (including  Android  and  iOS).  Facebook  mitigated  the 
threat by deploying a whitelist inside the WebView instance of its 
official  app,  which  only  allows  http,  https,  and  ftp  schemes. 
Google has not taken any actions so far [31]. Facebook awarded 
us  $1000  for  this  finding.  We  also  reported  to  Facebook  the 
second case (bypassing its app authentication mechanism on iOS), 
which is still under investigation. 
3.4  Attacks on Web-Accessing Utility Classes 
As  shown  in  ,  besides  intent  and  scheme,  origins  can  also  be 
crossed on a mobile OS when an app directly calls the methods of 
the WebView/HttpClient classes or registers their callback events. 
Here we show how this channel can be abused.  
3.4.1  Exploiting Callbacks (iOS) 
On  iOS,  we  studied  a  WebView  callback  method  the  Facebook 
app  registers,  shouldStartLoadWithRequest,  which  is  triggered 
each time the app’s WebView instance is navigated to a link.  If 
this  link  is  in  the  form  “fbrpc://appID=xyz&foo=123”,    the 
callback  method  (provided  by  Facebook)  creates  a  new  URL 
“fbxyz://foo=123” to invoke an app with the appID “xyz” and set 
its input argument to “123”. Note that this operation is different 
from the scheme-based invocation (from a web domain) described 
in Section 3.3.2, as in that case, a website directly runs a URL to 
invoke  the  target  app  on  the  mobile  device  (the  sender  of  the 
scheme  message  is  the  website),  while  here  such  a  URL  is 
actually created by the callback method, which is implemented by 
the Facebook app (the sender is the Facebook app). 
This mechanism can be exploited when a malicious link such as 
attacker.com  is  clicked  by  the  user  through  her  Facebook  app. 
When this happens, the malicious content loaded to the WebView 

instance  redirects  to  the  fbrpc  URL.  Then  the  callback  of  the 
Facebook app generates a new scheme URL to launch any app the 
adversary wants to run on the victim’s device with any argument 
value he sets. For example, we found that in this way, a popular 
app Pinterest can be activated by the adversary to sign onto the 
adversary’s  account  on  the  victim’s  device,  so  as  to  dump  the 
user’s data with Pinterest into the adversary’s account.  
3.4.2  Exploiting Header-attachment  (Android) 
We  also  studied  the  HttpClient  class, which is used by Android 
apps  for  direct  HTTP  communications  with  web  servers. 
HttpClient allows a developer to specify the URL of a request and 
an  HTTP  header.  The  header  is  attached  to  the  request.  In  the 
absence of origin-based protection, any header, including the one 
used for authentication, can be attached to a request sent to any 
website.  A  prominent  example  is  the  attack  case  described  in 
Section 3.2.1: the adversary invokes the Dropbox app’s Activity 
VideoPlayerActivity, which utilizes an HttpClient instance to load 
metadata  from  a  URL  with  the  user’s  authentication  header 
attached. Since the URL is manipulated by the adversary to point 
to attacker.com, without origin checks, the authentication header 
goes to the adversary.  
Note that this header-attachment issue by itself is a security flaw, 
as  admitted  by  the  Dropbox  security  team  (“Attaching  the 
Authorization  header  to  non-Dropbox  URLs  was  definitely  a 
serious security bug” [31]).  Actually the attack on a phone user’s 
Dropbox account as described in Section 3.2.1 is built upon two 
security flaws, i.e., the next-intent and header-attachment issue. 
4.  ORIGIN-BASED DEFENSE 
As  described  in  the  prior  section,  unlike  web  browsers,  today’s 
mobile OSes (i.e., Android and iOS) do not offer origin protection 
to the channels used by apps to communicate with each other or 
the  web.  As  a  result,  cross-origin  interactions  on  those  systems 
can be easily abused to undermine the user’s security and privacy, 
which  even  happens  to  highly  popular  apps  built  by  security-
savvy  developers.  Moreover,  even  after  the  problems  were 
reported, the developers still had hard time in fixing them. This 
makes us believe that a generic solution to the problem should be 
built into mobile OSes, which have the observations of messages’ 
origins, and the means to mediate the communication over those 
channels.    In  this  section,  we  elaborate  the  first  design  for such 
protection,  called  Morbs  (mobile  origin  based  security),  and  its 
implementation on Android. We released the source code of the 
Android implementation on GitHub [40]. 
4.1  Design 
Overview. Morbs is generic to iOS and Android. It is designed to 
achieve  browser-like  origin  protection:  1)  it  exposes  to  the 
developers  the  true  origins  of  the  messages  their  apps/websites 
receive,  enabling  them  to  build  protections  based  on  such 
information; 2) it allows the developers to specify their intentions, 
in  the  form  of  whitelists  of  origins  their  apps/websites  can  get 
messages  from  and  send  messages  to,  and  enforces  policies 
transparently within the OS.  
More  specifically,  an  app  or  a  web  service  that  asks  for  origin 
protection first communicates its intended sender/recipient origins 
(the  whitelists)  to  the  OS.  These  policies  are  enforced  by  a 
reference  monitor  that  mediates  different  mobile  channels.    The 
reference monitor is triggered by the messages delivered through 
these  channels.  Once  invoked,  it  identifies  the  origins  of  the 
messages, which are either apps or web domains, and checks their 

640for 

for  example,  “app://com.facebook.katana” 

policy  compliance  against  the  whitelists.  Those  running  against 
the policies are blocked by Morbs.   
A  unique  feature  of  Morbs  is  its  capability  to  connect  web 
activities  (within  WebView  instances  or  the  mobile  browser)  to 
the events that happen within the OS.  For example, it exposes the 
true origin of a message to a recipient app when confusion arises 
on  whether  the  message  comes  from  another  app  or  the  web 
domain  visited  by  that  app’s  WebView  instance.  It  also  helps  a 
web  server  specify  to  a  mobile  device  a  designate  app  on  the 
device  that  can  receive  the  server’s  scheme  message.  This 
capability is crucial for defeating unauthorized origin crossing on 
mobile devices. Following we elaborate our design. 
Defining mobile origin. For web content, an origin is defined as 
a  combination  of  scheme,  host  name,  and  port  number  [4]. 
However, this definition is insufficient for the origin protection on 
mobile platforms: here we need to consider not only web origins 
but  also  app  identities  and  other  local  origins.    To  maintain  the 
consistency  with  the  web  origins,  we  adopted  a  URL-like 
representation  for  those  new  origins,  such  as  “app://<appID>”,  
where <appID> is an app’s package name (Android) or bundle ID 
(iOS), 
the 
Facebook  app  on  Android  and  “app://com.getdropbox.Dropbox” 
for  the  Dropbox  app  on  iOS.  Likewise,  messages  from  trusted 
sources  like  the  OS  are  given  a  local  origin  “local://”.  For  web 
domains, we adhere to the traditional origin definition [4]. 
Exposing origins. When a message is created by an app/website, 
Morbs  sets  the  origin  attribute  (added  by  our  approach)  of  the 
message to its creator (i.e., an app, a web domain, or local). This 
attribute always goes with the message within the OS, until it gets 
to  its  recipient  app/website,  where  we  remove  the  attribute.  To 
help developers build their own protection, our design exposes the 
origin of a message through existing and new APIs. For example, 
on iOS, we can enhance the API for retrieving the bundle ID of 
the sender of a scheme message by returning the true origin of the 
message,  which  could  be  the  domain  of  the  web  content  within 
that app’s WebView instance. In this way, Facebook will be able 
to  find  out  that  the  scheme  message  it  gets  from  the  Yelp  app 
actually comes from a webpage Yelp displays, not the app itself. 
Therefore, the exploit described in Section 3.3.2 will be defeated. 
Default  policy.  It  is  well  known  that  the  browser  by  default 
enforces the SOP to the web content it hosts, but the same policy 
cannot be applied to all the apps on mobile platforms as it may 
disrupt their legitimate cross-origin communications. Our strategy 
is  to  implement  the  SOP  only  on  the  totally  unexpected  and 
insecure  channel.  An  example  is  the  next-intent  communication 
described in Section 3.2.1, which is unexpected, since the private 
Activity of an app should only be invoked by the app’s own intent 
when calling the startActivity API. Therefore, in this scenario, the 
SOP is always enforced.  
Setting  policies.  Morbs  allows  a  policy  to  be  specified  on  a 
channel between an app and a web domain (web policy), as well 
as  between  two  apps  (app  policy).  An  app  policy  defines 
legitimate inter-app communication, which goes through intent or 
scheme.  A  web  policy  is  about  app-web  interactions,  through 
scheme or web-accessing utility classes.  An app or a website sets 
a policy on a specific channel to notify Morbs the list of senders 
authorized  to  send  messages  to  it,  and  the  list  of  recipients 
allowed to receive the messages it sends.  
Setting a policy can be done through a new API setOriginPolicy, 
which an app can directly call.  Here is its specification: 

void setOriginPolicy(type, senderOrRecipient, channelID, origins) 

 
Here,  type  identifies  the  type  of  the  channel  (intent,  scheme,  or 
utility  class),  senderOrRecipient  specifies  sender  or  recipient, 
channelID indicates the channel ID, and origins is the whitelist. 
Once invoked, setOriginPolicy first identifies a channel by type, 
and channelID, which is an OS-wide unique string. For example, 
the ID for the intent channel that triggers LoginActivity within the 
Facebook app is “com.facebook.katana.LoginActivity”, in which 
“com.facebook.katana” is the Facebook app’s package name. For 
a scheme, its channel ID is the corresponding scheme field on a 
URL.  For  web-accessing  utility  classes,  they  are  identified  by 
their class instances within an app. After a channel is identified, 
the API then extracts the whitelist that regulates the sender or the 
recipient  (specified  in  senderOrRecipient)  through  this  channel 
from the origins parameter. 
Although  setOriginPolicy  offers  a  generic  interface  for  policy 
specification, it cannot be directly invoked by a website to set its 
policies.  To  address  this  problem,  Morbs  provides  mechanisms 
for  indirectly  accessing  this  API,  including  a  JavaScript  API 
setMobileAllowedOrigins,  through  which  the  dynamic  content 
from  a  website  can  set  policies  within  the  mobile  browser  or  a 
WebView instance, and a header mobile-allowed-origins in HTTP 
responses that inform the browser or a WebView instance of the 
parties  on  the  device  allowed  to  get  the  message.  The  app 
developer can also leverage other indirect mechanisms to define 
her policies whenever she is building the app’s functionality over 
a mobile channel. Specifically, Morbs allows the developer to set 
her policies regarding a scheme/intent her app claims within the 
app’s manifest file (for Android) or .plist (for iOS), under a new 
property allowedOrigins.  In this way, she can turn on our origin 
protection for her app without changing its code. Other ways for 
policy setting include a new argument allowedOrigins for the API 
that  delivers 
scheme/intent  messages,  and  a  new  API 
setAllowedOrigins used to define policies for utility classes such 
as WebView and HttpClient. An advantage of using these indirect 
ways  is  that  they  only  require  one  argument  (i.e., origins)  from 
the developer because other arguments are set by default. 
Enforcing  policies.    Morbs  runs  a  reference  monitor  to  enforce 
security  policies  on  different  channels.  Whenever  a  message  is 
delivered  over  a  channel,  the  reference  monitor  is  triggered  to 
identify its origin and calls a function checkOriginPolicy to check 
its policy compliance. The function’s specification is as follows: 

channel 

(type 

and 

bool checkOriginPolicy(type, senderOrRecipient, channelID, from, to) 

channelID).  Note 

 
Intuitively,  the  function  searches  Morbs  policy  base  to  find  out 
whether  the  current  sender  (specified  in  the  from  argument)  is 
allowed  to  deliver  the  message  to  the  recipient  (to)  through  the 
specific 
that 
checkOriginPolicy needs to be called twice (one for checking the 
sender  origin  against  the  recipient’s  policy  and  the  other  for 
checking  the  recipient  origin  against  the  sender’s  policy).  The 
message is allowed to go through only if both checks succeed.  
Both  setOriginPolicy  and  checkOriginPolicy  operate  on  the 
Morbs policy base that keeps all policies. setOriginPolicy inserts 
a  policy  into  the  database  and  checkOriginPolicy  searches  the 
database 
for  an  applicable  policy,  checking  whether  a 
sender/recipient origin is on the whitelist included in the policy. 
The  performance  of  this  compliance  check  is  critical  because  it 
needs  to  be  invoked  for  every  message  going  through  these 

641channels.  To  make  it  efficient,  Morbs  leverages  the  hash-table 
search to quickly locate a target within the database. 
4.2  Implementation 
We implemented our design on Android (Figure 2).  At the center 
of  our 
the 
ReferenceMonitor class, in which the most important function is 
checkOriginPolicy. They were built into the Thread class of the 
Dalvik  Virtual  Machine.  The setOriginPolicy API is open to all 
apps, while ReferenceMonitor is kept for the OS, which is only 
accessible to the code running inside the Android OS kernel.  

setOriginPolicy  API  and 

system  are 

the 

intents/schemes

AppApp
Apps

calls/schemes

class Activity {

startActivityForResult()

}

Dalvik VM

C
h
e
c
k
p
o

 

l
i
c
y

class Thread {

class WebViewCore {

handleMessage()

}

S
e
t
 
p
o

l
i
c
y

S
e
t
 
p
o

l
i
c
y

C
h
e
c
k
p
o

 

l
i
c
y

setOriginPolicy() API
class ReferenceMonitor {

checkOriginPolicy ()

}

}

Mobile Device

callbacks

class CallbackProxy {
handleMessage()

}

Check policy

Web

 

Figure 2 The framework of Morbs on Android 

In  the  presence  of  the  centralized  policy  compliance  check 
(checkOriginPolicy),  the  task  of  ReferenceMonitor  (i.e.,  policy 
enforcement) becomes trivial:  all we need to do here is pulling 
invoking  checkOriginPolicy  and  raising  an 
the  arguments, 
exception  to  drop  a  message  when  the  check  fails.  In  our 
implementation,  the  ReferenceMonitor  class  is  used  in  the  OS 
components  related  to  these  channels  to  conduct  mediation.  
Specifically,  for  intent  and  scheme,  the  enforcement  code  was 
placed  within the API startActivityForResult, which needs to be 
called  by  startActivity,  when  a  message  delivered  through  those 
channels attempts to start an Activity. Note that we chose not to 
do the security checks within the IPC mechanism: Android does 
not  recommend  inspecting  intent  data  in  IPC  because  the  intent 
instances are serialized for high performance data transport [29]. 
For  mediating  web-app  communications,  we  changed 
the 
handleMessage method within both the WebViewCore class and 
the CallbackProxy class. The two methods are the focal point of 
mobile browsers and WebView instances: all method invocations 
and  callback  handling  from  apps  need  to  go  through  them.  In 
addition,  the  execute  method  of  HttpClient  class  was  used  to 
mediate apps’ direct communication with web servers. 
Challenge  I:  origin  identification.    Morbs  attaches  an  origin 
attribute  to  every  message  exchanged  through  the  mobile 
channels. On Android, both intent and scheme channels utilize the 
intent  messages  (Section  2).  The  constructor  for  generating  an 
intent instance is hooked in our implementation to label an intent 
message  with  its  app  origin.  Specifically,  we  added  an  origin 
property  to  the  intent  class.  When  the  constructor  is  creating  a 
new  intent  instance,  it  retrieves  the  package  name  of  the  app 
initiating  the  intent  and  fills  in  the  origin  property  with  the 
package name.  For example, when the initiator is the Facebook 
app,  the  origin  property  should  be  marked  as  “app://com. 
facebook.katana”, in which “com.facebook.katana” is the package 

there 

to  understand 

name  of  Facebook  app.  However,  this  origin  is  not  easy  to 
identify in practice, since there is no API to help us find out the 
initiator directly. A simple solution is to get the whole call stack 
from  the  OS  through  getStackTrace  API,  and  then  inspect  it  to 
find out the caller. This approach turns out to be very expensive: 
in our test, extracting the call stack takes 1.35 ms in average. Our 
solution is to add an origin property to each thread that hosts an 
app. When the thread is created, the app’s origin is attached to the 
property.  Once  an  intent is initiated, the constructor then copies 
the origin information from the thread to the intent instance.  
Challenge II: response inspection. To enable a web server to set 
its policies to a mobile device, Morbs needs to inspect the HTTP 
response to find the header mobile-allowed-origins. The response 
is processed by Android’s native C++ libraries. Morbs (written in 
Java)  cannot  directly  access  it.  In  our  implementation,  we 
managed to get access to the header using Java Native Interface 
(JNI)  [30].  JNI  provides  an  API  called onReceivedData through 
which  C++  code  can  send  messages  to  Java  code.  To  inform 
Morbs of the content of the header, we modified the C++ code to 
the  header  mobile-allowed-origins  within  HTTP 
identify 
responses,  and  then  call  onReceivedData  to  deliver  all  policies 
described 
to  WebViewCore,  where  Morbs  uses 
setOriginPolicy to complete this policy setting process. 
5.  EVALUATION 
We evaluated the general design of Morbs and its implementation 
on  Android 
its  effectiveness,  performance, 
compatibility and utility to the app developers. 
5.1  Effectiveness 
We  ran  our  implementation  against  the  aforementioned  cross-
origin  attacks  (Section  3).  Specifically,  our  experiment  was 
conducted  on  Android  4.3  with  Morbs  running  within  an 
emulator.  We installed both the vulnerable apps discovered in our 
research and the attacker apps. In the experiments, we first ran the 
attacker  apps,  and  then  checked  whether  the  exploits  were 
blocked  by  Morbs  or  not.  Note  that  in  some  situations,  we  also 
need the developers to explicitly specify their whitelists of origins 
within  their  apps,  in  addition  to  the  default  policies.  In  the 
absence of those apps’ source code, we had to directly enter those 
app-specific policies (whitelists) into the OS. 
Preventing the exploits on intent. As described in Section 3.2.1, 
a malicious app can use the next-intent trick to invoke any private 
Activities of the victim app (the Dropbox app and the Facebook 
app).  The  content  saved  under  the  NEXT-INTENT  key  is 
essentially  an  intent,  which  needs  to  be  first  created  by  the 
malicious app before it is embedded into another intent (the one 
delivered  to  Dropbox  app’s  login  Activity).  Under  Morbs,  the 
intent constructor sets the origins of both intents to the malicious 
app,  which  cannot  be  changed  by  the  app.  As  a  result,  when 
startActivity  is  called  to  start  the  target  private  Activity,  our 
reference  monitor  immediately  finds  that  the  origin  of  the  next-
intent  is  not  the  victim  app  itself,  and  stops  this  invocation 
according  to  the  default  policy  (the  SOP).  Our  tests  confirmed 
that the vulnerabilities in both Dropbox app and Facebook app are 
fixed in this manner, without changing the apps’ code. 
Defeating  the  attacks  on  scheme.  For  the  fbconnect  problem 
described in Section 3.3.1, what Facebook wants to do is to return 
the data (e.g., secret tokens) from its server  to the app associated 
with the client_id parameter within the Dialog request, not anyone 
that  claims 
is 

scheme.  This 

fbconnect:// 

intention 

the 

642into 

communicated by the Facebook server to the mobile OS through a 
list  of  legitimate  recipient  origins  specified  using  its  HTTP 
response  header  or  the  JavaScript  API  provided  by  Morbs. 
Specifically  in  our  experiment,  we  inserted  the  header  “mobile-
allowed-origins:  app://com.facebook.katana” 
the  HTTP 
response  from  the  Facebook  server,  indicating  that  only  the 
Facebook app can receive the data, and observed that the scheme 
invocation  was  stopped  when  the  app  that  registered  the 
fbconnect://  scheme  was  not  the  Facebook  app.  A  video  demo 
about this case can be found at [31]. 
When  it  comes  to  the  apps  using  Dropbox  iOS  SDK  (the  first 
vulnerability  described  in  Section  3.3.2),  it  is  clear  that  their 
schemes  associated  with  Dropbox  are  only  supposed  to  be 
invoked  by  the  Dropbox  app.    Using  Morbs,  the  Dropbox  SDK 
embedded in the apps specifies “app://com.getdropbox.Dropbox” 
(i.e.,  the  Dropbox  app)  as  the  only  legitimate  sender  origin  for 
these schemes. As a result, our reference monitor ensures that any 
invocation  of  the  schemes  comes  only  from  the  Dropbox  app, 
which  defeats  the  attacks  from  a  malicious  webpage  (through 
Facebook app or the Google Plus app). In the case of the attack 
using  the  Yelp  app  (second  vulnerability  described  in  Section 
3.3.2), the problem comes from that the recipient of the scheme 
message, the Facebook app, cannot tell whether the origin of the 
message is indeed Yelp or a malicious website visited by the Yelp 
app’s  WebView  instance.  In  the  presence  of  Morbs,  the  true 
origin of the message is revealed as the website’s domain, which 
enables  Facebook  to  thwart  the  attack.  Note  that  we  did  not 
actually run those fixes as the problems were found on iOS. 
Mediating issues in web-accessing utility classes. For the case 
of  the  WebView  callback  within  the  Facebook  app  (Section 
3.4.1),  this  callback  should  only  respond  to  the  event  (i.e., 
processing  the  fbrpc  URL)  when  this  URL  comes  from  the 
domains  under  Facebook’s  control.  Let’s  assume  Facebook  app 
specifies “https://*.facebook.com” as the whitelist associated with 
the callback class UIWebViewDelegate, an operation that can be 
easily  done  using  Morbs.  As  a  result,  the  event  initiated  from 
“attacker.com” is ignored by the WebView, without triggering the 
callback 
from 
facebook.com  continue  to  be  handled.  We  further  evaluated  our 
implementation  against  the  exploit  on  the  HttpClient  class 
(Section 3.4.2). This time, we set “https://*.dropbox.com” as the 
legitimate origin on the whitelist for the instance of the HttpClient 
class  used  in  the  Dropbox  app.  After  that,  we  found  that  even 
after  the  adversary  crossed  the  origins  through  the  next-intent 
channel,  he  still  cannot  steal  the  Dropbox  authentication  header 
by sending requests to a non-dropbox.com URL, because it was 
blocked by our reference monitor according to the whitelist. 
5.2  Performance 
We evaluated the performance of our implementation on a Nexus 
4 development phone. We compared the overhead of Morbs with 
the overall delay the user experiences in the absence of Morbs, to 
understand the performance impact that our approach can have on 
cross-origin communications. In the experiments, we call a Java 
API nanoTime to collect timing measurements at a precision of 1 
nanosecond (i.e., 10-9 s).  To measure the performance of a Morbs 
operation,  we  repeated  it  10  times  to  get  its  average  execution 
time.  The  operations  we  studied  include  setting  policies  and 
checking policy compliance.  Among them, the compliance check 
is the focus of our evaluations, as the policy setting is just a one-
time task. More specifically, we measured the delays for sending 
messages  through  intent,  scheme,  and  utility  classes  in  the 
absence of Morbs, and then compared them with the time spent on 

shouldStartLoadWithRequest,  while 

those 

a policy compliance check.  In all the cases, the impact of Morbs 
was found to be negligible (below 1%). 
Performance of Morbs operations. On the Android OS with our 
Morbs  implementation,  we  ran  a  test  app  to  invoke  the 
setOriginPolicy API, and measured the time for setting a policy. 
On average, this operation took 0.354 ms, which involves storing 
the content of the policy to a policy database maintained by the 
OS.  To check the compliance with the policies, Morbs needs to 
search the database to find out whether the origin of the current 
sender or recipient is whitelisted. As described in Section 4.1, we 
leverage the hash-table search to quickly locate the policies.  To 
understand  the  performance  of  this  operation,  we  utilized  a  test 
app to invoke another test app through an intent message, which 
triggered  the  checkOriginPolicy  function.  We  found  that  the 
whole compliance check process took 0.219 ms on average. Note 
that policy enforcement over other channels all utilizes the same 
ReferenceMonitor  class,  which  is  expected  to  bring  in  similar 
average delay. 
Impacts  on  mobile  communications.  As  described  above,  the 
performance impact of setting policies should be minimum, since 
it just incurs a one-time cost. Also for the policies declared within 
a manifest file, they are set when the app is installed, which does 
not  affect  its  operations.  Therefore,  our  focus  was  policy 
compliance check.  
In  the  study,  we  measured  the  overall  delays  for  sending  a 
message through intent, scheme, and web-accessing utility classes 
without the policy compliance check. Table 1 shows the average 
delays  for  such  communication,  and  their  comparison  with  the 
overhead for a compliance check (0.219 ms). This gives a pretty 
good  picture  about  the  impact  the  check  can  have  on  such 
channels.  Specifically,  for  the  intent  channel,  we  measured  the 
time  interval  between  the  invocation  of  startActivity  and  the 
execution  of  performCreate  (the  first  API  the  target  Activity 
needs  to  call).  After  repeating  the  operation  for  10  times,  we 
observed  an  average  delay  of  42.142  ms  when  the  sender  and 
recipient were the same app, and 46.267 ms when they were not 
(see Table 1).  On the other hand, the compliance check took only 
an average 0.219 ms.  Therefore, the impact of this mediation on 
the  intent  communication  was  around  0.5%.    For  the  scheme 
message  delivered  between  two  apps,  it  goes  through  the  same 
intent  mechanism.  The  mediation  impact  of  Morbs  on  this 
communication  was  found  to  be  0.3%  on  average.  We  further 
measured  the  time  a  webpage  takes  to  invoke  an  app  through 
scheme,  between  the  event  when  the  method  handleMessage  in 
WebViewCore class is triggered to process the scheme URL, and 
when the performCreate API for the target test app is called. We 
found that this whole process took 115.301 ms and the impact of 
the policy checking there was 0.2%. 
When  we  take  into  account  the  delays  incurred  by  web-related 
operations,  particularly  those  performed  by  the  methods  and 
callbacks of WebView and HttpClient class, the extra time spent 
on  the  policy  compliance  check  can  be  comfortably  ignored.  
Specifically,  we  measured  the  waiting  time  for  loading  a  URL 
(specifically,  google.com)  through  HttpClient,  and  WebView.  
For  HttpClient,  we  measured  the  time  interval  between  the 
creation  of  a  class  instance  and  the  point  when  the  instance 
receives the HTTP response, which took 225.035 ms on average. 
For WebView, we measured the interval between the start of page 
loading 
completion 
(onPageFinished 
took  692.955  ms.  By 

called) 
is  called),  which 

(onPageStarted 

and 

is 

its 

643comparison,  the  time  Morbs  spends  on  the  compliance  check 
(0.219 ms) become unnoticeable. 

Table 1 Impact of policy compliance check 

Table 2 Comparison of current fixes and the fixes with Morbs 

Problems 

Fix w/o Morbs 

Fix w. Morbs 

Channel 

Type of 
Communication 

intent 

scheme 

utility 
classes 

in-app 
cross-app 
app-app 
web-app 
HttpClient 
WebView 

Communication 
Delay w/o 
Morbs (ms) 
42.142 
46.267 
64.077 
115.301 
225.035 
692.955 

Impact of Morbs 
policy checking 

0.52% 
0.47% 
0.34% 
0.19% 
0.10% 
0.03% 

5.3  Compatibility and Developer’s Effort 
An  important  goal  of  Morbs  is  to  maintain  compatibility  when 
possible and minimize the developer’s effort to use its protection. 
Following we elaborate our study on these two issues. 
Compatibility.  To  see  whether  our  implementation  can  work 
with existing apps, we loaded Android with Morbs into a Nexus 4 
development  phone  and  evaluated  the  operations  of  top  20  free 
apps downloaded from Google Play market. Those apps were first 
analyzed: we disassembled their binary code and found that all of 
them use intent, 12 claim various schemes and all need to use the 
web through WebView and HttpClient classes. We then analyzed 
their  functionalities  in  the  presence  or  absence  of  Morbs 
mediation, by clicking on all buttons and using all of the services 
we could find.  During the test, we did not observed any deviation 
of those apps’ behaviors with and without our mechanism. 
Developer’s  effort.    As  discussed  before,  to  use  Morbs,  the 
developer  only  need  to  specify  her  whitelists  through  the 
interfaces  (e.g.,  the  setOriginPolicy  API)  we  provide,  which  is 
straightforward for them to act on.  This is compared to the case-
by-case fixes that app developers are currently doing in response 
to  our  vulnerability  reports.  In  Table  1,  we  give  a  comparison 
with  regard  to  the  vulnerabilities  described  in  Section  3.    The 
ways they are fixed (or to be fixed) (“Fix w/o Morbs”) come from 
our  conversations  with  corresponding  software  vendors.  Here, 
how to fix the problem 4 (the exploit through Yelp app) and 5 (the 
callback loophole) is still unknown. 
As  we  can  see  from  the  table,  these  vulnerabilities  are  much 
easier to fix with the support of Morbs. Specifically, for the next-
intent  problem  (Section  3.2.1),  both  Dropbox  and  Facebook 
informed us that an effective fix takes time to build. Particularly, 
Dropbox explained that they need to “change the architecture” of 
their app, which involves non-trivial effort. In the presence of our 
origin-based  protection,  however,  this  next-intent  cross-origin 
loophole is fixed without requiring any modification to the apps.  
As another example, for the fbconnect issue described in Section 
3.3.1, Facebook chose to deprecate the use of fbconnect, which is 
a  core  feature  in  all  of  its  native  SDKs  and  official  apps.  This 
effort  needs  “a  several  month  deprecation  period”,  according  to 
Facebook.  Using Morbs, however, Facebook could easily fix the 
problem  without  touching  any  of  its  SDKs  and  apps,  by  simply 
adding an extra header, including the origins of the apps supposed 
to receive its message, to the HTTP response its server sends to 
mobile devices. Overall, as shown in the table, the current fixes to 
these  problems  are  all  case  by  case,  while  our  solution  is 
consistent  in  the  way  to  set  origin-based  security  policies 
(whitelist of authorized origins) and enforce the policies.  

next-intent 
(Section 3.2.1) 

fbconnect 
(Section 3.3.1) 

Change architecture of the 
Dropbox  app  and 
the 
Facebook app  

this 

Deprecate 
feature 
(affecting  all  apps  with 
Facebook  SDKs, 
and 
taking several months) 

Dropbox 
SDK 
3.3.2) 

iOS 
(Section 

Change both the Dropbox 
apps and SDKs  

Yelp 
(Section 3.3.2) 

issue  

Unknown 

callback  exploit 
(Section 3.4.1) 

Unknown 

HTTPClient 
exploit  (Section 
3.4.2) 

Change  to  the  Dropbox 
app,  adding  code 
for 
checking  whether  a  URL 
is  from  dropbox.  com 
when 
attaching 
authorization header  

No modification 

Facebook server specifies 
recipient whitelist by setting a 
header in HTTP response 
“mobile- allowed-origins: 
app://com.facebook.katana”  

Dropbox SDK specifies sender 
whitelist by adding an entry 
“allowedOrigins: 
app://com.getdropbox.Dropbox” 
under “URL scheme” in .plist file. 

No modification 

Facebook app specifies sender 
whitelist by calling 
WebViewClient:setAllowedOrigi
ns(“https://*.facebook.com”) 

Dropbox app specifies recipient 
whitelist by calling 
HTTPClient.setAllowedOrigins(“
https://*.dropbox.com”). 

6.  RELATED WORK 
Origin-based  protection  in  web  browsers. 
  Origin-based 
protection is a cornerstone for browser security. All modern web 
browsers enforce the same-origin policy (SOP) [4] to protect the 
web content from one origin against unauthorized access from a 
different  origin.  Always  at  the  center  of  browser  security  is  the 
attacks that circumvent this protection, such as XSS, CSRF, login 
CSRF, and the defense that reinforces the browser and makes the 
protection  hard  to  bypass  [1][2][3].  Our  research  shows  that 
serious cross-origin attacks can also happen on mobile platforms 
and therefore the origin-based protection is equally important to 
mobile security. 
Under the SOP, cross-origin communication needs to go through 
designated channels with proper mediation. A prominent example 
is the postMessage channel [5], through which the web content of 
one origin can send messages to another domain, and the browser 
ensures  that  the  recipient  knows  the  true  origin  of  the  sender.  
However, the web developer of the recipient domain still needs to 
come up with her own policy enforcement logic, which could be 
error-prone.  Alternatively,  the  browser  can  act  on  whitelisted 
origins  specified  by  the  developer.  An  example  is  the  Cross-
Origin  Resource  Sharing  mechanism  [6],  through  which  the 
content  from  a.com  can  request  resources  from  b.com  server 
using  XMLHttpRequest  [7].  The  server  authorizes  this  cross-
origin activity to the browser by attaching to its HTTP response a 
header “Access-Control-Allow-Origin: a.com”, a whitelist for the 
requestor a.com.  The browser then enforces this policy, sending 
the message only to a.com webpages.  
The design of Morbs is pretty much in line with those browser-
based  security  mechanisms.  We  bring  in  this  origin-based 
protection  to  mobile  platforms,  making  the  true  origin  of  each 
message observable to app/web developers and also helping them 
enforce their policies at the OS level.  

644from 

the 

proper 

consents 

Security  on  mobile  platforms.  The  security  framework  of 
Android is built on i) the sandbox model [8], which separates an 
app’s data and code execution from that of the rest of the system, 
and 2) the permission model [9], which grants each app different 
level  of  privileges  to  access  system  resources  under  the  user’s 
consent.  Prior  studies  mainly  focus  on  circumventing  such 
protection to obtain private user data (e.g., GPS location, phone 
contacts)  or  perform  privileged  operations  (e.g.,  sending  SMS 
messages)  without 
user 
[10][11][12][13][14][27].  Most  related  to  our  work  here  is 
permission re-delegation [10], in which an unprivileged app sends 
an  intent  to  another  app  with  a  proper  permission  to  act  on  its 
behalf, operating on the resources (e.g., GPS, user contacts, etc.) 
it  is  not  supposed  to  touch.  However,  this  problem  has  been 
studied  mainly  for  understanding  the  threat  to  mobile  devices’ 
local resources. What we investigated is the protection of an app’s 
web  resources,  which  has  not  been  explicitly  included  in 
Android’s  security  models.  Luo  et  al.  conducted  two  studies 
specifically  about  security  issues  related  to  WebView:  in  [42], 
they categorized existing issues raised by other researchers and a 
number of issues discovered by them. Many of these issues were 
shown  to  affect  Android  applications  that  use  the  open-source 
package DroidGap; in [43], they proposed a type of attack called 
“touchjacking”,  which  targets  the  weaknesses  of  WebView’s 
handling of touch events. 
To address those problems, numerous defense mechanisms have 
been  proposed  [17][18][19][20].  Particularly,  information-flow 
techniques, such as TaintDroid [15] and Vision [16], are used to 
track  the  propagation  of  sensitive  user  data  across  a  suspicious 
app  at  the  instruction  level.  Different  from  those  existing 
techniques, our protection mechanism is designed to keep track of 
the origin of the message exchanged between the initiator and the 
recipients  for  origin-based  mediation.  For  this  purpose,  we  only 
need  to  work  on  the  API  level  (given  that  the  OS  is  trusted), 
which  is  much  more  efficient.  A  related  technique  called  Quire 
[21]  enables  Android  to  trace  and  sign  the  whole  IPC  chain 
observed by the OS during intent messaging, so that the recipient 
of  an  intent  can  find  out  its  initiator.  However,  this approach is 
not  designed  to  determine  a  request’s  web  origin:  for  example, 
when an app is activated through a scheme URL generated by a 
malicious  webpage  displayed  in  the  WebView  instance  of  the 
Facebook app, looking at the IPC chain does not tell the recipient 
app that it is actually originated from the malicious domain.   
Similar call-sequence analyses have been done on iOS to detect 
information leaks through iOS apps [22][23].  The focus of these 
analyses  is  on  malicious  apps,  while  our  focus  is  on  protecting 
benign apps. 
7.  CONCLUSION AND FUTURE WORK 
Unlike  traditional  web  applications,  which  enjoy  browser-level 
origin-based protection, apps are hosted on mobile OSes, whose 
security  models  (e.g.,  sandbox  and  permission  models)  are  not 
designed  to  safeguard  resources  based  their  web  origins.  Our 
research shows that in the absence of such protection, the mobile 
channels  can  be  easily  abused  to  gain  unauthorized  access  to  a 
user’s sensitive web resources. We found 5 cross-origin issues in 
popular  SDKs  and  high-profile  apps  such  as  Facebook  and 
Dropbox,  which  can  be  exploited 
their  users’ 
authentication credentials and other confidential information such 
as text input. Moreover, without the OS support for origin-based 
protection,  not  only  is  app  development  shown  to  be  prone  to 
such cross-origin flaws, but the developer may also have trouble 
fixing the flaws even after they are discovered. This points to the 
urgent  need  of  building  origin-based  protection  into  mobile 

to  steal 

platforms. In our research, we designed and implemented the first 
such  protection  mechanism,  Morbs,  for  mediating  cross-origin 
communications at the OS level.  Our evaluation shows that the 
new  technique  effectively  and  efficiently  controls  the  risks  that 
come  with  the  communications,  and  can  also  be  conveniently 
utilized by the app and web developers.  
Our  current  implementation  is  for  Android.  Building  this  new 
protection  on  iOS  is  equally  important.  Also  interesting  is  the 
effort  to  automatically  analyze  existing  apps,  to  identify  their 
cross-origin  vulnerabilities  and  defend  them  using  the  origin 
protection we provided. More generally, given the trend that web 
services  are 
further 
investigations  are  needed  to  understand  how  to  better  protect 
users’  web  resources  on  mobile  OSes,  which  were  originally 
designed to safeguard a device’s local resources. 
8.  ACKNOWLEDGEMENTS 
We thank Seungyeop Han, Ravi Bhoraskar, and Jaeyeon Jung for 
their  help  on  monitoring  HTTPS  traffic  of  Android  emulator. 
Authors from Indiana University are supported in part by National 
Science Foundation CNS-1117106 and CNS-1223477. 
9.  REFERENCES 
[1]  Fogie, S., Grossman, J., Hansen, R., Rager, A., & Petkov, P. 

increasingly  delivered 

through  apps, 

D. (2007). XSS Attacks: Cross Site Scripting Exploits and 
Defense. Syngress Publishing. 

[2]  Auger, R. (2008). The cross-site request forgery (csrf/xsrf) 

faq. CGISecurity. com. Apr, 17. 

[3]  Barth, A., Jackson, C., & Mitchell, J. C. (2008, October). 

Robust defenses for cross-site request forgery. In 
Proceedings of the 15th ACM conference on Computer and 
communications security (pp. 75-88). ACM. 

[4]  Barth, A. (2011). The web origin concept. 
[5]  Cross-document messaging – HTML standard. 
http://www.whatwg.org/specs/web-apps/current-
work/multipage/web-messaging.html#web-messaging 

[6]  van Kesteren, A. (2010). Cross-origin resource sharing. W3C 

Working Draft WD-cors-20100727. 

[7]  Garrett, J. J. (2005). Ajax: A new approach to web 

applications. 

[8]  Android Developers: Security Tips. 

http://developer.android.com/training/articles/security-
tips.html 

[9]  Android Developers: Permissions. 

http://developer.android.com/guide/topics/security/permissio
ns.html 

[10] Felt, A. P., Wang, H. J., Moshchuk, A., Hanna, S., & Chin, 

E. (2011, August). Permission re-delegation: Attacks and 
defenses. In Proceedings of the 20th USENIX Security 
Symposium (Vol. 18, pp. 19-31). 

[11] Davi, L., Dmitrienko, A., Sadeghi, A. R., & Winandy, M. 

(2011). Privilege escalation attacks on android. In 
Information Security (pp. 346-360). Springer Berlin 
Heidelberg. 

[12] Grace, M., Zhou, Y., Wang, Z., & Jiang, X. (2012, 

February). Systematic detection of capability leaks in stock 
Android smartphones. In Proceedings of the 19th Annual 
Symposium on Network and Distributed System Security. 

[13] Schlegel, R., Zhang, K., Zhou, X., Intwala, M., Kapadia, A., 
& Wang, X. (2011, February). Soundcomber: A stealthy and 
context-aware sound trojan for smartphones. In Proceedings 

645of the 18th Annual Network and Distributed System Security 
Symposium (NDSS) (pp. 17-33). 

[14] Schrittwieser, S., Frühwirt, P., Kieseberg, P., Leithner, M., 
Mulazzani, M., Huber, M., & Weippl, E. (2012, February). 
Guess Who’s Texting You? Evaluating the Security of 
Smartphone Messaging Applications. In Proceedings of the 
19th Annual Symposium on Network and Distributed System 
Security. 

[15] Enck, W., Gilbert, P., Chun, B. G., Cox, L. P., Jung, J., 

McDaniel, P., & Sheth, A. N. (2010, October). TaintDroid: 
an information-flow tracking system for realtime privacy 
monitoring on smartphones. In Proceedings of the 9th 
USENIX conference on Operating systems design and 
implementation (pp. 1-6). 

[16] Gilbert, P., Chun, B. G., Cox, L. P., & Jung, J. (2011, June). 
Vision: automated security validation of mobile apps at app 
markets. In Proceedings of the second international 
workshop on Mobile cloud computing and services (pp. 21-
26). ACM. 

[17] Bugiel, S., Davi, L., Dmitrienko, A., Fischer, T., Sadeghi, A. 

R., & Shastry, B. (2012, February). Towards taming 
privilege-escalation attacks on Android. In 19th Annual 
Network & Distributed System Security Symposium (NDSS) 
(Vol. 17, pp. 18-25). 

[18] Shekhar, S., Dietz, M., & Wallach, D. S. (2012). Adsplit: 

Separating smartphone advertising from applications. CoRR, 
abs/1202.4030. 

[19] Fragkaki, E., Bauer, L., Jia, L., & Swasey, D. (2012). 

Modeling and enhancing Android’s permission system. In 
Computer Security–ESORICS 2012 (pp. 1-18). Springer 
Berlin Heidelberg. 

[20] Davi, L., Dmitrienko, A., Egele, M., Fischer, T., Holz, T., 
Hund, R., ... & Sadeghi, A. R. (2012, February). MoCFI: A 
framework to mitigate control-flow attacks on smartphones. 
In Proceedings of the 19th Annual Symposium on Network 
and Distributed System Security. 

[21] Dietz, M., Shekhar, S., Pisetsky, Y., Shu, A., & Wallach, D. 
S. (2011, August). Quire: Lightweight provenance for smart 
phone operating systems. In Proceedings of the 20th 
USENIX Security Symposium. 

[22] Egele, M., Kruegel, C., Kirda, E., & Vigna, G. (2011, 

February). PiOS: Detecting privacy leaks in iOS 
applications. In Proceedings of the Network and Distributed 
System Security Symposium. 

[23] Werthmann, T., Hund, R., Davi, L., Sadeghi, A. R., & Holz, 

T. (2013). PSiOS: Bring Your Own Privacy & Security to 
iOS Devices. 

[24] Hardy, N. (1988). The Confused Deputy:(or why capabilities 

might have been invented). ACM SIGOPS Operating 
Systems Review, 22(4), 36-38. 

[25] Hermandroid. “Launching an Android application from a 

URL”. http://androidsmith.com/2011/07/launching-an-
android-application-from-a-url/ 
[26] Apple URL Scheme Reference. 

http://developer.apple.com/library/ios/#featuredarticles/iPho
neURLScheme_Reference/Introduction/Introduction.html 
[27] Chin, E., Felt, A. P., Greenwood, K., & Wagner, D. (2011, 

June). Analyzing inter-application communication in 
Android. In Proceedings of the 9th international conference 

on Mobile systems, applications, and services (pp. 239-252). 
ACM. 

[28] Android Developers: WebView.addJavaScriptInterface. 

http://developer.android.com/reference/android/webkit/Web
View.html#addJavascriptInterface%28java.lang.Object,%20j
ava.lang.String%29 

[29] Android Developers: Parcel. 

http://developer.android.com/reference/android/os/Parcel.ht
ml 

[30] Android Developers: Java Native Interface. 

http://developer.android.com/training/articles/perf-jni.html 

[31] Supporting materials for this work. 

http://research.microsoft.com/en-us/um/people/ruiwan/mobile-
origin/index.html 

[32] A local file loaded from SD card to webview on Android can 

cross-domain. http://lists.grok.org.uk/pipermail/full-
disclosure/2012-February/085619.html 

[33] Android-apktool – A tool for reverse engineering Android 

apk files. http://code.google.com/p/android-apktool/ 

[34] AndroChef Java Decompiler. 

http://www.neshkov.com/ac_decompiler.html 

[35] Facebook Developers – Dialogs Overview. 

https://developers.facebook.com/docs/reference/dialogs/ 

[36] Android Developers – HttpClient. 

http://developer.android.com/reference/org/apache/http/clien
t/HttpClient.html 

[37] Android Developers – HttpURLConnection. 

http://developer.android.com/reference/java/net/HttpURLCo
nnection.html 

[38] iOS Developer Library – NSURLConnection Class 

Reference. 
http://developer.apple.com/library/ios/#documentation/Coco
a/Reference/Foundation/Classes/NSURLConnection_Class/
Reference/Reference.html#//apple_ref/occ/cl/NSURLConnec
tion 

[39] iOS Developer Library – Making HTTP and HTTPS 

Requests. 
http://developer.apple.com/library/ios/#documentation/Netw
orkingInternetWeb/Conceptual/NetworkingOverview/Worki
ngWithHTTPAndHTTPSRequests/WorkingWithHTTPAndH
TTPSRequests.html 

[40] The implementation of the mobile origin-based security 
mechanism (Morbs) on Android is published on GitHub. 
https://github.com/mobile-security/Morbs 

[41] PlainText – Dropbox text editing for iPhone, iPod touch, and 
iPad. https://itunes.apple.com/us/app/plaintext-dropbox-text-
editing/id391254385?mt=8 

[42] Luo, T., Hao, H., Du, W., Wang, Y., & Yin, H. (2011, 

December). Attacks on WebView in the Android system. In 
Proceedings of the 27th Annual Computer Security 
Applications Conference (pp. 343-352). 

[43] Luo, T., Jin, X., Ajai, A., & Du, W. Touchjacking attacks on 
web in android, ios, and windows phone. In Proceedings of 
5TH International Symposium on Foundations & Practice of 
Security (FPS 2012). 
 

 

646|
sec13-paper_zhu.pdf,|The Velocity of Censorship: High-Fidelity Detection 

of Microblog Post Deletions

Tao Zhu, Independent Researcher; David Phipps, Bowdoin College;  

Adam Pridgen, Rice University; Jedidiah R. Crandall, University of New Mexico;  

Dan S. Wallach, Rice University

Open access to the Proceedings of the 22nd USENIX Security Symposium is sponsored by USENIXThis paper is included in the Proceedings of the 22nd USENIX Security Symposium.August 14–16, 2013 • Washington, D.C., USAISBN 978-1-931971-03-4The Velocity of Censorship: High-Fidelity Detection of Microblog Post

Deletions

Tao Zhu

zhutao777@gmail.com
Independent Researcher

David Phipps

Computer Science
Bowdoin College

Adam Pridgen

Computer Science
Rice University

Jedidiah R. Crandall
Computer Science

University of New Mexico

Dan S. Wallach
Computer Science
Rice University

Abstract
Weibo and other popular Chinese microblogging sites are
well known for exercising internal censorship, to comply
with Chinese government requirements. This research
seeks to quantify the mechanisms of this censorship:
how fast and how comprehensively posts are deleted.
Our analysis considered 2.38 million posts gathered over
roughly two months in 2012, with our attention focused
on repeatedly visiting “sensitive” users. This gives us a
view of censorship events within minutes of their occur-
rence, albeit at a cost of our data no longer representing a
random sample of the general Weibo population. We also
have a larger 470 million post sampling from Weibo’s
public timeline, taken over a longer time period, that is
more representative of a random sample.

We found that deletions happen most heavily in the
first hour after a post has been submitted. Focusing
on original posts, not reposts/retweets, we observed that
nearly 30% of the total deletion events occur within 5–
30 minutes. Nearly 90% of the deletions happen within
the first 24 hours. Leveraging our data, we also consid-
ered a variety of hypotheses about the mechanisms used
by Weibo for censorship, such as the extent to which
Weibo’s censors use retrospective keyword-based cen-
sorship, and how repost/retweet popularity interacts with
censorship. We also used natural language processing
techniques to analyze which topics were more likely to
be censored.

1

Introduction

Virtually all measurements of Internet censorship are bi-
ased in some way, simply because it is not feasible to
test every keyword or check every post at small incre-
ments of time. In this paper, we describe our method for
tracking censorship on Weibo, a popular microblogging
platform in China, and the results of our measurements.
Our system focuses on a core set of users who are in-

terconnected through their social graph and tend to post
about sensitive topics. This biases us towards the content
posted by these particular users, but enables us to mea-
sure with high fidelity the speed of the censorship and
discern interesting patterns in censor behaviors.

Sina Weibo (weibo.com, referred to in this paper sim-
ply as “Weibo”) has the most active user community of
any microblog site in China [39]. Weibo provides ser-
vices which are similar to Twitter, with @usernames,
#hashtags, reposting, and URL shortening. In February
2012, Weibo had over 300 million users, and about 100
million messages sent daily [3]. Like Twitter in other
countries, Weibo plays an important role in the discourse
surrounding current events in China. Both professional
reporters and amateurs can provide immediate, first-hand
accounts and opinions of events as they unfold. Also like
Twitter, Weibo limits posts to 140 characters, but 140
characters in Chinese can convey significantly more in-
formation than in English. Weibo also allows embedded
photos and videos, as well as comment threads attached
to posts.

China employs both backbone-level filtering of IP
packets [5, 6, 11, 23, 37, 43] and higher level filtering
implemented in the software of, for example, blog plat-
forms [15, 20, 28], chat programs [13, 29] and search en-
gines [30, 41]. Work specific to Weibo [2, 9] is discussed
in more detail in Section 2. To our knowledge ours is the
first work to focus on how quickly microblog posts are
removed—on a scale of minutes after they are posted.
This fidelity in measurement allows us to not only accu-
rately measure the speed of the censorship, but also to
compare censorship speeds with respect to topics, censor
methods, censor work schedules, and other illuminating
patterns.

What our results illustrate is that Weibo employs
“defense-in-depth” in their strategy for filtering content.
Internet censorship represents a conflict between the cen-
sors, who seek to filter content according to some policy,
and the users who are subject to that censorship. Censor-

USENIX Association  

22nd USENIX Security Symposium  227

ship can serve to squelch conversations directly as well as
to chill future discussion with the threat of state surveil-
lance. Our goal in this paper is to catalog the wide variety
of mechanisms that Weibo’s censors employ.

This research has several major contributions:

• We describe the implementation of a method that
can detect a censorship event within 1–2 minutes of
its occurrence. A large amount of Weibo posts are
collected constantly via two APIs [26]. There are
more than 470 million posts from the public time-
line and 2.38 million posts from the user timeline in
our database.

• To further understand how the Weibo system can
react so quickly in terms of deleting posts with sen-
sitive content, we propose four hypotheses and at-
tempt to support each with our data. We also de-
scribe several experiments that shed light on cen-
sorship practices on Weibo. The overall picture we
illuminate in this paper is that Weibo employs a
distributed, defense-in-depth strategy for removing
sensitive content.

• Using natural language processing techniques that
overcome the usage of neologisms, named entities,
and informal language which typifies Chinese social
media, we perform a topical analysis of the deleted
posts and compare the deletion speeds for different
topics. We find that the topics where mass removal
happens the fastest are those that are hot topics in
Weibo as a whole (e.g., the Beijing rainstorms or a
sex scandal). We also find that our sensitive user
group has overarching themes throughout all topics
that suggest discussion of state power (e.g., Beijing,
government, China, and the police).

The rest of this paper is structured as follows. Sec-
tion 2 gives some basic background information about
microblogging and Internet censorship in China. Then
Section 3 describes the methods we used for our mea-
surement and analysis, followed by Section 4 that de-
scribes the timing of censorship events. Section 5 intro-
duces the natural language processing we applied to the
data and presents results from topical analysis. Finally,
we conclude with a discussion of various Weibo filtering
mechanisms in Section 6.

2 Background

Starting from 2010, when microblogs debuted in China,
not only have there been many top news stories where
the reporting was driven by social media, but social me-
dia has also been part of the story itself for a number
of prominent events [21, 38], including the protests of

Wukan [33], the Deng Yujiao incident [32], the Yao
Jiaxin murder case [35], and the Shifang protest [36].
There have also been events where social media has
forced the government to address issues directly, such as
the Beijing rainstorms in July 2012.

Chinese social media analysis is challenging [27].
One of many concerns that can hinder this work is the
general difficulty of mechanically processing Chinese
text. Western speakers (and algorithms) expect words
to be separated by whitespace or punctuation. In writ-
ten Chinese, however, there are no such word bound-
ary delimiters. The word segmentation problem in Chi-
nese is exacerbated by the existence of unknown words
such as named entities (e.g., people, companies, movies)
or neologisms (substituting characters that appear sim-
ilar to others, or otherwise coining new euphemisms
or slang expressions, to defeat keyword-based censor-
ship) [12]. Furthermore, since social media is heavily
centered around current events, it may well contain new
named entities that will not appear in any static lexi-
con [8].

Despite these concerns, Weibo censorship has been
the subject of previous research. Bamman et al. [2]
performed a statistical analysis of deleted posts, show-
ing that the presence of some sensitive terms indicated a
higher probability of the deletion of a post. Their work
also showed some geographic patterns in post deletion,
with posts from the provinces of Tibet and Qinghai ex-
hibiting a higher deletion rate than other provinces. Wei-
boScope [9] also collects deleted posts from Weibo, but
their strategy is to follow all users with a high number of
followers. This is in contrast to our strategy which is to
follow a core set of users who have a high rate of post
deletions, some of which have many followers and some
of which have few. The deletion events in these works
are measured with a resolution of hours or days. Our
system is able to detect deletion events at the resolution
of minutes.

3 Methodology

To have a better understanding of what the Weibo system
is targeting for censorship deletions, and how fast they do
so, we have developed a system which collects removed
posts on targeted users in almost real time.

Identifying the sensitive user group

3.1
In Weibo each IP address and Application Programming
Interface (API) has a rate limit for access to the service.
This forced us to make a number of engineering com-
promises, notably focusing our attention where we felt
we could find those posts most likely to be subject to
censorship. We decided to focus on users who we have

228  22nd USENIX Security Symposium 

USENIX Association

seen being censored in the past, under the assumption
that they will be more likely to be censored in the future.
We call this group of users the sensitive group.

We started with 25 sensitive users that we discov-
leveraging a list from China Digital
ered manually,
Times [4] of sensitive keywords which are not allowed
to be searched on Weibo’s server. To find our initial
sample, we searched using out-dated keywords that were
later un-banned. For example, 党产共 (Reverse of 共产
党, which means “Communist Party”) was found to be
banned on 4 April 2011, but found to not be banned on
20 October 2011, which means the we were able to ob-
tain some posts containing 党产共 when we searched for
this keyword after 20 October 2011. From the search re-
sults, we picked 25 users who stood out for posting about
sensitive topics.

Next, we needed to broaden our search to a larger
group of users. We assumed that anybody who has been
reposted more than five times by our sensitive users must
be sensitive as well. We followed them for a period of
time and manually measured how often their posts were
deleted. Any user with more than 5 deleted posts was
added to our pool of sensitive users.

After 15 days of this process, our sensitive group in-
cluded 3,567 users, and within this group we observed
more than 4,500 post deletions daily, including about
1,500 “permission denied” deletions. (See Section 3.3
for discussion on different types of deletion events.)
Roughly 12% of the total posts from our sensitive users
were eventually deleted. Further, we have enough of
these posts to be able to run topical analysis algorithms,
letting us extract the main subjects that Weibo’s censors
seemed concerned with on any given day.

We contrast these statistics with WeiboScope [9], de-
veloped at the University of Hong Kong in order to track
trends on Weibo concurrently with our own study. The
core difference between our work and WeiboScope is
that they track a large sample: around 300 thousand users
who each have more than 1000 followers. Despite this,
they report observing no more than 100 “permission de-
nied” deletions per day. WeiboScope’s results, therefore,
are perhaps more representative of the overall impact of
Weibo’s censorship as a fraction of total Weibo traffic,
while our work has more resolving power to consider the
speed and techniques employed by Weibo’s censors.

Because we do not have access to WeiboScope’s data,
we are limited in our ability to make direct comparisons
of our datasets. They did briefly support data down-
loads, and we extracted their “2,500 last permission de-
nied data” on 20 July 2012. This service has since been
closed. Our system went live following user timelines
on the same date, giving us a single day from which we
might compare our data. For 20 July 2012, WeiboScope
observed 54 permission-denied posts, while our system

observed 1,056.

(Our own system does not yet support public, real-
time downloads of our data, which among other issues
could make it easier for Weibo to shut it down. An appro-
priate means of disseminating real-time results or regular
summaries is future work for our group.)

While our methodology cannot be considered to yield
a representative sample of Weibo users overall, we be-
lieve it is representative of how users who discuss sensi-
tive topics will experience Weibo’s censorship. We also
believe our methodology enables us to measure the top-
ics that Weibo is censoring on any given day.

3.2 Crawling
Once we settled on our list of users to follow, we wanted
to follow them with sufficient fidelity to see posts as they
were made and measure how long they last prior to being
deleted. Our target sampling resolution was one minute.
We use two APIs provided by Weibo, allowing us to
query individual user timelines as well as the public time-
line1. Starting in July 2012, we queried each of our 3,500
users, once per minute, for which Weibo returns the most
recent 50 posts. Deleted posts outside of this 50-post
window are not detected by our system, meaning that we
may be underestimating the number of older posts that
get deleted.

We also queried the public timeline roughly once ev-
ery four seconds, for which Weibo returns 200 recent
posts. Half of these posts appear to be 1–5 minutes older
than real-time, and the other half are hours older.

Weibo does not support anonymous queries to its
servers, requiring us to create fake accounts on the ser-
vice. Weibo further enforces rate limits both on these
users’ queries as well as on source IP addresses, regard-
less of what user account is being used for the query. To
overcome these concerns, we used roughly 300 concur-
rent Tor circuits [24], driven from our research comput-
ing cluster. Our resulting data was stored and processed
on a four-node cluster using Hadoop and HBase [1].

If and when Weibo might make a concerted effort to
block us, it is easy to imagine a ongoing game where
they invent new detection strategies and we invent new
workarounds. So far, this has not been an issue.

3.3 Detecting deletions
An absent post may have been censored, or it may have
been deleted for any of a variety of other reasons. User

1The user timeline returns both original posts and retweeted posts
by that user, while the public timeline only returns original posts. Also,
the public timeline appears to be only a sampling of the total public
traffic.

USENIX Association  

22nd USENIX Security Symposium  229

accounts can also be closed, possibly for censorship pur-
poses. Users cannot delete their own account, only the
system can delete accounts. We conducted a variety of
short empirical tests to see if we could distinguish the
different cases. We concluded that we can detect two
kinds of deletions.

If a user deletes his or her own post, a query for that
post’s unique identifier will return a “post does not ex-
ist” error. We have observed this same error code re-
turned from censorship events and we refer to these, in
the remainder of the paper as general deletion. However,
there is another error code, “permission denied,” which
seems to indicate that the relevant database record still
exists but has been flagged by some censorship event.
We refer to these as permission-denied deletions or sys-
tem deletions. In either case, the post is no longer visible
to Weibo users.

The ratio of system deletions to general deletions in
our user timeline data set is roughly 1:2. In this paper, we
generally focus on posts that have been system deleted,
because there appears to be no way for a user to induce
this state. It can only be the result of a censorship event
(i.e., there are no censorship false positives in our system
deletion dataset). Because we followed a core set of users
who post on sensitive subjects, we did not find it neces-
sary to account for spam in our user timeline dataset.

Our crawler, which repeatedly fetches each sensitive
user’s personal timeline, is searching for posts that ap-
pear and then are subsequently deleted. If a post is in
our database but is not returned from Weibo, then we
issue a secondary query for that post’s unique ID to de-
termine what error message is returned. Ultimately, with
the speed of our crawler, we can detect a censorship event
within 1–2 minutes of its occurrence.

For each returned post from Weibo, there is a field
which records the creation time of the post. The life-
time of a post is the time difference between the time our
system detected the post being deleted and the creation
time. Therefore a post’s lifetime recorded by our system
is never shorter than its real lifetime, and never longer
than its real lifetime by more than two minutes.

4 Timing of censorship

For easier explanation we first give some definitions. A
post can be a repost of another post, and can have embed-
ded images. Also other users can repost reposts. If post
A is a repost of post B, we call post A a child post and
post B a parent post. If post A is not a repost of another
post, we call post A a regular post.

Using our user tracking method, from 20 July 2012 to
8 September 2012, we have collected 2.38 million user
timeline posts, with a 12.8% total deletion rate (4.5% for
system deletions and 8.3% for general deletions). Note

that this deletion rate is specific to our users and not rep-
resentative of Weibo as a whole. With a brief analysis,
we found that 82% of the total deletions are child posts,
and 75% of the total deletions have pictures either in
themselves or in their parent post.

(a) Sys. del. posts

(b) Sys. del. posts in first 2h

● ●

●

●●●

y
c
n
e
u
q
e
r
F

y
c
n
e
u
q
e
r
F

0
0
5

0
5

5

1

0
5

0
2

0
1

5

2

1

●

●●●
●
●●

●●

●●●

●●●●●●
●●●
●
●
●
●
●●●●
●
●●
●
●
●
●
●
●
●●
●
●
●●●●●
●●
●●●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●●
●●
●
●●●
●
●●●●
●●●
●●●
●●
●
●
●●●●
●●
●●
●
●●●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●●●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●●●●
●
●
●●
●
●●
●●●
●●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●●
●
●●
●
●●●
●●
●
●●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●●●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●

●

●

●●

●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●

●

●

●

●

●

●●
●
●
●
●●●
●
●
●●
●●
●●●●●
●●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●●●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●●●
●●
●
●
●
●●●●●
●●
●
●●
●
●
●
●●
●
●●●
●
●
●●●●●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●●

●
●
●●●
●
●
●
●

●
●

●

●
●
●
●

●

●

●
●
●●
●
●
●
●●
●●●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●●
●
●●●●
●
●
●
●
●●
●●●
●●
●
●
●
●
●
●
●
●●
●
●
●●●●●
●●
●
●
●●●●
●●●●
●
●
●
●●
●
●
●
●
●●
●
●●●
●
●●●
●
●
●
●●●
●
●
●
●●●
●●
●●
●●●
●●
●
●
●●
●●
●
●●
●●●●●●
●
●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●●
●●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●
●
●●●●
●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●
●●●●●●●●●●●●●
●●●
●●
●
●
●●
●●●●●●
●
●●●
●
●
●
●●●●
●●
●●●●●
●
●●●●●
●●
●
●●
●
●
●●
●
●●●
●●●
●●●
●●●●●●●
●●●●●●
●●●●●●●●●●●●
●●●●
●
●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●
●●●●●●●●●●●●●●●●●●●
●
●●●●●●
●●●●●●●●●●●●●●
●
●
●●●●
●
●
●●●●●
●
●●●●●●
●●●●●
●●●●
●●●●●●●●●
●
●●●●
●●
●
●●
●●●●●●
●
●●●●●●●●
●●
●●●●●
●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●● ●● ●

y
c
n
e
u
q
e
r
F

0
0
5
1

0
0
5

0

10

100

1000

100000

0

20

40

60

80

100 120

Post lifetime in minutes

Post lifetime in minutes

(c) Sys. del. reg. posts(txt)

(d) Sys. del. reg. posts(txt) in first 2h

● ●

●

●

●

●

●●

●

●

●

●
●

●●
●
●

●

●

●
●

●

●

●

●

●

●

●

●

●

●

●

●
●

●

●

●

●

●

●

●

●

●

●
●

●
●

●

●●

●

●
●
●
●
●
●

●
●

●

●

●
●

●

●
●

●

●

●
●

●
●●●

●

●
●

●
●

●

●

●

●

●●
●
●●

●●

●

●
●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●

●●
●
●
●

●

●

●
●

●

●
●

●
●
●
●
●
●
●
●●
●
●

●●●●
●
●●●
●●
●
●
●

●
●
●
●●
●
●●
●
●●
●●●●
●
●●
●
●
●●●
●●
●
●
●●
●
●
●●
●
●
●●
●
●
●
●

●
●
●

●

●

●

●
●

●
●
●
●●
●
●
●●
●
●●●
●
●●●
●
●
●●●●
●●●●●●●●●
●●●●●●●●●●●●●●●●●●●●
●●
●●●●●
●
●●●
●●
●
●●●●
●●●
●
●●
●
●●
●
●●
●
●
●●●
●
●●
●●●●
●●●
●●●●●
●
●●●●●●●
●●
●
●
●●●●●●●●●●●●●
●●●
●●
●●
●
●
●
●
●●

●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

y
c
n
e
u
q
e
r
F

0
8

0
6

0
4

0
2

0

●

10

100

1000

100000

0

20

40

60

80

100 120

Post lifetime in minutes

Post lifetime in minutes

Figure 1: Lifetime histograms. (a) and (b) are the life-
time histograms of all system deletions. (c) and (d) are
the lifetime histograms of regular text-only posts. (a)
and (c) show the histogram of the whole lifetime, (b)
and (d) only show the first two hours of the lifetime
histogram.

To demonstrate how long a post survives before it gets
deleted, we analyze the system deletion data set (see Sec-
tion 3.3). Figure 1 gives us a big picture of how fast the
Weibo system works for censorship purposes. The x axes
are the length of the lifetime divided into 5-minute bins,
and the y axes are the count of the deleted posts hav-
ing the lifetime in the corresponding bin. We note that
these figures have the distinctive shape of a power law or
long tailed distribution, implying that there is no partic-
ular time bound on Weibo’s censorship activity, despite
the bulk of it happening quickly, and that metrics like
mean and median are not as meaningful as they are in a
normal distribution.

We can see that the post bins with small lifetimes are
large. We zoom into the first 2 hours of data, which is
plotted in Figure 1 (c) and (d). This tells us that system
deletions start within 5 minutes, the same as text-only
regular posts. For both of them, the modal deletion age
appears to be between 5–10 minutes.

In our data set, 5% of the deletions happened in the
first 8 minutes, and within 30 minutes, almost 30% of

230  22nd USENIX Security Symposium 

USENIX Association

the deletions were finished. More than 90% of deletions
happened within one day after a post was submitted. This
demonstrates why a measurement fidelity on the order of
minutes, rather than days, is critical.

Considering the big data set that Weibo has to process,
the speed, especially the 5 to 10 minutes peak, is fast,
especially considering that the data cannot be processed
in a fully automated way. How can the Weibo system
find sensitive posts and remove them so quickly? On
the other hand, the long tails suggest that sensitive posts
can still be deleted even after an extended period. How
are those sensitive posts located by the moderators after
a month in their huge database? What factors affect a
post’s lifetime?

In this section, to find the answers to these questions,
we propose four hypotheses and then test them against
our data. Hypotheses 1 and 2 try to explain how the speed
of censorship on Weibo can be so fast. Hypothesis 3 ex-
plains why we see the long tails of the post lifetime for
censored posts in Figure 1. Hypothesis 4 tells us that the
deletion speed does not appear to be strongly related to
particular conversation topics, but rather to popular top-
ics (i.e., those that are being discussed on Weibo as a
whole according to our public timeline) where our core
sensitive users are putting a spin on the discussion that
involves themes of government power (see Section 5).

4.1 Post lifetime regression
Before we give our hypotheses, we first consider what
factors affect a post’s lifetime, regardless of the content
of the post.

For each post, besides the basic information about the
post itself, we also see an embedded picture, if present,
as well as a parent post identifier, if it is a repost. Also,
we know the number of followers and friends of each
user, as well as of any parent post’s user.

From the graphs in Figure 1, we decided to experi-
mentally fit a negative binomial regression to it to see
which factors affect the lifetime of a post. Table 1 and
Table 2 show the results for the regular posts and child
posts, respectively. Three asterisks (‘***’) indicates sta-
tistical significance, one asterisk (‘*’) indicates a coeffi-
cient that is not statistically significant, and no coefficient
is indicated with a dash (‘-’). We can regress the log life-
time for a regular post or a child post via:

+ b2(Friends #) +b 3(Posts #)

Regularlifetime) =Intercept + b1(PHasPic)

ln(¤(cid:30)
ln(⁄(cid:30)Childlifetime) =Intercept + b1(PHasPic)

+ b2(P.Friends #) +b 3(P.Posts #)

We examine the effect on post lifetime of: the exis-
tence of a picture, the number of friends and followers,
and the number of posts sent by this user. We found that,
for both regular and child posts, the existence of a pic-
ture affects the post’s lifetime the most. That is, posts
with pictures have shorter lifetimes than posts without
pictures. Some of the user attributes, such as number of
friends or number of posts, also affect post lifetime. We
note that the coefficients for these are relatively small,
but for users with large numbers of friends or who write
large numbers of posts, these factors can have a sig-
nificant impact on the speed of that users’ posts being
censored. However other attributes of a user, such as
whether a Weibo user is “verified” by Weibo (i.e., Weibo
knows who they are as part of newer Chinese require-
ments that crack down on pseudonyms unconnected to
real world identities) or the number of followers of a user,
are not statistically significant factors in a post’s lifetime.

Table 1: Factors affecting post lifetime (regular posts).

Factors
(Intercept)
Has picture
Number of friends
Number of posts
User verified
Number of followers

Coef
7.41
−4.07× 10−1
−2.42× 10−4
−5.23× 10−5

–
–

Stat. Sig.

***
***
***
***
-
-

Table 2: Factors affecting post lifetime (child posts)

Factors
(Intercept)
Parent has picture
Parent friends number
Parent posts number
Parent user verified
Parent followers number

Coef
6.27
−1.01× 10−1
−4.76× 10−5
6.84× 10−6
2.01× 10−1
–

Stat. Sig.

***
***
***
***
*
-

4.2 Hypotheses
As a distributed system with 70,000 posts per minute,
Weibo has above a 10% rate of deletion in the pub-
lic timeline (first observed by Bamman et al. [2]; we
have seen similar behavior). This high deletion rate can
be the result of many processes, including anti-spam
features, user deletions, as well as anti-censorship fea-
tures. Within the deletions that we believe are censor-
ship events, we note that 40% of the deletions in our user
timeline data set occur within the first hour after a post
has appeared. Clearly, Weibo exerts significant controls
over its content.

USENIX Association  

22nd USENIX Security Symposium  231

Before censors deal with the sensitive posts which are
already in the system, are there filters which do not allow
certain posts to enter the Weibo system? This question
leads to our first hypothesis.

Hypothesis 1 Weibo has filtering mechanisms as a
proactive, automated defense.

To find out if there are filtering mechanisms, we at-
tempted to post posts containing sensitive words from
the China Digital Times [4] and Tao et al. [41]. Here we
summarize the filtering mechanisms Weibo was found to
apply based on our observations.

• Explicit filtering: Weibo will inform a poster that
their post cannot be released because of sensitive
content.
For example, on 1 August 2012, we tried to post “政
法委书记” (Secretary of the Political and Legisla-
tive Committee). When we submitted a post with
this character string in it, a warning message says
“Sorry, since this content violates ‘Sina Weibo reg-
ulation rules’ or a related regulation or policy, this
operation cannot be processed.
If you need help,
please contact customer service.”

example when we

• Implicit filtering: Weibo sometimes suspends
posts until they can be manually checked, telling
the user that the delay is due to “server data syn-
chronization.”
submitted the post
For
‘youshenmefalundebanfa’ on the same day, 1
August 2012, Weibo responded with the mes-
sage “Your post has been submitted successfully.
Currently, there is a delay caused by server data
synchronization. Please wait for 1 to 2 minutes.
Thank you very much.” This delay, which fre-
quently takes much longer than the 1–2 minutes
suggested by Weibo, was triggered by our use of
the substring “falun”, pertaining to the Falun Gong
religion. In this example, it took more than 5 hours
for the post to appear.

• Camouflaged posts: Weibo also sometimes makes
it appear to a user that their post was successfully
posted, but other users are not able to see the post.
The poster receives no warning message in this
case.
On 1 August 2012 we submitted a post contain-
ing the substring “cgc” (Chen Guangcheng [31]),
and received no warning messages, so it seemed
to be published successfully to our user. When we
tried to access that post from another user account,
however, we were redirected to Weibo’s error page
which claimed the post does not exist.

We found these phenomena to be repeatable. Over the
course of our experiments, we selected a number of dif-
ferent subsets of the keyword list published by the China
Digital Times [4], trying to post them to Weibo manually.
We consistently found all of these same phenomena, al-
though the specific keywords on any list vary over time.
Figure 1 shows that the deletions happen most heav-
ily for a regular post within 5 to 10 minutes of it being
posted. While we believe this process to happen largely
via automation, it is instructive to estimate how much un-
aided human labor would otherwise be necessary. Sup-
pose an efficient worker could read 50 posts per minute,
including the reposts and figures included in the posts.
Then to read Weibo’s full 70,000 new posts [34] in one
minute, 1,400 simultaneous workers would be needed.
Assuming 8 hour shifts, 4,200 workers would then be
required. We can imagine that such a staff would have
a high error rate, owing to the repetitive nature of their
work. Such a labor force would also be relatively expen-
sive compared to automation. We instead conclude that
Weibo must be using a large amount of automation, per-
haps keyword-based as has been found in other systems
in China such as TOM-Skype [16]. This is likely com-
plemented with human efforts to evolve and refine the
filtering process.

Some of this refinement certainly results from a cen-
tralized list of topics. Other refinement may occur inter-
nally, through a smaller number of censors who look for
users finding new ways to misspell words or otherwise
work around existing filters. Our subsequent hypotheses
consider how this refinement occurs and delve into how
Weibo’s automation operates.

Hypothesis 2 Weibo targets specific users, such as those
who frequently post sensitive content.

Another way to achieve prompt response to sensitive
posts is to track users who are likely to post sensitive
content, using techniques similar to what we are doing.
The posts from those sensitive users could then be read
by moderators more often and more promptly than the
posts of other users.

To test this hypothesis, we plotted Figure 2. We
grouped users together who have the same number of
censorship events occurring to their posts. The x-axis
is the number of such deletions for each cohort of users.
The y-axis shows how long these to-be-censored posts
live. The clear downward trend is evidence that users
with larger deletion frequencies tend to observe faster
censorship of their work, supporting our hypothesis.

Even though this figure shows us that the more dele-
tion posts a user has, the faster the users’ posts tend to
be deleted, we cannot rule out other features which those
users have in common and that those features may lead to
the fast deletions. For example, they may tend to use the

232  22nd USENIX Security Symposium 

USENIX Association

)
s
e

t

u
n
m

i

(
 

e
m

i
t

e

f
i
l
 
t
s
o
P

0
0
2

0
0
1

0
5

y
t
i
s
n
e
D

0
3

.

0

5
2

.

0

0
2

.

0

5
1

.

0

0
1

.

0

5
0

.

0

0
0

.

0

1

2

5

10

20

50

100

200

0

10

20

30

40

50

60

Deletion counts

Standard deviation(miniutes)

Figure 2: Users’ median post lifetime in minutes vs.
the number of deletions for that user on a log-log
scale. Black circles show the median lifetime of posts
in the cohort, and the dotted blue bars show the 25%–
75% range.

same keywords, post from the same geographical area,
use the same kind of client platform, and so on. There is
a clear correlation between post lifetime and post dele-
tion counts, but correlation does not imply causation.

If the surveillance keyword list and targeting of spe-
cific users were the only mechanisms for removing sen-
sitive posts, then the histograms in Figure 1 would stop
at a certain time, say 1 or 2 days. However, 10% of
the deletions happen after one day, with some deletions
occurring one month or more after the post was posted.
Clearly, other mechanisms are in use for these long-tail
censorship events, which leads to our next hypothesis.

Hypothesis 3 When a sensitive post is found, a moder-
ator will use automated searching tools to find all of its
related reposts (parent, child, etc.), and delete them all
at once.

If this hypothesis is true, then the child posts which
repost a censored parent post should all be removed at the
same time. To test this hypothesis, we plot the histogram
of the standard deviation of the deletion time of the posts
sharing the same Repost Identification Number (rpid) in
Figure 3. In our system deleted posts dataset, over 82%
of reposted posts have a deletion time standard deviation
of less than 5 minutes, meaning that a sensitive post is
detected and then most of the other posts in a chain of
reposts are immediately deleted.

Figure 3: Reposts standard deviation histogram.

There are outliers with standard deviations as high as 5
days which suggest that the mass deletion strategy men-
tioned here is not the only method Weibo employs to
delete sensitive reposts. This leads to our next hypoth-
esis.

Hypothesis 4 Deletion speed is related to the topic.
That is, particular topics are targeted for deletion based
on how sensitive they are.

We performed topical analysis on the deleted posts.
The topical analysis methods we use are described
in Section 5.1. Here, to save space, we only list the top
topic in Table 3. (For further topical discussion, please
refer to our technical report [42].) The third column is the
response time for the censor to discover a sensitive topic.
Specifically, the response time here refers to the period
between the time when the first post on this topic ap-
peared in our user timeline data set and the time when the
Weibo system starts to delete the posts on this topic heav-
ily. These times were identified through manual analysis.
Even when a topic is still being actively censored, it does
not necessarily disappear. People may still discuss the
topic only to have their posts deleted. That is why some
topics appear twice or more in the table. When a topic
showed up again, there is no response time for it and we
indicate this with a dash (‘-’).

The main five topics extracted by Independent Com-
ponent Analysis (ICA, see Section 5) are: Qidong, Qian
Yunhui, Beijing Rainstorm, Diaoyu Island2 and Group
Sex. From Table 3, we can see that these topics have a

2Diaoyu Island is the number 3 top topic on 16 August.

USENIX Association  

22nd USENIX Security Symposium  233

Date

Top 1

Table 3: Blocked topics.

Response

Time (hours)

Judicial independence

Freedom of speech

Support Syrian rebels
Lying of gov. (Jixian)

7-20
7-21
7-22 Beijing rainstorms
7-23 Beijing rainstorms (Subway)
7-24 Beijing rainstormsa
7-25 Beijing rainstorms (Fangshan)
7-27 Beijing rainstorms (37 death)
7-28 Qidong
7-29 Qidong (Japanese reporter)
7-30 Complain gov. (Zhou Jun)
7-31
8-01 Complain gov. (Hongkong)
8-02
8-03 Qidong (Block the village)
8-04 One-Child Policy Abuse
8-05 Human Rights News
8-07 Qian Yunhui Accident
8-08 Qian Yunhui Accident
8-09 Group sex
8-10 RTLb
8-11
Tang Hui
8-12 Group sex
8-13 Corpse Plants in Dalian
8-14 Hongkong
8-15 Corpse Plants in Dalian
8-16 Corpse Plants in Dalian
8-17 Complain gov. (North Korea)
8-18

Zhou Kehua (faked)

21.32
12.20
2.55
1.62
2.65
2.58
0.82
1.18
2.25
5.73
2.00
45.30
7.35
31.58
33.42
24.63
10.87

–
0.78
3.65
33.42

–

532.50
70.98

–
–

19.83
16.37

aRefuse to donate for Beijing rainstorms.
bRe-education through labor.

relatively short lifetime compared to other topics. These
five topics were also hot topics in our public timeline dur-
ing this period.

This suggests that when sensitive users and a large
number of regular Weibo users are discussing the same
general topic, i.e., the topic is popular in both the user
timeline and public timeline, then extra resources are de-
voted to finding and deleting such posts3. In Section 5
we will show that the sensitive users in the user time-
line combine topics with common themes related to state
power (Beijing, government, China, country, police, and
people). This suggests that the censors consider the com-
bination of these themes with generally popular topics to
warrant extra resources.

3We have not ruled out other possibilities in our study, however,
such as that such topics are viewed by many users and therefore more
likely to be reported by regular users.

5 Topic extraction

Even though we are following a relatively modest num-
ber of Weibo authors, the volume of text we are capturing
is still too much to process manually. We need automatic
methods to classify the posts that we see, particularly
those which are deleted.

Automatic topic extraction is the process of identify-
ing important terms in the text that are representative of
the corpus as a whole. Topic extraction was originally
proposed by Luhn [19] in 1958. The basic idea is to as-
sign weights to terms and sentences based on their fre-
quency and some other statistical information.

However, when it comes to microblog text, standard
language processing tools become inapplicable [18, 40].
Microblogs typically contain short sentences and casual
language [7]. Unknown words, such as named entities
and neologisms often cause problems with these term-
based models. It can be especially challenging to extract
topics from Asian languages such as Chinese, Korean,
and Japanese, which have no spaces between words.

We applied the Pointillism approach [27] and TF*IDF
to extract hot topics. In the Pointillism model, a corpus
is divided into n-grams; words and phrases are recon-
structed from grams using external information (specifi-
cally, temporal correlations in the appearance of grams),
giving the context necessary to manage informal uses of
the language such as neologisms. Salton’s TF*IDF [10]
assigns weights to the terms of a document based on the
terms’ relative importance to that document compared to
the entire corpus.

We next explain how these techniques work together.

5.1 Algorithm
TF*IDF is a common method to determine the impor-
tance of words to a document in a corpus. The TF*IDF
value in our case is calculated as:

f (t,dday)× log

Total number of posts for the month

f (t,dmonth)

Here, f (t,d) means the frequency of the term t in doc-
ument d. We use trigrams as t, and documents d are sets
of posts over a certain period of time. dday is the deleted
posts we caught on day day. We use the posts of July,
2012 in the public timeline as IDF. f (t,dmonth) is the fre-
quency of term t in the public timeline in July, 2012.

First we calculate TF*IDF scores for all trigrams that
have more than 20 occurrences in a day. The top 1000
trigrams with the highest TF*IDF score will be fed to
our trigram connection algorithm, hereafter “Connector.”
We call these top 1000 trigrams the 1000-TFIDF list.

234  22nd USENIX Security Symposium 

USENIX Association

To connect trigrams back into longer phrases, Connec-
tor finds two trigrams which have two overlapping char-
acters. For instance, if there are ABC and BCD, Con-
nector will connect them to become ABCD. Sometimes
there is more than one choice for connecting trigrams,
e.g., there could also be BCE and BCF. Sometimes tri-
grams can even form a loop. To solve these problems,
we first build directed graphs for the trigrams with a high
TF*IDF score. Each node is a trigram, and edges indi-
cate the overlap information between two trigrams. For
example, if ABC and BCD can be connected to make
ABCD, then there is an edge from ‘ABC’ to ‘BCD’. Af-
ter all trigrams are selected, we use DFT (Depth First
Traversal) to output the nodes. During the DFT we check
to see if a node has been traversed already.
If so we
do not traverse it again. After the graphs have been tra-
versed, we obtain a set of phrases.

For example, the Connector output of the third most

popular topic on 4 August 2012 is:

1.头骨进京鸣冤。河北广平县上坡村76岁的农民冯

虎，其子在19
skull go Beijing to redress an injustice. The son of a 76
year old farmer Fenghu, from Shangpo village, Guang-
ping city, Hebei province, was ... at 19
2.头骨进京鸣冤。冯出示的头骨赴京鸣...
skull go Beijing to redress an injustice. The skull shown
by Feng go Beijing to redress an injustice...
3.头骨进京鸣冤。冯出示的头骨前额有一大窟窿，
他...
skull go Beijing to redress an injustice. There is a big hole
on the skull shown by Feng, he...
4.头骨进京鸣冤。冯出示的头骨前额有一个无罪的公
民...
skull go Beijing to redress an injustice. There is a inno-
cent citizen on the skull shown by Feng, he...
5.头骨进京鸣冤。冯出示的头骨进...
skull go Beijing to redress an injustice. The skull shown
by Feng enter...
6.头骨进京鸣冤。冯出示的头等舱
skull go Beijing to redress an injustice. The first class seat
shown by Feng...
7.【華聯社電】上访15年 老父携儿头骨...
Chinese Community report: petition 15 years, old father
bring the skull of his son...

In this example, the 7 outputs of Connector are trans-
lated in English, which is written in the next line after the
original Chinese phrase. Outputs 4 and 6 are incorrectly
connected. This is because the same trigrams are shared
by different stories that have high TF*IDF scores on the
same day. This problem can be solved by examining the

cosine similarity of the frequency of occurrence of the
first and the last trigram for each result.

Cosine similarity is used to judge whether two tri-

grams have correlated trends.

»(cid:31)n

< Ai,Bi >

2 ×»(cid:31)n

cos.Sim =

i=1 Ai

2

i=1 Bi

where <, > denotes an inner product between two vec-

tors. For details, please refer to Song et al. [27].

From the connected sentences, listed above, we can
begin to understand the general events that are driving
major sensitive topics of discussion on Weibo. Table 3
lists the top topics of the deleted posts from 20 July 2012
to 18 August 2012.
(A computer failure prevented us
from collecting data on 6 August 2012.) Note that we just
translated the posts from each topical cluster, we have not
confirmed the veracity of any of the claims of the Weibo
users’ posts that we translated.

Interestingly, besides named entities, we also extracted
three neologisms. They are 李W阳 (Li Wangyang, from
李 旺 阳), 六 圌 四 (June Fourth, from 六 四), 胡()涛
(Hu Jintao, from 胡锦涛, replacing the middle charac-
ter with open- and close-parentheses), and 启-东, 启\东
and 启/东 (Qidong, from 启东, inserting punctuation be-
tween the two characters). These neologisms became
popular enough that they stood out in our TF*IDF anal-
ysis.

5.2 Hot sensitive topics
Table 3 tells us the top topic for each day in terms of
having the highest TF*IDF scores—however, it does not
tell us which topics among these have been discussed for
the longest period of time by our users. Also, are there
some common themes behind those separate topics?

Here are the top 50 words which have appeared in the
1000-TFIDF list most frequently from 20 July 2012 to
20 August 2013, manually translated to English:

Beijing City, Liu Futang, secretary, Lujiang County,
Guo Jinlong, Qian Yunhui, City Government, Zhou Ke-
hua, Red Cross, Diaoyu Island, subprefect, water drain,
ordinary people, taxpayer, Fangshan district, Hagens, lo-
cal police station, office, Beijing, Qidong, government,
China, Japan, citizen, county’s head commissioner, re-
porter, mayor, corrupt official, freedom, country, re-
strain, keyhole report, wrist watch, police, national, rec-
ommend, American, repression, patriotic, democratic,
corpses, people, donation, cancel, opinion, reeducation
through labor, abolition, truck4

We used Independent Component Analysis (ICA) to
extract “independent signals” from those most important

4For clarity, we have elided close variants on China, Japan, and

Beijing from this list.

USENIX Association  

22nd USENIX Security Symposium  235

terms shown above. ICA [14] is a method to separate a
linearly mixed signal, x, into mutually independent com-
ponents, s.

Let X = [x1,x2, ...,xm]T be the observation mixture
matrix, consisting of m observed signals xi. Since X is
the linear composition of the independent components,
s, X can be modeled as:

X = AS =

m

(cid:31)i=1

aisi

A, the mixing matrix, gives the coefficients for linear
combinations of the independent signals, the rows of S.
Here, each word is represented by a row vector of
length 864 (36× 24), which contains the 36 days worth
of hourly frequency from 22 July 2012 to 2 Septem-
ber 2012. The 50 × 864 matrix X is fed to an ICA
program [25]. The number of independent components
number is set to 5, which retains almost 100% of the
eigenvalues.

There are six words that appear in almost every inde-
pendent signal: Beijing, government, China, country, po-
liceman, and people. This means that the sensitive user
group in our user timeline has these general themes that
cut across the many individual topics that they discuss,
which may explain why their posts are often subject to
censorship.

6 Discussion

Weibo appears to have a variety of other mechanisms that
do not fit neatly into our hypotheses, but which are in-
teresting to discuss. We first consider other aspects of
Weibo’s filtering, then we look at diurnal (time-of-day)
censorship behaviors, and finally we try to synthesize
some of our observations.

6.1 Weibo’s filtering mechanisms
Sina Weibo has a complex variety of censorship mech-
anisms, including both proactive and retroactive mecha-
nisms. Here we summarize the mechanisms Weibo may
apply. Proactive mechanisms, as we discussed in Hy-
pothesis 1, may include: explicit filtering, implicit fil-
tering, and camouflaged posts. Retroactive mechanisms
for removing content that has already been released may
include:

• Backwards reposts search: In our deleted posts
dataset over 82% of reposted posts have a standard
deviation of less than 5 minutes for deletion time,
meaning that a sensitive post is detected and then
most of the other posts in a chain of reposts are then
deleted (Hypothesis 3).

• Backwards keyword search: We also observed
that Weibo sometimes removes posts retroactively
in a way that causes spikes in the deletion rate of a
particular keyword within a short amount of time.
Here, we give two examples (兲朝 and 37人), out
of many that we witnessed, with a strong spike in
the deletion of posts containing that keyword.
We first consider 兲朝, Tian Chao, a neologism
for “Celestial Empire” where 兲 is an alternate
form for 天;
the substitute character is visually
similar to the original and also appears to be
constructed from the two distinct characters 王
八 ，meaning “bastard.”). The frequency of 兲
朝 in deleted posts, day by day, is the sequence
(6,3,0,0,2,2,0,3,0,2,3,3,2,1,2,0,0,1,0,0,0,5,4,4,2,14,3,6,4)
respectively from 28 July 2012 to 25 August 2012.
There is a concentrated deletion (14 censorship
events) of posts with this word within several
minutes on 22 August 2012, impacting posts that
were several weeks old at the time. It is likely that
a censor discovered this new phrase and ordered it
globally expunged.
Another example is the keyword 37人 (37 people).
There are 44 posts containing this keyword, which
were created from 2 days to 5 days before the cen-
sorship event, all removed together within 5 minutes
(03:25 to 03:30 27 July 2012). Those 44 posts are
from different users, have no common parent posts,
and have no common pictures. The only plausible
explanation for this concentrated deletion would ap-
pear to be a keyword-based deletion. The deletion
time at 3:25am Beijing time also strongly suggests
that there are moderators working in the early morn-
ing. To understand this workforce and its distributed
nature, we perform further analysis in Section 6.2.
• Monitoring specific users: Hypothesis 2 shows a
clear preference for Weibo’s censors to pay more at-
tention to users who seemingly like to discuss cen-
sored topics.

• Account closures: Weibo also closes users’ ac-
counts. There were over 300 user accounts closed
by the system from our sensitive user group (out
of over 3,500 users) during the roughly two month
period while when we collected data for their user
timelines.

• Search filtering: To prevent users from finding sen-
sitive information on weibo.com, Weibo also has a
frequently updated list of words [4] which cannot
be searched.

• Public timeline filtering: We believe that sensitive
topics are filtered out of the public timeline. This

236  22nd USENIX Security Symposium 

USENIX Association

filtering appears to be limited to only general topics
that have been known to be sensitive for a relatively
long time. In this paper all major results are based
on the user timeline, we only use the public time-
line for general results about major trending topics
in Weibo.

• User credit points: In May 2012, Sina Weibo an-
nounced a “user credit” points system [22] through
which users can report sensitive or rumor-based
posts to the administrators. We do not know the
extent to which the point system interacts with the
censorship mechanisms that we have already de-
scribed. It is possible that these reports “bubble up”
and help Weibo tune its automated filters, but we
have no way to observe this.

6.2 Time-of-day behavior
In our data, the time at which the censors are working and
deleting posts correlates more with the usage patterns of
regular users than with a typical day-time work schedule
(e.g., 8am to 5pm Beijing time). Figure 4 shows the to-
tal hourly deletions for different kinds of posts (on a log
scale) from 20 July to 8 September 2012. Both “general
deletions” and “system deletions” happen even very late
at night.

0
0
0
0
1

0
0
0
1

0
0
1

0
1

1

l

)
e
a
c
s
 
g
o
l
(
 
s
t

n
u
o
c
 

n
o

i
t

l

e
e
D

General del. posts
Sys. del. posts
Sys. del. child posts
Sys. del. child posts(pic)
Sys. del. child posts(text)
Sys. del. regular posts
Sys. del. regular posts(pic)
Sys. del. regular posts(text)

0 1 2 3 4 5 6 7 8 9

11

13

15

17

19

21

23

Hour

Figure 4: Post deletion amounts over 24 hours.

So do the censors respond as quickly during the night
as during day hours? We plotted the median lifetime of
the posts as a function of their deletion time in Figure 5.
The morning-hour spike suggests that the censors are be-
hind in the morning, both catching up on overnight posts

and dealing with a fresh influx of posts from morning
risers. They catch up by late morning or early afternoon.

)
s
e

t

u
n
m

i

(
 

e
m

i
t

e

f
i
l
 
t
s
o
P

0
0
6

0
0
5

0
0
4

0
0
3

0
0
2

0
0
1

0

General del. posts
Sys. del. posts

0 1 2 3 4 5 6 7 8 9

11

13

15

17

19

21

23

Hour

Figure 5: Post lifetime vs. deletion time of the day.

From Figure 4 and Figure 5 it is clear that, while a
significant fraction of the censors seem to work during
regular work hours, many do not.

6.3 Synthesis
Based on everything we have seen and observed, we
can begin to understand how Weibo censorship works.
Clearly, they are using a strong degree of automation to
help them delete posts that have been declared sensitive.
It is also clear that this process is relatively “loose,” in
the sense that there are few sharp rules that define what
gets deleted vs. what is allowed to remain. Given the
long-tailed distribution that we observe in post lifetimes
prior to censorship, it is clear that some posts are not
considered a high priority for censorship, such as if two
friends start conversing with each other using a new ne-
ologism, euphemism, or other coinage that would other-
wise be censorship-worthy. However, when those new
terms spread and grow, they are censored both proac-
tively and retroactively.

This suggests that Weibo is trying to strike a balance
between satisfying the legal requirements within which
it operates and the costs of running a fine-grained in-
strument of political censorship. Weibo must conduct
just enough censorship to satisfy government regulations
without being so intrusive as to discourage users from us-
ing their service. Among other issues, they must surely
be deeply concerned with false positives. If truly innocu-

USENIX Association  

22nd USENIX Security Symposium  237

ous posts disappeared with any regularity, Weibo’s users
might defect to a competing service.

It is unclear the extent to which Weibo is using natu-
ral language processing (NLP) algorithms to aid in their
work, versus having a stable of censors watching for
things to go viral and then using search tools to stamp
them out. Certainly, our use of fairly simple NLP tech-
niques helped reduce the workload of analyzing trend-
ing topics, so comparable techniques may well be in
use by Weibo. NLP techniques in a censor’s hands can
be thought of as a “force multiplier,” but it is unclear
whether they fundamentally change the game. Consider,
with English-language spam emails, the degree to which
spammers will try to evade automated spam classifica-
tion systems. These techniques and more could well
be applied to automated or manual rewriting of post-
ings, with the intent of avoiding automated censorship.
The results might not be as easy to read, but humans
will likely have an advantage at reading jumbled text,
at least until NLP algorithms are extended to deal with
them. Conversely, NLP techniques can cluster together
related terms, assisting censors to overcome such tech-
niques. At least so far, we have not seen evidence of
any sort of arms race between increasingly sophisticated
ways to avoid censorship and increasingly powerful cen-
sorship techniques.

In many ways, Internet censorship is related to intru-
sion detection. When our results in this paper are com-
pared to related work (see Section 1), including both IP-
layer filtering and application-level censorship, a picture
of Internet censorship in China emerges where “defense-
in-depth” is taken to a new level.
Intrusion detection
research has long focused on issues such as false pos-
itive vs.
false negative tradeoffs, viral spreading pat-
terns, polymorphic content, and the distinction between
different layers of abstraction (such as IP packets vs.
application-layer data). The so-called “Great Firewall of
China” and the accompanying application-layer censor-
ship that China’s domestic web services, such as Weibo,
carry out afford us an opportunity to study a real, national
scale intrusion detection system.

6.4 Major caveats
The most important caveat to keep in mind when inter-
preting our results is that we collected posts from a very
specific core set of users, built up from a “seed” group of
users who post about sensitive topics, which we call the
“user timeline.” Unless otherwise noted, such as when
results are from the public timeline, all results in this
paper are from the user timeline and therefore might be
biased by the differences between this core set of users
and the average Weibo user. All deletion rates, dele-
tion times, etc. must be interpreted in this light. In other

words, our sample users should not be considered to be
representative of the general population of Weibo.

Another important caveat is that our system does not
detect post deletions in the user timeline if the post
deleted is not one of the 50 most recent posts by the user
(see Section 3). This may affect our results about the
distribution of post deletions over time in Section 4.

7 Conclusion

Our research found that deletions happen most heavily in
the first hour after a post has been made (see Figure 1).
Especially for original posts that are not reposts, most
deletions occur within 30 minutes, accounting for 30%
of the total deletions of such posts. Nearly 90% of the
deletions of such posts happen within the first 24 hours
of the post.

With respect to the hypotheses enumerated in Sec-

tion 4, we make the following conclusions:

• Hypothesis 1: The Weibo system keeps more than
one keyword list, where each list triggers a different
kind of censorship behavior.

• Hypothesis 2: The clear downward trend in Figure 2
could be evidence that certain users are flagged for
closer scrutiny, but we have not ruled out other
causes in this paper.

• Hypothesis 3: Figure 3 shows that over 82% of re-
posted posts have a standard deviation of less than
5 minutes deletion time, meaning that a sensitive
post is detected and then most of the other posts in
a chain of reposts are then deleted.

• Hypothesis 4: As described in Section 4, us-
ing the methods described in Section 5 we find
that topics that were trends in the user timeline
and were also, according to the public timeline,
hot topics in public discussion as a whole about
events that happened during our month of data
collection (Qidong, Qian Yunhui, Beijing Rain-
storms, Diaoyu islands, and a group sex scandal)
had very short lifetimes. Recall that the deleted
posts in the user timeline included themes related
to state power (Beijing, government, China,
country, policeman, and people). This sug-
gests that such broadly discussed topics are tar-
geted with more censorship resources to limit cer-
tain kinds of discussion about the events.

Future work may reveal many mechanisms beyond
those we described here, and many different strategies
that Weibo uses to prioritize what content to delete. Our

238  22nd USENIX Security Symposium 

USENIX Association

results suggest that Weibo employs a distributed, hetero-
geneous strategy for censorship that has a great amount
of “defense-in-depth.”

One aspect of censorship that is not considered in our
analysis, but would be an interesting topic for future
work, is the interactions between social media and tra-
ditional media. Leskovec et al. [17] gives an interesting
analysis of the interplay between blogs and traditional
media during the 2008 U.S. Presidential election. Tradi-
tional media relevant to Weibo may include the state-run
media that is heavily censored, or off-shore news out-
lets that are uncensored but limited in availability and
sometimes offset from China’s news cycles by timezone
differences.

8 Acknowledgments

We would like to thank the anonymous reviewers and our
shepherd, Nikita Borisov, for helpful feedback. We owe
our deepest gratitude to Professors Stephanie Forrest,
Christopher Bronk, and George Luger for their feedback
and comments, and for encouraging us to go forward. We
are also grateful to Ben Edwards for insightful discus-
sions about potential future work. This material is based
upon work supported by the National Science Founda-
tion under Grant Nos. #0844880, #0905177, #1017602.
Jed Crandall is also supported by the Defense Advanced
Research Projects Agency CRASH program under grant
#P-1070-113237.

References
[1] APACHE SOFTWARE FOUNDATION. Apache HBase. http://

http://hbase.apache.org/.

[2] BAMMAN, D., O’CONNOR, B., AND SMITH, N. Censorship
and deletion practices in Chinese social media. First Monday 17,
3-5 (March 2012).

[3] CAO, B.

Sina’s Weibo outlook buoys

stock
Bloomberg, 28 February 2012.

Internet

gains: China overnight.
http://www.bloomberg.com/news/2012-02-28/sina-s-weibo-
outlook-buoys-internet-stock-gains-in-n-y-china-overnight.html.
[4] CHINA DIGITAL TIMES. 新 浪 微 博 搜 索 敏 感 词 列 表.

http://chinadigitaltimes.net/space/%E6%96%B0%E6%
B5%AA%E5%BE%AE%E5%8D%9A%E6%90%9C%E7%
B4%A2%E6%95%8F%E6%84%9F%E8%AF%8D.

[5] CLAYTON, R., MURDOCH, S. J., AND WATSON, R. N. M.
Ignoring the Great Firewall of China.
In 6th Workshop on
Privacy Enhancing Technologies (Cambridge, United Kingdom,
June 2006).

[6] CRANDALL, J. R., ZINN, D., BYRD, M., BARR, E., AND
EAST, R. ConceptDoppler: a weather tracker for internet censor-
ship. In Proceedings of the 14th ACM Conference on Computer
and Communications Security (CCS 2007) (Alexandria, Virginia,
Oct. 2007), ACM, pp. 352–365.

[7] ELLEN, J. All about microtext - a working definition and a sur-
vey of current microtext research within artificial intelligence and
natural language processing. In 3rd International Conference on
Agents and Artificial Intelligence, ICAART 2011 (Jan. 2011).

[8] ESPINOZA, A. M., AND CRANDALL, J. R. Work-in-progress:
Automated named entity extraction for tracking censorship of
current events. In the Proceedings of the USENIX Workshop on
Free and Open Communications on the Internet. (FOCI 2011)
(Aug. 2011).

[9] FU, K.-W., CHAN, C.-H., AND CHAU, M. Assessing censor-
ship on microblogs in China: Discriminatory keyword analysis
and the real-name registration policy. IEEE Internet Computing
(2013), 42–50.

[10] GERARD, S., AND CHRISTOPHER, B. Term-weighting ap-
Inf. Process. Manage. 24,

proaches in automatic text retrieval.
5 (Aug. 1988), 513–523.

[11] THE GLOBAL INTERNET FREEDOM CONSORTIUM. The Great
Firewall Revealed, Dec. 2002. http://www.internetfreedom.
org/files/WhitePaper/ChinaGreatFirewallRevealed.pdf.

[12] GOH, C.-L. Unknown Word Identification for Chinese: Mor-
phological Analysis. PhD thesis, Nara Institute of Science and
Technology, 2006.

[13] HUMAN RIGHTS WATCH. Race to the Bottom: Corporate
Complicity in Chinese Internet Censorship, Aug. 2006. http:
//www.unhcr.org/refworld/docid/45cb138f2.html.

[14] HYVARINEN, A., AND OJA, E.

Independent component anal-
ysis: algorithms and applications. Neural Networks 13 (2000),
411–430.

[15] KING, G., PAN, J., AND ROBERTS, M. E. How censorhsip in
China allows government criticism but silences collective expres-
sion. American Political Science Review 107 (2013), 1–18.

[16] KNOCKEL, J., CRANDALL, J. R., , AND SAIA, J. Three re-
searchers, five conjectures: An empirical analysis of TOM-Skype
censorship and surveillance, Aug. 2011.

[17] LESKOVEC, J., BACKSTROM, L., AND KLEINBERG, J. Meme-
tracking and the dynamics of the news cycle. In Proceedings of
the 15th ACM International Conference on Knowledge Discovery
and Data Mining (KDD ’99) (Paris, France, 2009), pp. 497–506.
[18] LI, J., LIU, Z., FU, Y., AND SHE, L. Chinese hot topic extrac-
tion based on web log. In Proceedings of the 2009 International
Conference on Web Information Systems and Mining (WISM ’09)
(Nov. 2009), pp. 103–107.

[19] LUHN, H. P. The automatic creation of literature abstracts. IBM

J. Res. Dev. 2, 2 (Apr. 1958), 159–165.

[20] MACKINNON, R. China’s censorship 2.0: How companies cen-

sor bloggers. First Monday 14, 2 (Feb. 2009).

[21] MAGISTAD, M. K. How weibo is changing china. YALEGlobal
Online, Aug. 2012. http://yaleglobal.yale.edu/content/how-
weibo-changing-china.

[22] MERIWETHER, A. A user contract for chinese microbloggers.

Herdict Blog (June 2012).

[23] PARK, J. C., AND CRANDALL, J. R. Empirical study of a
national-scale distributed intrusion detection system: Backbone-
level filtering of HTML responses in China. In Proceedings of the
30th International Conference on Distributed Computing Systems
(ICDCS 2010) (Genoa, Italy, June 2010).

[24] ROGER, D., NICK, M., AND PAUL, S. Tor:

the second-
In USENIX Security Symposium (San

generation onion router.
Diego, CA, 2004).

[25] SHEN, H., H ¨UPER, K., AND SEGHOUANE, A.-K. Geometric
optimisation and FastICA algorithms. In Proceedings of the 17th
International Symposium of Mathematical Theory of Networks
and Systems (MTNS 2006) (Kyoto, Japan, 2006), pp. 1412–1418.
[26] SINA. Sina Weibo API. http://open.weibo.com/wiki/API%

E6%96%87%E6%A1%A3/en.

USENIX Association  

22nd USENIX Security Symposium  239

[27] SONG, P., SHU, A., ZHOU, A., WALLACH, D. S., AND CRAN-
DALL, J. R. A pointillism approach for natural language process-
ing of social media. In IEEE International Conference on Natural
Language Processing and Knowledge Engineering (2012).

[28] TAO. China: Journey to the heart of Internet Censorship. Re-
porters Without Borders and Chinese Human Rights Defenders,
Oct. 2007. http://www.rsf.org/IMG/pdf/Voyage au coeur
de la censure GB.pdf.

[29] VILLENEUVE, N. BREACHING TRUST: An analysis of surveil-
lance and security practices on China’s TOM-Skype platform.
Citizen Lab, Munk Centre for International Studies, Univer-
sity of Toronto, Oct. 2008. http://www.nartv.org/mirror/
breachingtrust.pdf.

[30] WANG, N. Control of Internet search engines in China – a study
on Google and Baidu. Master’s thesis, Unitec New Zealand, Aug.
2008. http://unitec.researchbank.ac.nz/bitstream/handle/
10652/1272/fulltext.pdf?sequence=1.

[31] WIKIPEDIA. Chen Guangcheng. http://en.wikipedia.org/

wiki/Chen Guangcheng.

[32] WIKIPEDIA. Deng Yujiao incident. http://en.wikipedia.org/

wiki/Deng Yujiao incident.

[33] WIKIPEDIA. Protests of Wukan. http://en.wikipedia.org/

wiki/Protests of Wukan.

[34] WIKIPEDIA. Sina Weibo. en.wikipedia.org/wiki/Sina Weibo.
[35] WIKIPEDIA. Yao Jiaxin murder case. http://en.wikipedia.org/

wiki/Yao Jiaxin murder case.

[36] WIKIPEDIA. Shifang protest, 2012. http://en.wikipedia.org/

wiki/Shifang protest.

[37] WOLFGARTEN, S.

Investigating large-scale Internet con-
tent filtering. Dublin City Univeristy, Ireland, Aug. 2006.
http://www.security-science.com/mastering-internet-
security/internet-security-ebooks-and-documents/item/
investigating-large-scale-internet-content-filtering.

[38] YANG, L. Weibo’s impact on China’s society.

Journalism
Research Paper, May 2011. http://eportfolios.ithaca.edu/
lyang1/docs/jresearch/weibo/.

[39] YE, S. Sina Weibo controls the “holy shit idea of a genera-
tion,” launches new URL Weibo.com. http://techrice.com/
2011/04/07/sina-weibo-controls-the-holy-shit-idea-of-a-
generation-launches-new-url-weibo-com/.

[40] ZHAO, X., JIN, P., AND YUE, L. Analysis of long queries in a
large scale search log. In Future Generation Communication and
Networking Symposia (FGCNS ’08) (Dec. 2008), pp. 39–42.

[41] ZHU, T., BRONK, C., AND WALLACH, D. S. An analysis of
Chinese search engine filtering. CoRR abs/1107.3794 (2011).
http://arxiv.org/abs/1107.3794.

[42] ZHU, T., PHIPPS, D., PRIDGEN, A., CRANDALL, J. R., AND
WALLACH, D. S. Tracking and quantifying censorship on a Chi-
nese microblogging site. CoRR abs/1211.6166 (2012). http:
//arxiv.org/abs/1211.6166.

[43] ZITTRAIN, J., AND EDELMAN, B.

Internet filtering in China.

IEEE Internet Computing 7 (Mar. 2003), 70–77.

240  22nd USENIX Security Symposium 

USENIX Association

|
11_2_0.pdf,|Clear and Present Data: Opaque Traffic and its Security Implications for the Future

Andrew M. White∗, Srinivas Krishnan∗, Michael Bailey†, Fabian Monrose∗, Phillip Porras‡

∗University of North Carolina at Chapel Hill
{amw,krishnan,fabian}@cs.unc.edu

†University of Michigan

‡SRI International

mibailey@eecs.umich.edu

porras@csl.sri.com

Abstract—Opaque traffic,

i.e., traffic that is compressed
or encrypted,
incurs particularly high overhead for deep
packet inspection engines and often yields little or no useful
information. Our experiments indicate that an astonishing
89% of payload-carrying TCP packets — and 86% of bytes
transmitted — are opaque, forcing us to consider the challenges
this class of traffic presents for network security, both in the
short-term and, as the proportion of opaque traffic continues to
rise, for the future. We provide a first step toward addressing
some of these challenges by introducing new techniques for
accurate real-time winnowing, or filtering, of such traffic based
on the intuition that the distribution of byte values found in
opaque traffic will differ greatly from that found in transparent
traffic. Evaluation on traffic from two campuses reveals that
our techniques are able to identify opaque data with 95%
accuracy, on average, while examining less than 16 bytes of
payload data. We implemented our most promising technique
as a preprocessor for the Snort IDS and compared the per-
formance to a stock Snort instance by running both instances
live, on identical traffic streams, using a Data Acquisition and
Generation (DAG) card deployed within a campus network.
Winnowing enabled Snort to handle a peak load of 1.2Gbps,
with zero percent packet loss, and process almost one hundred
billion packets over 24 hours — a 147% increase over the
number processed by the stock Snort instance. This increase
in capacity resulted in 33,000 additional alerts which would
otherwise have been missed.

I. INTRODUCTION

The successful monitoring of security policies via Intru-
sion Detection Systems (IDS) critically depends on scalable
and accurate techniques for inspecting traffic. Deep packet
inspection (DPI) is a common method for performing such
inspection, especially when security policies require deter-
minations based on information not accurately reflected by
network ports, protocols, or hosts. In fact, the market for
DPI appliances alone reached almost $1 billion in 2009, and
continues to grow.1 Since DPI must deal with huge volumes
and significant heterogeneity of traffic, DPI designers often
trade off accuracy of detection with resource demands [34].
DPI systems cannot generally derive useful information from
opaque (i.e., encrypted or compressed) packets; thus, we
propose improving the performance versus quality curve
through the quick and accurate winnowing, i.e., filtering,
of opaque traffic, and evaluate a number of techniques for
doing so. Such techniques can improve the performance

of DPI engines by quickly and accurately separating out
low-value packets from the data stream they inspect; this
is particularly important in high-performance environments,
where the sheer volume of traffic can be staggering. At
our campus, encrypted traffic alone appears to make up
an average of 13% of the total
traffic, based solely on
port and protocol assumptions. We believe that the fraction
of encrypted traffic on real networks will only continue
to rise, as more web services follow in the footsteps of
Google and Facebook in offering encrypted connections and
the movement toward ubiquitous end-to-end encryption [2]
gains steam. In particular, private corporate networks, where
secure communications are often required, undoubtedly see
significantly higher proportions of encrypted traffic. The
class of opaque traffic encompasses not only encrypted
connections, but also compressed entities, which,
in the
modern era of streaming video, comprise an even more
significant fraction of network traffic: some sources indicate
that streaming video accounts for 35% of peak network
usage, a proportion which is only increasing.2

In fact, our experiments revealed a surprising preponder-
ance of opaque traffic: in a 24-hour weekday period, nearly
90% of TCP packets traversing a major university network
(with peak loads of 1.2Gbps) were found to be opaque. This
truly staggering figure suggests a broader issue for the secu-
rity community moving forward: as more and more payloads
become opaque to DPI engines, how can we detect and pre-
vent obscured threats? While content-based signatures will
continue to be relevant as a detection mechanism for direct
attacks (e.g., those exploiting buffer overflows in networking
routines) against networked systems, bypassing current DPI
engines can be as simple as encrypting the relevant exploit
code, particularly for indirect attack vectors (e.g., exploits
embedded in documents). Thus we, as a community, face
both an immediate need to separate the wheat from the
chaff—to winnow low-value, i.e., opaque, packets to enable
our existing detection methods to operate in high-speed
environments—and a long-term need to develop methods for
coping with attacks embedded in opaque, and particularly
encrypted, traffic. This work represents a first step toward
solving both problems: our techniques enable the fast and
accurate identification of opaque packets, either as chaff to

1Gartner’s Magic Quadrant for NIPS (Dec 2010).

2Sandvine’s Global Internet Phenomena Report (Fall 2011).

be discarded, or as the inputs to specialized detection engines
targeting opaque traffic.

Unfortunately, the identification of opaque network traffic
is very challenging. While signatures can identify many
known opaque protocols (e.g., SSL/TLS, SSH), some pro-
tocols (e.g., Bittorrent’s Message Stream Encryption) are
specifically designed to avoid signature matching. In addi-
tion, signature-based approaches for identifying new opaque
protocols require constructing and deploying new signatures
for each new protocol. More importantly, existing techniques
for identifying opaque data often require examination of
a large number of bytes, which can be computationally
and memory intensive on high-speed networks [5, 12].
Similarly, inspecting application-layer content-types to de-
termine opacity requires resource-intensive flow reassembly.
To compound matters, such detectors cannot rely on HTTP
content-types as they are often inaccurate (see §III).

As a final note, opaque traffic identification has value that
extends beyond filtering to policy monitoring and enforce-
ment. For instance, operators may wish to monitor and en-
force policies within their network (e.g., flagging encrypted
packets tunneled over unencrypted connections, an odd
practice used by botmasters to hide command-and-control

We take a first step toward addressing these challenges
by providing novel methods for quickly and accurately
determining whether a packet is opaque. Our techniques are
both port- and protocol-agnostic, allowing for the detection
of opaque traffic regardless of the port(s) or protocol(s)
involved. We concentrate on efficient techniques which can
operate on limited portions of packet payload, such as
small-sample fixed-size and sequential hypothesis tests, in
order to minimize overhead and allow scarce computational
resources to be reserved for high-value analyses.

In the present work, we necessarily evaluate our tech-
niques in isolation, i.e., without attempting to fully integrate
our approach with a particular system or methodology,
in order to minimize the number of factors potentially
biasing our results. That said, we envision the identification
of opaque traffic not as the relatively standalone system
portrayed herein — there is no “silver bullet” for network
intrusion detection — but rather as an efficient new tool
in the network analyst’s toolbox. In particular, forensic
recording systems, such as Time Machine [23], often avoid
archiving full connections in order to reserve limited storage
space for the higher-value packets at the beginning of a
connection. An admitted weakness of this design decision
is that attacks in the discarded portions of these connections
go unnoticed. Our techniques mitigate this concern to an
extent by providing a means for such a system to detect po-
tentially high-value plaintext packets in lengthy connections.
Similarly, our tests are simple and can be implemented on an
FPGA, providing a benefit even to systems which make use
of a hardware component, such as those following shunting
approach proposed by Gonzalez et al. [15].

messages in otherwise mundane HTTP traffic [6, 38]).
Similarly, one might wish to flag unencrypted traffic where
encrypted traffic is expected, such as might occur if an SSH
connection is subverted [27], or on institutional networks
where end-to-end encryption is required. Identifying opaque
traffic may also help to discover applications tunneled over
other protocols (e.g., video traffic tunneled over HTTP), or to
provide sanity checking for strongly typed networking [26].
Our contributions include: 1) the concept of opaque traffic
as a distinguishable class of network traffic; 2) the devel-
opment, evaluation and comparison of multiple techniques
for detecting such traffic (Section §II, Appendix §A); 3) an
operational analysis of modern network traffic with respect
to opacity (Section §III-A); and 4) an evaluation, at scale,
of the potential of our techniques for reducing the load on
modern intrusion detection systems (Section §III-B).

II. APPROACH

An important requirement of ours is to minimize the
amount of a packet’s payload that we must inspect, as doing
otherwise is both computationally and memory intensive
on high-speed networks [5, 12]. These overheads severely
restrict the numbers of samples available to us for any of
the tests we explore. Therefore, for the remaining discussion,
we propose detectors based on small-sample fixed-size hy-
pothesis tests and sequential hypothesis testing. The latter
allows us to make decisions quickly by examining only as
many samples as needed to support a given hypothesis.

i.e.,

Our detectors are based on determining whether the bytes
examined are drawn from a uniform distribution or some
other, unknown, distribution. Our first instinct, when faced
with the problem of measuring the uniformity of a set of
samples, was to use entropy-based measures. However, as
we show later, accurate entropy testing requires significantly
more samples than is practical in our setting, and is less
efficient than more direct methods based on the samples
themselves rather than on a derived statistic such as entropy.
Both encrypted and compressed traffic will exhibit high
entropy,
the distribution of bytes will be close to
uniform. In this paper, we consider these two cases as
belonging to the same equivalence class of “opaque” objects
as far as DPI engines are concerned. That is, regardless of
whether the packets that belong to a session are compressed
versus encrypted, they will be forced to go through the slow
path wherein the engine must analyze all packets of these
sessions, but will still fail to derive any useful information
in the vast majority of cases. Hence, from the perspective of
DPI engines, there is little value in attempting to analyze
these packets. As Cascarano et al. [5] observed in their
experimental evaluations, these slow paths incurred CPU
overheads that can be several orders of magnitude higher
than the average case for transparent traffic.

In our search for the best performing test for our goals,
we examine several fixed sample-size hypothesis tests. There

are a number of standard techniques for testing uniformity;
however, many of these are designed for testing the unifor-
mity of the outputs of a pseudo-random number generator,
and thus assume access to millions of samples (e.g., [25]).
These tests are unsuitable in our context because our sample
size is extremely limited. We instead evaluate the appropriate
fixed-size tests (the likelihood ratio test and the discrete
Kolmogorov-Smirnov test) and two variants of the sequential
probability ratio test. We assess the effectiveness of each
test in two scenarios, differentiated by the domain of the
underlying distribution. In the first scenario, we directly test
the distribution of byte values in the packet; for the second,
we instead test the distribution of entropy values over n-byte
“blocks” in the packet.

When the need arises for a probability density function for
the alternative hypothesis, we use a simple distribution based
on the intuition that plaintext packets will have a higher
frequency of bytes whose values are less than 128 (as are the
ASCII values). We parameterize this distribution by setting
δ to the cumulative density of those values. For example, at
δ = 0.75 the alternative hypothesis is that 75% of the bytes
in the packet have values less than 128.

Likelihood Ratio Test: A well-known theorem of hypoth-
esis testing is the Neyman-Pearson lemma, which states that
the most powerful test, i.e., that with the lowest expected
false positive rate for a given false negative rate, of one

For pedagogical reasons, we leave descriptions of the
less successful tests, along with the details of the parameter
exploration experiment itself, to §A2, and discuss only the
more effective tests here. In summary, the closely-related
likelihood ratio and sequential probability ratio tests, operat-
ing directly on the byte values instead of on derived entropy
values, consistently outperform the other tests. The poor
performance of the entropy tests is related to the birthday
problem, in that the (byte-)entropy of any n bytes is the same
unless there is a collision (e.g., there are two bytes which
share the same value). According to the birthday paradox,
the a priori probability of a collision in 8 bytes, even when
choosing only from only the 95 printable ASCII characters,
is only about 26%. This means that the entropy of 8 ASCII
bytes is indistinguishable from that of 8 unique bytes 74%
of the time. Thus entropy-based tests require substantially
more than the number of samples available in our context to
be effective. However, our parameter exploration experiment
(see §A2) reveals that the the more successful tests require
examining only 16 bytes of payload to be effective.

Since our goal is to discard opaque traffic (albeit en-
crypted or compressed), we let our null hypothesis be that
the packet is opaque and our general alternative be that
the packet is transparent. Specifically, the null hypothesis is
that the values of the bytes in the packet are approximately
uniformly distributed (e.g.,
is compressed or
encrypted); the alternative is that the distribution is non-
uniform (e.g., ASCII text).

the packet

Suppose we wish to test

simple hypothesis against another is the likelihood ratio
test [41]. For a single sample, the likelihood ratio statistic
is simply the ratio of the likelihood of the sample under the
alternative hypothesis to the likelihood of the sample under
the null hypothesis. For multiple samples, these likelihoods
are replaced with the corresponding joint likelihoods.
the simple null hypothesis
H0 : θ = θ0 against the simple alternative H1 : θ = θ1 given
a random sample ¯X. Then the Neyman-Pearson lemma [42,
Theorem 10.1] states that the most powerful test of H0
against H1 is that where one rejects H0 if Λ( ¯X) ≥ q and
accepts H0 if Λ( ¯X) < q, where q is determined by the
desired level of statistical significance and Λ( ¯X) is the ratio
of the likelihood of ¯X under the alternative to the likelihood
of ¯X under the null hypothesis.

Sequential Probability Ratio Test: Sequential analysis is
often used when there is a desire, such as in our context,
to limit the number of samples examined. In fact, Wald
and Wolfowitz have shown that, among all tests of two
simple hypotheses with fixed error probabilities, the sequen-
tial probability ratio test (SPRT) minimizes the expected
number of samples required for a decision under either
hypothesis [43]. Sequential
testing works by examining
samples one-by-one, rather than all at once, and evaluating
a decision function at each sample. This allows the test to
stop examining samples as soon as it has found enough
“evidence” for one hypothesis or the other.
Specifically, let α = P (accept H1 H0) be the proba-
bility of a false negative, that is, an erroneous prediction
that the vi are not uniformly distributed; similarly, define
β = P (accept H0 H1) as the probability of a false positive.
In order to perform the test, we iterate through some
sequence of samples X1, X2, ...; according to Wald’s theory
of sequential hypothesis testing [42], we choose at each
iteration m one of three actions: accept H0, accept H1,
or continue, as follows:

accept H0 if
accept H1 if
continue

Λm(X1, X2, ..., Xm) ≤ g0(m)
Λm(X1, X2, ..., Xm) ≥ g1(m)
otherwise.

Setting g0(m) = β
probabilities of false positives and false negatives.

1−α and g1(m) = 1−β

α gives the desired

A known drawback of this test is that it may not terminate
within an acceptable number of samples. We alleviate this
concern by exploring two variants, which we refer to as the
truncated and restricted SPRTs. For the truncated SPRT,
we specify a maximum number of samples which the test is
allowed to examine; if this limit is reached without making
a decision, then the fixed-size likelihood ratio test is applied
to the samples examined so far. The restricted SPRT, on
the other hand, works by sloping the decision boundaries
such that they intercept the axis after the given number of
samples. For the restricted case, we follow the approach

suggested by Bussgang and Marcus [4].
Based on the results of the parameter exploration exper-
iment (see §A2), we use the truncated sequential method
(α = 0.005, β = 0.005, δ = 0.85) for the remainder of the
experiments in this work.

III. EVALUATION

In order to ascertain the effectiveness of our techniques
in different scenarios, we perform a number of experiments
in both offline and online settings. First we show, in a
controlled, offline experiment, that our techniques are able
to accurately identify the opacity of different file types. We
then verify the accuracy of our techniques under real-world
network conditions by evaluating our detectors on traffic logs
and traces collected at two major universities, an analysis
which produced a number of interesting anomalies which we
investigated with the help of a network operator. Finally, we
show the utility of winnowing by comparing two otherwise
identical Snort IDS deployments under identical traffic loads.

A. Offline Analysis

File Type Identification: To gain an understanding of how
well our techniques were able to label the opacity of various
common data types, we collected a set of files with known
ground truth, including compressed archives and streams,
encrypted files, executable binaries, and text files. We also
attempted to cover different sub-types; e.g., we included five
different text file encodings. The details of this set, which
we believe to be a reasonable cross-section of common file
types, are presented in Table I.

We gathered a base set of files from multiple sources, then
applied standard compression and encryption algorithms to
these files to create the remainder of the dataset. For exe-
cutables, we used ELF binaries from the bin directories of
a basic Ubuntu Server 10.04.3 installation (that we used for
testing Bro and Snort). The files in each directory, including
a number of Perl scripts, were then individually compressed
using tar (which in turn uses gzip and bzip2) to provide
gzip and bz2 files and encrypted using openssl to
provide examples of the RC4 and AES ciphers. The text
files are the contents of the man path directory of the
same Ubuntu installation, uncompressed and processed by
groff into different encodings (ASCII, UTF-8, -16, and -
32, and latin1); the PDF files are the archive of proceedings
from a major computer security conference. Finally, the
images were JPEGs scraped from public websites, including
a departmental website and nasa.gov.

To simulate a network environment, we transmitted each
object over an individual TCP connection on the loopback
device of an Ubuntu virtual machine. We used nc to both
send and receive, and tcpdump to collect the resulting
packets. We report the proportion of opaque and transparent
packets observed for each type of file in Table I.

Our test labeled more than 95% of compressed, encrypted
and image file packets as opaque, as expected. Similarly,
even with multiple different encodings, our techniques la-
beled more than 99.99% of text file packets correctly as
transparent. Our test is less consistent on the executables.
Inspection of the binaries revealed a large number of null
bytes, suggesting that a more targeted alternative hypothesis,
perhaps counting only the set of printable ASCII bytes
rather than simply those with value less than 128, may
improve our results. However, this must be contrasted with
the straightforwardness and efficiency of checking whether
the value of a byte is greater than 128 (which amounts to
simply checking the high-order bit).

Type
compressed
encrypted
text
images
executable
pdf

Objects
1410
1410
5802
205
498
1006

Size (MB)

40
91
176
12
43
75

True Positive Rate

97.8
98.9
100.0
94.4
34.5
13.5

Table I

FILE TYPE ANALYSIS

Content Type Matching: To assess the accuracy and per-
formance of the techniques in §II, we instrumented the Bro
IDS (version 1.6-dev-1491) to log packet-level information
(with only coarse-grained payload information), providing
a form of ground truth. Due to privacy restraints, we were
unable to record full packet traces at scale; therefore, the
experiments in this section are performed on logs generated
by our instrumented version of Bro. For these experiments,
we collected two logs (log1 and log2) from two large
university campuses.3 Both logs were collected over several
hours during a weekday. For simplicity, we only consider
IPv4 TCP traffic. We labeled each packet by protocol using
Bro’s dynamic protocol detection [13], and restricted our
analysis to two encrypted protocols (SSL and SSH) and two
unencrypted protocols (HTTP and SMTP).

For each packet, we log the source and destination ports,
the HTTP message type, the HTTP message encoding, and
the payload length; we also store coarse-grained statistics
in the form of a binary value for each byte of the payload
(indicating whether the byte’s value is less than 128), and the
byte-value frequencies for each n-byte block of the payload.
The latter are needed to calculate sample entropy at the block
level and for the frequency-based tests (i.e., χ2 and discrete
K-S; see §A). In all cases, we only log information for the
first 256 bytes of the payload.

We labeled each packet as ‘opaque’ or ‘transparent’
according to the expected data type for that packet: SSL
and SSH packets are labeled opaque and SMTP packets are
labeled transparent. For HTTP, we labeled packets based

3Our efforts to maintain data privacy and ensure IRB compliance for this

collection effort are detailed in §VII.

on the HTTP Content-Type and Content-Encoding header
fields (as given by the sender) for the corresponding HTTP
message (see §B for details of the labeling). This allows
us to further restrict our attention to only those content-
types for which opacity should be predictable and consistent.
For instance, the HTTP 1.1 specification states that “HTTP
entities with no specified Content-Type should be identified
by inspection of the contents” [14]; we remove any such
packets from our analysis because we have no way of
determining ground truth. However, as we discovered during
the course of this work, the HTTP content-type headers are
often inaccurate and misleading (details are given in the
following sections).

Unfortunately, Bro suffers from performance problems
(see [12]) on high-speed networks, especially when port-
based analysis is disabled (as is necessary to force Bro to
determine protocols solely by inspection). Therefore, we dis-
card any HTTP packets which belong to flows in which Bro
experienced losses. We do so because the dropped packet(s)
could have contained essential labeling information (e.g., a
message header) necessary for determining the content-type
and encoding.

After filtering the logs down to only those packets which
met the criteria outlined above, over 39 million packets
(across ≈ 3.8 million bi-directional flows) remained from
log1 and over 24 million (across ≈ 2.3 million bi-
directional flows) remained from log2. The traffic makeup
for both logs is shown in Figure 1.

The content-type distribution (Figure 2) for the top content
types in both logs reveals some interesting contrasts between
the two. In particular, video types are prevalent in log2
while log1 consists mainly of web-browsing traffic (over
80% of log1 HTTP packets have content type ‘image/jpeg’
or ‘text/html’, compared to 40% in log2). The content-
encoding field is only specified for a small proportion of
packets in our logs, and the only significant encoding is
‘gzip,’ at 4.0% in log1 and 6.5% in log2. All other
encodings combined account for less than a tenth of a
percent of packets in each log.

We performed a large-scale analysis on both log1 and
log2; the overall results are given in Table II. Examining
only 16 bytes per packet, offset 8 bytes from the start of
each packet, we achieve a match rate, i.e., the percentage
of examples for which our techniques produced the same
label as expected from the content type, of 95.1% on log1
and 96.0% on log2. We refer to ‘match’ rates here, rather
than false-positive or false-negative rates, due to the large
quantity of mislabeled content-types we encountered.

In the case of encrypted traffic (i.e., TLS/SSL) we accu-
rately classified approximately 95% of the traffic. However,
the mismatches are particularly interesting. Figure 3 shows
the distribution of packet IDs, where a packet’s ID is its
position in the bi-directional flow (i.e., a packet with ID zero
is the first packet sent by the originator of the connection).

(a) log1

(b) log2

Figure 1. CDFs of payload size for the protocols examined in the two
campus network logs

log1

log2

Protocol
SSH
SSL/TLS
SMTP
HTTP
Total

Match Rate

94%
96%
86%
91%
94%

Examples Match Rate

7.3m
16.4m
3.35m
9.7m
36.7m

97%
94%
80%
85%
96%

Examples
157k
5.5m
1.4m
13.8m
20.8m

EXPERIMENTAL RESULTS (log1 AND log2)

Table II

Notice that 94.8% of the mismatches for SSL/TLS occur
within the first 5 packets, and 95% within the first 6 packets
for SSH. These packets are all in the connection set-up
phase and hence are not, in fact, encrypted. Moreover, these
connection set-up packets, particularly any SSL certificates
(typically around ID 2), may be of interest to DPI engines
even when the encrypted payload is not.

A closer analysis of our overall results reveals that 50% of
the transparent-as-opaque mismatches for log1, and 15%
for log2, are from SMTP traffic, which we surmise includes
opaque attachments for which we do not have accurate

0200400600800100012001400PayloadSize0.00.51.01.52.02.5NumberofPackets×107OverallSMTPSSHSSLHTTP0.00.20.40.60.81.0PercentageofPackets0200400600800100012001400PayloadSize0.00.20.40.60.81.01.21.41.6NumberofPackets×107OverallSMTPSSHSSLHTTP0.00.20.40.60.81.0PercentageofPacketsFigure 2. HTTP Content-Type Distribution

(a) SSL/TLS

(b) SSH

Figure 3. CDFs of Packet IDs

labeling information. Of the HTTP transparent-as-opaque
mismatches (Figure 4), many are PDF and Flash (both of
which can serve as a container for many other object types),
but a surprising proportion are of type text/plain.

We investigated the text/plain mismatches from
log1 further and concluded, based on the observed byte-
value distributions, that if these packets are in fact plaintext,
the method used for encoding is non-standard. Figure 5(a)
depicts the byte-value distribution for the mismatches and
for the text/plain packets where the encoding was
specified explicitly as one of UTF-8, UTF-16, US-ASCII

Figure 4. HTTP Mismatched Content-Types

or ISO-8859-1. As expected,
the latter distribution has
significant mass around the ASCII characters ‘A’-‘z’ (values
65 − 122 in Figure 5) while the former does not. A simi-
lar finding holds for the opaque-as-transparent mismatches
in the image/jpeg case: the distribution (bottom, Fig-
ure 5(b)) has striking similarities to that observed for the
case of known plaintext encodings (top, Figure 5(a)), and
moreover, is quite different from that of the opaque JPEGs
(top, Figure 5(b)). Unfortunately, without access to actual
payloads for these two traces, we cannot say what was the
underlying cause for the peculiar distribution in the content
flagged by our methods, but we believe this underscores the
utility of our techniques — i.e., we successfully identified
anomalous instances in both cases. We revisit this issue in
the next section.

Finally, we examined the number of iterations needed by
the truncated test to make a decision regarding each packet.
With the maximum sample size set at 16 bytes, 45% of the
packets can be classified in 12 bytes or less.

Operator Analysis: To gain deeper insights into the
issue of Content-Type mismatches, we performed another
experiment in which a resident network operator was able to
manually inspect payload data from a third dataset, for which
we were able to collect full payloads under the supervision of
our local network operator. This trace covers four weekday
afternoon hours of traffic from a university computer science
department, and consists of 27 million packets. To enable
this inspection, we instrumented Bro (version 2.0) to save
both HTTP entities and the payloads of TCP connections to
disk. We then ran the instrumented Bro against the port 80
(HTTP) and port 443 (SSL) traffic in the trace.

We determined the opacity of each packet, saving those
entities and connection payloads wherein the first five pack-
ets were mismatches. For the HTTP entities, we define a
mismatch as having an opacity different from that implied by
the Content-Type or Content-Encoding. For the streams on
port 443, we consider transparent packets to be mismatches.
We also captured relevant metadata, such as the URI, Host,
and MIME-type of the resulting file (determined using Bro’s
libmagic interface).

We discovered a number of mismatched HTTP entities
in the trace, of both the transparent-as-opaque and opaque-

0.00.10.20.30.4PercentageofHTTPPacketstext/htmlimage/jpegapp/pdfvideo/x-flvtext/plainimage/pngvideo/mp4image/gifapp/zipaudio/mp4trace1trace20246810PacketID0.00.20.40.60.81.0PercentageofPacketsAllFalseNegatives0246810PacketID0.00.20.40.60.81.0PercentageofPacketsAllFalseNegatives0.000.010.020.030.040.050.060.07PercentageofHTTPPacketsapp/pdftext/plainimage/jpegapp/x-shk-flashtext/htmltrace1trace2comprised: cleartext Yahoo Voice connection information
(including account names), names and email addresses from
a social networking chat site, and what appeared to be IM
conversations. As an aside, in looking at flagged mismatches
while testing, one of the authors discovered his AIM contact
list transmitted in the clear over port 443 by Adium!
B. Online Analysis

To further demonstrate the utility of winnowing opaque
traffic in a real-world environment, we implemented our
techniques as a preprocessor to the Snort IDS. Much of
Snort’s functionality, such as stream reassembly and HTTP
inspection, is implemented as preprocessors which are exe-
cuted after network and transport layer packet decoding but
before engaging the rule-matching engine. We positioned our
winnowing preprocessor to intercept packets before reaching
the stream reassembly module.4 This allows us to drop
opaque packets early enough to avoid incurring the overhead
(over 30% of Snort’s run-time in some experiments) of
stream reassembly and long before reaching the pattern
matching engine.

The ruleset used was provided by a major university,
which uses Snort primarily for post-facto forensics, and
contains 1606 rules. As a sanity check, and to provide
results on a public dataset which could therefore be easily
reproduced, we evaluated both stock Snort and Snort with
our winnowing preprocessor on the (admittedly controver-
sial) DARPA/MITLL 1999 intrusion detection dataset. Both
configurations produced exactly the same set of alerts.

For our online experiments, we made use of an Endace
9.2X2 Data Acquisition and Generation (DAG) card to run
two Snort (version 2.9.1.2) instances in parallel, one with
stock Snort and the other with winnowing enabled, on
live traffic. The DAG uses a packet processor to capture
and timestamp packets at line rate, and employs an on-
board direct memory access (DMA) engine to zero copy
packets into the host’s memory. The DMA engine can also
be programmed to duplicate traffic over multiple memory
buffers (called “streams”), which enables us to simultane-
ously run multiple experiments, thereby comparing different
approaches on the same traffic. We use a 10Gbps DAG
connected to a 2.53 Ghz Intel Xeon 6 core host with 16
GB of memory. As in our earlier experiments, we only
examined traffic on ports 22, 25, 80 and 443; this filtering
was performed on the DAG and therefore CPU resources
are used exclusively for payload inspection. Beyond the
winnowing preprocessor, there were no differences between
the two configurations: both used the same ruleset with the
default Snort options (including inspection of gzipped HTTP
entities and the bypassing of the detection algorithms by
SSL application data packets in established streams), and
each was allocated 2GB of memory on the DAG.

4Our preprocessor only drops packets with TCP payloads and therefore

does not interfere with connection tracking at the transport layer.

(a) text/plain (top: known encodings (transparent), bottom: opaque)

(b) image/jpeg (log-scaled; top: opaque, bottom: transparent)

Figure 5. Byte-value Distributions for Anomalies

as-transparent varieties. In the former case, many of these
mismatches were HTTP entities labeled with Content-Type
‘text/plain’ and no stated encoding; we determined from
the metadata that most of these were either images or
compressed bundles. These compressed bundles included
what appear to be updates to Symantec antivirus, extensions
to the Plex media center, and an installer for the DivX
video codec (a .cab file). The images were identified by
MIME-type as JPEGs. One interesting mismatch declared a
Content-Type of ‘text/javascript’, with no encoding, while
the MIME-type reported was ‘application/octet-stream’ and
the filename .gz.

Of the opaque-as-transparent mismatches, a number of
entities were labeled as JPEG and GIF images by Content-
and MIME- type but were flagged as transparent by our
techniques. Many of these contain significant text, which we
speculate may be metadata. We also found a large number
of streams on port 443 with predominantly transparent
packets. According to our network operator, these streams

050100150200250Value12345Count×1050.010.020.030.04Percentage12345Count×1050.010.020.030.04Percentage050100150200250Value104105106Count10−210−310−410−510−6Percentage104105106Count10−210−310−410−510−6Percentagerunning on a dedicated core. However, due to traffic bursts,
the stock Snort instances still dropped significant numbers of
packets despite our attempts to optimize its handling of the
traffic. While this underscores the need for techniques like
ours, we remind the reader that the advantages of winnowing
traffic are not limited to situations of overload: the significant
throughput increase allows systems to evaluate more rules
than stock configurations under the same traffic conditions.

(a) Packets

(b) Alerts

Figure 6. Number of alerts and number of packets processed by Snort
with winnowing and Snort without (in 15-minute windows).

Our primary experiment lasted for 24 hours and encom-
passed more than 7.6 terabytes of traffic; the load reached
1.2Gbps at peak. Figure 6(a) shows the number of packets
which each instance of Snort was able to process in 15-
minute intervals. Even at peak load, our winnowing Snort
instance is able to handle the full volume with zero percent
packet loss. In contrast, the stock Snort instance dropped
nearly 60% of the 98.9 billion packets observed during
the 24-hour window. In fact, the winnowing Snort instance
processed 147% more packets, resulting in over 33,000
additional alerts (see Figure 6(b)). We note that as the drops
suffered by the unmodified Snort are lost when Snort is
unable to copy packets out of the network buffer fast enough
to avoid old packets being overwritten by the kernel (or the
DAG, in this case), and therefore Snort has no control over
which packets are dropped [31].

In a second experiment, we partitioned one of the two
identical
the DAG into four disjoint
streams, each of which was passed to a stock Snort instance

traffic streams at

Figure 7. CDF of payload size for both opaque and transparent traffic.

We stress that, apart from the winnowing preprocessor,
instances in each experiment were identically
the Snort
configured and saw exactly the same traffic. The substantial
improvement in capacity arises due to the surprisingly high
proportion of opaque traffic, which, as discussed in §II,
incurs high CPU overheads (confirmed in our experiments
by an observed 30% increase in average per-packet pro-
cessing time compared to transparent packets). In total, an
astonishing 89% of the payload-carrying TCP packets ob-
served in our primary experiment were classified as opaque,
representing 86% of the bytes transmitted. We hypothesize
that this is due to the prevalence of streaming video services
such as Hulu, Netflix, and YouTube operating on port 80.
Differences in the packet size distributions (see Figure 7)
indicate that MTU-sized packets are far more often opaque
than not, which suggests that large, compressed streams
(e.g., the aforementioned streaming video), are prevalent.
These streams may also account for the bursty nature of
traffic observed in the multiple core experiment.

Operational Impact: Since winnowing opaque packets
fundamentally changes the mix of traffic which reaches the
rules-matching engine in an IDS, its influence could lead
to deeper insights about network activities. To evaluate this
further, we compared the alerts generated by both instances
of Snort. In total, the stock Snort instance generated 25,081
alerts, while the Snort instance augmented with winnowing
produced 58,297 alerts—a 118% increase. Of these, the
three most prevalent in both cases remained the same; two

20:000:004:008:0012:0016:0020:00Time0.00.20.40.60.81.01.21.41.61.8AnalyzedPackets×108stocksnortwinnowing20:000:004:008:0012:0016:0020:00Time020040060080010001200Alertsstocksnortwinnowing0200400600800100012001400PayloadSize1071081091010101110121013NumberofPackets(log-scale)opaquetransparentbothinstance, we argue that

overflow attack detections and a file-type identification alert
(for portable executable binaries). We found the differences
between the distributions of rules triggered by the stock
Snort and by Snort with winnowing to be particularly in-
teresting. One PowerPoint administrator privilege escalation
attack rule was triggered 2000% more times by the winnow-
ing Snort instance, an almost 10-fold increase over what one
might expect just from the increased traffic volume. We also
found a 760% increase in alerts for PDFs with embedded
Javascript. While ascertaining whether winnowing induces
any false positives is impossible due to the losses sustained
by the stock Snort
intelligently
dropping packets, as we do, is no more likely to induce false
positives than dropping random packets due to overload.
The only class of rules which produced fewer alerts when
winnowing opaque traffic were for HTTP connections to
blacklisted domains; these are the only clear false negatives.
Since HTTP headers are in plaintext, and hence not dropped
by Winnow, we suspect that the missed alerts are due to
Snort failing to recover from missing (opaque) packets in a
stream. Since parsing of HTTP headers is possible even in
the presence of missing payload packets, we believe such
alerts would be triggered if the IDS was better equipped to
recover from such midstream losses. Alternatively, a more
mature implementation of the winnowing preprocessor could
inform Snort of the dropped packets.

Nevertheless, the instantiation of our prototype on a major
campus network has already provided valuable information
to its network operators, clearly showing that their network
security exposure differs from what they originally thought.

IV. LIMITATIONS

While our focus on minimizing payload inspection sug-
gests a trivial method for an attacker to bypass DPI engines
using our technique (by padding the beginning of the packet
with ASCII bytes),
in current systems an attacker need
do nothing more than “encrypt” their exploit code (e.g., a
simple XOR may suffice) to bypass content-based signature
matching engines. Furthermore, our techniques comprise a
method for flagging the likely presence of such encrypted
objects based on nothing more than the ciphertext itself.

that

At first blush, it may appear that binary protocols present
an insurmountable problem for our methodology. However,
we point out
the majority of traffic (i.e., 80% on
our network) is HTTP, a text protocol. Furthermore, the
high-volume binary protocols (SSL, RTMP, and Bittorrent)
transport primarily,
if not exclusively, opaque data. We
believe that a different alternative hypothesis, such as that
mentioned earlier in the context of identifying file types,
could provide improved accuracy for binary protocols.

Additionally, while it may seem that the flow level, as
opposed to the packet-level, is the natural vantage point for
identifying opaque traffic, our work concentrates on packet-
level analysis. We argue that the presence of tunneled traffic,

and container formats such as PDF, mandates identification
of opaque traffic on a per-packet basis. In the specific case of
encrypted connections, many protocols (e.g., SSH and SSL)
begin with unencrypted, connection set-up packets; some
protocols (e.g., STARTTLS) are even designed specifically
to enable upgrading a connection from unencrypted to
encrypted mid-stream. Furthermore, packet-level techniques
can be used in situations where flow state is not kept, such as
on DAG cards. Finally, by performing packet-level analysis,
we can winnow opaque packets before they reach the stream
reassembly engine, significantly reducing overhead. That
said, there are benefits to incorporating flow-level analysis.
One interesting direction might be to limit packet-level
misclassifications by utilizing information from prior and
subsequent packets.

We remind the reader that our techniques are not intended
to be used in isolation, but rather as components in larger
systems. Therefore, the limitations of our techniques can
be mitigated by the strengths of the remainder of the system
architecture, such as by coordinating flow-level analysis with
packet-level opacity checking. This would provide a layered
defense against, e.g., attacks embedded in lengthy streams to
evade systems using selective packet discarding [31], which
is discussed in the next section, or approaches similar to the
afore-mentioned Time Machine (see Section I).

V. RELATED WORK

The related problem of forensic file and data type identi-
fication has been extensively explored over the past decade.
Many of these efforts have focused on analyses of byte
frequency distributions, i.e., the frequency of each byte value
in the data of interest. For the most part, these approaches
work by creating signatures for known file types (i.e., HTML
or PDF) based on byte frequency distributions, and then
comparing unknown files to pre-existing signatures to deter-
mine the file type (see, e.g., Ahmed et al. [1]). Veenman
[40] and Hall [17] examined entropy and compressibility
measurements as indicators of file type, while others have
explored techniques for directly modeling the structure of
files [16, 18, 21, 32]. The closest work to the problem at
hand is that of Conti et al. [8], who consider distinguishing
between random, encrypted and compressed files using k-
nearest-neighbor classification. Shannon entropy and the χ2
statistic, among other measures, are evaluated as distance
metrics over sliding windows of 1KB or more. Similar ideas
were explored by Shamir and Someren [33] for identifying
cryptographic keys on disk. However, as our empirical
analyses showed, the forensics scenario is fundamentally
distinct from the setting we explore in this work: for real-
time analysis on the network, the amount of data available
is constrained and computational resources are scarce.

Approaches based on byte-frequency [44, 46] and en-
tropy [22, 35, 36, 45] have also been applied in the context
of malware identification and analysis, both on disk and in

network traffic. These approaches are not well suited for our
task, as they require large sample sizes for their statistical
tests to be valid and/or impose high computational costs.

Most germane to our work is the scheme proposed by
Olivain and Goubault-Larrecq, which uses hypothesis test-
ing of the sample entropy statistic to determine whether
packets are encrypted [27]. Similarly, Dorfinger proposed
an approach for detecting encrypted traffic using entropy
estimation, intended as a pre-filtering step for a Skype flow
detection engine [3, 9–11, 39]. Their encryption detection
scheme shares much in common with that of Olivain and
Goubault-Larrecq, and is based on calculating the sample
entropy for the packet payload and comparing that entropy
value to the expected entropy value for a sequence of
uniformly randomly distributed bytes of the same length.
However, their entropy estimation approach does not scale
well to situations where the number of samples is small [28–
30]. For our entropy-based tests, we addressed this issue
head on by calculating the exact probability distribution
function (see §A)for the (byte-)entropy of n bytes, for small
n. Malhotra compared a number of standard statistical
tests, including the χ2 and Kolmogorov-Smirnov tests, for
identifying encrypted traffic [24]. As with Olivain and
Goubault-Larrecq, their approaches required at least 1 KB of
data per trial. In any case, our byte-value tests outperformed
all of these approaches.

From a systems perspective, similar notions have been
suggested in the context of improving the ability of NIDS
to weather inevitable traffic spikes by the selective discard-
ing [31] of packets when the system is under load. Similar
to Time Machine, Papadogiannakis et al. propose discarding
packets from lengthy flows, reasoning that these packets are
less likely to trigger alerts. We believe our approach is more
general, both in that we enable new policy decisions, as
discussed earlier, and our techniques can operate on flows
of any length. That said, our techniques are certainly ripe
to be employed in a load-dependent fashion, but we focus
on the more difficult load-independent setting in order to
explore the limits of our techniques.

Hardware-based approaches for reducing the load on
NIDS have also been proposed. That most closely related
to our work is the notion of shunting [15], in which packets
are matched in hardware against a dynamic list of rules
which the IDS/IPS updates periodically. For each packet, the
FPGA-based shunt decides, based on these lists, whether to
drop, pass, or forward the packet to the IDS/IPS. Again,
we see our techniques for the identification of opaque
traffic as complementary to the shunting approach. Since
the likelihood-ratio test can be, in the extreme, simplified to
counting high-value bits, it can be easily implemented on
an FPGA, allowing the NIDS to specify opacity-based rules
for packet forwarding decisions.

Lastly, the application of sequential hypothesis testing to
computer security problems is not new. For instance, Jung

et al. [20] applied sequential hypothesis testing to portscan
detection; Jaber and Barakat [19] proposed an iterative
approach that used the size and direction of each packet
to determine the most
likely application generating the
observed traffic; and Thatte et al. [37] proposed a distributed
denial-of-service detector based on a bivariate SPRT.

VI. SUMMARY

In this paper, we propose the notion of quick and accurate
winnowing of opaque traffic as a mechanism for reducing
the demands upon DPI engines, and introduce a number
of statistical techniques for performing such winnowing.
Our techniques are compared against those that might be
considered common wisdom, and through extensive evalua-
tion, we show that our statistical approaches perform best.
Our results demonstrate that we are able to identify opaque
data with 95% accuracy (Section §III-A). By implementing
our approach with the Snort IDS, we demonstrate that
winnowing vastly improves the rate at which an IDS can
process packets without adversely impacting accuracy. Our
experiments show that winnowing enables Snort to process
147% more packets and generate 135% more alerts over a
24-hour period featuring a peak traffic volume of 1.2Gbps.
It is our hope that the ability to classify traffic in the
ways proposed in this paper will provide significant benefits
by enabling second-line (e.g., DPI) analysis engines to
target each class with specialized approaches, rather than
attempting to apply heavy-weight analyses to a general
mix of traffic. That said, our real-world evaluations offer
a cautionary tale; our experiments indicate that the vast
majority—89% of payload-carrying TCP packets on our
network—of modern network traffic is opaque. This finding
was certainly a surprising result to us, and we believe it
may have far reaching consequences in its own right. At
the least, it calls into question the long-term viability of
DPI techniques, and warrants revived interest in finding new
ways for traffic monitoring and policy enforcement. We hope
that this paper stimulates discussions along those lines.

VII. ACKNOWLEDGMENTS

We thank Kevin Snow, Teryl Taylor, Vinod Yegneswaran
and the anonymous reviewers for their valuable suggestions
on ways to improve this paper. We also express our grat-
itude to our campus networking gurus (Bill Hays, Murray
Anderegg, and Jim Gogan) for their tremendous efforts in
helping deploy the infrastructure for this study, to Alex
Everett for providing access to university rulesets, and to
Jake Czyz for his assistance with data collection.

The researchers and their respective Technology Service
Offices have longstanding memorandums of understanding
(MoU) in place to collect anonymized network traffic. The
MoU covers specific uses and types of networking data,
as well as conditions for securing and accessing such data.
To be compliant with our pre-existing Institutional Review

Board (IRB) policies, all computations on payloads were
performed in memory. For this specific collection effort,
the Institutional Review Board (IRB) concluded that, as
we collect only coarse-grained statistics regarding packet
payloads, the activities in our application “do not meet the
regulatory definition of human subjects research under the
Common Rule” and are therefore not regulated.

This work was

supported in part by the Depart-
ment of Homeland Security (DHS) under contract num-
ber D08PC75388, the U.S. Army Research Office (ARO)
under Cyber-TA Grant no. W911NF-06-1-0316, and the
National Science Foundation (NSF) award no. 0831245.
Any opinions, findings, and conclusions or recommendations
expressed in this material are those of the authors and do
not necessarily reflect the views of DHS, NSF, or ARO.

REFERENCES

[1] I. Ahmed, K.-s. Lhee, H. Shin, and M. Hong. Fast file-
type identification. In Proceedings of the Symposium
on Applied Computing, pages 1601–1602, 2010.

[2] A. Bittau, M. Hamburg, M. Handley, D. Mazi`eres,
and D. Boneh. The case for ubiquitous transport-level
encryption. In USENIX Security Symposium, 2010.

[3] D. Bonfiglio, M. Mellia, M. Meo, D. Rossi, and
P. Tofanelli. Revealing Skype traffic: when randomness
plays with you. Comp. Commun. Review, pages 37–48,
2007.

[4] J. J. Bussgang and M. B. Marcus. Truncated sequential
IEEE Transactions on Information

hypothesis tests.
Theory, 13(3), July 1967.

[5] N. Cascarano, A. Este, F. Gringoli, F. Risso, and
L. Salgarelli. An experimental evaluation of the com-
In Global
putational cost of a DPI traffic classifier.
Telecommunications Conference, pages 1–8, 2009.

[6] K. Chiang and L. Lloyd. A case study of the Rustock
In Proceedings of the First
rootkit and spam bot.
Workshop on Hot Topics in Understanding Botnets,
2007.

[7] W. J. Conover. A Kolmogorov goodness-of-fit test for
discontinuous distributions. Journal of the American
Statistical Association, 67:591–596, 1972.

[8] G. Conti, S. Bratus, A. Shubina, B. Sangster, R. Rags-
dale, M. Supan, A. Lichtenberg, and R. Perez-Alemany.
Automated mapping of large binary objects using prim-
itive fragment type classification. Digital Investigation,
7(Supplement 1):S3 – S12, 2010.

[9] P. Dorfinger. Real-time detection of encrypted traffic
based on entropy estimation. Master’s thesis, Salzburg
University of Applied Sciences, 2010.

[10] P. Dorfinger, G. Panholzer, B. Trammell, and T. Pepe.
real-time
Entropy-based traffic filtering to support
In Proceedings of the 6th Interna-
Skype detection.
tional Wireless Communications and Mobile Comput-
ing Conference, pages 747–751, 2010.

[11] P. Dorfinger, G. Panholzer, and W. John. Entropy
estimation for real-time encrypted traffic identification.
In Traffic Monitoring and Analysis, volume 6613 of
Lecture Notes in Computer Science. 2011.

[12] H. Dreger, A. Feldmann, V. Paxson, and R. Sommer.
Operational experiences with high-volume network in-
In Proceedings of the 11th ACM
trusion detection.
conference on Computer and Communications Secu-
rity, 2004.

[13] H. Dreger, A. Feldmann, M. Mai, V. Paxson, and
R. Sommer. Dynamic application-layer protocol anal-
ysis for network intrusion detection. In Proceedings of
the 15th USENIX Security Symposium, 2006.

[14] R. Fielding, J. Gettys, J. Mogul, H. Frystyk, L. Masin-
ter, P. Leach, and T. Berners-Lee. RFC 2616, Hypertext
Transfer Protocol – HTTP/1.1, 1999.

[15] J. M. Gonzalez, V. Paxson, and N. Weaver. Shunting:
a hardware/software architecture for flexible, high-
performance network intrusion prevention. In Proceed-
ings of the 14th ACM conference on Computer and
Communications Security, 2007.

[16] J. Haggerty and M. Taylor. Forsigs: Forensic sig-
nature analysis of the hard drive for multimedia file
In Proceedings of IFIP International
fingerprints.
Information Security Conference, 2006.

[17] G. A. Hall.

Sliding window measurement for file
type identification. ManTech Security and Mission
Assurance, 2006.

[18] R. M. Harris. Using artificial neural networks for
forensic file type identification. Master’s thesis, Purdue
University, 2007.

[19] M. Jaber and C. Barakat. Enhancing application identi-
fication by means of sequential testing. In Proceedings
of the 8th International IFIP-TC 6 Networking Confer-
ence, 2009.

[20] J. Jung, V. Paxson, A. Berger, and H. Balakrishnan.
Fast portscan detection using sequential hypothesis
testing. In IEEE Symposium on Security and Privacy,
pages 211 – 225, may 2004.

[21] B. Li, Q. Wang, and J. Luo. Forensic analysis of
In International
document fragment based on SVM.
Conference on Intelligent Information Hiding and Mul-
timedia Signal Processing, Dec. 2006.

[22] R. Lyda and J. Hamrock. Using entropy analysis to
IEEE Security

find encrypted and packed malware.
and Privacy, 5:40–45, Mar. 2007.

[23] G. Maier, R. Sommer, H. Dreger, A. Feldmann, V. Pax-
son, and F. Schneider. Enriching network security
analysis with time travel. In Proceedings of the ACM
SIGCOMM 2008 Conference on Applications, Tech-
nologies, Architectures, and Protocols for Computer
Communications, pages 183–194, 2008.

[24] P. Malhotra. Detection of encrypted streams for egress
monitoring. Master’s thesis, Iowa State University,

2007.

[25] U. Maurer. A universal statistical test for random bit

generators. Journal of Cryptology, 5:89–105, 1992.

[26] C. Muthukrishnan, V. Paxson, M. Allman, and
A. Akella. Using strongly typed networking to architect
for tussle. In Proceedings of the 9th ACM SIGCOMM
Workshop on Hot Topics in Networks, 2010.

[27] J. Olivain and J. Goubault-Larrecq. Detecting sub-
verted cryptographic protocols by entropy checking.
Research Report LSV-06-13, Laboratoire Sp´ecification
et V´erification, ENS Cachan, France, June 2006.

[28] L. Paninski. Estimation of entropy and mutual infor-
mation. Neural Computation, 15(6):1191–1253, 2003.
[29] L. Paninski. Estimating entropy on m bins given fewer
IEEE Transactions on Information

than m samples.
Theory, 50(9):2200–2203, 2004.

[30] L. Paninski and M. Yajima. Undersmoothed kernel
entropy estimators. IEEE Transactions on Information
Theory, 54(9):4384–4388, 2008.

[31] A. Papadogiannakis, M. Polychronakis, and E. P.
Markatos. Improving the accuracy of network intrusion
detection systems under load using selective packet
the Third European
discarding.
Workshop on System Security, EUROSEC, 2010.

In Proceedings of

[32] V. Roussev and S. L. Garfinkel. File fragment clas-
sification - the case for specialized approaches. IEEE
International Workshop on Systematic Approaches to
Digital Forensic Engineering, pages 3–14, 2009.

[33] A. Shamir and N. v. Someren. Playing “hide and
In Proceedings of the Third
seek” with stored keys.
International Conference on Financial Cryptography,
pages 118–124, 1999.

[34] R. Sommer. Viable Network Intrusion Detection in
High-Performance Environments. Doktorarbeit, Tech-
nische Universit¨at M¨unchen, Munich, Germany, 2005.
[35] S. J. Stolfo, K. Wang, and W. jen Li. Fileprint analysis
for malware detection. Technical report, Columbia
University, 2005.

[36] S. M. Tabish, M. Z. Shafiq, and M. Farooq. Malware
detection using statistical analysis of byte-level file
In KDD Workshop on CyberSecurity and
content.
Intelligence Informatics, pages 23–31, 2009.

[37] G. Thatte, U. Mitra, and J. Heidemann. Parametric
Methods for Anomaly Detection in Aggregate Traffic.
ACM Transactions on Networking, 2011.

[38] K. Thomas and D. Nicol. The koobface botnet and the
In Malicious and Unwanted

rise of social malware.
Software (MALWARE), pages 63 –70, oct. 2010.

[39] B. Trammell, E. Boschi, G. Procissi, C. Callegari,
P. Dorfinger, and D. Schatzmann.
Identifying Skype
traffic in a large-scale flow data repository. In Traffic
Monitoring and Analysis, volume 6613 of Lecture
Notes in Computer Science, pages 72–85. 2011.

[40] C. J. Veenman. Statistical disk cluster classification for

file carving. In Proceedings of the Third International
Symposium on Information Assurance and Security,
pages 393–398, 2007.

[41] D. D. Wackerly, W. M. III, and R. L. Scheaffer.
Mathematical Statistics with Applications. Duxbury
Press, Pacific Grove, CA, sixth edition, 2002.

[42] A. Wald. Sequential tests of statistical hypotheses. The
Annals of Mathematical Statistics, 16(2):117–186, June
1945.

[43] A. Wald and J. Wolfowitz. Optimum character of the
sequential probability ratio test. Annals of Mathemati-
cal Statistics, 19(3):326–339, 1948.

[44] K. Wang and S. J. Stolfo. Anomalous payload-based
In Recent Advances in

network intrusion detection.
Intrusion Detection, pages 203–222, 2004.

[45] M. Weber, M. Schmid, D. Geyer, and M. Schatz.
Peat - a toolkit for detecting and analyzing malicious
In Proceedings of the 18th Annual Com-
software.
puter Security Applications Conference, pages 423–
431, 2002.

[46] L. Zhang and G. B. White. An approach to detect
executable content for anomaly based network intru-
In IEEE International Parallel and
sion detection.
Distributed Processing Symposium, pages 1–8, 2007.

APPENDIX

This appendix presents various methods for opaque traf-
fic identification, an extensive parameter space exploration
experiment to compare these methods and provide optimal
values for their parameters, the asymptotic efficiency of each
method, and a description of the HTTP content-type labeling
rules we used to determine ground truth in our experiments.

A. Comparison of Methods

1) Preliminaries: For all tests, let H0 be the hypothesis
that the vi are approximately uniformly distributed (e.g.,
is compressed or encrypted), and let H1 be
the packet
the alternative hypothesis, i.e., that the vi are distributed
according to some distribution that is not uniform (e.g.,
corresponding to a transparent packet).

In what follows, we examine approaches that can be
broadly classified under two different models, which we
refer to as operating on the entropy or byte-value domains.
For the entropy domain, the basic unit is the (byte-) entropy
of a block of bytes, where the number of bytes n in a block
is parameterized. In other words, if X is a random variable
in the entropy domain, then the support of X is the set of
all possible values for the byte-entropy of n bytes. If X is a
random variable in the byte-value domain, then the support
of X is the set of integers 0 through 255. That is, for the
byte-value domain, the basic unit is simply the byte.

Before delving into details of the various tests we explore,
we first present necessary notation, summarized in Table III.
To represent the packet payload, let v = {v1, v2, ..., vN} be

Symbol

n
k
N
M
vi
wi
v
w
α
β
δ
T

Meaning

size (in bytes) of a block

size of domain (e.g., 256 for bytes)
(maximum) number of payload bytes

(maximum) number of samples

observation (byte)
observation (block)

sequence of observations (bytes)
sequence of observations (blocks)

expected false negative rate
expected false positive rate
alternative hypothesis weight

offset (in bytes)

Table III
NOTATION

a sequence of observations (e.g., payload bytes), such that
vi ∈ {0, 1, ..., k−1} ∀ i ∈ {1, ..., N}. That is, for a sequence
v of bytes, k = 256. In the entropy domain, we operate on
a sequence w of blocks of bytes w = {v1, v2, ..., vN/n}.
Notice that while the density function for the byte-value
domain is well-known and easy to compute, the entropy
domain is more complicated. For many of the tests we
examine, we need to evaluate the probability mass function
(PMF) over possible values for the sample entropy of n
bytes for each hypothesis. Simple threshold checking of the
sample entropy fails to leverage the distribution of sample
entropy values, missing important
the
relative likelihood of the values observed.

information about

Due to space limitations, we omit

the details of the
derivation of this PMF: the essential idea is that the sample
entropy of n bytes is the same regardless of the arrangement
of those bytes. We can then enumerate the possible ways of
arranging n bytes, for small n, to calculate the corresponding
PMF. We implemented such an enumeration using NVIDIA’s
CUDA GPU parallel programming toolkit, yielding a PMF
for n = 8.

Discrete Kolmogorov-Smirnov Test: Another appropriate
test for determining whether a sample is drawn from a
particular distribution is the Kolmogorov-Smirnov (K-S)
test. The K-S test is often used because of its generality
and lack of restricting assumptions. The test is based on
quantifying the difference between the cumulative distri-
bution functions of the distributions of interest. However,
we note that the traditional Kolmogorov-Smirnov test is
only valid for continuous distributions, and Conover has
already shown that the p-values for the continuous K-S test
applied to discrete or discontinuous distributions can differ
substantially from those of the discrete analog. Therefore,
we apply the discrete version as described by Conover [7].
Lastly, we note that Pearson’s χ2 test also seems appro-
priate in this setting. However, Pearson’s χ2 test assumes
that samples are independent, the sample size is large, and
the expected cell counts are non-zero — but, the latter two
assumptions do not necessarily hold in our setting as the
goal is to examine as few bytes as possible. Nevertheless,
we include such analyses for the sake of comparison to

Domain

Entropy

0.001, 0.005, 0.01, 0.05
0.001, 0.005, 0.01, 0.05

0.65, 0.75, 0.85

Param.

α
β†
δ‡
N
T

Bytevalue∗
(no addl.)
(no addl.)
(no addl.)

4, 12, 20

8, 16, 24, 32, 48, 64, 80, 128

4, 12, 20, 28, 32, 40

0, 8, 16, 24

† Applicable only for sequential tests.
‡ Not applicable for χ2 or discrete K-S tests.
∗ In addition to those for the entropy tests.

PARAMETER SPACE EXPLORED BY DOMAIN

Table IV

prior work [24], but omit the derivation since the test is
in common usage and our application is no different.

2) Parameter Space Exploration: We now present the re-
sults of a parameter space exploration experiment examining
all of the hypothesis tests with varying parameter values.
We present ROC (receiver operating characteristic) plots
that simultaneously examine the true positive rate and false
positive rate as a parameter (e.g., a threshold) varies. A set
of ROC plots, one for each classifier, can then be used
to compare classifiers across a range of parameter values,
allowing one to judge the relative performance of classifiers.
There are several knobs we can turn. For the sequential
tests we explore the desired maximum false positive rate
α,
the desired maximum false negative rate β, and the
maximum number of payload bytes N. The parameter values
we examined for fixed tests included the desired significance
level α and number of payload bytes N. In all experiments,
the maximum number of samples for the sequential tests
equals the sample size for the fixed tests.

We consider different alternative hypotheses by changing
the relative weight, δ, of the lower 128 byte values versus
the higher 128 byte values. The δ parameter takes values
in (0.5, 1.0], and indicates the expected percentage of byte
values less than 128. By changing the value, we alter our
model of transparent traffic. In addition to measuring unifor-
mity, this class of alternative hypotheses has the advantages
of being extremely efficient to implement (as determining
whether a byte value is less than 128 is a simple bit mask
operation) and of accounting for the prevalence of ASCII
characters in network traffic.

Figure 8.

Illustration of length (N), offset (T ), and block size (n)

Finally, we explore starting our analysis at different points
within the packet, represented by the offset value T (in

Offset (T)0...T-1...Packet iBlock 0Block 1Block 0BlockSample(s){Block Size (n)EntropyTest0...T-1v0v1v2v3...Packet iSequence Size (N)viByte-ValueTestOffset (T)bytes). As shown pictorially in Figure 8, T indicates how
many bytes into the payload, i.e., past the end of the TCP
header, our analysis begins. The specific values explored for
each parameter are given in Table IV. Our dataset for the
exploration experiment consists of the first 100,000 packets
from trace1, and we consider only encrypted data as
positive examples.

Figure 9 shows a single point for each unique set of
parameter values (over 10,000) in our experiments. Superior
classifiers should evidence a high true positive rate and a
low false positive rate, represented on a ROC plot by a
predominance of points in the upper-left corner of the plot.
Since the byte-value tests evidence the points closest to
the upper-left corner, the plots indicate that the sequential
tests and the likelihood-ratio test in the byte-value domain
are able to more accurately classify packets than the other
tests. We can also compare our techniques based on their
theoretical computational efficiency (see §A3); again, the
byte-value sequential tests are the clear winners.

We also determine which specific values for the various
parameters provide the optimal performance. In order to
do so, we make use of the so-called F -score, which is
a weighted harmonic mean of the precision and recall
metrics.5 Precision is defined as the ratio of the number
of packets correctly classified as opaque, i.e., true positives,
to the total number of packets classified as opaque. Recall
is defined as the ratio of the number of true positives to
the number of packets which are opaque according to our
ground truth. Said another way, precision can be seen as
measuring how often packets labeled as opaque are actually
opaque (but says nothing about how often opaque packets
are correctly identified), while recall indicates how often the
method correctly identifies opaque packets (but says nothing
about how often transparent packets are mislabeled).

We now examine each parameter in isolation by varying
the parameter of interest while fixing the other parameters
at default values (δ = 0.85, α = β = 0.005, N = 32,
and T = 8). As might be expected, the number of bytes
examined has a substantial effect on the performance of
the detectors (Figure 10(a)); this effect drops off over time,
suggesting that less than 32 bytes are needed in the general
case to make a decision and less than 16 bytes are needed by
the truncated test in the byte-value domain. The byte-value
tests are the clear winners; all three sequential methods and
the likelihood-ratio method performed equally well, each
attaining precision over 90%. The entropy tests performed
unexpectedly poorly, in few cases obtaining scores close
to those of the other tests. In addition, we found that the
restricted entropy test did not behave as we expected with
regard to increasing the number of samples involved; an
investigation revealed no underlying patterns in labels of

5More precisely, we use the F1-score, where recall and precision are

evenly weighted.

the misclassified examples. We suspect that the particular
decision boundaries (§II) used are simply poorly suited for
the tests between entropy distributions.

With regard to changes in offset (Figure 10(b)), we see no
clear improvement in classification for offset values larger
than 8 bytes. Changes to the desired false positive and
negative rates similarly had limit impact on the performance
of the detectors, and so we omit those figures for brevity.
Performance when varying the δ parameter, which controls
the alternative hypothesis, peaks at δ = 0.85.

Changes to the desired false positive rate (Figure 11(a)),
similarly, have little impact on the performance of the
detectors (the case for the false negative rate β is the same;
we omit the figure for brevity). In either case, the parameters
have little effect on the entropy tests (due to the small
number of samples for the entropy tests, e.g., 4 samples
at N = 32 bytes); we therefore omit the entropy figures due
to space constraints.

We also examine changes to the δ parameter, which con-
trols the alternative hypothesis. Notice that in Figure 11(b),
the sequential
tests appear to fail most often when the
byte value distribution under H1 is assumed to be close
to uniform; this is due to a large number of trials failing to
make a decision, or forcibly making the wrong decision,
when faced with too few samples to effectively choose
between two similar distributions.

Summary of Findings: Our analysis shows that we can
narrow the field to the likelihood-ratio and sequential tests
in the byte-value domain based on accuracy rates alone.
Perhaps surprisingly,
the entropy-based methods do not
perform as accurately as those in the byte-value domain; we
believe that this indicates that accurate entropy tests require
more samples than are available in our context. In addition,
examining more than 16 bytes provided little benefit in
terms of accuracy. Interestingly, the offset parameter had
little effect in our tests. In summary, we found the truncated
sequential test to one of the most accuracy and efficient tests,
and therefore made use of this technique for the experiments
in the body of the text.

3) Theoretical Efficiency: We can also compare our tech-
niques based on their theoretical computational efficiency
(Table V), in terms of the number of online floating-point
operations required for each case. In the following, any
operations which can be precomputed are omitted. In the
byte-value domain, the sequential tests each require only a
single multiplication and comparison per iteration, so the
number of standard floating-point operations is no more
than 2M for M samples (in the byte-value domain, samples
are bytes, so M = N); the likelihood ratio test is the
same, but with equality in the relation. Working in the
entropy domain introduces the overhead both of binning
the samples (M = N/n operations) and of computing the
sample entropy (3k standard operations plus k logarithms).
Pearson’s χ2 test requires M operations to bin the samples

Figure 9. ROC plots for the techniques examined in this work.

and 3k operations, where k is the size of the domain (e.g.,
k = 256 for the byte-value domain), to sum the squared
difference between the observed count and expected count
over all bins. Finally,
is O(M 2).
Obviously, from a performance standpoint, neither the K-
S test nor the entropy tests are a good choice.

the discrete K-S test

Test Type
Sequential

Likelihood Ratio

Pearson’s χ2
Discrete K-S

Floating-Point Operations

≤ 2M
2M

M + 3k
O(M 2)

Table V

THEORETICAL EFFICIENCY

B. HTTP Labeling Rules

Base Type

image, video, audio

text

application
application
application

*

Sub-type

pdf, xml, flash

gzip, zip

*
*

*
*

Action/Label

opaque

transparent
transparent

opaque
omit
omit

HTTP CONTENT-TYPE FILTERING RULES

Table VI

To determine ground truth for HTTP packets, we first ex-
amine the content-encoding: if the strings ‘gzip’ or ‘deflate’
appear in the content-encoding, the packet is labeled opaque;
otherwise, the label is determined by content type. HTTP
content-type fields contain a base type, such as ‘text’ or
‘video’, a sub-type, such as ‘html’ or ‘mp4’, and optional
parameters, such as ‘charset=us-ascii’. For our filtering, if
the base type is ‘image’, ‘video’, or ‘audio’, the traffic
is labeled opaque; if the base type is ‘text’, the traffic is
labeled transparent. For the ‘application’ base type, we also
consider the sub-type: ‘zip’ and ‘x-gzip’ are considered
opaque, while ‘xml’, ‘pdf’, and ‘x-shockwave-flash’ are
considered transparent. While some of these sub-types (e.g.,
PDF) can be containers for many other formats, we choose to
be conservative and simply classify them as transparent since
we have no clear cut way of drawing the line between semi-
structured and more opaque formats. All other sub-types
are dropped, since hand-labeling of all potential content-
types is infeasible and error-prone. Our HTTP filtering and
labeling rules are summarized in Table VI (given in order
of application).

(a) F1-Score for sample size

(b) F1-Score for offset

Figure 10. Effects on accuracy when varying sample size and byte offset

(a) F1-Score for significance level

(b) F1-Score for alternative hypothesis

Figure 11. Effects on accuracy when varying significance level and alternative hypothesis

222324252627Length0.00.20.40.60.81.0F-ScoreBytevalueRestrictedLikelihoodRatioTruncated222324252627Length0.00.20.40.60.81.0F-ScoreBytevalueDiscreteK-SChisquare222324252627Length0.00.20.40.60.81.0F-ScoreEntropyRestrictedLikelihoodRatioTruncated22232425Offset0.00.20.40.60.81.0F-ScoreBytevalueRestrictedLikelihoodRatioTruncated22232425Offset0.00.20.40.60.81.0F-ScoreBytevalueDiscreteK-SChisquare22232425Offset0.00.20.40.60.81.0F-ScoreEntropyRestrictedLikelihoodRatioTruncated10−310−210−1Alpha0.00.20.40.60.81.0F-ScoreBytevalueRestrictedLikelihoodRatioTruncated10−310−210−1Alpha0.00.20.40.60.81.0F-ScoreBytevalueDiscreteK-SChisquare0.650.700.750.800.85Delta0.00.20.40.60.81.0F-ScoreBytevalueRestrictedLikelihoodRatioTruncated|
06547133.pdf,|2013 IEEE Symposium on Security and Privacy

Practical Control Flow Integrity & Randomization for Binary Executables

Chao Zhang1, Tao Wei1,2∗, Zhaofeng Chen1, Lei Duan1,

L´aszl´o Szekeres2,3+, Stephen McCamant2,4+, Dawn Song2, Wei Zou1

1Beijing Key Lab of Internet Security Technology

Institute of Computer Science and Technology

Peking University, China

Abstract—Control Flow Integrity (CFI) provides a strong
protection against modern control-flow hijacking attacks. How-
ever, performance and compatibility issues limit its adoption.
We propose a new practical and realistic protection method
called CCFIR (Compact Control Flow Integrity and Random-
ization), which addresses the main barriers to CFI adoption.
CCFIR collects all legal targets of indirect control-transfer in-
structions, puts them into a dedicated “Springboard section” in
a random order, and then limits indirect transfers to flow only
to them. Using the Springboard section for targets, CCFIR can
validate a target more simply and faster than traditional CFI,
and provide support for on-site target-randomization as well
as better compatibility. Based on these approaches, CCFIR can
stop control-flow hijacking attacks including ROP and return-
into-libc. Results show that ROP gadgets are all eliminated. We
observe that with the wide deployment of ASLR, Windows/x86
PE executables contain enough information in relocation tables
which CCFIR can use to find all legal instructions and jump
targets reliably, without source code or symbol information.

We evaluate our prototype implementation on common web
browsers and the SPEC CPU2000 suite: CCFIR protects large
applications such as GCC and Firefox completely automati-
cally, and has low performance overhead of about 3.6%/8.6%
(average/max) using SPECint2000. Experiments on real-world
exploits also show that CCFIR-hardened versions of IE6,
Firefox 3.6 and other applications are protected effectively.

I. INTRODUCTION

Many protection mechanisms including DEP (Data Ex-
ecution Prevention [1]), ASLR (Address Space Layout
Randomization [2][3]), GS/SSP (Stack Smashing Protec-
tor
[4][5]), and SafeSEH (Safe Structured Exception Han-
dling [6]) have gained wide adoption, and they are making it
more difficult for attackers to exploit vulnerabilities. These
mechanisms can mitigate various standard attacks, but these
reactive defenses can often be bypassed by advanced ex-
ploitation techniques [7][8]. Attacker countermeasures that
originally sounded impossible become easier and easier,
and sometimes automatable, over time. A better long term
approach is to focus on what we want to protect, and then
design protection measures accordingly.

A natural protection against control-flow hijacking attacks
is to enforce CFI (Control Flow Integrity [9]): a guarantee

*Corresponding author. Email: lenx.wei@gmail.com
+Work done while the authors were at UC Berkeley.

1081-6011/13 $26.00 © 2013 IEEE
DOI 10.1109/SP.2013.44

559

2University of California, Berkeley

3Stony Brook University
4University of Minnesota

that all control-flow transfers in a program will be the ones
intended in the original program (i.e., those represented in
the compiler’s control-flow graph). CFI can stop all control-
flow hijacking attacks, including sophisticated ROP exploits
(Return Oriented Programming [10][11][12]). CFI provides
a guarantee that is strong, and can be easily reasoned about
formally; this also makes it useful as a building block for
other protections [13][14]. The world would be a much more
secure place if every binary was protected with CFI.

Unfortunately, despite its long history (the original paper
proposing it was in 2005 [9]), CFI has not seen wide indus-
trial adoption. CFI suffers from a perception of complexity
and inefficiency: reported overheads (average/max) have
been as high as 7.7%/26.8% [13] and 15%/46% [9]. Many
CFI systems require debug information that is not available
in COTS applications, and cannot be deployed incrementally
because hardened modules cannot inter-operate with un-
hardened modules.

We propose a new practical and realistic protection
method called CCFIR (Compact Control Flow Integrity and
Randomization, pronounced “see-see-fur”), which fills most
of the gap between existing lightweight protection mecha-
nisms on one hand, and CFI on the other. It introduces low
performance overhead, and is compatible with unmodified
legacy binaries. These properties address the main barriers
to adopting CFI widely.

CCFIR enforces a policy on indirect control transfers that
prevents jumps to any but a white-list of locations; it also
distinguishes between calls and returns, and prevents unau-
thorized returns into sensitive functions. These restrictions
capture the most important aspects of CFI protection, with-
out requiring difficult and imprecise alias analysis. For effi-
ciency and compatibility, CCFIR performs this enforcement
by directing indirect control transfers through a dedicated
“Springboard section” that encodes target restrictions via
code alignment. The execution time overhead of this check-
ing is low, 3.6%/8.6% (average/max) over SPECint2000.
As a further layer of protection, the Springboard section
facilitates randomly permuting the allowed jump targets at
program startup, further increasing the difficulty of control-
flow injection.

We build CCFIR as a purely binary transformation. It does

not depend on source code or debug information. Instead,
it analyzes binary executables based on relocation tables
which are available with the wide deployment of ASLR. It
can be validated independently and deployed progressively.
We have applied it to parts of IE6 and Firefox 3.6 and 5
other applications to stop 10 known vulnerabilities. CCFIR
protects binaries as large as the 11MB xul.dll in Firefox
completely automatically.

In summary, our CCFIR protection approach has the

following key advantages:
• Robust protection: provides

against
control-flow hijacking attacks including return-to-libc and
ROP. ROP gadgets are eliminated.

strong defense

• On-site randomization: an additional lightweight layer of
protection beyond CFI to frustrate control-flow attacks.
• High performance: low overhead compared to previous

CFI implementations, only 3.6%/8.6% (average/max).

• Binary only: no source code or debug symbols required.
• Progressive deployment: protected and unprotected code

can inter-operate, without raising false alarms.

• Verifiable: can be verified independently.

The remainder of this paper is organized as follows: We
talk about related work in Section II, and then give an
overview of our approach in Section III. We describe the
design and implementation of our system in Section IV. Sec-
tion V gives our evaluation of performance and protection.
Section VI discusses security topics including remaining
possible attacks. Finally Section VII concludes.

II. RELATED WORK

Binary Disassembling and Rewriting. Schwarz et
al. [15] cover the disassembly problem in detail, including
two standard algorithms and a new combination. Their
approach also uses relocation tables, but less extensively.
A common challenge for disassembly is mixing of code
and data within the code section. Many other approach-
es [16][17][18] have depended heavily on heuristics which
with unjustified assumptions that miss some fraction of code.
With the deployment of DEP, compilers are more restricted
in the ways they can mix code and data. We believe we
are the first to point out that the binaries generated with
modern compilers’ security-sensitive modes (DEP, ASLR)
can be thoroughly analyzed using their relocation tables.
Some of the systematic binary rewriting modes we use were
previously proposed in systems such as Vulcan [19].

Control Flow Hijacking Attacks and Mitigation. Mem-
ory safety enforcement can protect against control-flow hi-
jacking attacks; it also defeats non-control-data attacks [20].
A representative technique is automatic bounds check-
ing [21][22]; however these techniques require recompilation
from source, and their performance overheads are too high
for practical deployment. Recently proposed techniques,
such as SoftBound [23] with CETS [24], also introduce
an overhead of 116%. Approaches that enforce data-flow

560

integrity [25] can also stop many kinds of exploits, but cause
a 2.5x slowdown. PointGuard [26] use pointer encryption to
protect function pointers from tampering. However it causes
compatibility issues and has weaknesses [27].

Modern operating systems widely adopt lightweight and
efficient protection mechanisms, like DEP [1], ASLR [2],
and SafeSEH [6]. However, advanced exploit techniques
like return-to-libc and ROP-based exploits [10][11] can
defeat
these protections. An indication of the power of
these techniques is that they can provide attackers a Turing
complete language for malicious functionality [10][28].

Some new mitigation techniques have focused on pro-
tecting against ROP [29], but this has also spawned newer
variants of attacks [12]. More comprehensive ROP protec-
tions, such as ROPdefenser [30] and ILR [18] introduce high
overhead. IPR [31] uses randomization in basic blocks with
minimal overhead, but provides only partial protection.

Control Flow Integrity. Abadi et al. introduced the term
CFI [9] and proposed a technique to enforce it. Rather
than trying to protect the integrity of function pointers and
return addresses, this technique marks the valid targets of
these indirect control transfers (i.e. function entry points and
landing points for returns) with unique identifiers (IDs), and
then inserts ID-checks before each indirect call or return
instruction. They propose identifying the set of valid targets
(i.e. the points-to set) through a precise control flow graph
(CFG) construction and enforcing control flow only to this
set for each indirect transfer instruction.

However, a precise CFG construction needs a sophisti-
cated pointer analysis, which is especially difficult without
source code or debug symbols. Compatibility is a problem:
hardened modules and un-hardened modules cannot inter-
operate, preventing incremental deployment which is often
needed in real systems. A further challenge is diversity of
IDs. The more IDs the code uses, the more restricted jumps
are, but any overlapping points-to-sets must be unified to
use the same ID. Sharing of jump targets such as library
functions can lead to many sites having only one ID.

In the absence of detailed analysis, Abadi et al. suggest
that using a single identifier for all sites (a 1-ID approach) or
one for calls and another for returns (a 2-ID approach) could
still provide substantial protection. Their implementation
used a conservative CFG in which any indirect call could
target any function whose address is taken. This allows
modular transformation of libraries, while still supporting
multiple return IDs for directly-called functions. However
calls and returns into untransformed code are still prohibited,
so this approach does not support incremental deployment.
CCFIR implements a 3-ID approach, which extends the 2-
ID approach by further separating returns to sensitive and
non-sensitive functions. This stops the jumps that would be
most useful to attackers, but the three-way separation can
be compactly represented in the Springboard section layout
without requiring separate ID values and checks.

MoCFI [32] applies CFI to ARM binaries, but due to the
imperfection of pointer analysis on binaries, they let statical-
ly unresolved calls/jumps transfer to any valid function entry.
CFIMon [33] utilizes the Performance Monitoring Unit in
modern processors for performance, but it is not reliable
in practice because of false negatives and false positives.
CPM [34] uses source code analysis to find all possible
targets of an indirect call and masks (e.g. bitwise AND) the
runtime target with these possible addresses to determine its
validity. Since the target information is encoded at the call
site statically, shared libraries are not supported. Control-
flow locking [35] implements a similar policy and has
similar restrictions. HyperSafe [36] provides integrity for
supervisor-mode code such as a hypervisor, including a CFI-
like technique that replaces jump targets with integer indexes
into function-specific tables. This approach can provide finer
granularity protection for returns compared to basic CFI,
but
it can not support modular compilation or dynamic
linking. Whole-system overhead was modest for benchmarks
dominated by I/O or user-space execution, but HyperSafe’s
approach would likely be significantly more expensive than
CCFIR if applied to CPU-bound user-space applications.

SFI (Software(-based) Fault Isolation [37][38][39]) uses
instruction rewriting but provides isolation (sandboxing)
rather than hardening, typically allowing jumps anywhere
within a sandboxed code region. CFI is also useful as a
foundation for enforcing SFI [13], or other higher level
policies, such as XFI [14] or write-integrity [40].

III. THE CCFIR APPROACH

The goal of CCFIR is to enforce control-flow integrity in
user mode applications by ensuring that the targets of all
indirect control transfer instructions are legal. We identify
the valid targets in binary modules and rewrite them so
that the valid targets can be distinguished from invalid ones
efficiently. Then we insert checks before each indirect con-
trol transfer instruction to make this distinction. To enforce
control-flow integrity fully, all modules have to be rewritten,
but this ideal is not always possible. To support incremental
deployment, our scheme allows unprotected libraries as well.

A. Assumptions

In this paper, we assume the following properties hold:
• ASLR and a W⊕X protection such as DEP are in use.
This usually holds in modern systems. In order to support
ASLR, modern compilers generate relocation tables in
target binaries, which are used by our binary analysis.
With DEP, compilers separate code and data sections
and thus ease disassembling. DEP also prohibits attackers
from tampering with code (including direct transfer in-
structions’ targets) or executing code in the data section.
• The target executable does not self-modify its code or
dynamically generate code. Traditional executables com-
piled from high level languages always satisfy this. For

561

executables that do not conform to this assumption, our
CCFIR scheme is not suitable because static rewriting
cannot enforce runtime control flow integrity.

• Limited information disclosure vulnerabilities are avail-
able to attackers. If intended functionality or a separate
information-disclosure bug allows attackers to read entire
memory regions such as the Springboard, or to selectively
reveal Springboard stub addresses of their choice, the
protection provided by randomization can be negated.

B. Protection Targets of CCFIR

There are several types of control flow transfers in user

mode Windows x86 binaries:
• Exceptions. When exceptions occur, the operating system
takes control and then transfers to user-defined exception
handlers. These handlers are only invoked by the OS and
will not be targets of indirect transfers. In addition, these
exception handlers are well protected by SafeSEH.

• Direct jmp/call and conditional jump (jo, jz etc.) instruc-
tions. Most jmp/call instructions fall into this category.
Their targets are fixed in the code, so DEP or W⊕X
protection prevents attackers from tampering with them.
• Indirect jmp/call instructions, as used for function pointers
and virtual method dispatch. Their targets usually are read
from memory and may be controlled by attackers.

• All ret instructions. Their targets are computed and pushed
onto the stack at run-time by corresponding call instruc-
tions. Attackers may overwrite the return address on the
stack to launch attacks like ROP and return-to-libc.
The first two kinds of transfers are already well protected,
so CCFIR protects the last two. As shorthand we refer to
an intended target of an indirect call or jump instruction as
a function pointer, and the target of a return instruction as
a return address.
C. Identify Indirect Transfer Targets

To protect targets of indirect transfers (i.e., return address-
es and function pointers) in binary executables, the binary
should be disassembled first. And then, we need to find
where all transfer targets are created and used, in order to
deploy further protection.

In general, it is challenging to disassemble an x86 PE
(Portable Executable format [41]) file correctly, because x86
is a CISC platform with variable-length instructions and a
dense encoding. However, we can take advantage of the fact
that ASLR and DEP are widely adopted on Windows. These
facts result in the following important deductions:
R1. ASLR-protected executables must have relocation ta-
bles, because absolute addresses in code must be relo-
cated when loading.

R2. Compilers can freely choose the starting address for a

function or a segment.

R3. Programmers get

the addresses of functions or in-
structions only through ways provided by high level

Original code section

Protected code section

call eax

5

6

ret

1

2

3

4

5

6

ret

call eax

1

2'

3

4

2''

3'

5'

Direct control transfer
Indirect control transfer

Springboard section
(all indirect targets 

are aligned)

(cid:258) (cid:258) (cid:258) 

  0x6800 0000-0x6fff ffff (27bit is 1)

DLL 1

free code

DLL 2

free code

DLL 1

springboard

DLL 2

springboard

  0x6000 0000-0x67ff ffff (27bit is 0)

(cid:258) (cid:258) (cid:258) 

  0x3800 0000-0x3fff ffff (27bit is 1)

EXE

free code

EXE

springboard

  0x3000 0000-0x37ff ffff (27bit is 0)

(cid:258) (cid:258) (cid:258) 

B
M
8
2
1

B
M
8
2
1

B
M
8
2
1

B
M
8
2
1

Figure 1: Illustration of CCFIR: a code section is split up into 2
sections, and all indirect control transfers (dashed lines) are only
permitted to flow to an aligned address in the Springboard section.

Figure 2: Memory layout for executables hardened
by CCFIR: only Springboard sections are placed
in a memory area for which the 27th bit is 0.

languages.1 Programs in high-level languages comply
with this rule, and even most inline assembly code does.
R4. If the targets of indirect call/jmp instructions are hard-
coded in binaries, they must be absolute addresses and
can be indexed through relocation tables (as rule R1).
R5. Compilers separate code and data sections (in order to
conform to DEP). In code sections, the only data which
can appear are special control structures, such as jump
tables for switch statements and exception tables.

These rules hold for binaries generated by modern com-
pilers today. Due to the rule R4, we can find most possible
indirect code entries. Then with the help of the export
table and the EntryPoint of the target PE file, it can be
disassembled recursively to identify all possible instructions.
Combined with other policies described in Section IV-B,
we take an approach that can disassemble a PE file com-
plying with rules R1∼R5 correctly and automatically. For
binaries not respecting R5, we can still identify most code
and data correctly and tag unidentified parts explicitly for
manual review. These remaining parts are usually small even
for large binaries, and can be easily reviewed.

As binaries can be disassembled correctly, we can identify
where transfer targets are created (i.e., all occurrences of
function pointers and return addresses) and where transfer
targets are used (i.e., all control-transfer instructions).
D. The Springboard and New Memory Layout

While CFI enforcement techniques have been used to
make software fault
isolation (SFI) more efficient [13],
we conversely use ideas of layout-based checking from
SFI [38][39] to make CFI enforcement more efficient.

1getpc() is a seldom-used method for addressing code and data in normal
binaries, although it’s more popular in malicious code. In our experiments
we find only one case of getpc in Windows binaries, setjmp() discussed in
Section IV-C2.

For each module, we introduce a new code section called
the Springboard. As shown in Figure 1, for each valid
indirect control-transfer target (e.g. nodes 5 and 3 in this
figure), the Springboard contains an associated unique stub
(nodes 5(cid:2) and 3(cid:2) respectively) containing a direct jump to the
given target. The nodes 2(cid:2) and 2(cid:2)(cid:2) are used to make sure the
node 3(cid:2) is aligned. Using techniques known from SFI, we
make sure that any indirect control-flow transfer instruction
can only jump to a code stub inside the Springboard. As
a result, diverting the execution to an attacker-supplied
arbitrary target becomes impossible.

The Springboard section is distinguishable from other
memory areas through the memory layout. As shown in Fig-
ure 2, it is enforced that any executable code section whose
address’s 27th bit is 0 can only be a Springboard section. In
other words we divide the program’s virtual memory space
into 128MB-large (227) slices, so that Springboard sections
are always in even slices, and other code sections are in odd
slices. Data sections are not constrained.

Real-world applications’ code sections are typically small-
er than 10MB, and they can be placed freely anywhere into
an odd 128MB memory slice, as long as the whole section
is inside the slice. Multiple Springboards or multiple code
sections can be contained in the same 128MB slice but
never mixed. Thus one bit testing instruction is capable of
checking if an address belongs to a Springboard section.

Make Valid Targets Distinguishable. In order to dis-
tinguish valid targets of indirect transfer instructions from
invalid targets (e.g. those supplied by attackers), valid targets
are all redirected to code stubs in the Springboard. Further,
to defeat advanced attacks like ROP and return-to-libc, code
stubs within the Springboard are further distinguishable.

First, function pointer stubs and return address stubs are
different. Second, return address stubs for return-landing

562

Executable

no
yes
yes
yes
yes
yes

Bits
3
*
*
*
1
0
0

26
*
*
*
*
1
0

27
*
1
0
0
0
0

2-0
***
***
!000
000
000
000

Meaning

Non-executable section
Normal code section
Springboard’s invalid entry
Springboard’s function pointer stub
Springboard’s sensitive return stub
Springboard’s normal return stub

CCFIR Static

Original PE 

File

Disassemble

BitCover

Disasm

info

redirect 
& validate
& randomize
BitRewrite

CCFIR Validator

Disassemble

BitCover

Disasm

info

Verify

BitVerify

Hardened PE file

Verification 

Result

Table I: Bit Mask of stubs in Springboard.

Figure 3: Architecture of CCFIR

points within sensitive library functions (e.g. system()
in libc) are different from return address stubs for normal
functions. In other words, there are three kinds of code stubs
in the Springboard (i.e., a 3-ID CFI implementation).

As shown in Table I, stubs inside the Springboards are
aligned and placed at distinguishable addresses based on
their types. Function pointer stubs are 8-byte aligned but not
16-byte aligned. All return address stubs are 16-byte aligned.
Further the 26th bits of sensitive return address stubs are 1,
while they are 0 for normal return stubs.

With this distinction, the type of indirect transfer targets
can be quickly determined at runtime, and a stricter security
policy can be enforced on indirect control transfers.

E. Enforcing Control Flow Integrity

Due to the careful design of the Springboard and stubs
alignment, one or two bit-testing instructions inserted before
an indirect control transfer are capable of validating its target
and so enforcing control flow integrity.

Indirect call and jump instructions can only jump to
function pointer stubs in the Springboard: In particular, they
are enforced to jump to targets within Springboard that are 8-
byte aligned but not 16-byte aligned. Moreover, there are no
function pointer stubs for sensitive functions in the Spring-
board because these functions are never used as targets of
indirect transfers, as discussed in Section IV-E. With this
enforcement, attackers cannot invoke invalid functions or
sensitive functions through indirect call/jmp instructions.

Return instructions in normal functions can only jump
to normal return address stubs in the Springboard, but not
In particular, these return
sensitive return address stubs:
instructions are enforced to jump to 16-byte aligned stubs
whose 26th bits are 0. With this enforcement, the capacity of
return-to-libc attacks is greatly constrained because exploits
cannot jump into sensitive functions. Especially, the Turing-
completeness of return-to-libc attacks [28] is broken.

Return instructions in sensitive functions can jump to
any return address stubs in the Springboard: In particular,
these return instructions are enforced to jump to 16-byte
aligned stubs regardless of their 26th bits.

It is worth noting that user programs may invoke sensitive
functions, and thus returns within sensitive functions may
jump to user functions. On the other hand, we have never
observed a need for sensitive functions to call user functions,

563

so CCFIR prohibits returns within user functions from
jumping to sensitive functions. While simple, we believe this
3-ID scheme achieves a good balance between compatibility
and safety.

In addition, all return instructions’ targets are 16-byte
aligned stubs in Springboard. And thus, attacks like ROP
that jump into the middle of instructions or basic blocks are
prohibited. It also greatly raises the bar for exploits which
weave together small snippets of code (e.g. gadgets).

CCFIR provides an extra protection to randomize the
order of the stubs inside the Springboard at load-time to
defeat guessing the addresses of function pointer and return
address stubs. Section IV-E will discuss this in detail.

IV. SYSTEM DESIGN & IMPLEMENTATION

CCFIR consists of three major modules: BitCover, Bi-
tRewrite & BitVerify. Its architecture is shown in Figure 3.
The first module BitCover disassembles a given PE file,
and identifies all indirect call/jmp/ret instructions and all
potential indirect control-transfer targets (Section IV-B).

BitRewrite statically rewrites the target PE file. In particu-
lar, it inserts Springboard sections for each module, encodes
valid transfer targets with pointers to Springboard stubs
(Section IV-C1), and instruments runtime checks before
indirect transfers to validate the targets (Section IV-C2).
BitRewrite also pays much attention to compatibility issues
(Section IV-D) to support incremental deployment.

In addition, BitRewrite introduces a further layer of pro-
tection, randomization, to increase the difficulty of attacks
(Section IV-E).

A separate module BitVerify checks whether a given bina-
ry conforms to our defined security policies (Section IV-F).
It is the last module before a binary is executed.

A. Background: Relocation Table

The relocation table is a feature of binary code required
to support dynamic linking and ASLR, and BitCover also
uses it to support disassembly. We use the following terms
to describe the structure of a PE-format relocation table:
• Relocation item: a 2-byte entry in the relocation table.
The lower 12 bits of an item are used together with a page
base to compute the address of a relocation slot;

.reloc

0x1000

0x31A4
0x3308

0x6000

0x3700
0x3E88

ImageBase: 
0x400000

.text

0x4011A4

mov eax,
0x401120

foo():

Input

PE file

Phase 1

Phase 2

Invalidate
code entries

Indentify control 

tables

code entry 
candidates

Other unknown 

data

Known control 

tables

Tag 

reliable 
code 
entries

Propagate 

valid
code 
entries

Output

Valid code 

entries

Valid control 

tables

Suspect code 

entries

Figure 4: Relocation table

Figure 5: Workflow of BitCover

• Relocation slot: a memory area that

is to contain a
relocation entry. For instance on a 32-bit archi-
tecture a typical slot will be 4 bytes long.

• Relocation entry:

the value to be stored in the
relocation slot at relocation (load) time. This is
usually the address of a function or global variable.
For example, in Figure 4, a relocation item 0x31A4 exists
in the relocation table (i.e. the .reloc section). The highest
4 bits (i.e. 0x3) indicate this is a normal relocation item.

The address of the relocation slot represented by this item
is 0x4011A4, which is the sum of the image base of the
PE (e.g. 0x400000), the relocatable page’s Relative Virtual
Address (e.g. 0x1000) and the relocation slot’s offset within
this page (e.g. 0x1A4, lower 12 bits of the relocation item).
The actual content stored in this relocation slot, i.e. the
is 0x401120. This is the address of a

relocation entry,
function foo, which needs to be updated at load time.

B. BitCover

The goal of BitCover is to identify all indirect control
transfer instructions and all valid transfer targets. As shown
in Figure 5, the workflow of Bitcover consists of two phases.
1) Phase 1: Explore the Code and Data: The EntryPoint
and entries in the export table of a PE file are possible
code entries. In addition, relocation entries are possible code
entries too according to rule R4 in Section III-C. For each
possible code entry, BitCover starts disassembling from it
recursively. Every executable instruction will be reachable
by recursive disassembly from some code entry.

However, not all export entries or relocation entries are
real code entries; for instance a relocation entry might
represent data rather than code. BitCover uses the following
heuristics to determine when disassembly reaches a byte
sequence that cannot be a valid instruction. If BitCover
encounters an invalid sequence during recursive exploration,
it marks the code entry from where it starts disassembling
as invalid.
H1. No invalid instruction are permitted.
H2. No instruction overlaps with another.
H3. A valid instruction must lead to other valid instructions.
H4. When disassembling starting from a code entry, all pos-
sible paths should stop at return instructions, indirect

jump instructions which jump to unknown targets, or
instructions which invoke must-terminate functions.

H5. All addresses’ sizes must be valid.
H6. If an instruction contains a relocation slot, the content
of this slot (i.e., a relocation entry) must be a valid
immediate value or offset.

H7. Instructions cannot start from relocation slots.
H8. All absolute addresses in code must be relocated, except

special hard-coded system values.

H9. I/O instructions and interrupt instructions are permitted

only in specific situations.

H10. Only specific segment registers can be used in code.
Here, a must-terminate function is one that will never
return to its caller, such as exit or abort in C/C++.
During exploration, BitCover also marks any function that
calls a must-terminate function unconditionally as a must-
terminate function itself. BitCover stops disassembling after
a path reaches a call to a must-terminate function, since
the bytes after that call would not be executed: they may
belong to another function or not be code at all. A function
that might call a must-terminate function under some but
not all circumstances we call a may-terminate function.
BitCover also analyzes which function may terminate, and
if it encounters an invalid byte sequence after a call to a
may-terminate function, it treats that call like a call to a
must-terminate function.

In phase 1, we also identify control tables like switch jump
tables [42]. We use both instruction and data characteristics
to distinguish switch jump tables from instructions. A switch
jump table must be an array of relocation slots containing
pointers which point to valid code entries. Following H7,
BitCover can accurately recognize any switch jump table
as data, i.e. not parts of instructions. In fact, heuristic H7
can filter out most control tables, including vtables for C++
objects, but not jump index tables [42] for switch statements,
because entries in these tables are not relocation slots.

After this phase, all candidate code entries and some
known control tables have been identified. There are still
some invalid candidate code entries and some unknown data
left. A second phase analysis is needed to remove all those
invalid candidate code entries, and identify control tables
like switch jump index tables from the unknown data.

564

2) Phase 2: Refine the Disassembling Result:

phase, BitCover removes unreachable entries,
suspect entries, and identifies remaining control tables.

In this
tags other

If a relocation entry does not exist in the export table
and is not the target of any direct jump or call (i.e., is
not explicitly a function pointer), and all relocation slots
containing this relocation entry are parts of some instructions
(and thus the relocation entry must be an offset or immediate
value according to H6), and this relocation entry will not be
assigned to a register (e.g., directly moved to registers), then
this relocation entry is called as an unreachable entry.

For example, if a relocation entry E is only used in in-
struction mov eax, E[ebx*4], then E is an unreachable
entry. For unreachable entries, there is no chance to transfer
their values to any registers or memory; the program cannot
use it as a valid indirect target. So, we can claim that:
R6. All unreachable entries must not be valid code entries.
Based on rule R6, we can filter out switch jump index
tables [42] and other remaining tables in code sections. In
addition, after filtering out unreachable entries, the remain-
ing code entries in phase 1 are all candidate entries. If a
candidate entry does not point to an entry in a known control
table, it must be a valid code entry, according to rule R5
in Section III-C. So, BitCover can disassemble the whole
program automatically.

For binaries not obeying R5, BitCover will find unknown
data in their code sections. In this case, BitCover tags the
location as a “suspected” target, and leave it for manual
review. In our experiments,
there are limited suspected
entries even in big binaries such as mshtml.dll, and an
expert can tag code entries in them quickly.
C. BitRewrite

The BitRewrite module carries out the central task, instru-
menting the binary to enforce control-flow integrity. This is
done in two steps:
• All valid indirect control transfer targets, e.g. function
pointers and return addresses, are modified by redirecting
them to unique stubs located inside the Springboard sec-
tion. This makes the validity of the addresses verifiable.
• Before each indirect control transfer instruction, e.g. cal-
l/jmp/ret, a special dynamic check is inserted, which
ensures that
is a valid stub in the
Springboard section.
1) Redirecting Indirect Control Transfer Targets: Bi-
tRewrite redirects both indirect call/jmp and ret instructions’
targets, i.e. function pointers and return addresses. As dis-
cussed in Section III-E, in order to enforce a better security
policy, function pointers and return addresses are redirected
to different kinds of stubs in the Springboard. In addition,
the ways function pointers and return addresses are created
and used are different. So, they are handled differently.

the transfer target

Redirecting function pointers. Function pointers in a
compiler-generated binary may be hard-coded in virtual

565

function tables, global variables and instructions. All these
occurrences of function pointers can be found, based on the
relocation table, as described in Section III.

As shown in Figure 6, as foo is loaded into a register
and may be a potential target of an indirect call, a unique
8-byte aligned stub foo sb in the Springboard is associated
with it. This stub contains a direct jump which will jump to
the entry point of foo. BitRewrite then replaces foo in the
instruction mov ecx, foo with foo sb.

Optimizations. As discussed in Section III-B, func-
tion pointers hard-coded in direct call/jmp instructions (e.g.,
call foo) and structured exception handlers used by the OS
are protected by DEP and SafeSEH and cannot be tampered
with by attackers. In addition,
these pointers cannot be
used by indirect transfer instructions. Thus we can improve
performance by no redirecting these pointers, and suffer no
loss of security or correctness.

Moreover, function pointers inside jump tables need not
be redirected. These pointers cannot be tampered with
because of DEP, and can only be targets of jump instructions
for switch statements, such as jmp jtable[eax*4]. When
we confirm that compilers implement jump table lookups
correctly, i.e., jumping out of this table is impossible, these
pointers can be safely skipped.

Redirecting return addresses. The most frequent indirect
control
transfer targets are return addresses, which also
makes them the most popular targets of attacks. Return
addresses are generally pushed onto the stack by corre-
sponding call instructions. To redirect valid return addresses,
BitRewrite relocates all call instructions.

Figure 7 shows the relocation of a direct call. Like
function pointers, a unique 16-byte aligned stub in the
Springboard (here back sb) is associated with each call site.
A direct jump instruction in this stub will jump back to the
original return-landing point (i.e. back). To make this stub
the new return address, the original call is replaced by a
jump to a new call instruction placed right before this stub.
This way, when the function is called, the return address
pushed onto the stack will be the verifiable address of the
stub back sb, which will seamlessly lead back the execution
after the original call site.

Figure 6 shows that indirect calls are modified similarly
to a direct call. The only difference is because the length of
a direct call instruction is 5 bytes, while the length of an
indirect one is 2. Hence their modified targets are back rsb-5
and back rsb-2 respectively. Moreover, as discussed in Sec-
tion III-D, for return-landing points in sensitive functions,
the 26th bits of associated return address stubs must be 1,
while they are 0 for normal return address stubs.

Having all indirect control transfer targets redirected to
their aligned stubs in the Springboard section makes legal
targets distinguishable from illegal ones.

2) Validating Indirect Control Transfers: As discussed
earlier, we focus on validating indirect call/jump and return

Original

Hardened

 mov ecx,foo

 ...

 call ecx
back:
 ...

 mov ecx,foo_sb
 ...
 test ecx,8
 jz error
 test ecx,M_F
 jnz error
 jmp back_sb-2
back:
 ...

 call ecx
back_sb:
 jmp back

foo:
 ...

 ret

foo:
 ...

 test [esp],M_R
 jnz error
 ret

foo_sb:
 jmp foo

 call foo
back:
 ...

Original

Hardened

foo:
 ...

 ret

foo:
 ...

 jmp back_sb-5
back:
 ...

 test [esp],M_R
 jnz error
 ret

 call foo
back_sb:
 jmp back

Direct control transfer

Indirect control transfer

(cid:894)(cid:912)(cid:887)(cid:849)(cid:878)(cid:849)(cid:865)(cid:937)(cid:873)(cid:865)(cid:865)(cid:865)(cid:865)(cid:865)(cid:872)
(cid:894)(cid:912)(cid:899)(cid:849)(cid:878)(cid:849)(cid:865)(cid:937)(cid:873)(cid:865)(cid:865)(cid:865)(cid:865)(cid:865)(cid:919)
(cid:849)(cid:849)(cid:849)(cid:849)(cid:849)(cid:849)(cid:928)(cid:931)
(cid:894)(cid:912)(cid:899)(cid:849)(cid:878)(cid:849)(cid:865)(cid:937)(cid:884)(cid:865)(cid:865)(cid:865)(cid:865)(cid:865)(cid:919)

Direct control transfer

Indirect control transfer

(cid:894)(cid:912)(cid:887)(cid:849)(cid:878)(cid:849)(cid:865)(cid:937)(cid:873)(cid:865)(cid:865)(cid:865)(cid:865)(cid:865)(cid:872)
(cid:894)(cid:912)(cid:899)(cid:849)(cid:878)(cid:849)(cid:865)(cid:937)(cid:873)(cid:865)(cid:865)(cid:865)(cid:865)(cid:865)(cid:919)
(cid:849)(cid:849)(cid:849)(cid:849)(cid:849)(cid:849)(cid:928)(cid:931)
(cid:894)(cid:912)(cid:899)(cid:849)(cid:878)(cid:849)(cid:865)(cid:937)(cid:884)(cid:865)(cid:865)(cid:865)(cid:865)(cid:865)(cid:919)

Figure 6: Rewriting of an indirect call and return

Figure 7: Rewriting of a direct call and return

instructions’ targets before the control transfers. The policy
our scheme enforces is the following:
• Indirect call/jump instructions’ targets must be function
pointer stubs (i.e 8-byte aligned but not 16-byte aligned)
in the Springboard.

• The target of a return back to a sensitive function can be

any valid return stub (i.e. 16-byte aligned).

• Any other return instruction’s target must be a valid
normal return stub (i.e., 16-byte aligned with the 26th
bit 0).
As discussed in Section III-D, this enforcement can be

performed using one or two bit-testing instructions.

For any indirect call/jump instruction, its target should be
in the Springboard (i.e. the 27th bit is 0) and only 8-bytes
aligned (i.e. the 0-2 bits are 0, but the 3rd bit is 1). Thus if
the target address is bitwise ANDed with 8, the result should
be non-zero. In addition, if the TARGET is bitwise ANDed
with the mask 0x8000007 (i.e. M_F in Figure 6), the result
should be zero. As shown in Figure 6, these bitwise AND
operations are performed with the test instruction. If one
of these conditions is violated, the control flow is directed
to a predefined error handler (i.e. error in Figure 6). In
our implementation, the error handler will log the buggy
EIP value and the invalid transfer target, and then terminate
the process. (To record the EIP, there is a separate copy of
error for each indirect call/jump and return.)

Figure 6 also shows how the validation is inserted before
return instructions. Before returning, the target of the return
is on the top of the stack, pointed to by the esp register. The
return address is then tested against a mask M_R. The mask
is 0x800000f for returns from functions called by sensitive
functions, and 0xc00000f for all other return instructions.

An Exceptional Case. The function longjmp()
ends with an indirect jump, but its target is a return address

saved by a call to setjmp(), and so is 16-byte aligned. Thus
the check for this special jmp instruction matches the check
for a return instruction: test ecx, 0xc00000f.

Optimizations. Indirect jump instructions which are
used for switch statements, such as jmp jtable[eax*4], do not
need dynamic checks. For any switch statement, regardless
of what its control expression is, the control flow in the
binary generated by modern compilers (e.g., GCC and VC)
is forced to one entry in its jump table. For example, GCC
first makes a bound check against eax (corresponding to the
case value in switch statements). If it exceeds the bound,
then eax is assigned with a default value (corresponding
to the default case). And then, the control flow transfers
through jmp jtable[eax*4]. In this way, the control flow is
always forced to the jump table entries and thus cannot be
hijacked by attackers. Thus BitRewrite skips validating these
indirect jump instructions, to improve performance.

D. Compatibility Issues

A protected module only allows indirect control transfers
whose targets are valid Springboard stubs. But the stubs are
not restricted to be within the current module’s Springboard
section. Stubs within other modules’ Springboard sections
are also permitted, since their addresses are compatible; they
are validated the same way. And thus if every module in a
program (i.e. the main program and all DLLs) is rewritten,
according to the scheme described in the previous section,
the separate modules will be compatible with each other in
any combination and the control-flow integrity is enforced.
However, rewriting all modules is not always possible
in practice (e.g. system DLLs on Windows 7 cannot be
altered). While control transfers from an unprotected module
to a protected one cause no problem, if there is an indirect
control transfer from the protected module to an unprotected

566

mov ecx,
 [foo_slot]

Original

Hardened

mov ecx,
[foo_slot_wrap]

.iat:
 ...

foo_slot:
 foo

.iat:
 ...

foo_slot:
 foo
gpa_slot:
 GetProcAddress

foo_slot_wrap:
 foo_sb

foo_sb:
 jmp [foo_slot]

gpa_slot_wrap:
 gpa_sb

gpa_sb:
 jmp gpa_wrap

gpa_wrap:
 ...
call [gpa_slot]
 ...
#fill new stub
#ret stub ptr

points to

Figure 8: Redirection of imported functions and GPA

one, the check will fail. In order to support the need for
incremental deployment, BitRewrite makes special efforts
for compatibility.

Compatibility issues come up when a protected module
returns to an unprotected module, or calls/jumps to an
external function in an unprotected module through
• an imported function pointer,
• a function pointer resolved at runtime by special API, e.g.,

GetProcAddress(), or

• a non-exported function pointer.

1) Imported function pointers: Most calls to external
functions are done through imported function pointers. Im-
ported function pointers are all stored in the import address
table (IAT) and then are accessed through IAT entries. For
example, in Figure 8, the imported function foo’s address is
stored in the .iat section (i.e., the foo slot). All references
to foo are accessed through its IAT entry foo slot, e.g., mov
ecx, [foo_slot]. Imported function pointers will be
resolved at load time by the dynamic linker and the IAT
entries will be updated.2 As a result, statically modifying
these IAT entries does not work.

To work around this issue, for each IAT entry, BitRewrite
generates a read-only and non-executable wrapper to replace
it. As shown in Figure 8, for the imported function foo, a
wrapper foo slot wrap is generated to replace foo slot. The
wrapper stores a function pointer which will jump to the
original imported function.

Optimizations. Because the IAT is read-only, im-
ported function pointers directly used in call/jmp instruc-
tions, such as call/jmp [foo slot], can also skip redirection
to improve performance.

2) Run-time resolved function pointers: Sometimes dy-
namic libraries are not loaded and linked at load time, but at
run-time using LoadLibrary. Function pointers in such cases

2Although the IAT will be updated by the dynamic linker, it is usually

read-only and non-executable at runtime.

567

can be obtained by the GetProcAddress call (comparable to
dlopen and dlsym in Unix). Since in this case the address
of a given function is computed at runtime, if the library
is unprotected, the redirection of this function can only be
made at runtime as well.

We leave stubs in the Springboard section which can
be filled at run-time. For this, write permission has to be
given for the page containing the stub, but only for the time
of the update. The run-time stub generation is carried out
by a special function which wraps GetProcAddress. As an
exception from the above described redirection technique
for imported functions, GetProcAddress is redirected to our
wrapper function, as depicted in Figure 8. This is possible
because the function GetProcAddress has to be imported.

to the wrapper. The wrapper will call

The stub code for GetProcAddress, i.e. gpa sb, does not
jump back to the original GetProcAddress function directly,
but
the original
GetProcAddress function, create a new stub for the returned
function pointer in one of the blank slots in the Springboard,
and return the pointer to the newly created stub instead of the
original function pointer. Only the page containing this stub
in the Springboard is writable for the time of this update.
This way all function pointers retrieved by GetProcAddress
are redirected to the Springboard section and 8-byte aligned.
3) Non-exported-function pointers: The overwhelming
majority of external functions are called either through
imports or resolved at runtime via GetProcAddress, i.e., they
are exported by an external module.

However, occasionally an external function that is not
exported by any external module can also be called, such
as through the vtable of an object
that exported by an
unprotected library. Since this function pointer is never
redirected to a Springboard stub anywhere, it will fail the
check in the protected module.

4) Return to unprotected module: It is also possible that
a protected function has to return to an unprotected module,
e.g. when a function is exported by the protected module and
invoked by the unprotected module. When the invocation
finishes, the protected function tries to return to unprotected
module, and then triggers a false alarm.

We handle these rare cases of 3) and 4) by running
BitCover on all libraries which can possibly be loaded by
a hardened module, but cannot be protected (e.g. Win-
dows system DLLs). The same algorithm described in
Section IV-B is used to collect all valid indirect transfer
targets. Instead of using this information for instrumenting
the binary, a hash table is built from the valid code pointers.
When the error handler is triggered at run-time, in case of a
failed check, this hash table is looked up as a final chance
to validate the target.

If the target being looked up is not in the hash table,
the lookup procedure will terminate the process. Otherwise,
the error handler will jump back to the original control
flow. To jump back, the error handler for each instrumented

validation is different. Each error handler saves registers
before calling the common hash table lookup procedure, and
then restores registers after the lookup and jumps back.

This hash table lookup scheme provides the same level
of protection as the previous alignment-based checking. We
also emphasize that since the vast majority of unprotected
targets are already covered by the first two categories (im-
port tables and GetProcAddress), the hash table is seldom
used and thus the introduced overhead is negligible. As
our experiments show, the hash tables for applications in
SPEC2000 [43] and SPEC2006 [44] are never looked up.

When all

the involved modules can be rewritten and
protected by CCFIR, none of these compatibility features
or their overhead are required. However when that is not
possible, CCFIR can still be applied to a single module,
which can work with other un-hardened modules. This
feature allows the incremental deployment of the protection
scheme, which we identified as an important requirement of
practicality.

E. Security Enforcement and Randomization

BitRewrite enforces that indirect call/jump instructions
can only jump to function stubs in the Springboard. Re-
turn instructions are constrained to jump to return address
stubs in the Springboard, and normal return instructions
are prohibited from jumping to sensitive return stubs. Thus
it is impossible for an attacker to inject a jump into the
middle of an instruction, or to an instruction in the middle
of a basic block. This greatly reduces the scope for attack
techniques based on stringing together small code snippets
such as ROP gadgets. However a buffer overflow could still,
for instance, allow an attacker to replace a function pointer
with a different legal function pointer, if the attacker guessed
its value or caused it to be leaked [45][46][47].

The first countermeasure is to harden sensitive functions.

These include:
• system and the execl, execv family of functions
in msvcrt.dll and WinExec, CreateProcess in ker-
nel32.dll. They can be used to execute a file or create a
process.

• The LoadLibrary and GetProcAddress functions

in kernel32.dll which can retrieve function addresses.

• memcpy and other memory operation functions.
• The VirtualProtect and VirtualAlloc family of

functions which can disable memory page protections.
• The fopen and CreateFile families of functions.
• The longjmp function. It

is the key to performing
branching in Turing-complete return-to-libc attacks [28].
• Other similar functions in application-specific libraries.

We suggest that these sensitive functions should only be
used via direct calls, and CCFIR raises an alert if they
are called indirectly. Thus we can assume that a binary
hardened by CCFIR has no function pointer stubs in the

568

Springboard for sensitive functions, and indirect jumps can-
not target them. Similarly, CCFIR’s prohibition of normal
return instructions from returning into the middle of sensitive
functions prevents attackers from accessing parts of their
functionality.

The second countermeasure is to introduce randomization.
Unlike other recent work [18][31], CCFIR randomizes each
stub in the Springboard at load-time. In particular, an extra
section is introduced in the PE file to record all redirected
stubs’ addresses, similar to the relocation table. This section
will only be used by the loader, and will not be mapped
into the process’s address space. So, attackers cannot steal
redirected stubs from this extra section.

With this extra section,

the loader can reorder those
redirected stubs in the Springboard. The stubs are randomly
moved to new addresses within the Springboard. And all
references to the stubs are updated accordingly. This load-
time reordering usually is very fast, as shown in Section V.
In our prototype, this load-time reordering is done by
custom bootstrap code planted in the executable. In the
future, this could be done using the loader.

The randomization introduced here is an orthogonal layer
of protection from the previous CFI-style checking. Even
if the randomization is totally disclosed, the original 3-ID
CFI still exists. Moreover, the location of each stub in the
Springboard is virtually independent. Attackers need a very
targeted disclosure (e.g., the stub for system) to launch an
attack, in contrast to ASLR where attackers can learn the
base address of a whole module and reveal all targets.
F. BitVerify

Separate verification provides an independent check of
whether the target obeys specified security policies. The
CCFIR verifier, BitVerify, checks whether a given binary
conforms to the following rules:
• Any executable section whose 27th bit is zero is a Spring-

board section.

• Code stubs in the Springboard section are all aligned.
• Dynamic checks have been inserted before all indirect

call/jmp/ret instructions that should not be skipped.

• Function pointers that should not be skipped have all been

rewritten and redirected to the Springboard section.

• All call instructions have been rewritten to make sure the
pushed return address points to the Springboard section.
These rules together guarantee that all indirect call/jmp
and ret
instructions in the executable can only flow to
valid code entries. Based on the results from BitCover,
BitVerify can get all valid code entries and indirect control
flow transfer instructions. Also, the Springboard section and
stubs in it are all identified. So this validation process is
straightforward and fast.

As required, BitVerify can also check extra require-
sensitive API
targets, as dis-

ments. For example, we can prohibit
VirtualProtect() from being legal
cussed in Section VI-A.

60
50
40
30

8

6

4

2

0

)
s
d
n
o
c
e
s
(
 
e
m

i

i
t
 
s
s
y
a
n
a

l

 
 
 
 

i

p
z
g

 
 
 
 
 
r
p
v

 
 
 
 
 
c
c
g

 
 
 
 
 
f
c
m

 
 
y
t
f

a
r
c

 
 
r
e
s
r
a
p

 
 
 
 
 

n
o
e

 
k
m
b
l
r
e
p

 
 
 
 
 

p
a
g

 
 
x
e
t
r
o
v

 
 
 

2
p
z
b

i

 analysis time (seconds)

 performance overhead

8%

6%

4%

2%

0%

d
a
e
h
r
e
v
o
 
e
c
n
a
m
r
o
f
r
e
p

 
c
e
r
e
c
a

f

 
 
 
s
a
c
u

l

 
 
 

d
3
a
m

f

k
c
a
r
t
x
s

i

 
 
 
 
i
s
p
a

 
 
 
 
 
t
r
a

 
 
 
 

a
s
e
m

 
 

e
k
a
u
q
e

 
 
 
 

p
m
m
a

 

i

e
s
w
p
u
w

 
 
 
 

i

m
w
s

 
 
 

d
i
r
g
m

 
 
 

l

u
p
p
a

 
 
l

e
g
a
g

l

 
 
 
f
l

o
w

t

 
 
 

i

p
z
g

 
 
 
 
r
p
v

 
 
 
 
c
c
g

 
 
 
 
f
c
m

 
y
t
f

a
r
c

 
r
e
s
r
a
p

 
 
 
 

n
o
e

k
m
b
l
r
e
p

 
 
 
 

p
a
g

 
x
e
t
r
o
v

 
 

2
p
z
b

i

 
f
l

o
w

t

 
 

i

e
s
w
p
u
w

 
 
 
 
 

i

m
 
w
s

 
 
 
 

 
 
 
 

d
i
r
g
m

l

u
p
p
a

 
 
 
 
 

a
s
e
m

 
 
 
l

e
g
a
g

l

 
 
 
 
 
 
t
r
a

 
 
 

e
k
a
u
q
e

 
 
c
e
r
e
c
a

f

 
 
 
 
 

p
m
m
a

 
 
 
 
s
a
c
u

l

 
 
 
 

d
3
a
m

f

 
k
c
a
r
t
x
s

i

 
 
 
 
 
i
s
p
a

Figure 9: Performance of BitCover and BitRewrite.

Figure 10: Performance overhead brought by CCFIR.

V. EVALUATION

We implement a prototype of CCFIR for x86 PE executa-
bles on the Windows platform. In this C++ implementation
prototype, BitCover uses an open source disassembler library
Udis86 [48] to parse x86 instructions. In addition the 8K
LOC of Udis86, BitCover and BitVerify take about 5k LOC,
while BitRewrite takes another 5k LOC and an additional
custom PE file parser takes 2k LOC.

We test CCFIR with the SPEC CPU2000 (consisting of
SPECint2000 and SPECfp2000) benchmark binaries [43],
SPECint2006 [44] and several COTS binaries including Fire-
fox 3.6 (denoted as FF3) and Internet Explorer 6 (denoted
as IE6)3 , to evaluate its overhead and protection.
A. Performance

SPECint2000 consists of 12 applications written in
C/C++, while SPECfp2000 consists of 4 applications written
in C/C++ and 10 in Fortran. We compile all these 16 C/C++
applications with Microsoft Visual Studio 2010 (abbreviated
MSVC2010). For the 10 applications written in Fortran
we use the GNU Fortran compiler (distributed with the
MinGW port of GCC). Because CCFIR can also protect
the return address,
the buffer security check (/GS flag)
provided by MSVC2010 and the stack smashing protection
(-fno-stack-protector) by GCC are turned off. For
each application, all modules are statically linked together
in order to get the approximate performance overhead of
applying CCFIR to the whole system. The experiments are
performed on a Windows 7 32-bit system, with an Intel
Core2 Duo CPU at 3.00GHz.

Then CCFIR is used to automatically disassemble and
rewrite all 26 benchmark binaries. We compare the function
pointer information determined by BitCover with the symbol
information from the source code, and confirm that BitCover
has no false positives or false negatives when parsing the
executables. The final binaries rewritten by BitRewrite are

3For newer browsers, newer OS is needed. But it is difficult to replace
modules in newer OS. In addition, there are few public available exploits
for newer browsers. So, we chose two old browsers as a benchmark here.

569

then run 9 times. The SPEC harness scripts check that the
hardened applications exhibit the same behavior and output
as their original counterparts.

For

the browser FF3 and IE6,

two core modules
xul.dll and mshtml.dll are hardened separately,
whereas other modules are left intact, to evaluate incremental
deployment. The module xul.dll in FF3 is very large
(more than 11MB) and has more than 67,000 functions.
CCFIR hardens it automatically without any problems. The
module mshtml.dll in IE6 is also fairly large, 3MB with
more than 15,000 functions. While BitCover identified a few
hundreds of suspects, an expert can tag code entries quick-
ly. These experiments are conducted in a virtual machine
running Windows XP SP3 with 512M memory and 1 core
CPU.

Because CCFIR currently does not support dynamically
generated code,
the JIT (just-in-time compiler) browser
option is turned off in the hardened browsers. (To provide
a similar protection, the JIT compiler should generate code
obeying the same restrictions as CCFIR. In addition, the JIT
should protect the generated code from tampering, which is
out of the scope of CCFIR.) We check that the hardened
browsers work fine and can visit popular websites.

1) Performance of Static Analysis: Figure 9 shows the
performance of BitCover and BitRewrite when analyzing the
SPEC CPU2000 benchmark. Only three of the benchmark
binaries, gcc, perlbmk and mesa, take more than 7
seconds. The other 23 applications take 1.8 seconds on
average. The analysis time is positively correlated with the
file size (especially the code segment size), the count of
function pointers and indirect call/jmp/ret instructions. For
example, gcc is 1,200KB large and takes 63 seconds, while
mgrid with 70KB takes only 0.14 seconds.

We also evaluate BitVerify’s performance. Experimental
results show that it can also verify binaries quickly. It takes
about 20 seconds to verify the 1.2MB gcc, about 37 seconds
for the 11MB xul.dll in FF3, and less than 10 seconds
for other programs in the benchmark suite.

It is worth noting that this static analysis overhead is

#fp

#imp #GPA #call

77
85
1000
73
88
78
1546
924
758
164
69
81

7
7
7
7
585
11
73
69
7
181
7
7
13
7

20
20
26
20
23
20
28
39
22
23
20
20

4
4
4
4
22
4
20
19
4
20
4
4
4
4

3
3
3
3
3
3
3
3
3
3
3
3

0
0
0
0
3
0
3
3
0
3
0
0
0
0

976
1578
12185
896
2135
1600
4391
7017
9991
3429
826
1385

127
82
104
118
8513
534
895
862
213
2014
98
1341
667
372

#ret

430
768
5628
392
930
751
2325
3229
2672
1715
367
674

31
15
21
27
3539
142
406
381
50
901
20
438
208
98

106
110
263
105
114
114
381
203
1352
124
103
109

35
30
31
29
495
62
105
103
55
132
43
77
92
40

App.

SPECint2000
gzip
vpr
gcc
mcf
crafty
parser
eon
perlbmk
gap
vortex
bzip2
twolf

SPECfp2000
wupwise
swim
mgrid
applu
mesa
galgel
art
equake
facerec
ammp
lucas
fma3d
sixtrack
apsi

Browsers
mshtml.dll
xul.dll

171
190
612
173
228
173
316
419
181
213
171
183

48
44
44
42
345
58
174
154
66
178
55
72
72
54

84.30
66.00
38.33
31.90
43.03
93.23
57.50
64.47
43.30
74.33
68.87
107.00

171.00
347.00
588.00
484.00
72.87
265.00
31.40
46.50
239.00
91.87
151.00
200.00
425.00
351.00

Avrg: 3.6385%
2.6097%
0.0001%
4.0870%
0.2089%
1.0845%
5.7562%
5.2174%
8.6002%
7.2363%
6.2334%
3.0978%
0.0000%

86.50
66.00
39.90
31.97
43.50
98.60
60.50
70.01
46.43
78.97
71.00
107.00

Avrg: 0.5855%
1.7544%
0.0000%
0.0000%
0.0000%
3.9799%
0.0000%
0.1060%
0.7168%
0.0000%
1.8143%
-0.6623%
0.5000%
0.0786%
0.0000%

174.00
347.00
588.00
484.00
75.77
265.00
31.43
46.83
239.00
93.53
150.00
201.00
425.33
351.00

2484
4437
42884
1791
7483
4400
10366
30949
20455
13408
1824
3987

255
116
161
182
21696
1515
1874
1710
826
5039
129
4161
3979
1126

78,676
273,437

0
0
0
0
0
0
0
0
0
0
0
0

0
134
166
172
0
952
0
0
775
0
0
0
3312
878

0
0

0
0
0
0
0
0
0
0
0
0
0
0

0
0
0
0
0
0
0
0
0
0
0
0
0
0

0
0

101
231
1181
80
290
159
440
605
439
488
91
262

67
48
49
165
531
281
89
93
127
217
121
1429
1463
201

140
301
1642
116
368
226
618
829
639
648
131
332

83
61
63
181
681
317
129
129
150
279
127
1633
1618
236

2,995
11,498

4,594
15,620

Table II: Statistical data of CCFIR when applying to applications

redirected fp/ret addr

validated inst

optimiz.
#skipped
fp/import

original

run
time

performance

new
run
time

overhead

#gadgets

file size (KB)

original

new

valid

original

new

modifications

#indirect
call/jmp

1,526
145,224

139
283

21
34

64,662
262,079

10,452
55,025

15,344
65,359

29,557
17,273

offline and does not influence the runtime performance.

2) Performance of Load-Time Randomization:

In our
prototype, the load-time randomization is done by bootstrap
code placed in the protected executable. Results show that
the load time randomization is very fast.

For mshtml.dll in IE6, there are less than 216 code
stubs in the Springboard section, and each stub occupies
less than 16 bytes. The whole memory movement when
reordering is less than 16 · 216 = 1M. And the evaluated
load time is about 16 milliseconds. Similarly, xul.dll has
less than 219 stubs and takes about 117 milliseconds.

3) Runtime Overhead on SPEC CPU2000: All the 26
applications in the SPEC CPU2000 benchmark are hardened
by CCFIR. Then the median run time over 9 trials is
evaluated. Figure 10 shows the performance overhead caused
by CCFIR, while Table II shows the detailed run time data.
When protecting targets of all indirect call/jmp/ret instruc-
tions, CCFIR introduces an overhead of 3.6% on the aver-
age over the SPECint2000 benchmark and only 0.59% for
SPECfp2000. The largest overhead is 8.6% on perlbmk,
an interpreter in which every opcode is implemented with an
indirect jump. For lucas, there is a slight speed-up, maybe
due to increased code alignment.

On SPECint2006, the average overhead is about 4.2%.
For space reasons, the detailed data are not listed here.

Compared with other protections, such as [13][49], CCFIR
is capable of protecting all binaries in the SPEC 2000/2006
benchmarks, with a reasonable overhead.

Statistics: Table II also lists the modifications made by
CCFIR to the SPEC CPU2000 applications and 2 browsers.
The columns under redirected fp/ret_addr in
the table represent the count of code entries redirected by
CCFIR, including hard-coded function pointers, imported
function pointers, pointers returned by GetProcAddress and
return addresses pushed by call instructions. Taking gcc as
an example, 1000 hard coded function pointers and 26 im-
ported functions are redirected, and GetProcAddress is called
3 times. Moreover, there are 12185 call instructions in the
whole application. All these 13214 (= 1000+26+3+12185)
code entries are redirected by CCFIR.

The

columns under validated instructions
record the count of indirect call/jmp/ret instructions which
are validated by CCFIR. For gcc, there are only 263 indirect
call/jmp instructions and 5628 ret instructions. So, targets of
5891 (= 263+5628) instructions are validated by CCFIR.

Performance Analysis: As discussed in Section IV-C,
BitRewrite skips redirecting some function pointers and
skips instrumenting checks for some indirect jumps. The
column under optimiz. in Table II counts how many
function pointers are skipped. For gcc, 612 function point-

570

ers are skipped, while only 1029 (=1000+26+3) pointers
are redirected. So, about 38% function pointers are not
redirected and thus the runtime overhead are greatly reduced.
For the original CFI, the attached ID (a potentially s-
low prefetchnta instruction) will always be executed
in direct control transfers. But for CCFIR, there are no
extra overheads in this case. In addition, the direct control
transfers cover most of the control transfers in applications.
And thus, CCFIR is much faster than original CFI.

Return instructions play a large part in CCFIR’s overall
performance. We also repeated our measurements (detailed
data omitted) in a mode in which CCFIR protects only
indirect call/jmp but not ret instructions. In this configuration
the overhead is 0.79% for SPECint2000, much smaller than
the 3.6% overhead when ret instructions are also protected.
The 10 Fortran applications in SPECfp2000 have few
indirectly used function pointers and imported functions, so
the overhead is much smaller than applications written in
C/C++, as those in SPECint2000.

4) Runtime Overhead on Real World Browsers.: One core
module each of FF3 and IE6 is hardened by CCFIR sepa-
rately, as described above. We attempted to test each browser
against the Sunspider [50] and Google V8 benchmarks [51].
Unfortunately the benchmarks do not support IE6, so
we only report results for Firefox (JIT is turned off).
The overhead caused by CCFIR was small. When testing
with Sunspider, the run time increases from 2130.7ms to
2150.3ms. When testing with the Google V8 benchmark,
the score drops from 369 to 361 (larger results are better).
B. Protection Effects

1) Eliminating ROP Gadgets: CCFIR can be used to
defend against ROP attacks because it will validate ret
instructions’ targets. Only instructions directly following a
call site can be the targets of ret instructions. As a result,
after applying CCFIR on the target binary, ROP gadgets that
do not directly follow call sites are unusable, including any
that start from the middle of legal instructions.

To evaluate this protection we count

the number of
gadgets in our benchmark applications. First, we use the tool
Mona [52] to count the gadgets in the original applications
and the rewritten applications. As shown in the columns
under #gadgets in Table II, after hardening, Mona only
finds gadgets in 7 out of the 26 applications, but none of
these gadgets will pass the validation of CCFIR.

2) Randomization Entropy: CCFIR’s load-time random-
ization makes it hard to guess the address of a target function
or a return site, and thus raises the bar for attackers to hijack
the control flow, including return-to-libc and ROP attacks.
Our Springboard’s size is 128MB (i.e. 227). All code stubs
are randomized within the Springboard. Each stub takes less
than 16 bytes and is aligned to 8 or 16 bytes.

This degree of randomization makes a brute-force search
infeasible. For each stub, there are 223 (=227/16) possible

Table III: Real World Exploit Samples Prevented by CCFIR.

ID
CVE-2011-0065
CVE-2010-0249
CVE-2010-3962
CVE-2011-1260
CVE-2005-1790
CVE-2008-0348
CVE-2010-5081
OSVDB-83362
CVE-2007-1195
OSVDB-82798

App
FF 3
IE 6
IE 6
IE 6
IE 6
coolplayer
RM-MP3
urlhunter
XM ftp
ComSndFTP

Vul Type
Use After Free
Use After Free
Use After Free
Mem. Corrupt
Mem. Corrupt
Stack Overflow
Stack Overflow
Stack Overflow
Format String
Format String

Vul Module
xul.dll
mshtml.dll
mshtml.dll
mshtml.dll
mshtml.dll
core exe
core exe
core exe
core exe
core exe

Protected
yes
yes
yes
yes
yes
yes
yes
yes
yes
yes

positions after load-time randomization. To chain k target
gadgets together, the attacker has to probe 223 · (223 − 1) ·
··· · (223 − k + 1) times in the worst case.

3) Protection against Real World Exploits: We also chose
10 publicly available exploits from Metasploit [53] against
FF3, IE6 and 5 other applications. These experiments are
performed in a virtual machine running Windows XP SP3
within a separate experiment network. Table III shows the
10 vulnerabilities attacked by exploits we used.

Taking CVE-2011-0065 as an example, this vulnerability
exists in Firefox 3.x before 3.6.17. It is a use-after-free
vulnerability which can cause arbitrary code execution, when
exploited by techniques such as heap spray [54].

After hardening the vulnerable module xul.dll with
CCFIR, we drive Firefox to access the attack URL again,
and the error handler added by CCFIR is triggered. The
remaining 9 exploits, which target IE6 and other 5 applica-
tions, are also prevented by CCFIR in a similar manner.

VI. DISCUSSION

A. Possible Attacks

To attack CCFIR, an attacker may:

(a) forge a valid target.
(b) change memory pages’ protection attributes to change

instructions directly or to add forged targets.

(c) use a dangerous target that is used by the program.
(d) jump to valid targets or chain them to launch attacks.
For (a), the attacker has to use a page which is writable
the same time. For modern programs

and executable at
protected by DEP, this depends on the attack (b).

Some APIs are inherently dangerous (e.g. WinExec), or
are dangerous because they can disable page protections (e.g.
VirtualProtect in the Virt* family). These functions
are rarely used through indirect calls in regular applications.
CCFIR raises an alert if such functions are called indirectly.
CCFIR randomizes their entry addresses to make it even
harder for attackers to guess or steal them, providing some
protection before developers provide a patch.

If a program calls Virt* functions directly and only
uses constant flProtect or flNewProtect arguments
which do not make the page executable, it will be immune
to such attacks as (a) or (b) after being hardened by

571

CCFIR. If a program calls Virt* functions to make a
page executable for JIT, attackers still have a chance to
utilize these functions, but again they need to penetrate the
randomization generated by CCFIR. We still suggest that the
program carefully check the arguments before such calls.

For (d), attackers’ abilities are greatly constrained by
CCFIR. As discussed in Section IV-E, indirect call/jmp are
enforced to flow to valid and non-sensitive function entry
points. In addition, normal return instructions cannot jump
into sensitive functions or any function entries. So,
the
Turing-completeness of return-to-libc attacks is broken. Be-
sides, ROP attacks that to jump to the middle of instructions
or basic blocks become impossible, while chaining gadgets
also becomes much more difficult.

When CCFIR is only applied to parts of a program,
attackers still have a chance to modify pointers flowing
to unprotected external modules. In this situation, ASLR
and other memory-allocation-based protection methods will
provide valuable defense and make attackers spend much
more effort to find the vulnerable locations. But of course
we still recommend applying CCFIR to the whole system to
provide the best protection.
B. Race Condition of Return Address

The code sequence that CCFIR uses to validate a return
address has a TOCTTOU (time of check to time of use) race
condition in a multi-threaded program. CCFIR checks the
value of [esp] and then executes ret in the next instruction,
but the return address is stored in memory in the interim,
where it could be modified by another thread.

The race could be avoided by storing the value in a
register, but this would have a substantial performance penal-
ty because it would disrupt the CPU’s branch prediction.
(Modern CPUs use a private shadow stack to predict the
targets of return instructions, while other indirect jumps use
a less sophisticated prediction mechanism.)

This race condition affects any other return protection
scheme that checks the return value in-place,
including
MSVC’s /GS, GCC’s SSP, and PittSFIeld [38]. However the
time window in the race is extremely small, so in practice
the odds of a successful attack will be small. To avoid the
possibility of repeated attacks within a process, CCFIR’s
validation will terminate a process immediately if it detects
an illegal return address.

VII. CONCLUSION

In this paper, we propose a new approach called CCFIR
to ensure that indirect control transfers jump only to known
targets. It can be used to enforce CFI, which provides a solid
base for software protection. It can block various attacks
against control transfers, including most ROP attacks.

CCFIR can be applied through binary rewriting on exe-
cutables generated by modern compilers. Its runtime over-
head is low (about 3.6% measured by SPECint2000). CC-
FIR’s techniques can also be used directly in the compilation
process to provide protections for software.

572

ACKNOWLEDGMENTS

This research was supported in part by the National
Natural Science Foundation of China under the grant No.
61003216 and 61003217; the Chinese NDRC InfoSec Foun-
dation under Grant No.[2010]3044; the NSF grants 0842695,
0831501, CCF-0424422 and CNS-0831298; the ONR grants
N000140911081 and N000140710928; an AFOSR grant
FA9550-09-1-0539; and a DARPA award HR0011-12-2-
005.

REFERENCES

[1] S. Andersen and V. Abella, “Data Execution Prevention:
Changes to Functionality in Microsoft Windows XP Service
Pack 2, Part 3: Memory Protection Technologies,” http://
technet.microsoft.com/en-us/library/bb457155.aspx, 2004.

[2] PaX Team, “PaX address

space layout

randomization

(ASLR),” http://pax.grsecurity.net/docs/aslr.txt, 2003.

[3] S. Bhatkar, D. C. DuVarney, and R. Sekar, “Address obfusca-
tion: an efficient approach to combat a broad range of memory
error exploits,” in USENIX Security Symposium, 2003.

[4] C. Cowan, C. Pu, D. Maier, J. Walpole, P. Bakke, S. Beat-
tie, A. Grier, P. Wagle, Q. Zhang, and H. Hinton, “Stack-
Guard: Automatic adaptive detection and prevention of buffer-
overflow attack,” in USENIX Security Symposium, 1998.

[5] M. Frantzen and M. Shuey, “StackGhost: Hardware facilitated

stack protection,” in USENIX Security Symposium, 2001.

[6] Microsoft Visual Studio 2005,

ex-
ception handlers,” http://msdn.microsoft.com/en-us/library/
9a89h429%28v=vs.80%29.aspx.

“Image has

safe

[7] J. Pincus and B. Baker, “Beyond stack smashing: Recent
advances in exploiting buffer overruns,” IEEE Symposium on
Security and Privacy, 2004.

[8] G. F. Roglia, L. Martignoni, R. Paleari, and D. Bruschi, “Sur-
gically returning to randomized lib(c),” in Annual Computer
Security Applications Conference (ACSAC), 2009.

[9] M. Abadi, M. Budiu, U. Erlingsson, and J. Ligatti, “Control-
flow integrity,” in ACM Conference on Computer and Com-
munications Security (CCS), 2005.

[10] H. Shacham, “The geometry of innocent flesh on the bone:
Return-into-libc without function calls (on the x86),” in
ACM Conference on Computer and Communications Security
(CCS), 2007.

[11] E. Buchanan, R. Roemer, H. Shacham, and S. Savage, “When
good instructions go bad: generalizing return-oriented pro-
gramming to RISC,” in ACM Conference on Computer and
Communications Security (CCS), 2008.

[12] S. Checkoway, L. Davi, A. Dmitrienko, A. Sadeghi,
H. Shacham, and M. Winandy, “Return-oriented program-
ming without returns,” in ACM Conference on Computer and
Communications Security (CCS), 2010.

[13] B. Zeng, G. Tan, and G. Morrisett, “Combining control-
flow integrity and static analysis for efficient and validated
data sandboxing,” in ACM Conference on Computer and
Communications Security (CCS), 2011.
´U. Erlingsson, M. Abadi, M. Vrable, M. Budiu, and G. Nec-
ula, “XFI: Software guards for system address spaces,” in
Symposium on Operating Systems Design and Implementation
(OSDI), 2006.

[14]

[15] B. Schwarz, S. Debray, and G. Andrews, “Disassembly of
executable code revisited,” in Working Conference on Reverse
Engineering, 2002.

[16] M. Prasad and T.-c. Chiueh, “A binary rewriting defense
against stack based buffer overflow attacks,” in USENIX
Annual Technical Conference, 2003.

[17] Hex-Rays SA, “IDA Pro: a cross-platform multi-processor
http://www.hex-rays.com/

debugger.”

disassembler
products/ida/index.shtml.

and

[18] J. Hiser, A. Nguyen-tuong, M. Co, M. Hall, and J. W. David-
son, “ILR : Where’d my gadgets go,” in IEEE Symposium on
Security and Privacy, 2012.

[19] A. Edwards, A. Srivastava, and H. Vo, “Vulcan: binary trans-
formation in a distributed environment,” Microsoft Research,
Tech. Rep. MSR-TR-2001-50, 2001.

[20] S. Chen, J. Xu, E. C. Sezer, P. Gauriar, and R. K. Iyer, “Non-
control-data attacks are realistic threats,” in USENIX Security
Symposium, 2005.

[21] W. Xu, D. C. DuVarney, and R. Sekar, “An efficient and
backwards-compatible transformation to ensure memory safe-
ty of C programs,” in SIGSOFT Symposium on Foundations
of Software Engineering (FSE), 2004.

[22] D. Dhurjati and V. Adve, “Backwards-compatible array
bounds checking for C with very low overhead,” in Inter-
national Conference on Software Engineering (ICSE), 2006.
[23] S. Nagarakatte, J. Zhao, M. M. Martin, and S. Zdancewic,
“SoftBound: highly compatible and complete spatial memory
safety for C,” in Conference of Programming Language
Design and Implementation (PLDI), 2009.

[24] ——, “CETS: compiler enforced temporal safety for C,” in
International Symposium on Memory Management (ISMM),
2010.

[25] M. Castro, M. Costa, and T. Harris, “Securing software by
enforcing data-flow integrity,” in Symposium on Operating
Systems Design and Implementation (OSDI), 2006.

[26] C. Cowan, S. Beattie, J. Johansen, and P. Wagle, “PointGuard:
Protecting pointers from buffer overflow vulnerabilities,” in
USENIX Security Symposium, 2003.

[27] S. Alexander, “Defeating Compiler-level Buffer Overflow

Protection,” The USENIX Magazine ;login:, Jun. 2005.

[28] M. Tran, M. Etheridge, T. Bletsch, X. Jiang, V. Freeh, and
P. Ning, “On the expressiveness of return-into-libc attacks,”
in International Symposium on Recent Advances in Intrusion
Detection (RAID), 2011.

[29] P. Chen, H. Xiao, X. Shen, X. Yin, B. Mao, and L. Xie,
“DROP: Detecting return-oriented programming malicious
code,” in International Conference on Information Systems
Security, 2009.

[30] L. Davi, A. Sadeghi, and M. Winandy, “ROPdefender: A
detection tool to defend against return-oriented programming
attacks,” in ACM Symposium on Information, Computer and
Communications Security (ASIACCS), 2011.

[31] V. Pappas, M. Polychronakis, and A. D. Keromytis, “Smash-
ing the gadgets: Hindering return-oriented programming using
in-place code randomization,” IEEE Symposium on Security
and Privacy, 2012.

[32] L. Davi, A. Dmitrienko, M. Egele, T. Fischer, T. Holz,
R. Hund, S. N¨urnberger, and A.-R. Sadeghi, “MoCFI: A
framework to mitigate control-flow attacks on smartphones,”
in Network and Distributed System Security Symposium
(NDSS), 2012.

[33] Y. Xia, Y. Liu, H. Chen, and B. Zang, “CFIMon: Detecting
violation of control flow integrity using performance coun-
ters,” in IEEE/IFIP International Conference on Dependable
Systems and Networks, 2012.

573

[34] P. Philippaerts, Y. Younan, S. Muylle, F. Piessens, S. Lach-
mund, and T. Walter, “Code pointer masking: hardening
applications against code injection attacks,” in Conference
on Detection of Intrusions and Malware, and Vulnerability
Assessment (DIMVA), 2011.

[35] T. Bletsch, X. Jiang, and V. Freeh, “Mitigating code-reuse
attacks with control-flow locking,” in Annual Computer Se-
curity Applications Conference (ACSAC), 2011.

[36] Z. Wang and X. Jiang, “HyperSafe: A lightweight approach
to provide lifetime hypervisor control-flow integrity,” in IEEE
Symposium on Security and Privacy, 2010.

[37] R. Wahbe, S. Lucco, T. Anderson, and S. Graham, “Efficient
software-based fault isolation,” in Symposium on Operating
Systems Principles (SOSP), 1994.

[38] S. McCamant and G. Morrisett, “Evaluating SFI for a CISC

architecture,” in USENIX Security Symposium, 2006.

[39] B. Yee, D. Sehr, G. Dardyk, J. B. Chen, R. Muth, T. Ormandy,
S. Okasaka, N. Narula, and N. Fullagar, “Native Client: A
sandbox for portable, untrusted x86 native code,” in IEEE
Symposium on Security and Privacy, 2009.

[40] P. Akritidis, C. Cadar, C. Raiciu, M. Costa, and M. Castro,
“Preventing memory error exploits with WIT,” in IEEE
Symposium on Security and Privacy, 2008.

[41] MSDN online

library,

“Microsoft Portable Executable
(PE) and Common Object File Format (COFF) Specifi-
cation,” http://msdn.microsoft.com/en-us/windows/hardware/
gg463119.aspx.

[42] C. Cifuentes and M. Van Emmerik, “Recovery of jump table
case statements from binary code,” in International Workshop
on Program Comprehension, 1999.

[43] J. L. Henning, “SPEC CPU2000: Measuring CPU Perfor-

mance in the New Millennium,” Computer, Jul. 2000.

[44] ——, “SPEC CPU2006 benchmark descriptions,” SIGARCH

Comput. Archit. News, vol. 34, pp. 1–17, Sep. 2006.

[45] R. Strackx, Y. Younan, P. Philippaerts, F. Piessens, S. Lach-
mund, and T. Walter, “Breaking the memory secrecy assump-
tion,” in European Workshop on System Security (EUROSEC),
2009.

[46] D. Blazakis, “Interpreter exploitation,” in Workshop on Of-

fensive Technologies (WOOT), 2010.

[47] F. J. Serna, “The info leak era on software exploitation,” in

Blackhat USA, 2012.

[48] V. Thampi, “Udis86 disassembler library for x86,” http:

//udis86.sourceforge.net/.

[49] R. Wartell, V. Mohan, K. W. Hamlen, and Z. Lin, “Binary
stirring: Self-randomizing instruction addresses of legacy x86
binary code,” in ACM Conference on Computer and Commu-
nications Security (CCS), 2012.

[50] WebKit.org, “SunSpider JavaScript Benchmark,” http://www.

webkit.org/perf/sunspider/sunspider.html.

[51] Google Inc., “V8 Benchmark Suite - version 6,” http://v8.

googlecode.com/svn/data/benchmarks/v6/run.html.

[52] Corelan Team, “MONA: a PyCommand plugin for Immunity

Debugger,” http://redmine.corelan.be/projects/mona, 2012.

[53] Metasploit Open Source Commitment, “Metasploit Penetra-
tion Testing Software & Framework,” http://metasploit.com.
[54] M. Daniel, J. Honoroff, and C. Miller, “Engineering heap
overflow exploits with JavaScript,” in Workshop on Offensive
Technologies (WOOT), 2008.

|
cset13-allodi.pdf,|MalwareLab: Experimentation with Cybercrime Attack Tools

Luca Allodi

DISI - University of Trento.

Via Sommarive 5, Povo (TN), Italy

Vadim Kotov

DISI - University of Trento.

Via Sommarive 5, Povo (TN), Italy

Fabio Massacci

DISI - University of Trento.

Via Sommarive 5, Povo (TN), Italy

Abstract
Cybercrime attack tools (i.e. Exploit Kits) are reportedly
responsible for the majority of attacks affecting home
users. Exploit kits are traded in the black markets at
different prices and advertising different capabilities and
functionalities. In this paper we present our experimental
approach in testing 10 exploit kits leaked from the mar-
kets that we deployed in an isolated environment, our
MalwareLab. The purpose of this experiment is to test
these tools in terms of resiliency against changing soft-
ware configurations in time. We present our experiment
design and implementation, discuss challenges, lesson
learned and open problems, and present a preliminary
analysis of the results.

built for this purpose and maintained at the University of
Trento, Italy. We discuss our design and implementation
methodology, and present the results of our analysis. We
also discuss strengths and weaknesses of our design, and
potential flaws of our implementation.

Section 2 and 3 give a brief background on exploit
kits and discuss related work respectively.
In Section
4 we report a first (failed) experiment design. The pa-
per then proceeds with describing a second experiment
design (Section 5) and implementation (Section 6). We
then discuss in Section 7 preliminary results of the exper-
iment. Section 8 presents open points and challenges we
identify in our design and implementation, and Section 9
concludes the paper.

1

Introduction

2 Background on exploit kits

In the cybercrime underground markets attack tools and
software artefacts are constantly traded [3, 10]; these are
responsible, reportedly, for about two thirds of user in-
fections worldwide [8]. A class of these tools, namely
“Exploit Kits”, seem to be particularly popular among
cyber-criminals, to the point that both industry [10] and
academia [3] got recently interested in the phenomenon.
An exploit kit is a software tool used by cyber crimi-
nals to deliver drive-by-download attacks. It is an HTTP
server-side application, that, based on request headers,
returns a page with an appropriate set of exploits to the
victim computer. Exploit kits are traded in the black mar-
kets and explicitly advertise the vulnerabilities they ex-
ploit. We performed a systematic exploration of under-
ground and public channels and gathered more than 30
exploit kits, spanning from 2007 to 2011.

In this paper we describe our experimental approach
to test exploit kits in terms of resiliency in time and
namely for how long an exploit kit would work consid-
ering the pace of software evolution. We conduct our ex-
periments in an isolated environment, the MalwareLab,

Exploit kits’ main purpose is to silently download and
execute malware on the victim machine by taking ad-
vantage of browser or plugin vulnerabilities. Errors in
applied programming interfaces or memory corruption
based vulnerabilities allow an exploit to inject a set of in-
structions (shellcode) into the target process. Shellcode
on its turn downloads an executable malware on the vic-
tim’s hard drive and executes it. The executable installed
on the target system is completely independent from the
exploit pack (see [3] for some statistics on the pairings).
Figure 1 depicts the generic scenario of drive-by-
download attack [3, 5]. A victim visits a compromised
web site, from which he/she gets redirected to the exploit
kit page. Various ways of redirection are possible: an
<iframe> tag, a JavaScript based page redirect etc. The
malicious web page then returns an HTML document,
containing exploits, which are usually hidden in an ob-
fuscated JavaScript code. If at least one exploit succeeds,
then the victim gets infected. An exploitation is success-
ful when the injected shellcode successfully downloads
and execute a malicious program on the victim system.

experimental design was therefore to measure, by con-
trolling the experiment for operating system and vulner-
able applications, how successful each exploit was at in-
fecting the vulnerable machine. We assumed that the
state of the machine memory (in particular the browser’s
heap) played a role in the successfulness of the ex-
ploit. We manually set up about 40 vulnerable configura-
tions per operating system (Windows XP and its Service
Packs). In order to be sure to measure only one exploit
at a time and avoid noise in the data, we paid extra at-
tention to the pre-existing vulnerabilities on the system.
For example CVE-2006-0003, a vulnerability widely ex-
ploited by most exploit kits, is a “system vulnerability”
that exploit kits reach through the ActiveX interface of
Internet Explorer 6. If we were testing for, say, a vul-
nerability in Java loaded within the context of Internet
Explorer 6, then we would have had no control above
a possible exploitation of CVE-2006-0003, which might
have created the false impression that the Java vulner-
ability was successfully exploited. Avoiding such con-
figurations required a significant amount of time. In the
example above, we run the vulnerable Java plugin in a
non-vulnerable (to our exploit kits) version of Firefox,
and run multiple tests to be sure that Firefox did not play
any role in the observed exploitation/non exploitation of
the Java vulnerability.

We controlled the state of the browser’s heap mem-
ory by automatically generating a random number of
HTML + JavaScript web pages that were loaded onto the
browser before sending a GET request to the exploit kit
tested in that run. However, we ended up measuring only
exploits that worked with 100% efficacy and exploits that
simply never worked. When we studied this issue in
more detail we found out that drive-by-download attacks
exploit the fact that each browser tab is in fact a sepa-
rate process (or a thread) with its own heap and memory
disposition, so the browsing history does not impact the
success of the attack.

Further investigations on the nature of exploit kits and
the exploits they bundle revealed why the design failed.
The exploits we tested either use Java vulnerabilities that
use internal Java VM resources to download and execute
malware (on which Windows defences have no effect)
or use a heap spray attack. The heap spray attack is an
exploitation technique against vulnerabilities in browsers
and other applications that allocate user data in the pro-
cess heap.
In the case of browser attacks the idea of
heap spray is to allocate hundreds of megabytes of pay-
load in the heap memory (by creating a big number of
JavaScript string variables) and then trigger the memory
corruption vulnerability to redirect the process (that is the
browser/tab/plugin) execution flow to the lower region
heap. The injected shellcode consists of a huge NOP-
sled (a set of NOP-like instructions) followed by the ac-

Figure 1: Scheme of drive-by-download attack

3 Related Work

The infection dynamics enforced by exploit kits are pre-
sented by Provos et al. in [6] and Rajab et al. in [8]. An
overview on the diffusion of exploit kits is also reported
by Grier et al. in [3], where they analyse DNS traffic to-
ward known malware-delivery domains, and by Allodi et
al. in [1], where an analysis of attacks against vulnera-
bilities in the exploit kits is given. These works related
on data recorded “in the wild” and differ therefore from
ours in the experimental approach.

A technical analysis of Exploit kits and their capa-
bilities are outlined by Kotov et al.
in [5]. Details on
heap spray attacks are given in [2], in which different
exploits are analysed. However, no systematic experi-
mental methodology is applied to test those exploits.

An overview of good practices when dealing with
malware experiments is provided by Rossow et al.
in
[9]. Realism, safeness, data categorisation are all top-
ics covered by the authors and considered in our work
as well. On a similar, more active and investigation-
oriented line, Kanich et al.
[4] underline the difficul-
ties of (safely) dealing with the cybercrime environment;
however, while they are addressing multiple interactions
with the underground, we are only measuring one snap-
shot of the underground community in our MalwareLab.

4 A (failed) first experiment design

Exploit kits are advertised on the black markets with a
“self-declared” infection efficacy, expressed as the per-
centage of “incoming users” the costumer may expect to
infect. The advertised ratios are usually around 15-25%
[1]. The frequency and trends of successful exploitations
depends on such factors as an operating system version,
type and version of a browser and its add-ons, presence
of security measures, and general system configuration.
From here we formulated our first question on exploit
kits. Question: How good are their exploits? Our first

2

CompromisedWeb Site1) Visit a compromized web site2) Redirect to an exploit kitExploit Kit3) Visit an exploit kit page4) Return exploits5) Download malwareVictimTable 1: Operating systems and respective release date.
*Configurations are right-censored with respect to the 6 years time window.

Op. system

Windows Xp

Windows Vista

Windows 7

Service Pack

None

1
2
3

1
2

1

None

None

Ysys

2001 - 2007
2002 - 2008
2004 - 2010
2008 - 2013*
2006 - 2012
2008 - 2013*
2008 - 2013*
2009 - 2013*
2011 - 2013*

tual malicious code. Since the heap is now filled with
the shellcode the probability that the instruction pointer
will reach the address where the malware is loaded ap-
proaches 1. We conclude that exploits bundled in exploit
kits are well engineered and designed to work disregard
of the memory state of the victim machine.

5 Design of the Experiment

With the second experiment design we aim at giving a
quantitative answer to the following research question:

Question: How resilient are exploit kits against software
configuration updates?

To answer this, we test exploit kits in a controlled envi-
ronment, our MalwareLab. The core of our design is the
generation of “reasonable” home-system configurations
to test against the infection mechanism and capabilities
of exploit kits. We test those configurations as running
on Windows XP, Windows Vista and Windows 7. Ta-
ble 1 reports versions and release dates of each operat-
ing system and service pack considered (from here on,
system). After an initial phase of application testing on
the selected systems, we fix the life-time of an operating
system to be 6 years for compatibility of software. Ysys
indicates the working interval of each operating system.
For our experiment we selected 10 exploit kits (see
Table 2) out of the 34, leaked from the black markets, we
gathered. Some of them proved to be not fully-functional
or impossible to be deployed (e.g. because of missing
functions). Out of those that were deployable and armed,
we selected 10 according to the following criteria: (a)
popularity of the exploit kit [10]; (b) year of release; (c)
unique functionality (e.g. only one of multiple versions
of the same kit family is selected).

Table 2: List of tested exploit kits

For some exploit kits we could not find the respective release advertisement on
the black markets, and therefore a precise date of release for the product cannot
be assessed. For those (*) we approximate the release date to the earliest mention
of that exploit kit in underground discussion forums and security reports. This
identifies an upper bound of the release date.

#
1
2
3
4
5
6
7
8
9
10

Name

Crimepack
Eleonore

Bleeding Life

Elfiesta

Shaman’s Dream

Gpack
Seo

Mpack
Icepack
Adpack

Version
3.1.3

1.4.4mod

2
1.8
2

UNK
UNK
0.86

platinum

UNK

Release Year

2010
2011
2010
2008*
2009*
2008
2010
2007*
2007
2007*

Table 3: Software versions included in the experiment.
Overall 9 software versions were excluded from the experiment setup because the
corresponding installation package was either not working or we could not find it
on the web.

Software
Mozilla Firefox
Microsoft Internet Explorer
Adobe Flash
Adobe Reader
Java

Versions

# of versions

1.5.0.2 - 17.0.1.0

6-10

9.0.16.0-11.5.502.135

8.0.0-10.1.4

1.5.0.7-7.10.0.0

Total

122
5
54
17
49
247

subject to a number of assumptions that define the cri-
teria themselves. For our experiment to be realistic, we
need to build configurations that are reasonable to ex-
ist at a certain point in time. As an example, we con-
sider unlikely to have Firefox 12, released in April 2012,
installed on the same machine with Adobe Flash 9, re-
leased 6 years earlier in June 2006. We therefore fix a
two-years window that defines which software can co-
exist. The window is based on the month and year of
release of a particular software. Since our oldest exploit
kit is from early 2007, we are testing software only re-
leased in the interval (2005,2013). Table 3 shows the
software versions we consider1.

Figure 2 exemplifies the mechanism to create con-
figurations. The algorithm to generate each config-
uration iterates through all years Ycon f from 2006 to
2013, and chooses at random a version of each soft-
ware (including “no version”, meaning that that soft-
ware is not installed for that configuration) that satisfy
YswRel ∈ [Ycon f − 1,Ycon f ]. For each Ycon f we generate 30
random configurations. Given the construction of YswRel,
we end up with seven windows and therefore 210 con-

5.1 Configuration selection
The automated installation of software configurations on
each machine followed the definition of a criteria to se-
lect software to be installed. As often happens, this is

1We did not include Google Chrome as it was first released halfway
through the timeline considered in our experiment (2008). Introduc-
ing Chrome samples in 2008 would have changed the probability of a
particular software to be selected. In turn, this would make comparing
time windows before and after 2008 statistically biased. We plan to
include Chrome in future experiment designs.

3

Each dot represents a tuple {software, version} that is eligible for selection to be
included in a configuration in a certain time window. The probability distribution
of each configuration is uniform (each tuple has the same likelihood of being
selected). We exemplify the selection mechanism by highlighting the {software,
version} tuples with the respective window colour. Note that because we treat
“no software version” as a “version” in the tuple, software can as well be not
selected for a certain configuration.

Figure 2: Random selection of configurations per soft-
ware with sliding windows of two years.

figurations per system reported in Table 1. However, as
underlined in Section 5, for compatibility reasons each
system has a time window of 6 years starting one year
before its release date. Because we want to measure the
resiliency of exploit kits, we keep the number of config-
urations per year constant (otherwise results would not
be comparable between different runs). This means that
some systems are tested, overall, against a lower number
of configurations than others. For example, Windows XP
Service Pack 1 (2002-2008) will be tested only against
configurations in the time windows{[2006, 2008),[2007-
2009))}2, which gives us 60 configurations. Windows
Vista with no Service Pack (2006-2012) will instead be
tested, for the same reason, with 180 configurations. This
guarantees that each exploit kit is tested for each system
against the same number of configurations per year.

The algorithm iterates through each configuration and
runs it against the available exploit kits. Figure 3 is a
representation of an experiment run for each system. At
each iteration i, we select the configuration con f i.
If
Ycon fi ∈ Ysys, we automatically install the selected soft-
ware on the virtual machine using the “silent install” in-
terface provided by the vendor or by the msi installer. A
configuration install is successful when all software in
that configuration is installed. For detection of unsuc-

2Note that the last year of the time window is not included. For
example, [2006,2008) includes configurations from January 2006 to
December 2007a.

4

This flowchart describes a full experiment run for each system in Table 1.
Configurations are generated in chronological order,
therefore if the first
control on YSys fails, every other successive configuration would as well and the
experiment ends. Snapshots enable us to re-use an identical installation of a
configuration multiple times.

Figure 3: Flowchart of an experimet run.

cessfully installed configurations, see Section 6.3.

When the installation process ends, we take a “snap-
shot” of the virtual machine. Every run for con fi will
restore and use this snapshot. The advantages of this
are twofold: at first we eliminate possible confounding
factors stemming from slightly different configurations,
because only the exploit kit changes; secondly, this is
also faster than re-installing the configuration every time,
which would have considerably stretched the (already
not short) completion time. When all exploit kits are
tested, a new configuration is eligible for selection.

5.2 Data collection
In the course of our experiment we keep track of (a) the
successfulness of the automated installation of a configu-
ration on a victim machine (VICTIM) at any given time;
(b) the successfulness of infection attempts from exploit
kits. This data is stored in two separate tables, Configu-
rations and Infections respectively.

1. Configurations is needed to control for VICTIM
configurations that were not successfully installed; this
way we can correctly attribute (un)successful exploita-
tion to the right set-ups. This is desirable when looking
for infection rates of single configurations or software.

2.

Infections stores information on each particular
configuration run against an exploit kit. We set our in-
fection mechanism to make a call to the Malware Dis-
tribution Server (MDS) each time it is executed on the
VICTIM machine. A “call back” to the MDS can in fact
only happen if the “malware” is successfully executed
on VICTIM. The MDS stores the record in Infections,
alongside (snapshot id,
toolkit version,
machine, IP, date, successful). Exploit kits have an “ad-

toolkit name,

ministrative panel” reporting infection rates [5]. How-
ever, we decide to implement our own mechanism be-
cause (a) it allows us to have more control on the data
in case of errors or unforeseen circumstances; (b) exploit
kits statistics may not be reliable (e.g. developers might
be incentivated in exaggerating infection rates).

To minimise detection [3], some exploit kits avoid at-
tacking the same machine twice (i.e. delivering the attack
the same IP). This behaviour is enabled by an internal
database controlled by the kit, independent from our In-
fections table. In some cases, e.g. when the experiment
run needs to be resumed from a certain configuration,
our Infections table may report un-successful attacks of
an exploit kit, when instead the exploit kit did not delib-
erately deliver the attack in the first place. We therefore
need to control for this possibility by resetting the exploit
kit statistics when needed.

6 Operational realization

In this Section we present the technical implementation
of our experiment design in its three key points: (1) vir-
tualised system infrastructure; (2) automated execution;
(3) operative data collection;

6.1 Virtualised System Infrastructure
When testing for malware, an isolated, virtualised infras-
tructure is desirable [9]. We set up a five machine net-
work that includes a Malware Distribution Server (MDS)
and four machines hosting the Victim Virtual Machines
(VICTIMs). Initially, the setup also included an IDS and
a network auditing infrastructure to log the traffic; how-
ever, to eliminate possible confounding factors caused
by the network monitoring and auditing, we decided to
eliminate this part of the infrastructure from the design
reported here. For practical purposes (i.e.
scripting),
all machines are run on a linux-based operating systems,
upon which the virtualised infrastructure is installed.

The purpose of the MDS is to deliver the attacks. Be-
cause of the nature of exploit kits, all we need to attack
VICTIMs is an Apache Web-Server listening on HTTP
port 80 upon which the kits are deployed. As mentioned,
we implemented and armed the exploit kits with our own
“malware”, Casper.exe (our Ghost-in-the-browser [7]) to
help us keep track of infected systems. In order to make
it compatible with all Windows versions we have linked
it statically with the appropiate libraries (e.g. Winsock).
Casper reads a special configuration information file that
we put on each victim machine and send its content to a
PHP script on the MDS by using the Winsock API. This
script (trojan.php) simply stores the received data along
with the VICTIM IP address and timestamp into the In-
fections table in our database.

6.2 Automated execution

We use VirtualBox to virtualise victim machines.
In
order to automate the tests we take advantage of the
tool that is shipped with VirtualBox called VBoxMan-
age.
It is a command line tool that provides all the
necessary functions to start/stop virtual machines, cre-
ate/delete snapshots and run commands in the guest oper-
ating system. The main program, responsible for running
the experiment is a Python script that makes a sequence
of calls to VBoxMange via subprocess Python module.3
At each run, our scripts read configurations.csv, a
file containing all the generated configurations for that
machine. The scripts iteratively install configurations
upon the VICTIM system. The mapping between soft-
ware version pointers in configurations.csv and the ac-
tual software to be installed is hard-coded in the core
of the implementation. The automated installation hap-
pens via the silent install interface bundled in the instal-
lation packages distributed by most software vendors.
However, because of a lack of a “standard” interface
and the inconsistencies between different versions of the
same software, we could not deploy one-solution for all
software. We used instead a “trial-and-error” approach
and online documentation to enumerate the arguments to
pass to the installers and map them with the right soft-
ware versions. Each configuration is then automatically
and iteratively run against every exploit kit on the MDS.
Despite the experiment being completely automated,
we found that some machines were failing at certain
points in the run, most often while saving snapshots or
uploading files to the VICTIMs. We therefore imple-
mented a “resume functionality” that allows us to “save”
the experiment at the latest valid configuration, and in
case of failure restore the run from that point.

6.3 Operational Data Collection

To reset exploit kits statistics and guarantee the sound-
ness of the statistics collected in the Configuration and
Infections tables, we have implemented a PHP script that
clears the records on delivered attacks the kit keeps. This
step was rather easy to accomplish: we used the code
snippets responsible for statistics reset in each exploit kit,
and copy-pasted them into a single script.

We keep track of software installations on the VICTIM
machines by means of a second dedicated script. To
build it, we manually checked where each program puts
its data on the file system at the installation. Because

3It should be noted that there is Python API for VirtualBox, that
allows to run VirtualBox commands directly from within the Python
environment. We tried to use it during our first (failed) experiment,
but had to switch to VBoxManage, because Python VirtualBox API
functions proved not to be very reliable on our machines.

5

Figure 4: Stacked barplot of configuration installs by
software.

Figure 5: Infection rates per time window.

it was impossible to look at every application installa-
tion directory we sampled a subset of programs to check
whether they always put data in the same place. Then we
wrote a batch file that checks for the presence of the cor-
responding data directories after the alleged installation.
The results of the batch file inspection are then passed to
a Python script on the host machine, sent to the MDS,
and stored in the Configurations table on our dataset.

To collect the infection data, when the MDS receives
a call from a VICTIM machine, the MDS adds a record
in the Infections table, setting the successful record to 0
(the default). When executed, Casper connects to the
MDS via a PHP page we set up (namely infection.php).
This updates the successful bit of the corresponding run
record in Infections to 1.

7 Preliminary Experimental Results

The automatic installation procedure proved to be rather
reliable. Figure 4 depicts a 100%-stacked barplot of con-
figuration installs by software. As one can see, Fire-
fox and Java were practically always successfully de-
ployed on the machine. In contrast, 6% of Adobe Ac-
robat and 21% of Flash installations were reported to be
not successfully completed. However, it proved practi-
cally unfeasible to manually check failures of our detec-
tion mechanism (e.g. the files for that software version
on that configuration may be on a different location). We
cannot therefore assess the level of false negatives our
detection mechanism generates.

Figure 5 reports an overview of the infection rates of
all exploit kits in each time window. Intuitively, because
the exploit kits are always the same, the general rate of
infection decreases with more up-to-date software. Ob-
servationally, from 2005 up to 2009 the success rate of

Figure 6: Infections per time window per exploit kit.

exploit kits seem not to be affected by system evolution.
A marked decrease in the performance of our exploit kits
starts only after 2010. This observation is confirmed by
looking at a break-up of volumes of infections per ex-
ploit kit per year, depicted in Figure 6. Generally speak-
ing, each exploit kit (apart from Bleeding Life) seem to
remain effective mainly within the first three time win-
dows, from 2005 to 2009. Eleonore, CrimePack and
Shaman lead the volume of infections in those years,
with Eleonore peaking at more than 100 infections for
2006-2008, which amounts at about 50% of the configu-
rations for that window. Interestingly, a few exploit kits
seem identical in terms of performance. Seo, mPack,
gPack, ElFiesta, AdPack, IcePack all perform identi-
cally throughout the experiment. Most exploit kits’s ef-
ficacy drops in the fourth time-window, were configura-
tions spanning from 2008 to 2010 are attacked. However,
Bleeding Life is here an outliner, as its efficacy in infect-
ing these machines rises and tops in 2009-2011 to more
than twice its infection rates for 2005-2009. After 2011,

6

In the
however, its infection capabilities drop to zero.
last but one time window (2010-2012), the only still ef-
fective exploit kits are Crimepack and Shaman. Overall
three types of exploit kits seem to emerge:

1. Lousy exploit kits. Some exploit kits in the markets
seem to be identical in terms of effectiveness in in-
fecting machines. Not only they perform equally,
but the identical trend throughout our experiment
suggests that the exploits they bundle are them-
selves identical. This may indicate that some ex-
ploit kits may be rip-offs of others, or that an exploit
kit author may re-brand the same product.

2. Long-term exploit kits. From our results, a subset
of exploit kits (in our case Crimepack and Shaman)
perform particularly well in terms of resiliency.
Crimepack and Shaman are the only two exploit
kits that remain active from 2005 to 2012, despite
not being the most recent exploit kits we deployed
(see Table 2). For example, in the period 2008-
2012 Shaman performs up to two times better than
Eleonore, despite being two years older.
In other
words, some exploit kits appear to be designed and
armed to affect a wider variety of systems in time
than the competition.

3. Time-specific exploit kits. As opposed to long-term
exploit kits, some kits seem to be extremely effec-
tive in short periods of time only to “die” shortly af-
ter. Eleonore and Bleeding Life belong to this cat-
egory. The former achieves the highest amount of
infection per time window in 2006-2008, and drops
then to the minimum within the next two years. The
latter is the only exploit kit capable of infecting “re-
cent” machines, i.e. those with configurations since
2009 on. Bleeding Life was in particular clearly de-
signed to attack machines around the period of the
release of the kit (2010).

8 Challenges and limitations

The experimental approach introduced with this work
also introduces a number of technical and design chal-
lenges that we believe are worth further discussion.

1. Scale of the experiment. We collected only a lim-
ited amount of software to be tested in our configura-
tions. This is due both to the lack of an automated way
to collect “historical” software versions, and for deploy-
ment reasons: unfortunately, no standard interface to au-
tomatically deploy an application on a system exists. Us-
ing Microsoft MSI demonstrated to be not universally ef-
fective, as some (in particular old) software failed to be
installed that way. We had therefore to manually check
for the “silent install” arguments to give in input to the

software install package. Apart from being a long op-
eration to complete, it is also obviously quite prone to
errors. These errors may be responsible for the failed in-
stallations reported in Figure 4. To enhance the scope of
the experiment and configure more “realistic” machines
with more software (e.g. Apple Quicktime, Real Player,
Opera and Chrome) and respective versioning manage-
ment, an automated way to gather and deploy such soft-
ware would positively enhance both the representative-
ness and the reliability of the experiment. However, we
are currently not aware of any efficient way to do that.

2. Configuration checking. Because serial, auto-
mated software installations may be subject to failure,
a reliable detection mechanism for failed installs should
be on top of the configuration setup process. Our im-
plementation guarantees no false positives (i.e. configu-
rations that were not installed but reported as such), but
its quality in terms of false negatives is not clear. Still,
to manually check its reliability is practically infeasible,
given the volume of possible configurations to inspect.

3. Assumptions. This experiment is run on top of

three main assumptions:
(a.) Regardless of the release date of an exploit kit, the
probability of attacking a configuration does not vary
with the year of the configuration. We believe that it
is reasonable to assume that configurations from 2006
are still visible in the wild in 2012; however, to guar-
antee the realism of the experiment a “discount factor”
should be applied to old machine runs. With the cur-
rent design, older machines are run against old exploit
kits with the same frequency as against new ones. This
represents a threat to the realism of the experiment, as
the probability of receiving a connection from a partic-
ular configuration is likely to decrease with time (i.e.
Prt (Con fx) > Prt+1(Con fx)). This discount factor is
however hard to assess, and further investigations may
be needed to discard unrealistic or biased assumptions.
(b.) Software configurations are valid solely within a cer-
tain time-frame. We fixed a two-years time window for
software deployment. We only have folk knowledge to
support the validity of this assumption. This point is
crucial as a different definition of “time window” may
change the results of the experiment (e.g. see long term
and short term exploit kits above).
(c.) Exploit functionality, differently from malware’s, is
not affected by virtual machines. We are aware that many
malicious programs have virtual machine detection func-
tionalities, that prevent them from being installed on vir-
tual environments [9]. However, to the best of our knowl-
edge no exploit detected in the wild and delivered by an
exploit kit performs such actions.

4. Technical challenges. The major issue in imple-
menting the malware experiment is making it realistic.
In our design we make a victim browser visit the mali-

7

cious web page and hold it for 60 seconds. In real world,
however, a lot of exploits get delivered by pop up win-
dows and banners which a victim user tends to close on
sight [11]. Thus, introducing the ‘time of exposure” of
the victim to the malicious page could increase the real-
ism of the experiment. Another aspect of a victim ma-
chine is its hardware configuration. We assign 1GB of
RAM and 1-core processor for each VICTIM, but this
may as well affect the realism of the experiment. For
instance, we found that some exploits allocate so much
data in the browser address space that on machines with
<300MB of RAM the virtual memory and swap file of
the VICTIM runs out, which eventually forces the kernel
to kill the process. Assessing to what degree machine
diversity is desirable is an open problem of our design.

9 Conclusion & lessons learned

In this paper we presented our MalwareLab, a platform
to experiment with cybercrime attack tools. The presen-
tation was focused on the technical and design issues
we faced. As a result, we can synthesise the following
lessons learned from our experience:

1. Exploit and possibly malware developers put extra-
effort in enhancing the reliability of their products.
This translates in hard-to-control experiments, be-
cause the number of possible confounding variables
the experimenter should control for grows unpre-
dictably.

2. Controls must be checked experimentally. Tech-
nically sound assumptions on the functionality of
software artefacts may prove to be completely
wrong or uneffective for the particular exploit or
malware that is being tested.

3. Experiments always need a “resume point” from
where restart failed runs. However, because of
technical or operative requirements a resume point
might not coincide with the last valid experiment
run. To avoid biased measurements is important to
assure that the experiment is restored to its origi-
nal state at the “moment of the resume”. This is
particularly relevant when the experimental setup is
distributed on a network of systems, and the exper-
iment fails on a single, or subset of, machines.

At a technical level, we find that exploit kits show
different capabilities in terms of resiliency of infections
in time. Our answer to the research question outlined in
Section 5 is therefore the following:

Answer: a clear distinction between high-quality and
low quality products exists. Some exploit kits seem engi-
neered to be effective for long periods of time, at the cost

of lower top-rates of infection; other exploit kits instead
seem developed to gather as many infections as possi-
ble in short periods of time, possibly around the time of
their release. These kits achieve higher top-rates than
the competition, but their resielincy is lower.

Acknowledgments

This work was partly supported by the projects EU-
IST-NOE-NESSOS and EU-SEC-CP-SECONOMICS
and MIUR-PRIN-TENACE Projects and the European
Union by the Erasmus Mundus Action 2 Programme.

References
[1] ALLODI, L., WOOHYUN, S., AND MASSACCI, F. Quantitative
assessment of risk reduction with cybercrime black market mon-
itoring. In In Proc. of IWCC’13 (2013).

[2] CORELAN,

T.
Heap

Exploit writing
demystified.

tutorial

part
https:

:

spraying

11
//www.corelan.be/index.php/2011/12/31/
exploit-writing-tutorial-part-11-heap-spraying-demystified/,
checked on 27.04.2013.

[3] GRIER, C., BALLARD, L., CABALLERO, J., CHACHRA, N.,
DIETRICH, C. J., LEVCHENKO, K., MAVROMMATIS, P., MC-
COY, D., NAPPA, A., PITSILLIDIS, A., PROVOS, N., RAFIQUE,
M. Z., RAJAB, M. A., ROSSOW, C., THOMAS, K., PAXSON,
V., SAVAGE, S., AND VOELKER, G. M. Manufacturing com-
promise: the emergence of exploit-as-a-service. In Proceedings
of the 2012 ACM conference on Computer and communications
security (New York, NY, USA, 2012), CCS ’12, ACM, pp. 821–
832.

[4] KANICH, C., CHACHRA, N., MCCOY, D., GRIER, C., WANG,
D. Y., MOTOYAMA, M., LEVCHENKO, K., SAVAGE, S., AND
VOELKER, G. M. No plan survives contact: experience with
cybercrime measurement. In Proc. of CSET’11 (2011).

[5] KOTOV, V., AND MASSACCI, F. Anatomy of exploit kits. pre-
liminary analysis of exploit kits as software artefacts. In Proc. of
ESSoS 2013 (2013).

[6] PROVOS, N., MAVROMMATIS, P., RAJAB, M. A., AND MON-
ROSE, F. All your iframes point to us. In Proc. of USENIX’08
(2008), pp. 1–15.

[7] PROVOS, N., MCNAMEE, D., MAVROMMATIS, P., WANG, K.,
AND MODADUGU, N. The ghost in the browser analysis of web-
based malware. In Proc. of HOTBOTS’07 (2007), pp. 4–4.

[8] RAJAB, M., BALLARD, L., JAGPAL, N., MAVROMMATIS, P.,
NOJIRI, D., PROVOS, N., AND SCHMIDT, L. Trends in circum-
venting web-malware detection. Tech. rep., Google, 2011.

[9] ROSSOW, C., J. DIETRICH, C., GRIER, C., KREIBICH, C.,
PAXSON, V., POHLMANN, N., BOS, H., AND VAN STEEN, M.
Prudent practices for designing malware experiments: Status quo
and outlook. In Proc. of the 33rd IEEE Symp. on Sec. & Privacy
(2012).

[10] SYMANTEC.

Analysis of Malicious Web Activity by Attack
Symantec, Available on the web at

Toolkits, online ed.
http://www.symantec.com/threatreport/topic.jsp?
id=threat activity trends&aid=analysis of malicious web activity,
2011. Accessed on June 1012.

[11] WASH, R. Folk models of home computer security. In Proc. of

SOUPS’10 (2010), ACM.

8

|
4-3.pdf,|On the Need of Physical Security for Small

Embedded Devices: a Case Study with

COMP128-1 Implementations in SIM Cards

Yuanyuan Zhou1, Yu Yu2, F.-X. Standaert3, J.-J. Quisquater3

1 Brightsight, Delft, The Netherlands.

2 East China Normal University and Tsinghua University, China.

3 ICTEAM/ELEN/Crypto Group, Universit´e catholique de Louvain, Belgium.

Abstract. Ensuring the physical security of small embedded devices is
challenging. Such devices have to be produced under strong cost con-
straints, and generally operate with limited power and energy budget.
However, they may also be deployed in applications where physical access
is indeed possible for adversaries. In this paper, we consider the case of
SIM cards to discuss these issues, and report on successful side-channel
attacks against several (old but still deployed) implementations of the
COMP128-1 algorithm. Such attacks are able to recover cryptographic
keys with limited time and data, by measuring the power consumption
of the devices manipulating them, hence allowing cards cloning and com-
munications eavesdropping. This study allows us to put forward the long
term issues raised by the deployment of cryptographic implementations.
It provides a motivation for improving the physical security of small em-
bedded devices early in their development. We also use it to argue that
public standards for cryptographic algorithms and transparent physical
security evaluation methodologies are important tools for this purpose.

1

Introduction

Protecting present information systems requires considering both hardware and
software security issues, with their specific risks and constraints. In general,
software attacks are cheaper and tools for performing them can be rapidly dis-
seminated. Yet, they are also easier to patch with code updates. By contrast,
hardware attacks are more difficult to perform, as they require laboratory equip-
ment that ranges from low-cost to highly expensive. But they can be more diffi-
cult to fix a posteriori, as hardware updates imply more expensive development
processes, and usually take place in the longer term. Hence, finding the best
balance between hardware and software security is a difficult task for system de-
signers. This concern is particularly critical with cryptographic implementations
that may be the target of fault insertion attacks [2] and side-channel attacks [11,
12, 19]. In the latter case (that will be our focus in this paper), the adversary
exploits physical information leakage such as the power consumption of the de-
vice running a cryptographic algorithm, in order to extract secret information
such as secret keys. As the power consumption of a device is expected to be
correlated with the data it manipulates, these attacks essentially proceed by
comparing key-dependent leakage predictions with actual measurements. When

no particular care is taken, cryptographic implementations frequently turn out
to be highly susceptible to side-channel attacks, as recently exhibited with re-
sults against the KeeLoq remote keyless entry systems (at CRYPTO 2009 [8]),
the Mifare DESFire contactless smart cards (at CHES 2011 [18]), or Xilinx’s
FPGA bitstream encryption mechanisms (at ACM CCS 2011 [16]).

Since side-channel attacks do not target algorithms but instances of their im-
plementation in various technologies, it is hard to design general solutions that
allow making any implementation of an algorithm secure. Hence, state-of-the-art
techniques to improve security against such attacks rely on heuristic assumptions
(e.g. the masking and hiding in [14]), and need to be confirmed by empirical eval-
uation. Note that although this situation raises challenging research problems
(e.g. discussed at the CHES workshops [6]), producing practically secure inte-
grated circuits is not out of reach. Nowadays, most smart card companies have
products evaluated by independent laboratories and granted with high security
levels by certification authorities, e.g. [1, 5]. But this improved security usually
comes at the cost of implementation overheads that may limit their practical
deployment. In addition, and although having certificates may be a good selling
point, obtaining them also takes time and money (see, e.g. the Common Crite-
ria [7] and EMVco [9]). Hence, while such certificates are a frequent requirement
for security products of government agencies and banking applications, they are
less usual in lower-cost applications using SIM or transport cards.

A typical example of this lack of general approaches for preventing side-
channel attacks was actually given by a team from IBM in 2002, for implemen-
tations of the COMP128-1 algorithm used in GSM communications. In a paper
from IEEE S&P [20], Rao et al. first showed that a straightforward application
of Differential Power Analysis (DPA) was not successful against the instances of
SIM cards they were analyzing (presumably because of some ad hoc countermea-
sures). Then, they observed that at the first round of COMP128-1’s compression
function, the substitution-box (S-box) consists of 512 values (i.e. are accessed by
a 9-bit index). It implies that on low-speed SIMs (with 8-bit CPU) this S-box
has to be implemented using two (typically equal-size) lookup tables. Knowing
which table is being accessed (which could be identified from the power traces)
could result in a key recovery with a maximum of 1000 random challenges, or
255 chosen ones, or just 8 adaptively chosen ones (i.e. as efficient as a binary
search). This data corresponds to the monitoring of a few minutes of SIM card
operations. In other words, while the standard DPA approach did not directly
lead to successful key recoveries, a slightly modified path taking advantage of the
implementation specificities did a perfect job. Fortunately, the attack (exploiting
the 8-bit addressing) was only applicable to 8-bit-CPU SIM cards. Since 2003,
the major operators have been gradually phasing out the use of legacy SIM by
issuing products equipped with 16-bit CPU data bus, ruling out this possibility.

In this paper, we take advantage of this SIM card example to discuss the
practical challenges raised by hardware security issues. For this purpose, we in-
vestigate the resistance of SIM cards from two different GSM operators and four
different manufacturers against DPA. Our experiments target implementations

of the COMP128-1 algorithm in 16-bit CPUs, that are secure against the IBM
2002 attack. They are also secure against the algorithmic collision attacks de-
scribed in [3]. While COMP128-1 is progressively being replaced by improved
versions, it is still deployed in commercial devices, and sometimes being dis-
tributed. We show how DPA can be used to recover its 128-bit secret key, allow-
ing cards cloning and communications eavesdropping. Depending on the targets
and measurement setup available to the adversary, the attacks require physical
access to the device ranging from minutes to a couple of hours. Interestingly, our
results can be seen as the methodological counterpart of the 2002 ones. While
the previous analysis in [20] targets instances of SIM cards (presumably) secure
against standard DPA attacks but weak against dedicated ones, our instances
are robust against the IBM attack but weak against standard DPA.

The important conclusions of this work are methodological. First, our results
exhibit the long term nature of physical security concerns. While cryptographic
implementations are not deployed as long as algorithms, they may remain in
service for a couple of years, and are not straightforward to upgrade. This ob-
servation makes a case for considering physical security as an important feature
of small embedded devices in general. Technical solutions exist to make side-
channel attacks significantly more difficult to perform, e.g. the previously men-
tioned masking and hiding. But they work best if considered early in a design
process. Second, we observe that public standards for cryptographic algorithms
are useful to improve the efficiency of countermeasures against physical attacks.
By contrast, the closed-source nature of COMP128-1 has significantly limited the
amount of research about its secure implementations. Finally, transparent and
reproducible (possibly standardized) methodologies for physical security evalu-
ations are required, in order to quantify physical security on a sound basis.

Contact with the operators. Our experiments have been performed in 2010.
The different operators exploiting the SIM cards that we discuss in this paper
have been contacted before publication of our results. Updates towards imple-
mentations of COMP128-2 and COMP128-3, including protections against side-
channel attacks, are under development (or maybe already deployed).

2 Background

For place constraints, details about the GSM infrastructure and previous works
on SIM cloning fraud and countermeasures have been deferred to the long version
of the paper [23]. In this section, we briefly recall the processing of the compres-
sion function in COMP128-1 and necessary basics on side-channel attacks.

COMP128-1 is a cryptographic hash function that takes a 32-byte input (i.e.
a 16-byte challenge RAND and a 16-byte secret key KI), and produces a 12-
byte output by iterating 8 rounds. In our attacks, the most important part of
this algorithm is its compression function that consists of 5 (sub-)rounds that
combine the key material and randomness. In particular, the sensitive operations
that we will target are the following data update occurring in the first round:

y = (KI[m] + 2·RAND[m]) mod 29−j,
z = (2·KI[m] + RAND[m]) mod 29−j,

that occur for 0 ≤ m ≤ 15 secret key bytes (with j the RAND byte index).
Side-channel attacks generally exploit the existence of data-dependent and
physically observable phenomenons caused by the execution of computing tasks
in present microelectronic devices. Typical examples of such information leakages
include the power consumption and the electromagnetic radiation of integrated
circuits. We will focus on the first one in the rest of this paper. The literature
usually divides such attacks in two classes. First, Simple Power Analysis (SPA)
attempts to interpret the power consumption of a device and deduce information
about its performed operations. This can be done by visual inspection of the
power consumption measurements in function of the time. SPA in itself does not
always lead to key recovery. For example with block ciphers, distinguishing the
encryption rounds does not reveal any sensitive information. Yet, it is usually an
important preliminary step in order to reduce the computational requirements
of more advanced attacks. Second, Differential Power Analysis (DPA) intends
to take advantage of data-dependencies in the power consumption patterns. In
its standard form [15], DPA is based on a divide-and-conquer strategy, in which
the different parts of a secret key (usually denoted as “subkeys”) are recovered
separately. The attack is best illustrated with an example. Say one targets the
first round of a block cipher, where the plaintext is XORed with a subkey and
sent through a substitution box S. DPA is made of three main steps:
1. For different plaintexts xi and subkey candidates k∗, the adversary predicts
intermediate values in the target implementation. For example, one could
predict S-box outputs and get values vk∗

i = S(xi ⊕ k∗).

2. For each of these predicted values, the adversary models the leakages. For
example, if the target block cipher is implemented in a CMOS-based micro-
controller, the model can be the Hamming weight (HW) of the predicted
values. One then obtains modeled leakages mk∗
3. For each subkey candidate k∗, the adversary compares the modeled leakages
with actual measurements, produced with the same plaintexts xi and a secret
subkey k. In the univariate DPA attacks (that we will apply), each mk∗
is compared independently with many single points in the traces, and the
subkey candidate that performs best is selected by the adversary.

i = HW(vk∗
i ).

i

Different statistical tools have been proposed to perform this comparison. In our
experiments, we will consider a usual DPA distinguisher, namely Pearson’s cor-
relation coefficient [4]. In this case, and denoting a leakage sample produced with
plaintext xi and subkey k as lk
i , the adversary selects the subkey candidate as:

) · (lk

)2 ·(cid:80)

k

i − l
i(lk

)
i − l

,

k

)2

(cid:80)
(cid:113)(cid:80)
i − mk∗
i(mk∗
i − mk∗
i(mk∗

˜k = argmax

k∗

where mk∗
are the sample means of the models and leakage samples. The
complete master key is recovered by repeating this procedure for every subkey.

and l

k

3 DPA attacks against implementations of the

COMP128-1 algorithm in SIM cards

3.1 Target SIM cards & measurement setup

In this section, we perform DPA attacks on four representative SIM cards de-
noted as #1,#2, #3 and #4. Besides corresponding to various operators and
manufacturers, the main difference between these implementations is that they
sometimes include protections against the algorithmic collision attacks described
in [3], next denoted as the “Indexed Challenges” (I-C) and “Collision Free” (C-
F) countermeasures. The details of these countermeasures are not necessary for
the understanding of the paper, but are given in the long version [23]. As sum-
marized in Table 1, SIM#1 and SIM#2 are susceptible to collision attacks in
20 000 and more queries, SIM#3 and SIM#4 are immune against them.

Table 1. Target SIM cards.

Manufact. Operator Countermeasure(s)

SIM#1
SIM#2
SIM#3
SIM#4

I
II
III
IV

A
B
B
B

Not Available
I-C
I-C + C-F
I-C + C-F

We used a LeCroy WavePro 950 oscilloscope to acquire the power traces, via
a small resistor of 25 Ohm between the GND of power supply and the GND
of a self-made Card-to-Terminal adapter. The Card-to-Terminal adapter was
tweaked to provide an external DC power to the test card via a Kenwood P18A
power supply (+5V), and to provide an external clock to the card via an Agilent
33120A function generator(5MHz Frequency, 2.2V Amplitude and 1.1V Offset).
We used a commercially available card reader and software to control the test
card during the acquisitions. In addition, we used a Keithley 488 GPIB card
(i.e. a PCI card installed inside a PC) to communicate with the oscilloscope.

3.2 Preprocessing of the traces

As usual when implementing side-channel attacks, we started by applying SPA
in order to identify the relevant parts of the power traces. This task is easy for
SIM#1 and SIM#2. As shown in the left part of Figure 1, we can identify the 8
iterative rounds of COMP128-1 by visual inspection. By further zooming on the
different iterations, we could even observe the 5 sub-rounds of the COMP128-1
compression function (see Figure 2 in [23]). Therefore, it is directly possible to
extract the parts of the power traces where to apply DPA for these two targets.
The situation slightly differs for SIM cards #3 and #4, where the Collision Free
countermeasure was implemented. As illustrated in the right part of Figure 1
(and Figure 6 in [23]), it is again possible to identify the COMP128-1 operations

Fig. 1. Left: a power trace from SIM#1. Right: a power trace from SIM#3.

(as well as the Indexed Challenges) in the power traces. Yet, the Collision Free
countermeasure includes a randomized memory writing operation (i.e. it uses
randomness to decide whether to store a current request or not). Therefore, the
length of the power traces varies for different inputs, which requires special care
for aligning the traces after acquisition. In order to deal with this situation, a
simple solution is to apply pattern matching techniques. That is, we selected a
characteristic pattern including the samples of interest for our DPA attacks, and
then systematically identified them in following traces using cross-correlation. As
the noise level in our measurements was relatively low, such a simple heuristic
was sufficient for performing successful key recoveries, as will be described next.

3.3 DPA attack results

Since no countermeasures in our target SIM cards prohibit random queries, we
generated our traces by repeatedly executing the COMP128-1 algorithm with
such inputs. Next, we applied exactly the divide-and conquer strategy focusing
on the intermediate values y and z at the first sub-round of the first round in the
implementation of COMP128-1, as described in Section 2. For each 0 ≤ m ≤ 15,
we built predictions for the 256 possible values of KI[m] and performed the com-
parison. The result of such a comparison for one of the 16 COMP128-1 subkeys
is given in Figure 2 for SIM#2 and SIM#3 (similar results are given for the
other targets in [23]). The figures plot the value of Pearson’s correlation coeffi-

Fig. 2. Left: DPA result against SIM#2. Right: DPA result against SIM#3.

cient over time, using y as a target value. We observe that a significant peak is
distinguishable at the time samples where the computation of y actually takes
place, and this peak only appears for the correct subkey candidate. As expected,
the situation was slightly more challenging for SIM#3 and SIM#4. This is due
to more noisy traces and the previously mentioned synchronization issue. Yet,
in both cases, a DPA peak remained clearly distinguishable, and we could al-
ways identify the COMP128-1 subkeys. Finally, we consistently recovered the
full key of SIM#1 and SIM#2 with an amount of traces in the hundreds range,
and this number extends to the thousands range for SIM#3 and SIM#4. These
estimated data complexities are in accordance with the work of Mangard at
CT-RSA 2004 [13], where it is shown that the number of measurement traces
needed to recover a subkey is inversely proportional to the square of the corre-
lation coefficient estimated for the correct key candidate. In practice, these data
complexities corresponds to a few minutes to a couple of hours of acquisition.

4 Conclusions & future work

Technically, it is not a surprise that weakly protected chips can be defeated by
side-channel attacks. Yet, our results exhibit (or recall) that such attacks are
relatively easy to implement, and are certainly accessible to determined adver-
saries. Taking the example of SIM cards, this can have severe consequences for
the security of GSM communications. Overall, the security of a system is always
as strong as its weakest point. Hence, distributing cryptographically-enhanced
chips without a sufficient care for physical security leads to unbalanced situa-
tions, as side-channel attacks may constitute a trapdoor to circumvent math-
ematical security. This is especially important for small embedded devices, for
which physical access may sometimes be granted to adversaries. In this respect, it
is more surprising that (somewhat) security sensitive applications do not always
build on certified chips (following what is done, e.g. for bank cards). Admit-
tedly, the target SIM cards investigated in this paper implement old versions of
the GSM algorithms, in old technologies. Nevertheless, some of these cards are
still in circulation and cards cloning is an important concern that could prevent
the adoption of new services. Hence, this situation illustrates the long term na-
ture of hardware security issues. It provides a general motivation for considering
them as an important element to take into account early in cryptographic de-
velopments. In this respect, we note that the use of proprietary algorithms in
commercial products significantly slows down progresses in securing their im-
plementation. In view of the implementation-specific nature of physical attacks,
it frequently turns out that protection mechanisms that are tailored to certain
cryptographic algorithms provide the best efficiency vs. security tradeoffs. For
example, secure implementations of the AES have been the subject of a large
literature over the last 10 years. By contrast, no similar analysis is available for
COMP128-1. Worse, the use of large (e.g. 512-bit) tables makes it hardly suitable
for implementation of countermeasures such as software masking [10]. Following
this observation and in the long term, considering protections against physical
attacks as a design criteria for cryptographic algorithms could be useful.

While resorting to certification would be an important step in improving the
security of SIM cards, we finally note that the procedures used by evaluation
laboratories could also benefit from an improved transparency. That is, currently
certified chips certainly rule out the possibility of simple attacks as we describe in
this paper. But it remains that the exact security level they guarantee is opaque
for the end-users, and this opaqueness generally increases as countermeasures are
added to the chips. Proposals of worst-case security evaluations aiming at lim-
iting the risks of a “false sense” of security could improve this situation [21, 22].
Considering the strongest available adversaries and taking advantage of the latest
cryptanalytic progresses during evaluations of cryptographic hardware appears
important in view of the difficulty to fix physical security breaches a posteri-
ori. Eventually, better reflecting side-channel evaluation methodologies in public
standards would be highly beneficial too. In this respect, it is noticeable that the
ISO 19790 draft standard on “security requirements for cryptographic modules”
(aka. FIPS-140-3 [17]) leaves the section on non-invasive attack methods essen-
tially optional to vendors, with little details about the evaluation procedures.

Acknowledgements. Yu Yu was supported by the National Basic Research Pro-
gram of China Grant 2011CBA00300, 2011CBA00301, the National Natural Science
Foundation of China Grant 61033001, 61172085, 61061130540, 61073174, 61103221,
11061130539, 61021004 and 60703031. F.-X. Standaert is an associate researcher of the
Belgian Fund for Scientific Research (FNRS-F.R.S.). This work has been funded in
part by the ERC project 280141 on CRyptographic Algorithms and Secure Hardware.

References

1. ANSSI.

Agence nationale de

systemes d’information,
http://www.ssi.gouv.fr/en/products/certified-products/, Retrieved on feb. 1,
2012.

la securite des

2. Dan Boneh, Richard A. DeMillo, and Richard J. Lipton. On the importance of
checking cryptographic protocols for faults (extended abstract). In Walter Fumy,
editor, EUROCRYPT, volume 1233 of LNCS, pages 37–51. Springer, 1997.

3. Mark Briceno, Ian Goldberg, and David Wagner. GSM Cloning. http://www.
isaac.cs.berkeley.edu/isaac/gsm-faq.html, 1998. Retrieved on Oct. 14, 2011.
4. Eric Brier, Christophe Clavier, and Francis Olivier. Correlation power analysis
with a leakage model. In Marc Joye and Jean-Jacques Quisquater, editors, CHES,
volume 3156 of Lecture Notes in Computer Science, pages 16–29. Springer, 2004.
information security, https://www.bsi.bund.de/

Federal office

5. BSI.

for

en/topics/certification/certification node.html, retrieved on feb. 1, 2012.

6. CHES. http://www.chesworkshop.org/.
7. Common Criteria. http://www.commoncriteriaportal.org/.
8. Thomas Eisenbarth, Timo Kasper, Amir Moradi, Christof Paar, Mahmoud Salma-
sizadeh, and Mohammad T. Manzuri Shalmani. On the power of power analysis
in the real world: A complete break of the keeloqcode hopping scheme. In David
Wagner, editor, CRYPTO, volume 5157 of LNCS, pages 203–220. Springer, 2008.

9. EMVco. http://www.emvco.com/. Retrieved on April 11, 2012.

10. Louis Goubin and Jacques Patarin. Des and differential power analysis (the ”du-
plication” method). In C¸ etin Kaya Ko¸c and Christof Paar, editors, CHES, volume
1717 of Lecture Notes in Computer Science, pages 158–172. Springer, 1999.

11. Paul Kocher. Timing attacks on implementations of Diffie-Hellman, RSA, DSS,
and other systems. In Neal Koblitz, editor, Advances in Cryptology—CRYPTO ’96,
volume 1109 of LNCS, pages 104–113. Springer-Verlag, 18–22 August 1996.

12. Paul Kocher, Joshua Jaffe, and Benjamin Jun. Differential power analysis.

In
Michael Wiener, editor, Advances in Cryptology—CRYPTO ’99, volume 1666 of
LNCS, pages 388–397. Springer-Verlag, 15–19 August 1999.

13. Stefan Mangard. Hardware countermeasures against dpa ? a statistical analysis of
their effectiveness. In Tatsuaki Okamoto, editor, CT-RSA, volume 2964 of Lecture
Notes in Computer Science, pages 222–235. Springer, 2004.

14. Stefan Mangard, Elisabeth Oswald, and Thomas Popp. Power analysis attacks -

revealing the secrets of smart cards. Springer, 2007.

15. Stefan Mangard, Elisabeth Oswald, and Francois-Xavier Standaert. One for all –
all for one: unifying standard differential power analysis attacks. IET Information
Security, 5(2):100–110, 2011.

16. Amir Moradi, Alessandro Barenghi, Timo Kasper, and Christof Paar. On the
vulnerability of fpga bitstream encryption against power analysis attacks: extract-
ing keys from xilinx virtex-ii fpgas.
In Yan Chen, George Danezis, and Vitaly
Shmatikov, editors, ACM CCS, pages 111–124. ACM, 2011.

17. National

Institute

of

Standards

and Technologies.

http://csrc.

nist.gov/publications/PubsDrafts.html. Retrieved on March 25, 2012.

18. David Oswald and Christof Paar. Breaking mifare desfire mf3icd40: Power analysis
and templates in the real world. In Bart Preneel and Tsuyoshi Takagi, editors,
CHES, volume 6917 of LNCS, pages 207–222. Springer, 2011.

19. Jean-Jacques Quisquater and David Samyde. Electromagnetic analysis (EMA):
Measures and counter-measures for smart cards. In Smart Card Programming and
Security (E-smart 2001) Cannes, France, volume 2140 of LNCS, pages 200–210,
September 2001.

20. Josyula R. Rao, Pankaj Rohatgi, Helmut Scherzer, and Stephane Tinguely. Parti-
tioning attacks: Or how to rapidly clone some gsm cards. In IEEE Symposium on
Security and Privacy, pages 31–44, 2002.

21. Fran¸cois-Xavier Standaert. Some hints on the evaluation metrics & tools for side-
channel attacks. proceedings of the nist non-invasive attacks testing workshop,
nara, japan, september 2011.

22. Fran¸cois-Xavier Standaert, Tal Malkin, and Moti Yung. A unified framework for
the analysis of side-channel key recovery attacks. In Antoine Joux, editor, EU-
ROCRYPT, volume 5479 of Lecture Notes in Computer Science, pages 443–461.
Springer, 2009.

23. Yuanyuan Zhou, Yu Yu, Francois-Xavier Standaert, and Jean-Jacques Quisquater.
On the Need of Physical Security for Small Embedded Devices: a Case Study with
COMP128-1 Implementations in SIM Cards (long version).

|