ahoo! is not affiliated with the authors of this page or responsible for its content.
Microsoft PowerPoint - DETER.CENIC.mar05b 3/11/2005
1
CENIC: High
CENIC: High
-
-
speed Connectivity for
speed Connectivity for
the DETER Testbed
the DETER Testbed
Terry V. Benzel
Terry V. Benzel
Bob Braden
Bob Braden
Information Sciences Institute
Information Sciences Institute
University of Southern California
University of Southern California
Anthony Joseph
Anthony Joseph
University of California at Berkeley
University of California at Berkeley 3/11/2005
2
Cyber Defense Technology
Cyber Defense Technology
Experimental Research (DETER)
Experimental Research (DETER)
& Evaluation Methods for Internet
& Evaluation Methods for Internet
Security Technology (EMIST)
Security Technology (EMIST)
Inadequate wide scale deployment of security
technologies
Despite 10+ years investment in network security
research
Lack of experimental infrastructure
Testing and validation in small to medium-scale
private research labs
Missing objective test data, traffic and metrics 3/11/2005
3
DETER + EMIST
DETER + EMIST
DETER & EMIST: companion projects
funded by NSF and DHS HSARPA
1) Design and construct a
testbed
for
network security experiments
2) Do research on
experimental
methodology
for network security
3) Do research on
network security 3/11/2005
4
Examples of Experiments
Examples of Experiments
DDoS attacks and defense
Understand dynamics, try various defense strategies
Worms
Understand worm dynamics
Try various defense strategies
Routing security
Understand BGP dynamics and threats
Build and test countermeasures
Advance intrusion detection
(and many more) 3/11/2005
5
DETER
DETER
Requirements
Requirements
(1)
(1)
Versatility: Support wide range of security scenarios.
Repeatability: Complete control of environment, for
repeatable experiments without artifacts.
Containment: Confine dangerous code.
Realism: Real router and end-system behavior.
Fidelity: Represent topology and traffic mix of
Internet.
Programmability: Add new algorithms to routers 3/11/2005
6
DETER
DETER
Requirements
Requirements
(2)
(2)
Accessibility: Remote control over Internet.
Efficiency: Testbed partitionable among
simultaneous independent experiments
Functionality: Rich set of traffic and
topology generators and experimental profiles.
Economy: Accomplish all this with very
limited $$ 3/11/2005
7
Basic Design Choice
Basic Design Choice
Cluster testbed
Many nodes in one laboratory
Dedicated local inter-node links => Perfectly
controlled
Prime example: Univ of Utah's Emulab
Distributed TB
Nodes scattered across Internet
Prime example: Planetlab
Links subject to "normal" Internet interference 3/11/2005
8
Basic Design Choice
Basic Design Choice
Two reasons to choose clusters for
DETERlab
1. Security & containment
would be impossible in distributed testbed.
2. Experimental repeatibility
There is no perfect solution
Use Utah's Emulab software
Objective: biggest scientific bang for the
bucks 3/11/2005
9
Deterlab
Deterlab
Experimental Plane
Experimental Plane
Basic experimental node in cluster:
high-end PC
Each node may be configured to:
Emulate an end node or a router,
Generate traffic,
Emulate link characteristics, or
Make measurements
Nodes identical in large groups
e.g., 32 or 64 3/11/2005
10
PC
160
N x 4 @1000bT
Data ports
PC
PC
Programmable Patch Panel (VLAN switch)
Switch Control
Interface
DETER Experimental Network
DETER Experimental Network
Cluster of N nearly identical experimental
nodes, interconnected dynamically into
arbitrary topologies using VLAN switch.
Pool of N processors 3/11/2005
11
Example DETER Topologies
Example DETER Topologies 3/11/2005
12
Experimental Plane
Experimental Plane
Internode links are 10/100/1Gbps
Working on integration of a few commercial
routers into the cluster, to provide realism
and increase heterogeneity
Will also add special-purpose hardware, e.g.:
high-speed synthetic traffic generators
hardware instrumentation devices 3/11/2005
13
The Fidelity Issue
The Fidelity Issue
Would Ideally Like:
Large and realistic topologies
Diverse, realistic nodes
and links
But:
Fidelity is expensive
Large-scale fidelity may be unnecessary for (maybe
even contrary to) good science
Plan to add limited
heterogeneity
and
realism
e.g.,
a few vendor routers, network processors 3/11/2005
14
Experimental Backplane
Experimental Backplane
Switch hardware
:
ISI: Cisco 6509, Nortel 5510 switches planned
UCB: Foundry 1500, Nortel 5510 switches planned
Example:
ISI currently has 72 nodes, with 4 1000bT
interfaces per node
All 10/100/1000 bT, VLANd
That is a LOT of wires 3/11/2005
15
Side view of the testbed. 3/11/2005
16
PC
Internet
160
Power
Controller
'Boss' Server
User Acct &
Data logging
N x 4 @1000bT
Data ports
N @100bT
Control ports
User Server
PC
PC Control Network VLAN
User
Control
DB
Node Serial
Line Server
Power Serial
Line Server
Web/DB/SNMP,
switch mgmt
User
files
Ethernet Bridge
with
Firewal
l
Programmable Patch Panel (VLAN switch)
'Gatekeeper'
DETER Testbed
Schematic 3/11/2005
17
DETERlab
DETERlab
Architecture
Architecture
Divide logical DETERlab cluster into two physical
clusters
At USC/ISI and at UC Berkeley
One control plane, and one entry point (ISI)
"Centrally-controlled federation"
Nodes from different clusters can be combined in one
experiment when user chooses
When Internet introduces variability that will be desirable or
at least tolerable 3/11/2005
18
Interconnecting Clusters
Interconnecting Clusters
One control site (ISI)
One user entry point, accounts, control
VLAN switches interconnected using IPsec
tunnels
Distinct pools of nodes to be allocated
User can control whether span multiple pools
IPsec tunnels should preserve security of link 3/11/2005
19
PC
User
Server
PC
Control Network
ISI Cluster
User
files
Cisco switch
Foundry switch
Node Serial
Line Server
'Boss'
Server
PC
PC
UCB Cluster
Node Serial
Line Server
Download
Server
Power
Contler
Power
Contler
PC
trunk
trunk
Control Network
Internet
IPsec
IPsec
User
FW
FW
C
E
N
I
C 3/11/2005
20
ISI/UCB Links
ISI/UCB Links
Two logical links:
Control plane link (layer-3 connection)
Experimental plane link (layer 2 connection
trunking ports between switches
For the experimental plane, this scheme
demands a very high-speed link between
Marina del Rey and Berkeley at least 1 Gbps
CENIC fills that need 3/11/2005
21
Using CENIC
Using CENIC
Straightened out confusion about routing, so now
have HPR routing
No surprise: the last mile is the hardest part
Network people don't actually LIE to us, but
It took awhile to track down which organization had a
100Mbps link in the path.
"Sure, you can have a 1Gbps link", but when we try to USE
it, alarms go off.
Traceroute shows 8 hops, half in CENIC 3/11/2005
22
Using CENIC
Using CENIC
1Gbps CENIC path is actually much smaller than
aggregate possible inter-cluster traffic
But an experiment that pushes this limit is probably a bad
experiment.
The Emulab control plane understands about the relatively
limited inter-switch connectivity of 1 Gbps.
Measured performance ISI<->UCB
(w/o IPSec!):
930 Mbps UDP
(TCP measurement in progress) 3/11/2005
23
IPsec Performance
IPsec Performance
--
--
Hard
Hard
Not easy to get desired throughput using IPsec.
First attempt: 100 Mbps, currently 200 Mbps.
Currently 200 Mbps using crypto boards
So far, not living up to specs
Trying many variations of hardware, software
May have to settle for 250, use 4-times striping 3/11/2005
24
Testbed
Testbed
Software
Software
Utah Emulab software in control plane.
Experimental node OS:
Current standard OS: RedHat Linux 7.3 or
FreeBSD 4.7
Moving soon to Red Hat 9.0, FreeBSD 4.10.
Users can load arbitrary code, in fact
Users have
root
access to all allocated nodes! 3/11/2005
25
Security and Containment
Security and Containment
Threats
To other experiments from inside isolation
failure
To DETER from inside
intrusion
.
To DETER from outside
To outside from inside
extrusion!!
Accidents 3/11/2005
26
Security is Critical
Security is Critical
Defenses employed by the DETER test-bed must balance
the requirements of containment, isolation, and
confidentiality with the need for remote management of
experiments
DETER experiments are categorized according to the
consequences of loss of containment, and procedures
applied according to that categorization 3/11/2005
27
Achieving Security
Achieving Security
Operational
Procedures for proposing and reviewing experiments
Guidelines for categorizing safety of experiments
Vetting of investigators and