Next: Optimization Techniques
Up: Overview of OOC Problems
Previous: Classification of OOC Problems
The need for high performance I/O resulted in designing
various tools on hardware and software levels. Their main
groups can be categorized as follows:
- Hardware support - addresses small scale parallelism and does not directly
address the important software issues.
An example of such technology is
RAID (Redundant Arrays of Inexpensive Disks)
system .
- Parallel file systems -
provide interface to high performance secondary storage
and mechanisms of remote file access.
They are a significant part of
High Throughput Computing (HTC)
environments .
- Virtual memory
,
(also referred to as paging mechanism)
allows the user to use a larger memory system than what is physically
installed in the system. This is achieved by storing
parts of memory contents (memory pages) on the disk and fetching
them as needed.
The swapping heuristics tends to be as general as possible
to handle a wide range of applications.
This technique is very convenient
as it is transparent to the user. However, it does not take into account
special application requirements. In addition,
many OOC applications require much more space than is guaranteed
by the paging mechanism.
- Low level APIs - (Application Programming Interface)
are sets of low-level routines usually provided
by the system.
They allow the user to explicitly handle the OOC data
and result in more efficient, but also less portable code.
To achieve portability various API standards were designed.
A well-established standard in the scientific community
is MPI-IO - an IO extension to the
MPI standard
which is particularly important in this thesis as the
library heavily depends on it.
- Programming libraries deal with system-specific details
and therefore enable a user to use the available I/O resources
efficiently.
Examples of such solutions include
aforementioned
Panda,
Jostle and
ViPIOS.
Next: Optimization Techniques
Up: Overview of OOC Problems
Previous: Classification of OOC Problems
Created by Katarzyna Zając