Parallel Processing

Sequential Processing: As seen in earlier topics, all processing on a single processor computer is essentially sequential processing. That is, each instruction is taken in turn from memory and then executed by the Central Processing Unit. This process continue (Fetch-Execute Cycle) until the program instruction set is exhausted.

Parallel Processing

In an attempt to process data more quickly, more processors can be added to the system. The problem then is to determine the best way to utilise these extra resources.

Within a single task environment, an analysis of the problem will determine the best method of utilising the extra processors.

Parallel methods

A number of different methods of processing data using parallel systems are defined. The choice of which method to choose is predetermined by the data to be processed and the architecture and software available.

Alk (1989) and Gibson & Rytter (1988) suggest the following definitions.

SIMD Architecture A Single Instruction Multiple Data system consists of parallel data processors acting in unison and using one instruction at a time but on different data stored nominally on each processor. Instructions are executed serially but parallel instructions are executed on each active processor simultaneously. Figure 1 shows a data stream being processed by n identical processors all carrying out the same instruction on the variant data.
MISD Architecture Multiple Instruction Single Data system has the same data item processed in many ways. Each processor has its own instruction set and it acts on the data item. The process embodies the concept of pipelining. Figure 2 depicts this architecture.
MIMD Architecture Multiple Data Multiple Instruction is where each processor has its own data set and instruction set. Each processor acts independently of each other and communication in some asynchronous user defined way. Figure 3 depicts this architecture.

Distributed Processing

In terms of large scale processing of data, distributed processing is where parts of the total data system is processed by multiple computers located over a distributed geographical location.

Example 1 - Databases

A database can be located in one central loaction but accessed and the data processed by many users of the system simultaneously. Typical systems include library database systems, on line shopping (e-commerce), On-line banking (e-banking) etc.

Example 2 - World Wide Web

WWW Internet access where Server Side Includes (SSI) are processed, Active Server Pages (ASP) are distibuted and CGI scripts are ran.

Example 3 - Pipelining

When large data sets are to be processed and especially when many seperate processed must be carried out on the each data element, many computers are used to process the data using a method called pipelining. Each computer has one specific task to carry out on the data element. Each element is passed to each computer in turn as iff going through a pipe. When the element gets to the end, all processing is complete and the job finished.

Example 4 - Search for Extra-terrestrial Intelligence (SETI) Program

Radio telescopes collect vast amonts of data. This data can be broken down into "lots" and sent to idle computers around the world for processing using the internet. When the data is processed, it is returned for collation purposes.

Author: Mike Leishman