Parallel Computing is the simultaneous use of multiple compute resources to solve a computational problem. To conduct parallel computing, the compute resources can include a single computer with multiple processors (Parallel Processing), an arbitrary number of computers connected by a network or a combination of both. The key benefit of parallel computing is to solve large and complex problems fast. Other benefits include: Taking advantage of non-local resources, Cost savings - using multiple "cheap" computing resources instead of a supercomputer and Overcoming memory constraints on single computer.

Typically, software has been written for serial computation that is executed by a single computer having a single Central Processing Unit (CPU) and Problems are solved by a series of instructions, executed one after the other by the CPU. Only one instruction may be executed at any moment in time. Parallel computing is an evolution of serial computing that attempts to emulate what has always been the state of affairs in the natural world: many complex, interrelated events happening at the same time, yet within a sequence.

To take advantage of parallel computing, the computational problem usually demonstrates characteristics:

• Broken apart into discrete pieces of work that can be solved simultaneously;
• Execute multiple program instructions at any moment in time;
• Solved in less time with multiple compute resources than with a single compute resource.

The matrix below defines the 4 possible classifications of computing architecture.

A Typical Software Infrastructure of a Parallel Computing System

Related Terms: Parallel Processing, Distributed Computing, Cluster Computing

Reference Links:
http://www.llnl.gov/computing/tutorials/parallel_comp/: Introduction to Parallel Computing
http://www.pcc.qub.ac.uk/tec/courses/intro/ohp/intro-ohp.html: Introduction to Parallel Computing