Accelerate Time-to-Solution with Star-P for Python
Productivity Breakthrough for MATLAB®, Python, R, and others. . .
Programming that took months, can be done in days; and simulation runs that took days can be done in minutes.
The Star-P open software platform delivers revolutionary results to scientists, engineers and analysts by enabling them to transparently use high performance computing resources, using familiar desktop tools such as MATLAB®, Python, R and many others.
|
|
10-100X Faster Computations
By transparently leveraging the parallel computing capability, Star-P enables simulations developed in desktop tools to be processed in parallel, dramatically accelerating computation time
10-100X Larger Data Sets
Using Star-P, desktop application users can work with large, distributed datasets - gigabytes and even terabytes in size - distributed across servers, clusters, and grids.
No Need for Low-level Parallel Programming
With Star-P, there is no need to use low-level languages and constructs of C, Fortran, and MPI, to take advantage of high performance computing resources. Using the Star-P Connect library API, users can leverage library functions from open source community and commercial vendors written in C or Fortran.
Simple and Efficient Parallel Computing for Science and Technology
The user interacts with standard Python desktop environment, enhanced with few simple Star-P commands
-
Tedious parallel programming is eliminated (No MPI, C, Fortran)
- Conversion from serial to parallel is easy and mostly automated
| |
Star-P with Python environment supports
- Serial computing (plain Python)
- Data-Parallel computing
- Task-Parallel computing (ppeval command)
- Additional functions for data- and task-parallel computing through
Star-P Connect™ Library API link
|
Functions Available with Star-P for Parallel Computing
-
Task-Parallel Operations: All NumPy and SciPy functions and operators, and many Python modules available online.
-
Data-Parallel Operations: ~100 functions and operators
- Unlimited user or community functions via Star-P Connect
- Star-P specific functions: ~20 programming or environment control and support functions
Star-P for Python Users
A high-level language of increasing importance today is Python,
and in particular the set of Python extensions contained in the
NumPy module. Starting with release 2.5, Interactive Supercomputing
offers a Python client as part of the Star-P suite of parallelization
tools (Figure 1). This Python client works in a way which will feel
comfortable to users of NumPy. In fact, the new Star-P Python client
will enable NumPy users to run their programs on a parallel supercomputer
with only a couple of trivial syntax changes.
If you are a Python user, Interactive Supercomputing's new Python
client means this: Using Star-P you can write a Python program handling
large matrix and array objects on a desktop PC. While developing
your code on the PC you may take advantage of Python's intrinsic
interactivity, and you can use any of the convenient IDE's available
for working in Python. Then, when you want to process extremely
large datasets, Star-P allows you to export your calculation to
a server supercomputer while still controlling your computation
from the interactive session running on the desktop PC.
Figure 1: Star-P's client-server architecture.
The client machine is the PC on the left. You interact with Python
on the client. The server supercomputer is shown on the right. The
server runs world-class parallel numerical libraries. Star-P mediates
the interaction of the client and server while you manipulate your
data on the client. When a computation must be performed on the
parallel server, Star-P automatically moves your functions and data
to the server for processing.
The goal is to enable ordinary scientists and engineers to write
number crunching programs in a comfortable high-level language,
and then immediately and seamlessly run their code on a parallel
computer. In particular, Star-P renders it unnecessary to re-code
an application into C in order to take advantage of the parallel-processing
MPI library. Star-P handles the details of MPI, allowing the computer
user to focus on his application, and avoid fooling around with
the complicated low-level details of parallel computing. In particular,
Star-P delivers interactive performance by automatically:
- Sending your computations to the parallel server
- Providing you access to built-in world-class parallel processing
libraries
- Enabling you to extend your code using community or user-specific
parallel libraries
- Managing inter-processor communication on the parallel server
- Managing memory allocation for large datasets
- Bundling and returning data from the server supercomputer to
your desktop PC for further analysis and visualization
Star-P will fundamentally transform your computing work flow since
it allows you to immediately run prototype code on a parallel server
without the necessity of porting it to C/C++ and MPI. Coupled with
the power and flexibility of Python, Star-P represents a shortcut
to getting the answers you want from your numerical algorithms!
The first Star-P client linked a client runing MATLAB® with a parallel
supercomputer. Python represents Interactive Supercomputing's second
client supporting a high-level language for numerical computation.
In particular, Star-P's Python client is envisioned to be a drop-in
replacement for the NumPy module. That means that any code targeted
to use NumPy may be ported to the Star-P platform by simply replacing
all numpy calls with starp calls, like this:
# NumPy version
import numpy
a = numpy.random.rand(5, 5)
b = numpy.linalg.inv(a)
c = a*b
# Star-P version
import starp
starp.defaultConnect('hostname', '/path/to/starp/installation')
# Establish connection to server
a = starp.numpy.random.rand(5, 5)
b = starp.numpy.linalg.inv(a)
c = a*b
As designed , every NumPy function has a Star-P equivalent which
is called in exactly the same way. Therefore, besides
the call to "starp.defaultConnect()", any NumPy program
can become a Star-P program simply by replacing the word "numpy"
with "starp.numpy". The difference is that Star-P operates on
parallelized arrays held on the parallel server!
Star-P also supports task-parallel computations, which refers to performing longer, non-communicating computations in parallel, and then gathering the results at the computation's end. Task parallel operations could sensibly run on separate threads (or separate processes) on a serial computer. Parallelized Monte-Carlo simulation is a classic example of task-parallelism. Star-P's Python client supports task-parallel computations explicitly by introducing a new function used to invoke parallel execution of any desired function. That new function is called "starp.ppeval()". Starp.ppeval handles the job of splitting up data, oversees execution of the function in parallel and gathers the returned results together.
|
Star-P Support
Blog
Email
Page
