- Rewrite your Python code by parallelizing or optimizing/replacing/tuning algorithm(s), e.g. using:
- Hadoop or Disco
- Open source implementations of MapReduce
- Parallel Python
- Message Passing Interface (MPI)
- Frequently used for numerical programming
- Bulk Synchronous Parallel (BSP)
- RPyC
- RPC for Distributed/Parallel Programming
- Twisted
- Network libraries for Distributed/Parallel Programming
- Profiling Tools
- Threading or Microthreads (Stackless)
- Use a tool that can speed up your code more or less unchanged
- Psyco
- Just in time compiler, note: this is probably the easiest approach to try
- Pyrex
- Write and compile Python with a flavor of C data structures
- Cython
- PyJs
- Compile (large subset of) Python to Javascript, note: probably more interesting for client development (ajax) than server side.
- Rpython
- Compile (large subset of) Python to native code, note: part of PyPy project
- PyStream
- GPULib
- Shedskin
- Compile (large subset of) Python to C++, some benchmarks
- Replace (parts of) your Python code with another language
- Simplified Wrapper and Interface Generator (SWIG)
- Use C/C++ from Python
- Fortran to Python Interface Generator (F2PY)
- Use Fortran from Python
- llvm-py
- Code and Compile to Assembler for running on Low-Level Virtual Machine
- CorePy
- Write Assembly Code in Python
- Weave
- PyInline
- Boost.Python
- Cinpy
Saturday, November 29, 2008
Tools for Accelerating Python
If you need to speed up your Python program there are several possible approaches, with varying degree of effort needed, here is (probably not complete) overview:
