Configuration

For the configuration, a configuration file is created for the client and engine when the IPython hub is started, usually in ~/.ipython/profile_default/security/.

1. If we don’t want to use the default profile, we should first create a new IPython profile with:

   $ pipenv run ipython profile create --parallel --profile=local
   [ProfileCreate] Generating default config file: '/Users/veit/.ipython/profile_parallel/ipython_config.py'
   [ProfileCreate] Generating default config file: '/Users/veit/.ipython/profile_parallel/ipython_kernel_config.py'
   [ProfileCreate] Generating default config file: '/Users/veit/.ipython/profile_parallel/ipcontroller_config.py'
   [ProfileCreate] Generating default config file: '/Users/veit/.ipython/profile_parallel/ipengine_config.py'
   [ProfileCreate] Generating default config file: '/Users/veit/.ipython/profile_parallel/ipcluster_config.py
   

   --parallel

   includes the configuration files for Parallel Computing (ipengine, ipcontroller etc.).

2. When the IPython controller and the engines are started, the files ipcontroller-engine.json and ipcontroller-client.json are generated in ~/.ipython/profile_default/security/.

ipcluster in mpiexec/mpirun mode

1. Creating the profile:

   $ pipenv run ipython profile create --parallel --profile=mpi
   [ProfileCreate] Generating default config file: '/Users/veit/.ipython/profile_mpi/ipython_config.py'
   [ProfileCreate] Generating default config file: '/Users/veit/.ipython/profile_mpi/ipython_kernel_config.py'
   [ProfileCreate] Generating default config file: '/Users/veit/.ipython/profile_mpi/ipcontroller_config.py'
   [ProfileCreate] Generating default config file: '/Users/veit/.ipython/profile_mpi/ipengine_config.py'
   [ProfileCreate] Generating default config file: '/Users/veit/.ipython/profile_mpi/ipcluster_config.py'
   

2. Editing of ipcluster_config.py:

   1. so that the MPI launcher can be used:

   c.IPClusterEngines.engine_launcher_class = 'MPIEngineSetLauncher'
   

3. The cluster can then be started with:

   $ pipenv run ipcluster start -n 4 --profile=mpi
   [IPClusterStart] Starting ipcluster with [daemon=False]
   [IPClusterStart] Creating pid file: /Users/veit/.ipython/profile_mpi/pid/ipcluster.pid
   [IPClusterStart] Starting Controller with LocalControllerLauncher
   [IPClusterStart] Starting 4 Engines with LocalEngineSetLauncher
   [IPClusterStart] Engines appear to have started successfully