DDPStrategy

class lightning.pytorch.strategies.DDPStrategy(accelerator=None, parallel_devices=None, cluster_environment=None, checkpoint_io=None, precision_plugin=None, ddp_comm_state=None, ddp_comm_hook=None, ddp_comm_wrapper=None, model_averaging_period=None, process_group_backend=None, timeout=datetime.timedelta(seconds=1800), start_method='popen', **kwargs)

기반 클래스: lightning.pytorch.strategies.parallel.ParallelStrategy

Strategy for multi-process single-device training on one or multiple nodes.

barrier(*args, **kwargs)

Synchronizes all processes which blocks processes until the whole group enters this function.

매개변수

name – an optional name to pass into barrier.

반환 형식

None

broadcast(obj, src=0)

Broadcasts an object to all processes.

매개변수

• obj (TypeVar(TBroadcast)) – the object to broadcast

• src (int) – source rank

반환 형식

TypeVar(TBroadcast)

model_to_device()

Moves the model to the correct device.

반환 형식

None

on_exception(exception)

Called when the trainer execution is interrupted by an exception.

반환 형식

None

optimizer_step(optimizer, closure, model=None, **kwargs)

Performs the actual optimizer step.

매개변수

• optimizer (Optimizer) – the optimizer performing the step

• closure (Callable[[], Any]) – closure calculating the loss value

• model (Union[LightningModule, Module, None]) – reference to the model, optionally defining optimizer step related hooks

• **kwargs – Any extra arguments to optimizer.step

반환 형식

Any

pre_backward(closure_loss)

Run before precision plugin executes backward.

반환 형식

None

predict_step(*args, **kwargs)

The actual predict step.

See predict_step() for more details

반환 형식

Union[Tensor, Dict[str, Any]]

reduce(tensor, group=None, reduce_op='mean')

Reduces a tensor from several distributed processes to one aggregated tensor.

매개변수

• tensor (Tensor) – the tensor to sync and reduce

• group (Optional[Any]) – the process group to gather results from. Defaults to all processes (world)

• reduce_op (Union[ReduceOp, str, None]) – the reduction operation. Defaults to ‘mean’/’avg’. Can also be a string ‘sum’ to calculate the sum during reduction.

반환 형식

Tensor

반환

reduced value, except when the input was not a tensor the output remains is unchanged

setup(trainer)

Setup plugins for the trainer fit and creates optimizers.

매개변수

trainer (Trainer) – the trainer instance

반환 형식

None

setup_environment()

Setup any processes or distributed connections.

This is called before the LightningModule/DataModule setup hook which allows the user to access the accelerator environment before setup is complete.

반환 형식

None

teardown()

This method is called to teardown the training process.

It is the right place to release memory and free other resources.

반환 형식

None

test_step(*args, **kwargs)

The actual test step.

See test_step() for more details

반환 형식

Union[Tensor, Dict[str, Any], None]

training_step(*args, **kwargs)

The actual training step.

See training_step() for more details

반환 형식

Union[Tensor, Dict[str, Any]]

validation_step(*args, **kwargs)

The actual validation step.

See validation_step() for more details

반환 형식

Union[Tensor, Dict[str, Any], None]

property root_device: torch.device

Return the root device.

반환 형식

device