FIT Profiler Report

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Action                                                                                                                                                                                                     	|  Mean duration (s)	|  Num calls      	|  Total time (s) 	|  Percentage %   	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Total                                                                                                                                                                                                      	|  -              	|  881489         	|  817.47         	|  100 %          	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  run_training_epoch                                                                                                                                                                                         	|  39.13          	|  20             	|  782.6          	|  95.735         	|
|  run_training_batch                                                                                                                                                                                         	|  0.025601       	|  20000          	|  512.02         	|  62.634         	|
|  [LightningModule]NeRFSystem.optimizer_step                                                                                                                                                                 	|  0.025135       	|  20000          	|  502.71         	|  61.495         	|
|  [Strategy]SingleDeviceStrategy.training_step                                                                                                                                                               	|  0.011458       	|  20000          	|  229.16         	|  28.033         	|
|  [TrainingEpochLoop].train_dataloader_next                                                                                                                                                                  	|  0.0079445      	|  20000          	|  158.89         	|  19.437         	|
|  [Strategy]SingleDeviceStrategy.backward                                                                                                                                                                    	|  0.0050869      	|  20000          	|  101.74         	|  12.445         	|
|  [Strategy]SingleDeviceStrategy.validation_step                                                                                                                                                             	|  1.1322         	|  64             	|  72.46          	|  8.8639         	|
|  [LightningModule]NeRFSystem.setup                                                                                                                                                                          	|  17.654         	|  1              	|  17.654         	|  2.1595         	|
|  [Callback]TQDMProgressBar.on_train_batch_end                                                                                                                                                               	|  0.00077524     	|  20000          	|  15.505         	|  1.8967         	|
|  [Strategy]SingleDeviceStrategy.batch_to_device                                                                                                                                                             	|  0.00024277     	|  20064          	|  4.8709         	|  0.59585        	|
|  [LightningModule]NeRFSystem.transfer_batch_to_device                                                                                                                                                       	|  0.00015188     	|  20064          	|  3.0473         	|  0.37277        	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_epoch_end                 	|  0.12209        	|  20             	|  2.4417         	|  0.29869        	|
|  [EvaluationEpochLoop].val_dataloader_idx_0_next                                                                                                                                                            	|  0.016451       	|  64             	|  1.0528         	|  0.12879        	|
|  [LightningModule]NeRFSystem.optimizer_zero_grad                                                                                                                                                            	|  4.2245e-05     	|  20000          	|  0.84491        	|  0.10336        	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_batch_end                 	|  2.481e-05      	|  20000          	|  0.49619        	|  0.060698       	|
|  [LightningModule]NeRFSystem.configure_gradient_clipping                                                                                                                                                    	|  2.2399e-05     	|  20000          	|  0.44798        	|  0.0548         	|
|  [Callback]GradientAccumulationScheduler.on_train_batch_end                                                                                                                                                 	|  6.6242e-06     	|  20000          	|  0.13248        	|  0.016207       	|
|  [Callback]TQDMProgressBar.on_after_backward                                                                                                                                                                	|  4.674e-06      	|  20000          	|  0.09348        	|  0.011435       	|
|  [LightningModule]NeRFSystem.on_validation_start                                                                                                                                                            	|  0.046609       	|  2              	|  0.093217       	|  0.011403       	|
|  [Callback]TQDMProgressBar.on_before_backward                                                                                                                                                               	|  4.2385e-06     	|  20000          	|  0.08477        	|  0.01037        	|
|  [Callback]TQDMProgressBar.on_before_optimizer_step                                                                                                                                                         	|  4.2343e-06     	|  20000          	|  0.084685       	|  0.010359       	|
|  [Callback]TQDMProgressBar.on_batch_start                                                                                                                                                                   	|  4.203e-06      	|  20000          	|  0.084061       	|  0.010283       	|
|  [Callback]GradientAccumulationScheduler.on_batch_start                                                                                                                                                     	|  4.0857e-06     	|  20000          	|  0.081714       	|  0.009996       	|
|  [Callback]TQDMProgressBar.on_train_batch_start                                                                                                                                                             	|  3.6583e-06     	|  20000          	|  0.073165       	|  0.0089502      	|
|  [Callback]TQDMProgressBar.on_before_zero_grad                                                                                                                                                              	|  3.5649e-06     	|  20000          	|  0.071298       	|  0.0087218      	|
|  [Callback]GradientAccumulationScheduler.on_batch_end                                                                                                                                                       	|  3.5351e-06     	|  20000          	|  0.070702       	|  0.0086489      	|
|  [LightningModule]NeRFSystem.on_before_batch_transfer                                                                                                                                                       	|  3.3708e-06     	|  20064          	|  0.067631       	|  0.0082732      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_batch_start               	|  3.3766e-06     	|  20000          	|  0.067532       	|  0.0082611      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_batch_start                     	|  3.1247e-06     	|  20000          	|  0.062493       	|  0.0076447      	|
|  [Callback]GradientAccumulationScheduler.on_train_batch_start                                                                                                                                               	|  2.9764e-06     	|  20000          	|  0.059527       	|  0.0072819      	|
|  [Callback]TQDMProgressBar.on_batch_end                                                                                                                                                                     	|  2.9624e-06     	|  20000          	|  0.059247       	|  0.0072477      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_batch_end                       	|  2.9487e-06     	|  20000          	|  0.058975       	|  0.0072143      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_optimizer_step           	|  2.9251e-06     	|  20000          	|  0.058501       	|  0.0071564      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_zero_grad                	|  2.8971e-06     	|  20000          	|  0.057942       	|  0.007088       	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_after_backward                  	|  2.8586e-06     	|  20000          	|  0.057171       	|  0.0069937      	|
|  [Callback]GradientAccumulationScheduler.on_before_backward                                                                                                                                                 	|  2.7525e-06     	|  20000          	|  0.05505        	|  0.0067341      	|
|  [Callback]GradientAccumulationScheduler.on_before_zero_grad                                                                                                                                                	|  2.7176e-06     	|  20000          	|  0.054352       	|  0.0066489      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_backward                 	|  2.7147e-06     	|  20000          	|  0.054294       	|  0.0066418      	|
|  [Callback]GradientAccumulationScheduler.on_before_optimizer_step                                                                                                                                           	|  2.6387e-06     	|  20000          	|  0.052774       	|  0.0064557      	|
|  [Callback]GradientAccumulationScheduler.on_after_backward                                                                                                                                                  	|  2.4876e-06     	|  20000          	|  0.049753       	|  0.0060862      	|
|  [Strategy]SingleDeviceStrategy.training_step_end                                                                                                                                                           	|  1.7196e-06     	|  20000          	|  0.034393       	|  0.0042072      	|
|  [LightningModule]NeRFSystem.on_before_optimizer_step                                                                                                                                                       	|  1.6927e-06     	|  20000          	|  0.033855       	|  0.0041414      	|
|  [Callback]TQDMProgressBar.on_validation_batch_end                                                                                                                                                          	|  0.00052797     	|  64             	|  0.03379        	|  0.0041335      	|
|  [LightningModule]NeRFSystem.on_train_batch_start                                                                                                                                                           	|  1.6661e-06     	|  20000          	|  0.033321       	|  0.0040761      	|
|  [LightningModule]NeRFSystem.training_step_end                                                                                                                                                              	|  1.6204e-06     	|  20000          	|  0.032408       	|  0.0039645      	|
|  [Strategy]SingleDeviceStrategy.on_train_batch_start                                                                                                                                                        	|  1.4566e-06     	|  20000          	|  0.029133       	|  0.0035638      	|
|  [LightningModule]NeRFSystem.on_before_zero_grad                                                                                                                                                            	|  1.4198e-06     	|  20000          	|  0.028396       	|  0.0034737      	|
|  [LightningModule]NeRFSystem.on_train_batch_end                                                                                                                                                             	|  1.397e-06      	|  20000          	|  0.02794        	|  0.0034179      	|
|  [LightningModule]NeRFSystem.on_after_batch_transfer                                                                                                                                                        	|  1.3534e-06     	|  20064          	|  0.027154       	|  0.0033217      	|
|  [LightningModule]NeRFSystem.on_after_backward                                                                                                                                                              	|  1.231e-06      	|  20000          	|  0.02462        	|  0.0030117      	|
|  [LightningModule]NeRFSystem.on_before_backward                                                                                                                                                             	|  1.0903e-06     	|  20000          	|  0.021806       	|  0.0026675      	|
|  [LightningModule]NeRFSystem.configure_optimizers                                                                                                                                                           	|  0.01531        	|  1              	|  0.01531        	|  0.0018729      	|
|  [LightningModule]NeRFSystem.validation_epoch_end                                                                                                                                                           	|  0.0045811      	|  2              	|  0.0091621      	|  0.0011208      	|
|  [Callback]TQDMProgressBar.on_train_epoch_end                                                                                                                                                               	|  0.00039402     	|  20             	|  0.0078804      	|  0.000964       	|
|  [Callback]TQDMProgressBar.on_train_epoch_start                                                                                                                                                             	|  0.00036589     	|  20             	|  0.0073179      	|  0.00089519     	|
|  [LightningModule]NeRFSystem.lr_scheduler_step                                                                                                                                                              	|  0.00014079     	|  20             	|  0.0028159      	|  0.00034446     	|
|  [Callback]TQDMProgressBar.on_validation_batch_start                                                                                                                                                        	|  3.4853e-05     	|  64             	|  0.0022306      	|  0.00027286     	|
|  [LightningModule]NeRFSystem.on_validation_model_eval                                                                                                                                                       	|  0.00057352     	|  2              	|  0.001147       	|  0.00014032     	|
|  [Callback]TQDMProgressBar.on_validation_end                                                                                                                                                                	|  0.00047615     	|  2              	|  0.00095229     	|  0.00011649     	|
|  [LightningModule]NeRFSystem.on_validation_model_train                                                                                                                                                      	|  0.00041711     	|  2              	|  0.00083422     	|  0.00010205     	|
|  [LightningModule]NeRFSystem.train_dataloader                                                                                                                                                               	|  0.00081853     	|  1              	|  0.00081853     	|  0.00010013     	|
|  [Callback]TQDMProgressBar.on_validation_start                                                                                                                                                              	|  0.00027374     	|  2              	|  0.00054748     	|  6.6972e-05     	|
|  [Callback]TQDMProgressBar.on_train_end                                                                                                                                                                     	|  0.00052765     	|  1              	|  0.00052765     	|  6.4546e-05     	|
|  [LightningModule]NeRFSystem.val_dataloader                                                                                                                                                                 	|  0.00050359     	|  1              	|  0.00050359     	|  6.1604e-05     	|
|  [Callback]GradientAccumulationScheduler.on_validation_batch_end                                                                                                                                            	|  7.3218e-06     	|  64             	|  0.0004686      	|  5.7323e-05     	|
|  [Callback]TQDMProgressBar.on_train_start                                                                                                                                                                   	|  0.00040705     	|  1              	|  0.00040705     	|  4.9793e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_batch_end            	|  4.2794e-06     	|  64             	|  0.00027388     	|  3.3503e-05     	|
|  [LightningModule]NeRFSystem.validation_step_end                                                                                                                                                            	|  3.8104e-06     	|  64             	|  0.00024386     	|  2.9832e-05     	|
|  [Callback]GradientAccumulationScheduler.on_validation_batch_start                                                                                                                                          	|  3.7439e-06     	|  64             	|  0.00023961     	|  2.9311e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_batch_start          	|  3.1036e-06     	|  64             	|  0.00019863     	|  2.4298e-05     	|
|  [LightningModule]NeRFSystem.on_validation_batch_end                                                                                                                                                        	|  2.1914e-06     	|  64             	|  0.00014025     	|  1.7157e-05     	|
|  [Callback]GradientAccumulationScheduler.on_train_epoch_start                                                                                                                                               	|  6.4405e-06     	|  20             	|  0.00012881     	|  1.5757e-05     	|
|  [Callback]GradientAccumulationScheduler.on_train_epoch_end                                                                                                                                                 	|  6.3824e-06     	|  20             	|  0.00012765     	|  1.5615e-05     	|
|  [LightningModule]NeRFSystem.on_validation_batch_start                                                                                                                                                      	|  1.7228e-06     	|  64             	|  0.00011026     	|  1.3488e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.setup                              	|  0.00010451     	|  1              	|  0.00010451     	|  1.2784e-05     	|
|  [Callback]TQDMProgressBar.on_epoch_start                                                                                                                                                                   	|  3.9205e-06     	|  22             	|  8.6251e-05     	|  1.0551e-05     	|
|  [Strategy]SingleDeviceStrategy.validation_step_end                                                                                                                                                         	|  1.3062e-06     	|  64             	|  8.3596e-05     	|  1.0226e-05     	|
|  [Callback]TQDMProgressBar.on_epoch_end                                                                                                                                                                     	|  3.6435e-06     	|  22             	|  8.0156e-05     	|  9.8054e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_end                  	|  3.8379e-05     	|  2              	|  7.6758e-05     	|  9.3897e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_epoch_start               	|  3.5769e-06     	|  20             	|  7.1539e-05     	|  8.7512e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_epoch_end                       	|  2.8791e-06     	|  22             	|  6.3339e-05     	|  7.7482e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_epoch_start                     	|  2.7055e-06     	|  22             	|  5.9521e-05     	|  7.2811e-06     	|
|  [Callback]GradientAccumulationScheduler.on_epoch_end                                                                                                                                                       	|  2.6076e-06     	|  22             	|  5.7367e-05     	|  7.0176e-06     	|
|  [Callback]GradientAccumulationScheduler.on_epoch_start                                                                                                                                                     	|  2.5268e-06     	|  22             	|  5.5591e-05     	|  6.8003e-06     	|
|  [LightningModule]NeRFSystem.on_epoch_end                                                                                                                                                                   	|  1.7772e-06     	|  22             	|  3.9099e-05     	|  4.7829e-06     	|
|  [LightningModule]NeRFSystem.on_train_epoch_end                                                                                                                                                             	|  1.899e-06      	|  20             	|  3.7979e-05     	|  4.646e-06      	|
|  [LightningModule]NeRFSystem.on_epoch_start                                                                                                                                                                 	|  1.6222e-06     	|  22             	|  3.5689e-05     	|  4.3658e-06     	|
|  [LightningModule]NeRFSystem.on_train_epoch_start                                                                                                                                                           	|  1.5095e-06     	|  20             	|  3.0191e-05     	|  3.6932e-06     	|
|  [LightningModule]NeRFSystem.on_validation_epoch_end                                                                                                                                                        	|  1.2545e-05     	|  2              	|  2.509e-05      	|  3.0692e-06     	|
|  [Callback]GradientAccumulationScheduler.on_train_start                                                                                                                                                     	|  1.969e-05      	|  1              	|  1.969e-05      	|  2.4087e-06     	|
|  [Callback]TQDMProgressBar.setup                                                                                                                                                                            	|  1.7581e-05     	|  1              	|  1.7581e-05     	|  2.1506e-06     	|
|  [Callback]GradientAccumulationScheduler.on_validation_end                                                                                                                                                  	|  8.4597e-06     	|  2              	|  1.6919e-05     	|  2.0697e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_start                     	|  1.478e-05      	|  1              	|  1.478e-05      	|  1.808e-06      	|
|  [Callback]GradientAccumulationScheduler.on_validation_start                                                                                                                                                	|  6.1453e-06     	|  2              	|  1.2291e-05     	|  1.5035e-06     	|
|  [LightningModule]NeRFSystem.configure_callbacks                                                                                                                                                            	|  1.1449e-05     	|  1              	|  1.1449e-05     	|  1.4005e-06     	|
|  [Callback]GradientAccumulationScheduler.on_train_end                                                                                                                                                       	|  9.5796e-06     	|  1              	|  9.5796e-06     	|  1.1719e-06     	|
|  [Strategy]SingleDeviceStrategy.on_validation_end                                                                                                                                                           	|  3.8552e-06     	|  2              	|  7.7104e-06     	|  9.4321e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_start                	|  3.7299e-06     	|  2              	|  7.4599e-06     	|  9.1256e-07     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_end                                                                                                                                                          	|  3.6904e-06     	|  2              	|  7.3807e-06     	|  9.0288e-07     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_start                                                                                                                                                        	|  3.3402e-06     	|  2              	|  6.6804e-06     	|  8.172e-07      	|
|  [Callback]TQDMProgressBar.on_pretrain_routine_start                                                                                                                                                        	|  6.6506e-06     	|  1              	|  6.6506e-06     	|  8.1356e-07     	|
|  [Callback]TQDMProgressBar.on_fit_end                                                                                                                                                                       	|  6.2902e-06     	|  1              	|  6.2902e-06     	|  7.6947e-07     	|
|  [Callback]TQDMProgressBar.on_fit_start                                                                                                                                                                     	|  5.6103e-06     	|  1              	|  5.6103e-06     	|  6.863e-07      	|
|  [LightningModule]NeRFSystem.on_fit_start                                                                                                                                                                   	|  5.1688e-06     	|  1              	|  5.1688e-06     	|  6.323e-07      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_end                       	|  5.0198e-06     	|  1              	|  5.0198e-06     	|  6.1407e-07     	|
|  [LightningModule]NeRFSystem.on_validation_end                                                                                                                                                              	|  2.4652e-06     	|  2              	|  4.9304e-06     	|  6.0313e-07     	|
|  [LightningModule]NeRFSystem.configure_sharded_model                                                                                                                                                        	|  4.8904e-06     	|  1              	|  4.8904e-06     	|  5.9823e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_epoch_start          	|  2.3898e-06     	|  2              	|  4.7795e-06     	|  5.8468e-07     	|
|  [Callback]TQDMProgressBar.teardown                                                                                                                                                                         	|  4.6901e-06     	|  1              	|  4.6901e-06     	|  5.7374e-07     	|
|  [Strategy]SingleDeviceStrategy.on_validation_start                                                                                                                                                         	|  2.3004e-06     	|  2              	|  4.6007e-06     	|  5.628e-07      	|
|  [Callback]GradientAccumulationScheduler.on_validation_epoch_start                                                                                                                                          	|  2.2994e-06     	|  2              	|  4.5989e-06     	|  5.6257e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_epoch_end            	|  2.2203e-06     	|  2              	|  4.4405e-06     	|  5.4321e-07     	|
|  [LightningModule]NeRFSystem.prepare_data                                                                                                                                                                   	|  4.4396e-06     	|  1              	|  4.4396e-06     	|  5.4309e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_epoch_end                                                                                                                                            	|  2.0349e-06     	|  2              	|  4.0699e-06     	|  4.9786e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_fit_end                         	|  3.8501e-06     	|  1              	|  3.8501e-06     	|  4.7098e-07     	|
|  [Callback]GradientAccumulationScheduler.on_fit_end                                                                                                                                                         	|  3.81e-06       	|  1              	|  3.81e-06       	|  4.6608e-07     	|
|  [Callback]TQDMProgressBar.on_configure_sharded_model                                                                                                                                                       	|  3.7896e-06     	|  1              	|  3.7896e-06     	|  4.6357e-07     	|
|  [Callback]GradientAccumulationScheduler.setup                                                                                                                                                              	|  3.6899e-06     	|  1              	|  3.6899e-06     	|  4.5138e-07     	|
|  [Callback]GradientAccumulationScheduler.teardown                                                                                                                                                           	|  3.6806e-06     	|  1              	|  3.6806e-06     	|  4.5024e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.teardown                           	|  3.6806e-06     	|  1              	|  3.6806e-06     	|  4.5024e-07     	|
|  [Callback]TQDMProgressBar.on_pretrain_routine_end                                                                                                                                                          	|  3.5297e-06     	|  1              	|  3.5297e-06     	|  4.3179e-07     	|
|  [Callback]TQDMProgressBar.on_before_accelerator_backend_setup                                                                                                                                              	|  3.4999e-06     	|  1              	|  3.4999e-06     	|  4.2814e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_pretrain_routine_start          	|  3.2e-06        	|  1              	|  3.2e-06        	|  3.9145e-07     	|
|  [Callback]GradientAccumulationScheduler.on_pretrain_routine_start                                                                                                                                          	|  3.0892e-06     	|  1              	|  3.0892e-06     	|  3.779e-07      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_configure_sharded_model         	|  3.0799e-06     	|  1              	|  3.0799e-06     	|  3.7676e-07     	|
|  [Callback]GradientAccumulationScheduler.on_fit_start                                                                                                                                                       	|  3.0501e-06     	|  1              	|  3.0501e-06     	|  3.7311e-07     	|
|  [LightningModule]NeRFSystem.on_train_end                                                                                                                                                                   	|  3.0305e-06     	|  1              	|  3.0305e-06     	|  3.7072e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_pretrain_routine_end            	|  3.0203e-06     	|  1              	|  3.0203e-06     	|  3.6947e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_accelerator_backend_setup	|  3.01e-06       	|  1              	|  3.01e-06       	|  3.6821e-07     	|
|  [Callback]GradientAccumulationScheduler.on_pretrain_routine_end                                                                                                                                            	|  2.9104e-06     	|  1              	|  2.9104e-06     	|  3.5602e-07     	|
|  [Callback]GradientAccumulationScheduler.on_before_accelerator_backend_setup                                                                                                                                	|  2.8806e-06     	|  1              	|  2.8806e-06     	|  3.5238e-07     	|
|  [Callback]GradientAccumulationScheduler.on_configure_sharded_model                                                                                                                                         	|  2.82e-06       	|  1              	|  2.82e-06       	|  3.4497e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_fit_start                       	|  2.7996e-06     	|  1              	|  2.7996e-06     	|  3.4247e-07     	|
|  [LightningModule]NeRFSystem.on_pretrain_routine_start                                                                                                                                                      	|  2.6198e-06     	|  1              	|  2.6198e-06     	|  3.2048e-07     	|
|  [LightningModule]NeRFSystem.on_validation_epoch_start                                                                                                                                                      	|  1.3048e-06     	|  2              	|  2.6096e-06     	|  3.1922e-07     	|
|  [LightningModule]NeRFSystem.teardown                                                                                                                                                                       	|  2.3702e-06     	|  1              	|  2.3702e-06     	|  2.8995e-07     	|
|  [LightningModule]NeRFSystem.on_fit_end                                                                                                                                                                     	|  2.1905e-06     	|  1              	|  2.1905e-06     	|  2.6796e-07     	|
|  [LightningModule]NeRFSystem.on_save_checkpoint                                                                                                                                                             	|  2.0508e-06     	|  1              	|  2.0508e-06     	|  2.5087e-07     	|
|  [Strategy]SingleDeviceStrategy.on_train_end                                                                                                                                                                	|  2.0396e-06     	|  1              	|  2.0396e-06     	|  2.495e-07      	|
|  [LightningModule]NeRFSystem.on_train_start                                                                                                                                                                 	|  1.5199e-06     	|  1              	|  1.5199e-06     	|  1.8593e-07     	|
|  [LightningModule]NeRFSystem.on_pretrain_routine_end                                                                                                                                                        	|  1.3495e-06     	|  1              	|  1.3495e-06     	|  1.6508e-07     	|
|  [Strategy]SingleDeviceStrategy.on_train_start                                                                                                                                                              	|  1.2498e-06     	|  1              	|  1.2498e-06     	|  1.5289e-07     	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
