FIT Profiler Report

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Action                                                                                                                                                                                                     	|  Mean duration (s)	|  Num calls      	|  Total time (s) 	|  Percentage %   	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Total                                                                                                                                                                                                      	|  -              	|  881585         	|  699.95         	|  100 %          	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  run_training_epoch                                                                                                                                                                                         	|  31.837         	|  20             	|  636.74         	|  90.968         	|
|  run_training_batch                                                                                                                                                                                         	|  0.026418       	|  20000          	|  528.35         	|  75.484         	|
|  [LightningModule]NeRFSystem.optimizer_step                                                                                                                                                                 	|  0.026162       	|  20000          	|  523.24         	|  74.754         	|
|  [Strategy]SingleDeviceStrategy.training_step                                                                                                                                                               	|  0.0093665      	|  20000          	|  187.33         	|  26.763         	|
|  [TrainingEpochLoop].train_dataloader_next                                                                                                                                                                  	|  0.0029346      	|  20000          	|  58.692         	|  8.3851         	|
|  [LightningModule]NeRFSystem.setup                                                                                                                                                                          	|  49.981         	|  1              	|  49.981         	|  7.1405         	|
|  [Strategy]SingleDeviceStrategy.backward                                                                                                                                                                    	|  0.0019782      	|  20000          	|  39.564         	|  5.6523         	|
|  [Strategy]SingleDeviceStrategy.validation_step                                                                                                                                                             	|  0.37612        	|  70             	|  26.328         	|  3.7614         	|
|  [Callback]TQDMProgressBar.on_train_batch_end                                                                                                                                                               	|  0.00046166     	|  20000          	|  9.2332         	|  1.3191         	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_epoch_end                 	|  0.17018        	|  20             	|  3.4036         	|  0.48626        	|
|  [Strategy]SingleDeviceStrategy.batch_to_device                                                                                                                                                             	|  0.0001221      	|  20070          	|  2.4506         	|  0.35011        	|
|  [LightningModule]NeRFSystem.transfer_batch_to_device                                                                                                                                                       	|  7.9733e-05     	|  20070          	|  1.6002         	|  0.22862        	|
|  [LightningModule]NeRFSystem.optimizer_zero_grad                                                                                                                                                            	|  1.9891e-05     	|  20000          	|  0.39781        	|  0.056834       	|
|  [EvaluationEpochLoop].val_dataloader_idx_0_next                                                                                                                                                            	|  0.0048693      	|  70             	|  0.34085        	|  0.048696       	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_batch_end                 	|  1.4903e-05     	|  20000          	|  0.29806        	|  0.042582       	|
|  [LightningModule]NeRFSystem.configure_gradient_clipping                                                                                                                                                    	|  1.4678e-05     	|  20000          	|  0.29357        	|  0.041941       	|
|  [Callback]GradientAccumulationScheduler.on_train_batch_end                                                                                                                                                 	|  4.3519e-06     	|  20000          	|  0.087038       	|  0.012435       	|
|  [Callback]TQDMProgressBar.on_after_backward                                                                                                                                                                	|  3.6342e-06     	|  20000          	|  0.072685       	|  0.010384       	|
|  [LightningModule]NeRFSystem.on_validation_start                                                                                                                                                            	|  0.032412       	|  2              	|  0.064825       	|  0.0092613      	|
|  [Callback]TQDMProgressBar.on_before_optimizer_step                                                                                                                                                         	|  3.1537e-06     	|  20000          	|  0.063074       	|  0.0090112      	|
|  [Callback]TQDMProgressBar.on_batch_start                                                                                                                                                                   	|  3.0692e-06     	|  20000          	|  0.061385       	|  0.0087698      	|
|  [Callback]TQDMProgressBar.on_before_backward                                                                                                                                                               	|  3.0108e-06     	|  20000          	|  0.060216       	|  0.0086028      	|
|  [Callback]TQDMProgressBar.on_before_zero_grad                                                                                                                                                              	|  2.8789e-06     	|  20000          	|  0.057577       	|  0.0082258      	|
|  [Callback]TQDMProgressBar.on_train_batch_start                                                                                                                                                             	|  2.7414e-06     	|  20000          	|  0.054828       	|  0.0078331      	|
|  [Callback]GradientAccumulationScheduler.on_batch_start                                                                                                                                                     	|  2.5794e-06     	|  20000          	|  0.051588       	|  0.0073702      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_batch_start                     	|  2.3966e-06     	|  20000          	|  0.047932       	|  0.0068479      	|
|  [Callback]GradientAccumulationScheduler.on_train_batch_start                                                                                                                                               	|  2.3603e-06     	|  20000          	|  0.047206       	|  0.0067441      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_batch_start               	|  2.3514e-06     	|  20000          	|  0.047028       	|  0.0067188      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_after_backward                  	|  2.2988e-06     	|  20000          	|  0.045977       	|  0.0065685      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_batch_end                       	|  2.2809e-06     	|  20000          	|  0.045617       	|  0.0065171      	|
|  [Callback]TQDMProgressBar.on_batch_end                                                                                                                                                                     	|  2.2758e-06     	|  20000          	|  0.045516       	|  0.0065027      	|
|  [Callback]GradientAccumulationScheduler.on_after_backward                                                                                                                                                  	|  2.1814e-06     	|  20000          	|  0.043629       	|  0.0062331      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_optimizer_step           	|  2.1157e-06     	|  20000          	|  0.042314       	|  0.0060453      	|
|  [Callback]GradientAccumulationScheduler.on_before_optimizer_step                                                                                                                                           	|  2.1055e-06     	|  20000          	|  0.04211        	|  0.0060162      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_zero_grad                	|  2.0824e-06     	|  20000          	|  0.041648       	|  0.0059502      	|
|  [Callback]GradientAccumulationScheduler.on_before_zero_grad                                                                                                                                                	|  2.0609e-06     	|  20000          	|  0.041218       	|  0.0058887      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_backward                 	|  2.0436e-06     	|  20000          	|  0.040873       	|  0.0058393      	|
|  [Callback]GradientAccumulationScheduler.on_before_backward                                                                                                                                                 	|  1.9967e-06     	|  20000          	|  0.039934       	|  0.0057052      	|
|  [Callback]GradientAccumulationScheduler.on_batch_end                                                                                                                                                       	|  1.9243e-06     	|  20000          	|  0.038485       	|  0.0054983      	|
|  [Callback]TQDMProgressBar.on_validation_batch_end                                                                                                                                                          	|  0.00041744     	|  70             	|  0.029221       	|  0.0041747      	|
|  [LightningModule]NeRFSystem.on_before_batch_transfer                                                                                                                                                       	|  1.2341e-06     	|  20070          	|  0.024768       	|  0.0035385      	|
|  [LightningModule]NeRFSystem.training_step_end                                                                                                                                                              	|  1.1356e-06     	|  20000          	|  0.022712       	|  0.0032448      	|
|  [LightningModule]NeRFSystem.on_after_batch_transfer                                                                                                                                                        	|  1.0426e-06     	|  20070          	|  0.020925       	|  0.0029895      	|
|  [LightningModule]NeRFSystem.on_train_batch_start                                                                                                                                                           	|  1.0068e-06     	|  20000          	|  0.020136       	|  0.0028768      	|
|  [Strategy]SingleDeviceStrategy.training_step_end                                                                                                                                                           	|  9.8016e-07     	|  20000          	|  0.019603       	|  0.0028006      	|
|  [LightningModule]NeRFSystem.on_before_backward                                                                                                                                                             	|  9.796e-07      	|  20000          	|  0.019592       	|  0.002799       	|
|  [LightningModule]NeRFSystem.on_train_batch_end                                                                                                                                                             	|  9.744e-07      	|  20000          	|  0.019488       	|  0.0027842      	|
|  [LightningModule]NeRFSystem.on_before_zero_grad                                                                                                                                                            	|  9.7087e-07     	|  20000          	|  0.019417       	|  0.0027741      	|
|  [LightningModule]NeRFSystem.on_after_backward                                                                                                                                                              	|  8.8987e-07     	|  20000          	|  0.017797       	|  0.0025426      	|
|  [Strategy]SingleDeviceStrategy.on_train_batch_start                                                                                                                                                        	|  8.7153e-07     	|  20000          	|  0.017431       	|  0.0024902      	|
|  [LightningModule]NeRFSystem.on_before_optimizer_step                                                                                                                                                       	|  8.1533e-07     	|  20000          	|  0.016307       	|  0.0023297      	|
|  [LightningModule]NeRFSystem.validation_epoch_end                                                                                                                                                           	|  0.0049763      	|  2              	|  0.0099526      	|  0.0014219      	|
|  [Callback]TQDMProgressBar.on_train_epoch_end                                                                                                                                                               	|  0.00026496     	|  20             	|  0.0052993      	|  0.00075709     	|
|  [Callback]TQDMProgressBar.on_train_epoch_start                                                                                                                                                             	|  0.00024315     	|  20             	|  0.004863       	|  0.00069476     	|
|  [Callback]TQDMProgressBar.on_validation_batch_start                                                                                                                                                        	|  5.3513e-05     	|  70             	|  0.0037459      	|  0.00053517     	|
|  [LightningModule]NeRFSystem.configure_optimizers                                                                                                                                                           	|  0.0030468      	|  1              	|  0.0030468      	|  0.00043529     	|
|  [LightningModule]NeRFSystem.lr_scheduler_step                                                                                                                                                              	|  9.552e-05      	|  20             	|  0.0019104      	|  0.00027293     	|
|  [Callback]TQDMProgressBar.on_validation_end                                                                                                                                                                	|  0.0006552      	|  2              	|  0.0013104      	|  0.00018721     	|
|  [LightningModule]NeRFSystem.on_validation_model_train                                                                                                                                                      	|  0.00047207     	|  2              	|  0.00094413     	|  0.00013488     	|
|  [LightningModule]NeRFSystem.on_validation_model_eval                                                                                                                                                       	|  0.00044601     	|  2              	|  0.00089201     	|  0.00012744     	|
|  [LightningModule]NeRFSystem.train_dataloader                                                                                                                                                               	|  0.00058991     	|  1              	|  0.00058991     	|  8.4279e-05     	|
|  [Callback]TQDMProgressBar.on_validation_start                                                                                                                                                              	|  0.0002622      	|  2              	|  0.0005244      	|  7.4918e-05     	|
|  [Callback]GradientAccumulationScheduler.on_validation_batch_end                                                                                                                                            	|  5.7582e-06     	|  70             	|  0.00040308     	|  5.7586e-05     	|
|  [Callback]TQDMProgressBar.on_train_end                                                                                                                                                                     	|  0.00035059     	|  1              	|  0.00035059     	|  5.0087e-05     	|
|  [LightningModule]NeRFSystem.val_dataloader                                                                                                                                                                 	|  0.0003032      	|  1              	|  0.0003032      	|  4.3317e-05     	|
|  [Callback]TQDMProgressBar.on_train_start                                                                                                                                                                   	|  0.000297       	|  1              	|  0.000297       	|  4.2431e-05     	|
|  [Callback]GradientAccumulationScheduler.on_validation_batch_start                                                                                                                                          	|  3.5095e-06     	|  70             	|  0.00024566     	|  3.5097e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_batch_end            	|  2.8935e-06     	|  70             	|  0.00020255     	|  2.8937e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_batch_start          	|  2.6929e-06     	|  70             	|  0.0001885      	|  2.6931e-05     	|
|  [LightningModule]NeRFSystem.validation_step_end                                                                                                                                                            	|  2.4829e-06     	|  70             	|  0.0001738      	|  2.483e-05      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_end                  	|  7.3209e-05     	|  2              	|  0.00014642     	|  2.0918e-05     	|
|  [LightningModule]NeRFSystem.on_validation_batch_end                                                                                                                                                        	|  1.8239e-06     	|  70             	|  0.00012767     	|  1.824e-05      	|
|  [Callback]GradientAccumulationScheduler.on_train_epoch_start                                                                                                                                               	|  5.5171e-06     	|  20             	|  0.00011034     	|  1.5764e-05     	|
|  [LightningModule]NeRFSystem.on_validation_batch_start                                                                                                                                                      	|  1.4493e-06     	|  70             	|  0.00010145     	|  1.4494e-05     	|
|  [Callback]GradientAccumulationScheduler.on_train_epoch_end                                                                                                                                                 	|  4.6639e-06     	|  20             	|  9.3278e-05     	|  1.3326e-05     	|
|  [Strategy]SingleDeviceStrategy.validation_step_end                                                                                                                                                         	|  1.1519e-06     	|  70             	|  8.0631e-05     	|  1.1519e-05     	|
|  [Callback]TQDMProgressBar.on_epoch_start                                                                                                                                                                   	|  3.2463e-06     	|  22             	|  7.1418e-05     	|  1.0203e-05     	|
|  [Callback]TQDMProgressBar.on_epoch_end                                                                                                                                                                     	|  3.1682e-06     	|  22             	|  6.97e-05       	|  9.9578e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_epoch_start               	|  2.9535e-06     	|  20             	|  5.907e-05      	|  8.4391e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_epoch_end                       	|  2.3457e-06     	|  22             	|  5.1605e-05     	|  7.3726e-06     	|
|  [Callback]GradientAccumulationScheduler.on_epoch_end                                                                                                                                                       	|  2.2127e-06     	|  22             	|  4.8678e-05     	|  6.9545e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_epoch_start                     	|  2.0995e-06     	|  22             	|  4.6189e-05     	|  6.5988e-06     	|
|  [Callback]GradientAccumulationScheduler.on_epoch_start                                                                                                                                                     	|  2.0908e-06     	|  22             	|  4.5997e-05     	|  6.5714e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.setup                              	|  3.751e-05      	|  1              	|  3.751e-05      	|  5.3589e-06     	|
|  [LightningModule]NeRFSystem.on_epoch_start                                                                                                                                                                 	|  1.5e-06        	|  22             	|  3.3e-05        	|  4.7147e-06     	|
|  [LightningModule]NeRFSystem.on_train_epoch_end                                                                                                                                                             	|  1.6055e-06     	|  20             	|  3.2109e-05     	|  4.5873e-06     	|
|  [LightningModule]NeRFSystem.on_epoch_end                                                                                                                                                                   	|  1.4322e-06     	|  22             	|  3.1509e-05     	|  4.5016e-06     	|
|  [LightningModule]NeRFSystem.on_train_epoch_start                                                                                                                                                           	|  1.308e-06      	|  20             	|  2.6161e-05     	|  3.7375e-06     	|
|  [Strategy]SingleDeviceStrategy.on_validation_end                                                                                                                                                           	|  8.605e-06      	|  2              	|  1.721e-05      	|  2.4587e-06     	|
|  [Callback]GradientAccumulationScheduler.on_validation_end                                                                                                                                                  	|  5.1949e-06     	|  2              	|  1.039e-05      	|  1.4844e-06     	|
|  [LightningModule]NeRFSystem.configure_callbacks                                                                                                                                                            	|  1.016e-05      	|  1              	|  1.016e-05      	|  1.4515e-06     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_end                                                                                                                                                          	|  5.0301e-06     	|  2              	|  1.006e-05      	|  1.4373e-06     	|
|  [Callback]GradientAccumulationScheduler.on_validation_start                                                                                                                                                	|  4.3451e-06     	|  2              	|  8.6902e-06     	|  1.2415e-06     	|
|  [Callback]TQDMProgressBar.on_fit_end                                                                                                                                                                       	|  7.6201e-06     	|  1              	|  7.6201e-06     	|  1.0887e-06     	|
|  [Callback]TQDMProgressBar.setup                                                                                                                                                                            	|  7.3202e-06     	|  1              	|  7.3202e-06     	|  1.0458e-06     	|
|  [LightningModule]NeRFSystem.on_train_end                                                                                                                                                                   	|  7.2001e-06     	|  1              	|  7.2001e-06     	|  1.0286e-06     	|
|  [LightningModule]NeRFSystem.configure_sharded_model                                                                                                                                                        	|  6.7204e-06     	|  1              	|  6.7204e-06     	|  9.6012e-07     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_start                                                                                                                                                        	|  3.2354e-06     	|  2              	|  6.4708e-06     	|  9.2446e-07     	|
|  [Callback]TQDMProgressBar.on_pretrain_routine_start                                                                                                                                                        	|  6.4597e-06     	|  1              	|  6.4597e-06     	|  9.2287e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_epoch_end            	|  3.1651e-06     	|  2              	|  6.3302e-06     	|  9.0437e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_epoch_end                                                                                                                                            	|  3.1451e-06     	|  2              	|  6.2902e-06     	|  8.9865e-07     	|
|  [Callback]TQDMProgressBar.on_configure_sharded_model                                                                                                                                                       	|  5.4697e-06     	|  1              	|  5.4697e-06     	|  7.8143e-07     	|
|  [Callback]GradientAccumulationScheduler.on_train_end                                                                                                                                                       	|  5.4603e-06     	|  1              	|  5.4603e-06     	|  7.801e-07      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_epoch_start          	|  2.5351e-06     	|  2              	|  5.0701e-06     	|  7.2435e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_epoch_start                                                                                                                                          	|  2.5104e-06     	|  2              	|  5.0208e-06     	|  7.173e-07      	|
|  [Strategy]SingleDeviceStrategy.on_validation_start                                                                                                                                                         	|  2.3548e-06     	|  2              	|  4.7097e-06     	|  6.7286e-07     	|
|  [Callback]GradientAccumulationScheduler.on_train_start                                                                                                                                                     	|  4.6995e-06     	|  1              	|  4.6995e-06     	|  6.7139e-07     	|
|  [Callback]TQDMProgressBar.teardown                                                                                                                                                                         	|  4.0997e-06     	|  1              	|  4.0997e-06     	|  5.8571e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_start                	|  2.0447e-06     	|  2              	|  4.0894e-06     	|  5.8424e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_start                     	|  4.0606e-06     	|  1              	|  4.0606e-06     	|  5.8012e-07     	|
|  [LightningModule]NeRFSystem.on_save_checkpoint                                                                                                                                                             	|  4.0606e-06     	|  1              	|  4.0606e-06     	|  5.8012e-07     	|
|  [Callback]TQDMProgressBar.on_fit_start                                                                                                                                                                     	|  3.8696e-06     	|  1              	|  3.8696e-06     	|  5.5284e-07     	|
|  [Callback]GradientAccumulationScheduler.on_configure_sharded_model                                                                                                                                         	|  3.7299e-06     	|  1              	|  3.7299e-06     	|  5.3288e-07     	|
|  [Callback]GradientAccumulationScheduler.setup                                                                                                                                                              	|  3.6908e-06     	|  1              	|  3.6908e-06     	|  5.273e-07      	|
|  [Callback]GradientAccumulationScheduler.on_fit_end                                                                                                                                                         	|  3.5902e-06     	|  1              	|  3.5902e-06     	|  5.1293e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_configure_sharded_model         	|  3.4599e-06     	|  1              	|  3.4599e-06     	|  4.943e-07      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_fit_end                         	|  3.39e-06       	|  1              	|  3.39e-06       	|  4.8432e-07     	|
|  [Callback]GradientAccumulationScheduler.teardown                                                                                                                                                           	|  3.3099e-06     	|  1              	|  3.3099e-06     	|  4.7288e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.teardown                           	|  3.1702e-06     	|  1              	|  3.1702e-06     	|  4.5292e-07     	|
|  [LightningModule]NeRFSystem.on_pretrain_routine_start                                                                                                                                                      	|  3.0193e-06     	|  1              	|  3.0193e-06     	|  4.3136e-07     	|
|  [LightningModule]NeRFSystem.on_validation_epoch_end                                                                                                                                                        	|  1.4897e-06     	|  2              	|  2.9793e-06     	|  4.2564e-07     	|
|  [Callback]TQDMProgressBar.on_before_accelerator_backend_setup                                                                                                                                              	|  2.9402e-06     	|  1              	|  2.9402e-06     	|  4.2005e-07     	|
|  [LightningModule]NeRFSystem.on_fit_start                                                                                                                                                                   	|  2.8405e-06     	|  1              	|  2.8405e-06     	|  4.0582e-07     	|
|  [Callback]TQDMProgressBar.on_pretrain_routine_end                                                                                                                                                          	|  2.6301e-06     	|  1              	|  2.6301e-06     	|  3.7575e-07     	|
|  [LightningModule]NeRFSystem.on_validation_epoch_start                                                                                                                                                      	|  1.2447e-06     	|  2              	|  2.4894e-06     	|  3.5566e-07     	|
|  [Callback]GradientAccumulationScheduler.on_fit_start                                                                                                                                                       	|  2.4596e-06     	|  1              	|  2.4596e-06     	|  3.514e-07      	|
|  [LightningModule]NeRFSystem.on_validation_end                                                                                                                                                              	|  1.2256e-06     	|  2              	|  2.4512e-06     	|  3.502e-07      	|
|  [Callback]GradientAccumulationScheduler.on_pretrain_routine_start                                                                                                                                          	|  2.21e-06       	|  1              	|  2.21e-06       	|  3.1574e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_fit_start                       	|  2.1998e-06     	|  1              	|  2.1998e-06     	|  3.1428e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_accelerator_backend_setup	|  2.1206e-06     	|  1              	|  2.1206e-06     	|  3.0297e-07     	|
|  [Callback]GradientAccumulationScheduler.on_before_accelerator_backend_setup                                                                                                                                	|  2.1104e-06     	|  1              	|  2.1104e-06     	|  3.015e-07      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_pretrain_routine_start          	|  2.0806e-06     	|  1              	|  2.0806e-06     	|  2.9724e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_end                       	|  2.0601e-06     	|  1              	|  2.0601e-06     	|  2.9432e-07     	|
|  [LightningModule]NeRFSystem.on_fit_end                                                                                                                                                                     	|  2.02e-06       	|  1              	|  2.02e-06       	|  2.886e-07      	|
|  [LightningModule]NeRFSystem.teardown                                                                                                                                                                       	|  1.9902e-06     	|  1              	|  1.9902e-06     	|  2.8434e-07     	|
|  [LightningModule]NeRFSystem.prepare_data                                                                                                                                                                   	|  1.9008e-06     	|  1              	|  1.9008e-06     	|  2.7156e-07     	|
|  [Callback]GradientAccumulationScheduler.on_pretrain_routine_end                                                                                                                                            	|  1.8999e-06     	|  1              	|  1.8999e-06     	|  2.7143e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_pretrain_routine_end            	|  1.8505e-06     	|  1              	|  1.8505e-06     	|  2.6438e-07     	|
|  [Strategy]SingleDeviceStrategy.on_train_end                                                                                                                                                                	|  1.3299e-06     	|  1              	|  1.3299e-06     	|  1.9e-07        	|
|  [Strategy]SingleDeviceStrategy.on_train_start                                                                                                                                                              	|  1.3197e-06     	|  1              	|  1.3197e-06     	|  1.8854e-07     	|
|  [LightningModule]NeRFSystem.on_train_start                                                                                                                                                                 	|  1.1194e-06     	|  1              	|  1.1194e-06     	|  1.5993e-07     	|
|  [LightningModule]NeRFSystem.on_pretrain_routine_end                                                                                                                                                        	|  1.0803e-06     	|  1              	|  1.0803e-06     	|  1.5434e-07     	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
