FIT Profiler Report

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Action                                                                                                                                                                                                     	|  Mean duration (s)	|  Num calls      	|  Total time (s) 	|  Percentage %   	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Total                                                                                                                                                                                                      	|  -              	|  881489         	|  2071.4         	|  100 %          	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  run_training_epoch                                                                                                                                                                                         	|  102.54         	|  20             	|  2050.9         	|  99.011         	|
|  run_training_batch                                                                                                                                                                                         	|  0.096174       	|  20000          	|  1923.5         	|  92.86          	|
|  [LightningModule]NeRFSystem.optimizer_step                                                                                                                                                                 	|  0.095886       	|  20000          	|  1917.7         	|  92.582         	|
|  [Strategy]SingleDeviceStrategy.training_step                                                                                                                                                               	|  0.024881       	|  20000          	|  497.63         	|  24.024         	|
|  [Strategy]SingleDeviceStrategy.backward                                                                                                                                                                    	|  0.0044837      	|  20000          	|  89.673         	|  4.3292         	|
|  [Strategy]SingleDeviceStrategy.validation_step                                                                                                                                                             	|  1.1227         	|  64             	|  71.854         	|  3.4689         	|
|  [TrainingEpochLoop].train_dataloader_next                                                                                                                                                                  	|  0.001722       	|  20000          	|  34.441         	|  1.6627         	|
|  [LightningModule]NeRFSystem.setup                                                                                                                                                                          	|  9.862          	|  1              	|  9.862          	|  0.47611        	|
|  [Callback]TQDMProgressBar.on_train_batch_end                                                                                                                                                               	|  0.00041987     	|  20000          	|  8.3974         	|  0.4054         	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_epoch_end                 	|  0.21408        	|  20             	|  4.2816         	|  0.2067         	|
|  [Strategy]SingleDeviceStrategy.batch_to_device                                                                                                                                                             	|  0.0001107      	|  20064          	|  2.2211         	|  0.10723        	|
|  [LightningModule]NeRFSystem.transfer_batch_to_device                                                                                                                                                       	|  7.0028e-05     	|  20064          	|  1.4051         	|  0.067832       	|
|  [LightningModule]NeRFSystem.optimizer_zero_grad                                                                                                                                                            	|  2.1268e-05     	|  20000          	|  0.42537        	|  0.020536       	|
|  [EvaluationEpochLoop].val_dataloader_idx_0_next                                                                                                                                                            	|  0.005758       	|  64             	|  0.36851        	|  0.017791       	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_batch_end                 	|  1.3612e-05     	|  20000          	|  0.27224        	|  0.013143       	|
|  [LightningModule]NeRFSystem.configure_gradient_clipping                                                                                                                                                    	|  1.242e-05      	|  20000          	|  0.2484         	|  0.011992       	|
|  [LightningModule]NeRFSystem.on_validation_start                                                                                                                                                            	|  0.048009       	|  2              	|  0.096019       	|  0.0046355      	|
|  [Callback]GradientAccumulationScheduler.on_train_batch_end                                                                                                                                                 	|  4.2615e-06     	|  20000          	|  0.085231       	|  0.0041147      	|
|  [Callback]TQDMProgressBar.on_after_backward                                                                                                                                                                	|  3.1938e-06     	|  20000          	|  0.063877       	|  0.0030838      	|
|  [Callback]TQDMProgressBar.on_batch_start                                                                                                                                                                   	|  3.0081e-06     	|  20000          	|  0.060161       	|  0.0029044      	|
|  [Callback]TQDMProgressBar.on_before_optimizer_step                                                                                                                                                         	|  2.9624e-06     	|  20000          	|  0.059247       	|  0.0028603      	|
|  [Callback]TQDMProgressBar.on_before_zero_grad                                                                                                                                                              	|  2.664e-06      	|  20000          	|  0.05328        	|  0.0025722      	|
|  [Callback]TQDMProgressBar.on_before_backward                                                                                                                                                               	|  2.6067e-06     	|  20000          	|  0.052135       	|  0.0025169      	|
|  [Callback]GradientAccumulationScheduler.on_batch_start                                                                                                                                                     	|  2.3896e-06     	|  20000          	|  0.047791       	|  0.0023072      	|
|  [Callback]TQDMProgressBar.on_train_batch_start                                                                                                                                                             	|  2.38e-06       	|  20000          	|  0.0476         	|  0.002298       	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_batch_start                     	|  2.304e-06      	|  20000          	|  0.04608        	|  0.0022246      	|
|  [Callback]GradientAccumulationScheduler.on_train_batch_start                                                                                                                                               	|  2.2273e-06     	|  20000          	|  0.044546       	|  0.0021505      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_batch_start               	|  2.2055e-06     	|  20000          	|  0.04411        	|  0.0021295      	|
|  [Callback]TQDMProgressBar.on_batch_end                                                                                                                                                                     	|  2.1239e-06     	|  20000          	|  0.042477       	|  0.0020507      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_after_backward                  	|  2.0999e-06     	|  20000          	|  0.041998       	|  0.0020275      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_optimizer_step           	|  2.0546e-06     	|  20000          	|  0.041093       	|  0.0019838      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_batch_end                       	|  2.0238e-06     	|  20000          	|  0.040476       	|  0.0019541      	|
|  [Callback]GradientAccumulationScheduler.on_before_optimizer_step                                                                                                                                           	|  2.0117e-06     	|  20000          	|  0.040235       	|  0.0019424      	|
|  [Callback]GradientAccumulationScheduler.on_after_backward                                                                                                                                                  	|  1.9898e-06     	|  20000          	|  0.039795       	|  0.0019212      	|
|  [Callback]GradientAccumulationScheduler.on_before_backward                                                                                                                                                 	|  1.9697e-06     	|  20000          	|  0.039395       	|  0.0019019      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_zero_grad                	|  1.921e-06      	|  20000          	|  0.03842        	|  0.0018548      	|
|  [Callback]GradientAccumulationScheduler.on_before_zero_grad                                                                                                                                                	|  1.9174e-06     	|  20000          	|  0.038348       	|  0.0018513      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_backward                 	|  1.9125e-06     	|  20000          	|  0.038251       	|  0.0018466      	|
|  [Callback]GradientAccumulationScheduler.on_batch_end                                                                                                                                                       	|  1.8464e-06     	|  20000          	|  0.036929       	|  0.0017828      	|
|  [Callback]TQDMProgressBar.on_validation_batch_end                                                                                                                                                          	|  0.0005248      	|  64             	|  0.033587       	|  0.0016215      	|
|  [LightningModule]NeRFSystem.on_before_batch_transfer                                                                                                                                                       	|  1.1205e-06     	|  20064          	|  0.022481       	|  0.0010853      	|
|  [LightningModule]NeRFSystem.training_step_end                                                                                                                                                              	|  1.1026e-06     	|  20000          	|  0.022051       	|  0.0010646      	|
|  [LightningModule]NeRFSystem.on_after_batch_transfer                                                                                                                                                        	|  1.079e-06      	|  20064          	|  0.021649       	|  0.0010451      	|
|  [LightningModule]NeRFSystem.on_train_batch_end                                                                                                                                                             	|  9.4856e-07     	|  20000          	|  0.018971       	|  0.00091587     	|
|  [Strategy]SingleDeviceStrategy.training_step_end                                                                                                                                                           	|  9.0366e-07     	|  20000          	|  0.018073       	|  0.00087253     	|
|  [LightningModule]NeRFSystem.on_train_batch_start                                                                                                                                                           	|  8.8485e-07     	|  20000          	|  0.017697       	|  0.00085436     	|
|  [LightningModule]NeRFSystem.on_before_optimizer_step                                                                                                                                                       	|  8.1162e-07     	|  20000          	|  0.016232       	|  0.00078365     	|
|  [LightningModule]NeRFSystem.on_after_backward                                                                                                                                                              	|  7.8658e-07     	|  20000          	|  0.015732       	|  0.00075948     	|
|  [LightningModule]NeRFSystem.on_before_zero_grad                                                                                                                                                            	|  7.6619e-07     	|  20000          	|  0.015324       	|  0.00073979     	|
|  [LightningModule]NeRFSystem.on_before_backward                                                                                                                                                             	|  7.5767e-07     	|  20000          	|  0.015153       	|  0.00073156     	|
|  [Strategy]SingleDeviceStrategy.on_train_batch_start                                                                                                                                                        	|  7.4508e-07     	|  20000          	|  0.014902       	|  0.00071941     	|
|  [LightningModule]NeRFSystem.validation_epoch_end                                                                                                                                                           	|  0.0048579      	|  2              	|  0.0097157      	|  0.00046905     	|
|  [LightningModule]NeRFSystem.configure_optimizers                                                                                                                                                           	|  0.0067627      	|  1              	|  0.0067627      	|  0.00032648     	|
|  [Callback]TQDMProgressBar.on_train_epoch_start                                                                                                                                                             	|  0.00022334     	|  20             	|  0.0044668      	|  0.00021564     	|
|  [Callback]TQDMProgressBar.on_train_epoch_end                                                                                                                                                               	|  0.00021971     	|  20             	|  0.0043941      	|  0.00021214     	|
|  [Callback]TQDMProgressBar.on_validation_batch_start                                                                                                                                                        	|  5.2142e-05     	|  64             	|  0.0033371      	|  0.00016111     	|
|  [LightningModule]NeRFSystem.lr_scheduler_step                                                                                                                                                              	|  9.5614e-05     	|  20             	|  0.0019123      	|  9.2319e-05     	|
|  [Callback]TQDMProgressBar.on_validation_end                                                                                                                                                                	|  0.00071406     	|  2              	|  0.0014281      	|  6.8945e-05     	|
|  [LightningModule]NeRFSystem.on_validation_model_train                                                                                                                                                      	|  0.00048744     	|  2              	|  0.00097488     	|  4.7064e-05     	|
|  [LightningModule]NeRFSystem.on_validation_model_eval                                                                                                                                                       	|  0.00044003     	|  2              	|  0.00088006     	|  4.2487e-05     	|
|  [LightningModule]NeRFSystem.train_dataloader                                                                                                                                                               	|  0.00052583     	|  1              	|  0.00052583     	|  2.5386e-05     	|
|  [Callback]GradientAccumulationScheduler.on_validation_batch_end                                                                                                                                            	|  7.443e-06      	|  64             	|  0.00047635     	|  2.2997e-05     	|
|  [Callback]TQDMProgressBar.on_train_end                                                                                                                                                                     	|  0.00046038     	|  1              	|  0.00046038     	|  2.2226e-05     	|
|  [Callback]TQDMProgressBar.on_validation_start                                                                                                                                                              	|  0.00019808     	|  2              	|  0.00039616     	|  1.9125e-05     	|
|  [Callback]GradientAccumulationScheduler.on_validation_batch_start                                                                                                                                          	|  5.1127e-06     	|  64             	|  0.00032722     	|  1.5797e-05     	|
|  [LightningModule]NeRFSystem.val_dataloader                                                                                                                                                                 	|  0.00027981     	|  1              	|  0.00027981     	|  1.3508e-05     	|
|  [Callback]TQDMProgressBar.on_train_start                                                                                                                                                                   	|  0.0002515      	|  1              	|  0.0002515      	|  1.2142e-05     	|
|  [LightningModule]NeRFSystem.validation_step_end                                                                                                                                                            	|  3.4873e-06     	|  64             	|  0.00022319     	|  1.0775e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_batch_start          	|  3.418e-06      	|  64             	|  0.00021875     	|  1.0561e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_batch_end            	|  3.4089e-06     	|  64             	|  0.00021817     	|  1.0532e-05     	|
|  [LightningModule]NeRFSystem.on_validation_batch_end                                                                                                                                                        	|  2.5602e-06     	|  64             	|  0.00016385     	|  7.9102e-06     	|
|  [LightningModule]NeRFSystem.on_validation_batch_start                                                                                                                                                      	|  2.2098e-06     	|  64             	|  0.00014143     	|  6.8277e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_end                  	|  5.7979e-05     	|  2              	|  0.00011596     	|  5.5982e-06     	|
|  [Callback]GradientAccumulationScheduler.on_train_epoch_start                                                                                                                                               	|  5.4218e-06     	|  20             	|  0.00010844     	|  5.235e-06      	|
|  [Strategy]SingleDeviceStrategy.validation_step_end                                                                                                                                                         	|  1.4136e-06     	|  64             	|  9.0468e-05     	|  4.3675e-06     	|
|  [Callback]GradientAccumulationScheduler.on_train_epoch_end                                                                                                                                                 	|  4.3633e-06     	|  20             	|  8.7267e-05     	|  4.213e-06      	|
|  [Callback]TQDMProgressBar.on_epoch_start                                                                                                                                                                   	|  3.0618e-06     	|  22             	|  6.736e-05      	|  3.2519e-06     	|
|  [Callback]TQDMProgressBar.on_epoch_end                                                                                                                                                                     	|  2.75e-06       	|  22             	|  6.0501e-05     	|  2.9208e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_epoch_start               	|  2.4779e-06     	|  20             	|  4.9558e-05     	|  2.3925e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_epoch_end                       	|  2.09e-06       	|  22             	|  4.5979e-05     	|  2.2198e-06     	|
|  [Callback]GradientAccumulationScheduler.on_epoch_end                                                                                                                                                       	|  2.0195e-06     	|  22             	|  4.443e-05      	|  2.1449e-06     	|
|  [Callback]GradientAccumulationScheduler.on_epoch_start                                                                                                                                                     	|  1.9713e-06     	|  22             	|  4.3369e-05     	|  2.0937e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_epoch_start                     	|  1.8832e-06     	|  22             	|  4.143e-05      	|  2.0001e-06     	|
|  [LightningModule]NeRFSystem.on_epoch_end                                                                                                                                                                   	|  1.6501e-06     	|  22             	|  3.6302e-05     	|  1.7526e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.setup                              	|  3.084e-05      	|  1              	|  3.084e-05      	|  1.4889e-06     	|
|  [LightningModule]NeRFSystem.on_train_epoch_end                                                                                                                                                             	|  1.4254e-06     	|  20             	|  2.8509e-05     	|  1.3763e-06     	|
|  [LightningModule]NeRFSystem.on_epoch_start                                                                                                                                                                 	|  1.0754e-06     	|  22             	|  2.3659e-05     	|  1.1422e-06     	|
|  [LightningModule]NeRFSystem.on_train_epoch_start                                                                                                                                                           	|  1.0875e-06     	|  20             	|  2.1749e-05     	|  1.05e-06       	|
|  [Callback]GradientAccumulationScheduler.on_validation_end                                                                                                                                                  	|  1.026e-05      	|  2              	|  2.052e-05      	|  9.9064e-07     	|
|  [LightningModule]NeRFSystem.configure_callbacks                                                                                                                                                            	|  1.0289e-05     	|  1              	|  1.0289e-05     	|  4.9674e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_start                                                                                                                                                	|  5.1446e-06     	|  2              	|  1.0289e-05     	|  4.9674e-07     	|
|  [LightningModule]NeRFSystem.on_validation_epoch_end                                                                                                                                                        	|  4.9998e-06     	|  2              	|  9.9996e-06     	|  4.8275e-07     	|
|  [Callback]GradientAccumulationScheduler.on_train_end                                                                                                                                                       	|  9.3896e-06     	|  1              	|  9.3896e-06     	|  4.533e-07      	|
|  [Strategy]SingleDeviceStrategy.on_validation_end                                                                                                                                                           	|  4.17e-06       	|  2              	|  8.34e-06       	|  4.0263e-07     	|
|  [LightningModule]NeRFSystem.on_save_checkpoint                                                                                                                                                             	|  7.7803e-06     	|  1              	|  7.7803e-06     	|  3.7561e-07     	|
|  [Callback]TQDMProgressBar.setup                                                                                                                                                                            	|  7.2392e-06     	|  1              	|  7.2392e-06     	|  3.4949e-07     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_end                                                                                                                                                          	|  3.5046e-06     	|  2              	|  7.0091e-06     	|  3.3838e-07     	|
|  [Callback]TQDMProgressBar.on_fit_end                                                                                                                                                                       	|  6.2389e-06     	|  1              	|  6.2389e-06     	|  3.012e-07      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_end                       	|  5.2899e-06     	|  1              	|  5.2899e-06     	|  2.5538e-07     	|
|  [Callback]GradientAccumulationScheduler.on_train_start                                                                                                                                                     	|  5.0999e-06     	|  1              	|  5.0999e-06     	|  2.4621e-07     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_start                                                                                                                                                        	|  2.4703e-06     	|  2              	|  4.9407e-06     	|  2.3852e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_start                	|  2.4047e-06     	|  2              	|  4.8093e-06     	|  2.3218e-07     	|
|  [Callback]TQDMProgressBar.on_fit_start                                                                                                                                                                     	|  4.7591e-06     	|  1              	|  4.7591e-06     	|  2.2975e-07     	|
|  [Callback]TQDMProgressBar.on_pretrain_routine_start                                                                                                                                                        	|  4.5998e-06     	|  1              	|  4.5998e-06     	|  2.2207e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_start                     	|  4.2403e-06     	|  1              	|  4.2403e-06     	|  2.0471e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_epoch_end                                                                                                                                            	|  2.0349e-06     	|  2              	|  4.0699e-06     	|  1.9648e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_epoch_end            	|  2e-06          	|  2              	|  4e-06          	|  1.9311e-07     	|
|  [Strategy]SingleDeviceStrategy.on_validation_start                                                                                                                                                         	|  1.9749e-06     	|  2              	|  3.9497e-06     	|  1.9068e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_epoch_start          	|  1.9451e-06     	|  2              	|  3.8901e-06     	|  1.878e-07      	|
|  [Callback]GradientAccumulationScheduler.on_validation_epoch_start                                                                                                                                          	|  1.9101e-06     	|  2              	|  3.8203e-06     	|  1.8443e-07     	|
|  [LightningModule]NeRFSystem.on_fit_start                                                                                                                                                                   	|  3.6303e-06     	|  1              	|  3.6303e-06     	|  1.7526e-07     	|
|  [Callback]TQDMProgressBar.on_configure_sharded_model                                                                                                                                                       	|  3.3397e-06     	|  1              	|  3.3397e-06     	|  1.6123e-07     	|
|  [Callback]GradientAccumulationScheduler.setup                                                                                                                                                              	|  3.2503e-06     	|  1              	|  3.2503e-06     	|  1.5692e-07     	|
|  [Callback]TQDMProgressBar.on_pretrain_routine_end                                                                                                                                                          	|  3.2503e-06     	|  1              	|  3.2503e-06     	|  1.5692e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_fit_start                       	|  3.2308e-06     	|  1              	|  3.2308e-06     	|  1.5597e-07     	|
|  [Callback]GradientAccumulationScheduler.on_fit_start                                                                                                                                                       	|  3.2e-06        	|  1              	|  3.2e-06        	|  1.5449e-07     	|
|  [Callback]TQDMProgressBar.on_before_accelerator_backend_setup                                                                                                                                              	|  3.16e-06       	|  1              	|  3.16e-06       	|  1.5255e-07     	|
|  [LightningModule]NeRFSystem.on_train_end                                                                                                                                                                   	|  3.01e-06       	|  1              	|  3.01e-06       	|  1.4532e-07     	|
|  [Callback]TQDMProgressBar.teardown                                                                                                                                                                         	|  2.9299e-06     	|  1              	|  2.9299e-06     	|  1.4145e-07     	|
|  [LightningModule]NeRFSystem.on_validation_end                                                                                                                                                              	|  1.3248e-06     	|  2              	|  2.6496e-06     	|  1.2792e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_accelerator_backend_setup	|  2.6301e-06     	|  1              	|  2.6301e-06     	|  1.2697e-07     	|
|  [Callback]GradientAccumulationScheduler.on_before_accelerator_backend_setup                                                                                                                                	|  2.55e-06       	|  1              	|  2.55e-06       	|  1.231e-07      	|
|  [Callback]GradientAccumulationScheduler.on_configure_sharded_model                                                                                                                                         	|  2.55e-06       	|  1              	|  2.55e-06       	|  1.231e-07      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_configure_sharded_model         	|  2.4699e-06     	|  1              	|  2.4699e-06     	|  1.1924e-07     	|
|  [LightningModule]NeRFSystem.on_validation_epoch_start                                                                                                                                                      	|  1.0952e-06     	|  2              	|  2.1905e-06     	|  1.0575e-07     	|
|  [LightningModule]NeRFSystem.on_pretrain_routine_start                                                                                                                                                      	|  2.17e-06       	|  1              	|  2.17e-06       	|  1.0476e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_fit_end                         	|  2.1597e-06     	|  1              	|  2.1597e-06     	|  1.0427e-07     	|
|  [LightningModule]NeRFSystem.configure_sharded_model                                                                                                                                                        	|  2.1402e-06     	|  1              	|  2.1402e-06     	|  1.0332e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_pretrain_routine_start          	|  2.1197e-06     	|  1              	|  2.1197e-06     	|  1.0233e-07     	|
|  [Callback]GradientAccumulationScheduler.on_fit_end                                                                                                                                                         	|  2.1104e-06     	|  1              	|  2.1104e-06     	|  1.0188e-07     	|
|  [Callback]GradientAccumulationScheduler.on_pretrain_routine_start                                                                                                                                          	|  2.0899e-06     	|  1              	|  2.0899e-06     	|  1.0089e-07     	|
|  [Callback]GradientAccumulationScheduler.teardown                                                                                                                                                           	|  1.9502e-06     	|  1              	|  1.9502e-06     	|  9.415e-08      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.teardown                           	|  1.9195e-06     	|  1              	|  1.9195e-06     	|  9.2666e-08     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_pretrain_routine_end            	|  1.8897e-06     	|  1              	|  1.8897e-06     	|  9.1227e-08     	|
|  [LightningModule]NeRFSystem.prepare_data                                                                                                                                                                   	|  1.8794e-06     	|  1              	|  1.8794e-06     	|  9.0732e-08     	|
|  [Callback]GradientAccumulationScheduler.on_pretrain_routine_end                                                                                                                                            	|  1.8599e-06     	|  1              	|  1.8599e-06     	|  8.9788e-08     	|
|  [LightningModule]NeRFSystem.on_pretrain_routine_end                                                                                                                                                        	|  1.7993e-06     	|  1              	|  1.7993e-06     	|  8.6866e-08     	|
|  [Strategy]SingleDeviceStrategy.on_train_end                                                                                                                                                                	|  1.79e-06       	|  1              	|  1.79e-06       	|  8.6416e-08     	|
|  [LightningModule]NeRFSystem.teardown                                                                                                                                                                       	|  1.6298e-06     	|  1              	|  1.6298e-06     	|  7.8683e-08     	|
|  [LightningModule]NeRFSystem.on_fit_end                                                                                                                                                                     	|  1.5302e-06     	|  1              	|  1.5302e-06     	|  7.3872e-08     	|
|  [Strategy]SingleDeviceStrategy.on_train_start                                                                                                                                                              	|  1.3802e-06     	|  1              	|  1.3802e-06     	|  6.6633e-08     	|
|  [LightningModule]NeRFSystem.on_train_start                                                                                                                                                                 	|  1.1697e-06     	|  1              	|  1.1697e-06     	|  5.6472e-08     	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
