FIT Profiler Report

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Action                                                                                                                                                                                                     	|  Mean duration (s)	|  Num calls      	|  Total time (s) 	|  Percentage %   	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Total                                                                                                                                                                                                      	|  -              	|  881265         	|  1781.5         	|  100 %          	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  run_training_epoch                                                                                                                                                                                         	|  86.726         	|  20             	|  1734.5         	|  97.361         	|
|  run_training_batch                                                                                                                                                                                         	|  0.077859       	|  20000          	|  1557.2         	|  87.406         	|
|  [LightningModule]NeRFSystem.optimizer_step                                                                                                                                                                 	|  0.077489       	|  20000          	|  1549.8         	|  86.991         	|
|  [Strategy]SingleDeviceStrategy.training_step                                                                                                                                                               	|  0.021496       	|  20000          	|  429.92         	|  24.132         	|
|  [TrainingEpochLoop].train_dataloader_next                                                                                                                                                                  	|  0.0060069      	|  20000          	|  120.14         	|  6.7435         	|
|  [Strategy]SingleDeviceStrategy.backward                                                                                                                                                                    	|  0.0022691      	|  20000          	|  45.383         	|  2.5474         	|
|  [LightningModule]NeRFSystem.setup                                                                                                                                                                          	|  32.22          	|  1              	|  32.22          	|  1.8086         	|
|  [Strategy]SingleDeviceStrategy.validation_step                                                                                                                                                             	|  0.47851        	|  50             	|  23.925         	|  1.343          	|
|  [Callback]TQDMProgressBar.on_train_batch_end                                                                                                                                                               	|  0.00066568     	|  20000          	|  13.314         	|  0.74731        	|
|  [Strategy]SingleDeviceStrategy.batch_to_device                                                                                                                                                             	|  0.00020645     	|  20050          	|  4.1393         	|  0.23234        	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_epoch_end                 	|  0.16441        	|  20             	|  3.2882         	|  0.18457        	|
|  [LightningModule]NeRFSystem.transfer_batch_to_device                                                                                                                                                       	|  0.00012954     	|  20050          	|  2.5973         	|  0.14579        	|
|  [LightningModule]NeRFSystem.optimizer_zero_grad                                                                                                                                                            	|  3.1968e-05     	|  20000          	|  0.63937        	|  0.035889       	|
|  [EvaluationEpochLoop].val_dataloader_idx_0_next                                                                                                                                                            	|  0.0087323      	|  50             	|  0.43662        	|  0.024508       	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_batch_end                 	|  2.063e-05      	|  20000          	|  0.41261        	|  0.02316        	|
|  [LightningModule]NeRFSystem.configure_gradient_clipping                                                                                                                                                    	|  2.0057e-05     	|  20000          	|  0.40113        	|  0.022516       	|
|  [LightningModule]NeRFSystem.on_validation_start                                                                                                                                                            	|  0.11553        	|  2              	|  0.23106        	|  0.01297        	|
|  [Callback]GradientAccumulationScheduler.on_train_batch_end                                                                                                                                                 	|  6.1947e-06     	|  20000          	|  0.12389        	|  0.0069543      	|
|  [Callback]TQDMProgressBar.on_after_backward                                                                                                                                                                	|  4.1076e-06     	|  20000          	|  0.082152       	|  0.0046113      	|
|  [Callback]TQDMProgressBar.on_batch_start                                                                                                                                                                   	|  3.9631e-06     	|  20000          	|  0.079262       	|  0.0044491      	|
|  [Callback]TQDMProgressBar.on_before_backward                                                                                                                                                               	|  3.767e-06      	|  20000          	|  0.075341       	|  0.004229       	|
|  [Callback]TQDMProgressBar.on_before_optimizer_step                                                                                                                                                         	|  3.474e-06      	|  20000          	|  0.06948        	|  0.0039         	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_batch_start               	|  3.2237e-06     	|  20000          	|  0.064475       	|  0.0036191      	|
|  [Callback]TQDMProgressBar.on_before_zero_grad                                                                                                                                                              	|  3.2046e-06     	|  20000          	|  0.064093       	|  0.0035976      	|
|  [Callback]TQDMProgressBar.on_train_batch_start                                                                                                                                                             	|  3.0671e-06     	|  20000          	|  0.061342       	|  0.0034432      	|
|  [Callback]GradientAccumulationScheduler.on_batch_start                                                                                                                                                     	|  2.9665e-06     	|  20000          	|  0.059331       	|  0.0033303      	|
|  [LightningModule]NeRFSystem.on_before_batch_transfer                                                                                                                                                       	|  2.907e-06      	|  20050          	|  0.058285       	|  0.0032716      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_batch_start                     	|  2.8298e-06     	|  20000          	|  0.056596       	|  0.0031768      	|
|  [Callback]TQDMProgressBar.on_batch_end                                                                                                                                                                     	|  2.5026e-06     	|  20000          	|  0.050052       	|  0.0028095      	|
|  [Callback]GradientAccumulationScheduler.on_train_batch_start                                                                                                                                               	|  2.4489e-06     	|  20000          	|  0.048979       	|  0.0027492      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_after_backward                  	|  2.3118e-06     	|  20000          	|  0.046235       	|  0.0025953      	|
|  [Callback]GradientAccumulationScheduler.on_before_backward                                                                                                                                                 	|  2.2752e-06     	|  20000          	|  0.045503       	|  0.0025542      	|
|  [Callback]GradientAccumulationScheduler.on_before_optimizer_step                                                                                                                                           	|  2.2452e-06     	|  20000          	|  0.044904       	|  0.0025205      	|
|  [Callback]GradientAccumulationScheduler.on_before_zero_grad                                                                                                                                                	|  2.2361e-06     	|  20000          	|  0.044722       	|  0.0025103      	|
|  [Callback]GradientAccumulationScheduler.on_after_backward                                                                                                                                                  	|  2.2158e-06     	|  20000          	|  0.044316       	|  0.0024875      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_batch_end                       	|  2.1893e-06     	|  20000          	|  0.043787       	|  0.0024578      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_optimizer_step           	|  2.1779e-06     	|  20000          	|  0.043558       	|  0.002445       	|
|  [LightningModule]NeRFSystem.on_after_batch_transfer                                                                                                                                                        	|  2.1572e-06     	|  20050          	|  0.043252       	|  0.0024278      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_zero_grad                	|  2.1374e-06     	|  20000          	|  0.042748       	|  0.0023995      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_backward                 	|  2.1121e-06     	|  20000          	|  0.042243       	|  0.0023712      	|
|  [Callback]GradientAccumulationScheduler.on_batch_end                                                                                                                                                       	|  2.109e-06      	|  20000          	|  0.042181       	|  0.0023677      	|
|  [Strategy]SingleDeviceStrategy.on_train_batch_start                                                                                                                                                        	|  1.4942e-06     	|  20000          	|  0.029884       	|  0.0016774      	|
|  [LightningModule]NeRFSystem.on_train_batch_start                                                                                                                                                           	|  1.4816e-06     	|  20000          	|  0.029633       	|  0.0016633      	|
|  [LightningModule]NeRFSystem.training_step_end                                                                                                                                                              	|  1.4667e-06     	|  20000          	|  0.029335       	|  0.0016466      	|
|  [LightningModule]NeRFSystem.on_after_backward                                                                                                                                                              	|  1.4636e-06     	|  20000          	|  0.029271       	|  0.001643       	|
|  [Strategy]SingleDeviceStrategy.training_step_end                                                                                                                                                           	|  1.4246e-06     	|  20000          	|  0.028492       	|  0.0015993      	|
|  [LightningModule]NeRFSystem.on_train_batch_end                                                                                                                                                             	|  1.3232e-06     	|  20000          	|  0.026463       	|  0.0014854      	|
|  [LightningModule]NeRFSystem.on_before_zero_grad                                                                                                                                                            	|  1.1533e-06     	|  20000          	|  0.023066       	|  0.0012947      	|
|  [Callback]TQDMProgressBar.on_validation_batch_end                                                                                                                                                          	|  0.00044288     	|  50             	|  0.022144       	|  0.001243       	|
|  [LightningModule]NeRFSystem.on_before_backward                                                                                                                                                             	|  1.1004e-06     	|  20000          	|  0.022007       	|  0.0012353      	|
|  [LightningModule]NeRFSystem.on_before_optimizer_step                                                                                                                                                       	|  1.035e-06      	|  20000          	|  0.0207         	|  0.0011619      	|
|  [LightningModule]NeRFSystem.validation_epoch_end                                                                                                                                                           	|  0.0045962      	|  2              	|  0.0091925      	|  0.00051599     	|
|  [Callback]TQDMProgressBar.on_train_epoch_end                                                                                                                                                               	|  0.00032851     	|  20             	|  0.0065702      	|  0.0003688      	|
|  [Callback]TQDMProgressBar.on_train_epoch_start                                                                                                                                                             	|  0.00026165     	|  20             	|  0.0052329      	|  0.00029373     	|
|  [LightningModule]NeRFSystem.configure_optimizers                                                                                                                                                           	|  0.0035498      	|  1              	|  0.0035498      	|  0.00019925     	|
|  [Callback]TQDMProgressBar.on_validation_batch_start                                                                                                                                                        	|  5.2568e-05     	|  50             	|  0.0026284      	|  0.00014754     	|
|  [LightningModule]NeRFSystem.lr_scheduler_step                                                                                                                                                              	|  0.00013096     	|  20             	|  0.0026192      	|  0.00014702     	|
|  [LightningModule]NeRFSystem.on_validation_model_eval                                                                                                                                                       	|  0.00071874     	|  2              	|  0.0014375      	|  8.0687e-05     	|
|  [Callback]TQDMProgressBar.on_validation_end                                                                                                                                                                	|  0.00063609     	|  2              	|  0.0012722      	|  7.1409e-05     	|
|  [LightningModule]NeRFSystem.on_validation_model_train                                                                                                                                                      	|  0.0005707      	|  2              	|  0.0011414      	|  6.4068e-05     	|
|  [Callback]TQDMProgressBar.on_validation_start                                                                                                                                                              	|  0.00041709     	|  2              	|  0.00083417     	|  4.6823e-05     	|
|  [LightningModule]NeRFSystem.train_dataloader                                                                                                                                                               	|  0.00078168     	|  1              	|  0.00078168     	|  4.3877e-05     	|
|  [Callback]TQDMProgressBar.on_train_start                                                                                                                                                                   	|  0.00041684     	|  1              	|  0.00041684     	|  2.3398e-05     	|
|  [LightningModule]NeRFSystem.val_dataloader                                                                                                                                                                 	|  0.00039385     	|  1              	|  0.00039385     	|  2.2107e-05     	|
|  [Callback]GradientAccumulationScheduler.on_validation_batch_end                                                                                                                                            	|  7.1949e-06     	|  50             	|  0.00035975     	|  2.0193e-05     	|
|  [Callback]TQDMProgressBar.on_train_end                                                                                                                                                                     	|  0.00025781     	|  1              	|  0.00025781     	|  1.4471e-05     	|
|  [Callback]GradientAccumulationScheduler.on_validation_batch_start                                                                                                                                          	|  3.8308e-06     	|  50             	|  0.00019154     	|  1.0751e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_batch_end            	|  3.3496e-06     	|  50             	|  0.00016748     	|  9.4009e-06     	|
|  [LightningModule]NeRFSystem.validation_step_end                                                                                                                                                            	|  2.9127e-06     	|  50             	|  0.00014564     	|  8.1748e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_batch_start          	|  2.8514e-06     	|  50             	|  0.00014257     	|  8.0027e-06     	|
|  [Callback]GradientAccumulationScheduler.on_train_epoch_start                                                                                                                                               	|  6.4624e-06     	|  20             	|  0.00012925     	|  7.2548e-06     	|
|  [Callback]GradientAccumulationScheduler.on_train_epoch_end                                                                                                                                                 	|  6.0471e-06     	|  20             	|  0.00012094     	|  6.7886e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_end                  	|  5.4765e-05     	|  2              	|  0.00010953     	|  6.148e-06      	|
|  [LightningModule]NeRFSystem.on_validation_batch_end                                                                                                                                                        	|  1.7412e-06     	|  50             	|  8.706e-05      	|  4.8868e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.setup                              	|  8.1619e-05     	|  1              	|  8.1619e-05     	|  4.5814e-06     	|
|  [Callback]TQDMProgressBar.on_epoch_start                                                                                                                                                                   	|  3.6153e-06     	|  22             	|  7.9536e-05     	|  4.4645e-06     	|
|  [LightningModule]NeRFSystem.on_validation_batch_start                                                                                                                                                      	|  1.5222e-06     	|  50             	|  7.611e-05      	|  4.2721e-06     	|
|  [Callback]TQDMProgressBar.on_epoch_end                                                                                                                                                                     	|  3.3255e-06     	|  22             	|  7.3162e-05     	|  4.1067e-06     	|
|  [Strategy]SingleDeviceStrategy.validation_step_end                                                                                                                                                         	|  1.2737e-06     	|  50             	|  6.3687e-05     	|  3.5748e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_epoch_start               	|  2.9944e-06     	|  20             	|  5.9888e-05     	|  3.3616e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_epoch_end                       	|  2.3927e-06     	|  22             	|  5.2638e-05     	|  2.9547e-06     	|
|  [Callback]GradientAccumulationScheduler.on_epoch_end                                                                                                                                                       	|  2.3195e-06     	|  22             	|  5.103e-05      	|  2.8644e-06     	|
|  [Callback]GradientAccumulationScheduler.on_epoch_start                                                                                                                                                     	|  2.2587e-06     	|  22             	|  4.9691e-05     	|  2.7892e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_epoch_start                     	|  2.1954e-06     	|  22             	|  4.8299e-05     	|  2.7111e-06     	|
|  [LightningModule]NeRFSystem.on_train_epoch_end                                                                                                                                                             	|  1.7229e-06     	|  20             	|  3.4458e-05     	|  1.9342e-06     	|
|  [LightningModule]NeRFSystem.on_epoch_start                                                                                                                                                                 	|  1.4932e-06     	|  22             	|  3.2851e-05     	|  1.8439e-06     	|
|  [LightningModule]NeRFSystem.on_epoch_end                                                                                                                                                                   	|  1.4463e-06     	|  22             	|  3.1818e-05     	|  1.786e-06      	|
|  [LightningModule]NeRFSystem.on_train_epoch_start                                                                                                                                                           	|  1.4398e-06     	|  20             	|  2.8796e-05     	|  1.6164e-06     	|
|  [LightningModule]NeRFSystem.on_validation_epoch_end                                                                                                                                                        	|  1.25e-05       	|  2              	|  2.4999e-05     	|  1.4033e-06     	|
|  [Callback]TQDMProgressBar.setup                                                                                                                                                                            	|  1.9469e-05     	|  1              	|  1.9469e-05     	|  1.0928e-06     	|
|  [Strategy]SingleDeviceStrategy.on_validation_end                                                                                                                                                           	|  7.9651e-06     	|  2              	|  1.593e-05      	|  8.9419e-07     	|
|  [LightningModule]NeRFSystem.on_fit_end                                                                                                                                                                     	|  1.583e-05      	|  1              	|  1.583e-05      	|  8.8854e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_end                                                                                                                                                  	|  7.0799e-06     	|  2              	|  1.416e-05      	|  7.9481e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_start                                                                                                                                                	|  6.7595e-06     	|  2              	|  1.3519e-05     	|  7.5884e-07     	|
|  [LightningModule]NeRFSystem.configure_callbacks                                                                                                                                                            	|  1.2849e-05     	|  1              	|  1.2849e-05     	|  7.2126e-07     	|
|  [Callback]TQDMProgressBar.on_fit_end                                                                                                                                                                       	|  1.055e-05      	|  1              	|  1.055e-05      	|  5.9219e-07     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_end                                                                                                                                                          	|  4.8801e-06     	|  2              	|  9.7603e-06     	|  5.4786e-07     	|
|  [Callback]GradientAccumulationScheduler.on_train_start                                                                                                                                                     	|  8.3698e-06     	|  1              	|  8.3698e-06     	|  4.6981e-07     	|
|  [Callback]TQDMProgressBar.on_pretrain_routine_start                                                                                                                                                        	|  8.1398e-06     	|  1              	|  8.1398e-06     	|  4.569e-07      	|
|  [Callback]TQDMProgressBar.teardown                                                                                                                                                                         	|  6.5695e-06     	|  1              	|  6.5695e-06     	|  3.6876e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_start                     	|  6.2902e-06     	|  1              	|  6.2902e-06     	|  3.5307e-07     	|
|  [LightningModule]NeRFSystem.configure_sharded_model                                                                                                                                                        	|  5.98e-06       	|  1              	|  5.98e-06       	|  3.3567e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_start                	|  2.9844e-06     	|  2              	|  5.9688e-06     	|  3.3504e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_epoch_end                                                                                                                                            	|  2.8596e-06     	|  2              	|  5.7193e-06     	|  3.2103e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_epoch_end            	|  2.8452e-06     	|  2              	|  5.6904e-06     	|  3.1941e-07     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_start                                                                                                                                                        	|  2.7507e-06     	|  2              	|  5.5013e-06     	|  3.088e-07      	|
|  [Strategy]SingleDeviceStrategy.on_train_end                                                                                                                                                                	|  5.4995e-06     	|  1              	|  5.4995e-06     	|  3.0869e-07     	|
|  [Callback]GradientAccumulationScheduler.on_fit_end                                                                                                                                                         	|  5.3104e-06     	|  1              	|  5.3104e-06     	|  2.9808e-07     	|
|  [Callback]TQDMProgressBar.on_fit_start                                                                                                                                                                     	|  5.1996e-06     	|  1              	|  5.1996e-06     	|  2.9186e-07     	|
|  [Callback]GradientAccumulationScheduler.teardown                                                                                                                                                           	|  5.1102e-06     	|  1              	|  5.1102e-06     	|  2.8684e-07     	|
|  [Callback]GradientAccumulationScheduler.setup                                                                                                                                                              	|  4.9602e-06     	|  1              	|  4.9602e-06     	|  2.7842e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_fit_end                         	|  4.8503e-06     	|  1              	|  4.8503e-06     	|  2.7226e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.teardown                           	|  4.7795e-06     	|  1              	|  4.7795e-06     	|  2.6828e-07     	|
|  [LightningModule]NeRFSystem.on_fit_start                                                                                                                                                                   	|  4.7395e-06     	|  1              	|  4.7395e-06     	|  2.6603e-07     	|
|  [Callback]GradientAccumulationScheduler.on_train_end                                                                                                                                                       	|  4.6901e-06     	|  1              	|  4.6901e-06     	|  2.6326e-07     	|
|  [LightningModule]NeRFSystem.on_pretrain_routine_start                                                                                                                                                      	|  4.1602e-06     	|  1              	|  4.1602e-06     	|  2.3352e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_epoch_start          	|  1.8547e-06     	|  2              	|  3.7095e-06     	|  2.0822e-07     	|
|  [Callback]TQDMProgressBar.on_configure_sharded_model                                                                                                                                                       	|  3.6703e-06     	|  1              	|  3.6703e-06     	|  2.0602e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_epoch_start                                                                                                                                          	|  1.8105e-06     	|  2              	|  3.621e-06      	|  2.0325e-07     	|
|  [Strategy]SingleDeviceStrategy.on_validation_start                                                                                                                                                         	|  1.81e-06       	|  2              	|  3.6201e-06     	|  2.032e-07      	|
|  [Callback]TQDMProgressBar.on_before_accelerator_backend_setup                                                                                                                                              	|  3.35e-06       	|  1              	|  3.35e-06       	|  1.8804e-07     	|
|  [Callback]GradientAccumulationScheduler.on_fit_start                                                                                                                                                       	|  3.3397e-06     	|  1              	|  3.3397e-06     	|  1.8746e-07     	|
|  [Callback]TQDMProgressBar.on_pretrain_routine_end                                                                                                                                                          	|  3.3099e-06     	|  1              	|  3.3099e-06     	|  1.8579e-07     	|
|  [LightningModule]NeRFSystem.on_validation_end                                                                                                                                                              	|  1.6354e-06     	|  2              	|  3.2708e-06     	|  1.8359e-07     	|
|  [Callback]GradientAccumulationScheduler.on_pretrain_routine_start                                                                                                                                          	|  3.2494e-06     	|  1              	|  3.2494e-06     	|  1.8239e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_fit_start                       	|  3.1404e-06     	|  1              	|  3.1404e-06     	|  1.7628e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_pretrain_routine_start          	|  3.0799e-06     	|  1              	|  3.0799e-06     	|  1.7288e-07     	|
|  [LightningModule]NeRFSystem.on_validation_epoch_start                                                                                                                                                      	|  1.465e-06      	|  2              	|  2.9299e-06     	|  1.6446e-07     	|
|  [Callback]GradientAccumulationScheduler.on_pretrain_routine_end                                                                                                                                            	|  2.8303e-06     	|  1              	|  2.8303e-06     	|  1.5887e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_pretrain_routine_end            	|  2.8303e-06     	|  1              	|  2.8303e-06     	|  1.5887e-07     	|
|  [Callback]GradientAccumulationScheduler.on_before_accelerator_backend_setup                                                                                                                                	|  2.8294e-06     	|  1              	|  2.8294e-06     	|  1.5882e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_accelerator_backend_setup	|  2.7996e-06     	|  1              	|  2.7996e-06     	|  1.5714e-07     	|
|  [LightningModule]NeRFSystem.prepare_data                                                                                                                                                                   	|  2.6198e-06     	|  1              	|  2.6198e-06     	|  1.4705e-07     	|
|  [LightningModule]NeRFSystem.teardown                                                                                                                                                                       	|  2.6096e-06     	|  1              	|  2.6096e-06     	|  1.4648e-07     	|
|  [Callback]GradientAccumulationScheduler.on_configure_sharded_model                                                                                                                                         	|  2.5099e-06     	|  1              	|  2.5099e-06     	|  1.4088e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_configure_sharded_model         	|  2.4103e-06     	|  1              	|  2.4103e-06     	|  1.3529e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_end                       	|  2.2007e-06     	|  1              	|  2.2007e-06     	|  1.2353e-07     	|
|  [LightningModule]NeRFSystem.on_save_checkpoint                                                                                                                                                             	|  2.0098e-06     	|  1              	|  2.0098e-06     	|  1.1281e-07     	|
|  [LightningModule]NeRFSystem.on_train_end                                                                                                                                                                   	|  1.75e-06       	|  1              	|  1.75e-06       	|  9.8227e-08     	|
|  [LightningModule]NeRFSystem.on_train_start                                                                                                                                                                 	|  1.7099e-06     	|  1              	|  1.7099e-06     	|  9.5979e-08     	|
|  [LightningModule]NeRFSystem.on_pretrain_routine_end                                                                                                                                                        	|  1.6391e-06     	|  1              	|  1.6391e-06     	|  9.2006e-08     	|
|  [Strategy]SingleDeviceStrategy.on_train_start                                                                                                                                                              	|  1.4398e-06     	|  1              	|  1.4398e-06     	|  8.0819e-08     	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
