FIT Profiler Report

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Action                                                                                                                                                                                                     	|  Mean duration (s)	|  Num calls      	|  Total time (s) 	|  Percentage %   	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Total                                                                                                                                                                                                      	|  -              	|  881937         	|  840.15         	|  100 %          	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  run_training_epoch                                                                                                                                                                                         	|  39.834         	|  20             	|  796.67         	|  94.825         	|
|  run_training_batch                                                                                                                                                                                         	|  0.025065       	|  20000          	|  501.31         	|  59.669         	|
|  [LightningModule]NeRFSystem.optimizer_step                                                                                                                                                                 	|  0.024592       	|  20000          	|  491.85         	|  58.542         	|
|  [Strategy]SingleDeviceStrategy.training_step                                                                                                                                                               	|  0.011201       	|  20000          	|  224.02         	|  26.665         	|
|  [TrainingEpochLoop].train_dataloader_next                                                                                                                                                                  	|  0.0079266      	|  20000          	|  158.53         	|  18.869         	|
|  [Strategy]SingleDeviceStrategy.backward                                                                                                                                                                    	|  0.0049945      	|  20000          	|  99.891         	|  11.89          	|
|  [Strategy]SingleDeviceStrategy.validation_step                                                                                                                                                             	|  1.0487         	|  92             	|  96.479         	|  11.483         	|
|  [LightningModule]NeRFSystem.setup                                                                                                                                                                          	|  26.069         	|  1              	|  26.069         	|  3.1029         	|
|  [Callback]TQDMProgressBar.on_train_batch_end                                                                                                                                                               	|  0.00080799     	|  20000          	|  16.16          	|  1.9234         	|
|  [Strategy]SingleDeviceStrategy.batch_to_device                                                                                                                                                             	|  0.00023786     	|  20092          	|  4.7791         	|  0.56884        	|
|  [LightningModule]NeRFSystem.transfer_batch_to_device                                                                                                                                                       	|  0.00014796     	|  20092          	|  2.9727         	|  0.35383        	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_epoch_end                 	|  0.1413         	|  20             	|  2.8261         	|  0.33638        	|
|  [EvaluationEpochLoop].val_dataloader_idx_0_next                                                                                                                                                            	|  0.015055       	|  92             	|  1.385          	|  0.16486        	|
|  [LightningModule]NeRFSystem.optimizer_zero_grad                                                                                                                                                            	|  4.3258e-05     	|  20000          	|  0.86516        	|  0.10298        	|
|  [LightningModule]NeRFSystem.configure_gradient_clipping                                                                                                                                                    	|  2.4008e-05     	|  20000          	|  0.48016        	|  0.057151       	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_batch_end                 	|  2.2987e-05     	|  20000          	|  0.45974        	|  0.054721       	|
|  [Callback]GradientAccumulationScheduler.on_train_batch_end                                                                                                                                                 	|  7.3243e-06     	|  20000          	|  0.14649        	|  0.017436       	|
|  [Callback]TQDMProgressBar.on_after_backward                                                                                                                                                                	|  5.4223e-06     	|  20000          	|  0.10845        	|  0.012908       	|
|  [Callback]GradientAccumulationScheduler.on_batch_start                                                                                                                                                     	|  4.3495e-06     	|  20000          	|  0.08699        	|  0.010354       	|
|  [Callback]TQDMProgressBar.on_batch_end                                                                                                                                                                     	|  4.2514e-06     	|  20000          	|  0.085028       	|  0.01012        	|
|  [Callback]TQDMProgressBar.on_before_zero_grad                                                                                                                                                              	|  4.1937e-06     	|  20000          	|  0.083874       	|  0.0099831      	|
|  [Callback]TQDMProgressBar.on_batch_start                                                                                                                                                                   	|  4.1467e-06     	|  20000          	|  0.082934       	|  0.0098712      	|
|  [Callback]TQDMProgressBar.on_before_optimizer_step                                                                                                                                                         	|  3.927e-06      	|  20000          	|  0.078539       	|  0.0093482      	|
|  [Callback]TQDMProgressBar.on_before_backward                                                                                                                                                               	|  3.859e-06      	|  20000          	|  0.07718        	|  0.0091864      	|
|  [Callback]TQDMProgressBar.on_train_batch_start                                                                                                                                                             	|  3.8511e-06     	|  20000          	|  0.077022       	|  0.0091675      	|
|  [LightningModule]NeRFSystem.on_before_batch_transfer                                                                                                                                                       	|  3.8032e-06     	|  20092          	|  0.076413       	|  0.0090951      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_batch_start               	|  3.6919e-06     	|  20000          	|  0.073837       	|  0.0087886      	|
|  [LightningModule]NeRFSystem.on_validation_start                                                                                                                                                            	|  0.036859       	|  2              	|  0.073718       	|  0.0087744      	|
|  [Callback]GradientAccumulationScheduler.on_before_zero_grad                                                                                                                                                	|  3.2241e-06     	|  20000          	|  0.064482       	|  0.007675       	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_batch_start                     	|  3.0902e-06     	|  20000          	|  0.061803       	|  0.0073562      	|
|  [Callback]GradientAccumulationScheduler.on_after_backward                                                                                                                                                  	|  3.0789e-06     	|  20000          	|  0.061578       	|  0.0073293      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_batch_end                       	|  3.0687e-06     	|  20000          	|  0.061373       	|  0.007305       	|
|  [Callback]GradientAccumulationScheduler.on_train_batch_start                                                                                                                                               	|  2.9261e-06     	|  20000          	|  0.058522       	|  0.0069656      	|
|  [Callback]GradientAccumulationScheduler.on_batch_end                                                                                                                                                       	|  2.7384e-06     	|  20000          	|  0.054768       	|  0.0065188      	|
|  [Callback]GradientAccumulationScheduler.on_before_backward                                                                                                                                                 	|  2.6814e-06     	|  20000          	|  0.053629       	|  0.0063832      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_after_backward                  	|  2.6726e-06     	|  20000          	|  0.053451       	|  0.0063621      	|
|  [LightningModule]NeRFSystem.on_after_batch_transfer                                                                                                                                                        	|  2.6542e-06     	|  20092          	|  0.053329       	|  0.0063475      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_zero_grad                	|  2.5923e-06     	|  20000          	|  0.051846       	|  0.006171       	|
|  [Callback]GradientAccumulationScheduler.on_before_optimizer_step                                                                                                                                           	|  2.5415e-06     	|  20000          	|  0.050829       	|  0.00605        	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_optimizer_step           	|  2.4716e-06     	|  20000          	|  0.049433       	|  0.0058838      	|
|  [Callback]TQDMProgressBar.on_validation_batch_end                                                                                                                                                          	|  0.00050085     	|  92             	|  0.046078       	|  0.0054845      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_backward                 	|  2.2445e-06     	|  20000          	|  0.044891       	|  0.0053432      	|
|  [LightningModule]NeRFSystem.on_after_backward                                                                                                                                                              	|  1.5876e-06     	|  20000          	|  0.031752       	|  0.0037793      	|
|  [LightningModule]NeRFSystem.on_train_batch_end                                                                                                                                                             	|  1.5385e-06     	|  20000          	|  0.030771       	|  0.0036625      	|
|  [LightningModule]NeRFSystem.on_before_optimizer_step                                                                                                                                                       	|  1.4953e-06     	|  20000          	|  0.029906       	|  0.0035596      	|
|  [Strategy]SingleDeviceStrategy.training_step_end                                                                                                                                                           	|  1.4878e-06     	|  20000          	|  0.029756       	|  0.0035418      	|
|  [LightningModule]NeRFSystem.training_step_end                                                                                                                                                              	|  1.4532e-06     	|  20000          	|  0.029065       	|  0.0034594      	|
|  [LightningModule]NeRFSystem.configure_optimizers                                                                                                                                                           	|  0.029051       	|  1              	|  0.029051       	|  0.0034578      	|
|  [LightningModule]NeRFSystem.on_before_zero_grad                                                                                                                                                            	|  1.3605e-06     	|  20000          	|  0.02721        	|  0.0032387      	|
|  [LightningModule]NeRFSystem.on_train_batch_start                                                                                                                                                           	|  1.2735e-06     	|  20000          	|  0.02547        	|  0.0030316      	|
|  [LightningModule]NeRFSystem.on_before_backward                                                                                                                                                             	|  1.2657e-06     	|  20000          	|  0.025314       	|  0.0030131      	|
|  [Strategy]SingleDeviceStrategy.on_train_batch_start                                                                                                                                                        	|  1.2547e-06     	|  20000          	|  0.025094       	|  0.0029868      	|
|  [Callback]TQDMProgressBar.on_train_epoch_end                                                                                                                                                               	|  0.00064325     	|  20             	|  0.012865       	|  0.0015313      	|
|  [LightningModule]NeRFSystem.validation_epoch_end                                                                                                                                                           	|  0.0041329      	|  2              	|  0.0082658      	|  0.00098384     	|
|  [Callback]TQDMProgressBar.on_train_epoch_start                                                                                                                                                             	|  0.00032922     	|  20             	|  0.0065844      	|  0.00078371     	|
|  [Callback]TQDMProgressBar.on_validation_batch_start                                                                                                                                                        	|  3.0555e-05     	|  92             	|  0.0028111      	|  0.00033459     	|
|  [LightningModule]NeRFSystem.lr_scheduler_step                                                                                                                                                              	|  0.00011352     	|  20             	|  0.0022703      	|  0.00027023     	|
|  [Callback]TQDMProgressBar.on_validation_end                                                                                                                                                                	|  0.0010792      	|  2              	|  0.0021583      	|  0.0002569      	|
|  [LightningModule]NeRFSystem.on_validation_model_train                                                                                                                                                      	|  0.00062435     	|  2              	|  0.0012487      	|  0.00014863     	|
|  [LightningModule]NeRFSystem.on_validation_model_eval                                                                                                                                                       	|  0.00058709     	|  2              	|  0.0011742      	|  0.00013976     	|
|  [Callback]GradientAccumulationScheduler.on_validation_batch_end                                                                                                                                            	|  7.4328e-06     	|  92             	|  0.00068382     	|  8.1392e-05     	|
|  [LightningModule]NeRFSystem.train_dataloader                                                                                                                                                               	|  0.00068192     	|  1              	|  0.00068192     	|  8.1166e-05     	|
|  [Callback]TQDMProgressBar.on_validation_start                                                                                                                                                              	|  0.00032955     	|  2              	|  0.00065911     	|  7.845e-05      	|
|  [Callback]TQDMProgressBar.on_train_end                                                                                                                                                                     	|  0.00049401     	|  1              	|  0.00049401     	|  5.8799e-05     	|
|  [Callback]GradientAccumulationScheduler.on_validation_batch_start                                                                                                                                          	|  4.1596e-06     	|  92             	|  0.00038268     	|  4.5549e-05     	|
|  [LightningModule]NeRFSystem.on_validation_batch_end                                                                                                                                                        	|  4.1377e-06     	|  92             	|  0.00038067     	|  4.5309e-05     	|
|  [Callback]TQDMProgressBar.on_train_start                                                                                                                                                                   	|  0.00034055     	|  1              	|  0.00034055     	|  4.0534e-05     	|
|  [LightningModule]NeRFSystem.validation_step_end                                                                                                                                                            	|  3.4382e-06     	|  92             	|  0.00031632     	|  3.765e-05      	|
|  [LightningModule]NeRFSystem.val_dataloader                                                                                                                                                                 	|  0.00030887     	|  1              	|  0.00030887     	|  3.6763e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_batch_end            	|  3.2884e-06     	|  92             	|  0.00030253     	|  3.6009e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_batch_start          	|  2.9848e-06     	|  92             	|  0.0002746      	|  3.2685e-05     	|
|  [Callback]GradientAccumulationScheduler.on_train_epoch_start                                                                                                                                               	|  8.009e-06      	|  20             	|  0.00016018     	|  1.9066e-05     	|
|  [LightningModule]NeRFSystem.on_validation_batch_start                                                                                                                                                      	|  1.7063e-06     	|  92             	|  0.00015698     	|  1.8685e-05     	|
|  [Strategy]SingleDeviceStrategy.validation_step_end                                                                                                                                                         	|  1.33e-06       	|  92             	|  0.00012236     	|  1.4564e-05     	|
|  [Callback]GradientAccumulationScheduler.on_train_epoch_end                                                                                                                                                 	|  5.763e-06      	|  20             	|  0.00011526     	|  1.3719e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_end                  	|  5.7329e-05     	|  2              	|  0.00011466     	|  1.3647e-05     	|
|  [Callback]TQDMProgressBar.on_epoch_start                                                                                                                                                                   	|  3.9522e-06     	|  22             	|  8.6949e-05     	|  1.0349e-05     	|
|  [Callback]TQDMProgressBar.on_epoch_end                                                                                                                                                                     	|  3.6347e-06     	|  22             	|  7.9963e-05     	|  9.5177e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_epoch_start               	|  3.379e-06      	|  20             	|  6.758e-05      	|  8.0438e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.setup                              	|  5.9349e-05     	|  1              	|  5.9349e-05     	|  7.0641e-06     	|
|  [Callback]GradientAccumulationScheduler.on_epoch_end                                                                                                                                                       	|  2.5959e-06     	|  22             	|  5.7109e-05     	|  6.7974e-06     	|
|  [Callback]GradientAccumulationScheduler.on_epoch_start                                                                                                                                                     	|  2.5368e-06     	|  22             	|  5.581e-05      	|  6.6429e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_epoch_end                       	|  2.4095e-06     	|  22             	|  5.3009e-05     	|  6.3094e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_epoch_start                     	|  2.407e-06      	|  22             	|  5.2953e-05     	|  6.3028e-06     	|
|  [LightningModule]NeRFSystem.on_train_epoch_end                                                                                                                                                             	|  1.7869e-06     	|  20             	|  3.5738e-05     	|  4.2537e-06     	|
|  [LightningModule]NeRFSystem.on_epoch_end                                                                                                                                                                   	|  1.4941e-06     	|  22             	|  3.287e-05      	|  3.9124e-06     	|
|  [LightningModule]NeRFSystem.on_epoch_start                                                                                                                                                                 	|  1.46e-06       	|  22             	|  3.212e-05      	|  3.8232e-06     	|
|  [LightningModule]NeRFSystem.on_train_epoch_start                                                                                                                                                           	|  1.5218e-06     	|  20             	|  3.0437e-05     	|  3.6227e-06     	|
|  [Callback]GradientAccumulationScheduler.on_validation_end                                                                                                                                                  	|  1.407e-05      	|  2              	|  2.814e-05      	|  3.3494e-06     	|
|  [Strategy]SingleDeviceStrategy.on_validation_end                                                                                                                                                           	|  6.9854e-06     	|  2              	|  1.3971e-05     	|  1.6629e-06     	|
|  [Callback]GradientAccumulationScheduler.on_validation_start                                                                                                                                                	|  6.0145e-06     	|  2              	|  1.2029e-05     	|  1.4318e-06     	|
|  [LightningModule]NeRFSystem.configure_callbacks                                                                                                                                                            	|  1.1221e-05     	|  1              	|  1.1221e-05     	|  1.3355e-06     	|
|  [Callback]TQDMProgressBar.on_fit_end                                                                                                                                                                       	|  9.8897e-06     	|  1              	|  9.8897e-06     	|  1.1771e-06     	|
|  [Callback]GradientAccumulationScheduler.on_train_end                                                                                                                                                       	|  9.2294e-06     	|  1              	|  9.2294e-06     	|  1.0985e-06     	|
|  [Callback]TQDMProgressBar.setup                                                                                                                                                                            	|  9.0506e-06     	|  1              	|  9.0506e-06     	|  1.0773e-06     	|
|  [LightningModule]NeRFSystem.on_validation_epoch_end                                                                                                                                                        	|  4.3749e-06     	|  2              	|  8.7498e-06     	|  1.0414e-06     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_start                                                                                                                                                        	|  4.2049e-06     	|  2              	|  8.4098e-06     	|  1.001e-06      	|
|  [Callback]TQDMProgressBar.on_validation_epoch_end                                                                                                                                                          	|  3.8054e-06     	|  2              	|  7.6108e-06     	|  9.0588e-07     	|
|  [Callback]TQDMProgressBar.on_pretrain_routine_start                                                                                                                                                        	|  6.7297e-06     	|  1              	|  6.7297e-06     	|  8.0101e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_epoch_start                                                                                                                                          	|  3.1204e-06     	|  2              	|  6.2408e-06     	|  7.4281e-07     	|
|  [Callback]TQDMProgressBar.teardown                                                                                                                                                                         	|  6.0499e-06     	|  1              	|  6.0499e-06     	|  7.2009e-07     	|
|  [Strategy]SingleDeviceStrategy.on_validation_start                                                                                                                                                         	|  2.9849e-06     	|  2              	|  5.9698e-06     	|  7.1056e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_epoch_start          	|  2.9495e-06     	|  2              	|  5.899e-06      	|  7.0213e-07     	|
|  [Callback]TQDMProgressBar.on_fit_start                                                                                                                                                                     	|  5.8599e-06     	|  1              	|  5.8599e-06     	|  6.9748e-07     	|
|  [LightningModule]NeRFSystem.on_fit_start                                                                                                                                                                   	|  5.8301e-06     	|  1              	|  5.8301e-06     	|  6.9393e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_start                	|  2.9048e-06     	|  2              	|  5.8096e-06     	|  6.9149e-07     	|
|  [Callback]GradientAccumulationScheduler.on_fit_end                                                                                                                                                         	|  5.2899e-06     	|  1              	|  5.2899e-06     	|  6.2964e-07     	|
|  [LightningModule]NeRFSystem.configure_sharded_model                                                                                                                                                        	|  4.9006e-06     	|  1              	|  4.9006e-06     	|  5.833e-07      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_end                       	|  4.8596e-06     	|  1              	|  4.8596e-06     	|  5.7842e-07     	|
|  [Callback]GradientAccumulationScheduler.teardown                                                                                                                                                           	|  4.76e-06       	|  1              	|  4.76e-06       	|  5.6656e-07     	|
|  [Callback]GradientAccumulationScheduler.on_train_start                                                                                                                                                     	|  4.7293e-06     	|  1              	|  4.7293e-06     	|  5.629e-07      	|
|  [LightningModule]NeRFSystem.on_validation_end                                                                                                                                                              	|  2.3451e-06     	|  2              	|  4.6901e-06     	|  5.5825e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_fit_end                         	|  4.5998e-06     	|  1              	|  4.5998e-06     	|  5.4749e-07     	|
|  [Callback]TQDMProgressBar.on_configure_sharded_model                                                                                                                                                       	|  4.5598e-06     	|  1              	|  4.5598e-06     	|  5.4273e-07     	|
|  [Callback]TQDMProgressBar.on_pretrain_routine_end                                                                                                                                                          	|  4.5197e-06     	|  1              	|  4.5197e-06     	|  5.3796e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_start                     	|  4.4899e-06     	|  1              	|  4.4899e-06     	|  5.3441e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_epoch_end                                                                                                                                            	|  2.2347e-06     	|  2              	|  4.4694e-06     	|  5.3198e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.teardown                           	|  4.3698e-06     	|  1              	|  4.3698e-06     	|  5.2011e-07     	|
|  [LightningModule]NeRFSystem.prepare_data                                                                                                                                                                   	|  4.3595e-06     	|  1              	|  4.3595e-06     	|  5.189e-07      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_epoch_end            	|  2.0955e-06     	|  2              	|  4.191e-06      	|  4.9883e-07     	|
|  [LightningModule]NeRFSystem.on_pretrain_routine_start                                                                                                                                                      	|  4.1192e-06     	|  1              	|  4.1192e-06     	|  4.903e-07      	|
|  [Callback]GradientAccumulationScheduler.setup                                                                                                                                                              	|  3.6201e-06     	|  1              	|  3.6201e-06     	|  4.3088e-07     	|
|  [Callback]TQDMProgressBar.on_before_accelerator_backend_setup                                                                                                                                              	|  3.5604e-06     	|  1              	|  3.5604e-06     	|  4.2378e-07     	|
|  [Callback]GradientAccumulationScheduler.on_pretrain_routine_end                                                                                                                                            	|  3.35e-06       	|  1              	|  3.35e-06       	|  3.9873e-07     	|
|  [LightningModule]NeRFSystem.on_save_checkpoint                                                                                                                                                             	|  3.3304e-06     	|  1              	|  3.3304e-06     	|  3.964e-07      	|
|  [Callback]GradientAccumulationScheduler.on_configure_sharded_model                                                                                                                                         	|  3.2494e-06     	|  1              	|  3.2494e-06     	|  3.8676e-07     	|
|  [Callback]GradientAccumulationScheduler.on_fit_start                                                                                                                                                       	|  3.2e-06        	|  1              	|  3.2e-06        	|  3.8089e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_pretrain_routine_end            	|  3.1497e-06     	|  1              	|  3.1497e-06     	|  3.749e-07      	|
|  [LightningModule]NeRFSystem.on_validation_epoch_start                                                                                                                                                      	|  1.5446e-06     	|  2              	|  3.0892e-06     	|  3.6769e-07     	|
|  [LightningModule]NeRFSystem.on_train_end                                                                                                                                                                   	|  2.9802e-06     	|  1              	|  2.9802e-06     	|  3.5472e-07     	|
|  [LightningModule]NeRFSystem.teardown                                                                                                                                                                       	|  2.9802e-06     	|  1              	|  2.9802e-06     	|  3.5472e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_configure_sharded_model         	|  2.9607e-06     	|  1              	|  2.9607e-06     	|  3.524e-07      	|
|  [Callback]GradientAccumulationScheduler.on_before_accelerator_backend_setup                                                                                                                                	|  2.9206e-06     	|  1              	|  2.9206e-06     	|  3.4763e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_fit_start                       	|  2.8405e-06     	|  1              	|  2.8405e-06     	|  3.381e-07      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_accelerator_backend_setup	|  2.7502e-06     	|  1              	|  2.7502e-06     	|  3.2734e-07     	|
|  [LightningModule]NeRFSystem.on_fit_end                                                                                                                                                                     	|  2.5099e-06     	|  1              	|  2.5099e-06     	|  2.9874e-07     	|
|  [Callback]GradientAccumulationScheduler.on_pretrain_routine_start                                                                                                                                          	|  2.4e-06        	|  1              	|  2.4e-06        	|  2.8566e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_pretrain_routine_start          	|  2.3702e-06     	|  1              	|  2.3702e-06     	|  2.8212e-07     	|
|  [Strategy]SingleDeviceStrategy.on_train_end                                                                                                                                                                	|  2.0498e-06     	|  1              	|  2.0498e-06     	|  2.4398e-07     	|
|  [LightningModule]NeRFSystem.on_pretrain_routine_end                                                                                                                                                        	|  1.6494e-06     	|  1              	|  1.6494e-06     	|  1.9632e-07     	|
|  [LightningModule]NeRFSystem.on_train_start                                                                                                                                                                 	|  1.18e-06       	|  1              	|  1.18e-06       	|  1.4045e-07     	|
|  [Strategy]SingleDeviceStrategy.on_train_start                                                                                                                                                              	|  9.6019e-07     	|  1              	|  9.6019e-07     	|  1.1429e-07     	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
