FIT Profiler Report

--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Action                                                                                                                                                         	|  Mean duration (s)	|  Num calls      	|  Total time (s) 	|  Percentage %   	|
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Total                                                                                                                                                          	|  -              	|  30303          	|  159.79         	|  100 %          	|
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  run_training_epoch                                                                                                                                             	|  150.38         	|  1              	|  150.38         	|  94.112         	|
|  run_training_batch                                                                                                                                             	|  0.1005         	|  1000           	|  100.5          	|  62.894         	|
|  [LightningModule]NeRFSystem.optimizer_step                                                                                                                     	|  0.10034        	|  1000           	|  100.34         	|  62.796         	|
|  [Strategy]SingleDeviceStrategy.validation_step                                                                                                                 	|  1.3501         	|  21             	|  28.353         	|  17.744         	|
|  [Strategy]SingleDeviceStrategy.training_step                                                                                                                   	|  0.025908       	|  1000           	|  25.908         	|  16.214         	|
|  [_TrainingEpochLoop].train_dataloader_next                                                                                                                     	|  0.019446       	|  1000           	|  19.446         	|  12.17          	|
|  [Strategy]SingleDeviceStrategy.backward                                                                                                                        	|  0.0033934      	|  1000           	|  3.3934         	|  2.1236         	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 1, 'train_time_interval': None}.on_train_epoch_end       	|  3.175          	|  1              	|  3.175          	|  1.987          	|
|  [LightningModule]NeRFSystem.setup                                                                                                                              	|  2.6569         	|  1              	|  2.6569         	|  1.6628         	|
|  [Callback]TQDMProgressBar.on_train_batch_end                                                                                                                   	|  0.00055814     	|  1000           	|  0.55814        	|  0.3493         	|
|  [Strategy]SingleDeviceStrategy.batch_to_device                                                                                                                 	|  0.00024105     	|  1021           	|  0.24611        	|  0.15402        	|
|  [_EvaluationLoop].val_next                                                                                                                                     	|  0.008007       	|  21             	|  0.16815        	|  0.10523        	|
|  [LightningModule]NeRFSystem.transfer_batch_to_device                                                                                                           	|  0.00015591     	|  1021           	|  0.15918        	|  0.099621       	|
|  [LightningModule]NeRFSystem.on_validation_start                                                                                                                	|  0.050714       	|  1              	|  0.050714       	|  0.031738       	|
|  [LightningModule]NeRFSystem.optimizer_zero_grad                                                                                                                	|  2.4965e-05     	|  1000           	|  0.024965       	|  0.015624       	|
|  [LightningModule]NeRFSystem.configure_gradient_clipping                                                                                                        	|  2.2549e-05     	|  1000           	|  0.022549       	|  0.014111       	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 1, 'train_time_interval': None}.on_train_batch_end       	|  1.6972e-05     	|  1000           	|  0.016972       	|  0.010621       	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 1, 'train_time_interval': None}.on_train_batch_start     	|  1.4065e-05     	|  1000           	|  0.014065       	|  0.0088023      	|
|  [LightningModule]NeRFSystem.configure_optimizers                                                                                                               	|  0.0070561      	|  1              	|  0.0070561      	|  0.0044159      	|
|  [Callback]TQDMProgressBar.on_validation_batch_end                                                                                                              	|  0.00030561     	|  21             	|  0.0064179      	|  0.0040165      	|
|  [LightningModule]NeRFSystem.on_validation_epoch_end                                                                                                            	|  0.0059868      	|  1              	|  0.0059868      	|  0.0037467      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 1, 'train_time_interval': None}.on_after_backward        	|  4.1985e-06     	|  1000           	|  0.0041985      	|  0.0026275      	|
|  [Callback]TQDMProgressBar.on_after_backward                                                                                                                    	|  3.887e-06      	|  1000           	|  0.003887       	|  0.0024326      	|
|  [Callback]TQDMProgressBar.on_before_zero_grad                                                                                                                  	|  2.7759e-06     	|  1000           	|  0.0027759      	|  0.0017372      	|
|  [Callback]TQDMProgressBar.on_before_optimizer_step                                                                                                             	|  2.6637e-06     	|  1000           	|  0.0026637      	|  0.001667       	|
|  [LightningModule]NeRFSystem.on_before_batch_transfer                                                                                                           	|  2.4703e-06     	|  1021           	|  0.0025222      	|  0.0015784      	|
|  [Callback]TQDMProgressBar.on_train_batch_start                                                                                                                 	|  2.4881e-06     	|  1000           	|  0.0024881      	|  0.0015571      	|
|  [Callback]TQDMProgressBar.on_before_backward                                                                                                                   	|  2.2645e-06     	|  1000           	|  0.0022645      	|  0.0014172      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 1, 'train_time_interval': None}.on_before_zero_grad      	|  1.737e-06      	|  1000           	|  0.001737       	|  0.0010871      	|
|  [Strategy]SingleDeviceStrategy.on_train_batch_start                                                                                                            	|  1.5634e-06     	|  1000           	|  0.0015634      	|  0.00097841     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 1, 'train_time_interval': None}.on_before_optimizer_step 	|  1.5251e-06     	|  1000           	|  0.0015251      	|  0.00095442     	|
|  [LightningModule]NeRFSystem.on_after_backward                                                                                                                  	|  1.4665e-06     	|  1000           	|  0.0014665      	|  0.00091778     	|
|  [LightningModule]NeRFSystem.on_train_batch_end                                                                                                                 	|  1.4567e-06     	|  1000           	|  0.0014567      	|  0.00091166     	|
|  [LightningModule]NeRFSystem.on_before_optimizer_step                                                                                                           	|  1.4369e-06     	|  1000           	|  0.0014369      	|  0.00089924     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 1, 'train_time_interval': None}.on_before_backward       	|  1.4143e-06     	|  1000           	|  0.0014143      	|  0.00088509     	|
|  [LightningModule]NeRFSystem.on_before_zero_grad                                                                                                                	|  1.4135e-06     	|  1000           	|  0.0014135      	|  0.00088459     	|
|  [LightningModule]NeRFSystem.on_before_backward                                                                                                                 	|  1.3957e-06     	|  1000           	|  0.0013957      	|  0.00087346     	|
|  [LightningModule]NeRFSystem.on_after_batch_transfer                                                                                                            	|  1.3475e-06     	|  1021           	|  0.0013758      	|  0.00086102     	|
|  [LightningModule]NeRFSystem.on_train_batch_start                                                                                                               	|  1.3521e-06     	|  1000           	|  0.0013521      	|  0.00084618     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 1, 'train_time_interval': None}.setup                    	|  0.0011705      	|  1              	|  0.0011705      	|  0.0007325      	|
|  [Callback]TQDMProgressBar.on_validation_start                                                                                                                  	|  0.00093116     	|  1              	|  0.00093116     	|  0.00058274     	|
|  [LightningModule]NeRFSystem.on_validation_model_eval                                                                                                           	|  0.00091127     	|  1              	|  0.00091127     	|  0.00057029     	|
|  [Callback]TQDMProgressBar.on_validation_batch_start                                                                                                            	|  3.3039e-05     	|  21             	|  0.00069382     	|  0.00043421     	|
|  [Callback]TQDMProgressBar.on_train_end                                                                                                                         	|  0.00053906     	|  1              	|  0.00053906     	|  0.00033736     	|
|  [LightningModule]NeRFSystem.on_validation_model_zero_grad                                                                                                      	|  0.00053903     	|  1              	|  0.00053903     	|  0.00033734     	|
|  [LightningModule]NeRFSystem.on_validation_model_train                                                                                                          	|  0.00047803     	|  1              	|  0.00047803     	|  0.00029916     	|
|  [Callback]TQDMProgressBar.on_validation_end                                                                                                                    	|  0.00044791     	|  1              	|  0.00044791     	|  0.00028032     	|
|  [Callback]TQDMProgressBar.on_train_start                                                                                                                       	|  0.00036525     	|  1              	|  0.00036525     	|  0.00022858     	|
|  [LightningModule]NeRFSystem.train_dataloader                                                                                                                   	|  0.00032991     	|  1              	|  0.00032991     	|  0.00020647     	|
|  [Callback]TQDMProgressBar.on_train_epoch_start                                                                                                                 	|  0.00020707     	|  1              	|  0.00020707     	|  0.00012959     	|
|  [Callback]TQDMProgressBar.on_train_epoch_end                                                                                                                   	|  0.00019123     	|  1              	|  0.00019123     	|  0.00011967     	|
|  [LightningModule]NeRFSystem.val_dataloader                                                                                                                     	|  0.00017393     	|  1              	|  0.00017393     	|  0.00010885     	|
|  [LightningModule]NeRFSystem.lr_scheduler_step                                                                                                                  	|  8.2309e-05     	|  1              	|  8.2309e-05     	|  5.1511e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 1, 'train_time_interval': None}.on_validation_batch_end  	|  3.2805e-06     	|  21             	|  6.889e-05      	|  4.3113e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 1, 'train_time_interval': None}.on_validation_batch_start	|  2.5181e-06     	|  21             	|  5.288e-05      	|  3.3094e-05     	|
|  [LightningModule]NeRFSystem.on_validation_batch_end                                                                                                            	|  2.3208e-06     	|  21             	|  4.8738e-05     	|  3.0501e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 1, 'train_time_interval': None}.on_validation_end        	|  3.84e-05       	|  1              	|  3.84e-05       	|  2.4031e-05     	|
|  [LightningModule]NeRFSystem.on_validation_batch_start                                                                                                          	|  1.5424e-06     	|  21             	|  3.239e-05      	|  2.027e-05      	|
|  [LightningModule]NeRFSystem.configure_callbacks                                                                                                                	|  2.7419e-05     	|  1              	|  2.7419e-05     	|  1.7159e-05     	|
|  [Callback]TQDMProgressBar.setup                                                                                                                                	|  2.364e-05      	|  1              	|  2.364e-05      	|  1.4795e-05     	|
|  [LightningModule]NeRFSystem.on_train_epoch_start                                                                                                               	|  7.2001e-06     	|  1              	|  7.2001e-06     	|  4.506e-06      	|
|  [LightningModule]NeRFSystem.prepare_data                                                                                                                       	|  6.9202e-06     	|  1              	|  6.9202e-06     	|  4.3308e-06     	|
|  [LightningModule]NeRFSystem.on_validation_end                                                                                                                  	|  6.76e-06       	|  1              	|  6.76e-06       	|  4.2306e-06     	|
|  [Strategy]SingleDeviceStrategy.on_validation_start                                                                                                             	|  5.9898e-06     	|  1              	|  5.9898e-06     	|  3.7486e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 1, 'train_time_interval': None}.on_validation_start      	|  5.9302e-06     	|  1              	|  5.9302e-06     	|  3.7113e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 1, 'train_time_interval': None}.on_train_end             	|  5.8501e-06     	|  1              	|  5.8501e-06     	|  3.6611e-06     	|
|  [LightningModule]NeRFSystem.on_fit_start                                                                                                                       	|  5.6499e-06     	|  1              	|  5.6499e-06     	|  3.5358e-06     	|
|  [Callback]TQDMProgressBar.on_fit_end                                                                                                                           	|  5.39e-06       	|  1              	|  5.39e-06       	|  3.3732e-06     	|
|  [Callback]TQDMProgressBar.teardown                                                                                                                             	|  3.6797e-06     	|  1              	|  3.6797e-06     	|  2.3028e-06     	|
|  [LightningModule]NeRFSystem.on_train_end                                                                                                                       	|  3.54e-06       	|  1              	|  3.54e-06       	|  2.2154e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 1, 'train_time_interval': None}.on_train_start           	|  3.3197e-06     	|  1              	|  3.3197e-06     	|  2.0775e-06     	|
|  [Callback]TQDMProgressBar.on_fit_start                                                                                                                         	|  3.2601e-06     	|  1              	|  3.2601e-06     	|  2.0402e-06     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_start                                                                                                            	|  3.0799e-06     	|  1              	|  3.0799e-06     	|  1.9275e-06     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_end                                                                                                              	|  2.6398e-06     	|  1              	|  2.6398e-06     	|  1.6521e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 1, 'train_time_interval': None}.on_train_epoch_start     	|  2.42e-06       	|  1              	|  2.42e-06       	|  1.5145e-06     	|
|  [Strategy]SingleDeviceStrategy.on_train_end                                                                                                                    	|  2.19e-06       	|  1              	|  2.19e-06       	|  1.3706e-06     	|
|  [Strategy]SingleDeviceStrategy.on_validation_end                                                                                                               	|  2.1202e-06     	|  1              	|  2.1202e-06     	|  1.3268e-06     	|
|  [LightningModule]NeRFSystem.on_save_checkpoint                                                                                                                 	|  2.0601e-06     	|  1              	|  2.0601e-06     	|  1.2892e-06     	|
|  [LightningModule]NeRFSystem.on_train_epoch_end                                                                                                                 	|  2.0196e-06     	|  1              	|  2.0196e-06     	|  1.2639e-06     	|
|  [LightningModule]NeRFSystem.on_validation_epoch_start                                                                                                          	|  1.9399e-06     	|  1              	|  1.9399e-06     	|  1.2141e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 1, 'train_time_interval': None}.on_validation_epoch_start	|  1.7299e-06     	|  1              	|  1.7299e-06     	|  1.0826e-06     	|
|  [LightningModule]NeRFSystem.on_fit_end                                                                                                                         	|  1.62e-06       	|  1              	|  1.62e-06       	|  1.0139e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 1, 'train_time_interval': None}.on_validation_epoch_end  	|  1.58e-06       	|  1              	|  1.58e-06       	|  9.8879e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 1, 'train_time_interval': None}.teardown                 	|  1.5702e-06     	|  1              	|  1.5702e-06     	|  9.8267e-07     	|
|  [LightningModule]NeRFSystem.teardown                                                                                                                           	|  1.5302e-06     	|  1              	|  1.5302e-06     	|  9.5761e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 1, 'train_time_interval': None}.on_fit_end               	|  1.4897e-06     	|  1              	|  1.4897e-06     	|  9.3226e-07     	|
|  [LightningModule]NeRFSystem.on_train_start                                                                                                                     	|  1.4701e-06     	|  1              	|  1.4701e-06     	|  9.2002e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 1, 'train_time_interval': None}.on_fit_start             	|  1.4598e-06     	|  1              	|  1.4598e-06     	|  9.1361e-07     	|
|  [Strategy]SingleDeviceStrategy.on_train_start                                                                                                                  	|  1.2899e-06     	|  1              	|  1.2899e-06     	|  8.0724e-07     	|
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
FIT Profiler Report

--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Action                                                                                                                                                          	|  Mean duration (s)	|  Num calls      	|  Total time (s) 	|  Percentage %   	|
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Total                                                                                                                                                           	|  -              	|  600724         	|  1791.8         	|  100 %          	|
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  run_training_epoch                                                                                                                                              	|  89.142         	|  20             	|  1782.8         	|  99.5           	|
|  run_training_batch                                                                                                                                              	|  0.064339       	|  20000          	|  1286.8         	|  71.815         	|
|  [LightningModule]NeRFSystem.optimizer_step                                                                                                                      	|  0.064164       	|  20000          	|  1283.3         	|  71.619         	|
|  [_TrainingEpochLoop].train_dataloader_next                                                                                                                      	|  0.020802       	|  20000          	|  416.04         	|  23.219         	|
|  [Strategy]SingleDeviceStrategy.training_step                                                                                                                    	|  0.01768        	|  20000          	|  353.6          	|  19.734         	|
|  [Strategy]SingleDeviceStrategy.backward                                                                                                                         	|  0.0032613      	|  20000          	|  65.226         	|  3.6402         	|
|  [Strategy]SingleDeviceStrategy.validation_step                                                                                                                  	|  0.97664        	|  42             	|  41.019         	|  2.2893         	|
|  [Callback]TQDMProgressBar.on_train_batch_end                                                                                                                    	|  0.0006755      	|  20000          	|  13.51          	|  0.75399        	|
|  [Strategy]SingleDeviceStrategy.batch_to_device                                                                                                                  	|  0.00026804     	|  20042          	|  5.3722         	|  0.29982        	|
|  [LightningModule]NeRFSystem.transfer_batch_to_device                                                                                                            	|  0.00018387     	|  20042          	|  3.6852         	|  0.20567        	|
|  [LightningModule]NeRFSystem.setup                                                                                                                               	|  2.9655         	|  1              	|  2.9655         	|  0.1655         	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None}.on_train_epoch_end       	|  0.12691        	|  20             	|  2.5382         	|  0.14166        	|
|  [LightningModule]NeRFSystem.optimizer_zero_grad                                                                                                                 	|  3.3129e-05     	|  20000          	|  0.66257        	|  0.036978       	|
|  [LightningModule]NeRFSystem.configure_gradient_clipping                                                                                                         	|  2.7664e-05     	|  20000          	|  0.55328        	|  0.030878       	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None}.on_train_batch_end       	|  2.1907e-05     	|  20000          	|  0.43813        	|  0.024452       	|
|  [_EvaluationLoop].val_next                                                                                                                                      	|  0.0052976      	|  42             	|  0.2225         	|  0.012418       	|
|  [LightningModule]NeRFSystem.on_before_batch_transfer                                                                                                            	|  4.2574e-06     	|  20042          	|  0.085327       	|  0.0047621      	|
|  [Callback]TQDMProgressBar.on_after_backward                                                                                                                     	|  3.8589e-06     	|  20000          	|  0.077178       	|  0.0043073      	|
|  [LightningModule]NeRFSystem.on_validation_start                                                                                                                 	|  0.034588       	|  2              	|  0.069176       	|  0.0038607      	|
|  [Callback]TQDMProgressBar.on_before_zero_grad                                                                                                                   	|  3.1041e-06     	|  20000          	|  0.062083       	|  0.0034648      	|
|  [Callback]TQDMProgressBar.on_before_backward                                                                                                                    	|  3.0066e-06     	|  20000          	|  0.060131       	|  0.0033559      	|
|  [Callback]TQDMProgressBar.on_before_optimizer_step                                                                                                              	|  2.9384e-06     	|  20000          	|  0.058768       	|  0.0032798      	|
|  [Callback]TQDMProgressBar.on_train_batch_start                                                                                                                  	|  2.9065e-06     	|  20000          	|  0.05813        	|  0.0032442      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None}.on_before_optimizer_step 	|  2.5037e-06     	|  20000          	|  0.050075       	|  0.0027946      	|
|  [LightningModule]NeRFSystem.on_after_batch_transfer                                                                                                             	|  2.1612e-06     	|  20042          	|  0.043316       	|  0.0024174      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None}.on_before_zero_grad      	|  1.9966e-06     	|  20000          	|  0.039932       	|  0.0022286      	|
|  [LightningModule]NeRFSystem.on_train_batch_start                                                                                                                	|  1.9717e-06     	|  20000          	|  0.039435       	|  0.0022009      	|
|  [Strategy]SingleDeviceStrategy.on_train_batch_start                                                                                                             	|  1.8947e-06     	|  20000          	|  0.037894       	|  0.0021149      	|
|  [LightningModule]NeRFSystem.on_train_batch_end                                                                                                                  	|  1.8443e-06     	|  20000          	|  0.036885       	|  0.0020585      	|
|  [LightningModule]NeRFSystem.on_before_backward                                                                                                                  	|  1.8216e-06     	|  20000          	|  0.036433       	|  0.0020333      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None}.on_after_backward        	|  1.7439e-06     	|  20000          	|  0.034879       	|  0.0019466      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None}.on_train_batch_start     	|  1.6816e-06     	|  20000          	|  0.033632       	|  0.001877       	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None}.on_before_backward       	|  1.5154e-06     	|  20000          	|  0.030307       	|  0.0016914      	|
|  [LightningModule]NeRFSystem.on_before_zero_grad                                                                                                                 	|  1.5106e-06     	|  20000          	|  0.030212       	|  0.0016861      	|
|  [LightningModule]NeRFSystem.on_before_optimizer_step                                                                                                            	|  1.4985e-06     	|  20000          	|  0.029969       	|  0.0016726      	|
|  [LightningModule]NeRFSystem.on_after_backward                                                                                                                   	|  1.3328e-06     	|  20000          	|  0.026657       	|  0.0014877      	|
|  [LightningModule]NeRFSystem.on_validation_epoch_end                                                                                                             	|  0.012316       	|  2              	|  0.024632       	|  0.0013747      	|
|  [Callback]TQDMProgressBar.on_validation_batch_end                                                                                                               	|  0.00049331     	|  42             	|  0.020719       	|  0.0011563      	|
|  [Callback]TQDMProgressBar.on_validation_start                                                                                                                   	|  0.0031039      	|  2              	|  0.0062078      	|  0.00034645     	|
|  [Callback]TQDMProgressBar.on_train_epoch_start                                                                                                                  	|  0.0002367      	|  20             	|  0.0047339      	|  0.0002642      	|
|  [LightningModule]NeRFSystem.configure_optimizers                                                                                                                	|  0.0039687      	|  1              	|  0.0039687      	|  0.00022149     	|
|  [Callback]TQDMProgressBar.on_train_epoch_end                                                                                                                    	|  0.00019201     	|  20             	|  0.0038403      	|  0.00021433     	|
|  [LightningModule]NeRFSystem.on_validation_model_eval                                                                                                            	|  0.0011508      	|  2              	|  0.0023017      	|  0.00012846     	|
|  [LightningModule]NeRFSystem.lr_scheduler_step                                                                                                                   	|  7.5707e-05     	|  20             	|  0.0015141      	|  8.4504e-05     	|
|  [LightningModule]NeRFSystem.on_validation_model_zero_grad                                                                                                       	|  0.00071186     	|  2              	|  0.0014237      	|  7.9457e-05     	|
|  [Callback]TQDMProgressBar.on_validation_batch_start                                                                                                             	|  3.0057e-05     	|  42             	|  0.0012624      	|  7.0453e-05     	|
|  [LightningModule]NeRFSystem.on_validation_model_train                                                                                                           	|  0.00046924     	|  2              	|  0.00093848     	|  5.2376e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None}.setup                    	|  0.00087145     	|  1              	|  0.00087145     	|  4.8635e-05     	|
|  [Callback]TQDMProgressBar.on_validation_end                                                                                                                     	|  0.00039908     	|  2              	|  0.00079817     	|  4.4546e-05     	|
|  [LightningModule]NeRFSystem.train_dataloader                                                                                                                    	|  0.00033688     	|  1              	|  0.00033688     	|  1.8801e-05     	|
|  [Callback]TQDMProgressBar.on_train_start                                                                                                                        	|  0.00028125     	|  1              	|  0.00028125     	|  1.5696e-05     	|
|  [LightningModule]NeRFSystem.val_dataloader                                                                                                                      	|  0.00027432     	|  1              	|  0.00027432     	|  1.5309e-05     	|
|  [Callback]TQDMProgressBar.on_train_end                                                                                                                          	|  0.00027401     	|  1              	|  0.00027401     	|  1.5292e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None}.on_validation_batch_end  	|  3.438e-06      	|  42             	|  0.00014439     	|  8.0586e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None}.on_validation_batch_start	|  2.5338e-06     	|  42             	|  0.00010642     	|  5.9392e-06     	|
|  [LightningModule]NeRFSystem.on_validation_batch_end                                                                                                             	|  2.1184e-06     	|  42             	|  8.8971e-05     	|  4.9654e-06     	|
|  [LightningModule]NeRFSystem.on_validation_batch_start                                                                                                           	|  1.5488e-06     	|  42             	|  6.5048e-05     	|  3.6303e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None}.on_train_epoch_start     	|  3.108e-06      	|  20             	|  6.216e-05      	|  3.4691e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None}.on_validation_end        	|  2.9769e-05     	|  2              	|  5.9538e-05     	|  3.3228e-06     	|
|  [LightningModule]NeRFSystem.on_train_epoch_end                                                                                                                  	|  1.7629e-06     	|  20             	|  3.5259e-05     	|  1.9678e-06     	|
|  [LightningModule]NeRFSystem.on_train_epoch_start                                                                                                                	|  1.748e-06      	|  20             	|  3.4959e-05     	|  1.9511e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None}.on_validation_start      	|  6.77e-06       	|  2              	|  1.354e-05      	|  7.5566e-07     	|
|  [Strategy]SingleDeviceStrategy.on_validation_start                                                                                                              	|  5.9854e-06     	|  2              	|  1.1971e-05     	|  6.6808e-07     	|
|  [LightningModule]NeRFSystem.configure_callbacks                                                                                                                 	|  1.112e-05      	|  1              	|  1.112e-05      	|  6.206e-07      	|
|  [LightningModule]NeRFSystem.on_validation_end                                                                                                                   	|  4.9251e-06     	|  2              	|  9.8501e-06     	|  5.4973e-07     	|
|  [Callback]TQDMProgressBar.setup                                                                                                                                 	|  8.9197e-06     	|  1              	|  8.9197e-06     	|  4.9781e-07     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_start                                                                                                             	|  3.065e-06      	|  2              	|  6.13e-06       	|  3.4211e-07     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_end                                                                                                               	|  2.7944e-06     	|  2              	|  5.5889e-06     	|  3.1191e-07     	|
|  [Callback]TQDMProgressBar.on_fit_end                                                                                                                            	|  5.4501e-06     	|  1              	|  5.4501e-06     	|  3.0417e-07     	|
|  [Callback]TQDMProgressBar.teardown                                                                                                                              	|  4.19e-06       	|  1              	|  4.19e-06       	|  2.3384e-07     	|
|  [LightningModule]NeRFSystem.prepare_data                                                                                                                        	|  4e-06          	|  1              	|  4e-06          	|  2.2324e-07     	|
|  [LightningModule]NeRFSystem.on_validation_epoch_start                                                                                                           	|  2e-06          	|  2              	|  4e-06          	|  2.2324e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None}.on_validation_epoch_start	|  1.7502e-06     	|  2              	|  3.5004e-06     	|  1.9535e-07     	|
|  [Strategy]SingleDeviceStrategy.on_validation_end                                                                                                                	|  1.7453e-06     	|  2              	|  3.4906e-06     	|  1.9481e-07     	|
|  [LightningModule]NeRFSystem.on_save_checkpoint                                                                                                                  	|  3.3197e-06     	|  1              	|  3.3197e-06     	|  1.8527e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None}.on_train_start           	|  3.16e-06       	|  1              	|  3.16e-06       	|  1.7636e-07     	|
|  [Callback]TQDMProgressBar.on_fit_start                                                                                                                          	|  2.9602e-06     	|  1              	|  2.9602e-06     	|  1.6521e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None}.on_train_end             	|  2.9397e-06     	|  1              	|  2.9397e-06     	|  1.6406e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None}.on_validation_epoch_end  	|  1.4252e-06     	|  2              	|  2.8503e-06     	|  1.5907e-07     	|
|  [LightningModule]NeRFSystem.on_fit_end                                                                                                                          	|  2.7101e-06     	|  1              	|  2.7101e-06     	|  1.5125e-07     	|
|  [LightningModule]NeRFSystem.on_train_end                                                                                                                        	|  2.3399e-06     	|  1              	|  2.3399e-06     	|  1.3059e-07     	|
|  [LightningModule]NeRFSystem.on_fit_start                                                                                                                        	|  2.1202e-06     	|  1              	|  2.1202e-06     	|  1.1833e-07     	|
|  [LightningModule]NeRFSystem.teardown                                                                                                                            	|  1.79e-06       	|  1              	|  1.79e-06       	|  9.9899e-08     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None}.teardown                 	|  1.6303e-06     	|  1              	|  1.6303e-06     	|  9.0985e-08     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None}.on_fit_start             	|  1.5199e-06     	|  1              	|  1.5199e-06     	|  8.4826e-08     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None}.on_fit_end               	|  1.4799e-06     	|  1              	|  1.4799e-06     	|  8.2591e-08     	|
|  [LightningModule]NeRFSystem.on_train_start                                                                                                                      	|  1.4598e-06     	|  1              	|  1.4598e-06     	|  8.1474e-08     	|
|  [Strategy]SingleDeviceStrategy.on_train_end                                                                                                                     	|  1.4501e-06     	|  1              	|  1.4501e-06     	|  8.0928e-08     	|
|  [Strategy]SingleDeviceStrategy.on_train_start                                                                                                                   	|  1.2298e-06     	|  1              	|  1.2298e-06     	|  6.8635e-08     	|
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
