FIT Profiler Report

--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Action                                                                                                                                                         	|  Mean duration (s)	|  Num calls      	|  Total time (s) 	|  Percentage %   	|
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Total                                                                                                                                                          	|  -              	|  30303          	|  162.62         	|  100 %          	|
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  run_training_epoch                                                                                                                                             	|  152.86         	|  1              	|  152.86         	|  93.997         	|
|  run_training_batch                                                                                                                                             	|  0.10374        	|  1000           	|  103.74         	|  63.793         	|
|  [LightningModule]NeRFSystem.optimizer_step                                                                                                                     	|  0.10358        	|  1000           	|  103.58         	|  63.693         	|
|  [Strategy]SingleDeviceStrategy.validation_step                                                                                                                 	|  1.3544         	|  21             	|  28.442         	|  17.49          	|
|  [Strategy]SingleDeviceStrategy.training_step                                                                                                                   	|  0.026285       	|  1000           	|  26.285         	|  16.164         	|
|  [_TrainingEpochLoop].train_dataloader_next                                                                                                                     	|  0.018655       	|  1000           	|  18.655         	|  11.471         	|
|  [LightningModule]NeRFSystem.setup                                                                                                                              	|  3.849          	|  1              	|  3.849          	|  2.3669         	|
|  [Strategy]SingleDeviceStrategy.backward                                                                                                                        	|  0.0034512      	|  1000           	|  3.4512         	|  2.1223         	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 1, 'train_time_interval': None}.on_train_epoch_end       	|  2.6658         	|  1              	|  2.6658         	|  1.6393         	|
|  [Callback]TQDMProgressBar.on_train_batch_end                                                                                                                   	|  0.00054894     	|  1000           	|  0.54894        	|  0.33756        	|
|  [Strategy]SingleDeviceStrategy.batch_to_device                                                                                                                 	|  0.00018962     	|  1021           	|  0.1936         	|  0.11905        	|
|  [_EvaluationLoop].val_next                                                                                                                                     	|  0.0073936      	|  21             	|  0.15527        	|  0.095479       	|
|  [LightningModule]NeRFSystem.transfer_batch_to_device                                                                                                           	|  0.0001345      	|  1021           	|  0.13732        	|  0.084444       	|
|  [LightningModule]NeRFSystem.on_validation_start                                                                                                                	|  0.06455        	|  1              	|  0.06455        	|  0.039694       	|
|  [LightningModule]NeRFSystem.optimizer_zero_grad                                                                                                                	|  2.3618e-05     	|  1000           	|  0.023618       	|  0.014524       	|
|  [LightningModule]NeRFSystem.configure_gradient_clipping                                                                                                        	|  2.3446e-05     	|  1000           	|  0.023446       	|  0.014418       	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 1, 'train_time_interval': None}.on_train_batch_end       	|  2.2641e-05     	|  1000           	|  0.022641       	|  0.013923       	|
|  [Callback]TQDMProgressBar.on_validation_batch_end                                                                                                              	|  0.00034188     	|  21             	|  0.0071795      	|  0.004415       	|
|  [LightningModule]NeRFSystem.configure_optimizers                                                                                                               	|  0.0058956      	|  1              	|  0.0058956      	|  0.0036254      	|
|  [LightningModule]NeRFSystem.on_validation_epoch_end                                                                                                            	|  0.0055492      	|  1              	|  0.0055492      	|  0.0034124      	|
|  [Callback]TQDMProgressBar.on_after_backward                                                                                                                    	|  3.7079e-06     	|  1000           	|  0.0037079      	|  0.0022801      	|
|  [Callback]TQDMProgressBar.on_before_zero_grad                                                                                                                  	|  2.9058e-06     	|  1000           	|  0.0029058      	|  0.0017869      	|
|  [Callback]TQDMProgressBar.on_before_optimizer_step                                                                                                             	|  2.6851e-06     	|  1000           	|  0.0026851      	|  0.0016512      	|
|  [LightningModule]NeRFSystem.on_before_batch_transfer                                                                                                           	|  2.3852e-06     	|  1021           	|  0.0024353      	|  0.0014975      	|
|  [Callback]TQDMProgressBar.on_before_backward                                                                                                                   	|  2.2507e-06     	|  1000           	|  0.0022507      	|  0.0013841      	|
|  [Callback]TQDMProgressBar.on_train_batch_start                                                                                                                 	|  2.2447e-06     	|  1000           	|  0.0022447      	|  0.0013804      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 1, 'train_time_interval': None}.on_before_zero_grad      	|  1.7763e-06     	|  1000           	|  0.0017763      	|  0.0010923      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 1, 'train_time_interval': None}.on_train_batch_start     	|  1.6232e-06     	|  1000           	|  0.0016232      	|  0.00099816     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 1, 'train_time_interval': None}.on_after_backward        	|  1.5002e-06     	|  1000           	|  0.0015002      	|  0.00092252     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 1, 'train_time_interval': None}.on_before_optimizer_step 	|  1.4815e-06     	|  1000           	|  0.0014815      	|  0.00091105     	|
|  [LightningModule]NeRFSystem.on_train_batch_end                                                                                                                 	|  1.4357e-06     	|  1000           	|  0.0014357      	|  0.00088285     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 1, 'train_time_interval': None}.on_before_backward       	|  1.4085e-06     	|  1000           	|  0.0014085      	|  0.00086613     	|
|  [LightningModule]NeRFSystem.on_after_batch_transfer                                                                                                            	|  1.3575e-06     	|  1021           	|  0.001386       	|  0.00085231     	|
|  [LightningModule]NeRFSystem.on_before_zero_grad                                                                                                                	|  1.3117e-06     	|  1000           	|  0.0013117      	|  0.00080661     	|
|  [LightningModule]NeRFSystem.on_train_batch_start                                                                                                               	|  1.2835e-06     	|  1000           	|  0.0012835      	|  0.00078926     	|
|  [LightningModule]NeRFSystem.on_after_backward                                                                                                                  	|  1.2101e-06     	|  1000           	|  0.0012101      	|  0.00074414     	|
|  [LightningModule]NeRFSystem.on_before_optimizer_step                                                                                                           	|  1.2071e-06     	|  1000           	|  0.0012071      	|  0.00074229     	|
|  [LightningModule]NeRFSystem.on_before_backward                                                                                                                 	|  1.127e-06      	|  1000           	|  0.001127       	|  0.00069304     	|
|  [Strategy]SingleDeviceStrategy.on_train_batch_start                                                                                                            	|  1.0883e-06     	|  1000           	|  0.0010883      	|  0.00066927     	|
|  [LightningModule]NeRFSystem.on_validation_model_eval                                                                                                           	|  0.0010848      	|  1              	|  0.0010848      	|  0.00066711     	|
|  [Callback]TQDMProgressBar.on_validation_start                                                                                                                  	|  0.0008382      	|  1              	|  0.0008382      	|  0.00051544     	|
|  [LightningModule]NeRFSystem.on_validation_model_train                                                                                                          	|  0.00057599     	|  1              	|  0.00057599     	|  0.0003542      	|
|  [Callback]TQDMProgressBar.on_train_end                                                                                                                         	|  0.00056876     	|  1              	|  0.00056876     	|  0.00034976     	|
|  [LightningModule]NeRFSystem.on_validation_model_zero_grad                                                                                                      	|  0.00055088     	|  1              	|  0.00055088     	|  0.00033876     	|
|  [Callback]TQDMProgressBar.on_validation_end                                                                                                                    	|  0.00050173     	|  1              	|  0.00050173     	|  0.00030854     	|
|  [Callback]TQDMProgressBar.on_validation_batch_start                                                                                                            	|  2.1688e-05     	|  21             	|  0.00045545     	|  0.00028008     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 1, 'train_time_interval': None}.setup                    	|  0.00042213     	|  1              	|  0.00042213     	|  0.00025959     	|
|  [LightningModule]NeRFSystem.train_dataloader                                                                                                                   	|  0.0002973      	|  1              	|  0.0002973      	|  0.00018282     	|
|  [Callback]TQDMProgressBar.on_train_start                                                                                                                       	|  0.0002917      	|  1              	|  0.0002917      	|  0.00017938     	|
|  [Callback]TQDMProgressBar.on_train_epoch_end                                                                                                                   	|  0.00021983     	|  1              	|  0.00021983     	|  0.00013518     	|
|  [Callback]TQDMProgressBar.on_train_epoch_start                                                                                                                 	|  0.00021481     	|  1              	|  0.00021481     	|  0.00013209     	|
|  [LightningModule]NeRFSystem.val_dataloader                                                                                                                     	|  0.00018445     	|  1              	|  0.00018445     	|  0.00011342     	|
|  [LightningModule]NeRFSystem.lr_scheduler_step                                                                                                                  	|  9.3418e-05     	|  1              	|  9.3418e-05     	|  5.7446e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 1, 'train_time_interval': None}.on_validation_batch_end  	|  3.8384e-06     	|  21             	|  8.0607e-05     	|  4.9569e-05     	|
|  [LightningModule]NeRFSystem.on_validation_batch_end                                                                                                            	|  3.0162e-06     	|  21             	|  6.3339e-05     	|  3.895e-05      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 1, 'train_time_interval': None}.on_validation_batch_start	|  2.4048e-06     	|  21             	|  5.0501e-05     	|  3.1055e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 1, 'train_time_interval': None}.on_validation_end        	|  4.8949e-05     	|  1              	|  4.8949e-05     	|  3.0101e-05     	|
|  [LightningModule]NeRFSystem.on_validation_batch_start                                                                                                          	|  1.9823e-06     	|  21             	|  4.1629e-05     	|  2.5599e-05     	|
|  [Callback]TQDMProgressBar.on_fit_end                                                                                                                           	|  1.152e-05      	|  1              	|  1.152e-05      	|  7.0841e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 1, 'train_time_interval': None}.on_train_end             	|  1.0559e-05     	|  1              	|  1.0559e-05     	|  6.4934e-06     	|
|  [Callback]TQDMProgressBar.teardown                                                                                                                             	|  8.34e-06       	|  1              	|  8.34e-06       	|  5.1286e-06     	|
|  [Callback]TQDMProgressBar.setup                                                                                                                                	|  7.96e-06       	|  1              	|  7.96e-06       	|  4.8949e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 1, 'train_time_interval': None}.on_validation_start      	|  6.6198e-06     	|  1              	|  6.6198e-06     	|  4.0708e-06     	|
|  [LightningModule]NeRFSystem.on_validation_end                                                                                                                  	|  6.17e-06       	|  1              	|  6.17e-06       	|  3.7942e-06     	|
|  [Strategy]SingleDeviceStrategy.on_validation_start                                                                                                             	|  4.4801e-06     	|  1              	|  4.4801e-06     	|  2.755e-06      	|
|  [LightningModule]NeRFSystem.on_train_end                                                                                                                       	|  4.38e-06       	|  1              	|  4.38e-06       	|  2.6934e-06     	|
|  [LightningModule]NeRFSystem.on_save_checkpoint                                                                                                                 	|  4.0298e-06     	|  1              	|  4.0298e-06     	|  2.4781e-06     	|
|  [LightningModule]NeRFSystem.teardown                                                                                                                           	|  3.8603e-06     	|  1              	|  3.8603e-06     	|  2.3739e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 1, 'train_time_interval': None}.on_train_start           	|  3.56e-06       	|  1              	|  3.56e-06       	|  2.1892e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 1, 'train_time_interval': None}.on_fit_end               	|  3.4599e-06     	|  1              	|  3.4599e-06     	|  2.1276e-06     	|
|  [LightningModule]NeRFSystem.configure_callbacks                                                                                                                	|  3.4203e-06     	|  1              	|  3.4203e-06     	|  2.1033e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 1, 'train_time_interval': None}.teardown                 	|  3.3998e-06     	|  1              	|  3.3998e-06     	|  2.0907e-06     	|
|  [Callback]TQDMProgressBar.on_fit_start                                                                                                                         	|  3.3202e-06     	|  1              	|  3.3202e-06     	|  2.0417e-06     	|
|  [LightningModule]NeRFSystem.on_fit_end                                                                                                                         	|  3.2499e-06     	|  1              	|  3.2499e-06     	|  1.9985e-06     	|
|  [Strategy]SingleDeviceStrategy.on_train_end                                                                                                                    	|  3.1297e-06     	|  1              	|  3.1297e-06     	|  1.9246e-06     	|
|  [Strategy]SingleDeviceStrategy.on_validation_end                                                                                                               	|  2.8796e-06     	|  1              	|  2.8796e-06     	|  1.7708e-06     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_end                                                                                                              	|  2.8196e-06     	|  1              	|  2.8196e-06     	|  1.7339e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 1, 'train_time_interval': None}.on_train_epoch_start     	|  2.5602e-06     	|  1              	|  2.5602e-06     	|  1.5744e-06     	|
|  [LightningModule]NeRFSystem.prepare_data                                                                                                                       	|  2.2599e-06     	|  1              	|  2.2599e-06     	|  1.3897e-06     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_start                                                                                                            	|  2.1197e-06     	|  1              	|  2.1197e-06     	|  1.3035e-06     	|
|  [LightningModule]NeRFSystem.on_train_epoch_end                                                                                                                 	|  2.0098e-06     	|  1              	|  2.0098e-06     	|  1.2359e-06     	|
|  [LightningModule]NeRFSystem.on_train_start                                                                                                                     	|  1.7299e-06     	|  1              	|  1.7299e-06     	|  1.0638e-06     	|
|  [LightningModule]NeRFSystem.on_fit_start                                                                                                                       	|  1.6503e-06     	|  1              	|  1.6503e-06     	|  1.0148e-06     	|
|  [LightningModule]NeRFSystem.on_train_epoch_start                                                                                                               	|  1.6098e-06     	|  1              	|  1.6098e-06     	|  9.8992e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 1, 'train_time_interval': None}.on_validation_epoch_end  	|  1.5302e-06     	|  1              	|  1.5302e-06     	|  9.4096e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 1, 'train_time_interval': None}.on_fit_start             	|  1.4701e-06     	|  1              	|  1.4701e-06     	|  9.0402e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 1, 'train_time_interval': None}.on_validation_epoch_start	|  1.4598e-06     	|  1              	|  1.4598e-06     	|  8.9772e-07     	|
|  [Strategy]SingleDeviceStrategy.on_train_start                                                                                                                  	|  1.4198e-06     	|  1              	|  1.4198e-06     	|  8.7309e-07     	|
|  [LightningModule]NeRFSystem.on_validation_epoch_start                                                                                                          	|  9.6019e-07     	|  1              	|  9.6019e-07     	|  5.9046e-07     	|
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
FIT Profiler Report

--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Action                                                                                                                                                          	|  Mean duration (s)	|  Num calls      	|  Total time (s) 	|  Percentage %   	|
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Total                                                                                                                                                           	|  -              	|  600724         	|  1819.1         	|  100 %          	|
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  run_training_epoch                                                                                                                                              	|  90.449         	|  20             	|  1809.0         	|  99.446         	|
|  run_training_batch                                                                                                                                              	|  0.066106       	|  20000          	|  1322.1         	|  72.681         	|
|  [LightningModule]NeRFSystem.optimizer_step                                                                                                                      	|  0.065934       	|  20000          	|  1318.7         	|  72.492         	|
|  [_TrainingEpochLoop].train_dataloader_next                                                                                                                      	|  0.020615       	|  20000          	|  412.29         	|  22.665         	|
|  [Strategy]SingleDeviceStrategy.training_step                                                                                                                    	|  0.017959       	|  20000          	|  359.17         	|  19.745         	|
|  [Strategy]SingleDeviceStrategy.backward                                                                                                                         	|  0.0032447      	|  20000          	|  64.894         	|  3.5674         	|
|  [Strategy]SingleDeviceStrategy.validation_step                                                                                                                  	|  0.85409        	|  42             	|  35.872         	|  1.972          	|
|  [Callback]TQDMProgressBar.on_train_batch_end                                                                                                                    	|  0.00068336     	|  20000          	|  13.667         	|  0.75133        	|
|  [Strategy]SingleDeviceStrategy.batch_to_device                                                                                                                  	|  0.00026333     	|  20042          	|  5.2777         	|  0.29013        	|
|  [LightningModule]NeRFSystem.setup                                                                                                                               	|  4.1124         	|  1              	|  4.1124         	|  0.22607        	|
|  [LightningModule]NeRFSystem.transfer_batch_to_device                                                                                                            	|  0.00018465     	|  20042          	|  3.7007         	|  0.20344        	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None}.on_train_epoch_end       	|  0.12886        	|  20             	|  2.5773         	|  0.14168        	|
|  [LightningModule]NeRFSystem.optimizer_zero_grad                                                                                                                 	|  2.7643e-05     	|  20000          	|  0.55287        	|  0.030393       	|
|  [LightningModule]NeRFSystem.configure_gradient_clipping                                                                                                         	|  2.6397e-05     	|  20000          	|  0.52794        	|  0.029023       	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None}.on_train_batch_end       	|  1.9777e-05     	|  20000          	|  0.39554        	|  0.021744       	|
|  [LightningModule]NeRFSystem.on_before_batch_transfer                                                                                                            	|  1.0249e-05     	|  20042          	|  0.20541        	|  0.011292       	|
|  [_EvaluationLoop].val_next                                                                                                                                      	|  0.0047102      	|  42             	|  0.19783        	|  0.010875       	|
|  [Callback]TQDMProgressBar.on_after_backward                                                                                                                     	|  4.2675e-06     	|  20000          	|  0.085351       	|  0.004692       	|
|  [Callback]TQDMProgressBar.on_before_zero_grad                                                                                                                   	|  3.9533e-06     	|  20000          	|  0.079066       	|  0.0043465      	|
|  [LightningModule]NeRFSystem.on_after_batch_transfer                                                                                                             	|  3.3287e-06     	|  20042          	|  0.066713       	|  0.0036674      	|
|  [Callback]TQDMProgressBar.on_train_batch_start                                                                                                                  	|  2.9418e-06     	|  20000          	|  0.058835       	|  0.0032343      	|
|  [Callback]TQDMProgressBar.on_before_optimizer_step                                                                                                              	|  2.8797e-06     	|  20000          	|  0.057594       	|  0.0031661      	|
|  [LightningModule]NeRFSystem.on_validation_start                                                                                                                 	|  0.025869       	|  2              	|  0.051739       	|  0.0028443      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None}.on_before_zero_grad      	|  2.4643e-06     	|  20000          	|  0.049285       	|  0.0027094      	|
|  [LightningModule]NeRFSystem.on_train_batch_start                                                                                                                	|  2.3489e-06     	|  20000          	|  0.046978       	|  0.0025825      	|
|  [Callback]TQDMProgressBar.on_before_backward                                                                                                                    	|  2.324e-06      	|  20000          	|  0.04648        	|  0.0025551      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None}.on_train_batch_start     	|  2.1717e-06     	|  20000          	|  0.043434       	|  0.0023877      	|
|  [LightningModule]NeRFSystem.on_after_backward                                                                                                                   	|  1.7265e-06     	|  20000          	|  0.03453        	|  0.0018982      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None}.on_after_backward        	|  1.7111e-06     	|  20000          	|  0.034222       	|  0.0018813      	|
|  [LightningModule]NeRFSystem.on_train_batch_end                                                                                                                  	|  1.6655e-06     	|  20000          	|  0.03331        	|  0.0018312      	|
|  [Strategy]SingleDeviceStrategy.on_train_batch_start                                                                                                             	|  1.5803e-06     	|  20000          	|  0.031606       	|  0.0017375      	|
|  [LightningModule]NeRFSystem.on_before_backward                                                                                                                  	|  1.5627e-06     	|  20000          	|  0.031254       	|  0.0017182      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None}.on_before_optimizer_step 	|  1.5118e-06     	|  20000          	|  0.030237       	|  0.0016622      	|
|  [LightningModule]NeRFSystem.on_before_zero_grad                                                                                                                 	|  1.4757e-06     	|  20000          	|  0.029514       	|  0.0016225      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None}.on_before_backward       	|  1.4075e-06     	|  20000          	|  0.02815        	|  0.0015475      	|
|  [LightningModule]NeRFSystem.on_before_optimizer_step                                                                                                            	|  1.3008e-06     	|  20000          	|  0.026015       	|  0.0014302      	|
|  [Callback]TQDMProgressBar.on_validation_batch_end                                                                                                               	|  0.00029512     	|  42             	|  0.012395       	|  0.00068139     	|
|  [LightningModule]NeRFSystem.on_validation_epoch_end                                                                                                             	|  0.0061058      	|  2              	|  0.012212       	|  0.00067131     	|
|  [Callback]TQDMProgressBar.on_train_epoch_end                                                                                                                    	|  0.00049271     	|  20             	|  0.0098543      	|  0.00054172     	|
|  [Callback]TQDMProgressBar.on_train_epoch_start                                                                                                                  	|  0.00023908     	|  20             	|  0.0047815      	|  0.00026286     	|
|  [LightningModule]NeRFSystem.configure_optimizers                                                                                                                	|  0.0045692      	|  1              	|  0.0045692      	|  0.00025118     	|
|  [Callback]TQDMProgressBar.on_validation_start                                                                                                                   	|  0.0010229      	|  2              	|  0.0020458      	|  0.00011246     	|
|  [LightningModule]NeRFSystem.on_validation_model_eval                                                                                                            	|  0.00093774     	|  2              	|  0.0018755      	|  0.0001031      	|
|  [Callback]TQDMProgressBar.on_validation_batch_start                                                                                                             	|  3.4279e-05     	|  42             	|  0.0014397      	|  7.9146e-05     	|
|  [LightningModule]NeRFSystem.lr_scheduler_step                                                                                                                   	|  6.4147e-05     	|  20             	|  0.0012829      	|  7.0527e-05     	|
|  [LightningModule]NeRFSystem.on_validation_model_zero_grad                                                                                                       	|  0.00049624     	|  2              	|  0.00099248     	|  5.456e-05      	|
|  [LightningModule]NeRFSystem.on_validation_model_train                                                                                                           	|  0.00048777     	|  2              	|  0.00097555     	|  5.3629e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None}.setup                    	|  0.00092569     	|  1              	|  0.00092569     	|  5.0888e-05     	|
|  [Callback]TQDMProgressBar.on_validation_end                                                                                                                     	|  0.00038487     	|  2              	|  0.00076975     	|  4.2316e-05     	|
|  [Callback]TQDMProgressBar.on_train_end                                                                                                                          	|  0.00045347     	|  1              	|  0.00045347     	|  2.4929e-05     	|
|  [Callback]TQDMProgressBar.on_train_start                                                                                                                        	|  0.00032924     	|  1              	|  0.00032924     	|  1.8099e-05     	|
|  [LightningModule]NeRFSystem.train_dataloader                                                                                                                    	|  0.00031397     	|  1              	|  0.00031397     	|  1.726e-05      	|
|  [LightningModule]NeRFSystem.val_dataloader                                                                                                                      	|  0.00027728     	|  1              	|  0.00027728     	|  1.5243e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None}.on_validation_batch_end  	|  3.1362e-06     	|  42             	|  0.00013172     	|  7.241e-06      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None}.on_validation_batch_start	|  2.6421e-06     	|  42             	|  0.00011097     	|  6.1004e-06     	|
|  [LightningModule]NeRFSystem.on_validation_batch_end                                                                                                             	|  2.3763e-06     	|  42             	|  9.9807e-05     	|  5.4867e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None}.on_validation_end        	|  3.608e-05      	|  2              	|  7.2159e-05     	|  3.9668e-06     	|
|  [LightningModule]NeRFSystem.on_validation_batch_start                                                                                                           	|  1.5285e-06     	|  42             	|  6.4199e-05     	|  3.5292e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None}.on_train_epoch_start     	|  2.749e-06      	|  20             	|  5.4979e-05     	|  3.0224e-06     	|
|  [LightningModule]NeRFSystem.on_train_epoch_start                                                                                                                	|  1.9014e-06     	|  20             	|  3.8027e-05     	|  2.0905e-06     	|
|  [LightningModule]NeRFSystem.on_train_epoch_end                                                                                                                  	|  1.61e-06       	|  20             	|  3.2199e-05     	|  1.7701e-06     	|
|  [LightningModule]NeRFSystem.on_fit_end                                                                                                                          	|  1.352e-05      	|  1              	|  1.352e-05      	|  7.4324e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None}.on_validation_start      	|  5.9793e-06     	|  2              	|  1.1959e-05     	|  6.574e-07      	|
|  [LightningModule]NeRFSystem.on_validation_end                                                                                                                   	|  5.2899e-06     	|  2              	|  1.058e-05      	|  5.8161e-07     	|
|  [Strategy]SingleDeviceStrategy.on_validation_start                                                                                                              	|  4.3751e-06     	|  2              	|  8.7502e-06     	|  4.8103e-07     	|
|  [Callback]TQDMProgressBar.setup                                                                                                                                 	|  8.0699e-06     	|  1              	|  8.0699e-06     	|  4.4363e-07     	|
|  [Callback]TQDMProgressBar.on_fit_end                                                                                                                            	|  6.76e-06       	|  1              	|  6.76e-06       	|  3.7162e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None}.on_train_end             	|  5.92e-06       	|  1              	|  5.92e-06       	|  3.2544e-07     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_end                                                                                                               	|  2.7253e-06     	|  2              	|  5.4506e-06     	|  2.9963e-07     	|
|  [Strategy]SingleDeviceStrategy.on_validation_end                                                                                                                	|  2.455e-06      	|  2              	|  4.9099e-06     	|  2.6991e-07     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_start                                                                                                             	|  2.3448e-06     	|  2              	|  4.6897e-06     	|  2.5781e-07     	|
|  [LightningModule]NeRFSystem.configure_callbacks                                                                                                                 	|  4.6301e-06     	|  1              	|  4.6301e-06     	|  2.5453e-07     	|
|  [Callback]TQDMProgressBar.teardown                                                                                                                              	|  3.9297e-06     	|  1              	|  3.9297e-06     	|  2.1603e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None}.on_train_start           	|  3.8501e-06     	|  1              	|  3.8501e-06     	|  2.1165e-07     	|
|  [LightningModule]NeRFSystem.on_train_end                                                                                                                        	|  3.349e-06      	|  1              	|  3.349e-06      	|  1.8411e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None}.on_validation_epoch_start	|  1.4249e-06     	|  2              	|  2.8498e-06     	|  1.5667e-07     	|
|  [Callback]TQDMProgressBar.on_fit_start                                                                                                                          	|  2.7996e-06     	|  1              	|  2.7996e-06     	|  1.539e-07      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None}.on_validation_epoch_end  	|  1.3954e-06     	|  2              	|  2.7907e-06     	|  1.5341e-07     	|
|  [LightningModule]NeRFSystem.prepare_data                                                                                                                        	|  2.61e-06       	|  1              	|  2.61e-06       	|  1.4348e-07     	|
|  [LightningModule]NeRFSystem.on_validation_epoch_start                                                                                                           	|  1.2249e-06     	|  2              	|  2.4498e-06     	|  1.3468e-07     	|
|  [LightningModule]NeRFSystem.on_save_checkpoint                                                                                                                  	|  2.3902e-06     	|  1              	|  2.3902e-06     	|  1.314e-07      	|
|  [Strategy]SingleDeviceStrategy.on_train_end                                                                                                                     	|  2.2403e-06     	|  1              	|  2.2403e-06     	|  1.2316e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None}.on_fit_start             	|  2.0396e-06     	|  1              	|  2.0396e-06     	|  1.1212e-07     	|
|  [LightningModule]NeRFSystem.on_train_start                                                                                                                      	|  1.9702e-06     	|  1              	|  1.9702e-06     	|  1.0831e-07     	|
|  [Strategy]SingleDeviceStrategy.on_train_start                                                                                                                   	|  1.6498e-06     	|  1              	|  1.6498e-06     	|  9.0697e-08     	|
|  [LightningModule]NeRFSystem.on_fit_start                                                                                                                        	|  1.6298e-06     	|  1              	|  1.6298e-06     	|  8.9596e-08     	|
|  [LightningModule]NeRFSystem.teardown                                                                                                                            	|  1.6093e-06     	|  1              	|  1.6093e-06     	|  8.847e-08      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None}.teardown                 	|  1.5502e-06     	|  1              	|  1.5502e-06     	|  8.5219e-08     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None}.on_fit_end               	|  1.5297e-06     	|  1              	|  1.5297e-06     	|  8.4092e-08     	|
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
