profiler ROCm tracing result was wrong

[demonsan]

I try to profile a resnet50 model based on https://pytorch.org/tutorials/intermediate/tensorboard_profiler_tutorial.html.
But the trace result in chrome://tracing is weird.

The profiler should record from 0ms but here record from nearly 250 ms. And the active step didn't launch the kernel on GPU.
The schedule was set up to wait=1, warmup=1, active=1 .

cc @aaronenyeshi @chaekit @sekyondaMeta @svekars @carljparker @NicolasHug @kit1980 @subramen @robieta

[ver2king]

/assigntome

[svekars]

This issue has been unassigned due to inactivity. If you are still planning to work on this, you can still send a PR referencing this issue.

[guptaaryan16]

/assigntome

[guptaaryan16]

I am not able to get any output in tensorboard after running the pt_profiler. It gives the following error and displays a blank page in PyTorch profiler tab, Is it due to the fact that the tensorboard profiler is being depreciated? I have tested this on colab and my local setup(MacBook Air M1).
Can someone confirm my findings by reproducing this on their end?
Although seems like we can visualise the trace here: https://ui.perfetto.dev/ as referred in this issue pytorch/kineto#805

> tensorboard --logdir=log --load_fast=false
TensorFlow installation not found - running with reduced feature set.
W1106 21:01:02.988501 8174490368 profile_plugin_loader.py:71] Unable to load profiler plugin. Import error: cannot import name 'builder' from 'google.protobuf.internal' (/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/google/protobuf/internal/__init__.py)
I1106 21:01:04.105515 6173667328 plugin.py:429] Monitor runs begin
I1106 21:01:04.106191 6173667328 plugin.py:444] Find run directory /Users/guptaaryan16/Desktop/OSS/test/result
I1106 21:01:04.107095 6190493696 plugin.py:493] Load run result
I1106 21:01:04.112741 6190493696 loader.py:57] started all processing
Serving TensorBoard on localhost; to expose to the network, use a proxy or pass --bind_all
TensorBoard 2.10.0 at http://localhost:6006/ (Press CTRL+C to quit)
I1106 21:01:05.205070 6190493696 plugin.py:497] Run result loaded
I1106 21:01:05.205360 6207320064 plugin.py:467] Add run result
W1106 21:01:12.815429 6190493696 application.py:558] path /data/index.js not found, sending 404
W1106 21:01:16.148443 6190493696 application.py:558] path /data/index.js not found, sending 404

Also I was thinking about rewriting this tutorial using torch/kineto, Any thoughts on that?
cc @svekars @carljparker