Benchmarks - Add LLaMA-2 Models

[dpower4]

Added llama benchmark - training and inference in accordance with the existing pytorch models implementation like gpt2, lstm etc.

• added llama fp8 unit test for better code coverage, to reduce memory required
• updated transformers version >= 4.28.0 for LLamaConfig
• set tokenizers version <= 0.20.3 to avoid 0.20.4 version issues with py3.8
• added llama2 to tensorrt
• llama2 tests not added to test_tensorrt_inference_performance.py due to large memory requirement for worker gpu. tests validated separately on gh200

[abuccts]

pls use python3 setup.py lint to check the format and run python3 setup.py format to format and code

[dpower4]

@abuccts can I get access to the unit test logs.

[codecov]

Codecov Report

Attention: Patch coverage is 87.70492% with 15 lines in your changes missing coverage. Please review.

Project coverage is 85.61%. Comparing base (4e6935a) to head (1570707).
Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
...bench/benchmarks/model_benchmarks/pytorch_llama.py	87.17%	15 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #668      +/-   ##
==========================================
+ Coverage   85.58%   85.61%   +0.03%     
==========================================
  Files          98       99       +1     
  Lines        7046     7165     +119     
==========================================
+ Hits         6030     6134     +104     
- Misses       1016     1031      +15     

Flag	Coverage Δ
cpu-python3.10-unit-test	71.21% <35.53%> (-0.64%)	⬇️
cpu-python3.7-unit-test	71.18% <36.06%> (-0.63%)	⬇️
cpu-python3.8-unit-test	71.22% <36.13%> (-0.62%)	⬇️
cuda-unit-test	83.42% <85.95%> (+0.03%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[guoshzhao]

LGTM, thanks! Please fix the UT failures with Python 1.10. And since the CUDA tests are running on K80 which is very old GPU, we can skip the "cuda-init-test", and just make sure "cpu-unit-test" can pass.

/__w/1/s/.eggs/setuptools_scm-8.1.0-py3.10.egg/setuptools_scm/_integration/setuptools.py:92: UserWarning: version of superbench already set
  warnings.warn(f"version of {dist_name} already set")
running lint
tests/analyzer/test_summaryop.py:7: error: Module "numpy" has no attribute "NaN"  [attr-defined]

[dpower4]

pytorch-llama validation on gh200.

[dpower4]

tokenizers Rust:cargo issue: huggingface/tokenizers#1691