use triton to init window info

[ANDgate99]

No description provided.

[ANDgate99]

batch_size: 45
b_seq_len: tensor([23, 45, 24, 31, 46, 32, 46, 27, 45, 42, 34, 30, 32, 31, 27, 35, 28, 34,
23, 41, 33, 21, 24, 49, 24, 49, 33, 29, 48, 20, 49, 23, 22, 31, 47, 22,
43, 29, 45, 32, 29, 36, 49, 23, 47], device='cuda:0',
dtype=torch.int32)
sliding_window: 32
b_attn_len: tensor([23, 32, 24, 31, 32, 32, 32, 27, 32, 32, 32, 30, 32, 31, 27, 32, 28, 32,
23, 32, 32, 21, 24, 32, 24, 32, 32, 29, 32, 20, 32, 23, 22, 31, 32, 22,
32, 29, 32, 32, 29, 32, 32, 23, 32], device='cuda:0',
dtype=torch.int32)

data = {
'inputs': 'Who is George Washington?',
"parameters": {
'do_sample': False,
'ignore_eos': False,
'max_new_tokens': 256,
}
}
{'generated_text': ['\n\nGeorge Washington was the first president of the United States. He was born on February 22, 1732, in Westmoreland County, Virginia. He was the son of Augustine Washington and Mary Ball Washington. He had two brothers and two sisters.\n\nGeorge Washington was a very important person in American history. He was the first president of the United States and he helped to create the country. He was also a general in the American Revolutionary War.\n\nGeorge Washington was born on February 22, 1732, in Westmoreland County, Virginia. He was the son of Augustine Washington and Mary Ball Washington. He had two brothers and two sisters.\n\nGeorge Washington was a very important person in American history. He was the first president of the United States and he helped to create the country. He was also a general in the American Revolutionary War.\n\nGeorge Washington was born on February 22, 1732, in Westmoreland County, Virginia. He was the son of Augustine Washington and Mary Ball Washington. He had two brothers and two sisters.\n\nGeorge Washington was a very important person in American history. He was'], 'count_output_tokens': 256, 'finish_reason': 'length'}

data = {
'inputs': 'What is AI?',
"parameters": {
'do_sample': False,
'ignore_eos': False,
'max_new_tokens': 256,
}
}
{'generated_text': ['\n\nArtificial Intelligence (AI) is a branch of computer science that deals with the simulation of intelligent behavior in computers.\n\nWhat is Machine Learning?\n\nMachine learning is a subset of artificial intelligence in the field of computer science that often uses statistical techniques to give computers the ability to “learn” (i.e., progressively improve performance on a specific task) with data, without being explicitly programmed.\n\nWhat is Deep Learning?\n\nDeep learning is a subset of machine learning that uses artificial neural networks to learn from data.\n\nWhat is a neural network?\n\nA neural network is a computational model inspired by the structure and function of biological neural networks.\n\nWhat is a convolutional neural network?\n\nA convolutional neural network (CNN) is a type of neural network, most commonly applied to analyzing visual imagery.\n\nWhat is a recurrent neural network?\n\nA recurrent neural network (RNN) is a type of artificial neural network where connections between nodes form a directed cycle.\n\nWhat is a generative adversarial network?\n\nA generative adversarial network (GAN) is a type of neural network that uses two neural networks to generate'], 'count_output_tokens': 256, 'finish_reason': 'length'}