[doc]: Stream Telemetry HLD #1795
Conversation
Pterosaur
force-pushed
the
hfc
branch
3 times, most recently
from
bc0e247
to
c14e643
Compare
| ### Architecture Design | ||
|
|
||
| This section covers the changes that are required in the SONiC architecture. In general, it is expected that the current architecture is not changed. | ||
| This section should explain how the new feature/enhancement (module/sub-module) fits in the existing architecture. |
There was a problem hiding this comment.
same here. these "guidance" words are better to be removed. very confusing to read.
There was a problem hiding this comment.
Sorry, Yes, there are many template "guidance" words I haven't removed them.
|
|
||
| #### Modules #### | ||
|
|
||
| ##### Netlink Module ##### |
There was a problem hiding this comment.
i know the numbered prefix are not there in the template, but the doc will be much more readable with it. we should consider add them here.
There was a problem hiding this comment.
Hi @zhangyanzhao , Do you have any suggestions or concerns about adding a number prefix to each title in the template?
|
|
||
| Pin CPU? | ||
|
|
||
|  |
There was a problem hiding this comment.
are we still uses the chunk size concept in our API?
There was a problem hiding this comment.
Or leverage the existing solution, time-based reporting?
Signed-off-by: Ze Gan <zegan@microsoft.com>
Pterosaur
force-pushed
the
hfc
branch
8 times, most recently
from
290f7f9
to
0604b27
Compare
Signed-off-by: Ze Gan <zegan@microsoft.com>
Signed-off-by: Ze Gan <zegan@microsoft.com>
Signed-off-by: Ze Gan <zegan@microsoft.com>
|
|
||
| - For high-frequency counters, the native IPFIX timestamp unit of seconds is insufficient. Therefore, we introduce an additional element, `observationTimeNanoseconds`, for each record to meet our requirements. | ||
| - The enterprise bit is always set to 1 for stats records. | ||
| - The element ID of IPFIX is derived from the object index. For example, for `Ethernet5`, the element ID will be `0x5 | 0x8000 = 0x8005`. |
There was a problem hiding this comment.
Having a variable element ID per SAI object will blow up the number of templates in the report (separate template ID per port, queue etc.).
There was a problem hiding this comment.
In my mind, I assume all stats(per port, queue etc) in one profile will be encoded into ONE template. I will generate a unique template ID for a profile.
|
|
||
| ### Data format | ||
|
|
||
| We will use IPFIX as the report format, with all numbers in the IPFIX message in network-order (Big-endian). |
There was a problem hiding this comment.
You cannot make the assumption that data will be Big- or Little-endian. If it is collected by the ASIC and DMAd to a ring buffer, then kernel driver would need to correct the endianness of each data set. We need to have a bit in the element ID or Enterprise number that would indicate endianness.
There was a problem hiding this comment.
Are you worried that endian conversion will consume too many CPU clocks?
| ``` | ||
| STREAM_TELEMETRY_GROUP|{{profile_name}}|{{group_name}} | ||
| "object_names": {{list of object name}} | ||
| "object_counters": {{list of stats of object}} |
There was a problem hiding this comment.
How do you plan to handle the update of the counter list or object list from the report POV? is it going to be a new template ID?
There was a problem hiding this comment.
Yes, I assume we will use a new template ID.
We treat any configuration update as a new profile. The previous session will be interrupted. The SAI report object will get a new SAI_TAM_REPORT_ATTR_REPORT_IPFIX_TEMPLATE_ID
Signed-off-by: Ze Gan <zegan@microsoft.com>
Signed-off-by: Ze Gan <zegan@microsoft.com>
Signed-off-by: Ze Gan <zegan@microsoft.com>
Signed-off-by: Ze Gan <zegan@microsoft.com>
Signed-off-by: Ze Gan <zegan@microsoft.com>
Signed-off-by: Ze Gan <zegan@microsoft.com>
Signed-off-by: Ze Gan <zegan@microsoft.com>
Signed-off-by: Ze Gan <zegan@microsoft.com>
Stream Telemetry HLD initial version