Skip to content

Basic federated support on Zephyr #73

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 73 commits into from
Oct 22, 2024
Merged

Basic federated support on Zephyr #73

merged 73 commits into from
Oct 22, 2024

Conversation

erlingrj
Copy link
Collaborator

This PR mainly adds an example of federated execution on Zephyr using TcpIpChannel

tanneberger and others added 30 commits October 3, 2024 23:36
Copy link
Contributor

Memory usage after merging this PR will be:

Memory Report

action_microstep_test_c

from to increase (%)
text 71299 39983 -43.92
data 960 744 -22.50
bss 480 480 0.00
total 72739 41207 -43.35

action_test_c

from to increase (%)
text 71099 39783 -44.05
data 960 744 -22.50
bss 480 480 0.00
total 72539 41007 -43.47

delayed_conn_test_c

from to increase (%)
text 75610 42758 -43.45
data 952 736 -22.69
bss 480 480 0.00
total 77042 43974 -42.92

event_queue_test_c

from to increase (%)
text 23406 26228 12.06
data 648 712 9.88
bss 320 480 50.00
total 24374 27420 12.50

physical_action_test_c

from to increase (%)
text 72123 41031 -43.11
data 961 761 -20.81
bss 1952 1952 0.00
total 75036 43744 -41.70

port_test_c

from to increase (%)
text 75576 42551 -43.70
data 952 736 -22.69
bss 480 480 0.00
total 77008 43767 -43.17

reaction_queue_test_c

from to increase (%)
text 23083 25905 12.23
data 648 712 9.88
bss 320 480 50.00
total 24051 27097 12.66

shutdown_test_c

from to increase (%)
text 67430 35997 -46.62
data 960 736 -23.33
bss 480 2624 446.67
total 68870 39357 -42.85

startup_test_c

from to increase (%)
text 66746 35313 -47.09
data 960 736 -23.33
bss 480 2400 400.00
total 68186 38449 -43.61

timer_test_c

from to increase (%)
text 66583 35131 -47.24
data 952 728 -23.53
bss 480 480 0.00
total 68015 36339 -46.57

trigger_data_queue_test_c

from to increase (%)
text 19344 22474 16.18
data 648 720 11.11
bss 320 480 50.00
total 20312 23674 16.55

Copy link
Contributor

Memory usage after merging this PR will be:

Memory Report

action_microstep_test_c

from to increase (%)
text 71299 39983 -43.92
data 960 744 -22.50
bss 480 480 0.00
total 72739 41207 -43.35

action_test_c

from to increase (%)
text 71099 39783 -44.05
data 960 744 -22.50
bss 480 480 0.00
total 72539 41007 -43.47

delayed_conn_test_c

from to increase (%)
text 75610 42758 -43.45
data 952 736 -22.69
bss 480 480 0.00
total 77042 43974 -42.92

event_queue_test_c

from to increase (%)
text 23406 26228 12.06
data 648 712 9.88
bss 320 480 50.00
total 24374 27420 12.50

physical_action_test_c

from to increase (%)
text 72123 41031 -43.11
data 961 761 -20.81
bss 1952 1952 0.00
total 75036 43744 -41.70

port_test_c

from to increase (%)
text 75576 42551 -43.70
data 952 736 -22.69
bss 480 480 0.00
total 77008 43767 -43.17

reaction_queue_test_c

from to increase (%)
text 23083 25905 12.23
data 648 712 9.88
bss 320 480 50.00
total 24051 27097 12.66

shutdown_test_c

from to increase (%)
text 67430 35997 -46.62
data 960 736 -23.33
bss 480 2624 446.67
total 68870 39357 -42.85

startup_test_c

from to increase (%)
text 66746 35313 -47.09
data 960 736 -23.33
bss 480 2400 400.00
total 68186 38449 -43.61

timer_test_c

from to increase (%)
text 66583 35131 -47.24
data 952 728 -23.53
bss 480 480 0.00
total 68015 36339 -46.57

trigger_data_queue_test_c

from to increase (%)
text 19344 22474 16.18
data 648 720 11.11
bss 320 480 50.00
total 20312 23674 16.55

Copy link
Contributor

Memory usage after merging this PR will be:

Memory Report

action_microstep_test_c

from to increase (%)
text 71299 39983 -43.92
data 960 744 -22.50
bss 480 480 0.00
total 72739 41207 -43.35

action_test_c

from to increase (%)
text 71099 39783 -44.05
data 960 744 -22.50
bss 480 480 0.00
total 72539 41007 -43.47

delayed_conn_test_c

from to increase (%)
text 75610 42758 -43.45
data 952 736 -22.69
bss 480 480 0.00
total 77042 43974 -42.92

event_queue_test_c

from to increase (%)
text 23406 26228 12.06
data 648 712 9.88
bss 320 480 50.00
total 24374 27420 12.50

physical_action_test_c

from to increase (%)
text 72123 41031 -43.11
data 961 761 -20.81
bss 1952 1952 0.00
total 75036 43744 -41.70

port_test_c

from to increase (%)
text 75576 42551 -43.70
data 952 736 -22.69
bss 480 480 0.00
total 77008 43767 -43.17

reaction_queue_test_c

from to increase (%)
text 23083 25905 12.23
data 648 712 9.88
bss 320 480 50.00
total 24051 27097 12.66

shutdown_test_c

from to increase (%)
text 67430 35997 -46.62
data 960 736 -23.33
bss 480 2624 446.67
total 68870 39357 -42.85

startup_test_c

from to increase (%)
text 66746 35313 -47.09
data 960 736 -23.33
bss 480 2400 400.00
total 68186 38449 -43.61

timer_test_c

from to increase (%)
text 66583 35131 -47.24
data 952 728 -23.53
bss 480 480 0.00
total 68015 36339 -46.57

trigger_data_queue_test_c

from to increase (%)
text 19344 22474 16.18
data 648 720 11.11
bss 320 480 50.00
total 20312 23674 16.55

Copy link
Contributor

Memory usage after merging this PR will be:

Memory Report

action_microstep_test_c

from to increase (%)
text 71299 39983 -43.92
data 960 744 -22.50
bss 480 480 0.00
total 72739 41207 -43.35

action_test_c

from to increase (%)
text 71099 39783 -44.05
data 960 744 -22.50
bss 480 480 0.00
total 72539 41007 -43.47

delayed_conn_test_c

from to increase (%)
text 75610 42758 -43.45
data 952 736 -22.69
bss 480 480 0.00
total 77042 43974 -42.92

event_queue_test_c

from to increase (%)
text 23406 26228 12.06
data 648 712 9.88
bss 320 480 50.00
total 24374 27420 12.50

physical_action_test_c

from to increase (%)
text 72123 41031 -43.11
data 961 761 -20.81
bss 1952 1952 0.00
total 75036 43744 -41.70

port_test_c

from to increase (%)
text 75576 42551 -43.70
data 952 736 -22.69
bss 480 480 0.00
total 77008 43767 -43.17

reaction_queue_test_c

from to increase (%)
text 23083 25905 12.23
data 648 712 9.88
bss 320 480 50.00
total 24051 27097 12.66

shutdown_test_c

from to increase (%)
text 67430 35997 -46.62
data 960 736 -23.33
bss 480 2624 446.67
total 68870 39357 -42.85

startup_test_c

from to increase (%)
text 66746 35313 -47.09
data 960 736 -23.33
bss 480 2400 400.00
total 68186 38449 -43.61

timer_test_c

from to increase (%)
text 66583 35131 -47.24
data 952 728 -23.53
bss 480 480 0.00
total 68015 36339 -46.57

trigger_data_queue_test_c

from to increase (%)
text 19344 22474 16.18
data 648 720 11.11
bss 320 480 50.00
total 20312 23674 16.55

Copy link
Contributor

Memory usage after merging this PR will be:

Memory Report

action_microstep_test_c

from to increase (%)
text 71299 39983 -43.92
data 960 744 -22.50
bss 480 480 0.00
total 72739 41207 -43.35

action_test_c

from to increase (%)
text 71099 39783 -44.05
data 960 744 -22.50
bss 480 480 0.00
total 72539 41007 -43.47

delayed_conn_test_c

from to increase (%)
text 75610 42758 -43.45
data 952 736 -22.69
bss 480 480 0.00
total 77042 43974 -42.92

event_queue_test_c

from to increase (%)
text 23406 26228 12.06
data 648 712 9.88
bss 320 480 50.00
total 24374 27420 12.50

physical_action_test_c

from to increase (%)
text 72123 41031 -43.11
data 961 761 -20.81
bss 1952 1952 0.00
total 75036 43744 -41.70

port_test_c

from to increase (%)
text 75576 42551 -43.70
data 952 736 -22.69
bss 480 480 0.00
total 77008 43767 -43.17

reaction_queue_test_c

from to increase (%)
text 23083 25905 12.23
data 648 712 9.88
bss 320 480 50.00
total 24051 27097 12.66

shutdown_test_c

from to increase (%)
text 67430 35997 -46.62
data 960 736 -23.33
bss 480 2624 446.67
total 68870 39357 -42.85

startup_test_c

from to increase (%)
text 66746 35313 -47.09
data 960 736 -23.33
bss 480 2400 400.00
total 68186 38449 -43.61

timer_test_c

from to increase (%)
text 66583 35131 -47.24
data 952 728 -23.53
bss 480 480 0.00
total 68015 36339 -46.57

trigger_data_queue_test_c

from to increase (%)
text 19344 22474 16.18
data 648 720 11.11
bss 320 480 50.00
total 20312 23674 16.55

Copy link
Contributor

Memory usage after merging this PR will be:

Memory Report

action_microstep_test_c

from to increase (%)
text 71299 39983 -43.92
data 960 744 -22.50
bss 480 480 0.00
total 72739 41207 -43.35

action_test_c

from to increase (%)
text 71099 39783 -44.05
data 960 744 -22.50
bss 480 480 0.00
total 72539 41007 -43.47

delayed_conn_test_c

from to increase (%)
text 75610 42758 -43.45
data 952 736 -22.69
bss 480 480 0.00
total 77042 43974 -42.92

event_queue_test_c

from to increase (%)
text 23406 26228 12.06
data 648 712 9.88
bss 320 480 50.00
total 24374 27420 12.50

physical_action_test_c

from to increase (%)
text 72123 41031 -43.11
data 961 761 -20.81
bss 1952 1952 0.00
total 75036 43744 -41.70

port_test_c

from to increase (%)
text 75576 42551 -43.70
data 952 736 -22.69
bss 480 480 0.00
total 77008 43767 -43.17

reaction_queue_test_c

from to increase (%)
text 23083 25905 12.23
data 648 712 9.88
bss 320 480 50.00
total 24051 27097 12.66

shutdown_test_c

from to increase (%)
text 67430 35997 -46.62
data 960 736 -23.33
bss 480 2624 446.67
total 68870 39357 -42.85

startup_test_c

from to increase (%)
text 66746 35313 -47.09
data 960 736 -23.33
bss 480 2400 400.00
total 68186 38449 -43.61

timer_test_c

from to increase (%)
text 66583 35131 -47.24
data 952 728 -23.53
bss 480 480 0.00
total 68015 36339 -46.57

trigger_data_queue_test_c

from to increase (%)
text 19344 22474 16.18
data 648 720 11.11
bss 320 480 50.00
total 20312 23674 16.55

Copy link
Collaborator Author

@erlingrj erlingrj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Self-review done

@erlingrj erlingrj requested a review from tanneberger October 21, 2024 20:09
Copy link
Contributor

Memory usage after merging this PR will be:

Memory Report

action_microstep_test_c

from to increase (%)
text 71299 43455 -39.05
data 960 744 -22.50
bss 480 480 0.00
total 72739 44679 -38.58

action_test_c

from to increase (%)
text 71099 43255 -39.16
data 960 744 -22.50
bss 480 480 0.00
total 72539 44479 -38.68

delayed_conn_test_c

from to increase (%)
text 75610 47542 -37.12
data 952 736 -22.69
bss 480 480 0.00
total 77042 48758 -36.71

event_queue_test_c

from to increase (%)
text 23406 26942 15.11
data 648 712 9.88
bss 320 480 50.00
total 24374 28134 15.43

physical_action_test_c

from to increase (%)
text 72123 44503 -38.30
data 961 761 -20.81
bss 1952 1952 0.00
total 75036 47216 -37.08

port_test_c

from to increase (%)
text 75576 47335 -37.37
data 952 736 -22.69
bss 480 480 0.00
total 77008 48551 -36.95

reaction_queue_test_c

from to increase (%)
text 23083 26619 15.32
data 648 712 9.88
bss 320 480 50.00
total 24051 27811 15.63

shutdown_test_c

from to increase (%)
text 67430 39208 -41.85
data 960 736 -23.33
bss 480 2624 446.67
total 68870 42568 -38.19

startup_test_c

from to increase (%)
text 66746 38524 -42.28
data 960 736 -23.33
bss 480 2400 400.00
total 68186 41660 -38.90

timer_test_c

from to increase (%)
text 66583 38342 -42.41
data 952 728 -23.53
bss 480 480 0.00
total 68015 39550 -41.85

trigger_data_queue_test_c

from to increase (%)
text 19344 22935 18.56
data 648 720 11.11
bss 320 480 50.00
total 20312 24135 18.82

Copy link
Contributor

Coverage after merging basic-federated-zephyr into main will be

35.69%

Coverage Report
FileStmtsBranchesFuncsLinesUncovered Lines
external/nanopb
   pb_common.c0%0%0%0%100, 100, 100, 100, 100, 102, 106, 110, 110, 110, 110, 110, 112, 116, 119, 122, 124, 126, 126, 126, 129–132, 14, 14, 14, 142–144, 15, 150–152, 152, 152, 152, 152, 154, 156, 158, 160–161, 163, 166, 168, 17, 171–172, 172, 172, 177, 18, 181, 184–185, 188, 190–192, 195, 197, 197, 197, 199, 20, 20, 20, 20, 20, 201, 201, 201, 203, 207, 210, 210, 210, 215, 22, 221, 224, 226, 226, 226, 229, 231, 231, 231–232, 232, 232, 235, 238, 238, 238, 24, 241–242, 246, 248, 248, 248, 25, 250, 254, 26, 260, 263, 265, 265, 265, 267, 269, 269, 269, 27, 272–273, 277, 28, 286–287, 29, 290, 292, 295, 297, 300, 302, 302, 302, 304, 306, 306, 306, 308, 308, 308, 308, 308, 310, 313, 313, 313, 313, 313, 315, 32, 320, 34, 36–41, 44, 46–48, 50–55, 58, 60–63, 65–70, 74, 74, 74, 77–78, 8, 82, 84, 84, 84, 86, 88, 88, 88–89, 89, 89–90, 90, 90, 93, 97
   pb_decode.c0%0%0%0%100, 1000–1001, 1003–1004, 1007, 1007, 1007, 1009, 1009, 1009, 1011, 1011, 1011–1012, 1012, 1012, 1016, 1016, 1016, 1022, 1022, 1022, 1024, 1024, 1024–1025, 1027, 1030, 1030, 1030, 1032, 1032, 1032, 1034, 1038, 1038, 1038, 104, 104, 104, 1042, 1042, 1042, 1042, 1042, 1045, 1045, 1045, 1047, 1047, 1047, 1049, 105, 105, 105, 1050, 1053, 1053, 1053, 1055, 1059, 1059, 1059, 1061, 1063, 1063, 1063–1064, 1066, 1066, 1066, 1069, 1074, 1074, 1074–1076, 108, 108, 108, 1082, 1082, 1082, 1082, 1082, 1084, 1084, 1084, 1089, 1089, 1089, 109, 109, 109, 1090, 1090, 1090, 1092, 1092, 1092, 1095–1097, 1100, 1103, 1103, 1103–1104, 1104, 1104, 1106–1107, 1110, 1110, 1110–1111, 1115, 1115, 1115–1116, 1116, 1116, 1118, 1118, 1118, 1123, 1125, 1125, 1125, 1129, 1129, 1129–1130, 1133, 1133, 1133, 1135, 1135, 1135–1136, 1136, 1136, 1140, 1140, 1140, 1142–1143, 1143, 1143, 1145, 1145, 1145, 115, 115, 115, 1151, 1154, 1158, 1158, 1158, 116, 1160, 1165, 1165, 1165–1166, 1168, 1170, 1170, 1170–1171, 1179, 118, 1182, 1186, 1193, 120, 125, 127, 127, 127–128, 128, 128, 131, 131, 131–132, 132, 132, 1336, 1341, 1346, 1349, 1349, 1349–1350, 1352–1353, 1356, 1359, 1359, 1359–1360, 1362, 1362, 1362–1363, 1365, 1367, 1370, 1377, 1377, 1377–1378, 138, 1382, 1389, 1393, 140, 1400, 1400, 1400–1401, 1405, 1416, 1420, 1422, 1425, 1427, 1427, 1427, 143, 1430, 1430, 1430–1431, 1434, 1434, 1434–1436, 1436, 1436–1438, 1438, 1438–1440, 1440, 1440–1441, 1443, 1443, 1443, 1445, 1445, 1445–1446, 1446, 1446, 1448, 1456, 1456, 1456, 1458, 1458, 1458–1459, 1463, 1463, 1463–1464, 1472, 1472, 1472–1473, 1475, 1479, 1479, 1479–1481, 1481, 1481–1483, 1483, 1483–1485, 1485, 1485–1486, 1488, 1488, 1488, 1490, 1490, 1490–1491, 1491, 1491, 1493, 1497, 1503, 1503, 1503–1504, 1506, 1506, 1506–1507, 1507, 1507, 1509–1510, 1510, 1510–1511, 1511, 1511, 1513, 1513, 1513, 1516, 1516, 1516, 1528, 1528, 1528–1529, 1529, 1529–1530, 1533–1534, 1537, 1541, 1543, 1543, 1543–1544, 1546, 1546, 1546–1547, 1547, 1547, 1550, 1552, 1552, 1552–1553, 1553, 1553, 1555,

project(reactor-uc-zephyr)

# TODO: Improve the way of pickng which Network channels to add to the build.
set(NETWORK_POSIX_TCP ON CACHE BOOL "blah")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

... "blah"

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes... using CMake options is irritating since I cannot just do set(NETWORK_POSIX_TCP ON) I need that whole line. We should address this with #75

#define DEFINE_FEDERATED_OUTPUT_CONNECTION(ConnectionName, BufferType) \
typedef struct { \
FederatedOutputConnection super; \
BufferType buffer[1]; \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this suffer from the same problem that also action suffered, where values would be overwritten?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, it does not have a TriggerDataQueue, it is only a temporary place to store the value in case it is overwritten with other calls to lf_set

@@ -17,6 +17,10 @@

#include "proto/message.pb.h"

#ifdef MIN
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why this?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because sometimes MIN is already defined, it is the case for Zephyr, so then we undef to avoid errors

memset(&self->receive_thread_stack, 0, TCP_IP_CHANNEL_RECV_THREAD_STACK_SIZE);
if (pthread_attr_init(&self->receive_thread_attr) < 0) {
LF_ERR(NET, "pthread_attr_init failed with %d", errno);
throw("");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add error message

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done!

Copy link
Contributor

Memory usage after merging this PR will be:

Memory Report

action_microstep_test_c

from to increase (%)
text 71299 43455 -39.05
data 960 744 -22.50
bss 480 480 0.00
total 72739 44679 -38.58

action_test_c

from to increase (%)
text 71099 43255 -39.16
data 960 744 -22.50
bss 480 480 0.00
total 72539 44479 -38.68

delayed_conn_test_c

from to increase (%)
text 75610 47900 -36.65
data 952 736 -22.69
bss 480 480 0.00
total 77042 49116 -36.25

event_queue_test_c

from to increase (%)
text 23406 26942 15.11
data 648 712 9.88
bss 320 480 50.00
total 24374 28134 15.43

physical_action_test_c

from to increase (%)
text 72123 44503 -38.30
data 961 761 -20.81
bss 1952 1952 0.00
total 75036 47216 -37.08

port_test_c

from to increase (%)
text 75576 47693 -36.89
data 952 736 -22.69
bss 480 480 0.00
total 77008 48909 -36.49

reaction_queue_test_c

from to increase (%)
text 23083 26619 15.32
data 648 712 9.88
bss 320 480 50.00
total 24051 27811 15.63

shutdown_test_c

from to increase (%)
text 67430 39208 -41.85
data 960 736 -23.33
bss 480 2624 446.67
total 68870 42568 -38.19

startup_test_c

from to increase (%)
text 66746 38524 -42.28
data 960 736 -23.33
bss 480 2400 400.00
total 68186 41660 -38.90

timer_test_c

from to increase (%)
text 66583 38342 -42.41
data 952 728 -23.53
bss 480 480 0.00
total 68015 39550 -41.85

trigger_data_queue_test_c

from to increase (%)
text 19344 22935 18.56
data 648 720 11.11
bss 320 480 50.00
total 20312 24135 18.82

@erlingrj
Copy link
Collaborator Author

@tanneberger I have addressed your comments. CI is failing, but the fixes are in the event-allocater branch. I am merging this now and planning on fixing everything with the event-allocator branch

@erlingrj erlingrj merged commit fa23939 into main Oct 22, 2024
0 of 2 checks passed
@erlingrj erlingrj deleted the basic-federated-zephyr branch October 25, 2024 23:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants