Skip to content

Commit

Permalink
[cppcon23] update slides
Browse files Browse the repository at this point in the history
  • Loading branch information
@ghorbanzade
ghorbanzade committed Oct 1, 2023
1 parent 2a303e9 commit 473e849
Show file tree
Hide file tree
Showing 4 changed files with 226 additions and 77 deletions.
8 changes: 4 additions & 4 deletions slides/cppcon23/components/Agenda.vue
View file
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
<script setup lang="ts">
const props = defineProps({ section: String });
const sections = {
intro: 'What is continuous regression testing?',
demo: 'How does regression testing work in practice?',
tooling: 'How to build a regression testing system?',
practice: 'How to use regression testing effectively?',
intro: 'What is continuous regression testing',
demo: 'How does regression testing work in practice',
tooling: 'How to build a regression testing system',
beyond: 'Going beyond finding behavioral regressions',
practice: 'How to use regression testing effectively',
safety: 'Establishing a culture of safety at scale'
};
</script>
Expand Down
210 changes: 137 additions & 73 deletions slides/cppcon23/pages/03_tooling.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -240,6 +240,22 @@ struct serializer {
};
```
<!--
I mentioned that all data passed to the value parameter of data capturing
functions are perfect forwarded to a serializer class template. This way,
we can use partial template specialization, a language feature available
in C++11, to convert any given type to a `data_point` variant that we know
how to handle and store efficiently.
We chose partial template specialization because the library can provide
specializations for common data types, and allow users to introduce further
specializations for their own user-defined types.
By disabling the primary template, as shown here, we ensure that the library
can generate easy to read and understand compile-time errors when a user
passes a data type that we don't know how to convert to a `data_point`.
-->
---
section: Data Serialization
slide: Specializing Standard Types
Expand Down Expand Up @@ -290,9 +306,19 @@ enum class internal_type : std::uint8_t {
</div>
</div>

<!--
Here's an example of how the library supports capturing signed integers.
We add a template specialization for all non-boolean types that are integral
and signed. We implement this specialization by copying the integer value into
our `data_point` variant.
Internally, this variant supports the core types listed on the right side of
the screen, where could you imagine that object and array are implemented
to store a map and a vector of other data_points, respectively.
-->

---
section: Data Serialization
slide: Specializing User-Defined Types - Interface
slide: Specializing User-Defined Types
---

```cpp
Expand All @@ -307,30 +333,19 @@ struct serializer<Date> {
};
```
---
section: Data Serialization
slide: Serializing User-Defined Types - Implementation
---
```cpp {1-3,14-17|5-10} {lines:true}
class object final {
public:
explicit object(std::string arg_name) : name(std::move(arg_name)), _v() {}
template <typename T>
object& add(std::string&& key, T&& value) {
using type = typename std::remove_cv<typename std::remove_reference<T>::type>::type;
_v.emplace(std::move(key), serializer<type>().serialize(std::forward<T>(value)));
return *this;
}
<!--
We expose object and array to make it convenient and elegant for users to
add specializations for their custom user-defined types.
/** ... */
Here's an example of a specialization for a Date object, serialized by
its member variables, year, month, and day. The add function here, is
implemented similar to the high-level touca::check, perfect forwarding the
value parameter to the serializer function, and returning a reference to this
to enable chaining.
private:
std::string name;
std::map<std::string, data_point> _v;
}
```
The outcome of this design is an implementation that is as expressive as we
can get with partial template specialization.
-->
---
section: Data Serialization
Expand All @@ -347,6 +362,17 @@ slide: Deeper Dive
</div>
</div>
<!--
But partial template specialization is not the only language feature that
enables designing extension points. Two years ago, at this conference, I
presented a practical overview of other approaches to creating extensible
libraries, including using concepts, ADL, and static reflection. And I tried
to show how the language is evolving towards simplicity and facilitating the
expression of intent. If you are interested in data serialization and like to
learn more about the implementation of the serialization system we just
reviewed, I hope that you find that talk insightful.
-->
---
section: Data Submission
slide: Low-Level API
Expand Down Expand Up @@ -378,7 +404,7 @@ Post::Status ClientImpl::post() const {
```

</div>
<div class="row-span-2 wsl-code-text-xs wsl-code-p-sm wsl-code-h-full">
<div v-click class="row-span-2 wsl-code-text-xs wsl-code-p-sm wsl-code-h-full">

```cpp
int main() {
Expand All @@ -402,6 +428,32 @@ int main() {
</div>
</div>

<!--
Let's now move on to data submission; when we've collected all our data points
of interest and want to submit them to the server.

On the top left code snippet on this slide is our test code from a few slides
ago that is using our high-level test framework that abstracts away certain
operations like error handling, progress reporting, and test results submission.
When our test workflow is fully executed for a given test case, the framework
submits all the captured data and flushes our singleton to free-up the memory
from data from that last executed test.

(click)

But the SDK is designed to make the use of this framework optional. If we are
not happy with this test framework or want to integrate Touca with an
existing test framework, we can take matters in our own hand and use the
lower-level API as shown in the right code snippet. In fact, the high-level
relies on this lower-level API to implement its features.

(click)

Now data submission is handled by the touca::post function that is implemented
to serialize all our captured data_point objects and submit them as a single
binary to the Touca server.
-->

---
section: Data Submission
slide: FlatBuffers Schema
Expand Down Expand Up @@ -452,55 +504,23 @@ root_type Messages;
</div>
</div>

---
section: Data Submission
slide: Serializing Test Cases
---

```cpp {|4-7}
std::vector<uint8_t> Testcase::serialize(const std::vector<Testcase>& testcases) {
flatbuffers::FlatBufferBuilder builder;
std::vector<flatbuffers::Offset<fbs::MessageBuffer>> messageBuffers;
for (const auto& tc : testcases) {
const auto& out = tc.flatbuffers();
messageBuffers.push_back(fbs::CreateMessageBufferDirect(builder, &out));
}
const auto& messages = fbs::CreateMessagesDirect(builder, &messageBuffers);
builder.Finish(messages);
const auto& ptr = builder.GetBufferPointer();
return {ptr, ptr + builder.GetSize()};
}
```
---
section: Data Submission
slide: Serializing Data Points
---
<div class="space-y-2 wsl-code-text-sm wsl-code-p-sm">
```cpp {|6}
std::vector<uint8_t> Testcase::flatbuffers() const {
/** ... */
std::vector<flatbuffers::Offset<fbs::Result>> fbsResultEntries;
for (const auto& result : _resultsMap) {
const auto& key = result.first.c_str();
const auto& value = result.second.val.serialize(builder);
fbsResultEntries.push_back(fbs::CreateResultDirect(builder, key, value));
}
/** ... */
}
```

```cpp {|3}
flatbuffers::Offset<fbs::TypeWrapper> data_point::serialize(
flatbuffers::FlatBufferBuilder& builder) const {
return touca::detail::visit(data_point_serializer_visitor(builder), _value);
}
)
```

</div>
<!--
We use FlatBuffers to perform this serialization, in a way that preserves the
type of our data points so that the server can compare them in their original
data types.

Here's the FlatBuffers schema used by Touca, where each captured data point is
represented by a Result object, and each performance data is serialized into
a Metric. This way, we can store all the collected data for each test case
in a Message object. We serialize each Message individually, then package our
messages into a Messages object which is then serialized and submitted to the
server.

This double packaging may feel excessive at first but it enables saving network
IO on the client side and, more importantly, it paves the way for much faster
data ingestion on the server side as the server can deserialize our top-level
Messages object but defer deserializing each Message to its workers.
-->

---
slide: Data ingestion w/ async processing
Expand All @@ -517,6 +537,27 @@ slide: Data ingestion w/ async processing
</LightOrDark>
</div>

<!--
Here's how the server handles an incoming submission when a test is run in
CI or on a decided test server. In these environments, the test framework
submits the data for a given test case and moves on to execute the next test
case. It doesn't wait for the server to compare test results because we're not
looking to fail the CI if we find a mismatch.

when the server receives the data, it deserializes the high-level Messages
object, and stores the binary content representing each Message into the data
store. Then it creates a job for each message to hand off the deserialization
and processing to Server Workers. This is when the server can acknowledge
receipt of the submitted data so that the test framework can move on to submit
next messages.

But the real job of processing messages is just getting started. The server
workers deserialize captured data, retrieve data from the base version of our
workflow, and compare the two with each other. This way, the next time the
web dashboard inquires about the status of the submitted messages, we can
show the results as compared against the baseline version.
-->

---
slide: Data ingestion w/ on-demand processing
---
Expand All @@ -532,6 +573,16 @@ slide: Data ingestion w/ on-demand processing
</LightOrDark>
</div>

<!--
Now much of this workflow is the same when we run our tests locally, either
using the CLI or by directly calling our test executable. The only exception is
that, in this case, we want the output of the test to report whether our
test results match that of the baseline version.

So instead of returning 204 after data ingestion, we wait until the messages
are compared and then report the status of the comparison result.
-->

---
slide: Data Retention
---
Expand All @@ -556,3 +607,16 @@ slide: Data Retention
</div>
</div>
</div>

<!--
Now remote storage of test results and comparison results is very convenient
but storage has a cost and when testing real-world systems at scale, we may
want to decide when to remove older test results.

To meet this need, Touca server includes a data retention service that is
configurable and helps remove older submitted data or move them to longer-term
archival solutions.

Touca also allows storing a local copy of captured data on the filesystem
in a format that can later be used to restore that data back into the server.
-->
80 changes: 80 additions & 0 deletions slides/cppcon23/pages/07_appendix.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
---
layout: center
---

Appendix

---
section: Data Serialization
slide: Serializing User-Defined Types - Implementation
---

```cpp {1-3,14-17|5-10} {lines:true}
class object final {
public:
explicit object(std::string arg_name) : name(std::move(arg_name)), _v() {}

template <typename T>
object& add(std::string&& key, T&& value) {
using type = typename std::remove_cv<typename std::remove_reference<T>::type>::type;
_v.emplace(std::move(key), serializer<type>().serialize(std::forward<T>(value)));
return *this;
}

/** ... */

private:
std::string name;
std::map<std::string, data_point> _v;
}
```
---
section: Data Submission
slide: Serializing Test Cases
---
```cpp {|4-7}
std::vector<uint8_t> Testcase::serialize(const std::vector<Testcase>& testcases) {
flatbuffers::FlatBufferBuilder builder;
std::vector<flatbuffers::Offset<fbs::MessageBuffer>> messageBuffers;
for (const auto& tc : testcases) {
const auto& out = tc.flatbuffers();
messageBuffers.push_back(fbs::CreateMessageBufferDirect(builder, &out));
}
const auto& messages = fbs::CreateMessagesDirect(builder, &messageBuffers);
builder.Finish(messages);
const auto& ptr = builder.GetBufferPointer();
return {ptr, ptr + builder.GetSize()};
}
```

---
section: Data Submission
slide: Serializing Data Points
---

<div class="space-y-2 wsl-code-text-sm wsl-code-p-sm">

```cpp {|6}
std::vector<uint8_t> Testcase::flatbuffers() const {
/** ... */
std::vector<flatbuffers::Offset<fbs::Result>> fbsResultEntries;
for (const auto& result : _resultsMap) {
const auto& key = result.first.c_str();
const auto& value = result.second.val.serialize(builder);
fbsResultEntries.push_back(fbs::CreateResultDirect(builder, key, value));
}
/** ... */
}
```

```cpp {|3}
flatbuffers::Offset<fbs::TypeWrapper> data_point::serialize(
flatbuffers::FlatBufferBuilder& builder) const {
return touca::detail::visit(data_point_serializer_visitor(builder), _value);
}
)
```

</div>
5 changes: 5 additions & 0 deletions slides/cppcon23/slides.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -89,3 +89,8 @@ layout: center
</div>
</div>
</div>

---
src: ./pages/07_appendix.md
sectionDefault: ''
---

0 comments on commit 473e849

Please sign in to comment.