This is a transcript of What's Up With That Episode 12, a 2024 video discussion between Sharon (yangsharon@chromium.org) and Peter (pkasting@chromium.org).
The transcript was automatically generated by speech-to-text software. It may contain minor errors.
Base is one of the lowest level directories in Chromium. What's in there? What should you be using? Why don't we just use the C++ standard library?
Notes:
Links:
00:00 SHARON: Hello, and welcome to "What's Up With That?," the series that demystifies all things Chrome. I'm your host, Sharon, and today we're talking about //base. What is it the base of? What are Chromium-specific types we use? How does it fit in with C++ at large? Today's special guest answering all of that and more is Peter. He's our newest //base owner, a longtime team member, and a driver behind style guide changes, C++ future allowances, and updating to new versions of C++. He has a series, C++ 201, on this channel that helped inspire this series. So welcome, Peter.
00:33 PETER: Thank you.
00:33 SHARON: So what is //base?
00:39 PETER: It's one of our lowest level directories, and it pretty much has all the low-level stuff everything else depends on.
00:45 SHARON: Sounds important. So can you tell us a bit more about what those specific things are and why they're important?
00:51 PETER: Yeah. So stuff goes in //base if it's broadly useful to lots of different unrelated places and if it's semantically fundamental, like maybe it would go in the STL, at least if it were broadly useful to C++, and if you can't move it to a better, more specific subdirectory. Anything that fits those is probably a good candidate.
01:17 SHARON: OK, so what are things that do versus don't belong in //base, then?
01:24 PETER: If it's something that is maybe useful for programming in general but we don't need it in Chrome specifically, then it doesn't go in //base. We're not trying to make a toolkit for general-purpose use outside Chrome. If it's just speculative-- you came up with a cool idea and you're hoping somebody might use it-- then probably don't put it in //base, at least until you can actually make various things use it. And also, it shouldn't overlap with anything that's already in the STL or absl or elsewhere in //base, at least, unless we have some clear guidance on, here's how you pick which alternative to use, and ideally, some tooling doing that too.
02:03 SHARON: Yeah, if you look around the Chromium codebase, you see base types, you see absl types, and you see standard library types. So when do we use each of these libraries? Because certain things-- if you were around before we used to use base optional. Now we use absl optional. Or do we use std optional now?
02:22 PETER: We do use std optional.
02:22 SHARON: So that changes, and that has changed over time. So when do we use which ones?
02:28 PETER: Yeah, we've used all three of those, actually. Generally, all else
being equal, we will prefer the STL if it's available and absl if it's there
and //base if it's not in either one. So with optional, we had an optional
before the STL had it because it got it in C++ 17, and we had optional long
before that. And then when we allowed absl, we switched to absl optional
because it was there. And then when we allowed C++ 17, we switched over to
that. So that's the most common thing. And then //base, in that case-- just be
used for stuff that supplements the STL. Things that only Chrome needs-- maybe
they're not usable by all C++ plus everywhere in the world, or we're
polyfilling something that we're hoping is in an upcoming standard. So we can
move faster than maybe the upstream library can because we only need to worry
about us. Some stuff supersedes things in the STL or absl. If we have problems
with particular APIs and we need to work around things or ban things-- like
bit_cast-- we have our own version of bit_cast that warns you if you're
doing something silly. If we're trying to integrate tightly with something in
the library to do more than what's in the spec, like we have our own version of
span, and that's partly because we want to integrate tighter lifetime checks.
03:59 SHARON: Yeah, we'll get into all the different types we have in //base in a bit. So what's absl? Because that's something that's newish to Chrome and didn't always exist there. So why did we start using it? What is it? Tell us more.
04:12 PETER: Yep. absl came out of other teams at Google. Some of the internal code in Google's-- you could think of it as Google's version of //base, internally. And they said, hey, this is broadly useful. We'd like to make it available to the open-source world. And they spent a lot of time, and they released this thing called absl. And we allowed that-- I don't remember-- I think around 2016 or something-- 2017? And I drove that process, in part because there was lots of useful stuff there. I think absl had variant at the time. And that was in an upcoming STL, and a lot of our code could have used variant. So I allowed us to use absl after getting all the necessary sign-offs. And you can think of it as the Google bits of Google's //base that they thought would be useful to the world.
05:14 SHARON: OK. Can you tell us a bit about the provenance of //base? Where did it come from? The name, I guess, is fairly self-explanatory, but a bit about the history of how we got here.
05:26 PETER: Yeah, Chrome started development in mid 2006, and we built things
internally in Google. So we used a lot of pieces of the same sorts of places
that absl itself came from, like core utilities in //base. And then once we
split off to our own repository-- and this is even before public launch-- we
couldn't depend on the rest of Google anymore. So we copied a few bits over. We
copied over, for example, a type called string_piece, which later became
proposed to the STL and became string_view. So we had an equivalent to
string_view back then. And we copied a few other bits. And then since then,
it's just been added to ad hoc.
06:06 SHARON: OK. Who owns //base now? Because everyone uses it, but there isn't a team that's just a dedicated //base team. So who is out here making sure that the fundamentals still work?
06:20 PETER: Yeah, there is no formal core //base team. There's currently an OWNERS file with 11 different people in it. Those folks all have other particular areas on the team that they nominally are in. Like, Dana is in the security side of things. And I work on the UI side of things. And to some degree, we bring those hats to //base, so we contribute to areas where we have particular expertise. A lot of it is just self-driven. People tend to gravitate there when they have an interest in mucking with really low-level C++, doing core tooling and API fixes. And so it's very much like any other directory in the codebase. Anyone is free to touch it. And if you want to become an owner of it, then you can get to touch it lots.
07:19 SHARON: [LAUGHS] Great. Yeah, as someone who is not a hardcore C++ person, I am very glad there are people who are and keep things running. So //base exists in the Chromium source directory. Where can you use it? Presumably everywhere within that. But can you tell us about where you can and can't use //base?
07:41 PETER: Yeah, mostly everywhere, although there are a few gotchas. So there are particular tiny pieces of //base that can't use the rest of //base for reasons. There's also a few pieces of our core code that we want to make sure don't depend on //base, like down deep in the installer or the Sandbox, like code that is extremely low-level, early stuff that needs to run without anything else being loaded. Otherwise, in Chrome, you can pretty much freely use things with a few exceptions. As you mentioned, stuff outside Chrome-- so most stuff in third party-- can't use //base. Interestingly, that sometimes includes first-party-ish code. The biggest thing that people get caught out by is probably Blink. Code in Blink can't just use any part of //base it wants. There's actually an allow-list that a Python script audits. And the main reason for that is that Blink uses a memory allocation thing called Oilpan, and it needs to make sure that all the APIs are safe to use with that. So that's why we have that. But this also means that things that don't live in the Chrome repository-- so V8 or Crashpad, our crash core utilities-- don't have //base. So often, they'll have their own forks or small copies or things like that.
09:06 SHARON: OK. Yeah. I'm sure if you work in any of those areas, you're pretty familiar with what you can and can't use. So let's get into the fun part and do a run-through of what exists in //base. So obviously, we're not going to cover all of it. It is a huge directory. But there are some types you see more than others, so let's run through some of them.
09:23 PETER: Yeah, this is the rare case when I actually think, because the
stuff in //base is so broadly used and so useful, it is worth team members'
time to go through the list themselves and just randomly scan down the files in
each directory and be like, oh, what's that, and look into it. I wouldn't
normally say that. Chrome is like millions of lines of code and I don't even
know how many files. But probably worth doing it at some point in base. But
yes, some things that I found when I was doing this, because I don't have all
this committed to memory-- PartitionAlloc. So if people have heard of
MiraclePtr, or even if not, Chrome has its own allocator called PartitionAlloc.
We use it pretty much everywhere. It's actually kind of its own standalone
project at this point. It's one of those pieces in //base that can't depend on
the rest of //base. But it lives inside //base. And if you have ever seen the
raw_ptr or raw_ref types-- which, at this point, it's hard to have worked
in Chromium and not seen raw_ptr somewhere-- then that lives in there. We
have expected. So this is basically a polyfill of the STL's std::expected
from C++ 23. So expected is a type that's mostly used as a return type from
functions. And it basically is two things. It's a value type for when your
function worked, which might be void if your function doesn't need to actually
return a value and you just wanted it to do some stuff, or else it's an error
type if the function failed, and then it can hold details. So you can think of
it like a special-purpose variant of two types, with some helpers and stuff
like that.
11:07 SHARON: What about optional, which seems related to that?
11:14 PETER: So optional and expected both are useful for functions that can succeed or fail. Really, the difference is, semantically, that optional is usually a value if it succeeds or else no value. So it tells you whether it succeeded, and it tells you what the value is if so. It doesn't tell you anything on failure, and it's not very useful for the case where success doesn't mean returning anything. So if it's a getter-- hey, give me this thing-- and it fails, then obviously, you return nothing. But if it's not a getter-- it's like, go paint some stuff on the screen, then the only return value is, yes, I painted it. So the fact that expected can return void in the success case makes it useful for that. And then expected can return details on errors. So that makes it more useful for when you want to pass more information back or handle failure or something. Not necessary if you actually-- there's no information needed on failure. Yeah, it didn't work. We don't care why. In that case, maybe optional is still a good idea.
12:24 SHARON: OK. Up next we have flat_map, flat_set. That feels like a
//base classic. It's one of the first things, when I was doing code reviews-- I
was like, oh, you could use a flat_map here.
12:32 PETER: Yeah, so this is another polyfill. flat_map and flat_set are
actually in C++ 23 also.
12:43 SHARON: OK. Which version of C++ is Chromium using now?
12:43 PETER: Chrome is currently on C++ 20. Basically, flat_map and
flat_set are-- let me-- maps and sets or associative containers, I might call
them, generally store things in a tree-like structure. So that gives them all
their properties of you can find or insert things in logarithmic time, and
removal is cheap and lots of other stuff. The problem with trees is that
because, in memory, they're just pointers to different areas of the heap, this
has really bad performance in terms of the constant factors. Because,
basically, every time you traverse to a new element, you're loading a new cache
line. So flat_map and flat_set store all their data in a contiguous block.
It's basically a sorted vector. And on the face of it, you'd think this would
be bad because sorted vectors, if you remove an element, you have to shift all
the other elements. And now it's linear time. And that's true. And for very
large maps and sets, map and set are way better than flat_map and flat_set
because big O is important. But for small ones, like the cache-line effects
dominate and flat_map-- is actually much more performant.
14:01 SHARON: What's a good heuristic for "big" or "small" in this case? It's one of those, like, this is Google. I forgot how to count that low.
14:07 PETER: We have a doc with more guidance on how to pick which type you
want. So if you look at base/containers/README.md, there's some information
there about, how big are these things, and how do I know what stuff to put in
it. My general mental thought is most code I run into is kind of two buckets.
Either it's like a dozen things or a thousand things. And those fit pretty well
into-- a dozen is a flat_map and a thousand is a map. If you're somewhere in
between-- you have a hundred things-- it's probably more iffy. Do some
benchmarking. But the other thing is performance is not always the concern
everywhere. I mean, performance matters, but if your code is not hot, then
binary size might matter more. Or just semantics or not having to change types
at an API boundary or whatever.
15:04 SHARON: Yeah, you and your code reviewer can go dig that out. Cool. What about FilePath?
15:09 PETER: FilePath. So as people who have worked across operating systems know, operating systems unfortunately choose to do things differently. Windows has to use a back slash instead of a forward slash because they like CP/M. And so we need abstractions to do things parse file paths or construct file paths. More subtly, the different operating systems also use different encodings for their file paths. Mac is UTF-8, which seems sane to me. Windows is UTF-16, which is kind of unfortunate, but historically sane. And then Linux is actually-- you don't know because Linux is literally whatever encoding was used by whoever wrote the file. And it could be anything, and you just use UTF-8 and hope for the best, which is what we do. And it mostly works. I don't know. There's scary comments in there, and I'm like, I don't want to touch this.
16:10 SHARON: Yeah. Some of the Linux stuff-- it's like, who owns this? And it's like--
16:15 PETER: And this is another one where there are STL utilities. So in this case, std::filesystem exists. I think that's C++ 17, off the top of my head. I'm less familiar because we ban it. And the reason we ban it is that the Google style guide bans it. We didn't actually make this call. And they banned it over security concerns, some testing concerns. I don't remember. Titus Winters, who was deeply involved with the C++ working group, had a lot of internal commentary on why he didn't think it was appropriate for Google use, and we didn't think Chrome differed enough from Google to make an exception.
16:58 SHARON: So a brief sidebar about style guides-- so there's a Google-wide style guide. And then there's a Chromium-specific style guide. So where do these things differ?
17:04 PETER: It's interesting. I actually just wrote a formal policy on that we have approved, but not yet written down in the tree. And basically, the formal answer is we differ whenever the consensus of the cxx mailing list says we differ. And cxx, by the way, is a mailing list that anybody in Chrome is welcome to join. It's a moderated list for non-members, primarily just to reduce spam. We're not actually trying to keep anybody out. There's no expertise bar to join it. You don't have to take a test and prove your C++ Foo. But that's where things get discussed, like, should we allow x, y, and z? For Googlers, you might be familiar with C-style internally. This is very much the Chromium version of the C-style mailing list. And mostly, our rule is, yeah, we do what Google style does, except in cases where there's a good Chrome reason to differ. And there are sometimes. Google doesn't ship to client machines, so it doesn't care about the size of updates or how much space it takes on the hard disk. So they don't care about binary size. The way that we do, especially on mobile platforms. They have different kinds of security concerns. They're concerned about the sorts of security holes that you could get on a server-side app. We're concerned about the sorts of security holes you could get from an attacker on the web. Those overlap. They're not quite the same. So there's a lot of subtleties why we might make a different call, but it means that in general, Chrome style is Google style with this set of changes to it that we document in our style guide.
18:43 SHARON: All right.
18:43 PETER: And yeah, get on the mailing list if you would like to kibitz and tell us that we're all idiots.
18:50 SHARON: [LAUGHS] I'm sure, yeah, everyone would love that. Cool. The next on our list is NoDestructor.
18:56 PETER: So, yes, this is one of the bazillion "I have a Singleton-ish sort of thing" types. So we have lazy instance. We have Singleton. We have NoDestructor. The rule is basically ignore all the others and use NoDestructor. The other things are old. They're deprecated. Don't use them. NoDestructor pretty much just tells the compiler, leak this, at program shutdown. Doesn't try to run it. And that has two nice effects. One is it prevents what you might call the destruction order fiasco, which is you're in the middle of tearing things down, and you don't know whether you can rely on other things that are also getting torn down. So who puts their gun down first, kind of thing. If nobody has to be destroyed, then you just don't care. And then the other thing that's nice is it just does less work on shutdown because really, at shutdown, ideally, what we want to do is two things-- write all the important data to disk. And then just kill the process. Like, die as fast as possible. Don't do anything else. Once all your data is written, you don't care. Tell the OS to wipe you off the system. So NoDestructor sort of helps get closer to that world.
20:10 SHARON: On a similar note, RAII Scope? Scoper?
20:10 PETER: Right. Yeah, anything for scoping stuff. So RAII, for people who haven't encountered that acronym, stands for Resource-- excuse me. Resource Allocation Is Initialization. And it really just means using C++'s objects and lifetimes in order to get stuff guaranteed to happen when something goes out of scope. So converting some manual calls of, do this on Enter; do this on Exit, into the lifetime of an object that handles those for you. And so we have things like AutoLock, which takes and releases a lock for you. Anything with "scoped" in the name-- Scoped Observation adds and removes an observer for you. AutoReset is actually trivia, the first thing I ever contributed to //base, at least that I recall. So I looked it up the other day. It dates back to 2009. It was called Scoped Bool at that point because it only did bools, and now it does everything. I was scared of templates in 2009 and did not know how to use them, so I only made it work with bools because templates. I know more about templates now, and my feeling then was totally justified. It was completely correct. But yes. So pretty much, anything you find in //base that has Auto or Scoped at the front of the name is some kind of scoping helper.
21:39 SHARON: OK. Yeah, I don't see AutoReset used a ton. Can you briefly mention how that works?
21:45 PETER: So AutoReset is just, set a value. And when I go out of scope, reset back to the old one. So you can do this, for example, if you want-- if your class has a member that is-- hey, I am inside the blah, blah, blah function right now, and therefore, you should allow or not allow this functionality to happen. A few classes have to have-- it's kind of hacky-- but something like that in their design. So you can use an AutoReset around setting that member within the particular scope that it's supposed to be set in. You can use AutoResets in tests, too. Say, OK, within this scope, make this variable be this. And then just automatically, when it goes out of scope, you put it back.
22:28 SHARON: Yeah, that's a handy one. Cool. In terms of template stuff maybe, span?
22:37 PETER: Yeah, so span is-- people might see span used a lot more lately because the security folks have been driving a process called spanification. And their goal with this is pretty much to get rid of all pointer arithmetic in the entire project. So span is something we call a view type. And I would actually to write a talk on view types at some point because I think they're less understood than they should be because they're very useful. But basically, a view type is some kind of a window into a block of contiguous objects that it doesn't own. It just says, here's some memory. Some things live there. You can think of this like a pointer because since arrays decay to pointers-- for example, an array that you have decayed to a pointer literally is a view onto that memory. A span is very similar to that, except that it also carries a size along with it, so it remembers how big it was. So in that sense, similar to std::array. But unlike stdarray, it's not managing the memory directly. It's just saying, no, the memory lives out somewhere else. So maybe it's in a vector. Maybe it's in an array. Maybe it's somewhere in the binary or it's on the heap. We don't actually care. Here's the pointer to it. And here's the size. And that's good because the fact that you have a size means all your APIs can bound check. So you can either handle things nicely, or you can crash the process on security holes or something. And you can also-- wow, my brain just completely died. Oh, yes, You can split spans or do other helpful things with them. Grab the first [INAUDIBLE] elements of the span, convert to other span types, things like that. So we provide those APIs. And that's what having a dedicated type for that gives you.
24:42 SHARON: This is maybe a bit of a silly question, but how do you write stuff in that memory, then, if--
24:47 PETER: So, since the span is just a window, it can either be a window of writeable or non-writable types. So if you have a span of int, it just means here's some ints, and you can read and write them. If you have a span of const int, that means that it's read only.
25:00 SHARON: OK, cool. So what about strong alias ID or type? There's a lot of types and whatnot. So--
25:13 PETER: Yeah, I don't see these used a ton, but they're occasionally useful. Basically, strong alias and ID type, which is a special case of strong alias, is a generalization of the idea of an enum class. So it's, I have this type that is implemented in terms of this underlying type. But semantically, there are different things, and you shouldn't conflate them. So just like you might have an enum class that it's really an int and its values are ints but don't just pass it to a function that takes ints. You want the compiler to yell at you because that's a different meaning. Strong alias is a way of basically saying, hey, I have an anything. You can do this for even non-numeric types. You can just say, it's really a this thing over here in the implementation, but the semantics are different.
26:10 SHARON: Right. I think you see it a lot for ID types of different classes, so you don't mix up what you're identifying with those numbers.
26:17 PETER: Right. Yeah, I mean, any kind of identifier runs this risk of type confusion. And if you can just get away with using an enum class directly, that's probably fine, too. But sometimes you can't.
26:30 SHARON: OK. All right. Next on our list is synchronization types, lock or available event, some examples of that?
26:37 PETER: So there's a whole //base synchronization folder. And there's lots of things in it. Lock is probably the most commonly used thing. This is a mutex if you need to do thread-safe stuff in Chrome. And actually a word about thread safety in Chrome, because people have asked this before-- if the code in Chrome doesn't state otherwise, it is not thread-safe. It's assumed to all be running on a single thread. It's probably assumed to be running on the primary thread. The thing we call in the browser process the UI thread. Obviously, in some directories that doesn't hold, and the whole directory will say something. But normally, thread-safe code-- code used across threads is the exception, not the norm. But where it is used-- lock is our mutex type. It's very much like std::mutex. In the case of //base synchronization APIs, it's really just a case of, we had all our own stuff and used it all well in advance of C++ 11 adding it all to the STL. And then we could migrate. It's easy to migrate. And I said earlier we like to use stuff from the STL when we can. So I should put the caveats on that. It's easy to do that when either of the following is true. One is the STL provides the exact same semantics. And the cost is there's no reason not to do it. And two is there's a huge win. We get way better perf or better integration with something in the STL, or better safety or something like that. And in the case of the STL synchronization stuff, migration is scary because any difference or bug in the implementation of the STL that we use for those is going to be very bad and hard to track because it's all really low-level, cryptic cross-thread stuff. And then we don't actually know if there is any gain. There's an open bug on performance testing some of these things against each other. But because migration is so scary, no one has bothered. Free opportunity. If you want, go perf test it. Find a big difference. And then migrate. And that's probably cool. Of course, if you perf test it and don't find any difference, then you just wasted your time. Ha-ha.
29:06 SHARON: [LAUGHS] RE the thread type-- so the most common ones you see, at least in the browser process, are the I/O and UI threads. So can you give us a quick rundown of what these two are and how they're different?
29:17 PETER: Yeah, so in the browser process, there's a whole bunch of different threads, but the two you mentioned are the two most common ones. The UI thread is the main thread of the process, and we call it the UI thread, in part because all of the actual UI interaction-- event handling, painting, et cetera-- is done on that thread. We don't do most of that off-process. We do do compositing off thread and things like that. But if you write code in views and it goes and paints pixels, then that code will be running on the UI thread. The I/O thread is more confusingly named because at first glance, a lot of engineers assume that means that's the thread where we do reads and writes to disk. And it actually doesn't mean that. It's interaction between the different threads of the browser. So the I/O thread is more like the coordinator thread, where it's responsible for communicating between browser and renderer processes, or between the network stack and different things. So the I/O thread actually never does-- it's not supposed to touch disk at all because disk blocks, which is why you do it off thread to begin with. And the I/O thread, since it's coordinating all of the other things, needs to be very responsive. So in fact, there's other places-- there used to be something called the file thread. I can't remember if it still exists. We now have the thread pool that you can use to do some of these longer running blocking tasks. And then they'll probably communicate their results back directly, but if not, they'll use the I/O thread to coordinate that.
30:59 SHARON: OK. All right. Back to our //base walkthrough. So up next is Time.
31:04 PETER: Yeah, so another bit of trivia here. Before I was a //base owner, which-- I've only been a //base owner since March of this year, 2024-- but I have been a //base time owner for many years. So there are more than 11 //base OWNERS if you start counting the OWNERS of various subdirectories. And time has lots of things in it. But the three core classes, which are all in time.h, are Time, TimeTicks and TimeDelta. TimeDelta is easy because TimeDelta is just the difference between two Times or two TimeTicks. It's constexpr, and it's type-safe. And this means that if you want to store a value 100 milliseconds, even at compile time, even as a constant at the top of your .cc file, do not do, int blah, blah underscore MS equals 100. That's not type-safe. So it's very easy to accidentally add that to some value with different units. So yes, TimeDelta is type-safe. So if you say, auto, blah, blah equals base milliseconds 100, then that's 100 milliseconds. And it not only says what it is in the code, but you can't add it to the wrong units. The compiler will check you. And it doesn't-- It's not expensive. It's compiled into the binary. So that's great. Time versus TimeTicks is more subtle. These represent two slightly different versions of What Time is It? So Time is like a wall clock, human readable time. So a time like 3:57 PM, this time zone, on this date-- that's a time. And human readable makes it really good for messages to humans or saying, this happened at this point, but really bad for doing calculations with because human times are messy. They skip forward in daylight savings things. They also skip backwards in daylight savings things. There's leap seconds. There's all sorts of complexities. So trying to find out the difference between two times is not as easy as it might seem. So then for that, we have TimeTicks, which is based on a monotonically increasing counter that runs while the process is running. And that's much more useful for saying, OK, 15 seconds from now, I want this to happen, or something like that. There are still even gotchas with that because what does that clock do if the user puts their machine to sleep? Does it keep running or not? And so there's a lot of commentary in the code about what you do and when, et cetera. But fundamentally, that's the difference between those.
33:44 SHARON: So in the stuff I've looked at, I haven't seen too much use of Time. Where is heavy usage of all this Time stuff?
33:57 PETER: There is stuff in a number of different places. So for example, the media code needs heavy usage of Time because it needs to know when to schedule things. The network code might use Time for computing rates. So if it's like OK, I'm going to bandwidth-limit this, or I need to know how fast the user is downloading. A lot of UI code needs to use time to display various things to users. For example, the scheduled and update-- critical update for Chrome. You need to restart your machine within 30 minutes type of thing-- needs to use Times. And then we use Times a lot when time stamping things that come in from sync or that we save to disk. Cases like that, we often need to know, is this sufficiently out of date? Do we need to go get a new one?
34:42 SHARON: OK, sounds good. All right. Next on our exploration is value in //base.
34:55 PETER: Yeah, //base value-- I think subject to the longest running
code-health migration thing. We had a code-health thing going for //base value
for. I don't even know how many years to migrate APIs. And actually,
ironically, I have almost never used //base value. So I always thought like,
why are we spending so much time doing this? But //base value is basically a
C++ class that abstracts, what kinds of values can you store in JSON? And JSON
matters because JSON is how we store all of our preferences. It's also a good
abstraction for values that come to and from JavaScript. But all that is
handled differently. Like, V8 and Blink worry more about that. And usually, by
the time you get to stuff in //base, they have dealt with those sorts of things
already. So mostly, where you'll see value is when you're going to and from the
pref store. Preferences are the backing abstraction, also, for sync. So
anything that's synced-- you'll probably go through //base value. That gives it
some things that, to a C++-only programmer, would be odd. For example, that you
can store a double in it, but you can't store an int64_t. And that makes
sense if you think in terms of JSON doesn't have the concept of a 64-bit int.
So that's why //base value models that. But this also means that value is not a
good type to use for a member of your class or a general-purpose thing to pass
around in APIs. Most of the time, you have one specific type. Use that type. If
you have multiple things, use a variant. You only really want to use value at
the boundary level of-- you're serializing to or from some kind of storage that
uses values, and then after that, you put it in its own dedicated type.
36:45 SHARON: OK, cool. All right. Next up, we have numerics. Numbers-- we like those.
36:52 PETER: I got my shirt in here. So yes, the numerics library has a number
of useful things. It has some mathematical constants. Actually, C++ now has a
lot of these. We used to have our own constant for pi and square root of 2 and
things like that that you would need to use a lot. And now C++ 20 has those,
and we use those more widely. But we still have others. And we have some basic
conversion functions like, if you're converting between degrees and radians
don't write the code yourself. Just use our code. Not only is calling a
function more readable than doing the math inline, but it prevents you
accidentally going the wrong way or something like that. More interestingly, we
also have a bunch of safe math libraries, so we have math operators and types
that will either clamp out-of-range calculations and values, or they will, in
fact, check fail and crash your process when bad stuff happens. And these are
not only useful in the cases you would expect like, oh, I have some data coming
over the network. I should probably check whether the size they want is sane--
that sort of thing. But also cases you might not expect, casts to smaller size
within the code. If you're going to use a static cast, the coder should be able
to tell locally that that's provably safe, like you literally just checked that
the size is less than such and such. So of course, it has to fit. Yeah, in that
case, just use a static_cast. But if it's coming into some function and
you're not guaranteed, don't make people go read it and find out that 13 other
functions later, the transitive closure of x proves that this can be done. Use
a checked_cast. And then even more surprisingly, conversions between integer
and floating point types-- neither one can accurately represent the other. So
you should use the safe math functions. You should not be doing things like
calling std::round and then just casting to an int. That doesn't work.
38:55 SHARON: Right.
38:55 PETER: I have a screed on this that I wrote at one point.
39:00 SHARON: OK. Don't roll your own math. All right. What about other ranges we have in //base? We talked about a couple earlier, I think, but are there more?
39:06 PETER: Well, we talked about some-- we talked about span as a view type. I don't know that we talked about anything with ranges. So //base ranges was a backport of the range-based algorithms in std::ranges, in C++ 20. So these are basically-- everyone hated the old algorithms because they're so cumbersome. You always have to pass, my long vector name dot begin, my long vector name dot end. And this was even more annoying if you actually had to go to the trouble of stuffing something into a temporary just so that you could do that because it was coming from some other function. So the range algorithms provide this surprisingly nice piece of sugar by just letting you take a range-like object directly. And there's lots of complexities to, what is a range-like object? So we had back ported that. They give you a few other nice things. They have projections on all the algorithms, which lets you do some cool stuff without having to manually unwrap stuff. But basically, that's what //base ranges was.
40:16 SHARON: OK, cool. Up next is something we mentioned a bit earlier, but
general string stuff. So you mentioned string_view and whatnot. So are there
other string things in //base to know about?
40:27 PETER: There's conversions between various types of encodings. In
particular, Windows APIs are basically UTF-16. Mac APIs are basically UTF-8.
JavaScript is pretty much UTF-16. POSIX is pretty much UTF-8. There's lots of
disagreement, and therefore, we end up doing this. So //base makes it easy to
do this, although even better than doing an easy conversion is not doing the
conversion. If you can write your APIs or storage such that you actually don't
need to convert, that's better. I have fixed up code where once I trace through
the 10 call chains, I discovered that, actually, we wanted the same type at the
beginning and the end. We just converted back and forth about four times along
the way. So fix that. Don't do that. There's utilities to split and tokenize
strings. Probably not good to write your own HTML parser in this. That's why we
have Blink. But if you're just doing some super trivial thing, we've got that.
And then some of the bigger ones-- we have StringPrintF, which is basically C's
sprintf(), where you want to do a formatted output, but into a string buffer
instead of onto the screen. So we have something like that in StringPrintF,
except that it returns a std::string. It's harder to misuse. It checks a lot of
your format stuff at compile time. I also find it overused. We have cases where
we StringPrintF with a format string that has no substitutions in it at all,
which-- kind of strange. We have things that just concatenate strings, which
could be StrCat. And then in general, lots of stuff is cryptic. And then I just
mentioned StrCat. StrCat is basically a special-purpose function for doing
string concatenation really quickly. You should not just blindly use StrCat for
all concatenation. If you have two strings and you do string plus string,
that's the shortest, most readable, and it turns out, most performant way of
doing it. You should just do that. People have this idea of, oh yeah, string
plus is terribly slow. Don't ever use that. Actually, it's great if you're only
doing it once. If you're doing it over and over and over, it's terrible, but
not because the implementation of plus is bad. It's because that basically is N
squared. You have to potentially resize the string bigger and bigger and
bigger. So StrCat lets you take a whole list of things to concatenate, and it
does it all at once, which means it's linear time to do that. And StrCat also
works very nicely with string_views, so you can mix strings, string_views,
C-style strings, et cetera, which is not possible with things like plus. So
very, very useful function. Underused in my opinion.
43:14 SHARON: OK, go check it out. So you listened to a bunch of all these different platforms that Chrome runs on and a lot of stuff that lives-- because of things like //base, you don't have to really worry about what platform Chrome is running on when you're working on things. For example, in content, we don't really have to worry about this. There's some Android-specific stuff, but that's not because of actual-- that's for other reasons. So how much of the magic that goes into making Chrome run across these different platforms lives in //base, versus somewhere else?
43:54 PETER: Certainly more of it. So things like FilePath have to understand,
fairly directly, the differences between operating systems. We also have a file
in //base called compiler_specific.h, which is a very low-level-- here's a
bunch of macros. And they differ by platform or by compiler. So C++ 20 gained a
new attribute called-- shoot. Is it no_tail_padding? No. It's-- oh,
no_unique_address. Yes, C++ 20 gained something called no_unique_address.
The only reason I mention it is because on Windows, Clang does something
different to match Microsoft. And so compiler_specific abstracts that detail
away and says if you use our macro, then you get the same behavior everywhere.
Things like that. That said, there are still plenty of cases where code outside
//base needs to understand this. Since I work in UI, the examples that come to
mind are in UI, like being a good platform citizen on the different OSes often
means doing different things in terms of what keys do stuff, or how do you
handle different events, or where should the buttons on the OS surfaces be, or
things like that. How do fonts get rendered? Those are all things that wouldn't
be handled in //base because they're higher-level concerns. They're stuff
happening up in a UI layer somewhere. And probably, code in other directories
has to do that kind of thing as well. But certainly, //base will take care of
anything that you might think of as like a Unix versus Windows or POSIX versus
Windows or Mac or something like that. Android versus iOS. API-level
difference. That kind of thing will often be handled more at the //base level.
45:46 SHARON: OK, cool. So re macros-- a bunch of macros live in //base. We previously had another Peter on to talk about DCHECK(). Do we have some updates there, if you want to give us a rundown?
46:00 PETER: So I think your episode with pbos (Peter) was filmed in 2022.
46:09 SHARON: It was a while ago.
46:09 PETER: And then since then, he's been continuing to do hard work. So anything I mention here is pretty much credit to pbos. But we've changed our guidance on some of these things. So the guidance used to be, basically-- we have DCHECK() and CHECK(), and they both kind of mean, this shouldn't happen. Well, this should be true, and crash if it isn't. And the guidance used to pretty much be, use DCHECK() for everything. Except, use CHECK() for things that are security sensitive. And now the guidance is effectively reversed. It's basically, use CHECK() for everything. Only use DCHECK() if this is provably performance-disastrous here. And the big reason for that is we're finding increasingly that a lot of our crashes and security problems in the field come from violating the code's invariants. So we get to somewhere in the code, and the invariant that even had a DCHECK() that said, this shouldn't be true here-- it was violated. So something got goofed up somewhere. And by converting all of these things into CHECK()s more, initially, it risks making the product more crashy. But assuming that we do it in a slow enough way and we fix things quickly as they come up, we eventually get to a state where we're actually enforcing our invariants and not just saying, well, we're pretty sure this is true in production, but we don't want to take the perf hit to do it.
47:35 SHARON: Yeah, and avoiding those weird states is important because that's what attackers look for, of, once we're in this weird state, all bets are off kind of thing, and we can just do whatever. So--
47:43 PETER: Yes.
47:43 SHARON: --eliminating those.
47:48 PETER: C++ 26 has something in this space called Contracts that they're working on, where you can annotate a function to basically say, these are the preconditions and postconditions and things. I don't know how that will turn out, and I don't know whether we'll want that at the time, it ships. The other thing to mention here is that-- I said that rolling this out can sometimes be hard. When we say, oh, our guidance is, use CHECK() unless it's perf-disastrous, a lot of the questions that I get are things like, well, what if I'm pretty sure this is true, but I mean, I don't know for certain? And I don't want to ship this thing and crash everyone in the wild. Shouldn't I use a DCHECK() if that's the case? And we have a couple tools for dealing with that. And one of them is the feature flag. Pretty much everything should be developed under a feature flag, unless it really, truly doesn't make sense to do so. And that's a way that you can say, oh, hey, we noticed everybody under this feature is crashing. Turn it off. But the other thing is that pbos added something called NotFatalUntil, which is a way of saying, hey, I'm putting this in. I'm explicitly going to make it fatal in the future. But for right now, I just want to collect crash stacks and data on it and not actually crash in the field. And that's a good tool that people can use to implement things in a cautious way.
49:11 SHARON: Is that the same as DumpWithoutCrashing()?
49:11 PETER: So NotFatalUntil, I believe, uses DumpWithoutCrashing() to implement things. DumpWithoutCrashing() is a way of explicitly just saying. I don't want this to be fatal. I just want to collect stuff. NotFatalUntil is a way of marking a CHECK(), as having that behavior, and it will automatically switch over at a certain milestone. So you say, NotFatalUntil M-136, and then when M-136 rolls around, bang, that becomes fatal. And everybody--
49:40 SHARON: Everyone's crashing. Yeah.
49:40 PETER: Hopefully not, because hopefully you caught and fixed all the problems with it before then. But yes.
49:46 SHARON: So in terms of other macros we see a lot, maybe more as a-- not something you hopefully see as much in production, but in tests and general debugging is logging and various logging-adjacent macros. Can you tell us a bit about those?
50:00 PETER: Yes. If you're on a team where you know, I have to be able to debug only from the log output. There's no other way. And I know how to collect it, and I know what I'm going to do with it, and I'm going to clean it up eventually when we fix the problem. Then if all those are true, go for it and log. But otherwise, no.
50:16 SHARON: If you do want to collect data of, say, certain values, of certain variables out in the field that you can't get locally, we have better ways to do that.
50:27 PETER: Yeah. There's debugging utilities. So you mentioned DumpWithoutCrashing(), and that's a way to send back a lot of data to us as if there was a crash. And then when you do that, you can use something called debug Alias(), where you can force a particular variable's value to get captured by the crash data, because normally, the crash data will include things like, well, these variables were on the stack. But in a release build, a lot of things are optimized away, so you can't guarantee something like that. So aliasing a variable using that particular //base utility is one way to make sure that-- we want to capture this, this, this, in this dump, for sure. Do it.
51:05 SHARON: Is that the thing we also call "crash keys?"
51:11 PETER: I think it uses crash keys to implement it. I haven't looked at this very recently. So--
51:17 SHARON: OK, I think we'll end our meander through //base at that point. But there are many types we didn't cover. So how do people find those? Should they try to remember everything we just mentioned?
51:28 PETER: Yes, this is the, Mr. Johnson, may I be excused; my brain is full, moment. So there's no way to remember all the different things in //base, as far as I can tell. I'm an owner, and I constantly find things that I'm like, oh, I didn't know we had this. Oh, that would have been useful. So I try to tell people, like, yeah, you can get better with some of this stuff. The biggest way to do this is just practice. Practice is much better than just raw, focused effort on polishing a single thing. Just write more CLs. Look more at the //base APIs. Use more things. Put them in practice. Send a bunch of stuff. If it's not perfect, I don't care. If it's a monotonic improvement over what we've got, I'll stamp that immediately and say, sure, let's move forward, and just do it more. But in the limit, nobody can remember all this stuff. And I feel a little bit bad that-- I think the message that a lot of people get in code review is like, what the heck are you doing? Why didn't you use a base blah blah, that you've never heard of? And you're like, I'm sorry that I am not a genius like you. So actually, I feel overwhelmed and incompetent, et cetera, a lot of the time, too. And it's because this is a hard problem and it's a big space. //base is huge. Chrome is huge.
52:54 SHARON: Monotonically increasingly huge.
52:54 PETER: C++ is huge. The web is huge. Nobody-- no human being is capable of being an expert in any of these areas. It's too big. And therefore, like if we can have compassion for each other, that's good. I hope, increasingly, we give people more encouragement and opportunity to succeed and not just, hey, avoid failure harder, because that's just a route to everybody getting burned out and miserable.
53:20 SHARON: Yeah, I mean, Chrome is fun because a lot of people have been around for a while, so they have a better grasp of things, I guess, because it used to be simpler. So it's easier to patch in those incremental changes, whereas when you come in now, it's so much stuff. It's like, oh, my God. What? What's happening. And it gets harder and harder to start. And there's not that many new people at Chrome, relatively, so we kind of don't have that constant reminder of, oh, this is hard, and this is what people find hard now. So--
53:55 PETER: Yes, anybody coming into Chrome-- it's enormous and overwhelming. And I mean, it overwhelms me, and I've been here since the inception. So it's very much true.
54:06 SHARON: I think everyone is overwhelmed, no matter how long they've been here. It's just what they are overwhelmed by changes as you go.
54:18 PETER: I have proposed in the past doing more formal training, not just classes or talks or something, but direct one on one-- here, you watch me step through this kind of problem and do this. And I'll watch you, and let's give each other feedback. And very much more apprenticeship model than just like lecturer model stuff in Chrome. I think that would be good. I think readability reviews would be useful. I used to be a C++ readability reviewer at Google. And all of those things-- it's been difficult to get organizational traction to actually go do those. So shameless plug-- if people think that would be useful for them and you want to do that with me or somebody else, let me know, and I will try to make it happen. And if you think that's a terrible idea, don't let me know. I don't need more discouragement right now.
55:15 SHARON: Yeah. I mean, everyone in Chrome is incredibly helpful and friendly. There's people who you think, oh, they must be so busy. But they're always so willing to help and talk and whatever.
55:21 PETER: And I think one of the keys to making that happen is finding the right people for questions and then not turning it into no good deed goes unpunished. So I've been guilty of this, where someone was helpful to me, and then I immediately just rammed 500 followups down their throat. And they, internally, were like, I think I will not be helpful in the future. That might work better. So I try to tell people, hey, as much as possible, instead of sending chats or emails to one specific person--
56:00 SHARON: Post them to the mailing list.
56:00 PETER: --post them on a mailing lists. Put them in chat threads. Come hang out on Slack. There's this weird dichotomy of teams in Chrome that use Slack, and teams in Chrome that do not use Slack at all, no. I don't really care, in terms of what your team wants to do. Neither one is wrong. But in terms of, can everybody else see it and make use of it, if you come over to Slack, then, yes, that can happen. If you're in your team's chat room, probably not. So that's my, you should all be in Slack if you would like to have your conversations visible and possibly helped by other people on the team.
56:40 SHARON: In Slack, it's much easier to search and use and the other stuff we use. So if you want to be able to find an answer to something you asked a long time ago, it's going to be much easier in Slack than--
56:51 PETER: Hey, now, you sound dangerously like someone who does not believe that all Google products are the best for all situations.
57:00 SHARON: OK, so something you have mentioned a few times, and we've touched on is updating C++ versions. So you are quite well-acquainted with that. So what goes into going from, say, C++ 17 to 20?
57:07 PETER: Yeah, I helped with 14 and 17, and then I pretty much drove 20. So just so people understand what it means when we say, well, what version of C++, pretty much, at any given time, Chrome has some version of C++ that it says it formally supports. So we say, right now, that we formally support C++ 20. And what that means in practice is really that we pass, like, dash std equals C++ 20 to the compiler and the linker when we build stuff. So this really means that's the version that's in our build files that we tell the compiler to use. And then it will complain about stuff outside that. That means you can also use earlier stuff. Whether you can use later stuff is a matter of whether the compiler will let you. C++ 20 introduced designated initializers. But in fact, you could use them before then, in part because compilers would allow that. And Google style guide, said, yes, and we're OK with that. So it's a function of that stuff. So we have that. And we have a guide called c++-features.md that says, here's all the stuff in the different language versions that you are, are not allowed to use. And when we decide that we want to go to a new version, like say, if we want to go to C++ 23 right now, the first thing that somebody will look at-- there's no formal owner of this. There's no timetable that says, thou shalt pull this new version into Chrome at this time, and this is the team that will do it. It's very much like, no one owns it, and it will happen if it happens. And the reason that I did 20 is because I wanted 20. It actually was even weirder than that. I wanted us to be on 20 in hopes that I could use MSVC to test something specific at low level in //base. And then that snowballed because I was just like, well, I'll just roll the C++ version. How hard could it be? A year later, we rolled. So--
59:15 SHARON: That's not bad. That's pretty good.
59:15 PETER: So what I might try doing first is simply see, does the toolchain support it? So that means go change the version we passed to the compiler and see if it compiles. Some stuff gets deprecated in new versions. And you maybe have to go fix that. And then the next thing is, well, do we want to allow it? Like, it does no good to say you can use C++ 23 if all of the features aren't implemented yet, and they're broken, and you don't know that they're broken because the compiler says that they work, but they don't. This was the case for a number of things in the past, where people would be like, yeah, you can theoretically use this. Don't use it. It doesn't work. So once those things are true, and we know the toolchain pretty much supports most of what we want to use, not all of-- the rule is, not all of, because toolchains have lagging things. Actually, Clang slash libc++ just finished C++ 17 within the last couple of months. They finished the last bits in order to mark it as fully complete on their thing. Obviously, C++ has been out for a long time, and we've been using it for a long time, and so has everybody else. So we don't wait for 100% to be complete. We wait for enough. And then we pull in the new thing, fix everything, and write a bunch of updates to the features doc that says, you can and can't use these parts. And normally, we're somewhat conservative. We don't ban everything. We ban anything that doesn't work. But we might also ban some other stuff just because, yeah, we probably want this, but let's get ourselves onto the new version before we have to think about, what's the migration plan to this feature? So let's just, for now, block it. And we'll think of that after the fact. And that's what happened with C++ 20. We blocked a lot of stuff initially. And then in the months following that update, we've allowed more things. The std ranges stuff earlier was an example. I had said, that's what //base ranges was. And then if anybody out there was asking, well wait a minute, what do you mean, was? It's still there. I see it in the tree. What do you mean? It's because std ranges is actually, now, the approved way of going, and the old thing is deprecated. We will get rid of it at some point. There is a migration plan. But that's the path that all of the new things follow. So there's a whole list of stuff in C++ 20 that we want to allow. And I push aggressively to move forward if we can. And it happens when it happens.
61:59 SHARON: OK. So if someone wants to be the person who updates to C++ 23 or generally get more involved in //base kind of things, become another //base owner, what kind of things can you start out by doing now to go into that direction?
62:14 PETER: Yeah, it's funny because in the Notes doc that I wrote myself where this talk-- all of my notes to myself are at least very negative like, don't do this; don't do this. And I'm thinking, wow, how discouraging. Actually, the best answer is, go do it. A lot of these things-- like I said, with C++ 20, it happened because I was just like, let's do this. How hard can it be? Many bad events have started with those words. But with a lot of things. If you want to do stuff in //base or if you want to make major C++ changes in Chrome, just do it. We have this cultural principle now, think like an owner. Part of what that means is it actually is OK for you to go make major changes. You do not need to go get permission. Now, you may want to talk to some knowledgeable people so that you don't do something silly like, oh, well, I just spent four months making this possible, only to find that you had blocked it because-- other reason that I was unaware of or something. But that's very different than, hey, I don't know whether I'm allowed to do anything. Can I touch //base? No this is the special, sacred area. It's just like every other directory. Send CLs to an owner, and we'll do stuff. In fact, for a lot of people, that's enough. The C++ 20 upgrade I did I was not a //base owner for, and I still updated the whole C++ version that Chrome uses. And I reviewed all //base ranges. I did a whole bunch of LSCs to change our string types. All of that stuff happened without being a //base owner. You probably don't need to be in an OWNERS file to have special credentials or anything to do a lot of this stuff. If you're interested and you like doing it, then go do it. If you actually want to become a //base owner or something like that, then, yeah, like probably, eventually, you'll need to have some decent C++ skill. I don't like to use words like, expert, partly because no one knows what they mean, but also partly because I think the people who are the biggest experts in some area understand how much they don't know. And therefore, there's this sort of Dunning-Kruger like-- the people who are not experts are like, I'm an expert. And then the experts are like, I'm not an expert. So that happens. But know some stuff. Don't be totally terrified by templates. Maybe feel pretty comfortable with most of the stuff I covered in C++ 201. Some breadth of experience. If you can have a rough idea that, yeah, this would probably be useful several places in Chrome; I think we probably do this multiple places, then that's a useful instinct to be able to have. And you're comfortable tackling API design-level questions. So not just the implementation, but also, what direction do we want to move the code //base in? And is this a safe concept, that people should think at this level? Those kinds of considerations are things that you want to be comfortable thinking about. And then finally, fundamentally, you have to have somebody who's established trust over time. //base OWNERS get an automatic OWNERS override capability. This is not intended to be like a carrot of, go become a //base owner so you can have this. It's mostly just like a, hey, if you're touching this, you probably need to make a lot of changes that touch a lot of directories across the code to fix API usage or something. So just for convenience sake, we're going to give these people OWNERS override permission. There's still a request that you not use it on your own CLs. I'm not supposed to 00 plus 1 my own change after somebody reviews it, that sort of thing. And because of that level of power, we want to be sure that people who we're giving this to are people who can be trusted to use that in a responsible way. So somebody who's only been known to the project for three weeks-- even if they're the world-- Herb Sutter comes in and is like, hey, I am the head of the C++ committee. I know lots. Can I be a //base owner? It's like, probably no, not right away, not because we don't think you're good. It's just because you don't know Chrome yet, and we don't know you. And so a lot of building relationship over time.
66:48 SHARON: Sounds good. Yeah, so if you want to get more into it, cxx is both a mailing list open to the public, as well as a Slack channel. Lots of other Slack channels, other related ones, too.
67:06 PETER: Yeah, there is a //base Slack channel.
67:06 SHARON: A //base Slack channel.
67:06 PETER: And it's funny, because many of us are in many Slack channels and will have conversations simultaneously across several of them, which is a little odd. But yes, most of our day-to-day work gets discussed on Slack. So things like, how do I use this space API? Or do you think these things should change this way? That sort of question is very Slack-level. And then more formal like, shall this feature be allowed or banned? That's a mailing list-type question. So people are welcome to come participate in both of those as they desire to do so.
67:43 SHARON: OK. Well, thank you so much. This was very fun to see-- get a glimpse into everything that is in phase, and hopefully, people will find it interesting and want to get more involved.
67:56 PETER: All right.
67:56 SHARON: All right. Thank you very much. You could do both. This is a fairly low-budget production, as you can tell by the pens holding up--
68:02 PETER: And the fact that you-- where is second-- oh, they're--