For seven years and counting, Gil Yehuda, Senior Director of Open Source at Oath Inc. (which owns the Yahoo and AOL brands), has led the open source program at Yahoo. Now with an expanded scope, he is gearing up to grow his team and improve the program. The company’s formal open source program office serves as a hub to connect all open source activities across the company, he says, but it didn’t start out that way.
As with many other companies, the open source program started informally with a group of diligent engineers and a few legal people. But the ad hoc group soon realized it needed a more formal program if it was going to be able to scale to address more issues and achieve specific business goals. With a formal program in place, they are poised to achieve its goals.
The top five of Oath’s numerous open source goals, according to Yehuda, are:
- Keep aligned with the industry on open source technology standards by avoiding creating unique tech stacks that Oath alone would have to manage at its own expense.
- Make it easy for engineers to interact with open source as users and as contributors.
- Be viewed as an open source friendly company for partnerships and collaborations.
- Be known as a great place for engineers to work on open source projects.
- Give back to the Open Source community by sharing code and practices.
Measuring and monitoring success requires the right tools and attitudes. Yehuda says at Oath they actively solicit and listen to the needs of their many engineering teams, track all their work transparently in JIRA, and spread the work across many teams who help with the process.
"We have custom tools we use to check code and manage projects, but we’re hoping to work more with our peers in the TODO Group on tooling we can share across many of our peer open source program offices," he said.
Yahoo helped make Apache Hadoop the cornerstone of the big data revolution when it took the early code and created a team around it to help it scale to internet-scale. They agreed to publish it all as open source. When the need for real-time processing came to the forefront, Yahoo created S4 and open sourced it too, but then discovered Storm was just published too, and it looked more promising. The team ditched their own code and put their efforts into helping make Storm even better.
"We applied to Apache Storm what we learned from Hadoop and S4," Yehuda said. "Our goal was to make it great, even though it kind of competed with our own first stab."
Storm is a success today, and the company runs it alongside Hadoop to power many of its products. They added machine learning and high-scale data serving capabilities by adding Vespa Engine, to their platforms, and then published that too. And they helped other machine learning projects scale better too, all by publishing open source.
"We’ve leveraged our expertise with Storm to help both Caffe and TensorFlow achieve better scalability. We don’t own these solutions exclusively. Rather we share our code and help others -- all the while we get to leverage our expertise to build one of the industry’s most scalable platforms for our use," he said. "This saves us money while making us a fantastic place to work on projects that impact hundreds of millions of people."
The program office worked on strategy, legalities, and execution of these and similar projects. Leveraging open source licensing and processes effectively was a key element throughout. Now as Oath, this work continues and expands.
Yehuda cited three key lessons he learned managing an open source program:
- Be a service to the engineers, not a barrier.
- Accept that challenges will be never-ending.
- Run the program office like you run an open source project: Be transparent in the way decisions are made and be open to input and collaboration from everyone.
"There are so many edge cases that come up -- partnerships, acquisitions, unclear contract terms -- and we simply need to be open to learn, explore, and come up with an answer to every open source related question. But the most rewarding part of my job is when people tell me they joined our company because they knew about our open source friendly culture. You know, we’re always looking for open source talent, and I’m hiring into the program office." added Yehuda.