SOSP Workshop on Large Scale Distributed Systems and Middleware (LADIS) 2009

We are in the midst of a computing revolution. As the cost of provisioning hardware and software stacks grows, and the cost of securing and administering these complex systems grows even faster, we're seeing a shift towards computing clouds. For cloud service providers, there is efficiency from amortizing costs and averaging usage peaks. Internet portals like Yahoo! have long offered application services, such as email for individuals and organizations. Companies are now offering services such as storage and compute cycles, enabling higher-level services to be built on top. In this talk, I will discuss Yahoo!'s vision of cloud computing, and describe some of the key initiatives, highlighting the technical challenges involved in designing hosted, multi-tenanted data management systems.

Raghu Ramakrishnan is Chief Scientist for Audience & Cloud Computing, and a Fellow at Yahoo!, where he heads the Community Systems group. He has been Professor of Computer Sciences at the University of Wisconsin-Madison, and was founder and CTO of QUIQ, a company that pioneered question-answering communities, powering Ask Jeeves' AnswerPoint. His research is in the area of database systems, with a focus on data mining, online communities, and web-scale data management. He received the ACM SIGKDD Innovation Award in 2008 and the ACM SIGMOD Contributions Award in 1999. He is a Fellow of the ACM and a Fellow of the IEEE, served as Chair of ACM SIGMOD, is on the Board of Trustees of the VLDB Endowment, and served as editor-in-chief of the Journal of Data Mining and Knowledge Discovery. He received his Ph.D. from the University of Texas at Austin.

Running a highly scalable, reliable and cost efficient utility computing platform such as Amazon Web Services involves a number of challenges and opportunities. Designing an architecture that provides resources in an elastic, on-demand manner that is able to withstand fault events such as the loss of entire data centers without impacting a customer's application is just one of the challenges our AWS developers encounter. This talk will focus on core design principles for all AWS services and the lessons learned in acquiring those principles.

Marvin Theimer is a Senior Principal Engineer at Amazon. Marvin received a PhD in Computer Science from Stanford University and has spent most of his career in research, having worked at IBM Almaden, Xerox PARC, and Microsoft Research on topics including distributed operating systems, ubiquitous computing, weakly-consistent replicated systems, peer-to-peer file systems, and global-scale peer-to-peer event notification systems. In 2003 he migrated from research to product development and has worked on web services, grid computing, high-performance computing, eCommerce platforms, and utility computing at Microsoft, Google, and Amazon.

As part of implementing the many products and services offered by Google, we have built a collection of systems and tools that simplify the storing and processing of large-scale data sets, and the construction of heavily-used public services based on these data sets. These systems are intended to work well in Google's computational environment, which consists of large numbers of commodity machines connected by commodity networking hardware. Our systems handle issues like storage reliability and availability in the face of machine failures, and our processing tools make it relatively easy to write robust computations that run reliably and efficiently on thousands of machines. In this talk I'll highlight some of the systems we have built, and discuss some challenges and future directions for new systems.

Jeff Dean joined Google in 1999 and is currently a Google Fellow in Google's Systems Infrastructure Group. He has co-designed/implemented five generations of Google's crawling, indexing, and query serving systems, and co-designed/implemented major pieces of Google's initial advertising and AdSense for Content systems. He is also a co-designer and co-implementor of Google's distributed computing infrastructure, including the MapReduce and BigTable systems, has worked on system software for statistical machine translation, and implemented on a variety of internal and external developer tools.

Like other large-scale Internet service companies, Microsoft has an in-house data storage platform supporting transactions, on-line update and a sophisticated query language. It's called SQL Server, and it's deployed on thousands of servers in Windows Live. Using SQL at this scale leads to a number of challenges in areas such as replication, data consistency, backup, reporting, and schema evolution. This talk will look at several of these issues and the tradeoffs we've made in Windows Live to address them.

David Nichols is a software developer in the Windows Live group at Microsoft, working on large-scale storage systems. He came to Microsoft by the acquisition of PlaceWare, Inc. where he was a co-founder and principal rchitect. The PlaceWare web conferencing product became Microsoft Office Live Meeting. Before PlaceWare, he was a researcher at Xerox PARC, working on the Tapestry and Jupiter collaboration systems. David has a Ph.D. in Computer Science from Carnegie Mellon University, where he was one of the designers of the Andrew File System.