radartower

Seattle scales

Back (a few hours ago) from the Google Scalability Conference. Most of my colleagues no-showed, but Brandon delivered. Highlights included Swapnil Patil’s talk on GIGA+ massively scalable directories for shared file systems (billions of files). PDF here (a bit out of date compared to the updated version we received at the conference, but certainly good enough). This quote describes some of the magic of the system:

A novel property of Giga+ is that each client caches its own partition-to-server map (P2SMap), without using a traditional, synchronous cache consistency protocol. As the directory grows, at high insert rates, the directory is partitioned on more servers. Clients will not immediately know about the partitions created at the server due to an operation sent by another client. As a result, all clients will end up having different, out-of-date copies of the P2SMap. Despite the inconsistent copies of P2SMap, Giga+ ensures that the clients’ requests are forwarded to the correct server

Also very interesting was Vijay Menon’s efficient and perfectly clear presentation on Transactional Memory. Vijay provided an overview of TM, in hardware and software, and proceeded to peel the onion a bit on software-based TM. Of special note were some of the troubles (TOCTTOU, for example) that can occur with previous attempts at concurrency, including Java synchronized methods. Vijay stressed that there are a number of research and implementation issues to be addressed around TM. Performance, fairness, and semantics (especially where rollback is not feasible) were all identified as areas of improvement. Doug Lee’s work related to concurrent hashmaps was discussed during the Q&A as particularly interesting, using methods similar to “true” TM. In a day full of presentations on clusters and clouds, Vijay’s presentation illustrated that scalability is a concern for even solitary machines.

Notes from another attendee can be found here.

I snagged two Google LED throwies for my boys. All attendees received a Y Power Splitter because, well, I don’t know.