Happy Small Business Week.

Our first AdWords customer was a small business selling live mail-order lobsters. It’s been a long time since then, but a majority of our customers are still small businesses, who play a vital role not only for Google, but for the American economy. More than 60 percent of new jobs each year come from small businesses.



This Small Business Week, we want to celebrate you. We’re grateful to you for everything you do for us and our communities. Whether you fix people’s cars, offer music lessons to aspiring musicians, or make the world’s best homemade ice cream—when you do what you love, our lives get better.

As part of the celebration, we’ll be highlighting some amazing small businesses across the country, so keep an eye on the Google+ Your Business page. And in the meantime, check out some of the Google tools that are designed to help you take care of business.

Happy Small Business Week.

Posted by Lisa Gevelber, VP Marketing, Americas

Our continued commitment to combating child exploitation online

The Internet has been a tremendous force for good—increasing access to information, improving people’s ability to communicate and driving economic growth. But like the physical world, there are dark corners on the web where criminal behavior exists.

In 2011, the National Center for Missing & Exploited Children’s (NCMEC’s) Cybertipline Child Victim Identification Program reviewed 17.3 million images and videos of suspected child sexual abuse. This is four times more than what their Exploited Children's Division (ECD) saw in 2007. And the number is still growing. Behind these images are real, vulnerable kids who are sexually victimized and victimized further through the distribution of their images.

It is critical that we take action as a community—as concerned parents, guardians, teachers and companies—to help combat this problem.

Child sexual exploitation is a global problem that needs a global solution. More than half of the images and videos sent to NCMEC for analysis are found to have been uploaded to U.S. servers from outside the country. With this in mind, we need to sustain and encourage borderless communication between organizations fighting this problem on the ground. For example, NCMEC’s CyberTipline is able to refer reports regarding online child sexual exploitation to 66 countries, helping local law enforcement agencies effectively execute their investigations.

Google has been working on fighting child exploitation since as early as 2006 when we joined the Technology Coalition, teaming up with other tech industry companies to develop technical solutions. Since then, we’ve been providing software and hardware to helping organizations all around the world to fight child abuse images on the web and help locate missing children.

There is much more that can be done, and Google is taking our commitment another step further through a $5 million effort to eradicate child abuse imagery online. Part of this commitment will go to global child protection partners like the National Center for Missing & Exploited Children and the Internet Watch Foundation. We’re providing additional support to similar heroic organizations in the U.S., Canada, Europe, Australia and Latin America.

Since 2008, we’ve used “hashing” technology to tag known child sexual abuse images, allowing us to identify duplicate images which may exist elsewhere. Each offending image in effect gets a unique ID that our computers can recognize without humans having to view them again. Recently, we’ve started working to incorporate encrypted “fingerprints” of child sexual abuse images into a cross-industry database. This will enable companies, law enforcement and charities to better collaborate on detecting and removing these images, and to take action against the criminals. Today we’ve also announced a $2 million Child Protection Technology Fund to encourage the development of ever more effective tools.

We’re in the business of making information widely available, but there’s certain “information” that should never be created or found. We can do a lot to ensure it’s not available online—and that when people try to share this disgusting content they are caught and prosecuted.

Update June 17: Clarified language around NCMEC's Child Victim Identification Program and CyberTipline.

Posted by Jacquelline Fuller, Director, Google Giving

Introducing Project Loon: Balloon-powered Internet access

The Internet is one of the most transformative technologies of our lifetimes. But for 2 out of every 3 people on earth, a fast, affordable Internet connection is still out of reach. And this is far from being a solved problem.

There are many terrestrial challenges to Internet connectivity—jungles, archipelagos, mountains. There are also major cost challenges. Right now, for example, in most of the countries in the southern hemisphere, the cost of an Internet connection is more than a month’s income.

Solving these problems isn’t simply a question of time: it requires looking at the problem of access from new angles. So today we’re unveiling our latest moonshot from Google[x]: balloon-powered Internet access.

We believe that it might actually be possible to build a ring of balloons, flying around the globe on the stratospheric winds, that provides Internet access to the earth below. It’s very early days, but we’ve built a system that uses balloons, carried by the wind at altitudes twice as high as commercial planes, to beam Internet access to the ground at speeds similar to today’s 3G networks or faster. As a result, we hope balloons could become an option for connecting rural, remote, and underserved areas, and for helping with communications after natural disasters. The idea may sound a bit crazy—and that’s part of the reason we’re calling it Project Loon—but there’s solid science behind it.

Balloons, with all their effortless elegance, present some challenges. Many projects have looked at high-altitude platforms to provide Internet access to fixed areas on the ground, but trying to stay in one place like this requires a system with major cost and complexity. So the idea we pursued was based on freeing the balloons and letting them sail freely on the winds. All we had to do was figure out how to control their path through the sky. We’ve now found a way to do that, using just wind and solar power: we can move the balloons up or down to catch the winds we want them to travel in. That solution then led us to a new problem: how to manage a fleet of balloons sailing around the world so that each balloon is in the area you want it right when you need it. We’re solving this with some complex algorithms and lots of computing power.

Now we need some help—this experiment is going to take way more than our team alone. This week we started a pilot program in the Canterbury area of New Zealand with 50 testers trying to connect to our balloons. This is the first time we’ve launched this many balloons (30 this week, in fact) and tried to connect to this many receivers on the ground, and we’re going to learn a lot that will help us improve our technology and balloon design.

Over time, we’d like to set up pilots in countries at the same latitude as New Zealand. We also want to find partners for the next phase of our project—we can’t wait to hear feedback and ideas from people who’ve been working for far longer than we have on this enormous problem of providing Internet access to rural and remote areas. We imagine someday you'll be able to use your cell phone with your existing service provider to connect to the balloons and get connectivity where there is none today.

This is still highly experimental technology and we have a long way to go—we’d love your support as we keep trying and keep flying! Follow our Google+ page to keep up with Project Loon’s progress.

Onward and upward.

Posted by Mike Cassidy, Project Lead

Excellent Papers for 2012

Posted by Corinna Cortes and Alfred Spector, Google Research

Googlers across the company actively engage with the scientific community by publishing technical papers, contributing open-source packages, working on standards, introducing new APIs and tools, giving talks and presentations, participating in ongoing technical debates, and much more. Our publications offer technical and algorithmic advances, feature aspects we learn as we develop novel products and services, and shed light on some of the technical challenges we face at Google.

In an effort to highlight some of our work, we periodically select a number of publications to be featured on this blog. We first posted a set of papers on this blog in mid-2010 and subsequently discussed them in more detail in the following blog postings. In a second round, we highlighted new noteworthy papers from the later half of 2010 and again in 2011. This time we honor the influential papers authored or co-authored by Googlers covering all of 2012 -- covering roughly 6% of our total publications.  It’s tough choosing, so we may have left out some important papers.  So, do see the publications list to review the complete group.

In the coming weeks we will be offering a more in-depth look at some of these publications, but here are the summaries:

Algorithms and Theory

Online Matching with Stochastic Rewards
Aranyak Mehta*, Debmalya Panigrahi [FOCS'12]
Online advertising is inherently stochastic: value is realized only if the user clicks on the ad, while the ad platform knows only the probability of the click. This paper is the first to introduce the stochastic nature of the rewards to the rich algorithmic field of online allocations. The core algorithmic problem it formulates is online bipartite matching with stochastic rewards, with known click probabilities. The main result is an online algorithm which obtains a large fraction of the optimal value. The paper also shows the difficulty introduced by the stochastic nature, by showing how it behaves very differently from the classic (non-stochastic) online matching problem.

Matching with our Eyes Closed
Gagan Goel*, Pushkar Tripathi* [FOCS'12]
In this paper we propose a simple randomized algorithm for finding a matching in a large graph. Unlike most solutions to this problem, our approach does not rely on building large combinatorial structures (like blossoms) but works completely locally. We analyze the performance of our algorithm and show that it does significantly better than the greedy algorithm. In doing so we improve a celebrated 18 year old result by Aronson et. al.

Simultaneous Approximations for Adversarial and Stochastic Online Budgeted Allocation
Vahab Mirrokni*, Shayan Oveis Gharan, Morteza Zadimoghaddam, [SODA'12]
In this paper, we study online algorithms that simultaneously perform well in worst-case and average-case instances, or equivalently algorithms that perform well in both stochastic and adversarial models at the same time. This is motivated by online allocation of queries to advertisers with budget constraints. Stochastic models are not robust enough to deal with traffic spikes and adversarial models are too pessimistic. While several algorithms have been proposed for these problems, each algorithm was known to perform well in one model and not both, and we present new results for a single algorithm that works well in both models.

Economics and EC

Polyhedral Clinching Auctions and the Adwords Polytope
Gagan Goel*, Vahab Mirrokni*, Renato Paes Leme [STOC'12]
Budgets play a major role in ad auctions where advertisers explicitly declare budget constraints. Very little is known in auctions about satisfying such budget constraints while keeping incentive compatibility and efficiency. The problem becomes even harder in the presence of complex combinatorial constraints over the set of feasible allocations. We present a class of ascending-price auctions addressing this problem for a very general class of (polymatroid) allocation constraints including the AdWords problem with multiple keywords and multiple slots.

HCI

Backtracking Events as Indicators of Usability Problems in Creation-Oriented Applications
David Akers*, Robin Jeffries*, Matthew Simpson*, Terry Winograd [TOCHI '12]
Backtracking events such as undo can be useful automatic indicators of usability problems for creation-oriented applications such as word processors and photo editors. Our paper presents a new cost-effective usability evaluation method based on this insight.

Talking in Circles: Selective Sharing in Google+
Sanjay Kairam, Michael J. Brzozowski*, David Huffaker*, Ed H. Chi*, [CHI'12]
This paper explores why so many people share selectively on Google+: to protect their privacy but also to focus and target their audience. People use Circles to support these goals, organizing contacts by life facet, tie strength, and interest.

Information Retrieval

Online selection of diverse results
Debmalya Panigrahi, Atish Das Sarma, Gagan Aggarwal*, and Andrew Tomkins*, [WSDM'12]
We consider the problem of selecting subsets of items that are simultaneously diverse in multiple dimensions, which arises in the context of recommending interesting content to users. We formally model this optimization problem, identify its key structural characteristics, and use these observations to design an extremely scalable and efficient algorithm. We prove that the algorithm always produces a nearly optimal solution and also perform experiments on real-world data that indicate that the algorithm performs even better in practice than the analytical guarantees.

Machine Learning

Large Scale Distributed Deep Networks
Jeffrey Dean, Greg S. Corrado*, Rajat Monga, Kai Chen, Matthieu Devin, Quoc V. Le, Mark Z. Mao, Marc’Aurelio Ranzato, Andrew Senior, Paul Tucker, Ke Yang, Andrew Y. Ng, NIPS 2012;
In this paper, we examine several techniques to improve the time to convergence for neural networks and other models trained by gradient-based methods. The paper describes a system we have built that exploits both model-level parallelism (by partitioning the nodes of a large model across multiple machines) and data-level parallelism (by having multiple replicas of a model process different training data and coordinating the application of updates to the model state through a centralized-but-partitioned parameter server system). Our results show that very large neural networks can be trained effectively and quickly on large clusters of machines.

Open Problem: Better Bounds for Online Logistic Regression
Brendan McMahan* and Matthew Streeter*, COLT/ICML'12 Joint Open Problem Session, JMLR: Workshop and Conference Proceedings.
One of the goals of research at Google is to help point out important open problems--precise questions that are interesting academically but also have important practical ramifications. This open problem is about logistic regression, a widely used algorithm for predicting probabilities (what is the probability an email message is spam, or that a search ad will be clicked). We show that in the simple one-dimensional case, much better results are possible than current theoretical analysis suggests, and we ask whether our results can be generalized to arbitrary logistic regression problems.

Spectral Learning of General Weighted Automata via Constrained Matrix Completion
Borja Balle and Mehryar Mohri*, NIPS 2012.
Learning weighted automata from finite samples drawn from an unknown distribution is a central problem in machine learning and computer science in general, with a variety of applications in text and speech processing, bioinformatics, and other areas. This paper presents a new family of algorithms for tackling this problem for which it proves learning guarantees. The algorithms introduced combine ideas from two different domains: matrix completion and spectral methods.

Machine Translation

Improved Domain Adaptation for Statistical Machine Translation
Wei Wang*, Klaus Macherey*, Wolfgang Macherey*, Franz Och* and Peng Xu*, [AMTA'12]
Research in domain adaptation for machine translation has been mostly focusing on one domain. We present a simple and effective domain adaptation infrastructure that makes an MT system with a single translation model capable of providing adapted, close-to-upper-bound domain-specific accuracy while preserving the generic translation accuracy. Large-scale experiments on 20 language pairs for patent and generic domains show the viability of our approach.

Multimedia and Computer Vision

Reconstructing the World's Museums
Jianxiong Xiao and Yasutaka Furukawa*, [ECCV '12]
Virtual navigation and exploration of large indoor environments (e.g., museums) have been so far limited to either blueprint-style 2D maps that lack photo-realistic views of scenes, or ground-level image-to-image transitions, which are immersive but ill-suited for navigation. This paper presents a novel vision-based 3D reconstruction and visualization system to automatically produce clean and well-regularized texture-mapped 3D models for large indoor scenes, from ground-level photographs and 3D laser points. For the first time, we enable users to easily browse a large scale indoor environment from a bird's-eye view, locate specific room interiors, fly into a place of interest, view immersive ground-level panoramas, and zoom out again, all with seamless 3D transitions.

The intervalgram: An audio feature for large-scale melody recognition
Thomas C. Walters*, David Ross*, Richard F. Lyon*, [CMMR'12]
Intervalgrams are small images that summarize the structure of short segments of music by looking at the musical intervals between the notes present in the music. We use them for finding cover songs - different pieces of music that share the same underlying composition. Wedo this by comparing 'heatmaps' which look at the similarity between intervalgrams from different pieces of music over time. If we see a strong diagonal line in the heatmap, it's good evidence that the songs are musically similar.

General and Nested Wiberg Minimization
Dennis Strelow*, [CVPR'12]
Eriksson and van den Hengel’s CVPR 2010 paper showed that Wiberg’s least squares matrix factorization, which effectively eliminates one matrix from the factorization problem, could be applied to the harder case of L1 factorization. Our paper generalizes their approach beyond factorization to general nonlinear problems in two sets of variables, like perspective structure-from-motion. We also show that with our generalized method, one Wiberg minimization can also be nested inside another, effectively eliminating two of three sets of unknowns, and we demonstrated this idea using projective struture-from-motion

Calibration-Free Rolling Shutter Removal
Matthias Grundmann*, Vivek Kwatra*, Daniel Castro, Irfan Essa*, International Conference on Computational Photography '12. Best paper.
Mobile phones and current generation DSLR’s, contain an electronic rolling shutter, capturing each frame one row of pixels at a time. Consequently, if the camera moves during capture, it will cause image distortions ranging from shear to wobbly distortions. We propose a calibration-free solution based on a novel parametric mixture model to correct these rolling shutter distortions in videos that enables real-time rolling shutter rectification as part of YouTube’s video stabilizer.

Natural Language Processing

Vine Pruning for Efficient Multi-Pass Dependency Parsing
Alexander Rush, Slav Petrov*, The 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL '12), Best Paper Award.
Being able to accurately analyze the grammatical structure of sentences is crucial for language understanding applications such as machine translation or question answering. In this paper we present a method that is up to 200 times faster than existing methods and enables the grammatical analysis of text in large-scale applications. The key idea is to perform the analysis in multiple coarse-to-fine passes, resolving easy ambiguities first and tackling the harder ones later on.

Cross-lingual Word Clusters for Direct Transfer of Linguistic Structure
Oscar Tackstrom, Ryan McDonald*, Jakob Uszkoreit*, North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL '12), Best Student Paper Award.
This paper studies how to build meaningful cross-lingual word clusters, i.e., clusters containing lexical items from two languages that are coherent along some abstract dimension. This is done by coupling distributional statistics learned from huge amounts of language specific data coupled with constraints generated from parallel corpora. The resulting clusters are used to improve the accuracy of multi-lingual syntactic parsing for languages without any training resources.

Networks

How to Split a Flow
Tzvika Hartman*, Avinatan Hassidim*, Haim Kaplan*, Danny Raz*, Michal Segalov*, [INFOCOM '12]
Decomposing a flow into a small number of paths is a very important task arises in various network optimization mechanisms. In this paper we develop an an approximation algorithm for this problem that has both provable worst case performance grantees as well as good practical behavior.

Deadline-Aware Datacenter TCP (D2TCP)
Balajee Vamanan, Jahangir Hasan*, T. N. Vijaykumar, [SIGCOMM '12]
Some of our most important products like search and ads operate under soft-real-time constraints. They are architected and fine-tuned to return results to users within a few hundred milliseconds. Deadline-Aware Datacenter TCP is a research effort into making the datacenter networks deadline aware, thus improving the performance of such key applications.

Trickle: Rate Limiting YouTube Video Streaming
Monia Ghobadi, Yuchung Cheng*, Ankur Jain*, Matt Mathis* [USENIX '12]
Trickle is a server-side mechanism to stream YouTube video smoothly to reduce burst and buffer-bloat. It paces the video stream by placing an upper bound on TCP’s congestion window based on the streaming rate and the round-trip time. In initial evaluation Trickle reduces the TCP loss rate by up to 43% and the RTT by up to 28%. Given the promising results we are deploying Trickle to all YouTube servers.

Social Systems

Look Who I Found: Understanding the Effects of Sharing Curated Friend Groups
Lujun Fang*, Alex Fabrikant*, Kristen LeFevre*, [Web Science '12]. Best Student Paper award.
In this paper, we studied the impact of the Google+ circle-sharing feature, which allows individual users to share (publicly and privately) pre-curated groups of friends and contacts. We specifically investigated the impact on the growth and structure of the Google+ social network. In the course of the analysis, we identified two natural categories of shared circles ("communities" and "celebrities"). We also observed that the circle-sharing feature is associated with the accelerated densification of community-type circles.

Software Engineering

AddressSanitizer: A Fast Address Sanity Checker
Konstantin Serebryany*, Derek Bruening*, Alexander Potapenko*, Dmitry Vyukov*, [USENIX ATC '12].
The paper “AddressSanitizer: A Fast Address Sanity Checker” describes a dynamic tool that finds memory corruption bugs in C or C++ programs with only a 2x slowdown. The major feature of AddressSanitizer is simplicity -- this is why the tool is very fast.

Speech

Japanese and Korean Voice Search
Mike Schuster*, Kaisuke Nakajima*, IEEE International Conference on Acoustics, Speech, and Signal Processing [ICASSP'12].
"Japanese and Korean voice search" explains in detail how the Android voice search systems for these difficult languages were developed. We describe how to segment statistically to be able to handle infinite vocabularies without out-of-vocabulary words, how to handle the lack of spaces between words for language modeling and dictionary generation, and how to deal best with multiple ambiguities during evaluation scoring of reference transcriptions against hypotheses. The combination of techniques presented led to high quality speech recognition systems--as of 6/2013 Japanese and Korean are #2 and #3 in terms of traffic after the US.

Google's Cross-Dialect Arabic Voice Search
Fadi Biadsy*, Pedro J. Moreno*, Martin Jansche*, IEEE International Conference on Acoustics, Speech, and Signal Processing [ICASSP 2012].
This paper describes Google’s automatic speech recognition systems for recognizing several Arabic dialects spoken in the Middle East, with the potential to reach more than 125 million users. We suggest solutions for challenges specific to Arabic, such as the diacritization problem, where short vowels are not written in Arabic text. We conduct experiments to identify the optimal manner in which acoustic data should be clustered among dialects.

Deep Neural Networks for Acoustic Modeling in Speech Recognition
Geoffrey Hinton*, Li Deng, Dong Yu, George Dahl, Abdel-rahman Mohamed, Navdeep Jaitly, Andrew W. Senior*, Vincent Vanhoucke*, Patrick Nguyen, Tara Sainath, Brian Kingsbury, Signal Processing Magazine (2012)"
Survey paper on the DNN breakthrough in automatic speech recognition accuracy.

Statistics

Empowering Online Advertisements by Empowering Viewers with the Right to Choose
Max Pashkevich*, Sundar Dorai-Raj*, Melanie Kellar*, Dan Zigmond*, Journal of Advertising Research, vol. 52 (2012).
YouTube’s TrueView in-stream video advertising format (a form of skippable in-stream ads) can improve the online video viewing experience for users without sacrificing advertising value for advertisers or content owners.

Structured Data

Efficient Spatial Sampling of Large Geographical Tables
Anish Das Sarma*, Hongrae Lee*, Hector Gonzalez*, Jayant Madhavan*, Alon Halevy*, [SIGMOD '12].
This paper presents fundamental results for the "thinning problem": determining appropriate samples of data to be shown on specific geographical regions and zoom levels. This problem is widely applicable for a number of cloud-based geographic visualization systems such as Google Maps, Fusion Tables, and the developed algorithms are part of the Fusion Tables backend. The SIGMOD 2012 paper was selected among the best papers of the conference, and invited to a special best-papers issue of TODS.

Systems

Spanner: Google's Globally-Distributed Database
James C. Corbett*, Jeffrey Dean*, Michael Epstein*, Andrew Fikes*, Christopher Frost*, JJ Furman*, Sanjay Ghemawat*, Andrey Gubarev*, Christopher Heiser*, Peter Hochschild*, Wilson Hsieh*, Sebastian Kanthak*, Eugene Kogan*, Hongyi Li*, Alexander Lloyd*, Sergey Melnik*, David Mwaura*, David Nagle*, Sean Quinlan*, Rajesh Rao*, Lindsay Rolig*, Dale Woodford*, Yasushi Saito*, Christopher Taylor*, Michal Szymaniak*, Ruth Wang*, [OSDI '12]
This paper shows how a new time API and its implementation can provide the abstraction of tightly synchronized clocks, even on a global scale. We describe how we used this technology to build a globally-distributed database that supports a variety of powerful features: non-blocking reads in the past, lock-free snapshot transactions, and atomic schema changes.

Getting healthy just got a little easier

We’re all looking for ways to get a little healthier and smarter about the choices we make. Having tools and information at your fingertips might help bring a bit of motivation to your routine, and of course good tunes and a strong community doesn’t hurt either.

What’s in that cupcake?
Want to know how many calories are in a cupcake, or how much potassium is in a banana? You can now find nutrition information for over 1,000 foods in search - helping you stay informed about what you eat more quickly and easily. While using voice search, on desktop, your iPhone, or Android device you can ask, “how many calories are in a cupcake?” and you can follow-up and ask, “how about a cookie?” without needing to repeat parts of your question. Fruits and vegetables don’t have labels, and it’s often hard to track down the nutritional info for wine or more complex dishes like a burrito, so type or tap the microphone and easily ask your question for these foods and more.

Explore what’s around you, on two wheels
If you want a change of scenery from the gym, use Google Maps on your Android device to find nearby biking routes. Mount your device on your handlebars to see the turn-by-turn directions and navigation, or use speaker-mode to hear voice-guided directions for more than 330,000 miles of trails and paths around the world. Dark green lines on the map show dedicated bike trails and paths without cars, light green lines show streets with dedicated bike lanes, and dashed green lines show other streets recommended for cycling.

Team up to get fit
Looking to get healthy with a friend? Join a Google+ Community and connect with others that share your diet and exercise goals. Check out Communities such as Eating Right and Fitness & Weight Loss for motivation, tips and inspiration to keep you on track. Use Hangouts On Air to learn what experts like The Biggest Loser are saying about nutrition or jump into a yoga class.

Don’t stop the music
A good beat will keep you moving and motivated. Sign up for All Access, our new music subscription service, and you can listen to millions of songs from Google Play Music. Build an awesome workout mix or start a radio station from your favorite pop song like “We Can’t Stop!” Miley Cyrus says it best.

Keep track—no matter which device you’re on
Counting calories? Apps such as Diet Diary can be easily accessed through Chrome or on your mobile device—that way it’s with you when it‘s on your mind. If spreadsheets are more your style, try one of several Google Docs templates, like this weekly meal planner.

Get inspired by the pros
Need a little more motivation? Why not watch fitness gurus do their thing on YouTube: you can watch Sadie Nardini and her amazing yoga classes, or Cassey Ho will get you in top shape for summer - all in the comfort of your own living room.

Posted by Roya Soleimani, Google Search team

Play Cube Slam face-to-face against your friends

My friends and I used to play videogames all the time, squashed together on the couch, engaged in structured intellectual discourse about exactly how badly we were going to destroy each other. Now that we live spread out around the world, it’s a bit harder to dance in each other’s faces and yell “booyah!” every time we win a game. Enter: Cube Slam.

Cube Slam is a video game that you can play face-to-face against your friends. It’s a Chrome Experiment built using WebRTC, an open web technology that lets you video chat right in the browser without installing any plug-ins. That means you can quickly and easily play Cube Slam with your friends, no matter where they are in the world, just by sharing a link.

To win Cube Slam, hit the cube against your friend’s screen three times until the screen explodes. Shields, obstacles, and gravity fields change with every new level, and you can unlock power-ups including fireballs, lasers, multi-balls, mirrored controls, bulletproof shields, fog, ghost balls, time bombs, resized paddles, extra lives and death balls––though you might want to avoid the death balls. If none of your friends are online, you can always play against Bob the Bear and see what level you can reach. If you install the Cube Slam app, you can even play Bob when you’re offline.

Cube Slam’s graphics are rendered in WebGL and CSS 3D, and its custom soundtrack is delivered dynamically through Web Audio. WebRTC, which enables the two-person game, is available on desktop Chrome and Chrome OS, and will be available on mobile later this year. In the meantime, you can play Cube Slam against Bob the Bear on your phone or tablet. To learn more about what’s going on under the hood, see our technology page and Chromium blog post.

Play a friend. Play a bear. Have fun!

Posted by Clem Wright, Google Creative Lab, Ursine Diversion Division

Improving Photo Search: A Step Across the Semantic Gap

Posted by Chuck Rosenberg, Image Search Team

Last month at Google I/O, we showed a major upgrade to the photos experience: you can now easily search your own photos without having to manually label each and every one of them. This is powered by computer vision and machine learning technology, which uses the visual content of an image to generate searchable tags for photos combined with other sources like text tags and EXIF metadata to enable search across thousands of concepts like a flower, food, car, jet ski, or turtle.

For many years Google has offered Image Search over web images; however, searching across photos represents a difficult new challenge. In Image Search there are many pieces of information which can be used for ranking images, for example text from the web or the image filename. However, in the case of photos, there is typically little or no information beyond the pixels in the images themselves. This makes it harder for a computer to identify and categorize what is in a photo. There are some things a computer can do well, like recognize rigid objects and handwritten digits. For other classes of objects, this is a daunting task, because the average toddler is better at understanding what is in a photo than the world’s most powerful computers running state of the art algorithms.

This past October the state of the art seemed to move things a bit closer to toddler performance. A system which used deep learning and convolutional neural networks easily beat out more traditional approaches in the ImageNet computer vision competition designed to test image understanding. The winning team was from Professor Geoffrey Hinton’s group at the University of Toronto.

We built and trained models similar to those from the winning team using software infrastructure for training large-scale neural networks developed at Google in a group started by Jeff Dean and Andrew Ng. When we evaluated these models, we were impressed; on our test set we saw double the average precision when compared to other approaches we had tried. We knew we had found what we needed to make photo searching easier for people using Google. We acquired the rights to the technology and went full speed ahead adapting it to run at large scale on Google’s computers. We took cutting edge research straight out of an academic research lab and launched it, in just a little over six months. You can try it out at photos.google.com.

Why the success now? What is new? Some things are unchanged: we still use convolutional neural networks -- originally developed in the late 1990s by Professor Yann LeCun in the context of software for reading handwritten letters and digits. What is different is that both computers and algorithms have improved significantly. First, bigger and faster computers have made it feasible to train larger neural networks with much larger data. Ten years ago, running neural networks of this complexity would have been a momentous task even on a single image -- now we are able to run them on billions of images. Second, new training techniques have made it possible to train the large deep neural networks necessary for successful image recognition.

We feel it would be interesting to the research community to discuss some of the unique aspects of the system we built and some qualitative observations we had while testing the system.

The first is our label and training set and how it compares to that used in the ImageNet Large Scale Visual Recognition competition. Since we were working on search across photos, we needed an appropriate label set. We came up with a set of about 2000 visual classes based on the most popular labels on Google+ Photos and which also seemed to have a visual component, that a human could recognize visually. In contrast, the ImageNet competition has 1000 classes. As in ImageNet, the classes were not text strings, but are entities, in our case we use Freebase entities which form the basis of the Knowledge Graph used in Google search. An entity is a way to uniquely identify something in a language-independent way. In English when we encounter the word “jaguar”, it is hard to determine if it represents the animal or the car manufacturer. Entities assign a unique ID to each, removing that ambiguity, in this case “/m/0449p” for the former and “/m/012x34” for the latter. In order to train better classifiers we used more training images per class than ImageNet, 5000 versus 1000. Since we wanted to provide only high precision labels, we also refined the classes from our initial set of 2000 to the most precise 1100 classes for our launch.

During our development process we had many more qualitative observations we felt are worth mentioning:

1) Generalization performance. Even though there was a significant difference in visual appearance between the training and test sets, the network appeared to generalize quite well. To train the system, we used images mined from the web which did not match the typical appearance of personal photos. Images on the web are often used to illustrate a single concept and are carefully composed, so an image of a flower might only be a close up of a single flower. But personal photos are unstaged and impromptu, a photo of a flower might contain many other things in it and may not be very carefully composed. So our training set image distribution was not necessarily a good match for the distribution of images we wanted to run the system on, as the examples below illustrate. However, we found that our system trained on web images was able to generalize and perform well on photos.

A typical photo of a flower found on the web.

A typical photo of a flower found in an impromptu photo.

2) Handling of classes with multi-modal appearance. The network seemed to be able to handle classes with multimodal appearance quite well, for example the “car” class contains both exterior and interior views of the car. This was surprising because the final layer is effectively a linear classifier which creates a single dividing plane in a high dimensional space. Since it is a single plane, this type of classifier is often not very good at representing multiple very different concepts.

3) Handling abstract and generic visual concepts. The system was able to do reasonably well on classes that one would think are somewhat abstract and generic. These include "dance", "kiss", and "meal", to name a few. This was interesting because for each of these classes it did not seem that there would be any simple visual clues in the image that would make it easy to recognize this class. It would be difficult to describe them in terms of simple basic visual features like color, texture, and shape.

Photos recognized as containing a meal.

4) Reasonable errors. Unlike other systems we experimented with, the errors which we observed often seemed quite reasonable to people. The mistakes were the type that a person might make - confusing things that look similar. Some people have already noticed this, for example, mistaking a goat for a dog or a millipede for a snake. This is in contrast to other systems which often make errors which seem nonsensical to people, like mistaking a tree for a dog.

Photo of a banana slug mistaken for a snake.

Photo of a donkey mistaken for a dog.

5) Handling very specific visual classes. Some of the classes we have are very specific, like specific types of flowers, for example “hibiscus” or “dhalia”. We were surprised that the system could do well on those. To recognize specific subclasses very fine detail is often needed to differentiate between the classes. So it was surprising that a system that could do well on a full image concept like “sunsets” could also do well on very specific classes.

Photo recognized as containing a hibiscus flower.

Photo recognized as containing a dahlia flower.

Photo recognized as containing a polar bear.

Photo recognized as containing a grizzly bear.

The resulting computer vision system worked well enough to launch to people as a useful tool to help improve personal photo search, which was a big step forward. So, is computer vision solved? Not by a long shot. Have we gotten computers to see the world as well as people do? The answer is not yet, there’s still a lot of work to do, but we’re closer.

Art, Copy & Code: sending kisses around the world

Thanks to modern technology you can connect with your loved ones by sending a quick note, a photo of your cat, even a smile :) around the world in seconds. But one of humanity’s most iconic forms of communication—the kiss—has been left out in the cold. Now, though, you can send a kiss to anyone, anywhere in the world, through Burberry Kisses, a new campaign from Burberry and Google. And not just any kiss, but your kiss.

To get started, simply visit kisses.burberry.com and pucker up in front of your webcam (this works best on Chrome). Using unique kiss-detection technology, the site will detect the outline of your actual lips, which you can choose to dress up with a Burberry lipstick color. If you’re using your touch screen mobile or tablet, you can actually kiss your screen (you might want to wipe it off first) and your lip outline will be taken from there. After that, write a short message and send it to someone from your Google+ friends list or via email. Then sit back and see the envelope with your message fly from your city to the receiver’s destination across a 3D landscape. The receiver gets an email, from which they can see the same journey, read your message and hopefully respond with a kiss of their own.



For an example, see this message I sent to my mom this morning. All the kisses being sent around the world can be seen in a real-time interactive map, capturing the story of the world’s love. You don’t have to kiss and tell: all kisses are private unless you choose to share.

Burberry Kisses is the latest campaign in our Art, Copy & Code project, an ongoing series of brand partnerships to re-imagine how brands tell stories in a connected world. With this project, we’ve tried to create a beautiful experience that comes to life across all screens, and helps connect you to the people who are important to you, wherever they are. For more details on the campaign, see our agency blog or visit our website.

Posted by Aman Govil, Art, Copy & Code Project Lead

Asking the U.S. government to allow Google to publish more national security request data

This morning we sent the following letter to the offices of the Attorney General and the Federal Bureau of Investigation. Read the full text below. -Ed.

Dear Attorney General Holder and Director Mueller

Google has worked tremendously hard over the past fifteen years to earn our users’ trust. For example, we offer encryption across our services; we have hired some of the best security engineers in the world; and we have consistently pushed back on overly broad government requests for our users’ data.

We have always made clear that we comply with valid legal requests. And last week, the Director of National Intelligence acknowledged that service providers have received Foreign Intelligence Surveillance Act (FISA) requests.

Assertions in the press that our compliance with these requests gives the U.S. government unfettered access to our users’ data are simply untrue. However, government nondisclosure obligations regarding the number of FISA national security requests that Google receives, as well as the number of accounts covered by those requests, fuel that speculation.

We therefore ask you to help make it possible for Google to publish in our Transparency Report aggregate numbers of national security requests, including FISA disclosures—in terms of both the number we receive and their scope. Google’s numbers would clearly show that our compliance with these requests falls far short of the claims being made. Google has nothing to hide.

Google appreciates that you authorized the recent disclosure of general numbers for national security letters. There have been no adverse consequences arising from their publication, and in fact more companies are receiving your approval to do so as a result of Google’s initiative. Transparency here will likewise serve the public interest without harming national security.

We will be making this letter public and await your response.

David Drummond
Chief Legal Officer

90 ideas to change the world: Announcing Google Science Fair Finalists

Many great scientists developed their curiosity for science at an early age and in January we called on the brightest young minds from around the world to send us their ideas to change the world. Our 2013 Google Science Fair attracted an exciting and diverse range of entries, with thousands of submissions from more than 120 countries.

After a busy few months for the judges, we’re ready to reveal our 90 regional finalists for the 2013 Google Science Fair. It was no easy task selecting these projects, but in the end their creativity, scientific merit and global relevance shined through. This year’s finalists projects range from using banana peels to produce bioplastics to research into a green treatment for contaminated water. Other projects include a study on the effects of video gaming on the cognitive function of the brain and the evaluation of wireless transmission of electricity.

The 90 Regional Finalists come from all over the world.

For the second year, we’ll also be recognizing the Scientific American Science in Action Award. This award honors a project that makes a practical difference by addressing an environmental, health or resources challenge. From the 90 finalists’ projects, 15 were nominated for this year’s award.

On June 27 we’ll announce the 15 global finalists and the winner of the Science in Action Award. These young scientists will then be flown to Google’s California headquarters for the last round of judging and a celebratory event on September 23.

Thank you to everyone who submitted a project—we really appreciate all your hard work. Congratulations to our 90 regional finalists!

Posted by Sam Peter, Google Science Fair team