Facebook is using machine learning to self-tune its myriad services

« Back

Facebook is using machine learning to self-tune its myriad services 06.29.2018 18:12

Regardless of what you may think of Facebook  as a platform, they run a massive operation, and when you reach their level of scale you have to get more creative in how you handle every aspect of your computing environment.

Engineers quickly reach the limits of human ability to track information, to the point that checking logs and analytics becomes impractical and unwieldy on a system running thousands of services. This is a perfect scenario to implement machine learning, and that is precisely what Facebook has done.

The company published a blog post today about a self-tuning system they have dubbed Spiral. This is pretty nifty, and what it does is essentially flip the idea of system tuning on its head. Instead of looking at some data and coding what you want the system to do, you teach the system the right way to do it and it does it for you, using the massive stream of data to continually teach the machine learning models how to push the systems to be ever better.

In the blog post, the Spiral team described it this way: “Instead of looking at charts and logs produced by the system to verify correct and efficient operation, engineers now express what it means for a system to operate correctly and efficiently in code. Today, rather than specify how to compute correct responses to requests, our engineers encode the means of providing feedback to a self-tuning system.”

They say that coding in this way is akin to declarative code, like using SQL statements to tell the database what you want it to do with the data, but the act of applying that concept to systems is not a simple matter.

“Spiral uses machine learning to create data-driven and reactive heuristics for resource-constrained real-time services. The system allows for much faster development and hands-free maintenance of those services, compared with the hand-coded alternative,” the Spiral team wrote in the blog post.

If you consider the sheer number of services running on Facebook, and the number of users trying to interact with those services at any given time, it required sophisticated automation, and that is what Spiral is providing.

The system takes the log data and processes it through Spiral, which is connected with just a few lines of code. It then sends commands back to the server based on the declarative coding statements written by the team. To ensure those commands are always being fine-tuned, at the same time, the data gets sent from the server to a model for further adjustment in a lovely virtuous cycle. This process can be applied locally or globally.

The tool was developed by the team operating in Boston, and is only available internally inside Facebook. It took lots of engineering to make it happen, the kind of scope that only Facebook could apply to a problem like this (mostly because Facebook is one of the few companies that would actually have a problem like this).


- Это интересно
How much does consulting cost?
News
02.10.18
Venture and growth investors are doing a lot to speed up the rise of these worker-bots. So far this year, they’ve poured hundreds of millions into developers of robotic process automation technology, the term to describe software used for performing a series of tasks previously carried out by humans.
02.10.18

What if blockchain turned out to be just what emerging economies were after?

 

02.10.18

While Clinton certainly did not appear to be a Shingy-esque blockchain evangelist onstage, he delivered a targeted amount of enthusiasm about new technologies like blockchain and artificial intelligence in enhancing accessibility and shaping the country’s economic future.

02.10.18
It might be the only way we can break Facebook’s hold on our lives, it could cement Apple’s reputation as a privacy-minded service provider, and more importantly, it wouldn’t be that hard to get people to use it.
02.10.18
Instagram’s co-founders announced that Adam Mosseri, the platform’s vice president of product, is now in charge. Mosseri will oversee all functions of the business and recruit a new executive team, Kevin Systrom and Mike Krieger said in a statement. The co-founders announced their departure last week — more than six years after Facebook purchased the company for $1 billion. “We remain excited for the future of Instagram in the coming years as we transition from being leaders at Instagram to being just two users in a billion,” said Systrom and Krieger.
Address:
2288 Homecrest Avenue 2FL Brooklyn NY 11229