Airbnb Opens Data, Machine Learning Code

Seeking to be a good host for code, too, Airbnb flung open the doors to several of its internal engineering innovations Thursday, releasing parts of its data infrastructure as well as a machine learning package as open source.

Airflow and Aerosolve join a third resource, Airpal, which Airbnb made open source earlier this year. Airbnb also announced airbnb.io as a destination for all of its open-source projects, supplementing its existing Github page.

Mike Curtis, Airbnb's vice president of engineering, made the announcements at the company's developer-focused OpenAir conference in San Francisco Thursday morning.

For Airbnb, which makes money by operating a marketplace for unconventional lodging in people's homes and apartments, not by selling software, releasing its in-house software projects as open source serves to bolster its image as a technical innovator. There was a lot of talk about the company's "engineering brand" at OpenAir. Allowing others to examine and use its code should, in theory, help the company recruit engineers and retain its current technical employees by publicizing their coding feats.

Unclogging Data Pipelines

Data pipelines are the systems companies use to ingest and move the vast amounts of data their websites, mobile apps, and other systems generate. Airflow allows you to programmatically author, schedule, and monitor data pipelines.

Airflow came about because the company wasn't satisfied with the data-pipeline infrastructure it used beforehand.

"We found that what we were applying was at least one generation behind what we actually needed," said Curtis. "We had created a monster. The data warehouse was kind of a mess."

Now all of Airbnb's data pipeplines use Airflow.

"The data warehouse was kind of a mess," said Airbnb VP of Engineering Mike Curtis at OpenAir 2015 in San Francisco.

Teaching Machines To Learn

Aerosolve represents Airbnb's latest efforts in machine learning, the creation of systems that automate the continuous improvement of algorithms. It powers a new product the company unveiled on Thursday, Price Tips.

Price Tips gives Airbnb hosts recommendations on how to price their listing every day of the year. Curtis said that initial testing of Price Tips indicated that if a host made the price for their listing within five percent of the price recommended, the listing was four times as likely to get booked. That kind of intelligence is what Aerosolve is meant to enable.

Will you use Airflow or Aerosolve? Let us know in the comments.