- Many of the largest tech companies are hiring for a new, in-demand role.
- Called the analytics engineer, the position has become one of the hottest jobs in tech.
- Analytics engineers use a tool called dbt and can earn about $ 127,000, according to Glassdoor.
In the past few years, a new role in the world of data analytics has become one of the hottest jobs in tech.
While data analysts and data scientists previously had to go through incredibly messy data sets to report insights, a new role called the analytics engineer is tasked with cleaning up that data and making it more usable for them.
Analytics engineers are in demand at some of the largest companies – Apple, Amazon,
, and Instacart are all hiring analytics engineers in some form. They specialize in using dbt, a tool to make it easier to build cleaner data sets that analysts can use.
“You can change your LinkedIn title from analyst to analytics engineer, and you instantly get a 20% pay bump and a bunch of job offers,” said Jeremiah Lowin, CEO of Prefect, a data-orchestration startup. “That was enabled in large part by dbt. It is a phenomenal example of category creation, which companies suck at these days.”
Glassdoor, which crowdsources salaries for specific roles, suggests the typical salary for an analytics engineer is about $ 127,000. But it can vary – a job listing for a senior analytics engineer at Discord suggests the role pays more than $ 200,000 a year, while one from CircleCI says it pays $ 106,250 to $ 125,000. (Companies must list salary bands in New York and Colorado.)
While data analysts are primed to jump into this new position, Tristan Handy, the CEO of Dbt Labs, said the roles are substantially different. And while analytics engineers take on some tasks traditionally performed by data engineers, it’s not just a rebrand.
“Along with the different job responsibilities come different career ladders and, most importantly, different compensation,” Handy said. “Data professionals who make this shift are compensated extremely well for having done so.”
One startup, Dbt Labs, is responsible for popularizing the analytics engineer
A Philadelphia startup founded in 2016 is essentially responsible for the emergence of the analytics engineer.
Dbt Labs, which created the tool critical to the role, is valued at $ 4.2 billion. Analysts can use dbt to translate complex, messy data sets into cleaner, easier-to-use tables in Snowflake or Redshift.
Data engineers have traditionally used Spark or Python to move data from multiple sources and build tables in Snowflake and other SQL warehouses. Rather than learning Python, data analysts could use dbt and the language they know best, SQL, to do what the data engineers do.
In the past few years dbt has caught the eye of larger companies, which are increasingly moving data to warehousing tools like Snowflake.
“In late 2018, early 2019 we started to see a real identity emerge around analysts who were frustrated with waiting in long data-engineering queues so started to learn the practices necessary to just do the work themselves,” Handy said. “Dbt was a part of this journey for many people, but at the time there was also a rise in companies attempting to build dbt-like tools in-house to empower analysts and people looking for a new name for what they were doing.”
Bob Muglia, a former CEO of Snowflake, said the increasing prevalence of SQL has made tools like dbt – and the analytics-engineer role – substantially more in demand.
“Dbt is winning in that space,” he said. “They have definitely built the biggest community.”
Analytics engineers are also often asked to have experience with Airflow, a popular open-source big-data tool to schedule data-crunching tasks. Astronomer, the startup that helps manage Airflow, said it’s making it easier to use for analytics engineers.
“Traditionally, Airflow was great for people from more traditional data-engineering backgrounds, but we see this emergence of people who are less technical but want to crunch data, do analysis and model building,” said Laurent Paris, Astronomer’s senior vice president of research. and development. “And we want to cater to them even with Airflow.”
Though dbt is extremely popular and has essentially crafted a new role, its creator wants to expand beyond cleaning and organizing data using SQL – Handy said he expects the role to grow as dbt launches new tools like tracking essential metrics.