I had recently a data engineering challenge: I had to deliver data, wrangled in Spark (personal choice, but since Spark and Databricks are all the hype, why not, right?), into a data warehouse provisioned in Yandex’s own Clickhouse. For those of you who are unaware of it, Clickhouse is an analytical, column-oriented, blazingly fast database, open sourced by their creators. I was, at the moment, unfamiliar with the technology. And if the fun was not enough, I had to dockerize the entire solution, so I had to deal with network communication and APIs. …

Javier Tapia

Data artisan, currently working as a data engineer in a digital transformation consultancy practice.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store