Data Engineer (Databricks)
The person in this role will work with project teams developing a modern data platform in a cloud environment for global brands. They will work in an international environment, specializing in BI & Big Data cloud architecture and the latest technologies in that area, and promote the principles of the DataOps manifesto.
Responsibilities:
- Design, build, and optimize scalable data pipelines in Databricks platform using SQL/Python/Spark
- Developing existing projects in the Microsoft Azure environment to gain valuable insights for the business.
- Ensuring that data is modelled and processed according to the architecture and both functional and non-functional requirements
- Planning and implementing processing pipelines for structured and unstructured data (e.g. video and images).
- Working to automate and optimize internal processes in Azure.
- Collaborating with cross-functional and international teams both internally and externally.
Requirements:
- Minimum 3 years' experience working with Databricks platform (Delta Lake, Workflows/Jobs, DLT, Unity Catalog)
- Strong experience with programming in Python
- Solid understanding of SQL and relational databases.
- Knowledge of Data Warehouse, Business Intelligence and ETL/ELT data processing
- Familiarity with Medallion Architecture
- Very good knowledge of English (particular emphasis on written English)
- Proactive approach to tasks, problem-solving attitude and critical thinking skills.
- Flexibility, independence and responsibility for assigned tasks.
- A constant desire to improve your skills and learn new technologies.
- Knowledge of Azure cloud components for data storage and processing: Azure Data Lake Storage, Azure Event Hubs, Azure Data Factory, SQL Database, Synapse, Fabric
- Experience with other big data technologies such as Hadoop, Hive, Kafka, and Flink would be an asset.
We offer:
- Global projects in clouds — we work with clients from all over the world based on modern cloud technologies
- Certification reimbursement — we fund exams, certifications from Microsoft, AWS, Databricks, Snowflake
- Time to learn — 60 paid hours per year
- Flexible approach — you can choose to work from home or meet at our offices
- Personalized benefits — medical care, subsidized sports packages, language tuition, new employee referral bonus (up to PLN 15,000) as well as annual and media bonus
Polski
English
