Principal/Lead Databricks Engineer – Fully Remote – 12‑Month Contract

About the role

We are looking for a Principal/Lead Databricks Engineer to join a global technology consultancy on a large‑scale data transformation project. This fully remote, 12‑month contract role will involve designing and delivering scalable data platforms using modern cloud and AI technologies.

Key responsibilities

• Build and optimise Databricks and Apache Spark solutions.
• Develop scalable data pipelines and transformation workflows.
• Write clean, maintainable Python code.
• Improve data platform performance and reliability.
• Collaborate with technical teams and stakeholders.
• Support best practices around code quality and documentation.

Required profile

• 5+ years of experience in software or data engineering.
• Strong experience with Databricks and Apache Spark.
• Proficiency in Python.
• Experience with Delta Lake, Delta Tables and Parquet.
• Solid understanding of data processing and ETL workflows.
• Experience working with cloud‑based data platforms.

Required skills

• Databricks
• Apache Spark
• Python
• Delta Lake
• Delta Tables
• Parquet
• Azure cloud
• Azure Synapse / Fabric
• FastAPI
• Docker
• Kubernetes
• Microservices architecture