Databricks Data Engineer/Solution Architect (On site or remote)

Jetfly Group is a leading European private aviation group headquartered in Luxembourg. With over 25 years of experience and more than 500 passionate professionals, we bring together specialized business units offering a complete aviation ecosystem, including fractional ownership, charter services, aircraft management, maintenance, and training.

Based in Luxembourg, Jetfly Aviation is the fractional ownership arm of the Jetfly Group. We operate a fleet of 70 Pilatus aircraft (PC-12 and PC-24), along with the Cirrus SF50, combining Swiss precision and exceptional service to deliver a refined private flying experience,  exclusively for over 450 co-owners across Europe.

We are looking for a Databricks Data Engineer / Solution Architect who can lead the design, implementation, and optimization of our data platform and pipelines. This person will be both a strategic thinker and a doer: designing architectures, standards, best practices, and writing code, building pipelines, integrating data sources, ensuring reliability, performance and security. 

Key Responsibilities 

• Lead the end-to-end design, development, deployment and maintenance of data pipelines using Databricks, ensuring best practices in architecture, performance, scalability, reliability, and cost optimization. 

• Architect solutions for ingestion, transformation, storage and serving layers (e.g. streaming & batch), ensuring they meet business requirements (latency, volume, quality). 

• Collaborate with stakeholders (operations, finance, safety, etc.) to understand business needs and translate them into technical requirement and data models. 

• Define and enforce data governance, data quality, metadata management, security, access controls, lineage. 

• Work with cloud infrastructure (Azure) to provision, monitor and optimize data compute, storage, and networking resources. 

• Implement CI/CD, testing, monitoring, alerting for data pipelines. 

• Mentor and coach more junior engineers, define best practices, code reviews, knowledge sharing. 

• Stay current with developments in Databricks, distributed systems, data engineering trends - propose innovations. 

• Support data architecture decisions: what tools, frameworks, storage formats, etc. are appropriate. 

• Troubleshoot performance issues, data discrepancies, failures; ensure high availability, disaster recovery. 

Technical Skills: 

• Deep hands-on experience with Databricks (or similar platforms like Spark on cloud), including notebooks, jobs, clusters, Delta Lake or equivalent. 

• Strong proficiency in data engineering: building batch & streaming pipelines; ETL/ELT; data modeling; experience with both structured & unstructured data. 

• Solid programming skills in Python, Scala or SQL (at least two). 

• Experience with cloud platforms (ideally Azure), infrastructure as code (Terraform, ARM templates, or similar). 

• Familiarity with data storage technologies: relational databases, NoSQL, data lakes, Delta Lake / Parquet / parquet-like formats. 

• Understanding of distributed computing, partitioning, scaling, performance tuning. 

• Hands-on experience with data governance, data lineage, data quality frameworks. 

• Experience with CI/CD tooling, version control, automated testing, monitoring
  and alerting. 

Soft Skills: 

• Excellent problem-solving mindset: able to decompose complex business problems, propose pragmatic and scalable solutions. 

• Ownership and
  accountability:
  takes responsibility end-to-end. 

• Collaboration and
  teamwork: works well in cross-functional teams, mentoring/knowledge sharing. 

• Adaptability: able to move fast, learn new technologies / approaches, adjust to changing priorities. 

• Detail-orientation: ensuring data quality, reliability, security. 

Nice-to-Have: 

• Experience in aviation or other regulated industries (safety, compliance, operations). 

• Experience with machine learning pipelines or real-time analytics. 

• Knowledge of Luxembourg or European data regulations (GDPR, etc.). 

• Experience with BI tools / data visualization for stakeholders. 

• Certifications in Databricks, cloud providers.