Azure Data Engineer

Fox IT is a Portuguese consulting company that is, for the 2nd consecutive year, in the TOP of the 5% best SMEs in Portugal. Our Mission is to bring a new approach to the IT market, promoting a fairer exchange between us, our consultants and our customers.

 

We are looking for a Azure Data Engineer to join our team in Portugal.

 

Job description and core responsibilities:

·        Work with the team (technical Lead/Architect/other team members) and customer focal point to understand the business need and design/implement the technical data management solution; 

·        Design and implement Data Solutions based on given requirements, specially using Microsoft Azure cloud stack. Design and implement complex ELT/ETL processes using Integration Services, Azure Data Factory, LogicApps, DataBricks, PySpark, among others;

·        Implement and optimize complex transformations on different data sources such as JSON, XML, EXCEL and CSV Files;

·        Design and implement Data Lakes and Delta Lakes;

·        Improve system performance by conducting tests, troubleshooting and integrating new elements;

·        Design, implement and maintain Azure DevOps pipelines for CI/CD.

 

 

Mandatory Tech Skills:

·         5+ years experience as Data Engineer using Microsoft Azure environments

•       Microsoft Azure platform expertise;

•       Microsoft Azure platform Data services expert knowledge;

•       Proven experience working with Data Factory, Logic Apps, Azure Functions, Databricks, Synapse and Azure Devops;

•       Python, PySPark, Spark SQL, T-SQL expert knowledge and experience • Proven experience working with Data Lakes and Delta lakes;

•       Proven experience writing complex transformations and ingestion pipelines using Data Factory, Databricks and PySpark;

•       Proven experience working with relational database engines such as SQL Server and Oracle;

•       Excellent scripting skills with PowerShell, bash Shell and Python;

•       Proven experience building CI/CD pipelines in Azure using, at least, scripting and ARM templates.

 

Location: 100% remote

Take the Leap!