Azure Data Engineer
Fox IT is a Portuguese consulting company that is, for the 2nd consecutive year, in the TOP of the 5% best SMEs in Portugal. Our Mission is to bring a new approach to the IT market, promoting a fairer exchange between us, our consultants and our customers.
We are looking for a Azure Data Engineer to join our team in Portugal.
Job description and core responsibilities:
· Work with the team (technical Lead/Architect/other team members) and customer focal point to understand the business need and design/implement the technical data management solution;
· Design and implement Data Solutions based on given requirements, specially using Microsoft Azure cloud stack. Design and implement complex ELT/ETL processes using Integration Services, Azure Data Factory, LogicApps, DataBricks, PySpark, among others;
· Implement and optimize complex transformations on different data sources such as JSON, XML, EXCEL and CSV Files;
· Design and implement Data Lakes and Delta Lakes;
· Improve system performance by conducting tests, troubleshooting and integrating new elements;
· Design, implement and maintain Azure DevOps pipelines for CI/CD.
Mandatory Tech Skills:
· 5+ years experience as Data Engineer using Microsoft Azure environments
• Microsoft Azure platform expertise;
• Microsoft Azure platform Data services expert knowledge;
• Proven experience working with Data Factory, Logic Apps, Azure Functions, Databricks, Synapse and Azure Devops;
• Python, PySPark, Spark SQL, T-SQL expert knowledge and experience • Proven experience working with Data Lakes and Delta lakes;
• Proven experience writing complex transformations and ingestion pipelines using Data Factory, Databricks and PySpark;
• Proven experience working with relational database engines such as SQL Server and Oracle;
• Excellent scripting skills with PowerShell, bash Shell and Python;
• Proven experience building CI/CD pipelines in Azure using, at least, scripting and ARM templates.
Location: 100% remote
Take the Leap!