Data Engineer (Azure), remote work
Opublikowano 1 dzień temu
We are currently recruiting for our client for the position of: Data Engineer (Azure).
Sector: energy sector
Your responsibilities::
- Build and run Azure based Data Platform.
- Handle multiple Terabytes of different domains data with focus on IoT data using Stream and Batch processing.
- Craft and maintain world-class, high traffic analytical applications and services with a team of experienced data engineers and scientists.
- Develop data models and process raw data at scale to make it available for analysis.
- Build scalable data pipelines using state-of-the-art technologies in the Microsoft Azure Cloud.
- Implement, run and deploy algorithms in cooperation with data scientists using the Hadoop ecosystem (e.g.Spark, Hive, HDFS) and the Microsoft Azure Stack).
What we expect:
- Certification University degree in Computer Science or related fields of study (Mathematics, Physics).
- At least 5 Years of experience in architecting and building Data Lake and Enterprise Analytics Solutions.
- Ability to optimize 'Big Data’ data pipelines, design Data Lake architecture and data sets.
- Experience with Design and Architecture of relational SQL and NoSQL databases.
- Advanced hands-on SQL, Python, pySpark and HDFS knowledge and experience working with relational databases for data querying and retrieval.
- Experience with Design and Architecture of Azure big data frameworks: Azure Databricks, Azure Data Factory, Azure ML, SQL Data Warehouse, HDInsights, Stream Analytics, Azure Data Explorer – or their open source counterparts: Kafka, Spark, Blob Storages.
- Knowledge of DevOps Processes (including CI/CD) and infrastructure as a code fundamentals.
- Knowledge of Data Quality issues and methods of dealing with them.
- Nice to have: experience in designing, building and integrating REST APIs.
- Fluent in English, German language will be considered as a plus.
Offered:
- 180 zł/h (B2B),
- Remote work,
- Professional development in a large corporation,
- A work-life balance preserved,
- Medical Care, Sports packages.
