Data Engineer
Petrel AI
1.000.000 to 1.500.000 KZT per month
Almaty
Responsibilities:
• Real-time data ingestion
• Building fault tolerant distributed systems for large datasets
Requirements:
• Proficient understanding of distributed computing principles
• Management of Hadoop cluster, with all included services; ability to solve any ongoing issues with operating Hadoop clusters
• Proficiency with Hadoop 2.x, MapReduce, HDFS
• Experience with building stream-processing systems, using solutions such as Storm or Spark-Streaming
• Good knowledge of Big Data querying tools, such as Pig, Hive and Impala
• Experience with Spark, Jenkins, Sqoop, HDInsight, U-SQL, Java, Python, JSON, Linux
• Experience with integration of data from multiple data sources and working with large datasets
• Experience with NoSQL databases
• Knowledge of various ETL techniques and frameworks Extract Transform Load
• Experience with various messaging systems Apache Kafka
• Experience with Big Data ML toolkits
• Good understanding of Lambda Architecture, along with its advantages and drawbacks
• Advanced data analysis skills including advanced SQL query capabilities
• Deep experience in data modeling, data analysis and relational database design
• Ability to work both independently and as part of a team; ability to work under pressure and handle multiple projects and deadlines
Optional:
• Previous work experience in industrial company with IoT
Contacts:
a.murzabayev@petrelai.com
+77019959055
Petrel AI
1.000.000 to 1.500.000 KZT per month
Almaty
Responsibilities:
• Real-time data ingestion
• Building fault tolerant distributed systems for large datasets
Requirements:
• Proficient understanding of distributed computing principles
• Management of Hadoop cluster, with all included services; ability to solve any ongoing issues with operating Hadoop clusters
• Proficiency with Hadoop 2.x, MapReduce, HDFS
• Experience with building stream-processing systems, using solutions such as Storm or Spark-Streaming
• Good knowledge of Big Data querying tools, such as Pig, Hive and Impala
• Experience with Spark, Jenkins, Sqoop, HDInsight, U-SQL, Java, Python, JSON, Linux
• Experience with integration of data from multiple data sources and working with large datasets
• Experience with NoSQL databases
• Knowledge of various ETL techniques and frameworks Extract Transform Load
• Experience with various messaging systems Apache Kafka
• Experience with Big Data ML toolkits
• Good understanding of Lambda Architecture, along with its advantages and drawbacks
• Advanced data analysis skills including advanced SQL query capabilities
• Deep experience in data modeling, data analysis and relational database design
• Ability to work both independently and as part of a team; ability to work under pressure and handle multiple projects and deadlines
Optional:
• Previous work experience in industrial company with IoT
Contacts:
a.murzabayev@petrelai.com
+77019959055
Комментарии