Report this job

---

What is the problem? *

Incorrect company
Incorrect location
Job is expired
Job may be a scam
Other

Job Title: Data/AI EngineerLocation: NYC(Onsite) Duration: Long Term Job Description: The focus of this role is to assist in the utilization of the Full Next Gen OSS Technology stack of Elasticsearch, Kubernetes, Kafka, StreamSets, Spark, Hadoop, Hive and Micro-services developed in Java. Role Responsibilities: Work as part of an agile team to perform the tasks allocated. Attend daily stand-up meetings to provide status. Participate in Sprint planning activities and provide estimates. Be proactive, raise issues quickly and collaborate with the team. Utilize reporting to course correct as projects progress Answer ad hoc requests for information on short timeline. Requirements and Experience: Work in a software lifecycle-based environment providing requirement analysis, architecture design, version control, milestones, testing, release iteration and deployment. Develop and implement infrastructure, configuration management and deployment of solutions using the following or similar infrastructure automation and software development technologies which may include Elastic, Ansible, Puppet, Chef, Git, Azure DevOps/GitHub/Gitlab/Bitbucket. Develop solutions following continuous integration, continuous delivery, and automation projects practices. Design and deploy efficient high-volume analytics systems for multi-terabyte datasets. Design and develop data pipeline applications that can move and transform large quantities of data. Design solutions for Hadoop-based machine learning platforms to analyze data across unlinked systems. Effectively collaborate with a multi-team/work stream environment comprised of customer resources and subcontractors. Understand the data schema and work on resolving data quality issues. Experience working for an IT systems integrator in a customer-facing role Work efficiently with limited supervision with all levels of staff in a high activity / fast paced environment. Ability to motivate teams to produce quality materials within tight timeframes while simultaneously managing various responsibilities. Strong interpersonal skills, ability to debate, dialogue, negotiate, influence and work with both internal and external stakeholders collaboratively and constructively. Good understanding of Big Data concepts and experience with GitHub and CI/CD tools Experience working on Micro-service design, Java Spring framework, Kafka, RabbitMQ. Experience with writing and optimizing SQL Queries Experience with Programming languages such as Scala, Java, Python. Experience working on highly scalable, highly concurrent and low latency systems Experience in Data processing in Batch and Real time with design and implementation using Pyspark/Spark Scala. Experience with Spark framework and related tools (PySpark, Scala, SparkR, Spark SQL, Spark UI) Experience in event driven architecture design/solution using Kafka. Strong expertise in SQL/HQL with Hive/Impala experience Experience in orchestration tools like Airflow