SOFTWARE ENGINEER ( HADOOP ) Job at Spry Info Solutions, INC, Pennsylvania

aFNzWFMxOURvWG8zcTRwSC9talFIWThEVWc9PQ==
  • Spry Info Solutions, INC
  • Pennsylvania

Job Description

Looking for a Software Engineer with knowledge in Hadoop and Spark. Experience with data mining and stream processing technologies (Kafka, Spark Streaming, Akka Streams). Responsibilities: Translate functional and technical requirements into detailed specifications running on AWS using services EC2, ECS, RDS Aurora MySQL, SQS, SNS, KMS, Athena. Migrate current Prometheus and DARQ applications to manage multi-account AWS cloud environment containers, scalable computing platforms Docker (ECS), Kubernetes. Develop Spark code using Scala and Spark-SQL/Streaming for faster testing and processing of data. Create, optimize, and troubleshoot complex SQL queries to retrieve and analyze data from databases such as Redshift, Oracle, MS SQL Server, MySQL, and PostgreSQL. Design ETL transformations and jobs using Pentaho Kettle Spoon designer 5.7.12 and Pentaho Data Integration Designer and schedule them on ETL WFE application Carte Server. Design, code, test, and customize RHQ reports for Market systems data and provide data quality solutions to meet client requirements. Develop various complex queries for different data sources Nasdaq Data Warehouse, Revenue Management System and work on performance tuning of queries. Create scripts for automation processes for data ingestion. Build and deploy artifacts (RPMs) and services using GitLab pipelines to Dev, QC, and Prod AWS accounts in cloud platform. Validate DARQ data, reports and manage document libraries on collaboration site using Confluence. Requirements: The minimum education requirements to perform the above job duties are a Bachelor’s degree in Computer Science, Information Technology, or a related technical field. #J-18808-Ljbffr Spry Info Solutions, INC

Job Tags

Similar Jobs

North Highland

Instructional Designer Job at North Highland

 ...change and transformation consultancy. Simply put, we make change happen like no one else. And we're on the hunt for a new Instructional Designer. HOW WILL YOU MAKE CHANGE HAPPEN Instructional Designers needed to prepare the organization for the... 

Jobot

Locum Radiologist Job at Jobot

 ...Locum Radiologist needed in Beautiful Wisconsin! This Jobot Consulting Job is hosted by: Megan Gordon Are you a fit? Easy Apply now by clicking the "Apply" buttonand sending us your resume. Salary: $100 - $1,000 per hour A bit about us: A Hospital in Wisconsin... 

Aramark

Cook - Medina Hospital - Patient - Cleveland Clinic Job at Aramark

Do you love to cook? Are you passionate about food? As a Cook on our team, youll help bring recipes to life while being able to flex your creativity! Surrounded by fresh ingredients and the best equipment, youll be immersed in a world that goes beyond food preparation...

Concierge Care- Daytona, FL

Certified Nursing Assistant - CNA - Nurses Aide Job at Concierge Care- Daytona, FL

Start this week as a Certified Nursing Assistant - CNA - Nurses Aide! Our response time is usually within a few hours and our goal is to help get you in process and working this week!Pay between $16-20 per hour depending on the location, client, and level of care.... 

Chicago Protective Apparel

Pattern Maker/Sample Maker Job at Chicago Protective Apparel

 ...welding, glass handling, chemical & petroleum, and maintenance industries. What We Do At MWCPA, we are building a world where Safety is foundational, Performance is enhanced, and Trust in the brand is unshakable. We create and elevate protection beyond your...