Data Engineer

Job Post Index 23DF7A-Sy##
Location Virtual, US
Description: Description
  • Collaborate and work closely with team to build data platforms.
  • Maintain and manage Hadoop clusters in development and production environments.
  • Assemble large, complex data sets that meet functional/non-functional business requirements.
  • Work with team members and functional leads to understand existing data requirements and validation rules to support moving existing data warehouse workloads into a distributed data platform.
  • Create custom software components (e.g. specialized UDFs) and analytics applications.
  • Employ a variety of languages and tools to marry systems together.
  • Recommend ways to improve data reliability, efficiency and quality.
  • Implement and automate high-performance algorithms, prototypes and predictive models.
Skills: Qualifications
  • College degree or equivalent in technology-related field (computer science, engineering, information technology, etc.)
  • Varying modern technology stack experience, who can adapt to new client environments and learn new technologies quickly.
  • Experience building data pipelines to connect analytics stacks, client data visualization tools and external data sources.
  • Knowledge of cloud and distributed systems principles
  • Experience with Python, R, sh/bash and JVM-based languages including Scala and Java.
  • Experience with Hadoop family languages including Pig and Hive.
  • Experience with high performance data libraries including Spark, NumPy and TensorFlow.
  • Experience building stream-processing applications using Storm, Spark-Streaming, Kafka and MQ.
  • Ability to manually and programmatically interact with relational and NoSQL databases
  • Intermediate SQL programming and query performance tuning techniques
  • 1+ years in data engineering
  • Practical education or experience in a technology related field – could include bootcamps and focused educational programs
  • Project experience in delivering technology solutions in a team-based environment, ideally Agile development

Apply For This Job

  • Accepted file types: docx, doc, pdf, pages, txt.