Big Data Software Engineer

Technology · Chicago, Illinois
Department Technology
Employment Type Contractor
Minimum Experience Experienced

SoHo Dragon represents a financial services firm with offices in Chicago, IL that needs to hire a Big Data Software Engineer .


As a Big Data Software Engineer, you will support the development of our new data lake. This is a new build project that will use the Microsoft AZURE Infrastructure to develop and deploy a market leading solution. As member of the Feature Team, you will work in autonomy, which includes the following development tasks:

  • Design, develop and implement technical solutions in Microsoft Azure environments using the Java programming language, Spark, Hive, Avro, Parquet, Kubernetes, Java Spring Boot, Elastic Search, Logstash, Kibana, Oozie, Airflow and PostGres SQL. ‘
  • Work with distributed systems with huge volume of data
  • Good working knowledge troubleshooting performance issue
  • Support deconstruction of customer requests into detailed stories by interacting with the Product Owner
  • Deliver working code that meets acceptance criteria as well as meets the definition of done at different levels
  • Write code, deploy scripts, unit test, check code to source code repository, and monitor delivery pipeline activity to ensure product quality and consistency
  • Conduct testing, deployment, and production activities of the team to ensure production stability, applying the guidelines provided by the chapters
  • Engage in pair programming to write high quality code that’s easy to understand and support
  • Write tests - very often before the associated code - at unit level with Junit, Mockito and in BDD style with Cucumber
  • Attend backlog refinement and planning sessions to discuss and estimate (small, medium, large) upcoming stories

Technical Skills:

  • Experience with Java 8
  • Good working knowledge of distributed systems.
  • Must have working experience with Spark jobs and troubleshooting performance issue.
  • Must have working experience in dealing with high volume of data – batch and realtime processing.
  • Cloud experience is must – AWS or Azure
  • Sound knowledge of Spring Boot or another Java back-end framework, Kafka, Elastic Search, Kibana, & Kubernetes
  • Strong experience with Cloud & Big Data technology (Spark, Hive, Avro, Parquet) & Scala/Kotlin, Airflow
  • Strong experience with relational databases development including Oracle PL/SQL, MS SQL Server.
  • Strong experience with development life cycle products such as SVN, CVS, Maven, Hudson
  • AZURE or AWS Development Environment
  • Designing RESTful APIs and integrating third party RESTful APIs.
  • Working familiarity with code revising and branding, ideally Git.


  • Comfortable working in agile methodologies, ideally Scrum.
  • Experience with automated testing approaches - test driven development, unit testing, integration testing, and BDD testing.
  • Exposure to continuous integration tools.
  • Understanding of service-oriented architectures and message brokers.
  • Strong analytical skills and problem-solving ability. Able to break down complex problems into simple manageable units, develop solutions for each unit, and integrate them back into the whole. 
  • Results oriented, able to set goals and priorities that maximize the use of resources available to consistently deliver quality results.
  • Team-oriented, client-focused and open to different ideas/viewpoints. Self-awareness of own behavior and work style, as well as tolerant of different needs and viewpoints
  • Financial industry experience highly preferable


  • Experience working in a global / international financial IT environment, where there are a broad range of policies and procedures.
  • Possess first-hand experience working in a real Change and Release management environment
  • Experience with business analysis methodologies, processes and documentation standards

Experience Needed:

  • 10 plus years of experience working as Senior Programmer Analyst role in a financial industry environment

Thank You

Your application was submitted successfully.

  • Location
    Chicago, Illinois
  • Department
  • Employment Type
  • Minimum Experience