NorthBay Solutions is looking for a highly skilled and motivated Senior Big Data Engineer (AWS+Python+Hadoop) with solid experience in data engineering and a passion for Big Data. The ideal candidate will have experience of over 3+ years to 6 years specifically in Data Engineering.
You will have the chance to join a global, multi-cultural team of Data Engineers, and be part of the growth and development of it, participate in a large variety of exciting and challenging projects and work with our exceptional team of Technology professionals.
Technology Stack Used & Required Knowledge:
- Over 3+ years of experience on ETL Data Engineer, Python, Java, Bigdata (Hadoop, spark, hive)
- Excellent understanding of ETL cycle.
- Experience in creating and driving large-scale ETL pipelines in AWS-based environment
- Experience with integration of data from multiple data sources.
- Good knowledge and understanding of Big data engineering, ETL, Data Modeling, and Data Architecture.
- Strong software development and programming skills with a focus on data using Python/ PySpark and/or Scala for data engineering.
- Experience and understanding of various core AWS services such as IAM, Cloud Formation, EC2, S3, EMR/Spark, Glue, Lambda, Athena, and Redshift will be a plus
- Understanding of data types/ handling of different data models.
- Experience with the AWS data management tools such as Data lake and Databricks or AWS Snowflake is a plus
- Understanding of descriptive and exploratory statistics, predictive modelling, evaluation metrics, decision trees, machine learning algorithms is a plus
- Good scripting and programming skills.
- Should have experience in Jira and Agile development methodologies.
- Must be open to learn and the ability to grasp new technologies quickly.
- Be motivated and passionate about technology.
- Team player with strong integrity and professionalism who can work well with others in the group.
- Self-starter with a proven ability to take initiative and strong problem-solving skills.
To keep it short, below are some key responsibilities:
- Design and develop using Big Data technologies.
- Design and develop cutting edge Big Data, and Cloud technologies to build critical, highly complex distributed systems from scratch
- Design, develop and manage complex ETL jobs and pipelines.
- Act as a team member, consulting other teammates as your new systems transform the way this entire firm leverages data
- Deploying data models into production environments. This entails providing the model with data stored in a warehouse or coming directly from sources, configuring data attributes, managing computing resources, setting up monitoring tools, etc.
- Support and helping them learn and improve their skills, as well as innovate and iterate on best practices.
- Solve complex issues and provide guidance to the team members when needed.
- Make improvement and process recommendations that have an impact on the business
About NorthBay
NorthBay, an 8+ Year Award Winning AWS Premier Consulting Partner, is an Global Analytics, DB/App Migration, App Dev and DevOps professional services company. NorthBay's intense focus is unparalleled in the areas of Big Data (Data Lakes, Data Platforms, Enterprise Data Warehouse), Database & Application Migrations and Machine Learning/AI. NorthBay has earned the Analytics, DevOps, Mobile; Education Competency and is also a Public Sector Partner and we are also certified for AWS Well Architected Reviews and Database Freedom Migrations.
What We Offer
· Consulting environment with a hyper-growth company
· Multi-cultural environment
· An opportunity to solve complex business challenges and problems for large corporations
· International Exposure and Experience.
· Competitive salary and benefits
· Paid holidays and vacations
· Medical OPD & Health Insurance
· Fuel Allowance
· Employee Life Insurance
· GYM/Health Club Fee Reimbursement
· Mobile & Internet Allowance
· Opportunities to make a difference in a highly productive environment
