(USA) Senior Data Engineer, E2E
Walmart
Job description
What you'll do at
Position Summary...
What you'll do...
We are looking for a Senior Data Engineering to build robust data pipelines and services which support the scaling and deployment of cutting-edge machine learning and deep learning modeling works for our Retail Intelligence team.
About The Team
The Retail Intelligence team is leading the AI strategy for the company and is specifically charged with intercepting, redesigning, and reframing our end-to-end systems to accelerate customer retention and spend while improving our associates' productivity and turnover. We will reframe business challenges into ML problems, rapidly prototype new algorithms in a toy-scale garage setting, iterate to acceptance with the business, design and execute experiments to test for viability of our models, and work with Tech to productionalize and scale the end-state solution.
At Walmart, we help people save money, so they can live better. This mission serves as the foundation for every decision we make and drives us to create the future of retail. We can't do that without the best talent - talent that is innovative, curious, and driven to create exceptional experiences for our customers.
As part of Retail Intelligence Team, this role will be an enabler of our journey to be the world's leading data-driven retailer. We are seeking an individual who will be responsible to establish robust data pipelines and services. We are looking for a highly motivated, resourceful, team-oriented individual to drive the data engineering process. Our ideal candidate is an exceptionally talented data engineer with an outstanding track record of working with very large data sets and building robust data pipelines. You will be working with structured/unstructured Data sets, building large scale Data processing platforms, implementing world class data governance and operational controls, solving complex performance challenges. This role will report up to the Director of Data Science.
What You'll Do:
• Understanding Business Context Requires knowledge of: Industry and environmental factors; Common business vernacular; Business practices across two or more domains such as product, finance, marketing, sales, technology, business systems, and human resources and in-depth knowledge of related practices; Directly relevant business metrics and business areas. To provide recommendations to business stakeholders to solve complex business issues. Develops business cases s for projects with a projected return on investment or cost savings. Translates business requirements into projects, activities, and tasks and aligns to overall business strategy and develops domain specific artifact. Serves as an interpreter and conduit to connect business needs with tangible solutions and results. Identify and recommend relevant business insights pertaining to their area of work.
• Work with business partner, product managers and data scientist closely to design feature-based solutions and implement them in an agile fashion.
• Design, develop and build database to power Big Data analytical systems.
• Design data integration pipeline architecture and ensure successful creation of the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using Spark, DataProc, SQL, Airflow and other technologies.
• Build robust and scalable applications using SQL, Scala/Python and Spark.
• Work on creating data ingestion processes to maintain Global Data lake on Google cloud or Azure
• Engage with architects and senior technical leads to create and enhance complex software components.
• Design, configure and implement systems that can scale to process terabytes of data between heterogeneous systems on premise and cloud.
• Develop proof-of-concept prototype with fast iteration and experimentation.
• Develop and maintain design documentation, test cases, performance and monitoring and performance evaluation.
• Familiar with CI/CD pipeline, perform continuous integration and deployment using Jenkins, Git, and Concord
• Deploy and monitor products on Cloud platforms
• Develop and implement best-in-class monitoring processes to enable data applications meet SLAs
• Guide the team technically for end-to-end solution lifecycle.
What You'll Bring ...
• 3+ years of experience on Big Data development and data pipeline development
• 2+ years of Experience in GCP/Azure cloud platforms
• Demonstrates up-to-date expertise in Data Engineering, complex data pipeline development.
• Architect, design, develop, implement, and tune distributed data processing pipelines that process large volume of data; focusing on scalability, low -latency, and fault-tolerance in every system built.
• Exposure to Data Governance (Data Quality, Metadata Management, Security, etc.)
• Experience with Java, Scala and/or Python to write data pipelines and data processing layers.
• Demonstrates expertise in writing complex, highly optimized queries across large data sets over BigQuery or Presto
• Proven working expertise with Big Data Technologies Spark Scala/PySpark, DataProc and SQL
• Retail experience and knowledge of commercial data is a huge plus.
• Experience with BI Tool Tableau or Looker or PowerBI is a plus.
• Experience on real time data streaming and processing using Kafka and/or Spark streaming is a plus.
The above information has been designed to indicate the general nature and level of work performed in the role. It is not designed to contain or be interpreted as a comprehensive inventory of all responsibilities and qualifications required of employees assigned to this job. The full Job Description can be made available as part of the hiring process.
About Walmart Global Tech
Imagine working in an environment where one line of code can make life easier for hundreds of millions of people. That's what we do at Walmart Global Tech. We're a team of software engineers, data scientists, cybersecurity expert's and service professionals within the world's leading retailer who make an epic impact and are at the forefront of the next retail disruption. People are why we innovate, and people power our innovations. We are people-led and tech-empowered. We train our team in the skillsets of the future and bring in experts like you to help us grow. We have roles for those chasing their first opportunity as well as those looking for the opportunity that will define their career. Here, you can kickstart a great career in tech, gain new skills and experience for virtually every industry, or leverage your expertise to innovate at scale, impact millions and reimagine the future of retail.
Flexible, hybrid work:
We use a hybrid way of working that is primarily in office coupled with virtual when not onsite. Our campuses serve as a hub to enhance collaboration, bring us together for purpose and deliver on business needs. This approach helps us make quicker decisions, remove location barriers across our global team and be more flexible in our personal lives.
Benefits:
Benefits: Beyond our great compensation package, you can receive incentive awards for your performance. Other great perks include 401(k) match, stock purchase plan, paid maternity and parental leave, PTO, multiple health plans, and much more.
Equal Opportunity Employer:
Walmart, Inc. is an Equal Opportunity Employer - By Choice. We believe we are best equipped to help our associates, customers and the communities we serve live better when we really know them. That means understanding, respecting and valuing diversity- unique styles, experiences, identities, ideas and opinions - while being inclusive of all people.
Minimum Qualifications...
Outlined below are the required minimum qualifications for this position. If none are listed, there are no minimum qualifications.
Option 1: Bachelor's degree in Computer Science and 3 years' experience in software engineering or related field. Option 2: 5 years' experience in
software engineering or related field. Option 3: Master's degree in Computer Science and 1 year's experience in software engineering or related
field.
2 years' experience in data engineering, database engineering, business intelligence, or business analytics.
Preferred Qualifications...
Outlined below are the optional preferred qualifications for this position. If none are listed, there are no preferred qualifications.
Data engineering, database engineering, business intelligence, or business analytics, ETL tools and working with large data sets in the cloud, Master's degree in Computer Science or related field and 3 years' experience in software engineering
Primary Location...
702 SW 8TH ST, BENTONVILLE, AR 72716, United States of America
About Walmart
At Walmart, we help people save money so they can live better. This mission serves as the foundation for every decision we make, from responsible sourcing to sustainability-and everything in between. As a Walmart associate, you will play an integral role in shaping the future of retail, tech, merchandising, finance and hundreds of other industries-all while affecting the lives of millions of customers all over the world. Here, your work makes an impact every day. What are you waiting for?
Walmart, Inc. is an Equal Opportunity Employer- By Choice. We believe we are best equipped to help our associates, customers, and the communities we serve live better when we really know them. That means understanding, respecting, and valuing diversity- unique styles, experiences, identities, abilities, ideas and opinions- while being inclusive of all people.
Position Summary...
What you'll do...
We are looking for a Senior Data Engineering to build robust data pipelines and services which support the scaling and deployment of cutting-edge machine learning and deep learning modeling works for our Retail Intelligence team.
About The Team
The Retail Intelligence team is leading the AI strategy for the company and is specifically charged with intercepting, redesigning, and reframing our end-to-end systems to accelerate customer retention and spend while improving our associates' productivity and turnover. We will reframe business challenges into ML problems, rapidly prototype new algorithms in a toy-scale garage setting, iterate to acceptance with the business, design and execute experiments to test for viability of our models, and work with Tech to productionalize and scale the end-state solution.
At Walmart, we help people save money, so they can live better. This mission serves as the foundation for every decision we make and drives us to create the future of retail. We can't do that without the best talent - talent that is innovative, curious, and driven to create exceptional experiences for our customers.
As part of Retail Intelligence Team, this role will be an enabler of our journey to be the world's leading data-driven retailer. We are seeking an individual who will be responsible to establish robust data pipelines and services. We are looking for a highly motivated, resourceful, team-oriented individual to drive the data engineering process. Our ideal candidate is an exceptionally talented data engineer with an outstanding track record of working with very large data sets and building robust data pipelines. You will be working with structured/unstructured Data sets, building large scale Data processing platforms, implementing world class data governance and operational controls, solving complex performance challenges. This role will report up to the Director of Data Science.
What You'll Do:
• Understanding Business Context Requires knowledge of: Industry and environmental factors; Common business vernacular; Business practices across two or more domains such as product, finance, marketing, sales, technology, business systems, and human resources and in-depth knowledge of related practices; Directly relevant business metrics and business areas. To provide recommendations to business stakeholders to solve complex business issues. Develops business cases s for projects with a projected return on investment or cost savings. Translates business requirements into projects, activities, and tasks and aligns to overall business strategy and develops domain specific artifact. Serves as an interpreter and conduit to connect business needs with tangible solutions and results. Identify and recommend relevant business insights pertaining to their area of work.
• Work with business partner, product managers and data scientist closely to design feature-based solutions and implement them in an agile fashion.
• Design, develop and build database to power Big Data analytical systems.
• Design data integration pipeline architecture and ensure successful creation of the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using Spark, DataProc, SQL, Airflow and other technologies.
• Build robust and scalable applications using SQL, Scala/Python and Spark.
• Work on creating data ingestion processes to maintain Global Data lake on Google cloud or Azure
• Engage with architects and senior technical leads to create and enhance complex software components.
• Design, configure and implement systems that can scale to process terabytes of data between heterogeneous systems on premise and cloud.
• Develop proof-of-concept prototype with fast iteration and experimentation.
• Develop and maintain design documentation, test cases, performance and monitoring and performance evaluation.
• Familiar with CI/CD pipeline, perform continuous integration and deployment using Jenkins, Git, and Concord
• Deploy and monitor products on Cloud platforms
• Develop and implement best-in-class monitoring processes to enable data applications meet SLAs
• Guide the team technically for end-to-end solution lifecycle.
What You'll Bring ...
• 3+ years of experience on Big Data development and data pipeline development
• 2+ years of Experience in GCP/Azure cloud platforms
• Demonstrates up-to-date expertise in Data Engineering, complex data pipeline development.
• Architect, design, develop, implement, and tune distributed data processing pipelines that process large volume of data; focusing on scalability, low -latency, and fault-tolerance in every system built.
• Exposure to Data Governance (Data Quality, Metadata Management, Security, etc.)
• Experience with Java, Scala and/or Python to write data pipelines and data processing layers.
• Demonstrates expertise in writing complex, highly optimized queries across large data sets over BigQuery or Presto
• Proven working expertise with Big Data Technologies Spark Scala/PySpark, DataProc and SQL
• Retail experience and knowledge of commercial data is a huge plus.
• Experience with BI Tool Tableau or Looker or PowerBI is a plus.
• Experience on real time data streaming and processing using Kafka and/or Spark streaming is a plus.
The above information has been designed to indicate the general nature and level of work performed in the role. It is not designed to contain or be interpreted as a comprehensive inventory of all responsibilities and qualifications required of employees assigned to this job. The full Job Description can be made available as part of the hiring process.
About Walmart Global Tech
Imagine working in an environment where one line of code can make life easier for hundreds of millions of people. That's what we do at Walmart Global Tech. We're a team of software engineers, data scientists, cybersecurity expert's and service professionals within the world's leading retailer who make an epic impact and are at the forefront of the next retail disruption. People are why we innovate, and people power our innovations. We are people-led and tech-empowered. We train our team in the skillsets of the future and bring in experts like you to help us grow. We have roles for those chasing their first opportunity as well as those looking for the opportunity that will define their career. Here, you can kickstart a great career in tech, gain new skills and experience for virtually every industry, or leverage your expertise to innovate at scale, impact millions and reimagine the future of retail.
Flexible, hybrid work:
We use a hybrid way of working that is primarily in office coupled with virtual when not onsite. Our campuses serve as a hub to enhance collaboration, bring us together for purpose and deliver on business needs. This approach helps us make quicker decisions, remove location barriers across our global team and be more flexible in our personal lives.
Benefits:
Benefits: Beyond our great compensation package, you can receive incentive awards for your performance. Other great perks include 401(k) match, stock purchase plan, paid maternity and parental leave, PTO, multiple health plans, and much more.
Equal Opportunity Employer:
Walmart, Inc. is an Equal Opportunity Employer - By Choice. We believe we are best equipped to help our associates, customers and the communities we serve live better when we really know them. That means understanding, respecting and valuing diversity- unique styles, experiences, identities, ideas and opinions - while being inclusive of all people.
Minimum Qualifications...
Outlined below are the required minimum qualifications for this position. If none are listed, there are no minimum qualifications.
Option 1: Bachelor's degree in Computer Science and 3 years' experience in software engineering or related field. Option 2: 5 years' experience in
software engineering or related field. Option 3: Master's degree in Computer Science and 1 year's experience in software engineering or related
field.
2 years' experience in data engineering, database engineering, business intelligence, or business analytics.
Preferred Qualifications...
Outlined below are the optional preferred qualifications for this position. If none are listed, there are no preferred qualifications.
Data engineering, database engineering, business intelligence, or business analytics, ETL tools and working with large data sets in the cloud, Master's degree in Computer Science or related field and 3 years' experience in software engineering
Primary Location...
702 SW 8TH ST, BENTONVILLE, AR 72716, United States of America
About Walmart
At Walmart, we help people save money so they can live better. This mission serves as the foundation for every decision we make, from responsible sourcing to sustainability-and everything in between. As a Walmart associate, you will play an integral role in shaping the future of retail, tech, merchandising, finance and hundreds of other industries-all while affecting the lives of millions of customers all over the world. Here, your work makes an impact every day. What are you waiting for?
Walmart, Inc. is an Equal Opportunity Employer- By Choice. We believe we are best equipped to help our associates, customers, and the communities we serve live better when we really know them. That means understanding, respecting, and valuing diversity- unique styles, experiences, identities, abilities, ideas and opinions- while being inclusive of all people.