The Data Engineer - Costco Logistics BI is accountable for the cease-to-cease statistics pipelines to strengthen Costco Logistics reporting. This position is centered on statistics engineering to construct and supply computerized statistics pipelines from a plethora of inner and outside statistics sources. The Data Engineer will companion with product owners, and engineering and statistics platform groups to layout, construct, test, and automate statistics pipelines that might be relied upon throughout the employer because of the unmarried supply of truth.
ROLE - Develops and operationalizes statistics pipelines to make statistics to be had for consumption (Costco Logistics BI).
- Works with statistics architects and statistics/BI engineers to layout statistics pipelines and recommends ongoing optimization of statistics garage, statistics ingestion, statistics best, and orchestration.
- Designs, develops, and implements ETL/ELT methods with the use of IICS (Informatica Cloud).
- Uses MySQL to enhance and accelerate the transport of our statistics merchandise and offerings.
- Uses Azure offerings consisting of Azure SQL DW (Synapse), ADLS, Azure Event Hub, and Azure Data Factory to enhance and accelerate the transport of our statistics merchandise and offerings.
- Implements massive statistics and NoSQL answers through growing scalable statistics processing structures to force high-price insights to the organization.
- Identifies, designs, and implements inner system improvements: automating guide methods, and optimizing statistics transport.
- Identifies approaches to enhance statistics reliability, efficiency, and best of statistics management.
- Communicates technical standards to non-technical audiences each written and verbal.
- Performs peer critiques for different statistics engineer?s work.
REQUIRED - 3+ years? experience engineering and operationalizing statistics pipelines with big and complicated datasets.
- 2+ years? experience with Informatica PowerCenter.
- 2+ years? experience with Informatica IICS.
- 2+ years? experience operating with Cloud technologies; consisting of ADLS, Azure Databricks, Spark, Azure Synapse, Cosmos DB, and different massive statistics technologies.
- 2+ years? experience with Data Modeling, ETL, and Data Warehousing.
- 2+ years? experience imposing statistics integration strategies consisting of event/message primarily based totally integration (Kafka, Azure Event Hub), ETL.
- 2+ years? experience with Git / Azure DevOps.
- Extensive experience operating with numerous statistics sources; SQL, SQL Server database, flat files (CSV, delimited), Web API, XML.
- Advanced SQL skills; Understanding of relational databases, commercial enterprise statistics, and the capacity to put in writing complicated SQL queries towards quite a few statistics sources.
- Strong expertise in database garage standards; Data Lake, Relational Databases, NoSQL, Graph, and Data Warehousing.
- Able to work in a fast-moving agile improvement environment.
Recommended - Microsoft Azure/comparable certifications.
- Experience handing over statistics answers via agile software program improvement methodologies.
- Experience with PowerShell, Python or comparable scripting language.
- Experience with UC4 Job Scheduler.
- Exposure to the retail industry.
- Excellent verbal and written communique skills.
- BA/BS in Computer Science, Engineering, or equal software program/offerings experience.
Pay Ranges: - Level 1 - $75,000 - $110,000
- Level 2 - $100,000 - $135,000
Apply Job!