Description:
Viral Square is looking for an experienced and highly motivated Senior Data Engineer to join our growing data team. In this role, you will support company-wide data management, system integration, and analytics initiatives that drive business intelligence and decision-making. You’ll collaborate closely across departments to design scalable data solutions, integrate multiple business systems, and ensure our data infrastructure remains robust, secure, and efficient. This is a full-time, onsite position reporting directly to the Data Engineering Manager.
How You Will Achieve It:
- Design and build scalable and reliable data infrastructure and pipelines for ingesting, processing, and transforming data from diverse sources. Implement data validation and quality checks to ensure accuracy and consistency.
- Monitor and maintain the performance of data solutions, including troubleshooting and optimizing queries, data pipelines, and storage systems.
- Design and maintain data models, schemas, and database structures to support reporting and analytics.
- Collaborate with product managers, stakeholders, analysts, and other data engineers to define requirements and data specifications.
- Collaborate with network engineers to define networking infrastructure requirements to support the data infrastructure and pipelines.
- Collaborate with the security team to ensure all deliverables include the necessary controls and compliance standards regarding data access, data security, data sensitivity and data confidentiality.
- Perform preliminary assessment of new data sources. Assist project managers with estimating level of effort and refining the project scope to identify requirements and tasks.
- Document technical designs, pipelines, data sets, and best practices.
- Proactively stay informed of the latest advancements in data engineering, data architecture, and data privacy regulations.
What You Need to Achieve It:
- BS degree in Computer Science or Engineering, Math, Statistics, Informatics, Information Systems, or another quantitative field required.
- 4-5 years of experience in a data engineering role, with expertise in Microsoft Azure cloud: Azure Data Factory, Data Pipelines, Data Bricks, Azure Functions, Azure CLI, Integration Runtime, Azure Synapse Analytics, SQL Pools, SQL Database and data warehousing.
- Experience with Python, pySpark, SQL, and ETL development.
- Experience in agile methodologies, Azure Devops, Azure Repos, CI/CD pipelines for automating build, test, and deployment processes.
- Experience with Microsoft Power Platform, Power Apps, Power Automate, Dataverse.
- Experience with Microsoft Fabric preferred.
- Familiar with Azure Networking, VNets, DNS, routing, load balancing, security.
- Familiarity with Microsoft Dynamics 365 Finance and Operations, and Microsoft Dynamics CRM Sales data a plus.
- Excellent problem-solving and communication skills, with the ability to work effectively in a remote, cross-functional team environment.