SWITS Digital is seeking an experienced Senior Data Engineer for a Contract-to-Hire opportunity across Bangalore, Pune, Gurgaon, and Coimbatore.
The ideal candidate will have at least 5 years of experience in data engineering, with strong hands-on expertise in Python, PySpark, Databricks, and real-time data streaming. The role requires experience building and maintaining large-scale data pipelines, processing high-volume data, and developing reliable data solutions using cloud platforms.
Candidates should have a strong understanding of SQL, relational databases, big data technologies, and distributed data processing. Experience with Azure is preferred, although candidates with AWS experience will also be considered.
Key Responsibilities:
- Design, develop, and maintain scalable data pipelines using Python and PySpark.
- Build and optimise data processing workflows using Databricks.
- Develop real-time data streaming solutions using Spark Structured Streaming, Kafka, Azure Event Hubs, or Amazon Kinesis.
- Implement batch and streaming data processing solutions for high-volume datasets.
- Develop and maintain data transformation workflows and data integration pipelines.
- Work with Azure, AWS, or GCP to build and operate cloud-based data engineering solutions.
- Write efficient SQL queries and work with relational databases and big data platforms.
- Monitor data pipelines, troubleshoot processing issues, and improve reliability and performance.
- Apply appropriate data quality checks and validation practices.
- Collaborate with cross-functional teams to understand data requirements and deliver suitable engineering solutions.
- Contribute to technical design discussions and recommend improvements to data architecture and processing workflows.
- Support code quality, maintainability, and operational reliability across data engineering solutions.
- Mentor junior engineers and contribute to knowledge sharing within the team.
Mandatory Skills:
- Minimum 5 years of professional experience in data engineering.
- Strong hands-on experience with Python and PySpark.
- Experience with Databricks; Azure Databricks is preferred, while AWS experience is also acceptable.
- Practical expertise in real-time data streaming using Spark Structured Streaming, Kafka, Azure Event Hubs, or Kinesis.
- At least 2 years of experience with Azure, AWS, or GCP.
- Strong SQL skills.
- Experience working with relational databases and big data platforms.
- Strong problem-solving, communication, and collaboration skills.
- Ownership and accountability, with the ability to work across teams and mentor junior engineers.
Good-to-Have Skills:
- Delta Lake.
- Unity Catalog.
- Databricks Workflows.
- Kubernetes.
- Terraform.
- Experience developing large-scale, high-volume data pipelines.
- Telecom or subscription-based industry experience.
Employment Type:
Contract-to-Hire.
Locations:
Bangalore, Karnataka; Pune, Maharashtra; Gurgaon, Haryana; or Coimbatore, Tamil Nadu, India.
Interview Process:
Virtual or face-to-face interview.
The supplied announcement does not specify a salary range, exact work arrangement, educational qualification, or number of vacancies.
SWITS DIGITAL Private Limited is an IT services and consulting company headquartered in Chennai, Tamil Nadu, India. The company specialises in technology solutions, talent acquisition, digital transformation, and AI-powered workforce management.Its t... Read More
