SMX is hiring a Data Engineer responsible for designing, building, and maintaining scalable data architecture that powers our AI, analytics and reporting. Collaborate closely with Analysts, AI Engineers, and Software Engineers to transform raw, unstructured data into high-quality, accessible data pipelines. Optimizing data systems, building them from the ground up, and ensuring peak data reliability and performance. This role is essential for making data accessible, reliable, and usable across the enterprise.
Essential Duties and Responsibilities
- Design, construct, install, test, and maintain highly scalable data management systems and robust ELT/ETL pipelines across cloud and on-prem systems.
- Build infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using cloud technologies. Support API strategy, data modeling, and domain-driven data structures.
- Implement monitoring systems to ensure data integrity, quality, and security across all storage and processing layers. Implement data quality checks, validation, lineage tracking, and metadata management.
- Prepare and optimize data for AI/ML models, semantic search, analytics, and mission applications.
- Identify, design, and implement internal process improvements, such as automating manual processes and optimizing data delivery for greater scalability. Automate ingestion and transformation processes using industry-standard tools and patterns. Troubleshoot pipeline issues and optimize for performance and cost.
- Work with analytics and business teams to understand their data requirements and deliver production-ready datasets. Collaborate with AI engineers and analysts to ensure datasets meet requirements.
Required Skills & Experience
- Bachelor's degree in Computer Science, Information Technology, Data Engineering
- 4+ years of progressive professional experience in a data engineering or backend software engineering role
- Security clearance required (Secret or higher)
- Strong proficiency in Python, SQL, Java.
- Experience with cloud data warehouses such as Snowflake and AWS Redshift.
- Familiarity with distributed computing tools like Spark, Databricks, and Kafka.
- Experience with Azure Data Factory, AWS Glue, or similar data integration platforms.
- Experience with workflow management tools like Apache Airflow.
- Hands-on experience with AWS, GCP, or Azure cloud environments. Familiarity with multi-cloud data ecosystem patterns.
