Senior Data Engineer

Liminalis a global market intelligence and strategic advisory firm specializing in digital identity, financial crime and compliance, and IT security technology solutions across industries while also catering to the private equity and venture capital community.
Founded in ****,Liminaloffers strategic and analytical services supporting executive decision-making at all product and business lifecycle stages.
We advise some of the world's most prominent business leaders, investors, and policymakers on building, acquiring, and investing in the next generation of solutions and technologies.
We provide access to proprietary data and analysis, strategic frameworks, and integrated insights on the industry's only market intelligence platform.
Every major company in the world has started focusing on the next generation of digital identity technologies as a necessity for continued growth and security.
Our team works with a myriad of organizations, from Fortune 100s to startups, across industries including financial services, technology, telecommunications, and the P2P economy.
AtLiminal, we help businesses build solutions, execute strategies, invest intelligently, and connect with key decision-makers.
We know that it's in the sharing of discovery and insights that groundwork is laid, problems are solved, and entire sectors advance at the speed of light.
Keeping information to ourselves delays progress for all.
AtLiminal, we don't just respond to the market;
we define it.
About therole This role focuses on building and maintaining strongdata architectures ,pipelines, andsystemsthat support the effective collection, storage, and processing of data across multiple departments.
TheData Engineerwill play a pivotal role in ensuring thescalability ,reliability, andperformanceof our data systems.
With a strong background indata engineering ,cloud infrastructure, anddata pipeline automation, the Data Engineer will work on projects frominitial design to deployment, supporting the seamless integration ofdata workflowsinto product and operational teams.
What you\ 'll do Cross-Department DataSolutions:Collaboratewith various departments to understanddata needs, assesstechnical feasibility, and design efficientdata engineering solutionsto support organizational initiatives.
Implementscalable data workflowsthat optimizedata availability ,quality, andaccessibilityfor AI, business analytics, and other internal teams.
Support product teams in transitioning maturedata pipelinesand systems to ensure alignment withproduct goalsandtechnical requirements.
Design ,implement, and maintaindata pipelinesthat ingest, process, and transformlarge-scale datasetsfor internal applications, including AI and machine learning models.
Build efficientETL(Extract, Transform, Load) processes that streamline the movement ofdatabetween systems, databases, and analytics platforms.
Optimizedata flows to ensurehigh performance ,low latency, andscalability, adapting pipelines to handle bothbatchandreal-time processing.
Develop and maintaincloud-based data infrastructureon a major cloud platform (e.G.,AWS ,Azure ,GCP, or similar), ensuring data systems arerobust ,cost-effective, andperformant.
Implementdata storage solutionsanddistributed databasesthat ensureseamless integrationwith other internal systems.
Leveragecloud servicesforscalable data processingand storage, ensuring that infrastructure can supportgrowing datasetsand organizational demands.
Data Quality & Governance:Establishdata validation processesto ensuredata quality ,consistency, andintegrityacross all pipelines and systems.
Collaborate withdata scientistsandanalyststo ensure data isstructuredandformattedfor optimal use inanalyticsandAI applications.
Ensurecompliancewithdata governance policiesand best practices fordata privacy ,security, andauditability.
Automation & Monitoring:Implementautomationfor data processing workflows, reducingmanual interventionand ensuring consistent delivery ofhigh-quality data.
Set upmonitoringandalerting systemsforpipeline health ,performance metrics, anddata anomaliesto proactively address any issues.
Continuouslyoptimizeexisting data systems and pipelines to improveperformance, reduce errors, and enhancereliability.
Maintaincomprehensive documentationofdata architectures ,data pipeline designs, andsystem integrationsto facilitate clear communication and collaboration.
Documenttechnical workflows, processes, andsystem configurationsto ensure smooth handoffs and enable other teams to leverage data assets effectively.
Collaborate withcross-functional teams, includingdata scientists ,product developers, andbusiness stakeholders, to ensure data solutions align withorganizational goals.
Qualifications 5+ years of experienceindata engineering ,data architecture, andsystem design, with extensive experience building and optimizinglarge-scale data sys