We are seeking a talented Data Platform Developer to join our team in developing a scalable data collection, storage, and distribution platform. This platform will house data from various sources, including vendors, research providers, exchanges, prime brokers, and through web scraping. Your work will make critical data available to systematic and fundamental portfolio managers, as well as enterprise functions such as Operations, Risk, Trading, and Compliance. Additionally, you will play a key role in developing internal data products and analytics to drive data-driven decision-making.
Key Responsibilities
- Web Scraping. Design and implement web scraping solutions using scripts, APIs, and tools to gather data from various sources.
- Data Platform Development. Build and maintain a greenfield data platform utilizing Snowflake and AWS technologies.
- Pipeline Enhancement. Analyze existing data pipelines and implement enhancements to accommodate new requirements and improve performance.
- Data Provider Onboarding. Collaborate with stakeholders to onboard new data providers, ensuring seamless integration into the platform.
- Data Migration Projects. Participate in data migration initiatives to transfer and consolidate data from legacy systems to the new platform.
Mandatory Skills
- SQL. Proficient in writing and optimizing SQL queries for data manipulation and retrieval.
- Python. Strong programming skills in Python for data processing and automation tasks.
- Linux. Experience with Linux environments, including command-line operations and scripting.
- Containerization. Familiarity with Docker and Kubernetes for deploying and managing containerized applications.
- DevOps. Solid understanding of DevOps practices, including CI/CD using Jenkins.
- AWS. Experience with Amazon Web Services for cloud infrastructure and services.
- Communication. Excellent communication skills to collaborate with cross-functional teams and stakeholders.
Nice-to-Have Skills
- Market Data Projects. Experience working on market data projects or within capital markets.
- Airflow. Familiarity with Apache Airflow for orchestrating complex data workflows.