Manulife is a leading international financial services provider, and they are seeking an Associate Data Scientist to design, evaluate, and deploy LLM-based applications. This role involves collaborating on projects that utilize cutting-edge LLM technologies, ensuring model performance and alignment while gaining real-world experience in AI workflows.
Responsibilities
- Design and evaluate LLM-based applications, including RAG pipelines, agentic systems, and chat interfaces
- Conduct LLM red-teaming, bias detection, and alignment testing using comparative and statistical analysis techniques
- Build and maintain automated evaluation frameworks for LLM outputs using embedding-based and prompt-based methods
- Develop interactive dashboards and visualizations using Power BI to communicate insights and model performance to stakeholders
- Integrate and test LLMs using OpenAI APIs, Databricks, and Azure AI Services
- Collaborate on the development of fine-tuning and pre-training pipelines for domain-specific use cases
- Work with Vector Databases (e.g., FAISS, MongoDB, Neo4j) to support RAG and semantic search capabilities
- Participate in code reviews, documentation, and agile development cycles with the engineering team
Skills
- Minimum 1 year of professional (non-academic) experience in data science, machine learning, or AI-related roles
- Strong programming skills in Python and SQL; experience with JavaScript is a plus
- Bachelor's or master's degree in computer science, Data Science, AI, or a related field
- Hands-on experience with LLM evaluation, bias detection, or alignment research
- Proficiency in data visualization tools, especially Power BI (experience with Tableau or similar tools is a bonus)
- Familiarity with LangChain, RAG, Ollama, LLM fine-tuning, and embedding-based analysis
- Experience with cloud platforms such as Microsoft Azure and Databricks
- Knowledge of databases including PostgreSQL, MongoDB, Neo4j, and FAISS
- Strong analytical thinking, attention to detail, and a passion for building responsible AI systems
Benefits
- Health, dental, mental health, vision, short- and long-term disability, life and AD&D insurance coverage, adoption/surrogacy and wellness benefits, and employee/family assistance plans
- Various retirement savings plans (including pension and a global share ownership plan with employer matching contributions)
- Financial education and counseling resources
- Generous paid time off program in Canada includes holidays, vacation, personal, and sick days
- Full range of statutory leaves of absence
Company Overview
Apply Now