Lead QA Engineer (Python, TypeScript)

Overview:

We are seeking a Lead Engineer – AI Application QA to define and drive the quality assurance strategy for AI-enabled financial applications and enterprise platforms. This role combines technical leadership with hands-on testing across application functionality, AI-powered user experiences, APIs, data flows, and integrated business workflows.You will lead manual and automated testing practices, establish measurable acceptance criteria, and embed continuous testing throughout the software delivery lifecycle. Working closely with Engineering, Product, Architecture, and AI teams, you will ensure that applications deliver reliable, secure, and contextually appropriate outcomes that meet business requirements.

Responsibilities:

• Own the end-to-end QA strategy, test planning, and execution for AI-enabled applications, covering functional, integration, regression, usability, accessibility, and performance testing.
• Lead and mentor QA engineers, review test designs and automation code, and maintain consistent testing standards across internal and vendor teams.
• Define risk-based coverage and acceptance criteria for business workflows, including AI-assisted interactions and their downstream actions.
• Validate AI application behavior against approved scenarios, checking response relevance, factual consistency with authorized source content, instruction adherence, and appropriate handling of unsupported requests.
• Test conversational experiences, including multi-turn interactions, context retention, session isolation, ambiguous inputs, and recovery from errors.
• Validate document ingestion and retrieval workflows, ensuring responses use relevant source content, provide accurate references where required, and respect user access permissions.
• Test AI-driven orchestration and tool integrations, including correct action execution, confirmation steps, authorization boundaries, retries, and prevention of duplicate or unintended actions.
• Validate application safeguards against prompt injection, sensitive information disclosure, and unauthorized access across users or tenants.
• Verify graceful handling of unavailable AI services, timeouts, incomplete responses, and dependency failures, including fallback behavior and user-facing messages.
• Build and maintain scalable automation frameworks for UI, API, integration, and data validation testing.Maintain reusable business scenarios, representative test data, and application regression suites to assess the impact of changes to prompts, configurations, integrations, and application releases.
• Integrate automated smoke and regression testing into CI/CD pipelines, supporting daily regression and timely feedback.Validate financial calculations, reconciliations, reporting, and audit trails within AI-assisted and conventional application workflows.Assess end-to-end responsiveness, concurrent usage, and application stability under expected and peak workloads.
• Collaborate with Security and Compliance teams to verify application controls against applicable requirements.Drive defect triage and root-cause analysis, translating production findings into improved test coverage.
• Maintain requirements traceability, test evidence, defects, and quality reporting in Azure DevOps.Communicate release readiness, unresolved defects, and residual risks; ensure QA sign-off is followed by Product sign-off before production release.

Qualifications:

To qualify for this role, you must have:

• A bachelor’s degree in Computer Science, Engineering, or a related field.At least 10 years of QA experience, including 3 years  technical leadership in  enterprise environments.
• Hands-on experience testing AI-enabled applications, such as conversational assistants, enterprise search, document processing, or automated business workflows.Strong expertise in functional testing, exploratory testing, test design, and automation engineering.
• Experience developing and maintaining automation frameworks for UI, API, and end-to-end integration testing.An understanding of application concepts such as prompts, conversational context, retrieval-augmented generation (RAG), and AI-driven tool execution.
• The ability to design repeatable acceptance checks for applications whose responses can vary, using clear business rules and expected outcomes.
• Strong SQL skills and experience validating data flows, transformations, and reporting outputs.Experience validating financial workflows, access controls, and audit requirements.
• Familiarity with Agile delivery, Git, CI/CD pipelines, and cloud-based applications.Strong analytical and communication skills, with the ability to explain quality risks and support release decisions.Experience mentoring engineers and coordinating testing across multidisciplinary teams.

Technical Exposure

Relevant experience with an appropriate combination of the following technologies is desirable:

• Test automation: Selenium, Playwright, Cypress, REST Assured.
• Programming languages: Python, TypeScript; Java for REST Assured-based automation.
• AI application testing: Conversational workflows, RAG applications, prompt regression, response verification, and tool/API integration testing.
• API testing: Postman, REST APIs, authentication, authorization, and service integration.
• CI/CD and test management: Azure DevOps, GitHub Actions.
• Azure services: App Services, Azure Functions, Azure Kubernetes Service (AKS).
• Data platforms: SQL, Azure Data Factory.
• Domain knowledge: Financial systems, reconciliation, compliance controls, and reporting.