About Toloka

At Toloka AI we create data that powers leading GenAI models and innovations. We work with frontier labs, big tech, renowned AI startups, enterprises and non-profit research organizations worldwide. We use a combination of Experts + Crowd + Tech Platform to teach AI models to reason and evaluate their efficacy and safety. We have experts in more than 50 different domains—from doctors and lawyers to physicists and engineers—and boast one of the most diverse global crowds, representing over 100 countries and speaking 40+ languages. We are a well-funded startup with an enviable portfolio of clients including Anthropic, Amazon, Microsoft, Poolside, Recraft, and Shopify.

Recently, we secured strategic investment led by Bezos Expeditions and Nebius Group with participation from Mikhail Parakhin, CTO of Shopify and board advisor to leading GenAI companies, who now serves as our Chairman of the Board. Our remote-first team is globally distributed around the world: USA, UK, the Netherlands, Serbia, and more.

About the Team

We are the ML team inside Toloka — we build the machine-learning products that power the platform itself, so every project running on Toloka is faster, cheaper, and more reliable.

A few examples of what we own:

• LLM QA — the core technology behind Toloka's automated quality-check mechanism. Every annotation flowing through Self-Service is reviewed by an LLM agent we design, train, and operate.
• Model distillation and fine-tuning — adapting frontier and open-source models to Toloka's tasks to hit the right quality at the right cost.
• Evaluation, benchmarking, cost modeling, and model selection across providers.

We own the full chain. The same team designs the ML solution, ships it to production, keeps it running 24/7, analyzes the results coming back from real projects, and feeds that signal into the next iteration. No hand-off between research, engineering, and operations — it's all us.

About the Position

As a Senior ML Engineer, you will design, train, and deploy the AI agents that drive our core products. You will focus on end-to-end ML tasks—including fine-tuning and Reinforcement Learning (RL)—to build resilient agentic workflows in Python. You will own the full lifecycle of your models, closing the loop from research and benchmarking to production scaling and monitoring.

What you’ll do

• Train, fine-tune, and distill ML models (including RL approaches) to power autonomous AI agents.
• Build and operate agentic workflows in Python, handling complex reasoning and hybrid human-expert interactions.
• Own evaluation and benchmarking, selecting foundational models and establishing cost models.
• Manage the full ML lifecycle: design solutions, ship to production, and monitor real-time signals.
• Implement observability metrics tailored for agent logic, model performance, and system reliability.

What we're looking for

• 3+ years of experience in ML: Strong background in model training, fine-tuning, and Reinforcement Learning (RL);
• 1+ year in Agent Development: Practical experience building, evaluating, and launching autonomous AI agents;
• Agentic Frameworks: Proven experience working with frameworks such as LangChain, LlamaIndex, or AutoGen;
• Open-Source LLMs: Practical knowledge of model distillation and adapting open-source models (e.g., Llama, Mistral);
• Python Mastery: Advanced proficiency in Python with a drive to apply disciplined software engineering standards to ML;
• End-to-End Ownership: Ability to work across the entire chain, from research to production operations;
• Language: Fluency in English (B2 or above).

What we can offer

• You will be part of an international, dynamic environment that drives innovation and sets new standards in the AI and technology sector.
• Competitive compensation package including base salary, bonus, and ESOP.
• Paid PTO and benefits will vary depending on location.
• We offer a full remote or hybrid model (if you are based in NL or Serbia).
• IT setup and home office allowances.

Equal Opportunity Employer:

Toloka is committed to providing equal opportunity and fostering an inclusive environment. We welcome applications from all qualified individuals and do not discriminate on the basis of race, religion, color, national origin, sex, sexual orientation, gender identity, age, marital status, veteran status, disability, or any other characteristic protected by applicable law. Selection decisions are made based on qualifications, merit, and business need.

[Important Notice] Scam Alert Regarding Fake Job Postings

It has come to our attention that an individual or group is fraudulently impersonating Toloka to post fake jobs and solicit personal information from applicants. Please be aware:

• Official Communication: Our recruiting team will only contact you from an official "toloka.ai" email address. We will NEVER use Gmail, Yahoo, Tolokainc, toloka.inc, or other personal or seemingly business email accounts.
• Our Process: We will never ask for your bank account details, credit card number, or any fees as part of the application or interview process.
• Official Listings: All legitimate job openings are posted on our official careers page: https://toloka.ai/careers#job-list

What to do: If you see a suspicious job posting or have been contacted by someone you suspect is a scammer, please do not provide any personal information. Instead, report the incident to us directly at security@toloka.ai and report the profile/post to LinkedIn.We are taking this matter very seriously and are working with the appropriate parties to resolve it.

Thank you for your vigilance!

To learn how we collect, use, disclose, and store personal data, check out our Privacy Notice.

Company Intro

At Toloka AI we create data that powers leading GenAI models and innovations. We work with frontier labs, big tech, renowned AI startups, enterprises and non-profit research organizations worldwide. We use a combination of Experts + Crowd + Tech Platform to teach AI models to reason and evaluate their efficacy and safety. We have experts in more than 50 different domains—from doctors and lawyers to physicists and engineers—and boast one of the most diverse global crowds, representing over 100 countries and speaking 40+ languages. We are a well-funded startup with an enviable portfolio of clients including Anthropic, Amazon, Microsoft, Poolside, Recraft,  and Shopify.

Recently, we secured strategic investment led by Bezos Expeditions and Nebius Group with participation from Mikhail Parakhin, CTO of Shopify  and board advisor to leading GenAI companies, who now serves as our Chairman of the Board. Our remote-first team is globally distributed around the world: USA, UK, the Netherlands,  Serbia, and more. 

About position

The Forward Deployed AI Engineer Engineer is Toloka's most senior client-facing technical role. You will partner directly with CTOs, VPs of Engineering and ML, research leaders, and applied AI teams, helping them solve complex AI data challenges and translate ambitious model goals into practical, scalable solutions.

This is a hands-on role. You will design and build the data-generation, labeling, and evaluation pipelines that power the next generation of AI models. Rather than training the models yourself, you'll help clients identify the right data strategy and own the solution from problem discovery through implementation and delivery.

Beyond delivering client solutions, you'll help shape how Toloka approaches AI solution engineering - establishing best practices, raising the technical bar across engagements, and mentoring Solution Engineers and Technical Consultants.

Why this role is different

Every engagement is different. One week you might be designing evaluation pipelines for frontier foundation models, the next helping an enterprise build domain-specific reasoning datasets or synthetic data workflows.

This role sits at the intersection of consulting, engineering, and AI. You'll work directly with some of the world's leading AI companies, helping shape how next-generation models are trained and evaluated.

What you'll do

Executive partnership

• Act as the primary technical counterpart to CTOs, VPs of Engineering, and research/engineering leadership.
• Lead executive conversations using a structured, answer-first (BLUF) approach.
• Manage escalations and expectations with composure and integrity.
• Build long-term trusted relationships by recommending evidence-based solutions.

Data strategy & needs diagnosis

• Ask excellent questions to uncover the client's real need - the "question behind the question." Understand their model, which metrics they want to move, and how they intend to train or evaluate it.
• Draw on a solid understanding of how LLMs are trained and fine-tuned to have credible conversations with their technical leaders, understand their data strategy, and proactively propose the data that will solve their problem - with options and rationale ("based on your goal, you likely need this, or this").

Solution engineering & delivery (hands-on)

• Design and build the data solutions yourself: configure data-labeling components and quality controls, develop user interfaces and AI-driven solutions (e.g. agentic systems, RAG, synthetic data generation), and integrate them into automated, multi-stage pipelines that produce data for AI training and evaluation.
• Architect and reason about complex, multi-stage solutions end to end; run experiments to prove the pipeline delivers data of the required quality and speed; iterate from MVP toward production.
• Provide technical leadership across multiple client engagements, establish reusable engineering standards and best practices, and mentor Solution Engineers and Technical Consultants to raise the overall technical bar of the organization.

Commercial ownership

• Own delivery end to end - timelines, quality, and scalability - and the commercial outcome: Gross Margin and Contribution Margin, with the levers, trade-offs, and next checkpoints named, not just described.
• Identify and drive expansion opportunities.

About You

No one is expected to meet every requirement below. If you bring a strong combination of client-facing communication, AI/LLM expertise, and hands-on engineering experience, we'd love to hear from you.

• Executive communication. You can confidently lead conversations with CTOs, VPs, and senior technical stakeholders - clear, concise, structured, persuasive, and calm under pressure. Professional English (C1+) is essential.
• Strong understanding of modern LLM development. You understand how modern LLMs are trained, fine-tuned, and evaluated (including SFT, RLHF/RLAIF, DPO/PPO, reward modeling, and LoRA/PEFT). You're comfortable discussing these topics with senior technical stakeholders and translating their goals into effective AI data solutions.
• Hands-on solution engineering. You've designed complex AI solutions, built multi-stage pipelines, and developed AI-driven systems (e.g. agentic workflows, RAG, or synthetic data generation). You're comfortable working in Python (NumPy, Pandas), integrating APIs, and independently prototyping LLM-powered solutions.
• Solid software engineering foundations. You understand version control, testing, MVP thinking, and iterative development. Experience building production software is a strong plus.
• Exceptional discovery and problem framing. You enjoy working through ambiguity, asking the right questions, and translating business or research goals into clear AI data and evaluation strategies.
• Seniority and track record. 8+ years delivering complex AI, ML, or data projects end to end, with experience influencing technical direction beyond individual projects. Experience establishing engineering standards, mentoring engineers, or leading cross-functional technical initiatives is highly valued. Experience in top-tier strategy consulting (McKinsey, Bain, BCG) and/or as an applied ML / LLM engineer is a strong advantage.
• Ownership mindset. You take responsibility for outcomes, make thoughtful trade-offs between time, cost, and quality, and are comfortable owning both technical and commercial success.

Nice to have

• Hands-on experience training or fine-tuning LLMs and/or building agentic systems (helps you reason about client needs - though the role itself is about building data solutions, not training models).
• Advanced degree (MSc/PhD) in AI, CS, or a related field.
• Experience in crowdsourcing and/or data-centric AI, and with fast-paced, multi-project delivery for frontier AI clients.

What we can offer

• Freelance (B2B) collaboration, with a path into a project team if things go well;
• Flexible, fully remote schedule (40 hours per week);
• Work at the leading edge of AI development, alongside a dedicated and dynamic team of experts;
• Projects with customers that are AI industry leaders and well-known household names;
• Friendly community.

[Important Notice] Scam Alert Regarding Fake Job Postings

It has come to our attention that an individual or group is fraudulently impersonating Toloka to post fake jobs and solicit personal information from applicants.Please be aware:

• Official Communication: Our recruiting team will only contact you from an official "toloka.ai" email address. We will NEVER use Gmail, Yahoo, Tolokainc, toloka.inc,  or other personal or seemingly business email accounts.
• Our Process: We will never ask for your bank account details, credit card number, or any fees as part of the application or interview process.
• Official Listings: All legitimate job openings are posted on our official careers page: https://toloka.ai/careers#job-list

Software engineers and AI agents alike suffer from the same problem: finding that one person or place that will answer their tough, specific question. Many solutions promise to solve this with similarity search in vector databases. Unfortunately, finding the answer is often a puzzle with pieces to be collected across a myriad of contradictory sources and cannot be solved without surgical search and careful reasoning. 

Spectrum collects data from an organization’s code, docs, and issues, and organizes knowledge in a unified ontology that AI agents can efficiently search through and reason over. We aim to revolutionize the semantic layer space for software-building organizations and move beyond specs that fall out of sync with code, introducing a living spec – one that’s extracted from the whole system and used to keep it aligned. Spectrum is meant to be the single source of truth for all product and architectural knowledge.

Spectrum is a resident of JetBrains' startup incubator, with startup speed and autonomy, and backed by 25 years of developer tooling expertise. We are looking for a top-class ML Engineer who will help us shape the future of software development. You will own our AI and ML engineering stack and help define the research agenda for our team. Your technical vision and design decisions will directly shape the product and determine its success.

Your responsibilities will include:

• Designing and building the ML/LLM solution for data ingestion, knowledge extraction, retrieval, and subsequent reasoning.
• Creating the datasets, metrics, and pipelines that drive measurable improvements across the system.
• Architecting and improving agents for context retrieval, knowledge extraction, and data alignment, which includes prompt engineering, model selection, and inference optimization.
• Establishing MLOps practices, including orchestration, observability, and experiment tracking.
• Collaborating with the engineering team on system design and with JetBrains Research on the research agenda.
• Defining hiring criteria, growing the ML team, and shaping the ML team culture.

We expect you to have:

• A proven track record as an ML/AI Lead.
• At least five years of experience in ML/AI systems, with at least two years focused on LLMs and generative AI.
• A deep understanding of the LLM ecosystem, including model architectures and fine-tuning approaches.
• Hands-on experience with:
• Prompt engineering and LLM pipeline design, including evaluation.
• Agentic frameworks such as LangChain, LlamaIndex, LangSmith, smolagents, or an equivalent.
• Vector databases and retrieval-augmented generation (RAG) patterns.
• Deploying and scaling LLM-powered applications using APIs (e.g. OpenAI or Anthropic) or open-source models.
• Strong Python skills – Kotlin knowledge would be a plus.
• Excellent communication skills, with the ability to explain complex technical concepts to diverse audiences.
• Proficiency in English, both written and verbal. 

Our ideal candidate would have:

• Experience with ontologies, knowledge graphs, or graph-based reasoning.
• Experience in early-stage startups – you enjoy the zero-to-one phase.
• The ability to think strategically about product-led AI, beyond just training models in isolation.
• A background in code analysis, developer tools, or software engineering research.
• Experience with multi-agent systems or complex agentic workflows.
• Actively contributed to relevant open-source projects or publications.

What we offer

• A competitive salary and JetBrains benefits.
• A generous runway and corporate resources with startup autonomy.

#LI-KP1