Description
Toptal is hiring a remote ML Inference Engineer – Terminal-Bench Task Designer (full-time, senior). The role is open to people living in any country. The employer has not published a salary. Working hours: 60 to 75 minutes per task. You apply directly on the employer's site.
About the role
We are building a benchmark that tests how well AI agents handle real expert work in a terminal. We are hiring ML inference and serving experts to turn real, paid tasks from their jobs into expert-level Terminal-Bench challenges. Each task includes the environment, a working solution, and an automated check that proves the result is correct.
What you'll do
- Design expert-level Terminal-Bench tasks that turn real, paid ML inference work from your own experience into challenges an AI agent must solve end to end.
- Define the files, systems and environments each task runs on, such as model serving with streaming, so the challenge is realistic, reproducible and command-line solvable.
- Write a working reference solution for every task and verify it runs end to end, sized for about 60 to 75 minutes of expert effort.
- Build automated checks that objectively prove a result is correct, including cases where ML outputs go wrong silently, such as numerical drift or metric bugs.
- Package tasks with Docker and Git so they are reproducible, and document the systems, tools, data and rules involved so reviewers can validate difficulty.
- Iterate on tasks based on reviewer feedback and agent test runs, adjusting difficulty and fixing ambiguities so each task stays expert-level, fair and objectively gradable.
Skills
- Python
- PyTorch
Who can apply
This job is open to people living in any country. There are no location restrictions. Working hours: 60 to 75 minutes per task.
How to apply
- 1Check that you live where this job hires (any country) and can work 60 to 75 minutes per task.
- 2Click “Apply Now” to open Toptal's application page.
- 3Apply soon: listings close when the role is filled.
Interested in this role?
Apply directly to Toptal. It's free, and HRjobs never charges candidates.
