Project Sceptrum - Multilingual AI Coding Evaluation Specialist - German

Join a short, paid project evaluating AI coding agents' performance with instructions in German.

We are seeking a multilingual AI coding evaluation specialist to join a project testing AI coding agents with instructions in German. As a specialist, you will play a crucial role in assessing the agents' performance and providing valuable insights.

About the Project Welo Data is helping evaluate AI coding agents' performance with instructions in languages other than English. The project involves giving agents a real software engineering task in a code repository and checking whether they solved it using automated tests.

Roles There are two roles available:

• Translator: Translate English software engineering task briefs into German, maintaining requirements, constraints, and success conditions.
• Reviewer: Compare translations with English sources, check for errors, and log corrections.

Requirements

• Native or near-native writing in German and strong reading ability in English.
• Professional experience in translation, editing, review, or technical writing, or a software background with excellent writing in German.
• Comfort reading technical text; coding experience is a plus.
• Close attention to detail.

Project Details

• Duration: One week, starting mid-November.
• Time commitment: 6 hours per person.
• Compensation: Varies depending on programming language.
• Work location: Fully remote.

Selection Process

1. Screening questions.
2. Language test.
3. Programming skills assessment.
4. Financial onboarding and role placement.
5. Production.

Benefits

• Limitless flexibility with project-based opportunities.
• Optional access to AI and Large Language Model workshops.
• Global contributor community with responsive guidance and support.