DataArt

remote • ARM • senior • Не указана

# Platform Engineer with AWS AgentCore Evaluation ArmeniaBulgariaCyprusGeorgiaKazakhstanLatviaPolandRomaniaSerbiaUkraine ## Technology stack AWS AgentCore Evaluation API, AWS AgentCore Runtime, AWS CloudWatch, CloudWatch Embedded Metric Format (EMF), GitHub Actions, CI/CD Pipelines, Python, AWS Services, Monitoring and Observability Tools, Quality Evaluation Frameworks ## Project overview The project focuses on establishing a scalable evaluation and monitoring platform for AI agents and intelligent applications. The solution provides automated deployment gates, online quality assessment, operational visibility, and regression detection capabilities to help maintain high standards of reliability and performance. ## Team Medium team (10-20 people) You will work closely with platform engineers, DevOps specialists, AI engineers, and cloud architects. The team collaborates on automation, observability, deployment processes, and operational excellence, ensuring that AI systems meet quality requirements throughout the software delivery lifecycle. ## Position overview We are looking for a Platform Engineer with experience in CI/CD automation, cloud observability, and AI platform operations to help build and maintain quality evaluation pipelines for agent based systems. In this role, you will integrate automated evaluation frameworks into deployment workflows, develop monitoring solutions, and support continuous quality validation across production environments. ## Responsibilities - Integrate AWS AgentCore Evaluation API into CI/CD pipelines to support automated deployment quality gates - Configure and maintain on demand evaluation workflows as part of build and release processes - Develop and support GitHub Actions workflows for automated evaluation execution and deployment validation - Configure and manage online evaluation modes with appropriate sampling strategies - Build and maintain CloudWatch dashboards to track evaluation metrics for agents and evaluators over time - Design and configure CloudWatch alarms to identify quality degradation and operational issues - Collaborate with AI and platform engineering teams to define evaluation thresholds and quality standards - Monitor evaluation results and contribute to regression detection and release validation processes - Support observability, reporting, and operational monitoring initiatives across AI platforms - Create and maintain technical documentation related to evaluation workflows, monitoring, and deployment automation ## Requirements - 3+ years of experience in platform engineering, DevOps engineering, or a related field - Experience integrating quality validation processes into CI/CD pipelines - Hands on experience designing and maintaining deployment gates for automated release workflows - Experience building and supporting AWS CloudWatch dashboards and monitoring solutions - Experience configuring CloudWatch alarms and operational alerting strategies - Knowledge of cloud infrastructure, automation, and deployment best practices - Experience working with monitoring, observability, and platform operations tools - Strong analytical and troubleshooting skills - Good written and verbal communication skills ## Nice to have - Experience with AWS AgentCore Evaluation online mode configuration and sampling strategies - Experience using evaluations to compare runtime versions and identify regressions - Knowledge of CloudWatch Embedded Metric Format (EMF) for structured metric collection and reporting - Experience with AI platform monitoring and evaluation frameworks - Exposure to automated quality assurance processes for machine learning or agent based systems - Experience supporting enterprise scale cloud platforms and deployment pipelines ## FAQ for Candidates Work on global projects, grow your career in a supportive, flexible, and innovative tech environment. We help cover the cost of IT certifications and provide access to top-tier courses and learning platforms. View current openings and take the next step with us. ### What does DataArt do, and which industries does it serve? DataArt is a global software engineering company that helps businesses build powerful data, analytics, and AI solutions. We work with clients across a range of industries — including Finance, Healthcare & Life Sciences, Consumer Goods & Retail, Travel, Media & Entertainment, Mobility, and Manufacturing. Learn more about what we do [here](https://www.dataart.com/company/about-us). ### What's the work-life balance like at DataArt? Does DataArt offer remote work options? DataArt supports flexible work formats to help you find the balance that works best for you. You can choose to work from the office, go hybrid, or stay fully remote—each option comes with equal opportunities for growth. We’ll help set you up with secure access and the equipment you need. With 46 remote and onsite official locations, you can join us from almost anywhere in the world. Learn more about how we work [here](https://www.dataart.team/career/workplace-options). ### What is DataArt's company culture like? At DataArt, we put people first—fostering a culture built on trust, flexibility, and professional growth. We believe in open communication, mutual respect, and the freedom to choose how and where you work. Diversity, equity, and inclusion are core to our values, and we actively support a workplace where everyone can thrive. From mental health support to global sustainability efforts, we aim to create a healthy, empowering environment for all. [Read more about our culture and values](https://www.dataart.team/career/culture). ### What is the typical career path at DataArt? There's no one-size-fits-all career path at DataArt—your growth is yours to shape. Whether you want to deepen your technical skills, move into management or sales, or even switch professions entirely, you'll have the support to do it. With exam fees fully covered, you can also earn professional certifications, like AWS, Azure, or Google Cloud. With tools like the Professional Development Map, the Talent Lab, and access to expert mentoring, we help you build your desired career. Learn how we support both [newcomers starting their careers](https://www.dataart.team/start-career) and experienced [professionals looking to grow further](https://www.dataart.team/advance-career). ### What can I expect in terms of compensation and benefits at DataArt? DataArt offers competitive compensation along with a range of thoughtful benefits that support your well-being, growth, and daily comfort. You’ll get flexible vacation and sick leave, mental health programs, and access to a corporate laptop or BYOD option. We also offer bonuses for referrals, parental leave, and smooth exit and return-to-work options. [Read more about how things work at DataArt](https://www.dataart.team/career/culture). ### How does DataArt approach employee retention and turnover? At DataArt, we focus on creating an environment where people want to stay and grow. With 95% of employees recommending us to a friend on Glassdoor and a 100% CEO approval rating, we’re proud of the trust we’ve built. From mentoring programs and professional development services to support and conflict resolution programs, we invest in our colleagues' growth and wellbeing—because when people feel valued, they stick around. Learn more about our company >[here](https://www.dataart.team/). ### What is the interview process like, and how can I prepare? Our interview process is designed to be thorough, transparent, and supportive. It starts with a CV review, followed by an HR interview to discuss your background and goals. You'll then go through communication and technical assessments, where we evaluate your English skills and knowledge of relevant technologies. If all goes well, you’ll meet someone from the project team to learn more about the work, and we’ll guide you every step of the way—including helping you prep for any client interviews. Read more and get tips for each stage [here](https://www.dataart.team/career/how-to-get-in). ### What skills and experience does DataArt look for in candidates? DataArt looks for candidates with strong technical skills, analytical thinking, and a commitment to continuous learning. We value experience in relevant technologies and industries, adaptability, and excellent communication skills. If you’re a junior or just starting out, don’t worry—our mentorship programs and training will support your growth from day one. The most important thing is your willingness to learn and develop professionally in our collaborative, people-first environment. [Read more about development in DataArt.](https://www.dataart.team/start-career) ### How does DataArt support learning and professional development? At DataArt, continuous learning is part of our culture. You’ll have access to expert mentors, leadership guidance, and thousands of courses from top platforms like Udemy and LinkedIn Learning. You can shape your own path—whether it’s growing within your role or switching to a new one. [Explore how we support your growth.](https://www.dataart.team/start-career)