The growing interest and use of AI-assisted and agentic engineering intersects with ROOST’s work in three ways: As a community of software developers, coding assistance models are becoming part of developer workflows, but the quality and effectiveness of using AI-assistance varies widely across users. As an open source code base, recent proof-of-concepts “rewriting” existing open source software (OSS) projects raise fundamental questions about how OSS will be used in the future. And third, as a trust and safety project – a field that has long been using machine learning and large language models – incorporating any technology advancement has to be done within the context of sensitive safety systems protecting millions of users.

This post shares our observations, questions-on-our-minds, and the principles guiding how we’re currently navigating these intersections. What these look like will continuously evolve, but we’ll always anchor our approach in our mission: to radically reshape online safety technology by making critical safety tools openly available, auditable, and relevant for all.

The stakes are not abstract. ROOST projects now protect hundreds of millions of users online every day, meaning bugs and issues in ROOST code could directly impact how a real, vulnerable person is protected.

AI coding makes strong SDLC practices more important than ever

The rise of AI-assisted and agentic coding emphasizes the importance of strong software development lifecycle (SDLC) practices for OSS being used in critical systems. In open source development, a code contribution may come from a pseudonymous contact, who’s under no obligation to respond to your questions or help troubleshoot the code after they’ve shared it with you. While code should always be verified, when working inner-company, there are pathways for staying connected to teammates and accountability mechanisms for their future cooperation if needed.

That difference shapes what it means to be an OSS project maintainer. A maintainer reviewing and accepting code contributions wants to make sure they fully understand a code change, because they assume it’s them, not the contributor, who is responsible for this code from here on out.

This is where AI-assisted or fully agentically generated contributions can create friction in open source projects. As highlighted by Mark Russinovich, CTO of Microsoft Azure, and Scott Hanselman, VP of Developers at Microsoft AI, developers with strong foundations in software engineering bring the ability to spot anti-patterns or errors in an approach to their use of AI; developers who have not yet built these foundations have to defer design decisions and assumptions to the model. A new approach contributed to an open source project could be relevant, but if the contributor isn’t able to pass the reasoning on to another maintainer, it sets the maintainer up for future trouble to take on code they don’t understand.

This challenge, combined with high volumes of low-value contributions (aka “AI slop”), has understandably caused some open source projects to ban AI contributions outright to respect maintainer time.

For ROOST, we want developers to be able to use the tools that help them do their best work, including AI tools. But because our projects are used in production to make high-stakes decisions, we treat them as “critical safety systems,” and that informs our risk tolerance.

Our current approach is to hold AI-assisted contributions to the same standard as any other contributed code – the author must have permission to contribute the work, be able to explain the contribution, and present the change in a way that is reviewable, maintainable, and debuggable by others. (That means code changes that a human reviewer can digest, not multi-thousand-line changes.) This policy recently got a shout out from GitHub for its effective simplicity. At the same time, we’re ramping up our software development lifecycle (SDLC) practices to ensure there are clear project conventions and strong quality and security checks across all contributions. We have added AGENTS.md files to ROOST projects to make our project conventions machine readable and improve the experience and output of developers using AI-assistance.

Whether imported or referenced, open source remains foundational

There have been multiple high-profile “rewrites” of existing open source projects by AI labs and teams. As the models had relevant open source projects in their training sets, and in some cases were prompted to reference particular bodies of work, it’s spawned the idea that in the future, open source software won’t be directly imported by a user, but rather used as a “reference” for a model to write a user-customized version.

When using open source software, many users ultimately do end up making a change to their production version. It could be something like deciding which of the compatible databases to use, or it could be a larger change to tailor the project to existing internal services or platforms. ROOST’s mission is helping everyone have access to better safety tools, and that includes smaller teams with few engineering resources. Entertaining the notion of “open source as a reference,” could this approach streamline user onboarding and get safety tools deployed faster?

The delicate balance with customizing open source software is that the user is responsible for maintaining any changes not in the main project, and making sure those changes are compatible with future releases from the main project. This is sometimes referred to as “carrying patches” or being “off main.” Furthermore, generating an initial code base is only a very small part of successfully running software, particularly infrastructure-like software like ROOST projects. Being too far “off main” can also impact how service reliability and troubleshooting are done. If code base customization isn’t done with foresight, users find themselves losing the engineering efficiency of using shared open source software, and with gnarly upgrade paths, and quickly accruing technical debt.

In two ROOST community agentic coding experiments, Coop and Osprey were used as references, with design intentions and prompts to simplify the projects’ services and dependencies. While the outputs are fully working, performant prototypes, they are both derivatives far away (eg “off main”) from the referenced projects (not even in the same languages, for example).

These experiments highlighted that if trust and safety teams are going to use ROOST projects as “references” yet get the engineering efficiency and community support of open source, teams need a way to tightly guide and steer derivative creation.

One idea is, similar to how AGENTS.md files put boundaries and standards on how to write code for a project, are there similar “prompt packs” or machine-readable instructions that can help users deploy ROOST tools quickly in their environments, but stay within a bound of customization that prevents future technical debt? How can contributors create these kinds of packs or instructions to have consistent, reliable deployment results, and without the details of a user’s environment?

The ROOST Playground: Enabling community experimentation

While these questions feel theoretical, we’ve already seen some creative users successfully prototype agents that deploy ROOST instances. Sharing their lessons with the community, and generalizing them into developer tool kits – including handling more sophisticated environment customizations – could be on the near horizon.

That’s why we spun up the “ROOST Playground,” a space to learn from what’s working and celebrate the creative ways our community is building and experimenting with ROOST tools. In the playground you’ll find links to public proofs-of-concept, modified branches, and deployment methods all relating to ROOST tools. AI-forward projects include agentic rewrites of ROOST projects, experimenting with specific design changes, and an investigation agent that runs atop Osprey. We hope this playground encourages new ideas as well as practical lessons for in-production ROOST users, and if you create an experiment or extension with ROOST projects, we encourage you to add it to the Playground!

Anchored in our mission: online safety for all

To date, we’ve seen a variety of approaches to AI in the ROOST community. Some community members or organizations come from strong ML backgrounds – or have enthusiastically increased their knowledge – and are experimenting with what’s possible with applied AI engineering. For others, their priority is establishing or refining strong foundations to their safety workflows, or they aren’t as interested in incorporating agentic tools into their workflows. As with any technology, there is no uniform approach or adoption rate. We’ll continue to support organizations wherever they are, anchored in our goal of improving online safety for all.

These are not areas with determined outcomes. Our duty to our users and our determined belief in making online safety within reach of all means we’re closely watching how software development and Trust & Safety will look tomorrow, while providing users with reliable, effective safety tools for today.