Extending Versity S3 Gateway with a Shared Library Plugin Framework

The Versity S3 Gateway (VersityGW) is an open-source, high-performance S3 translation server written in Go. While Versity leads development, the project is intentionally open to community innovation. Anyone can contribute, and contributions don’t just solve local challenges—they strengthen the gateway for everyone. One notable contribution has come from engineers at CERN, who developed and contributed a shared library plugin framework for the Versity S3 Gateway.

This new capability makes it easier to add support for different storage backends by writing them as standalone plugins that can be dynamically loaded at runtime. CERN’s work is a great example of how organizations can extend the gateway to fit their needs while giving back to the community.

What the Plugin Framework Enables

The shared-library plugin system adds flexibility without complicating the core gateway:

• Dynamic loading – Backends are written and maintained independently, then loaded into the Versity S3 Gateway at runtime.
• Focused development – Developers only implement storage operations like putObject() and getObject(). The Versity S3 Gateway handles the rest—protocol parsing, authentication, routing, multipart handling, logging, error responses, and metrics.
• Direct integration – Plugins can talk directly to backend APIs without going through POSIX, improving efficiency.
• Simpler adoption – New backends can be integrated without forking or patching core code.

This approach gives storage engineers a clean, practical way to add S3 access to their systems. Because plugins live outside the main repo, they can be shared, adapted, or customized by different groups. That makes this model not only practical for developers but also collaborative—an ecosystem where one contribution can be useful to many.

CERN’s EOS Plugin

Alongside the shared-library framework itself, CERN developed the first reference implementation: the EOS S3 plugin (eoss3). This plugin provides a concrete example of how to extend the Versity S3 Gateway to integrate a large-scale production storage system.

With this architecture, standard S3 clients can seamlessly interact with EOS, CERN’s distributed storage platform that underpins much of the data analysis for high-energy physics. EOS manages 100s of Petabytes of scientific data produced by experiments at the Large Hadron Collider, making S3 access a critical step in enabling broader interoperability with external tools and workflows.

CERN also emphasized in their presentation on the S3 Gateway for EOS that this plugin approach brings several benefits:

• Familiar interfaces for researchers – Scientists and engineers can use the S3-compatible tools they already know, while EOS continues to manage data at massive scale behind the scenes.
• Fine-grained access control – EOS’s existing authentication and namespace controls remain in place, extended through the Versity S3 Gateway.
• Security and IAM integration – The gateway model supports encryption and AWS IAM-style policies, aligning EOS more closely with modern cloud storage expectations.
• Modularity and reusability – By building the EOS plugin as an independent shared library, CERN demonstrated how institutions can extend the Versity S3 Gateway without needing to fork or modify the gateway itself.

For other organizations, the EOS plugin serves as a reference design: a working example of how to connect a complex storage system to the Versity S3 Gateway. The source code and documentation in the EOS S3 plugin GitHub repository provide a valuable starting point for developers planning to integrate their own backends.

CERN’s EOS plugin is both a practical solution for their research community and a blueprint for collaboration, showing how specialized storage platforms can be exposed as fully S3-compatible services using the Versity S3 Gateway.

Explore & Contribute: Key Versity S3 Gateway Resources

Below is a curated list of key resources—code, documentation, and presentations—to explore, test, or contribute to the project:

Open Source + Open Collaboration

The Versity S3 Gateway is licensed under Apache 2.0, giving users and organizations the freedom to adapt, extend, and deploy the gateway without restrictions. CERN’s work is a clear example of the power of open source: a feature created to solve one institution’s challenge became a capability that anyone can use and build upon.

We’re excited to see what the community contributes next. Whether it’s new plugins, performance enhancements, expanded protocol support, improved monitoring, or better documentation—every contribution makes the gateway stronger. The Versity S3 Gateway’s open design creates space for innovation across the stack, and we look forward to seeing how the community continues to evolve it.