Skip to content

Latest commit

 

History

History
499 lines (311 loc) · 40 KB

cubefs-graduation-dd.md

File metadata and controls

499 lines (311 loc) · 40 KB

CubeFS Graduation Due Diligence

Graduation Evaluation Summary for CubeFS

Criteria Evaluation

Kevin Wang conducted the due diligence of CubeFS who applied for graduation. The project has completed the criteria that show its maturity at graduation.

The following criteria implementations are noteworthy to call out:

  • A stable and easily maintainable distributed file system, with excellent concurrent write performance, scalability and extensibility.
  • Has a strong and growing community of maintainers and adopters, ensuring its long-term sustainability.
  • Provided diverse channels for community users and contributors to interact, and public meeting links, recordings and notes can be easily found.
  • Provided high-quality, well-organized documentation and practical examples to help users quickly learn and master the project.
  • The project completed the third party security audit by Ada Logics with no high-level or critical issues found. The project team actively resolved the 12 detected issues, clearly demonstrated commitment to security, which is praiseworthy.

The following actions were provided to the project that were considered blocking but have since been resolved:

  • Removed the Project Lead role, previously held by one individual and considered conflicting with community neutrality. And instead, established a Technical Steering Committee (TSC) with a defined number of seats and neutrality requirements.
  • Updated the governance documents to clarify the management rules for subprojects.
  • Added a RELEASE.md file, including updating the release process to reflect the latest engineering principles criteria.
  • Updated the governance documents to inlcude roadmap changing process.

The following recommendations were provided to the project that are non-blocking in the TOC's assessment but should be completed by the project to ensure continued viability of the project:

  • TOC Reviewer recommends finalizing the ongoing Governance Review by the TAG Contributor Strategy to achieve a more comprehensive community governance.
  • TOC Reviewer recommends to extend its current management rules to cover all repos under the GitHub organization, including non-subproject repos, and archive any repos that are no longer being maintained.
  • Make full use of cubefs-community repo, as the canonical location for community governance-related documents.
  • To better support project adopters, TOC Reviewer suggests keeping deep engagement with them and improving the project's extensibility and code history management. This will facilitate easier tracking of community updates.
  • To foster a more inclusive global community, TOC Reviewer recommends making a plan for global community development. This plan may include initiatives like English-language community meetings and cultivating contributors from various regions to better support adopters worldwide.
  • TOC Reviewer recommends to organizing dedicated TSC meeting, in order to keep TSC members engaged.
  • To enhance community decision-making transparency, the TOC Reviewer recommends the project provide explicit records of voting processes, e.g. manual vote counts or using gitvote.

Adoption Evaluation

The adopter interviews reflect the production grade usage of CubeFS for the applied Graduation level. According to the feedback, CubeFS simplifies the interaction between users and storage infrastructure by providing a high-performance, scalable, reliable and easy-to-maintain distributed file system. Developers leverage CubeFS to address the challenges of complex distributed file systems, including large-scale data storage, high concurrent access, data reliability and the complexity of system maintenance. Adopters have been using CubeFS in the production environment for many years, managing data ranging from hundreds of terabytes to petabytes and supporting the access of millions of clients. Overall, the stability, reliable performance and active community of CubeFS have built the trust of adopters.

Final Assessment

The TOC has found the project to have satisfied the criteria for Graduation.

Application Process Principles

Suggested

N/A

Required

  • Give a presentation and engage with the domain specific TAG(s) to increase awareness

    The CubeFS team has given presentation on the TAG Storage meeting on April 24, 2024. link to recording

    The project has requested Governance Review by TAG Contributor Strategy at: cncf/tag-contributor-strategy#649

  • TAG provides insight/recommendation of the project in the context of the landscape

    TAG Storage recommendation can be found in this doc.

    TAG Storage has reviewed CubeFS for its graduation request, provided suggestions for updating CubeFS-csi and CubeFS-helm projects to resolve CVEs, and the team responded quickly to address the issues. We recommend that the CubeFS team continue to keep these projects up to date. We believe that the project’s health is sound in general, the customer adoption has been increasing, and the project is at a mature state and it is ready to move to the graduation level.

  • All project metadata and resources are vendor-neutral.

  • Review and acknowledgement of expectations for graduated projects and requirements for moving forward through the CNCF Maturity levels.

    • Met during Project's application on 10-Apr-2024 as a kick-off meeting.

    The initial PR was submitted on 5-Aug-2023. The TOC Reviewer was assigned on 9-Apr-2024.

    On 10-Apr-2024, the TOC Reviewer and project maintainers met during the kick-off meeting of the Graduation process, discussed the changes and update in the TOC repo regarding the criteria and expectations for moving levels, release, and freeze time period for KubeCons.

    The project provided updates using the recommended template on 18-Jun-2024.

    The TOC Reviewer and project maintainers met multiple time during the due diligence review. Some suggested action items were provided during the review, and the project maintainers have been highly responsive throughout the process.

    Completion of this due diligence document, resolution of concerns raised, and presented for public comment satisifies the Due Diligence Review criteria.

  • Additional documentation as appropriate for project type, e.g.: installation documentation, end user documentation, reference implementation and/or code samples.

Governance and Maintainers

Note: this section may be augmented by the completion of a Governance Review from TAG Contributor Strategy.

Suggested

  • Governance has continuously been iterated upon by the project as a result of their experience applying it, with the governance history demonstrating evolution of maturity alongside the project's maturity evolution.

    CubeFS has been continuously updating governance doc to reflect project growth, some examples are:

Required

Contributors and Community

Note: this section may be augmented by the completion of a Governance Review from TAG Contributor Strategy.

Suggested

Required

Engineering Principles

Suggested

N/A

Required

  • Document project goals and objectives that illustrate the project’s differentiation in the Cloud Native landscape as well as outlines how this project fulfills an outstanding need and/or solves a problem differently.

    Project goal from project README.md#what-can-you-build-with-cubefs:

    As an open-source distributed storage, CubeFS can serve as your datacenter filesystem, data lake storage infra, and private or hybrid cloud storage. In particular, CubeFS enables the separation of storage/compute architecture for databases and AI/ML applications.

    Some key features of CubeFS include:

    • Multiple access protocols such as POSIX, HDFS, S3, and its own REST API
    • Highly scalable metadata service with strong consistency
    • Performance optimization of large/small files and sequential/random writes
    • Multi-tenancy support with better resource utilization and tenant isolation
    • Hybrid cloud I/O acceleration through multi-level caching
    • Flexible storage policies, high-performance replication or low-cost erasure coding

  • Document what the project does, and why it does it - including viable cloud native use cases.

    CubeFS Introduction: link

    According to CubeFS doc, typical use cases are:

    • Big Data Analytics
    • Deep Learning/Machine Learning
    • Container Shared Storage
    • Database & Middleware
    • Online Services
    • Traditional NAS to Cloud

  • Document and maintain a public roadmap or other forward looking planning document or tracking mechanism.

    CubeFS has a public roadmap doc at ROADMAP.md

  • Roadmap change process is documented.

    CubeFS documentes its roadmap rules and changing process in GOVERNANCE.md#roadmap

  • Document overview of project architecture and software design that demonstrates viable cloud native use cases, as part of the project's documentation.

  • Document the project's release process and guidelines publicly in a RELEASES.md or equivalent file that defines:

    • Release expectations (scheduled or based on feature implementation) CubeFS documents their release frequency as needed (beta and official releases), which can be regarded as based on feature implementation.

    • Tagging as stable, unstable, and security related releases CubeFS uses beta to mark their unstable releases. Ref: RELEASE.md#types-of-releases. Security release process is documented at: security-release-process.md

    • Information on branch and tag strategies

      Type Versioning Branch Frequency
      beta vX.Y.Z-beta release-X.Y.Z-Beta as needed (at branch time)
      official vX.Y.Z release-X.Y.Z as needed (post beta)

    • Branch and platform support and length of support No specific description of platform supported, according to the artifacts-included-in-the-release, currently only amd64 binaries are maintained by the community. Length of support clearly documented, support latest 3 minor releases.

    • Artifacts included in the release.

    • Additional information on topics such as LTS and edge releases are optional. Release expectations are a social contract between the project and its end users and hence changes to these should be well thought out, discussed, socialized and as necessary agreed upon by project leadership before getting rolled out. Each release note specifies the artifacts included in the release. Example: CubeFS v3.3.2 release notes

  • History of regular, quality releases.

    History of CubeFS releases and changelogs: https://github.com/cubefs/cubefs/releases

Security

Note: this section may be augemented by a joint-assessment performed by TAG Security.

Suggested

Required

  • Clearly defined and discoverable process to report security issues.

    CubeFS has a clear security vulnerability report guide at: SECURITY.md

  • Enforcing Access Control Rules to secure the code base against attacks (Example: two factor authentication enforcement, and/or use of ACL tools.)

  • Document assignment of security response roles and how reports are handled.

    The CubeFS Security Release Process documents response roles and process of handling reports.

  • Document Security Self-Assessment.

    https://github.com/cubefs/cubefs/blob/6617aa1eb7bf6b63bfacc2c266eeb711c650973f/security/CubeFS-self-assessment.md

  • Third Party Security Review.

    • Moderate and low findings from the Third Party Security Review are planned/tracked for resolution as well as overall thematic findings, such as: improving project contribution guide providing a PR review guide to look for memory leaks and other vulnerabilities the project may be susceptible to by design or language choice ensuring adequate test coverage on all PRs.

    CubeFS has passed the Third Party Security Review, Ref: CubeFS-Security-Audit-2023-report. All found issues have been fixed, ref: page4 in the report "Executive summary". Security advisories of the fixes: link

  • Achieve the Open Source Security Foundation (OpenSSF) Best Practices passing badge.

    CubeFS has achieved OpenSSF Best Practices passing badge: https://www.bestpractices.dev/en/projects/6232

Ecosystem

Suggested

N/A

Required

  • Publicly documented list of adopters, which may indicate their adoption level (dev/trialing, prod, etc.)

    The ADOPTERS.md documentes adopters with adoption level and success stories.

  • Used in appropriate capacity by at least 3 independent + indirect/direct adopters, (these are not required to be in the publicly documented list of adopters)

    The project provided the TOC with a list of adopters for verification of use of the project at the level expected, i.e. production use for graduation. CubeFS has been adopted by a large base of end users, most of which prefer to remain anonymous.

  • TOC verification of adopters.

    CubeFS Maintainers provided the TOC sponsor with a list of 7 users who agreed to be interviewed for the Graduation Due Diligence process. Refer to the Adoption portion of this document for details.

  • Clearly documented integrations and/or compatibility with other CNCF projects as well as non-CNCF projects.

Adoption

Adopter 1 - NetEase / Internet

NetEase has integrated CubeFS into its cloud-native infrastructure since 2020, using it as a distributed file system solution. After a thorough evaluation of various options, CubeFS was selected for its superior scalability and reliability. Currently in production, NetEase's deployment spans over 200 servers and manages more than 15 petabytes of user data.

The core features of CubeFS that NetEase finds most valuable are high availability, fault tolerance, and the ability to handle large data volumes efficiently. These functionalities have made CubeFS an integral part of their cloud operations. The project's compatibility with Kubernetes has also been a significant advantage.

During the implementation phase, NetEase found the documentation quite helpful, although there was an initial challenge in understanding the architecture. They received community support that helped them overcome these initial hurdles.

NetEase upgrades their deployment of CubeFS with one minor release per year and patch releases as needed. CubeFS releases are usually stable and upgrades can be done with automated scripts in most cases.

Looking ahead, NetEase intends to increase its engagement with the CubeFS community and explore additional features that could enhance their cloud services. They view the project as a valuable asset that has not only met their storage needs but also contributed to their learning and development in distributed systems.

The adoption of CubeFS has been a significant benefit for NetEase, enhancing its cloud services with a robust and scalable storage solution. The project's stability and clear scope and roadmap have built trust with NetEase, making it a reliable component for its cloud-native infrastructure needs.

August, 2024

Ref: Adopter Interview - NetEase

Adopter 2 - BEIKE / Housing Transactions and Services

BEIKE has been using CubeFS since early 2020, starting with version 2.3. Currently, they are using version 3.3.1 in a large-scale production environment with a total capacity of 30PB (triple-replica redundancy), and some legacy environments are still running on version 2.4.

The main motivation for adoption was CubeFS's overall read/write and concurrency performance, scalability, as well as the ability to seamlessly integrate with existing infrastructure.

The adoption resulted in saving storage costs and easier debugging and problem resolution. The disk throttling feature has been useful, and the fault tolerance and data consistency features have been crucial in maintaining service continuity, even during peak business periods.

During the migration from NFS to CubeFS, they faced challenges with decreased write performance for small files. They also met issues with the HDFS data integration tool due to lack of maintenance. And BEIKE found the CubeFS documentation to be comprehensive and the community support to be responsive, which has been instrumental in the successful integration and ongoing maintenance of the system.

The community is open and friendly, BEIKE mainly participates in discussions in the WeChat group and attends monthly meetings, and contributes back when possible. Looking ahead, BEIKE hopes to be more involved in feature planning, enhancement discussions, and foster deeper collaboration with other community members.

Overall, CubeFS's strengths include concurrent read and write operations and efficient bandwidth utilization. Areas for improvement include cleaner git commit history management, making features pluggable, and improving compatibility for upgrading historical versions. The community also needs to improve its promotion and outreach efforts.

September, 2024

Ref: Adopter Interview - BEIKE

Adopter3 - Live Streaming and Short Video

Adopter3 started to use CubeFS in the second half of 2022. It began with testing and was deployed for production in January 2023. The key reason for adopting CubeFS was its ability to handle concurrent writes and the capability to horizontally scale metadata nodes. These were major advantages compared to HDFS and CephFS, which had constraints in performance, maintenance, and handling small files.

CubeFS has been convenient to use with its FUSE mounting. It has been integrated with the Kubernetes clusters and is also used directly on physical machines in some scenarios. The community documentation has been helpful to the adopter for learning about APIs, RESTful commands, and the architecture. Some internal business units have transitioned from CephFS to CubeFS, benefiting from reduced maintenance time. The replacement of HDFS with CubeFS is in progress.

In the future, Adopter3 plans to use the EC (Erasure Coding) feature when the community's architecture becomes more stable. Currently, they are focusing on replacing HDFS. Adopter3 has 3 committers in the community, interacting with maintainers primarily through WeChat group discussions.

Overall, the CubeFS project is considered stable. However, there is room for improvement in areas such as covering more corner cases, promoting the project more effectively, and enhancing the format and content of issue and PR descriptions to facilitate better maintenance.

September, 2024

Note: Adopter3 preferred to remain anonymous. The interview details are kept in a private file for CNCF TOC review.

Adopter4 - Online Retailing

JD.com has been using CubeFS since 2018 as the foundation for its entire storage infrastructure. This adoption supports a diverse range of unstructured storage needs across the company's retail operations, including middleware, online and offline business, big data, and AI training, etc. In production for more than 6 years, they have multiple clusters, with the largest one consisting of over 4,000 servers and a total storage capacity exceeding 300TB, serving over 1 million clients concurrently. They use the 2020 version from the community and manually backport bug fixes.

JD.com choses CubeFS because it is customizable for their specific scenarios and supports large-scale clusters. CubeFS supports operating on the same dataset with different protocols simultaneously, like POSIX and S3, while alternatives like CephFS support only one. The adopter has also evaulated MooseFS, however it didn't met their requirements in scalability and stability.

JD.com has found the project documentation invaluable, particularly the design documents, which facilitated a deeper understanding and customization of CubeFS. The adoption has led to significant value, including reduced maintenance and resource costs, and improved resource utilization, reaching up to 98% in some environments.

From the adopter's perspective, the CubeFS community has rational governance and responsive maintainers. JD.com participates by sharing internal solutions, having maintainers review PRs, and interacting via GitHub and WeChat groups.

JD.com has a positive view of the CubeFS community, with rational governance and responsive maintainers. JD.com participates by sharing internal solutions, having maintainers review PRs, and interacting via GitHub and WeChat groups. They will continue to actively engage, contribute solutions, propose features, seeing the relationship as mutually beneficial.

Overall, CubeFS has strengths such as handling extreme scenarios and meeting enterprise requirements. JD.com suggests that the project could benefit from more community events and wider adoption promotion to reach its full potential.

October, 2024

Ref: Adopter Interview - JD.com