Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Enhancement] Use JRaft to Implement Controller #7300

Closed
1 task done
yulangz opened this issue Sep 6, 2023 · 1 comment · Fixed by #7301
Closed
1 task done

[Enhancement] Use JRaft to Implement Controller #7300

yulangz opened this issue Sep 6, 2023 · 1 comment · Fixed by #7301
Labels
type/enhancement

Comments

@yulangz
Copy link
Contributor

yulangz commented Sep 6, 2023 •
edited
Loading

Before Creating the Enhancement Request

  • I have confirmed that this should be classified as an enhancement rather than a bug/feature.

Summary

The Controller is implemented based on DLedger. DLedger is a raft repository specifically designed for RocketMQ with many optimizations made for CommitLog. However, when implementing Controller, these optimizations are not needed, and DLedger does not support log compression (Snapshot), so more mature raft libraries such as JRaft can be considered.

Motivation

Compared to DLedger, jRaft is more mature and reliable, and supports snapshot function, which can solve the problem of unlimited log growth.

Describe the Solution You'd Like

Use JRaft to Implement Controller.

Describe Alternatives You've Considered

Use other mature Raft libraries.

Additional Context

RIP: https://docs.google.com/document/d/1mpzTv1vnWxQwPGsHj6Ng2fK9aL9f6MZFw7ZgvW5284o/edit#heading=h.av2kaasaafu6

No response
@yulangz yulangz mentioned this issue Sep 6, 2023
Merged
@vongosling
Copy link
Member

I must clarify one thing, DLedger is not specifically designed for Apache RocketMQ. It is still in the process of continuous evolution.

RongtongJin pushed a commit that referenced this issue Jan 29, 2024
@yulangz @leizhiyuan
[ISSUE #7300] jRaft-Controller Implemention (#7301)
8df53df 
* jRaft-Controller Implemention

* fix bazel build

* reformat code

* remove fury dependence

* fix bazel build

* resolve conflict

* clear code

* Optimize code style

* Optimize code style

* Optimize code style

* revert style only change
fix code style
rollback an unexpected modification.

* Update Producer.java

* chore: move raft startup to start method

* chore: use jraftconfig to collect all configs about jraft

* fix: fix wrong store path because init use string constant

* fix: fix CONTROLLER_NOT_LEADER error in follower

* chore: seperate jraft and controller log

* chore: fix conflict with develop

* chore: add comment to clear the filter logic

* feat: triggerElectMaster will retry when failed

* feat: when controller all restart, we use a timestamp to trace the first heartbeat, avoid to elect again

* fix: implements Serializable to enable snapshot serialize

* fix: use for loop to simple the elect retry

* chore: update jraft version

* chore: opt import

---------

Co-authored-by: leizhiyuan <leizhiyuan@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
type/enhancement
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[ISSUE #7300] jRaft-Controller Implemention yulangz/rocketmq
2 participants
@vongosling @yulangz