[ISSUE #7142] Add command `RocksDBConfigToJson` to inspect rocksdb content #7180

Ziy1-Tan · 2023-08-14T05:02:20Z

Which Issue(s) This PR Fixes

Brief Description

case get topic config
mqadmin RocksDBConfigToJson -p /home/xxx/store/config/topics -t topics

	"topicConfigTable":{
		"SCHEDULE_TOPIC_XXXX":{
			"readQueueNums":18,
			"perm":6,
			"writeQueueNums":18,
			"topicFilterType":"SINGLE_TAG",
			"topicName":"SCHEDULE_TOPIC_XXXX",
			"attributes":{},
			"order":false,
			"topicSysFlag":0
		},
		"RMQ_SYS_TRANS_HALF_TOPIC":{
			"readQueueNums":1,
			"perm":6,
			"writeQueueNums":1,
			"topicFilterType":"SINGLE_TAG",
			"topicName":"RMQ_SYS_TRANS_HALF_TOPIC",
			"attributes":{},
			"order":false,
			"topicSysFlag":0
		},
// ...
}

case get subscription config
mqadmin RocksDBConfigToJson -p /home/xxx/store/config/subscriptionGroups -t subscriptionGroups

{
	"subscriptionGroupTable":{
		"CID_ONSAPI_OWNER":{
			"retryMaxTimes":16,
			"whichBrokerWhenConsumeSlowly":1,
			"brokerId":0,
			"groupRetryPolicy":{
				"type":"CUSTOMIZED"
			},
			"consumeFromMinEnable":true,
			"consumeTimeoutMinute":15,
			"groupName":"CID_ONSAPI_OWNER",
			"groupSysFlag":0,
			"consumeEnable":true,
			"consumeBroadcastEnable":true,
			"retryQueueNums":1,
			"notifyConsumerIdsChangedEnable":true,
			"consumeMessageOrderly":false,
			"attributes":{}
		},
//...
}

How Did You Test This Change?

TBD

allow append to file

RongtongJin · 2023-08-14T06:21:16Z

tools/src/main/java/org/apache/rocketmq/tools/command/metadata/KvConfigToJsonCommand.java

+import java.util.HashMap;
+import java.util.Map;
+
+public class KvConfigToJsonCommand implements SubCommand {


How about renaming to RocksDBConfigToJsonCommand？

Good! Let me take a look.

Signed-off-by: Ziy1-Tan <ajb4596984460@gmail.com>

fuyou001 · 2023-08-14T07:19:56Z

tools/src/main/java/org/apache/rocketmq/tools/command/metadata/RocksDBConfigToJsonCommand.java

+    @Override
+    public void execute(CommandLine commandLine, Options options, RPCHook rpcHook) throws SubCommandException {
+        String path = commandLine.getOptionValue("path").trim();
+        if (StringUtils.isEmpty(path) || !new File(path).exists()) {


support default path configuration？

Metadata is stored in different paths, it may be more appropriate to specify the path.

fuyou001 · 2023-08-14T07:22:30Z

tools/src/main/java/org/apache/rocketmq/tools/command/metadata/RocksDBConfigToJsonCommand.java

+                }
+                topicsJsonConfig.put("topicConfigTable", (JSONObject) JSONObject.toJSON(topicConfigTable));
+                final String topicsJsonStr = JSONObject.toJSONString(topicsJsonConfig, true);
+                System.out.print(topicsJsonStr + "\n");


The topic size is too large, causing Out Of Memory (OOM) errors.

You could use the stream approach to write to the file, instead of reusing the previous code.

How about mqadmin rocksDBConfig -p /home/xxx/store/config/topics -t topics > topics.json?

RongtongJin · 2023-08-14T11:47:30Z

tools/src/main/java/org/apache/rocketmq/tools/command/metadata/RocksDBConfigToJsonCommand.java

+
+    @Override
+    public String commandName() {
+        return "RocksDBConfigToJson";


rocksDBConfigToJson, lowercase the first letter like any other command.

codecov-commenter · 2023-08-15T03:37:49Z

Codecov Report

Merging #7180 (4366490) into develop (f534501) will decrease coverage by 0.05%.
Report is 4 commits behind head on develop.
The diff coverage is 24.00%.

@@              Coverage Diff              @@
##             develop    #7180      +/-   ##
=============================================
- Coverage      42.52%   42.48%   -0.05%     
- Complexity      9357     9363       +6     
=============================================
  Files           1149     1150       +1     
  Lines          82043    82093      +50     
  Branches       10701    10706       +5     
=============================================
- Hits           34889    34877      -12     
- Misses         42792    42850      +58     
- Partials        4362     4366       +4

Files Changed	Coverage Δ
.../apache/rocketmq/tools/command/MQAdminStartup.java	`0.00% <0.00%> (ø)`
...s/command/metadata/RocksDBConfigToJsonCommand.java	`24.48% <24.48%> (ø)`

... and 21 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

RongtongJin changed the title ~~feat: add command kvConfigToJson to inspect rocksdb content~~ [ISSUE #7142] Add command kvConfigToJson to inspect rocksdb content Aug 14, 2023

Ziy1-Tan force-pushed the kv_json branch from ff4b58c to d5c212d Compare August 14, 2023 06:17

RongtongJin reviewed Aug 14, 2023

View reviewed changes

feat: add command RocksDBConfigToJson to inspect rocksdb content

c8c446d

Signed-off-by: Ziy1-Tan <ajb4596984460@gmail.com>

Ziy1-Tan force-pushed the kv_json branch from d5c212d to c8c446d Compare August 14, 2023 06:36

Ziy1-Tan changed the title ~~[ISSUE #7142] Add command kvConfigToJson to inspect rocksdb content~~ [ISSUE #7142] Add command RocksDBConfigToJson to inspect rocksdb content Aug 14, 2023

fuyou001 reviewed Aug 14, 2023

View reviewed changes

RongtongJin reviewed Aug 14, 2023

View reviewed changes

refactor: fix style

4366490

RongtongJin approved these changes Aug 17, 2023

View reviewed changes

RongtongJin merged commit 744167b into apache:develop Aug 23, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ISSUE #7142] Add command `RocksDBConfigToJson` to inspect rocksdb content #7180

[ISSUE #7142] Add command `RocksDBConfigToJson` to inspect rocksdb content #7180

Ziy1-Tan commented Aug 14, 2023 •

edited

Loading

RongtongJin Aug 14, 2023

Ziy1-Tan Aug 14, 2023

fuyou001 Aug 14, 2023

Ziy1-Tan Aug 15, 2023

fuyou001 Aug 14, 2023

Ziy1-Tan Aug 15, 2023 •

edited

Loading

RongtongJin Aug 14, 2023

Ziy1-Tan Aug 15, 2023

codecov-commenter commented Aug 15, 2023

[ISSUE #7142] Add command RocksDBConfigToJson to inspect rocksdb content #7180

[ISSUE #7142] Add command RocksDBConfigToJson to inspect rocksdb content #7180

Conversation

Ziy1-Tan commented Aug 14, 2023 • edited Loading

Which Issue(s) This PR Fixes

Brief Description

How Did You Test This Change?

TBD

RongtongJin Aug 14, 2023

Choose a reason for hiding this comment

Ziy1-Tan Aug 14, 2023

Choose a reason for hiding this comment

fuyou001 Aug 14, 2023

Choose a reason for hiding this comment

Ziy1-Tan Aug 15, 2023

Choose a reason for hiding this comment

fuyou001 Aug 14, 2023

Choose a reason for hiding this comment

Ziy1-Tan Aug 15, 2023 • edited Loading

Choose a reason for hiding this comment

RongtongJin Aug 14, 2023

Choose a reason for hiding this comment

Ziy1-Tan Aug 15, 2023

Choose a reason for hiding this comment

codecov-commenter commented Aug 15, 2023

Codecov Report

[ISSUE #7142] Add command `RocksDBConfigToJson` to inspect rocksdb content #7180

[ISSUE #7142] Add command `RocksDBConfigToJson` to inspect rocksdb content #7180

Ziy1-Tan commented Aug 14, 2023 •

edited

Loading

Ziy1-Tan Aug 15, 2023 •

edited

Loading