Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ISSUE #7560] [RIP-68] Support RocketMQ ACL 2.0 #7725

Merged
merged 80 commits into from
Mar 18, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
80 commits
Select commit Hold shift + click to select a range
55ca65f
xinz moxing
Nov 9, 2023
fe6f60a
init acl 2.0 code.
Nov 9, 2023
06945d6
init acl 2.0 code.
Dec 20, 2023
fd686bd
init acl 2.0 code.
Dec 20, 2023
0c8b413
init acl 2.0 code.
Dec 20, 2023
b77e4e6
add fastjson2 to bazel
Dec 20, 2023
acd18d3
Merge remote-tracking branch 'upstream/develop' into feature/support_…
Dec 21, 2023
1cbc0a0
Change the Pipeline to HandlerChain
Dec 21, 2023
28cb71a
Add ContextInitPipeline for remoting and gRPC.
Dec 22, 2023
f44e480
optimize the code.
Dec 22, 2023
358f4d0
optimize the code.
Dec 22, 2023
904b007
add init user api, optimize some code and fix some bugs.
Dec 25, 2023
bf287bb
Add cache for auth metadata.
Dec 25, 2023
9663aa8
remove no use dependency
Dec 25, 2023
b69a959
Add annotation to request header
Dec 25, 2023
f49d833
Fix no topics request header.
Dec 25, 2023
16af549
add whitelist and optimize some code.
Dec 26, 2023
861d79e
optimize some code.
Dec 26, 2023
915164f
add tests, and optimize some code.
Dec 27, 2023
ed93ea4
add test for AuthorizationMetadataManager
Dec 27, 2023
f6edd75
add test for AuthorizationEvaluator and AuthenticationEvaluator.
Dec 27, 2023
206b1e4
add test for DefaultAuthenticationContextBuilder.
Dec 27, 2023
5fb1f47
change the method of init user.
Dec 28, 2023
bdcdf65
add copy user and acl command.
Dec 31, 2023
05e5f40
test and fix admin bugs.
Jan 3, 2024
a939a45
change the broker auth from rpchook to pipeline.
Jan 3, 2024
5966afa
fix auth admin bugs.
Jan 4, 2024
29e52ff
change the symbol to constants.
Jan 4, 2024
bc2b53e
add strategy for authentication and authorization.
Jan 5, 2024
9d522c7
fix tests
Jan 5, 2024
865d029
rollback maven version
Jan 5, 2024
bd5b123
fix test bugs.
Jan 5, 2024
678a004
fix check style bugs.
Jan 5, 2024
90332b0
support authConfig is null.
Jan 5, 2024
66a05eb
fix bugs
Jan 5, 2024
8b0bb3a
fix tests.
Jan 8, 2024
3cb0020
fix tests.
Jan 8, 2024
fa77a95
optimize codes.
Jan 8, 2024
8241027
Merge remote-tracking branch 'upstream/develop' into feature/support_…
Jan 9, 2024
9cc2333
fix conflicts with develop.
Jan 9, 2024
3e26667
set auth default provider.
Jan 12, 2024
b245307
change handler to chain.
Jan 16, 2024
5a71fa1
Merge remote-tracking branch 'upstream/develop' into feature/support_…
Jan 16, 2024
7988e40
add action for heartbeat request header.
Jan 16, 2024
67c7558
add action for heartbeat request header.
Jan 16, 2024
02d1821
add action for heartbeat request header.
Jan 16, 2024
8eb5eeb
fix header import.
Jan 16, 2024
e840386
fix unused import
Jan 16, 2024
38f4e49
add tests.
Jan 18, 2024
d167579
add tests.
Jan 18, 2024
8f8b3e4
add tests.
Jan 18, 2024
2d28beb
add tests.
dingshuangxi888 Jan 18, 2024
9f2077c
add audit logs for auth.
dingshuangxi888 Jan 22, 2024
fd35b75
change the acl create and update logic
dingshuangxi888 Jan 24, 2024
cff5807
remote Comparable
dingshuangxi888 Jan 25, 2024
c65615d
Merge remote-tracking branch 'upstream/develop' into feature/support_…
dingshuangxi888 Jan 25, 2024
c0012d4
merge master
dingshuangxi888 Jan 25, 2024
933aa5a
部分字段不要序列化
dingshuangxi888 Jan 25, 2024
3a7c3f2
fix test
dingshuangxi888 Jan 25, 2024
4e7a185
fix test
dingshuangxi888 Jan 25, 2024
a728695
modify the log file.
dingshuangxi888 Jan 26, 2024
b9bcfc5
change error message
dingshuangxi888 Jan 31, 2024
939588b
Merge remote-tracking branch 'upstream/develop' into feature/support_…
dingshuangxi888 Mar 5, 2024
3e35478
change the version.
dingshuangxi888 Mar 5, 2024
9a21f1c
fix sourceIp for grpc
dingshuangxi888 Mar 7, 2024
8f8fd53
fix username is null.
dingshuangxi888 Mar 7, 2024
74eaa82
fix username is null.
dingshuangxi888 Mar 7, 2024
e7c6279
优化无权限日志
dingshuangxi888 Mar 8, 2024
1c2b3b7
优化无权限日志
dingshuangxi888 Mar 8, 2024
ee030be
优化无权限日志
dingshuangxi888 Mar 8, 2024
73df6f0
optimize the code.
dingshuangxi888 Mar 11, 2024
7f8c4ed
optimize the code.
dingshuangxi888 Mar 12, 2024
782305e
修改鉴权失败错误码信息
dingshuangxi888 Mar 13, 2024
0b4f025
修改鉴权失败错误码信息
dingshuangxi888 Mar 13, 2024
24d0176
修改AUTH字段命名
dingshuangxi888 Mar 14, 2024
3351a0e
修复端口转发场景中,获取客户端IP错误问题
dingshuangxi888 Mar 15, 2024
dd52e86
fix test.
dingshuangxi888 Mar 15, 2024
98eda1a
fix test.
dingshuangxi888 Mar 15, 2024
12afde9
add haproxy message extension and fix test
dingshuangxi888 Mar 18, 2024
1bd9ba6
Merge remote-tracking branch 'upstream/develop' into feature/support_…
dingshuangxi888 Mar 18, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
add strategy for authentication and authorization.
  • Loading branch information
ShuangxiDing committed Jan 5, 2024
commit bc2b53e8ce747262314e1bcc26b961fba97f4362
Original file line number Diff line number Diff line change
Expand Up @@ -16,64 +16,25 @@
*/
package org.apache.rocketmq.auth.authentication;

import java.util.ArrayList;
import java.util.List;
import java.util.function.Supplier;
import org.apache.commons.lang3.StringUtils;
import org.apache.rocketmq.auth.authentication.context.AuthenticationContext;
import org.apache.rocketmq.auth.authentication.exception.AuthenticationException;
import org.apache.rocketmq.auth.authentication.factory.AuthenticationFactory;
import org.apache.rocketmq.auth.authentication.provider.AuthenticationProvider;
import org.apache.rocketmq.auth.authentication.strategy.AuthenticationStrategy;
import org.apache.rocketmq.auth.config.AuthConfig;
import org.apache.rocketmq.common.utils.ExceptionUtils;

public class AuthenticationEvaluator {

private final AuthConfig authConfig;
private final List<String> authenticationWhitelist = new ArrayList<>();
private final AuthenticationProvider<AuthenticationContext> authenticationProvider;
private final AuthenticationStrategy authenticationStrategy;

public AuthenticationEvaluator(AuthConfig authConfig) {
this(authConfig, null);
}

public AuthenticationEvaluator(AuthConfig authConfig, Supplier<?> metadataService) {
this.authConfig = authConfig;
this.authenticationProvider = AuthenticationFactory.getProvider(authConfig);
if (this.authenticationProvider != null) {
this.authenticationProvider.initialize(authConfig, metadataService);
}
if (StringUtils.isNotBlank(authConfig.getAuthenticationWhitelist())) {
String[] whitelist = StringUtils.split(authConfig.getAuthenticationWhitelist(), ",");
for (String rpcCode : whitelist) {
this.authenticationWhitelist.add(StringUtils.trim(rpcCode));
}
}
this.authenticationStrategy = AuthenticationFactory.getStrategy(authConfig, metadataService);
}

public void evaluate(AuthenticationContext context) {
if (context == null) {
return;
}
if (!authConfig.isAuthenticationEnabled()) {
return;
}
if (this.authenticationProvider == null) {
return;
}
if (this.authenticationWhitelist.contains(context.getRpcCode())) {
return;
}
try {
this.authenticationProvider.authenticate(context).join();
} catch (AuthenticationException ex) {
throw ex;
} catch (Throwable ex) {
Throwable exception = ExceptionUtils.getRealException(ex);
if (exception instanceof AuthenticationException) {
throw (AuthenticationException) exception;
}
throw new AuthenticationException("Failed to authentication the request", exception);
}
this.authenticationStrategy.evaluate(context);
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -18,11 +18,12 @@

import com.google.protobuf.GeneratedMessageV3;
import io.grpc.Metadata;
import io.netty.channel.ChannelHandlerContext;
import org.apache.rocketmq.remoting.protocol.RemotingCommand;

public interface AuthenticationContextBuilder<AuthenticationContext> {

AuthenticationContext build(Metadata metadata, GeneratedMessageV3 request);

AuthenticationContext build(RemotingCommand request);
AuthenticationContext build(ChannelHandlerContext context, RemotingCommand request);
}
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@

import com.google.protobuf.GeneratedMessageV3;
import io.grpc.Metadata;
import io.netty.channel.ChannelHandlerContext;
import java.nio.charset.StandardCharsets;
import java.util.HashMap;
import java.util.Map;
Expand Down Expand Up @@ -47,6 +48,7 @@ public class DefaultAuthenticationContextBuilder implements AuthenticationContex
public DefaultAuthenticationContext build(Metadata metadata, GeneratedMessageV3 request) {
try {
DefaultAuthenticationContext context = new DefaultAuthenticationContext();
context.setChannelId(metadata.get(GrpcConstants.CHANNEL_ID));
context.setRpcCode(request.getDescriptorForType().getFullName());
String authorization = metadata.get(GrpcConstants.AUTHORIZATION);
if (StringUtils.isEmpty(authorization)) {
Expand Down Expand Up @@ -94,15 +96,16 @@ public DefaultAuthenticationContext build(Metadata metadata, GeneratedMessageV3
}

@Override
public DefaultAuthenticationContext build(RemotingCommand request) {
public DefaultAuthenticationContext build(ChannelHandlerContext context, RemotingCommand request) {
HashMap<String, String> fields = request.getExtFields();
if (MapUtils.isEmpty(fields)) {
throw new AuthenticationException("authentication field is null");
}
DefaultAuthenticationContext context = new DefaultAuthenticationContext();
context.setRpcCode(String.valueOf(request.getCode()));
context.setUsername(fields.get(SessionCredentials.ACCESS_KEY));
context.setSignature(fields.get(SessionCredentials.SIGNATURE));
DefaultAuthenticationContext result = new DefaultAuthenticationContext();
result.setChannelId(context.channel().id().asLongText());
result.setRpcCode(String.valueOf(request.getCode()));
result.setUsername(fields.get(SessionCredentials.ACCESS_KEY));
result.setSignature(fields.get(SessionCredentials.SIGNATURE));
// Content
SortedMap<String, String> map = new TreeMap<>();
for (Map.Entry<String, String> entry : fields.entrySet()) {
Expand All @@ -114,8 +117,8 @@ public DefaultAuthenticationContext build(RemotingCommand request) {
map.put(entry.getKey(), entry.getValue());
}
}
context.setContent(AclUtils.combineRequestContent(request, map));
return context;
result.setContent(AclUtils.combineRequestContent(request, map));
return result;
}

public String hexToBase64(String input) throws DecoderException {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,20 @@

public abstract class AuthenticationContext {

private String channelId;

private String rpcCode;

private Map<String, Object> extInfo;

public String getChannelId() {
return channelId;
}

public void setChannelId(String channelId) {
this.channelId = channelId;
}

public String getRpcCode() {
return rpcCode;
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@

import com.google.protobuf.GeneratedMessageV3;
import io.grpc.Metadata;
import io.netty.channel.ChannelHandlerContext;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentMap;
import java.util.function.Function;
Expand All @@ -29,6 +30,8 @@
import org.apache.rocketmq.auth.authentication.manager.AuthenticationMetadataManagerImpl;
import org.apache.rocketmq.auth.authentication.provider.AuthenticationMetadataProvider;
import org.apache.rocketmq.auth.authentication.provider.AuthenticationProvider;
import org.apache.rocketmq.auth.authentication.strategy.AuthenticationStrategy;
import org.apache.rocketmq.auth.authentication.strategy.StatefulAuthenticationStrategy;
import org.apache.rocketmq.auth.config.AuthConfig;
import org.apache.rocketmq.remoting.protocol.RemotingCommand;

Expand Down Expand Up @@ -103,6 +106,19 @@ public static AuthenticationEvaluator getEvaluator(AuthConfig config, Supplier<?
return computeIfAbsent(EVALUATOR_PREFIX + config.getConfigName(), key -> new AuthenticationEvaluator(config, metadataService));
}

@SuppressWarnings("unchecked")
public static AuthenticationStrategy getStrategy(AuthConfig config, Supplier<?> metadataService) {
try {
Class<? extends AuthenticationStrategy> clazz = StatefulAuthenticationStrategy.class;
if (StringUtils.isNotBlank(config.getAuthenticationStrategy())) {
clazz = (Class<? extends AuthenticationStrategy>) Class.forName(config.getAuthenticationStrategy());
}
return clazz.getDeclaredConstructor(AuthConfig.class, Supplier.class).newInstance(config, metadataService);
} catch (Exception e) {
throw new RuntimeException(e);
}
}

public static AuthenticationContext newContext(AuthConfig config, Metadata metadata, GeneratedMessageV3 request) {
AuthenticationProvider<AuthenticationContext> authenticationProvider = getProvider(config);
if (authenticationProvider == null) {
Expand All @@ -111,12 +127,12 @@ public static AuthenticationContext newContext(AuthConfig config, Metadata metad
return authenticationProvider.newContext(metadata, request);
}

public static AuthenticationContext newContext(AuthConfig config, RemotingCommand command) {
public static AuthenticationContext newContext(AuthConfig config, ChannelHandlerContext context, RemotingCommand command) {
AuthenticationProvider<AuthenticationContext> authenticationProvider = getProvider(config);
if (authenticationProvider == null) {
return null;
}
return authenticationProvider.newContext(command);
return authenticationProvider.newContext(context, command);
}

@SuppressWarnings("unchecked")
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@

import com.google.protobuf.GeneratedMessageV3;
import io.grpc.Metadata;
import io.netty.channel.ChannelHandlerContext;
import java.util.concurrent.CompletableFuture;
import java.util.function.Supplier;
import org.apache.rocketmq.auth.config.AuthConfig;
Expand All @@ -31,5 +32,5 @@ public interface AuthenticationProvider<AuthenticationContext> {

AuthenticationContext newContext(Metadata metadata, GeneratedMessageV3 request);

AuthenticationContext newContext(RemotingCommand command);
AuthenticationContext newContext(ChannelHandlerContext context, RemotingCommand command);
}
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@

import com.google.protobuf.GeneratedMessageV3;
import io.grpc.Metadata;
import io.netty.channel.ChannelHandlerContext;
import java.util.concurrent.CompletableFuture;
import java.util.function.Supplier;
import org.apache.rocketmq.auth.authentication.builder.AuthenticationContextBuilder;
Expand Down Expand Up @@ -52,8 +53,8 @@ public DefaultAuthenticationContext newContext(Metadata metadata, GeneratedMessa
}

@Override
public DefaultAuthenticationContext newContext(RemotingCommand command) {
return this.authenticationContextBuilder.build(command);
public DefaultAuthenticationContext newContext(ChannelHandlerContext context, RemotingCommand command) {
return this.authenticationContextBuilder.build(context, command);
}

protected HandlerChain<DefaultAuthenticationContext, CompletableFuture<Void>> newHandlerChain() {
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.rocketmq.auth.authentication.strategy;

import java.util.ArrayList;
import java.util.List;
import java.util.function.Supplier;
import org.apache.commons.lang3.StringUtils;
import org.apache.rocketmq.auth.authentication.context.AuthenticationContext;
import org.apache.rocketmq.auth.authentication.exception.AuthenticationException;
import org.apache.rocketmq.auth.authentication.factory.AuthenticationFactory;
import org.apache.rocketmq.auth.authentication.provider.AuthenticationProvider;
import org.apache.rocketmq.auth.config.AuthConfig;
import org.apache.rocketmq.common.utils.ExceptionUtils;

public abstract class AbstractAuthenticationStrategy implements AuthenticationStrategy {

protected final AuthConfig authConfig;
protected final List<String> authenticationWhitelist = new ArrayList<>();
protected final AuthenticationProvider<AuthenticationContext> authenticationProvider;

public AbstractAuthenticationStrategy(AuthConfig authConfig, Supplier<?> metadataService) {
this.authConfig = authConfig;
this.authenticationProvider = AuthenticationFactory.getProvider(authConfig);
if (this.authenticationProvider != null) {
this.authenticationProvider.initialize(authConfig, metadataService);
}
if (StringUtils.isNotBlank(authConfig.getAuthenticationWhitelist())) {
String[] whitelist = StringUtils.split(authConfig.getAuthenticationWhitelist(), ",");
for (String rpcCode : whitelist) {
this.authenticationWhitelist.add(StringUtils.trim(rpcCode));
}
}
}

protected void doEvaluate(AuthenticationContext context) {
if (context == null) {
return;
}
if (!authConfig.isAuthenticationEnabled()) {
return;
}
if (this.authenticationProvider == null) {
return;
}
if (this.authenticationWhitelist.contains(context.getRpcCode())) {
return;
}
try {
this.authenticationProvider.authenticate(context).join();
} catch (AuthenticationException ex) {
throw ex;
} catch (Throwable ex) {
Throwable exception = ExceptionUtils.getRealException(ex);
if (exception instanceof AuthenticationException) {
throw (AuthenticationException) exception;
}
throw new AuthenticationException("Failed to authentication the request", exception);
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.rocketmq.auth.authentication.strategy;

import org.apache.rocketmq.auth.authentication.context.AuthenticationContext;

public interface AuthenticationStrategy {

void evaluate(AuthenticationContext context);
}
Loading