MinIO文件夹临时鉴权

烂柯 发布于 2022-12-28 747 次阅读

简介

MinIO 一款高性能、分布式的对象存储系统. 它是一款开源、 S3 兼容的软件产品, 可以100%的运行在标准硬件。

​ 由于开发场景中有大量小文件或图片，使用时需要对某个文件夹进行临时授权访问，而在对象存储中安全级别教高，基本只有对单个文件对象进行临时授权访问（分享），因此并不太复合我们项目的使用场景，文件夹鉴权方案没有找到合适的（有可能是我关键词没有找对）。

​ 最初方案是直接将minio存储桶设置为公共只读通过Yarp（Yarp.ReverseProxy）代理自定义鉴权，考虑到有上传功能存在暴露真实地址，于是便打算自己调整源码临时权限（对go并不熟，希望大家指出相关问题或优化）。

官网：https://min.io/

国内官网：https://www.minio.org.cn/

github：https://github.com/minio

一 、环境

安装golang（https://golang.google.cn/）环境，设置国内镜像源，使用到的IDE有 vs code、vs2022

go env -w GOPROXY=https://goproxy.cn,direct

image-20221228103832380

二、搞到源码

分别fork minio服务端源码和SDK源码并拉取。

image-20221228101505390

image-20221228102915055

三、分析定位分享权限

1、整体"分析代码"结构

这个是了解项目的首要步骤，但是我这里做分析，找到主入口简单看下就可以了（感兴趣的小伙伴可以找下专门解析Minio的博客）

2、通过mc客户端生成分享链接

image-20221228104620934

http://192.168.21.140:8060/test/g.jpg?

X-Amz-Algorithm=AWS4-HMAC-SHA256&

X-Amz-Credential=minioadmin%2F20221228%2Fus-east-1%2Fs3%2Faws4_request&

X-Amz-Date=20221228T024609Z&X-Amz-Expires=3600&

X-Amz-SignedHeaders=host

&X-Amz-Signature=0043bf03da583b6690fd98a901cee8f3ab3c4299c1e7cc81011bda1d015a9311

3、通过链接中的鉴权内容在code中进行全局搜索

image-20221228104858670

根据分享的请求方式及signature、函数注释，定位我们需要调整的doesPresignedSignatureMatch函数。因为我不太

image-20221228105058674

4、跑起来

配置调试文件 launch.json就可以直接F5调试运行了，或者通过命令 go run main.go server ../../data 进行运行测试

{
    // 使用 IntelliSense 了解相关属性。 
    // 悬停以查看现有属性的描述。
    // 欲了解更多信息，请访问: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Launch Package",
            "type": "go",
            "request": "launch",
            "mode": "auto",
            "program": "main.go",
            "args": ["server","../../data"]
        }
    ]
}

5、分析修改方案

根据调试请求结果（这里的请求分析最好和客户端SDK一起调试运行会更容易些）来考虑如何修改进而减少影响。

因为我的场景需要对文件夹进行权限控制，当然这可能降低了文件的安全性，这就要在SDK上做好一定的授权控制。

整体思路是在授权时将文件夹路径进入链接及加密内容中，鉴权时获取当前请求绝对路径判断是否在授权文件夹中并进行权限验证，如果请求中没有文件夹路径则按照单文件授权进行验证（这样做主要是减少对minio的影响）。

image-20221228112448867

四、修改分享权限

1、添加参与鉴权的header

headers.go

    // Signature V4 related contants.
    AmzContentSha256        = "X-Amz-Content-Sha256"
    AmzDate                 = "X-Amz-Date"
    AmzAlgorithm            = "X-Amz-Algorithm"
    AmzExpires              = "X-Amz-Expires"
    AmzSignedHeaders        = "X-Amz-SignedHeaders"
    AmzSignature            = "X-Amz-Signature"
    AmzCredential           = "X-Amz-Credential"
    AmzSecurityToken        = "X-Amz-Security-Token"
    AmzDecodedContentLength = "X-Amz-Decoded-Content-Length"
    AmzTrailer              = "X-Amz-Trailer"
    //授权文件夹
    AmzFolderPath           = "X-Amz-Folder-Path"

2、修改鉴权逻辑

signature-v4.go

func doesPresignedSignatureMatch(hashedPayload string, r *http.Request, region string, stype serviceType) APIErrorCode {
    ...
    defaultSigParams := set.CreateStringSet(
        xhttp.AmzContentSha256,
        xhttp.AmzSecurityToken,
        xhttp.AmzAlgorithm,
        xhttp.AmzDate,
        xhttp.AmzExpires,
        xhttp.AmzSignedHeaders,
        xhttp.AmzCredential,
        xhttp.AmzSignature,
        //授权文件夹
        xhttp.AmzFolderPath,
    )
    ...
    //校验请求路径与文件夹路径是否一致
    //校验请求路径与文件夹路径是否一致
    queryFolderPath := req.Form.Get(xhttp.AmzFolderPath)
    reqUrlPath := s3utils.EncodePath(req.URL.Path)
    if  len(queryFolderPath)==0{
        queryFolderPath=reqUrlPath
    }
    if !strings.HasPrefix(reqUrlPath,queryFolderPath) {
        return ErrSignatureDoesNotMatch
    }
    ...
    // Verify finally if signature is same.

    // Get canonical request.
    presignedCanonicalReq := getCanonicalRequest(extractedSignedHeaders, hashedPayload, encodedQuery, queryFolderPath, req.Method)
    ...
}

3、修改SDK授权逻辑

V4Authenticator.cs

    ...
    internal string PresignURL(HttpRequestMessageBuilder requestBuilder, int expires, string region = "",
        string sessionToken = "", DateTime? reqDate = null, string queryFolderPath = "")
    {
        ...
        var canonicalRequest = GetPresignCanonicalRequest(requestBuilder.Method, presignUri, headersToSign, queryFolderPath);
        ...
        // Return presigned url.
        var signedUri = new UriBuilder(presignUri)
        {
            Query = $"{requestQuery}{headers}{(string.IsNullOrWhiteSpace(queryFolderPath) ? string.Empty : $"&X-Amz-Folder-Path={Uri.EscapeDataString(queryFolderPath)}")}&X-Amz-Signature={signature}"
        };
        ...
    }
    ...

ObjectOperationsArgs.cs

...
public class PresignedGetFolderPathArgs : ObjectArgs<PresignedGetFolderPathArgs>
{
    public PresignedGetFolderPathArgs()
    {
        RequestMethod = HttpMethod.Get;
    }

    internal int Expiry { get; set; }
    internal DateTime? RequestDate { get; set; }
    internal string FolderPath { get; set; }

    internal override void Validate()
    {
        base.Validate();
        if (!utils.IsValidExpiry(Expiry))
            throw new InvalidExpiryRangeException("expiry range should be between 1 and " +
                                                  Constants.DefaultExpiryTime);
        if (string.IsNullOrWhiteSpace(FolderPath) || FolderPath.LastIndexOf('/') + 1 != FolderPath.Length)
            throw new InvalidObjectNameException(FolderPath, "文件夹路径需以'/'结尾");
        FolderPath = string.Concat("/",BucketName, "/", FolderPath.TrimStart('/'));
    }

    public PresignedGetFolderPathArgs WithExpiry(int expiry)
    {
        Expiry = expiry;
        return this;
    }

    public PresignedGetFolderPathArgs WithRequestDate(DateTime? d)
    {
        RequestDate = d;
        return this;
    }
    /// <summary>
    /// 设置分享文件夹
    /// </summary>
    /// <param name="folderPath">不包含桶名</param>
    /// <returns></returns>
    public PresignedGetFolderPathArgs WithFolderPath(string folderPath)
    {
        FolderPath = folderPath;
        //添加默认值绕过Object Name校验
        ObjectName = "default";
        return this;
    }
}
...

MinioClientBuilder.cs

    ...
    Task<string> PresignedGetFolderPathAsync(string uri, PresignedGetFolderPathArgs args);
    ...

IObjectOperations.cs

    ...
    Task<string> PresignedGetObjectAsync(PresignedGetObjectArgs args);
//新增函数
    /// <summary>
    ///  Presigned get url - returns a presigned url to access an object's data without credentials.URL can have a maximum
    ///     expiry of
    ///     up to 7 days or a minimum of 1 second.Additionally, you can override a set of response headers using reqParams.
    /// </summary>
    /// <param name="uri">访问时的请求地址（参与签名影响访问）</param>
    /// <param name="args"></param>
    /// <returns></returns>
    Task<string> PresignedGetFolderPathAsync(string uri,PresignedGetFolderPathArgs args);
    ...

ObjectOperations.cs

    ...
    public async Task<string> PresignedGetObjectAsync(PresignedGetObjectArgs args)
    {
        args.Validate();
        var requestMessageBuilder = await CreateRequest(args).ConfigureAwait(false);
        var authenticator = new V4Authenticator(Secure, AccessKey, SecretKey, Region,
            SessionToken);
        return authenticator.PresignURL(requestMessageBuilder, args.Expiry, Region, SessionToken, args.RequestDate);
    }

    /// <summary>
    ///  获取授权请求参数
    /// </summary>
    /// <param name="uri">访问时的请求地址（参与签名影响访问）</param>
    /// <param name="args"></param>
    /// <returns></returns>
    public async Task<string> PresignedGetFolderPathAsync(string uri,PresignedGetFolderPathArgs args)
    {
        args.Validate();
        var requestMessageBuilder = await CreateRequest(args).ConfigureAwait(false);
        var authenticator = new V4Authenticator(Secure, AccessKey, SecretKey, Region,
            SessionToken);
        requestMessageBuilder.RequestUri=new Uri(uri);
        string uriString = authenticator.PresignURL(requestMessageBuilder, args.Expiry, Region, SessionToken, args.RequestDate,args.FolderPath);
        return new Uri(uriString).Query;
    }
    ...

五、测试

1、启动minio服务端，并上传测试文件

image-20221228155023270

//文件结构如下
--test
----g.jpg
----test1
------g.jpg
----test2
------g.jpg

2、SDK调用生成请求参数

...
var minio = new MinioClient()
    .WithEndpoint("192.168.21.109:9000")
    .WithCredentials("minioadmin",
    "minioadmin")
    .WithSSL(false)
    .Build();
//这里的uri参数和下面测试访问的地址保持一致，否则鉴权不通过
string url = minio.PresignedGetFolderPathAsync("http://127.0.0.1:9000",new PresignedGetFolderPathArgs()
    .WithBucket("test")
    .WithFolderPath("/")
    .WithExpiry(10_000))
    .Result;
...

3、生成根文件夹请求参数

image-20221228155813354

4、生成test1文件夹请求参数

image-20221228161604167

5、生成test2文件夹请求参数

image-20221228161729060

六、MinIO打包

//设置环境变量
set GOARCH=amd64
//linux windows
set GOOS=linux
go build
//go build main.go

七、代码地址

https://github.com/VectorlanKe/minio

https://github.com/VectorlanKe/minio-dotnet

烂柯