Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

有几个问题 #5

Open
2512132839 opened this issue Jan 26, 2025 · 22 comments
Open

有几个问题 #5

2512132839 opened this issue Jan 26, 2025 · 22 comments

Comments

@2512132839
Copy link

全部视频和博客仔细的看了,来来回回折腾了几个小时,部署后出现了一些问题不知道是项目本身有小bug还是哪里没操作对。

1.我直接部署的是B2 + R2 + GitHub + GitLab的,全部配置完后图片文件都能上传,检测节点状态也有对应存储的状态显示。但是通过picgo选择渠道R2/B2上传后,访问链接“https://自定义域/文件名" 时,显示出来的都是404,只有通过action“s3_to_github.yml“拉到自己配置的node仓库后,再次访问链接才有显示,镜像到gitlab也没用。

Image
尝试了”指定文件获取平台“这几个命令包括"from=where",只有github是有效果的,其余全都是404。

2.访问对应链接的github渠道的图片,如果图片名是中文,也是会出现404

Image

3.关于b2通过action备份到github过程中,恰好github备份策略选择'true'。如果出现失败,会导致两边都没有对应的文件。

这是我的B2/R2picgo配置:

Image

Image
感谢大佬开源,希望大佬指点一下

@fscarmen2
Copy link
Owner

项目太久太久了,视频出来后又优化了一下,我要花点时间看看先。

@fscarmen2
Copy link
Owner




麻烦你替代以下的,即是从 // 用户配置区域结束 ================================= 下面的所有直接替换,再测试一下

// 用户配置区域结束 =================================

// 检查配置是否有效
function hasValidConfig() {
  // 检查 GitHub 配置
  const hasGithub = GITHUB_PAT && GITHUB_USERNAME && GITLAB_CONFIGS && 
                    GITLAB_CONFIGS.length > 0 && 
                    GITLAB_CONFIGS.some(config => config.name && config.id && config.token);

  // 检查 GitLab 配置
  const hasGitlab = GITLAB_CONFIGS && 
                    GITLAB_CONFIGS.length > 0 && 
                    GITLAB_CONFIGS.some(config => config.name && config.id && config.token);

  // 检查 R2 配置
  const hasR2 = R2_CONFIGS && 
                R2_CONFIGS.length > 0 && 
                R2_CONFIGS.some(config => 
                  config.name && 
                  config.accountId && 
                  config.accessKeyId && 
                  config.secretAccessKey && 
                  config.bucket
                );

  // 检查 B2 配置
  const hasB2 = B2_CONFIGS && 
                B2_CONFIGS.length > 0 && 
                B2_CONFIGS.some(config => 
                  config.name && 
                  config.endPoint && 
                  config.keyId && 
                  config.applicationKey && 
                  config.bucket
                );

  return {
    github: hasGithub,
    gitlab: hasGitlab,
    r2: hasR2,
    b2: hasB2
  };
}

// AWS SDK 签名相关函数开始 =================================

// 获取签名URL
async function getSignedUrl(config, method, path) {
  const region = 'auto';
  const service = 's3';

  // 根据配置类型确定 host 和认证信息
  const host = config.endPoint
    ? config.endPoint  // B2 配置使用 endPoint
    : `${config.accountId}.r2.cloudflarestorage.com`;  // R2 配置使用默认格式

  // 根据配置类型确定认证信息
  const accessKeyId = config.endPoint ? config.keyId : config.accessKeyId;
  const secretKey = config.endPoint ? config.applicationKey : config.secretAccessKey;

  const datetime = new Date().toISOString().replace(/[:-]|\.\d{3}/g, '');
  const date = datetime.substr(0, 8);

  const canonicalRequest = [
    method,
    '/' + path,
    '',
    `host:${host}`,
    'x-amz-content-sha256:UNSIGNED-PAYLOAD',
    `x-amz-date:${datetime}`,
    '',
    'host;x-amz-content-sha256;x-amz-date',
    'UNSIGNED-PAYLOAD'
  ].join('\n');

  const stringToSign = [
    'AWS4-HMAC-SHA256',
    datetime,
    `${date}/${region}/${service}/aws4_request`,
    await sha256(canonicalRequest)
  ].join('\n');

  const signature = await getSignature(
    secretKey,
    date,
    region,
    service,
    stringToSign
  );

  const authorization = [
    `AWS4-HMAC-SHA256 Credential=${accessKeyId}/${date}/${region}/${service}/aws4_request`,
    `SignedHeaders=host;x-amz-content-sha256;x-amz-date`,
    `Signature=${signature}`
  ].join(', ');

  return {
    url: `https://${host}/${path}`,
    headers: {
      'Authorization': authorization,
      'x-amz-content-sha256': 'UNSIGNED-PAYLOAD',
      'x-amz-date': datetime,
      'Host': host
    }
  };
}

// SHA256 哈希函数
async function sha256(message) {
  const msgBuffer = new TextEncoder().encode(message);
  const hashBuffer = await crypto.subtle.digest('SHA-256', msgBuffer);
  return Array.from(new Uint8Array(hashBuffer))
    .map(b => b.toString(16).padStart(2, '0'))
    .join('');
}

// HMAC-SHA256 函数
async function hmacSha256(key, message) {
  const keyBuffer = key instanceof ArrayBuffer ? key : new TextEncoder().encode(key);
  const messageBuffer = new TextEncoder().encode(message);

  const cryptoKey = await crypto.subtle.importKey(
    'raw',
    keyBuffer,
    { name: 'HMAC', hash: 'SHA-256' },
    false,
    ['sign']
  );

  const signature = await crypto.subtle.sign(
    'HMAC',
    cryptoKey,
    messageBuffer
  );

  return signature;
}

// 获取签名
async function getSignature(secret, date, region, service, stringToSign) {
  const kDate = await hmacSha256('AWS4' + secret, date);
  const kRegion = await hmacSha256(kDate, region);
  const kService = await hmacSha256(kRegion, service);
  const kSigning = await hmacSha256(kService, 'aws4_request');
  const signature = await hmacSha256(kSigning, stringToSign);

  return Array.from(new Uint8Array(signature))
    .map(b => b.toString(16).padStart(2, '0'))
    .join('');
}

// AWS SDK 签名相关函数结束 =================================

// 检查服务函数
async function getGitHubUsername(pat) {
  const url = 'https://api.github.com/user';
  try {
    const response = await fetch(url, {
      headers: {
        'Authorization': `token ${pat}`,
        'Accept': 'application/vnd.github.v3+json',
        'User-Agent': 'Cloudflare Worker'
      }
    });

    if (response.status === 200) {
      const data = await response.json();
      return data.login;
    } else {
      console.error('GitHub API Error:', response.status);
      return 'Unknown';
    }
  } catch (error) {
    console.error('GitHub request error:', error);
    return 'Error';
  }
}

// 修改文件路径处理函数
function getFilePath(basePath, requestPath) {
  // 移除开头的斜杠
  const cleanRequestPath = requestPath.replace(/^\//, '');
  
  // 如果没有设置 basePath,直接返回请求路径
  if (!basePath) return cleanRequestPath;
  
  // 组合基础路径和请求路径
  return `${basePath}/${cleanRequestPath}`;
}


// 检查 GitHub 仓库
async function checkGitHubRepo(owner, repo, pat) {
  const repoUrl = `https://api.github.com/repos/${owner}/${repo}`;
  const contentsUrl = `https://api.github.com/repos/${owner}/${repo}/contents/${DIR}`;

  const headers = {
    'Authorization': `token ${pat}`,
    'Accept': 'application/vnd.github.v3+json',
    'User-Agent': 'Cloudflare Worker'
  };

  try {
    // 并发请求获取仓库信息和目录内容
    const [repoResponse, contentsResponse] = await Promise.all([
      fetch(repoUrl, { headers }),
      fetch(contentsUrl, { headers })
    ]);

    const repoData = await repoResponse.json();

    if (repoResponse.status !== 200) {
      throw new Error(`Repository error: ${repoData.message}`);
    }

    if (contentsResponse.status !== 200) {
      return [`working (${repoData.private ? 'private' : 'public'})`, 0, 0];
    }

    const contentsData = await contentsResponse.json();

    // 计算文件数量和总大小
    const fileStats = contentsData.reduce((acc, item) => {
      if (item.type === 'file') {
        return {
          count: acc.count + 1,
          size: acc.size + (item.size || 0)
        };
      }
      return acc;
    }, { count: 0, size: 0 });

    return [
      `working (${repoData.private ? 'private' : 'public'})`,
      fileStats.count,
      fileStats.size
    ];

  } catch (error) {
    console.error(`Error checking GitHub repo ${repo}:`, error);
    return [`error: ${error.message}`, 0, 0];
  }
}

// 检查 GitLab 项目
async function checkGitLabProject(projectId, pat) {
  const projectUrl = `https://gitlab.com/api/v4/projects/${projectId}`;
  const filesUrl = `https://gitlab.com/api/v4/projects/${projectId}/repository/tree?ref=main&path=${DIR}&recursive=true&per_page=10000`;

  try {
    const [projectResponse, filesResponse] = await Promise.all([
      fetch(projectUrl, {
        headers: { 'PRIVATE-TOKEN': pat }
      }),
      fetch(filesUrl, {
        headers: { 'PRIVATE-TOKEN': pat }
      })
    ]);

    if (projectResponse.status === 200) {
      const projectData = await projectResponse.json();
      let fileCount = 0;

      if (filesResponse.status === 200) {
        const filesData = await filesResponse.json();
        fileCount = filesData.filter(item => item.type === 'blob').length;
      }

      return [
        `working (${projectData.visibility})`,
        projectData.owner.username,
        fileCount
      ];
    } else if (projectResponse.status === 404) {
      return ['not found', 'Unknown', 0];
    } else {
      return ['disconnect', 'Unknown', 0];
    }
  } catch (error) {
    console.error('GitLab project check error:', error);
    return ['disconnect', 'Error', 0];
  }
}

// 检查 R2 存储
async function checkR2Storage(r2Config) {
  try {
    const listPath = `${r2Config.bucket}`;
    const signedRequest = await getSignedUrl(r2Config, 'GET', listPath);

    const response = await fetch(signedRequest.url, {
      headers: signedRequest.headers
    });

    let fileCount = 0;
    let totalSize = 0;

    if (response.ok) {
      const data = await response.text();
      const keys = data.match(/<Key>([^<]+)<\/Key>/g) || [];
      const sizes = data.match(/<Size>(\d+)<\/Size>/g) || [];

      keys.forEach((key, index) => {
        const filePath = key.replace(/<Key>|<\/Key>/g, '');
        if (filePath.startsWith(DIR + '/')) {
          fileCount++;
          const size = parseInt(sizes[index]?.replace(/<Size>|<\/Size>/g, '') || String(0), 10);
          totalSize += size;
        }
      });
    }

    const status = response.ok ? 'working' : 'error';

    return [
      status,
      r2Config.name,
      r2Config.bucket,
      fileCount,
      formatSize(totalSize)
    ];
  } catch (error) {
    console.error('R2 Storage error:', error);
    return ['error', r2Config.name, 'connection failed', 0, '0 B'];
  }
}

// 检查 B2 存储
async function checkB2Storage(b2Config) {
  try {
    const listPath = `${b2Config.bucket}`;
    const signedRequest = await getSignedUrl(b2Config, 'GET', listPath);

    const response = await fetch(signedRequest.url, {
      headers: signedRequest.headers
    });

    let fileCount = 0;
    let totalSize = 0;

    if (response.ok) {
      const data = await response.text();
      const keys = data.match(/<Key>([^<]+)<\/Key>/g) || [];
      const sizes = data.match(/<Size>(\d+)<\/Size>/g) || [];

      keys.forEach((key, index) => {
        const filePath = key.replace(/<Key>|<\/Key>/g, '');
        if (filePath.startsWith(DIR + '/')) {
          fileCount++;
          const size = parseInt(sizes[index]?.replace(/<Size>|<\/Size>/g, '') || String(0), 10);
          totalSize += size;
        }
      });
    }

    const status = (response.status === 404 || response.status === 403 || response.ok) ? 'working' : 'error';

    return [
      status,
      b2Config.name,
      b2Config.bucket,
      fileCount,
      formatSize(totalSize)
    ];
  } catch (error) {
    console.error('B2 Storage error:', error);
    return ['error', b2Config.name, 'connection failed', 0, '0 B'];
  }
}

// 文件大小格式化函数
function formatSize(sizeInBytes) {
  if (sizeInBytes >= 1024 * 1024 * 1024) {
    return `${(sizeInBytes / (1024 * 1024 * 1024)).toFixed(2)} GB`;
  } else if (sizeInBytes >= 1024 * 1024) {
    return `${(sizeInBytes / (1024 * 1024)).toFixed(2)} MB`;
  } else {
    return `${(sizeInBytes / 1024).toFixed(2)} kB`;
  }
}

export default {
  async fetch(request, env, ctx) {
    const url = new URL(request.url);
    const from = url.searchParams.get('from')?.toLowerCase();
    const validConfigs = hasValidConfig();

    // 获取完整的请求路径
    const requestPath = decodeURIComponent(url.pathname);
    const FILE = requestPath.split('/').pop();
    const fullPath = DIR ? `${DIR}` : '';

    // 直接使用 GITLAB_CONFIGS 中的 name 作为 GitHub 仓库名
    const githubRepos = GITLAB_CONFIGS.map(config => config.name);

    // 只在没有 from 参数时才检查和使用缓存
    let cacheResponse;
    if (!from) {
      const cacheUrl = new URL(request.url);
      const cacheKey = new Request(cacheUrl.toString(), request);
      const cache = caches.default;
      cacheResponse = await cache.match(cacheKey);

      if (cacheResponse) {
        return cacheResponse;
      }
    }

    // 检查状态页面
    if (url.pathname === `/${CHECK_PASSWORD}`) {
      let result = '';
      let hasAnyValidConfig = false;

      try {
        // GitHub 状态检查
        if (validConfigs.github) {
          hasAnyValidConfig = true;
          result += '=== GitHub Status ===\n';
          const username = await getGitHubUsername(GITHUB_PAT);
          for (const repo of githubRepos) {
            const [status, fileCount, totalSize] = await checkGitHubRepo(GITHUB_USERNAME, repo, GITHUB_PAT);
            const formattedSize = formatSize(totalSize);
            result += `GitHub: ${repo} - ${status} (Username: ${username}, Files: ${fileCount}, Size: ${formattedSize})\n`;
          }
        }

        // GitLab 状态检查
        if (validConfigs.gitlab) {
          hasAnyValidConfig = true;
          result += result ? '\n=== GitLab Status ===\n' : '=== GitLab Status ===\n';
          for (const config of GITLAB_CONFIGS) {
            const [status, username, fileCount] = await checkGitLabProject(config.id, config.token);
            result += `GitLab: Project ID ${config.id} - ${status} (Username: ${username}, Files: ${fileCount})\n`;
          }
        }

        // R2 状态检查
        if (validConfigs.r2) {
          hasAnyValidConfig = true;
          result += result ? '\n=== R2 Storage Status ===\n' : '=== R2 Storage Status ===\n';
          for (const config of R2_CONFIGS) {
            const [status, name, bucket, fileCount, totalSize] = await checkR2Storage(config);
            result += `R2 Storage: ${name} - ${status} (Bucket: ${bucket}, Files: ${fileCount}, Size: ${totalSize})\n`;
          }
        }

        // B2 状态检查
        if (validConfigs.b2) {
          hasAnyValidConfig = true;
          result += result ? '\n=== B2 Storage Status ===\n' : '=== B2 Storage Status ===\n';
          for (const config of B2_CONFIGS) {
            const [status, name, bucket, fileCount, totalSize] = await checkB2Storage(config);
            result += `B2 Storage: ${name} - ${status} (Bucket: ${bucket}, Files: ${fileCount}, Size: ${totalSize})\n`;
          }
        }

        // 如果没有任何有效配置
        if (!hasAnyValidConfig) {
          result = 'No storage services configured.\n';
        } else {
          result = 'Storage status:\n\n' + result;
        }

      } catch (error) {
        result += `Error during status check: ${error.message}\n`;
      }

      return new Response(result, {
        headers: {
          'Content-Type': 'text/plain',
          'Access-Control-Allow-Origin': '*'
        }
      });
    }

    const startTime = Date.now();
    let requests = [];

    // 检查特定服务的请求是否有效
    if (from) {
      if (from === 'github' && !validConfigs.github) {
        return new Response('GitHub service is not configured.', {
          status: 400,
          headers: { 'Content-Type': 'text/plain', 'Access-Control-Allow-Origin': '*' }
        });
      }
      if (from === 'gitlab' && !validConfigs.gitlab) {
        return new Response('GitLab service is not configured.', {
          status: 400,
          headers: { 'Content-Type': 'text/plain', 'Access-Control-Allow-Origin': '*' }
        });
      }
      if (from === 'r2' && !validConfigs.r2) {
        return new Response('R2 storage service is not configured.', {
          status: 400,
          headers: { 'Content-Type': 'text/plain', 'Access-Control-Allow-Origin': '*' }
        });
      }
      if (from === 'b2' && !validConfigs.b2) {
        return new Response('B2 storage service is not configured.', {
          status: 400,
          headers: { 'Content-Type': 'text/plain', 'Access-Control-Allow-Origin': '*' }
        });
      }
    }

    // 生成存储请求
    const generateStorageRequests = async () => {
      let requests = [];

      if (validConfigs.r2) {
        const r2Requests = await Promise.all(R2_CONFIGS.map(async (r2Config) => {
          const r2Path = `${r2Config.bucket}/${fullPath}/${FILE}`.replace(/\/+/g, '/');
          const signedRequest = await getSignedUrl(r2Config, 'GET', r2Path);
          return {
            url: signedRequest.url,
            headers: signedRequest.headers,
            source: 'r2',
            repo: `${r2Config.name} (${r2Config.bucket})`
          };
        }));
        requests = [...requests, ...r2Requests];
      }

      if (validConfigs.b2) {
        const b2Requests = await Promise.all(B2_CONFIGS.map(async (b2Config) => {
          const b2Path = `${b2Config.bucket}/${fullPath}/${FILE}`.replace(/\/+/g, '/');
          const signedRequest = await getSignedUrl(b2Config, 'GET', b2Path);
          return {
            url: signedRequest.url,
            headers: signedRequest.headers,
            source: 'b2',
            repo: `${b2Config.name} (${b2Config.bucket})`
          };
        }));
        requests = [...requests, ...b2Requests];
      }

      return requests;
    };

    // 处理不同类型的请求
    if (from === 'where') {
      if (validConfigs.github) {
        const githubRequests = githubRepos.map(repo => ({
          url: `https://api.github.com/repos/${GITHUB_USERNAME}/${repo}/contents/${fullPath}/${FILE}`,
          headers: {
            'Authorization': `token ${GITHUB_PAT}`,
            'Accept': 'application/vnd.github.v3+json',
            'User-Agent': 'Cloudflare Worker'
          },
          source: 'github',
          repo: repo,
          processResponse: async (response) => {
            if (!response.ok) throw new Error('Not found');
            const data = await response.json();
            return {
              size: data.size,
              exists: true
            };
          }
        }));
        requests = [...requests, ...githubRequests];
      }

      if (validConfigs.gitlab) {
        const gitlabRequests = GITLAB_CONFIGS.map(config => ({
          url: `https://gitlab.com/api/v4/projects/${config.id}/repository/files/${encodeURIComponent(`${fullPath}/${FILE}`.replace(/\/+/g, '/'))}/raw?ref=main`,
          headers: {
            'PRIVATE-TOKEN': config.token
          },
          source: 'gitlab',
          repo: config.name,
          processResponse: async (response) => {
            if (!response.ok) throw new Error('Not found');
            const data = await response.json();
            const size = atob(data.content).length;
            return {
              size: size,
              exists: true
            };
          }
        }));
        requests = [...requests, ...gitlabRequests];
      }

      const storageRequests = await generateStorageRequests();
      const storageWhereRequests = storageRequests.map(request => ({
        ...request,
        processResponse: async (response) => {
          if (!response.ok) throw new Error('Not found');
          const size = response.headers.get('content-length');
          return {
            size: parseInt(size),
            exists: true
          };
        }
      }));
      requests = [...requests, ...storageWhereRequests];

    } else {
      // 获取文件内容模式
      if (from === 'github' && validConfigs.github) {
        requests = githubRepos.map(repo => ({
          url: `https://raw.githubusercontent.com/${GITHUB_USERNAME}/${repo}/main/${fullPath}/${FILE}`,
          headers: {
            'Authorization': `token ${GITHUB_PAT}`,
            'User-Agent': 'Cloudflare Worker'
          },
          source: 'github',
          repo: repo
        }));
      } else if (from === 'gitlab' && validConfigs.gitlab) {
        requests = GITLAB_CONFIGS.map(config => ({
          url: `https://gitlab.com/api/v4/projects/${config.id}/repository/files/${encodeURIComponent(`${fullPath}/${FILE}`.replace(/\/+/g, '/'))}/raw?ref=main`,
          headers: {
            'PRIVATE-TOKEN': config.token
          },
          source: 'gitlab',
          repo: config.name
        }));
      } else if ((from === 'r2' && validConfigs.r2) || (from === 'b2' && validConfigs.b2)) {
        requests = await generateStorageRequests();
        requests = requests.filter(req => req.source === from);
      } else if (!from) {
        if (validConfigs.github) {
          const githubRequests = githubRepos.map(repo => ({
            url: `https://raw.githubusercontent.com/${GITHUB_USERNAME}/${repo}/main/${fullPath}/${FILE}`,
            headers: {
              'Authorization': `token ${GITHUB_PAT}`,
              'User-Agent': 'Cloudflare Worker'
            },
            source: 'github',
            repo: repo
          }));
          requests = [...requests, ...githubRequests];
        }

        if (validConfigs.gitlab) {
          const gitlabRequests = GITLAB_CONFIGS.map(config => ({
            url: `https://gitlab.com/api/v4/projects/${config.id}/repository/files/${encodeURIComponent(`${fullPath}/${FILE}`)}/raw?ref=main`,
            headers: {
              'PRIVATE-TOKEN': config.token
            },
            source: 'gitlab',
            repo: config.name
          }));
          requests = [...requests, ...gitlabRequests];
        }

        const storageRequests = await generateStorageRequests();
        requests = [...requests, ...storageRequests];
      }
    }

    // 处理请求和响应
    try {
      if (requests.length === 0) {
        throw new Error('No valid source specified or no valid configurations found');
      }

      const fetchPromises = requests.map(request => {
        const { url, headers, source, repo, processResponse } = request;

        return fetch(new Request(url, {
          method: 'GET',
          headers: headers
        })).then(async response => {
          if (from === 'where' && typeof processResponse === 'function') {
            try {
              const result = await processResponse(response);
              const endTime = Date.now();
              const duration = endTime - startTime;

              const formattedSize = result.size > 1024 * 1024
                ? `${(result.size / (1024 * 1024)).toFixed(2)} MB`
                : `${(result.size / 1024).toFixed(2)} kB`;

              return {
                fileName: FILE,
                size: formattedSize,
                source: `${source} (${repo})`,
                duration: `${duration}ms`
              };
            } catch (error) {
              throw new Error(`Not found in ${source} (${repo})`);
            }
          } else {
            if (!response.ok) {
              throw new Error(`Not found in ${source} (${repo})`);
            }
            return response;
          }
        }).catch(error => {
          throw new Error(`Error in ${source} (${repo}): ${error.message}`);
        });
      });

      const result = await Promise.any(fetchPromises);

      let response;
      if (from === 'where') {
        response = new Response(JSON.stringify(result, null, 2), {
          headers: {
            'Content-Type': 'application/json',
            'Access-Control-Allow-Origin': '*'
          }
        });
      } else if (result instanceof Response) {
        const blob = await result.blob();
        const headers = {
          'Content-Type': result.headers.get('Content-Type') || 'application/octet-stream',
          'Access-Control-Allow-Origin': '*'
        };

        if (from) {
          headers['Cache-Control'] = 'no-store, no-cache, must-revalidate, proxy-revalidate';
          headers['Pragma'] = 'no-cache';
          headers['Expires'] = '0';
        }

        response = new Response(blob, {
          status: 200,
          headers: headers
        });
      } else {
        throw new Error("Unexpected result type");
      }

      if (!from && from !== 'where') {
        const cacheUrl = new URL(request.url);
        const cacheKey = new Request(cacheUrl.toString(), request);
        const cache = caches.default;
        ctx.waitUntil(cache.put(cacheKey, response.clone()));
      }

      return response;

    } catch (error) {
      const sourceText = from === 'where'
        ? 'in any repository'
        : from
          ? `from ${from}`
          : 'in any configured storage';

      const errorResponse = new Response(
        `404: Cannot find ${FILE} ${sourceText}. ${error.message}`,
        {
          status: 404,
          headers: {
            'Content-Type': 'text/plain',
            'Access-Control-Allow-Origin': '*'
          }
        }
      );

      return errorResponse;
    }
  }
}

@2512132839
Copy link
Author

2512132839 commented Jan 27, 2025

今天又试了一下,上传到R2、B2同时备份到github/gitlab的图片以“自定义域名/文件名”的形式都能显示了,但是目前的问题是:
1.关于文件名,涉及到中文的话,显示界面是404乱码的

Image

2.在‘true’的情况下,对于B2的action我发现好像有点问题,时不时会出现如下图的这种错误,会导致B2那边删除了文件,github这边没存到文件。甚至有时候action成功了,日志显示放在对应的仓库节点node后,也会出现两边的文件都没存储到的情况。B2的存储桶权限我是按照博客和视频给予了应该没有出入,麻烦大佬复现一下
Image

辛苦大佬修复下

@fscarmen2
Copy link
Owner

今天又试了一下,上传到R2、B2同时备份到github/gitlab的图片以“自定义域名/文件名”的形式都能显示了

是用今天发的代码?

@2512132839
Copy link
Author

2512132839 commented Jan 27, 2025

今天又试了一下,上传到R2、B2同时备份到github/gitlab的图片以“自定义域名/文件名”的形式都能显示了

是用今天发的代码?

嗯,picgo的S3插件我也换成一模一样的了,昨天的不太一样

@fscarmen2
Copy link
Owner

在‘true’的情况下,对于B2的action我发现好像有点问题,时不时会出现如下图的这种错误,会导致B2那边删除了文件,github这边没存到文件。甚至有时候action成功了,日志显示放在对应的仓库节点node后,也会出现两边的文件都没存储到的情况。B2的存储桶权限我是按照博客和视频给予了应该没有出入,麻烦大佬复现一下

我看你的文件名含有中文的,暂时先不要用,全部英文来试一下。

@2512132839
Copy link
Author

2512132839 commented Jan 27, 2025

在‘true’的情况下,对于B2的action我发现好像有点问题,时不时会出现如下图的这种错误,会导致B2那边删除了文件,github这边没存到文件。甚至有时候action成功了,日志显示放在对应的仓库节点node后,也会出现两边的文件都没存储到的情况。B2的存储桶权限我是按照博客和视频给予了应该没有出入,麻烦大佬复现一下

我看你的文件名含有中文的,暂时先不要用,全部英文来试一下。

大佬,试过了,启动action后显示成功了,R2的文件移过去了实际也确实移过去了。而B2的日志也显示移过去了。但是实际上仓库的对应节点并没有。B2的原本文件也被删除了

Image

Image

@fscarmen2
Copy link
Owner

是一直如此还是偶尔?

@2512132839
Copy link
Author

是一直如此还是偶尔?

昨天是偶尔,今天换代码后,B2 action试了三次都是这样

@fscarmen2
Copy link
Owner

换代码只是处理了显示路径,不会影响ac的,一时搞不清了,难道b2官方改了东西?你先用着r2

@fscarmen2
Copy link
Owner

这几天断断续续加了个从各个节点删除文件的接口,具体可以看看项目 README.md。
另外你使用中,b2 文件备份不到 Github ,是不是有子路径的?还是什么情况下发生 ?总是还是偶尔?

@2512132839
Copy link
Author

这几天断断续续加了个从各个节点删除文件的接口,具体可以看看项目 README.md。 另外你使用中,b2 文件备份不到 Github ,是不是有子路径的?还是什么情况下发生 ?总是还是偶尔?

我看了看之前的action的记录好像b2备份到github一直都是不行🤥。b2桶下里就一个文件夹files,里面就一张英文命名的测试图片。
worker里对应的集群目录也是'files',最新代码也更新了,就单单只有b2备份到node节点仓库有问题。大佬复现的出来吗?

Image

Image

@fscarmen2
Copy link
Owner

好的,我花个时间测试一下这里,这段时间没有处理这块。花了挺多时间处理删除文件的接口,这个对于我和其他网友都挺有用,有时候上传图片马上发现没有用,可以能立刻快速删除

@2512132839
Copy link
Author

好的,我花个时间测试一下这里,这段时间没有处理这块。花了挺多时间处理删除文件的接口,这个对于我和其他网友都挺有用,有时候上传图片马上发现没有用,可以能立刻快速删除

好的感谢大佬,大佬有考虑搞一个web的ui界面吗?

@fscarmen2
Copy link
Owner

我试了一只,只有在 B2 桶出现以下问题的时候,才会有可能报错

像这些有带 * 号的文件,其实已经是删除的了,但删除不干净,导致同步 ac 时报错。
Image

Image

以下是把所有的删除后,恢复正常了

Image

你看一下你的报错信息是什么,清空桶后再加测试文件试试。

@fscarmen2
Copy link
Owner

大佬有考虑搞一个web的ui界面吗?

有能力自己弄一个方案专属的当然是最好的。

但我对ui这个完全不会,连问ai都不知道如何问,也不会调试或者故障处理。

方案最大的作用就是上传、访问和删除呢。

上传方面:可以用 PicGO ,PicList,盘络助手,兰空图床这些应用或者浏览器插件这些相对成熟的方案。已经可以满足上传的需求啦。

删除接口又刚做好。

还是由于技术不够,所以暂时没有计划弄了。

@2512132839
Copy link
Author

2512132839 commented Feb 3, 2025

你看一下你的报错信息是什么,清空桶后再加测试文件试试。

b2和github仓库以及对应令牌全部重新弄了一遍,和之前一样显示成功,什么报错都没。但b2到github还是一样不行,对应节点仍然是空的。

Image

Image

Image

@fscarmen2
Copy link
Owner

fscarmen2 commented Feb 4, 2025

已经复现并更新了指令,你把 s3_to_github.yml文件里的 uses: fscarmen2/r2_to_github@v1.0.2 最后改为 v1.0.3,你试试后告诉我啦。

@2512132839
Copy link
Author

已经复现并更新了指令,你把 s3_to_github.yml文件里的 uses: fscarmen2/r2_to_github@v1.0.2 最后改为 v1.0.3,你试试后告诉我啦。

可以了,感谢大佬付出。
目前还有一个问题:
第一种情况:在b2/r2迁移图片到github(删除),同时备份到gitlab后。直接“自定义/图片名”是可以访问的,但是"?from=where或者from=gitlab"会显示404被拒绝,而from=github则是正常显示。
第二种情况:在b2/r2备份图片到github(不删除),同时备份到gitlab后。直接“自定义/图片名”是可以访问的,但是"from=gitlab"会显示404被拒绝,而from=github和from=b2/r2则是正常显示。

Image

Image

Image

@fscarmen2
Copy link
Owner

fscarmen2 commented Feb 6, 2025

之前添加子路径支持功能的时候有些地方没有处理好。修了一下,我这里测试可以,麻烦你抽时间帮看看。如果你也没有问题,我就更新到项目里了。

// 用户配置区域结束 =================================

// 检查配置是否有效
function hasValidConfig() {
  // 检查 GitHub 配置
  const hasGithub = GITHUB_PAT && GITHUB_USERNAME && GITLAB_CONFIGS &&
                    GITLAB_CONFIGS.length > 0 &&
                    GITLAB_CONFIGS.some(config => config.name && config.id && config.token);

  // 检查 GitLab 配置
  const hasGitlab = GITLAB_CONFIGS &&
                    GITLAB_CONFIGS.length > 0 &&
                    GITLAB_CONFIGS.some(config => config.name && config.id && config.token);

  // 检查 R2 配置
  const hasR2 = R2_CONFIGS &&
                R2_CONFIGS.length > 0 &&
                R2_CONFIGS.some(config =>
                  config.name &&
                  config.accountId &&
                  config.accessKeyId &&
                  config.secretAccessKey &&
                  config.bucket
                );

  // 检查 B2 配置
  const hasB2 = B2_CONFIGS &&
                B2_CONFIGS.length > 0 &&
                B2_CONFIGS.some(config =>
                  config.name &&
                  config.endPoint &&
                  config.keyId &&
                  config.applicationKey &&
                  config.bucket
                );

  return {
    github: hasGithub,
    gitlab: hasGitlab,
    r2: hasR2,
    b2: hasB2
  };
}

// AWS SDK 签名相关函数开始 =================================

// 获取签名URL
async function getSignedUrl(config, method, path, queryParams = {}) {
  const region = config.endPoint ? config.endPoint.split('.')[1] : 'auto';
  const service = 's3';
  const host = config.endPoint || `${config.accountId}.r2.cloudflarestorage.com`;
  const accessKeyId = config.endPoint ? config.keyId : config.accessKeyId;
  const secretKey = config.endPoint ? config.applicationKey : config.secretAccessKey;
  const datetime = new Date().toISOString().replace(/[:-]|\.\d{3}/g, '');
  const date = datetime.substr(0, 8);

  // 确保路径正确编码,但保留斜杠
  const encodedPath = path.split('/')
    .map(segment => encodeURIComponent(segment))
    .join('/');

  // 构建规范请求
  const canonicalHeaders = `host:${host}\nx-amz-content-sha256:UNSIGNED-PAYLOAD\nx-amz-date:${datetime}\n`;
  const signedHeaders = 'host;x-amz-content-sha256;x-amz-date';

  // 按字母顺序排序查询参数
  const sortedParams = Object.keys(queryParams).sort().reduce((acc, key) => {
    acc[key] = queryParams[key];
    return acc;
  }, {});

  const canonicalQueryString = Object.entries(sortedParams)
    .map(([key, value]) => `${encodeURIComponent(key)}=${encodeURIComponent(value)}`)
    .join('&');

  const canonicalRequest = [
    method,
    '/' + encodedPath,
    canonicalQueryString,
    canonicalHeaders,
    signedHeaders,
    'UNSIGNED-PAYLOAD'
  ].join('\n');

  const stringToSign = [
    'AWS4-HMAC-SHA256',
    datetime,
    `${date}/${region}/${service}/aws4_request`,
    await sha256(canonicalRequest)
  ].join('\n');

  const signature = await getSignature(
    secretKey,
    date,
    region,
    service,
    stringToSign
  );

  const authorization = [
    `AWS4-HMAC-SHA256 Credential=${accessKeyId}/${date}/${region}/${service}/aws4_request`,
    `SignedHeaders=${signedHeaders}`,
    `Signature=${signature}`
  ].join(', ');

  const url = `https://${host}/${encodedPath}${canonicalQueryString ? '?' + canonicalQueryString : ''}`;

  return {
    url,
    headers: {
      'Authorization': authorization,
      'x-amz-content-sha256': 'UNSIGNED-PAYLOAD',
      'x-amz-date': datetime,
      'Host': host
    }
  };
}

// SHA256 哈希函数
async function sha256(message) {
  const msgBuffer = new TextEncoder().encode(message);
  const hashBuffer = await crypto.subtle.digest('SHA-256', msgBuffer);
  return Array.from(new Uint8Array(hashBuffer))
    .map(b => b.toString(16).padStart(2, '0'))
    .join('');
}

// HMAC-SHA256 函数
async function hmacSha256(key, message) {
  const keyBuffer = key instanceof ArrayBuffer ? key : new TextEncoder().encode(key);
  const messageBuffer = new TextEncoder().encode(message);

  const cryptoKey = await crypto.subtle.importKey(
    'raw',
    keyBuffer,
    { name: 'HMAC', hash: 'SHA-256' },
    false,
    ['sign']
  );

  const signature = await crypto.subtle.sign(
    'HMAC',
    cryptoKey,
    messageBuffer
  );

  return signature;
}

// 获取签名
async function getSignature(secret, date, region, service, stringToSign) {
  const kDate = await hmacSha256('AWS4' + secret, date);
  const kRegion = await hmacSha256(kDate, region);
  const kService = await hmacSha256(kRegion, service);
  const kSigning = await hmacSha256(kService, 'aws4_request');
  const signature = await hmacSha256(kSigning, stringToSign);

  return Array.from(new Uint8Array(signature))
    .map(b => b.toString(16).padStart(2, '0'))
    .join('');
}

// AWS SDK 签名相关函数结束 =================================

// 检查服务函数
async function getGitHubUsername(pat) {
  const url = 'https://api.github.com/user';
  try {
    const response = await fetch(url, {
      headers: {
        'Authorization': `token ${pat}`,
        'Accept': 'application/vnd.github.v3+json',
        'User-Agent': 'Cloudflare Worker'
      }
    });

    if (response.status === 200) {
      const data = await response.json();
      return data.login;
    } else {
      console.error('GitHub API Error:', response.status);
      return 'Unknown';
    }
  } catch (error) {
    console.error('GitHub request error:', error);
    return 'Error';
  }
}

// 修改文件路径处理函数
function getFilePath(basePath, requestPath) {
  // 移除开头的斜杠
  const cleanRequestPath = requestPath.replace(/^\//, '');

  // 如果没有设置 basePath,直接返回请求路径
  if (!basePath) return cleanRequestPath;

  // 组合基础路径和请求路径
  return `${basePath}/${cleanRequestPath}`;
}

// 检查 GitHub 仓库
async function checkGitHubRepo(owner, repo, pat) {
  const repoUrl = `https://api.github.com/repos/${owner}/${repo}`;

  const headers = {
    'Authorization': `token ${pat}`,
    'Accept': 'application/vnd.github.v3+json',
    'User-Agent': 'Cloudflare Worker'
  };

  try {
    // 获取仓库信息,确定默认分支
    const repoResponse = await fetch(repoUrl, { headers });
    const repoData = await repoResponse.json();

    if (repoResponse.status!== 200) {
      throw new Error(`Repository error: ${repoData.message}`);
    }

    const defaultBranch = repoData.default_branch;
    const contentsUrl = `https://api.github.com/repos/${owner}/${repo}/git/trees/${defaultBranch}?recursive=1`;

    // 获取文件树信息
    const contentsResponse = await fetch(contentsUrl, { headers });
    if (contentsResponse.status!== 200) {
      const contentsErrorData = await contentsResponse.json();
      throw new Error(`Contents error: ${contentsErrorData.message}`);
    }

    const contentsData = await contentsResponse.json();

    let fileCount = 0;
    let totalSize = 0;

    if (contentsData.tree) {
      for (const item of contentsData.tree) {
        // 检查是否是文件
        if (item.type === 'blob' && (DIR === '' || item.path.startsWith(DIR + '/'))) {
          fileCount++;
          totalSize += item.size || 0;
        }
      }
    }

    return [
      `working (${repoData.private ? 'private' : 'public'})`,
      fileCount,
      totalSize
    ];

  } catch (error) {
    console.error(`Error checking GitHub repo ${repo}:`, error);
    return [`error: ${error.message}`, 0, 0];
  }
}

// 检查 GitLab 项目
async function checkGitLabProject(projectId, pat) {
  const projectUrl = `https://gitlab.com/api/v4/projects/${projectId}`;
  const treeUrl = `https://gitlab.com/api/v4/projects/${projectId}/repository/tree?recursive=true&per_page=100&path=${DIR}`;

  try {
    const [projectResponse, treeResponse] = await Promise.all([
      fetch(projectUrl, {
        headers: { 'PRIVATE-TOKEN': pat }
      }),
      fetch(treeUrl, {
        headers: { 'PRIVATE-TOKEN': pat }
      })
    ]);

    if (projectResponse.status === 200) {
      const projectData = await projectResponse.json();
      let fileCount = 0;

      if (treeResponse.status === 200) {
        const treeData = await treeResponse.json();
        // 只计算文件,不计算目录
        fileCount = treeData.filter(item => item.type === 'blob').length;
      }

      return [
        `working (${projectData.visibility})`,
        projectData.owner.username,
        fileCount
      ];
    } else if (projectResponse.status === 404) {
      return ['not found', 'Unknown', 0];
    } else {
      return ['disconnect', 'Unknown', 0];
    }
  } catch (error) {
    console.error('GitLab project check error:', error);
    return ['disconnect', 'Error', 0];
  }
}

// 检查 R2 存储
async function checkR2Storage(r2Config) {
  try {
    // 列出所有文件
    const listRequest = await getSignedUrl(r2Config, 'GET', r2Config.bucket, {
      'list-type': '2',
      'prefix': DIR ? `${DIR}/` : ''  // 添加目录前缀筛选
    });

    const response = await fetch(listRequest.url, {
      headers: {
        ...listRequest.headers,
        'Host': `${r2Config.accountId}.r2.cloudflarestorage.com`
      }
    });

    let fileCount = 0;
    let totalSize = 0;

    if (response.ok) {
      const data = await response.text();

      // 使用正则表达式匹配所有文件信息
      const contents = data.match(/<Contents>[\s\S]*?<\/Contents>/g) || [];

      for (const content of contents) {
        const keyMatch = content.match(/<Key>([^<]+)<\/Key>/);
        const sizeMatch = content.match(/<Size>(\d+)<\/Size>/);

        if (keyMatch && sizeMatch) {
          const key = keyMatch[1];
          // 只计算文件,不计算目录
          if (!key.endsWith('/')) {
            fileCount++;
            totalSize += parseInt(sizeMatch[1]);
          }
        }
      }
    }

    return [
      'working',
      r2Config.name,
      r2Config.bucket,
      fileCount,
      formatSize(totalSize)
    ];
  } catch (error) {
    console.error('R2 Storage error:', error);
    return ['error', r2Config.name, 'connection failed', 0, '0 B'];
  }
}

// 检查 B2 存储
async function checkB2Storage(b2Config) {
  try {
    // 构建列出文件的请求,移除 delimiter 参数以获取所有子目录
    const signedRequest = await getSignedUrl(b2Config, 'GET', b2Config.bucket, {
      'prefix': DIR ? `${DIR}/` : ''
    });

    const response = await fetch(signedRequest.url, {
      headers: {
        ...signedRequest.headers,
        'Host': b2Config.endPoint
      }
    });

    let fileCount = 0;
    let totalSize = 0;

    if (response.ok) {
      const data = await response.text();
      // 使用正则表达式匹配所有文件信息
      const keyRegex = /<Key>([^<]+)<\/Key>/g;
      const sizeRegex = /<Size>(\d+)<\/Size>/g;

      let keyMatch;
      while ((keyMatch = keyRegex.exec(data)) !== null) {
        const key = keyMatch[1];
        // 只计算文件,不计算目录,并确保文件在指定目录下
        if (!key.endsWith('/') && (!DIR || key.startsWith(DIR + '/'))) {
          fileCount++;
          // 获取对应的文件大小
          const sizeMatch = /<Size>(\d+)<\/Size>/g.exec(data.slice(keyMatch.index));
          if (sizeMatch) {
            totalSize += parseInt(sizeMatch[1]);
          }
        }
      }

      return [
        'working',
        b2Config.name,
        b2Config.bucket,
        fileCount,
        formatSize(totalSize)
      ];
    } else {
      throw new Error(`Failed to list bucket: ${response.status} ${response.statusText}`);
    }

  } catch (error) {
    console.error('B2 Storage error:', error);
    return ['error', b2Config.name, b2Config.bucket, 0, '0 B'];
  }
}

// 删除 GitHub 仓库中的文件
async function deleteGitHubFile(repo, filePath, pat) {
  // 构建完整的文件路径,包含 DIR
  const fullPath = DIR ? `${DIR}/${filePath.replace(/^\/+/, '')}` : filePath.replace(/^\/+/, '');
  const url = `https://api.github.com/repos/${GITHUB_USERNAME}/${repo}/contents/${fullPath}`;

  try {
    // 先检查文件是否存在
    const getResponse = await fetch(url, {
      headers: {
        'Authorization': `token ${pat}`,
        'Accept': 'application/vnd.github.v3+json',
        'User-Agent': 'Cloudflare Worker'
      }
    });

    if (getResponse.status === 404) {
      return '文件不存在';
    }

    if (!getResponse.ok) {
      const errorData = await getResponse.json();
      return `删除失败:(${errorData.message})`;
    }

    const fileData = await getResponse.json();

    // 执行删除操作
    const deleteResponse = await fetch(url, {
      method: 'DELETE',
      headers: {
        'Authorization': `token ${pat}`,
        'Accept': 'application/vnd.github.v3+json',
        'User-Agent': 'Cloudflare Worker'
      },
      body: JSON.stringify({
        message: `Delete ${fullPath}`,
        sha: fileData.sha
      })
    });

    if (deleteResponse.ok) {
      return '删除成功';
    } else {
      const errorData = await deleteResponse.json();
      return `删除失败:(${errorData.message})`;
    }
  } catch (error) {
    console.error('GitHub delete error:', error);
    return `删除失败:(${error.message})`;
  }
}

// 删除 GitLab 项目中的文件
async function deleteGitLabFile(projectId, filePath, pat) {
  // 构建完整的文件路径,包含 DIR
  const fullPath = DIR ? `${DIR}/${filePath.replace(/^\/+/, '')}` : filePath.replace(/^\/+/, '');
  const encodedPath = encodeURIComponent(fullPath);
  const url = `https://gitlab.com/api/v4/projects/${projectId}/repository/files/${encodedPath}`;

  try {
    // 执行删除操作
    const deleteResponse = await fetch(url, {
      method: 'DELETE',
      headers: {
        'PRIVATE-TOKEN': pat,
        'Content-Type': 'application/json'
      },
      body: JSON.stringify({
        branch: 'main',
        commit_message: 'Delete file: ' + fullPath
      })
    });

    // 获取响应数据
    const errorData = await deleteResponse.json().catch(() => ({}));

    // 处理文件不存在的所有可能情况
    if (deleteResponse.status === 404 ||
      errorData.message === 'A file with this name doesn\'t exist' ||
      errorData.message?.includes('file does not exist') ||
      errorData.message?.includes('File not found')) {
      return '文件不存在';
    }

    // 处理删除成功的情况
    if (deleteResponse.ok ||
      errorData.message?.includes('reference update') ||
      errorData.message?.includes('reference does not point')) {
      return '删除成功';
    }

    return `删除失败:(${errorData.message || '未知错误'})`;
  } catch (error) {
    console.error('GitLab delete error:', error);
    if (error.message?.includes('file') && error.message?.includes('exist')) {
      return '文件不存在';
    }
    return `删除失败:(${error.message})`;
  }
}

// 删除 R2 存储中的文件
async function deleteR2File(r2Config, filePath) {
  // 构建完整的文件路径,包含 DIR
  const fullPath = DIR ? `${DIR}/${filePath.replace(/^\/+/, '')}` : filePath.replace(/^\/+/, '');

  try {
    // 1. 首先列出所有文件
    const listRequest = await getSignedUrl(r2Config, 'GET', r2Config.bucket, {
      'list-type': '2',
      'prefix': fullPath  // 使用精确的前缀匹配
    });

    const listResponse = await fetch(listRequest.url, {
      headers: {
        ...listRequest.headers,
        'Host': `${r2Config.accountId}.r2.cloudflarestorage.com`
      }
    });

    if (!listResponse.ok) {
      throw new Error(`Failed to list objects: ${listResponse.statusText}`);
    }

    // 解析响应
    const listData = await listResponse.text();
    const contents = listData.match(/<Contents>[\s\S]*?<\/Contents>/g) || [];
    let fileExists = false;

    // 精确匹配文件路径
    for (const content of contents) {
      const keyMatch = content.match(/<Key>([^<]+)<\/Key>/);
      if (keyMatch && keyMatch[1] === fullPath) {
        fileExists = true;
        break;
      }
    }

    if (!fileExists) {
      return '文件不存在';
    }

    // 2. 删除文件
    const deleteRequest = await getSignedUrl(r2Config, 'DELETE', `${r2Config.bucket}/${fullPath}`);

    const deleteResponse = await fetch(deleteRequest.url, {
      method: 'DELETE',
      headers: {
        ...deleteRequest.headers,
        'Host': `${r2Config.accountId}.r2.cloudflarestorage.com`
      }
    });

    if (!deleteResponse.ok) {
      const deleteResponseText = await deleteResponse.text();
      throw new Error(`Failed to delete: ${deleteResponse.status} - ${deleteResponseText}`);
    }

    return '删除成功';
  } catch (error) {
    console.error('R2 delete error:', error);
    return `删除失败:(${error.message})`;
  }
}

// 删除 B2 存储中的文件
async function deleteB2File(b2Config, filePath) {
  // 构建完整的文件路径,包含 DIR
  const fullPath = DIR ? `${DIR}/${filePath.replace(/^\/+/, '')}` : filePath.replace(/^\/+/, '');

  try {
    // 1. 首先列出所有文件
    const listObjectsRequest = await getSignedUrl(b2Config, 'GET', b2Config.bucket, {
      'list-type': '2',
      'prefix': fullPath
    });

    const listResponse = await fetch(listObjectsRequest.url, {
      headers: {
        ...listObjectsRequest.headers,
        'Host': b2Config.endPoint
      }
    });

    if (!listResponse.ok) {
      throw new Error(`Failed to list objects: ${listResponse.statusText}`);
    }

    // 解析 XML 响应
    const listData = await listResponse.text();
    const keyRegex = /<Key>([^<]+)<\/Key>/g;
    const fileExists = Array.from(listData.matchAll(keyRegex))
      .some(match => match[1] === fullPath);

    if (!fileExists) {
      return '文件不存在';
    }

    // 2. 获取文件的版本信息
    const versionsRequest = await getSignedUrl(b2Config, 'GET', b2Config.bucket, {
      'versions': '',
      'prefix': fullPath,
      'list-type': '2'
    });

    const versionsResponse = await fetch(versionsRequest.url, {
      headers: {
        ...versionsRequest.headers,
        'Host': b2Config.endPoint,
        'x-amz-date': versionsRequest.headers['x-amz-date'],
        'Authorization': versionsRequest.headers['Authorization']
      }
    });

    if (!versionsResponse.ok) {
      const responseText = await versionsResponse.text();
      console.error('Version listing response:', responseText);
      throw new Error(`Failed to list versions: ${versionsResponse.status} - ${responseText}`);
    }

    const versionsData = await versionsResponse.text();

    // 解析版本信息
    const versionMatch = versionsData.match(/<Version>[\s\S]*?<VersionId>([^<]+)<\/VersionId>[\s\S]*?<\/Version>/);
    if (!versionMatch) {
      throw new Error('No version information found');
    }

    const versionId = versionMatch[1];

    // 3. 删除指定版本的文件
    const deleteRequest = await getSignedUrl(b2Config, 'DELETE', `${b2Config.bucket}/${fullPath}`, {
      'versionId': versionId
    });

    const deleteResponse = await fetch(deleteRequest.url, {
      method: 'DELETE',
      headers: {
        ...deleteRequest.headers,
        'Host': b2Config.endPoint,
        'x-amz-date': deleteRequest.headers['x-amz-date'],
        'Authorization': deleteRequest.headers['Authorization']
      }
    });

    if (!deleteResponse.ok) {
      const deleteResponseText = await deleteResponse.text();
      throw new Error(`Failed to delete: ${deleteResponse.status} - ${deleteResponseText}`);
    }

    return '删除成功';
  } catch (error) {
    console.error('B2 delete error:', error);
    return `删除失败:(${error.message})`;
  }
}

// 文件大小格式化函数
function formatSize(sizeInBytes) {
  if (sizeInBytes >= 1024 * 1024 * 1024) {
    return `${(sizeInBytes / (1024 * 1024 * 1024)).toFixed(2)} GB`;
  } else if (sizeInBytes >= 1024 * 1024) {
    return `${(sizeInBytes / (1024 * 1024)).toFixed(2)} MB`;
  } else {
    return `${(sizeInBytes / 1024).toFixed(2)} kB`;
  }
}

export default {
  async fetch(request, env, ctx) {
    // 获取请求 URL 对象
    const url = new URL(request.url);

    // 从 URL 的查询参数中获取 'from' 参数并转换为小写
    const from = url.searchParams.get('from')?.toLowerCase();

    // 检查是否有有效的配置,调用 `hasValidConfig()` 函数
    const validConfigs = hasValidConfig();

    // 获取请求路径并解码(对 URL 编码进行解码)
    const requestPath = decodeURIComponent(url.pathname);

    // 添加根路径项目介绍
    if (requestPath === '/') {
      return new Response(
        '欢迎来到文件托管集群!(File Hosting Cluster)\n' +
        '这是一个分布式存储集群项目,旨在提供高效的文件存储和管理服务。\n\n' +
        '项目链接: https://github.com/fscarmen2/pic-hosting-cluster\n' +
        '视频介绍: https://youtu.be/5i-86oBLWP8\n\n' +
        '您可以使用以下操作:\n' +
        '1. 从集群所有节点获取文件: /<文件名>\n' +
        '2. 指定从 Github 获取文件: /<文件名>?from=github\n' +
        '3. 指定从 Gitlab 获取文件: /<文件名>?from=gitlab\n' +
        '4. 指定从 Cloudflare R2 获取文件: /<文件名>?from=r2\n' +
        '5. 指定从 Backblaze B2 获取文件: /<文件名>?from=b2\n' +
        '6. 查找文件信息: /<文件名>?from=where\n' +
        '7. 查各节点状态: /<自定义密码>\n' +
        '8. 删除文件: /<自定义密码>/del?file=<文件名>',
        {
          headers: {
            'Content-Type': 'text/plain; charset=UTF-8',
            'Access-Control-Allow-Origin': '*'
          }
        }
      );
    }

    // 从路径中提取文件名(即路径的最后一部分)
    const FILE = requestPath.split('/').pop();

    // 获取子目录路径,移除开头和结尾的斜杠
    const subPath = requestPath.substring(1, requestPath.lastIndexOf('/')).replace(/^\/+|\/+$/g, '');

    // 如果 DIR 存在,拼接 DIR 和子目录路径;否则仅使用子目录路径
    const fullPath = DIR ? `${DIR}/${subPath}` : subPath;

    // 检查请求路径是否匹配删除请求(支持 'delete' 或 'del')
    const isDeleteRequest = requestPath.match(new RegExp(`^/${CHECK_PASSWORD}/(delete|del)$`));

    // 检查是否是未授权的删除请求
    const isUnauthorizedDelete = requestPath.match(/^\/(delete|del)$/);
    if (isUnauthorizedDelete) {
      const file = url.searchParams.get('file');
      if (!file) {
        return new Response(
          '需要指定要删除的文件。\n' +
          '正确的删除格式为: /<自定义密码>/del?file=文件路径\n' +
          '例如: /<自定义密码>/del?file=example.png',
          {
            status: 403,
            headers: {
              'Content-Type': 'text/plain; charset=UTF-8',
              'Access-Control-Allow-Origin': '*'
            }
          }
        );
      }

      return new Response(
        '需要密码验证才能删除文件。\n' +
        '要删除文件 ' + file + ' 的正确格式为:\n' +
        '/<自定义密码>/del?file=' + file,
        {
          status: 403,
          headers: {
            'Content-Type': 'text/plain; charset=UTF-8',
            'Access-Control-Allow-Origin': '*'
          }
        }
      );
    }

    // 从 GITLAB_CONFIGS 中获取每个配置的 name 作为 GitHub 仓库名
    const githubRepos = GITLAB_CONFIGS.map(config => config.name);

    // 只在没有 from 参数时才检查和使用缓存
    let cacheResponse;
    if (!from) {
      const cacheUrl = new URL(request.url);
      const cacheKey = new Request(cacheUrl.toString(), request);
      const cache = caches.default;
      cacheResponse = await cache.match(cacheKey);

      if (cacheResponse) {
        return cacheResponse;
      }
    }

    // 检查状态页面
    if (url.pathname === `/${CHECK_PASSWORD}`) {
      let result = '';
      let hasAnyValidConfig = false;

      try {
        // GitHub 状态检查
        if (validConfigs.github) {
          hasAnyValidConfig = true;
          result += '=== GitHub Status ===\n';
          const username = await getGitHubUsername(GITHUB_PAT);
          for (const repo of githubRepos) {
            const [status, fileCount, totalSize] = await checkGitHubRepo(GITHUB_USERNAME, repo, GITHUB_PAT);
            const formattedSize = formatSize(totalSize);
            result += `GitHub: ${repo} - ${status} (Username: ${username}, Files: ${fileCount}, Size: ${formattedSize})\n`;
          }
        }

        // GitLab 状态检查
        if (validConfigs.gitlab) {
          hasAnyValidConfig = true;
          result += result ? '\n=== GitLab Status ===\n' : '=== GitLab Status ===\n';
          for (const config of GITLAB_CONFIGS) {
            const [status, username, fileCount] = await checkGitLabProject(config.id, config.token);
            result += `GitLab: Project ID ${config.id} - ${status} (Username: ${username}, Files: ${fileCount})\n`;
          }
        }

        // R2 状态检查
        if (validConfigs.r2) {
          hasAnyValidConfig = true;
          result += result ? '\n=== R2 Storage Status ===\n' : '=== R2 Storage Status ===\n';
          for (const config of R2_CONFIGS) {
            const [status, name, bucket, fileCount, totalSize] = await checkR2Storage(config);
            result += `R2 Storage: ${name} - ${status} (Bucket: ${bucket}, Files: ${fileCount}, Size: ${totalSize})\n`;
          }
        }

        // B2 状态检查
        if (validConfigs.b2) {
          hasAnyValidConfig = true;
          result += result ? '\n=== B2 Storage Status ===\n' : '=== B2 Storage Status ===\n';
          for (const config of B2_CONFIGS) {
            const [status, name, bucket, fileCount, totalSize] = await checkB2Storage(config);
            result += `B2 Storage: ${name} - ${status} (Bucket: ${bucket}, Files: ${fileCount}, Size: ${totalSize})\n`;
          }
        }

        // 如果没有任何有效配置
        if (!hasAnyValidConfig) {
          result = 'No storage services configured.\n';
        } else {
          result = 'Storage status:\n\n' + result;
        }

      } catch (error) {
        result += `Error during status check: ${error.message}\n`;
      }

      return new Response(result, {
        headers: {
          'Content-Type': 'text/plain; charset=UTF-8',
          'Access-Control-Allow-Origin': '*'
        }
      });
    }

    // 添加删除路由
    if (isDeleteRequest) {
      const file = url.searchParams.get('file');
      if (!file) {
        return new Response('Missing "file" parameter', {
          status: 400,
          headers: { 'Content-Type': 'text/plain; charset=UTF-8', 'Access-Control-Allow-Origin': '*' }
        });
      }

      let result = `Delete:${file}\n`;

      // GitHub 状态
      if (validConfigs.github) {
        result += '\n=== GitHub Status ===\n';
        const githubRepos = GITLAB_CONFIGS.map(config => config.name);
        for (const repo of githubRepos) {
          const status = await deleteGitHubFile(repo, file, GITHUB_PAT);
          result += `GitHub: ${repo} - working (private) ${status}\n`;
        }
      }

      // GitLab 状态
      if (validConfigs.gitlab) {
        result += '\n=== GitLab Status ===\n';
        for (const config of GITLAB_CONFIGS) {
          const status = await deleteGitLabFile(config.id, file, config.token);
          const projectData = await fetch(`https://gitlab.com/api/v4/projects/${config.id}`, {
            headers: { 'PRIVATE-TOKEN': config.token }
          }).then(res => res.json());
          result += `GitLab: Project ID ${config.id} - working (${projectData.visibility}) ${status}\n`;
        }
      }

      // R2 存储状态
      if (validConfigs.r2) {
        result += '\n=== R2 Storage Status ===\n';
        for (const config of R2_CONFIGS) {
          const status = await deleteR2File(config, file);
          result += `R2 Storage: ${config.name} - working ${status}\n`;
        }
      }

      // B2 存储状态
      if (validConfigs.b2) {
        result += '\n=== B2 Storage Status ===\n';
        for (const config of B2_CONFIGS) {
          const status = await deleteB2File(config, file);
          result += `B2 Storage: ${config.name} - working ${status}\n`;
        }
      }

      return new Response(result, {
        headers: {
          'Content-Type': 'text/plain; charset=UTF-8',
          'Access-Control-Allow-Origin': '*'
        }
      });
    }

    const startTime = Date.now();
    let requests = [];

    // 检查特定服务的请求是否有效
    if (from) {
      if (from === 'github' && !validConfigs.github) {
        return new Response('GitHub service is not configured.', {
          status: 400,
          headers: { 'Content-Type': 'text/plain', 'Access-Control-Allow-Origin': '*' }
        });
      }
      if (from === 'gitlab' && !validConfigs.gitlab) {
        return new Response('GitLab service is not configured.', {
          status: 400,
          headers: { 'Content-Type': 'text/plain', 'Access-Control-Allow-Origin': '*' }
        });
      }
      if (from === 'r2' && !validConfigs.r2) {
        return new Response('R2 storage service is not configured.', {
          status: 400,
          headers: { 'Content-Type': 'text/plain', 'Access-Control-Allow-Origin': '*' }
        });
      }
      if (from === 'b2' && !validConfigs.b2) {
        return new Response('B2 storage service is not configured.', {
          status: 400,
          headers: { 'Content-Type': 'text/plain', 'Access-Control-Allow-Origin': '*' }
        });
      }
    }

    // 生成存储请求
    async function generateStorageRequests() {
      let requests = [];

      // 处理请求路径,保留子目录结构
      const getStoragePath = (filePath) => {
        return filePath.replace(/^\/+/, '').replace(/\/+/g, '/');
      };

      if (validConfigs.r2) {
        const r2Requests = await Promise.all(R2_CONFIGS.map(async (r2Config) => {
          // 构建包含子目录的完整路径
          const storagePath = getStoragePath(`${subPath}/${FILE}`);
          const r2Path = `${r2Config.bucket}/${DIR}/${storagePath}`;

          const signedRequest = await getSignedUrl(r2Config, 'GET', r2Path);
          return {
            url: signedRequest.url,
            headers: {
              ...signedRequest.headers,
              'Accept': '*/*'
            },
            source: 'r2',
            repo: `${r2Config.name} (${r2Config.bucket})`
          };
        }));
        requests = [...requests, ...r2Requests];
      }

      if (validConfigs.b2) {
        const b2Requests = await Promise.all(B2_CONFIGS.map(async (b2Config) => {
          // 构建完整路径,注意 B2 需要包含 bucket 名称
          const storagePath = getStoragePath(`${subPath}/${FILE}`);
          const b2Path = `${b2Config.bucket}/${DIR}/${storagePath}`;

          const signedRequest = await getSignedUrl({
            endPoint: b2Config.endPoint,
            keyId: b2Config.keyId,
            applicationKey: b2Config.applicationKey,
            bucket: b2Config.bucket
          }, 'GET', b2Path);

          return {
            url: signedRequest.url,
            headers: {
              ...signedRequest.headers,
              'Host': b2Config.endPoint,
              'Accept': '*/*'
            },
            source: 'b2',
            repo: `${b2Config.name} (${b2Config.bucket})`
          };
        }));
        requests = [...requests, ...b2Requests];
      }

      return requests;
    }

    // 处理不同类型的请求
    if (from === 'where') {
      if (validConfigs.github) {
        const githubRequests = githubRepos.map(repo => ({
          url: `https://api.github.com/repos/${GITHUB_USERNAME}/${repo}/contents/${getFilePath(DIR, `${subPath}/${FILE}`)}`,
          headers: {
            'Authorization': `token ${GITHUB_PAT}`,
            'Accept': 'application/vnd.github.v3+json',
            'User-Agent': 'Cloudflare Worker'
          },
          source: 'github',
          repo: repo,
          processResponse: async (response) => {
            if (!response.ok) throw new Error('Not found');
            const data = await response.json();
            return {
              size: data.size,
              exists: true
            };
          }
        }));
        requests = [...requests, ...githubRequests];
      }

      if (validConfigs.gitlab) {
        const gitlabRequests = GITLAB_CONFIGS.map(config => ({
          // GitLab where 查询 URL
          url: `https://gitlab.com/api/v4/projects/${config.id}/repository/files/${encodeURIComponent(getFilePath(DIR, `${subPath}/${FILE}`))}?ref=main`,
          headers: {
            'PRIVATE-TOKEN': config.token
          },
          source: 'gitlab',
          repo: config.name,
          processResponse: async (response) => {
            if (!response.ok) throw new Error('Not found');
            const data = await response.json();
            return {
              size: data.size,
              exists: true
            };
          }
        }));
        requests = [...requests, ...gitlabRequests];
      }

      const storageRequests = await generateStorageRequests();
      const storageWhereRequests = storageRequests.map(request => ({
        ...request,
        processResponse: async (response) => {
          if (!response.ok) throw new Error('Not found');
          const size = response.headers.get('content-length');
          return {
            size: parseInt(size),
            exists: true
          };
        }
      }));
      requests = [...requests, ...storageWhereRequests];

    } else {
      // 获取文件内容模式
      if (from === 'github' && validConfigs.github) {
        requests = githubRepos.map(repo => ({
          url: `https://raw.githubusercontent.com/${GITHUB_USERNAME}/${repo}/main/${fullPath}/${FILE}`,
          headers: {
            'Authorization': `token ${GITHUB_PAT}`,
            'User-Agent': 'Cloudflare Worker'
          },
          source: 'github',
          repo: repo
        }));
      } else if (from === 'gitlab' && validConfigs.gitlab) {
        requests = GITLAB_CONFIGS.map(config => ({
          // GitLab 文件获取 URL
          url: `https://gitlab.com/api/v4/projects/${config.id}/repository/files/${encodeURIComponent(getFilePath(DIR, `${subPath}/${FILE}`))}/raw?ref=main`,
          headers: {
            'PRIVATE-TOKEN': config.token
          },
          source: 'gitlab',
          repo: config.name
        }));
      } else if ((from === 'r2' && validConfigs.r2) || (from === 'b2' && validConfigs.b2)) {
        requests = await generateStorageRequests();
        requests = requests.filter(req => req.source === from);
      } else if (!from) {
        if (validConfigs.github) {
          const githubRequests = githubRepos.map(repo => ({
            url: `https://raw.githubusercontent.com/${GITHUB_USERNAME}/${repo}/main/${fullPath}/${FILE}`,
            headers: {
              'Authorization': `token ${GITHUB_PAT}`,
              'User-Agent': 'Cloudflare Worker'
            },
            source: 'github',
            repo: repo
          }));
          requests = [...requests, ...githubRequests];
        }

        if (validConfigs.gitlab) {
          const gitlabRequests = GITLAB_CONFIGS.map(config => ({
            // GitLab URL 构建方式
            url: `https://gitlab.com/api/v4/projects/${config.id}/repository/files/${encodeURIComponent(getFilePath(DIR, `${subPath}/${FILE}`))}/raw?ref=main`,
            headers: {
              'PRIVATE-TOKEN': config.token
            },
            source: 'gitlab',
            repo: config.name
          }));
          requests = [...requests, ...gitlabRequests];
        }

        const storageRequests = await generateStorageRequests();
        requests = [...requests, ...storageRequests];
      }
    }

    // 处理请求和响应
    try {
      if (requests.length === 0) {
        throw new Error('No valid source specified or no valid configurations found');
      }

      const fetchPromises = requests.map(request => {
        const { url, headers, source, repo, processResponse } = request;

        return fetch(new Request(url, {
          method: 'GET',
          headers: headers
        })).then(async response => {
          if (from === 'where' && typeof processResponse === 'function') {
            try {
              const result = await processResponse(response);
              const endTime = Date.now();
              const duration = endTime - startTime;

              const formattedSize = result.size > 1024 * 1024
                ? `${(result.size / (1024 * 1024)).toFixed(2)} MB`
                : `${(result.size / 1024).toFixed(2)} kB`;

              return {
                fileName: FILE,
                size: formattedSize,
                source: `${source} (${repo})`,
                duration: `${duration}ms`
              };
            } catch (error) {
              throw new Error(`Not found in ${source} (${repo})`);
            }
          } else {
            if (!response.ok) {
              throw new Error(`Not found in ${source} (${repo})`);
            }
            return response;
          }
        }).catch(error => {
          throw new Error(`Error in ${source} (${repo}): ${error.message}`);
        });
      });

      const result = await Promise.any(fetchPromises);

      let response;
      if (from === 'where') {
        response = new Response(JSON.stringify(result, null, 2), {
          headers: {
            'Content-Type': 'application/json',
            'Access-Control-Allow-Origin': '*'
          }
        });
      } else if (result instanceof Response) {
        const blob = await result.blob();
        const headers = {
          'Content-Type': result.headers.get('Content-Type') || 'application/octet-stream',
          'Access-Control-Allow-Origin': '*'
        };

        if (from) {
          headers['Cache-Control'] = 'no-store, no-cache, must-revalidate, proxy-revalidate';
          headers['Pragma'] = 'no-cache';
          headers['Expires'] = '0';
        }

        response = new Response(blob, {
          status: 200,
          headers: headers
        });
      } else {
        throw new Error("Unexpected result type");
      }

      if (!from && from !== 'where') {
        const cacheUrl = new URL(request.url);
        const cacheKey = new Request(cacheUrl.toString(), request);
        const cache = caches.default;
        ctx.waitUntil(cache.put(cacheKey, response.clone()));
      }

      return response;

    } catch (error) {
      const sourceText = from === 'where'
        ? 'in any repository'
        : from
          ? `from ${from}`
          : 'in any configured storage';

      const errorResponse = new Response(
        `404: Cannot find ${FILE} ${sourceText}. ${error.message}`,
        {
          status: 404,
          headers: {
            'Content-Type': 'text/plain',
            'Access-Control-Allow-Origin': '*'
          }
        }
      );

      return errorResponse;
    }
  }
}

@2512132839
Copy link
Author

之前添加子路径支持功能的时候有些地方没有处理好。修了一下,我这里测试可以,麻烦你抽时间帮看看。如果你也没有问题,我就更新到项目里了。

目前没啥问题了,路径能正常显示了。辛苦大佬了

@fscarmen2
Copy link
Owner

目前没啥问题了,路径能正常显示了。辛苦大佬了

感谢支持和反馈,有遇到其他问题再告诉我。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants