Skip to content

Commit

Permalink
add vercel limit fix
Browse files Browse the repository at this point in the history
  • Loading branch information
sunwu51 committed Jun 9, 2024
1 parent bd3f0a8 commit f1213c6
Show file tree
Hide file tree
Showing 201 changed files with 156 additions and 51 deletions.
1 change: 1 addition & 0 deletions .vercelignore
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
.github
.git
*.gif
*.ppt
*.pptx
97 changes: 97 additions & 0 deletions 24.06/vercel依赖大小超250M解决.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
---
title: vercel依赖大小超250M解决
tags:
- vercel
- blog
- next.js
---

# 现象
笔记ci流程挂了

![image](https://i.imgur.com/xw6vza2.png)

特别大的单个文件没有,都是一些碎文件。

![image](https://i.imgur.com/aVLj1cT.png)

主要还是一些图片文件比较大,这些图片有两类
- 1 为了兼容新老博客系统,单独copy到`public/oriimg/`目录下的图片。
- 2 为了避免`imgur`被墙,导致看不到图片,把所有图片拉取到了本地。

# 解决
## 解决本地图片问题
针对第一个情况,删除`public/oriimg/`目录,将原来重定向到`oriimg`路径的代码修改为直接指向`github`.
```diff
- node.properties.src = src.replace('./img', '/oriimg/' + month);
+ node.properties.src = src.replace('./img', `https://raw.githubusercontent.com/sunwu51/notebook/master/${month}/img`);
```

然后`.vercelignore`中,就可以忽略`*.gif`格式的文件,动图格式的文件比较大,只有早期的笔记中才用了`gif`

## 解决imgur下载图片问题
直接注释掉`imgur`的下载显然就解决了,但是这样图片链接是直连`imgur`,这样会导致国内用户不翻墙看不到图片,另一个解决方案是运行时转发请求。

即我们把`public/imgur`目录删除掉,然后创建`app/api/imgur/route.js`一个api函数,来处理get请求,将所有的`/api/imgur?filename=xx`的请求转发到`https://i.imgur.com/${filename}`,实现代理,这样只要能访问我的域名,就能代理到`imgur`保证图片的可访问性。

```js :route.js
export async function GET(request) {
// 从请求 URL 中获取参数
const { searchParams } = new URL(request.url);
const filename = searchParams.get('filename');

// 确保文件名存在
if (!filename) {
return new Response(JSON.stringify({ error: 'Filename is required' }), {
status: 400,
headers: {
'Content-Type': 'application/json',
},
});
}

// 目标服务器的 URL
const targetUrl = `https://i.imgur.com/${filename}`;

console.log(targetUrl)
try {
// 转发请求到目标服务器
const response = await fetch(targetUrl, {
headers :{
"user-agent": "curl/7.84.0",
"accept": "*/*"
},
});

// 获取目标服务器的响应
const responseBody = await response.arrayBuffer();

// 返回目标服务器的响应
return new Response(responseBody, {
status: response.status,
headers: {
// 复制目标服务器的 Content-Type 和其他相关头信息
'Content-Type': response.headers.get('Content-Type') || 'application/octet-stream',
'Content-Length': response.headers.get('Content-Length'),
},
});
} catch (error) {
// 错误处理
console.error('Error forwarding request:', error);
return new Response(JSON.stringify({ error: 'Error forwarding request' }), {
status: 500,
headers: {
'Content-Type': 'application/json',
},
});
}
}
```

接下来修改插件代码,把所有`imgur`域名,改为我们自己的代理地址。
```mjs :rehypePlugins/rehype-image-src-modifier.mjs
if (process.env.NODE_ENV === 'production' && src.startsWith('https://i.imgur.com/')) {
let picName = src.replace('https://i.imgur.com/', '/api/imgur?filename=');
node.properties.src = picName;
}
```
52 changes: 52 additions & 0 deletions app/api/imgur/route.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
export async function GET(request) {
// 从请求 URL 中获取参数
const { searchParams } = new URL(request.url);
const filename = searchParams.get('filename');

// 确保文件名存在
if (!filename) {
return new Response(JSON.stringify({ error: 'Filename is required' }), {
status: 400,
headers: {
'Content-Type': 'application/json',
},
});
}

// 目标服务器的 URL
const targetUrl = `https://i.imgur.com/${filename}`;

console.log(targetUrl)
try {
// 转发请求到目标服务器
const response = await fetch(targetUrl, {
headers :{
"user-agent": "curl/7.84.0",
"accept": "*/*"
},
});

// 获取目标服务器的响应
const responseBody = await response.arrayBuffer();

// 返回目标服务器的响应
return new Response(responseBody, {
status: response.status,
headers: {
// 复制目标服务器的 Content-Type 和其他相关头信息
'Content-Type': response.headers.get('Content-Type') || 'application/octet-stream',
'Content-Length': response.headers.get('Content-Length'),
},
});
} catch (error) {
// 错误处理
console.error('Error forwarding request:', error);
return new Response(JSON.stringify({ error: 'Error forwarding request' }), {
status: 500,
headers: {
'Content-Type': 'application/json',
},
});
}
}

24 changes: 0 additions & 24 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 0 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,6 @@
"esbuild": "^0.21.4",
"esbuild-plugin-css-modules": "^0.3.0",
"framer-motion": "^11.2.10",
"https-proxy-agent": "^7.0.4",
"mdx-bundler": "^10.0.2",
"next": "14.2.3",
"node-fetch": "^3.3.2",
Expand Down
Binary file removed public/oriimg/18.1/kube1.gif
Binary file not shown.
Binary file removed public/oriimg/18.1/kube1.jpg
Binary file not shown.
Binary file removed public/oriimg/18.1/kube2.gif
Binary file not shown.
Binary file removed public/oriimg/18.1/kube2.jpg
Binary file not shown.
Binary file removed public/oriimg/18.1/kube3.gif
Binary file not shown.
Binary file removed public/oriimg/18.1/kube3.jpg
Binary file not shown.
Binary file removed public/oriimg/18.1/kube4.gif
Binary file not shown.
Binary file removed public/oriimg/18.1/kube5.gif
Binary file not shown.
Binary file removed public/oriimg/18.1/kube6.gif
Binary file not shown.
Binary file removed public/oriimg/18.1/kube7.gif
Binary file not shown.
Binary file removed public/oriimg/18.1/kube8.gif
Binary file not shown.
Binary file removed public/oriimg/18.1/video1.jpg
Binary file not shown.
Binary file removed public/oriimg/18.10/1.gif
Binary file not shown.
Binary file removed public/oriimg/18.10/2.gif
Binary file not shown.
Binary file removed public/oriimg/18.10/3.gif
Binary file not shown.
Binary file removed public/oriimg/18.10/iot1.png
Binary file not shown.
Binary file removed public/oriimg/18.10/iot2.jpg
Binary file not shown.
Binary file removed public/oriimg/18.10/iot3.png
Binary file not shown.
Binary file removed public/oriimg/18.10/iot4.png
Binary file not shown.
Binary file removed public/oriimg/18.10/iot6.png
Binary file not shown.
Binary file removed public/oriimg/18.10/r1.jpg
Binary file not shown.
Binary file removed public/oriimg/18.10/r2.jpg
Binary file not shown.
Binary file removed public/oriimg/18.10/r3.jpg
Binary file not shown.
Binary file removed public/oriimg/18.10/r4.jpg
Binary file not shown.
Binary file removed public/oriimg/18.11/bit1.gif
Binary file not shown.
Binary file removed public/oriimg/18.11/bit1.jpg
Diff not rendered.
Binary file removed public/oriimg/18.11/bit2.gif
Diff not rendered.
Binary file removed public/oriimg/18.11/bit2.jpg
Diff not rendered.
Binary file removed public/oriimg/18.11/bit3.jpg
Diff not rendered.
Binary file removed public/oriimg/18.11/bit4.jpg
Diff not rendered.
Binary file removed public/oriimg/18.11/discord1.gif
Diff not rendered.
Binary file removed public/oriimg/18.11/discord1.jpg
Diff not rendered.
Binary file removed public/oriimg/18.11/discord2.gif
Diff not rendered.
Binary file removed public/oriimg/18.11/discord2.jpg
Diff not rendered.
Binary file removed public/oriimg/18.11/discord3.gif
Diff not rendered.
Binary file removed public/oriimg/18.11/discord3.jpg
Diff not rendered.
Binary file removed public/oriimg/18.11/discord4.gif
Diff not rendered.
Binary file removed public/oriimg/18.11/discord5.gif
Diff not rendered.
Binary file removed public/oriimg/18.11/discord6.gif
Diff not rendered.
Binary file removed public/oriimg/18.11/email.gif
Diff not rendered.
Binary file removed public/oriimg/18.11/iot5.jpg
Diff not rendered.
Binary file removed public/oriimg/18.11/js0.jpg
Diff not rendered.
Binary file removed public/oriimg/18.11/js1.jpg
Diff not rendered.
Binary file removed public/oriimg/18.11/js2.jpg
Diff not rendered.
Binary file removed public/oriimg/18.12/1.png
Diff not rendered.
Binary file removed public/oriimg/18.12/2.png
Diff not rendered.
Binary file removed public/oriimg/18.12/circleCI1.png
Diff not rendered.
Binary file removed public/oriimg/18.12/circleCI2.png
Diff not rendered.
Binary file removed public/oriimg/18.12/circleCI3.png
Diff not rendered.
Binary file removed public/oriimg/18.12/jupyter1.gif
Diff not rendered.
Binary file removed public/oriimg/18.12/jupyter1.png
Diff not rendered.
Binary file removed public/oriimg/18.12/jupyter2.gif
Diff not rendered.
Binary file removed public/oriimg/18.12/jupyter2.png
Diff not rendered.
Binary file removed public/oriimg/18.12/jupyter3.png
Diff not rendered.
Binary file removed public/oriimg/18.12/jupyter4.png
Diff not rendered.
Binary file removed public/oriimg/18.12/mongo1.gif
Diff not rendered.
Binary file removed public/oriimg/18.12/mongo2.jpg
Diff not rendered.
Binary file removed public/oriimg/18.12/mongo3.jpg
Diff not rendered.
Binary file removed public/oriimg/18.2/electron.jpg
Diff not rendered.
Binary file removed public/oriimg/18.2/firebase.gif
Diff not rendered.
Binary file removed public/oriimg/18.2/firebase1.jpg
Diff not rendered.
Binary file removed public/oriimg/18.2/firebase2.jpg
Diff not rendered.
Binary file removed public/oriimg/18.2/firebase3.jpg
Diff not rendered.
Binary file removed public/oriimg/18.2/firebase4.jpg
Diff not rendered.
Binary file removed public/oriimg/18.2/firebase5.jpg
Diff not rendered.
Binary file removed public/oriimg/18.2/firebase6.jpg
Diff not rendered.
Binary file removed public/oriimg/18.2/firebase7.jpg
Diff not rendered.
Binary file removed public/oriimg/18.2/index1.jpg
Diff not rendered.
Binary file removed public/oriimg/18.2/index2.jpg
Diff not rendered.
Binary file removed public/oriimg/18.2/index3.jpg
Diff not rendered.
Binary file removed public/oriimg/18.2/iptables.gif
Diff not rendered.
Binary file removed public/oriimg/18.2/iptables2.gif
Diff not rendered.
Binary file removed public/oriimg/18.2/iptables3.gif
Diff not rendered.
Binary file removed public/oriimg/18.2/iptables4.jpg
Diff not rendered.
Binary file removed public/oriimg/18.2/kube1.jpg
Diff not rendered.
Binary file removed public/oriimg/18.2/kube2.jpg
Diff not rendered.
Binary file removed public/oriimg/18.2/kube3.jpg
Diff not rendered.
Binary file removed public/oriimg/18.2/kube4.1.jpg
Diff not rendered.
Binary file removed public/oriimg/18.2/kube4.jpg
Diff not rendered.
Binary file removed public/oriimg/18.2/kube5.jpg
Diff not rendered.
Binary file removed public/oriimg/18.2/kube6.jpg
Diff not rendered.
Binary file removed public/oriimg/18.2/kube7.gif
Diff not rendered.
Binary file removed public/oriimg/18.2/shell.jpg
Diff not rendered.
Binary file removed public/oriimg/18.2/sortO.jpg
Diff not rendered.
Binary file removed public/oriimg/18.4/k8sing.jpg
Diff not rendered.
Binary file removed public/oriimg/18.4/k8sui.jpg
Diff not rendered.
Binary file removed public/oriimg/18.5/1.gif
Diff not rendered.
Binary file removed public/oriimg/18.5/1.jpg
Diff not rendered.
Binary file removed public/oriimg/18.5/2.gif
Diff not rendered.
Binary file removed public/oriimg/18.5/2.jpg
Diff not rendered.
Binary file removed public/oriimg/18.5/3.gif
Diff not rendered.
Binary file removed public/oriimg/18.6/1.gif
Diff not rendered.
Binary file removed public/oriimg/18.6/git1.gif
Diff not rendered.
Binary file removed public/oriimg/18.6/git2.gif
Diff not rendered.
Binary file removed public/oriimg/18.6/git3.gif
Diff not rendered.
Binary file removed public/oriimg/18.6/git4.gif
Diff not rendered.
Binary file removed public/oriimg/18.6/git5.jpg
Diff not rendered.
Binary file removed public/oriimg/18.7/1.gif
Diff not rendered.
Binary file removed public/oriimg/18.7/1.jpg
Diff not rendered.
Binary file removed public/oriimg/18.7/2.gif
Diff not rendered.
Binary file removed public/oriimg/18.7/3.gif
Diff not rendered.
Binary file removed public/oriimg/18.8/1.gif
Diff not rendered.
Binary file removed public/oriimg/18.8/1.jpg
Diff not rendered.
Binary file removed public/oriimg/18.8/10.jpg
Diff not rendered.
Binary file removed public/oriimg/18.8/11.jpg
Diff not rendered.
Binary file removed public/oriimg/18.8/12.jpg
Diff not rendered.
Binary file removed public/oriimg/18.8/13.jpg
Diff not rendered.
Binary file removed public/oriimg/18.8/14.jpg
Diff not rendered.
Binary file removed public/oriimg/18.8/15.jpg
Diff not rendered.
Binary file removed public/oriimg/18.8/16.jpg
Diff not rendered.
Binary file removed public/oriimg/18.8/17.jpg
Diff not rendered.
Binary file removed public/oriimg/18.8/2.jpg
Diff not rendered.
Binary file removed public/oriimg/18.8/3.jpg
Diff not rendered.
Binary file removed public/oriimg/18.8/4.jpg
Diff not rendered.
Binary file removed public/oriimg/18.8/5.jpg
Diff not rendered.
Binary file removed public/oriimg/18.8/6.jpg
Diff not rendered.
Binary file removed public/oriimg/18.8/7.jpg
Diff not rendered.
Binary file removed public/oriimg/18.8/8.jpg
Diff not rendered.
Binary file removed public/oriimg/18.8/9.jpg
Diff not rendered.
Binary file removed public/oriimg/18.8/fire_database.gif
Diff not rendered.
Binary file removed public/oriimg/18.8/fire_functions.gif
Diff not rendered.
Binary file removed public/oriimg/18.8/fire_functions2.gif
Diff not rendered.
Binary file removed public/oriimg/18.8/fire_hosting.gif
Diff not rendered.
Binary file removed public/oriimg/18.8/fire_hosting2.gif
Diff not rendered.
Binary file removed public/oriimg/18.8/fire_storage.gif
Diff not rendered.
Binary file removed public/oriimg/18.8/sql1.jpg
Diff not rendered.
Binary file removed public/oriimg/18.9/clazloader1.jpg
Diff not rendered.
Binary file removed public/oriimg/18.9/jconsole.gif
Diff not rendered.
Binary file removed public/oriimg/18.9/jconsole.jpg
Diff not rendered.
Binary file removed public/oriimg/18.9/jvm1.gif
Diff not rendered.
Binary file removed public/oriimg/18.9/jvm2.gif
Diff not rendered.
Binary file removed public/oriimg/18.9/jvmmem.jpg
Diff not rendered.
Binary file removed public/oriimg/18.9/jvmstack.png
Diff not rendered.
Binary file removed public/oriimg/18.9/stack1.gif
Diff not rendered.
Binary file removed public/oriimg/18.9/stack2.gif
Diff not rendered.
Binary file removed public/oriimg/18.9/threadlocal.jpg
Diff not rendered.
Binary file removed public/oriimg/18.9/vsvm.gif
Diff not rendered.
Binary file removed public/oriimg/18.9/vsvm2.gif
Diff not rendered.
Binary file removed public/oriimg/19.01/1.jpg
Diff not rendered.
Binary file removed public/oriimg/19.01/2.jpg
Diff not rendered.
Binary file removed public/oriimg/19.01/auth1.gif
Diff not rendered.
Binary file removed public/oriimg/19.01/auth1.jpg
Diff not rendered.
Binary file removed public/oriimg/19.01/auth2.jpg
Diff not rendered.
Binary file removed public/oriimg/19.01/auth3.jpg
Diff not rendered.
Binary file removed public/oriimg/19.01/auth4.jpg
Diff not rendered.
Binary file removed public/oriimg/19.01/await1.jpg
Diff not rendered.
Binary file removed public/oriimg/19.01/await2.jpg
Diff not rendered.
Binary file removed public/oriimg/19.01/await3.png
Diff not rendered.
Binary file removed public/oriimg/19.01/await4.png
Diff not rendered.
Binary file removed public/oriimg/19.01/await5.png
Diff not rendered.
Binary file removed public/oriimg/19.01/dock1.jpg
Diff not rendered.
Binary file removed public/oriimg/19.01/dock2.gif
Diff not rendered.
Binary file removed public/oriimg/19.01/dock2.png
Diff not rendered.
Binary file removed public/oriimg/19.01/dock3.png
Diff not rendered.
Binary file removed public/oriimg/19.01/http2-1.jpg
Diff not rendered.
Binary file removed public/oriimg/19.01/http2-2.jpg
Diff not rendered.
Binary file removed public/oriimg/19.01/http2-3.jpg
Diff not rendered.
Binary file removed public/oriimg/19.01/next1.gif
Diff not rendered.
Binary file removed public/oriimg/19.01/next1.jpg
Diff not rendered.
Binary file removed public/oriimg/19.01/next2.gif
Diff not rendered.
Binary file removed public/oriimg/19.01/next2.jpg
Diff not rendered.
Binary file removed public/oriimg/19.01/sub1.gif
Diff not rendered.
Binary file removed public/oriimg/19.01/sub1.jpg
Diff not rendered.
Binary file removed public/oriimg/19.01/sub2.jpg
Diff not rendered.
Binary file removed public/oriimg/19.01/sub3.jpg
Diff not rendered.
Binary file removed public/oriimg/19.01/sub4.jpg
Diff not rendered.
Binary file removed public/oriimg/19.01/sub5.jpg
Diff not rendered.
Binary file removed public/oriimg/19.02/bfs1.png
Diff not rendered.
Binary file removed public/oriimg/19.02/bfs2.png
Diff not rendered.
Binary file removed public/oriimg/19.02/bfs3.png
Diff not rendered.
Binary file removed public/oriimg/19.02/bfs4.png
Diff not rendered.
Binary file removed public/oriimg/19.02/bfs5.png
Diff not rendered.
Binary file removed public/oriimg/19.02/dfs1.png
Diff not rendered.
Binary file removed public/oriimg/19.02/dfs2.png
Diff not rendered.
Binary file removed public/oriimg/19.02/dfs3.png
Diff not rendered.
Binary file removed public/oriimg/19.02/dfs4.png
Diff not rendered.
Binary file removed public/oriimg/19.02/sketch.gif
Diff not rendered.
Binary file removed public/oriimg/19.02/visbug1.gif
Diff not rendered.
Binary file removed public/oriimg/19.03/bs1.jpg
Diff not rendered.
Binary file removed public/oriimg/19.03/bs2.jpg
Diff not rendered.
Binary file removed public/oriimg/19.03/bs3.jpg
Diff not rendered.
Binary file removed public/oriimg/19.03/bs4.jpg
Diff not rendered.
Binary file removed public/oriimg/19.03/sort1.gif
Diff not rendered.
Binary file removed public/oriimg/19.04/1.gif
Diff not rendered.
Binary file removed public/oriimg/19.04/2.gif
Diff not rendered.
Binary file removed public/oriimg/19.04/3.gif
Diff not rendered.
Binary file removed public/oriimg/19.04/4.gif
Diff not rendered.
Binary file removed public/oriimg/19.04/db1.jpg
Diff not rendered.
Binary file removed public/oriimg/19.04/mr1.jpg
Diff not rendered.
Binary file removed public/oriimg/19.04/pc1.jpg
Diff not rendered.
Binary file removed public/oriimg/19.04/shell1.gif
Diff not rendered.
Binary file removed public/oriimg/19.04/vps1.jpg
Diff not rendered.
Binary file removed public/oriimg/19.04/vps2.jpg
Diff not rendered.
Binary file removed public/oriimg/19.04/vps3.jpg
Diff not rendered.
Binary file removed public/oriimg/19.04/web1.jpg
Diff not rendered.
32 changes: 6 additions & 26 deletions rehypePlugins/rehype-image-src-modifier.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,7 @@ import { visit } from 'unist-util-visit';
import fetch from 'node-fetch';
import fs from 'fs'
import path from 'path';
import { HttpsProxyAgent } from 'https-proxy-agent';

// const proxyUrl = 'http://localhost:7890';
// const agent = new HttpsProxyAgent(proxyUrl);

function rehypeImageSrcModifier() {
return (tree) => {
Expand Down Expand Up @@ -34,32 +31,15 @@ function rehypeImageSrcModifier() {
if (node.tagName === 'img' && node.properties && node.properties.src) {
const src = node.properties.src;
if (src.startsWith('./img')) {
node.properties.src = src.replace('./img', '/oriimg/' + month);
node.properties.src = src.replace('./img', `https://raw.githubusercontent.com/sunwu51/notebook/master/${month}/img`);
}
else if (src.startsWith('img')) {
node.properties.src = src.replace('img', '/oriimg/' + month);
node.properties.src = src.replace('img', `https://raw.githubusercontent.com/sunwu51/notebook/master/${month}/img`);
}

// 如果本地启动无法翻墙,请注释掉下面imgur这段代码
else if (process.env.NODE_ENV ==='production' && src.startsWith('https://i.imgur.com/')) {
let picName = src.replace('https://i.imgur.com/', '');
fetch(src, {agent: undefined, headers :{
"user-agent": "curl/7.84.0",
"accept": "*/*"
}}).then(response => {
if (!response.ok) {
throw new Error(`Failed to fetch ${src}: ${JSON.stringify(response)} ${response.status}`);
}
return response.buffer();
})
.then(buffer => {
if (!fs.existsSync(path.join(process.cwd(), "public", "imgur"))) {
fs.mkdirSync(path.join(process.cwd(), "public", "imgur"));
}
// 将图片数据写入本地文件
fs.writeFileSync(path.join(process.cwd(), "public", "imgur", picName), buffer);
})
node.properties.src = "/imgur/" + picName;
// 如果本地启动无法翻墙,本地build请注释掉下面imgur这段代码
else if (process.env.NODE_ENV === 'production' && src.startsWith('https://i.imgur.com/')) {
let picName = src.replace('https://i.imgur.com/', '/api/imgur?filename=');
node.properties.src = picName;
}
}
});
Expand Down

0 comments on commit f1213c6

Please sign in to comment.