Is it possible to use files stored inside github repo? #130
Comments
|
Currently, Velite does not support remote content sources. You can consider using git submodule with automated workflows to implement it. |
|
Thank you, I will try to do that. |
Will Velite supporting remote content sources be an expected feature in the later future? |
|
At present, we can only say that it is possible, The current consideration is to implement it through a custom loader. Although the loader architecture is not designed to support this. maybe future: import { defineLoader } from 'velite'
export default defineLoader({
// name: 'json',
test: /^http\:\/\//,
load: pattern => {
// fetch data from remote
return new VFile({ ... })
}
}) |
It would be so awesome if possible. Velite does not officially support remote content sources, but you can approximate that functionality by creating a custom loader. The idea is to treat a remote URL as if it were a local file, then make an HTTP request within the loader to retrieve and parse the data. Although the loader architecture is not designed exclusively for network operations, the example below illustrates a possible approach.
Example code: import { defineConfig, defineLoader } from 'velite'
import { VFile } from 'vfile'
const remoteJsonLoader = defineLoader({
// This pattern will capture URLs that begin with http:// or https://
test: /^https?:\/\//,
load: async vfile => {
// The URL is stored in vfile.path
const url = vfile.path
try {
const response = await fetch(url)
if (!response.ok) {
throw new Error(`Failed to fetch data from ${url} (status: ${response.status})`)
}
const text = await response.text()
let data
try {
data = JSON.parse(text)
} catch (err) {
throw new Error(`Invalid JSON structure from ${url}`)
}
// Return an object that Velite recognizes.
// Typically, { data, content } or a similar structure
return {
data, // This can be treated as frontmatter data
content: '' // Remote JSON might have no separate "content" body
}
} catch (err) {
throw new Error(`Remote JSON loader error: ${err.message}`)
}
}
})
export default defineConfig({
collections: {
posts: {
// The pattern array may include both local and remote references
pattern: [
'posts/**/*.md',
'https://example.com/posts.json'
],
// Additional schema, transform, or collection settings can be defined here
}
},
loaders: [
remoteJsonLoader
]
})Usage details:
To enable Velite to read remote JSON as if it were local MDX content, you need to structure the remote JSON in a format that Velite can process. Velite typically handles content as objects with Example: Remote JSON FormatAssume the remote JSON is stored at [
{
"data": {
"title": "My First Remote Post",
"date": "2024-06-01",
"slug": "my-first-remote-post"
},
"content": "# Hello from Remote\n\nThis is MDX content stored remotely.\n"
},
{
"data": {
"title": "My Second Remote Post",
"date": "2024-06-02",
"slug": "my-second-remote-post"
},
"content": "# Another Remote Post\n\nHere is another example of remote MDX content.\n"
}
]
The Custom Loader will treat each object in this JSON array as an individual document. Custom Loader ImplementationThe following example shows how to create a Custom Loader to fetch remote JSON and integrate it into Velite's pipeline. import { defineConfig, defineLoader } from 'velite'
import { VFile } from 'vfile'
const remoteJsonLoader = defineLoader({
// Match URLs starting with http:// or https://
test: /^https?:\/\//,
load: async vfile => {
const url = vfile.path
try {
const response = await fetch(url)
if (!response.ok) {
throw new Error(`Failed to fetch data from ${url} (status: ${response.status})`)
}
const text = await response.text()
let jsonData
try {
// Assume the JSON structure is an array of objects with { data, content } fields
jsonData = JSON.parse(text)
} catch (err) {
throw new Error(`Invalid JSON structure from ${url}`)
}
// If Velite supports processing multiple documents, return an array of vfiles
const vfiles = jsonData.map((item, idx) => ({
data: item.data,
content: item.content,
path: `${url}?doc=${idx}` // Assign a unique path for each document
}))
return vfiles
} catch (err) {
throw new Error(`Remote JSON loader error: ${err.message}`)
}
}
})
export default defineConfig({
collections: {
posts: {
name: 'Post',
pattern: [
'posts/**/*.md', // Local files
'https://example.com/posts.json' // Remote JSON source
],
schema: ({ s }) =>
s.object({
data: s.any(),
content: s.mdx() // Interpret content as MDX
})
}
},
loaders: [remoteJsonLoader]
})Rendering in a Next.js ProjectAfter running import { posts } from '@/.velite'
export default function Page() {
return (
<main>
{Array.isArray(posts) &&
posts.map((post, i) => (
<article key={i}>
<h2>{post.data.title}</h2>
{/* For rendering MDX content, use a component that dynamically executes the MDX code */}
</article>
))}
</main>
)
}Key Considerations
By following this approach, remote JSON data can be seamlessly integrated into Velite, treated like local MDX files, and rendered as part of your Next.js blog or application. |
|
I made gitpick, a CLI tool to sync directories from GitHub repositories and more. Run 
Say your files are located at a valid GitHub URL with a directory: You can sync them to a target directory in your project using: npx gitpick https://github.com/zce/velite/tree/main/docs awesome-docs
If you need to watch for changes, just add gitpick https://github.com/zce/velite/tree/main/docs -wAdditionally, you can use this tool to clone any file or directory from any repository. For example: gitpick https://github.com/nrjdalal/awesome-templates/tree/main/next.js-apps/next.js-proThe above command will clone the Run I would love feedback from you all. Let's go! |
@AuJezus checkout gitpick's watch mode to do just that
|
This is awesome! |
|
@bobbytxt do try and let me know if it's working for you |
|
You can put gitpick in your import { defineConfig, s } from 'velite'
import { spawn } from 'node:child_process'
spawn('npx', ['gitpick', 'zce/velite', 'content/github', '-w'])
.stdout.on('data', data => console.log(`${data}`))
/*
this will sync zce/velite repository, to ./content/github
you can sync directories with gitpick too, check repo out for usage
*/
export default defineConfig({
// ... velite.config.js
})When you need to sync once or during build, just switch If you need any help with that, feel free to reach out to me at X. |
compatible with edge runtime? |
what exactly does that mean? care to explain a little more |
Hello, I really like velite, it's a great project so please keep working on it :).
I would like to store my files on github repo, so I could use this as hosting for my .mdx files and each website that will consume these files will be able to pass it's own react components to render to html.
I know it's possible to get files from github using contentLayer, but I would like to use velite.
The text was updated successfully, but these errors were encountered: