Similar to projectz

[balupton]

https://github.com/bevry/projectz has been around for a few years now and seems to already accomplish what this project aims to do. It is used on all bevry and docpad repos. Plan is to extend it with a plugin system in the coming months. Happy to improve it as necessary.

[billiegoose]

Whoa. ProjectZ looks cool. More reactions forthcoming as I have time.

[davidmarkclements]

I think the distinction here, is between small discrete module vs all encompassing framework - both good in their own ways

[zeke]

package-json-to-readme has served me well for the last few years, but I really need to dust it off and make some improvements. It's currently useful for getting a README started but after that you're on your own. I would love to borrow some of the concepts from mos and jsmd to help keep READMEs up to date over time.

[zeke]

I'd kinda like to bring https://github.com/zeke/standard-markdown into the mix somehow too, though maybe that's too opinionated.

[balupton]

fwiw the plan is to make projectz have a plugin system - bevry/projectz#85 - the only thing preventing this is established need and developer time

[billiegoose]

What I'd love is a stupidly simple system. My idea would be to take README.md, split it by level 2 headers (those starting with ##) and throw each section to a plugin:

• # Name of Project: dumps the name and description from package.json
• ## Installation: dumps installation instructions, adds --global from preferGlobal in package.json, and lists the names of the commands installed from bin
• ## License adds a pretty license from the license, author, and maintainers fields of package.json

And after the tool has delegated each section of the README to its plugin, it just concatenates them back together.

And this is the critical bit: Any ## header whose name isn't recognized by a plugin is just ignored. So for example, in this case I could add a ## Usage section anywhere, re-run the command, and it would be unmodified while the sections under plugin control got re-written.

The plugin system I think would work using peer dependencies. E.g. you install update-readme, which (like babel) by itself simply does an identity transform. Then to auto generate / auto update the README, you'd add update-readme-title, update-readme-license, update-readme-install and the ecosystem would be like express or gulp, where people just iterate and make whatever plugins they want, like update-readme-better-title-from-github or whatever, and we wouldn't try to control it. (It would be very decentralized.) Also, we wouldn't tie it to using package.json even. Individual plugins could do it however they want - with the one restriction that "one plugin = one section" so it's dead simple to understand and you don't worry about plugins interfering with each other. And (just like browserify) we might have to also have to allow "global" plugins to handle cases like: change project-name everywhere (useful whenever you fork a project and need to rename it!), code-linting like standard-markdown.

Then I'd say we just steal the browserify syntax for plugins:

"scripts": {
  "readme": "update-readme -p title -p license -p install -g [ replace-name -i --use-git-history -o=false ]"
}

This would not be nearly as general and versatile as projectz which actually syncs data from multiple source files (package.json, bower.json, component.json, jquery.json) and updates MANY types of files (README, CONTRIBUTING, LICENSE, BACKERS, HISTORY). However, it is simple enough that we might just pull it off in our spare time without having to invest a ton of time.

Ooh, one final thought: Instead of hard-coding it to README.md, we could make the filename an argument. Then you could make plugins for each separate file:

"scripts": {
  "readme": "update-readme -p title -p license README.md",
  "license": "update-readme -p full-license LICENSE.md",
  "history": "update-readme -p git-history-file HISTORY.md",
  "update": "npm run readme; npm run license; npm run history"
}

Hmm. Maybe a better name than update-readme would be update-markdown. (Looks to see if it's available.) Holy cow! We're halfway done! @bahmutov is one of those geniuses, along with mafintosh, etc, who frequently have already invented a module to do exactly what I was planning to do.

[billiegoose]

Hey @bahmutov I see update-markdown has zero dependents on npmjs.org. Did you ever think about making a plugin ecosystem around update-markdown?

[bahmutov]

Hi everyone,

Thanks @wmhilton any day when I am mentioned in one sentence with mafintosh like that is a great day and a reason to toast a beer at lunch 😄

I have no objections to refactoring update-markdown to work with plugins or changing it. Will be happy to setup semantic-release for faster updates, greenkeeper.io, etc and invite collaborators to move quickly.

[billiegoose]

Alright, I'm excited about this and am gonna see if I can pump out a stupidly simple MVP in the next few hours. https://github.com/update-doc/update-doc

[billiegoose]

Alright, it's taking longer than I expected because as I thought deeper about how Markdown files are structured I decided to make a hierarchical parser that deals with h1-h6 headers rather than just split on h2 headers.

One of my goals is for a perfect identity transform (you could also call it idempotency I guess?) so that I have 100% confidence that a) re-running the tool won't create extraneous diffs that create noise in the git history and b) that plugins will only touch that one area of the README and everything else stays completely untouched. Because there's nothing more frustrating than adding a tool to automate your life and make everything easier and then finding it subtly interferes with other things. (Maybe changes in the README trigger the build process to update the project's docs website, maybe you're using a separate linter, maybe you have no newline at the end of the README file specifically because it gets concatenated some weird way and bundled into the JS...)

I rambled there a bit. Anyway, I'm addressing the problem by brute force, by testing

assert(readme == render(parse(readme)))

on every package in the npm registry. My current record is 23,149 readmes before an error, but I'm about to fix that error and then hopefully it'll get through all 400,000 packages in the registry.