Moving the new ToH to the Wiki

[richb-hanover]

The comments on the forum seem positive. Congratulations!

I think the next step would be to figure out how to move the new, improved ToH to the openwrt.org site. Here are some steps you'll need to know:

1. Do you have an account to edit the Wiki?
2. There's a "playground" that allows you to create your own test pages. It's at: https://openwrt.org/playground/soif (or https://openwrt.org/playground/wxop, if you prefer). Click the pencil with "+" to create a new page at that URL
3. You will need to know that the OpenWrt wiki is managed by the Dokuwiki software. I don't specifically know how you'd integrate a bunch of Javascript into it...

[thess]

I think it would be easy to move this new ToH to a separate URl served by the same Nginx instance as the Wiki. The bigger issue here is the missing cross references/links.

[richb-hanover]

@thess @soif I suspect you're right. I imagine it could be straightforward to construct a URL that had all the columns, filters, etc. that showed the "right stuff".

But I wonder whether it's possible to "embed" the new ToH into a standard wiki page so that all the headings/menus/etc. are preserved....

[soif]

Adding my new Table of Hardware (TOH) to DokuWiki seems not to be straightforward. DokuWiki doesn't allow including plain HTML or JavaScript by default for security reasons.

It may be feasible to use the Dokuwiki iframe plugin. This would allow keeping the current OpenWRT site layout (header, left menu, footer), but if the TOH were hosted elsewhere (e.g., on GitHub), it would pose a significant security risk. I'm not sure if the DokuWiki admins would be able to restrict the usage of this plugin to particular wiki users only.

The best approach would likely be to host the page directly on the OpenWrt webserver, perhaps on a subdomain, similar to how the current firmware selector is implemented.

I don't have any contacts within the OpenWrt team, but if they were aware of this project and willing to adopt it, they would surely know the best way to integrate it. Modifying index.html to add an OpenWrt-like header would then be a matter of just a few minutes

[richb-hanover]

Both Ted Hess (@thess) and I are in the OpenWrt team, so you do have contacts. I'm not sure about the process of getting the new ToH accepted into the main system but we can look into this.

I agree with Ted's suggestion that it's better use a subdomain on openwrt.org, similar to the Firmware Selector. And like the Firmware Selector, I imagine that it would be best to create an "OpenWrt-flavored" header/footer instead of trying to clone the wiki's template exactly.

Here are a bunch of random thoughts of tasks to move this forward.

• The ToH site can remain on github.io. It's low traffic, so Github won't get mad at us.
• There's a way to set up a CNAME on a github.io site so that it could become toh.openwrt.org (or whatever)
• The source code should be accepted in a directory under the main OpenWrt repo
• The new ToH index.html should get a makeover to "echo" the main site's styling
• As part of the project, several of the existing ToH pages should be converted to the new view, just to demonstrate its utility
• It would be useful to produce a guide for converting existing ToH tables to help others convert their tables

Another couple thoughts:

• Am I correct that some process distills down the hundreds of TechData pages to produce the toh.json file? It would be useful to document that process. (Even though this isn't strictly the responsibility of this project, it should be written down somewhere.)
• Would it be possible to add new fields to the TechData pages and have them picked up automatically in the toh.json file, and then, if selected, shown in the new ToH?

Thanks again!

[soif]

Happy to have some official OpenWrt contacts now! 😎

Regarding hosting: (If the OpenWrt team intends to integrate this project)

• I think the best option would be to transfer the repo under the https://github.com/openwrt organization, exactly like firmware-selector, to something like "toh-openwrt-org" (if you intend to use it as toh.openwrt.org, for example)
• When you're ready, there's a procedure on Github to transfer ownership from me to OpenWrt
• This would allow the OpenWrt organization to get full control of the repo, and easily manage if you want to CNAME the repo, or daily clone it on your own OpenWrt web server(s)
• This would also secure the code, in case I become unavailable, etc.
• Putting me as one of the maintainers for this particular repo at OpenWrt would certainly be a good idea, at least for the moment 😄
• This would also likely attract more developers to collaborate on the project
• I would obviously donate the code to OpenWrt, and change the License (if needed) to whatever is required

Code:

The new ToH index.html should get a makeover to "echo" the main site's styling

No problem. Would you like me to mimic the firmware selector page look?

As part of the project, several of the existing ToH pages should be converted to the new view, just to demonstrate its utility
It would be useful to produce a guide for converting existing ToH tables to help others convert their tables

I'm not sure I understand what you mean 👶

Am I correct that some process distills down the hundreds of TechData pages to produce the toh.json file?
It would be useful to document that process. (Even though this isn't strictly the responsibility of this project, it should be written down somewhere.)

I obviously don't do it myself. I simply use the toh.json file that is certainly automatically built by some script, somewhere. I have absolutely no idea how this is done.

Would it be possible to add new fields to the TechData pages and have them picked up automatically in the toh.json file, and then, if selected, shown in the new ToH?

If the maintainer of the "distiller" script has anticipated this in their code, or slightly changes it to include new fields, from my side, including it in the new TOH is just a matter of adding the fields definition into the toh_conf.js. In fact, without doing anything, the new field(s) would automatically appear at the end (right) of the table. The configuration line to add simply gives the column a name, and defines how the sort is setuped.

Maybe it's time to speak to your colleagues in the OpenWrt organization and ask them if they are interested in integrating this project or not.

[thess]

Before we charge ahead with anything here, we should at least see what the current authors: @jow- and/or @aparcar have to add.

The existing ToH javascript is in https://github.com/openwrt/toh and deployed nightly by gh-actions. It reads a CSV data file generated by a SQL shell script which dumps the DB nightly and then imports it into gh-pages using a package called Zola

The toh.json and toh.json.gz are generated by a nightly Perl script from the hardware text pages / forms served by Doku.

I'm sure there is a connection between these 2 operations but I haven't found it yet.

[soif]

Dear @jow- , @aparcar : Any feedback about this enhanced OpenWrt TOH ?

[aparcar]

Sorry for the slow response I'll try to react on this over the weekend! Thanks for you work and interest this looks great

[soif]

Nice reminder for @aparcar and @jow- : Your feedback would be greatly appreciated... 😸

[soif]

Hi guys.
Any feedback, please ?

[aparcar]

Hi, I think it's great work and I'm happy to replace the current implementation. Could you create a PR against the existing implementation? So with the current approach this comment is then magically converted to HTML/JS

<!-- ToH: {
  "source": "json",
  "paging": true,
  "shownColumns": [
    "brand", 
    "model", 
    "version",
    "supportedcurrentrel",
    "devicepage", 
    "deviceid"
  ],
  "filterColumns": {
    "supportedcurrentrel": "!^(EOL|-|)$"
  }
} -->

[aparcar]

I thinks the deployment job on GitHub isn't used after all, @jow- created a script (Perl?) to generate the JSON, it's hosted directly on the openwrt.org domain here https://openwrt.org/toh.json

From my side it's fine to drop the current views of the wiki and add toh.openwrt.org, serving this page like we do with firmware-selector.openwrt.org.

If you want that, please write an email to the email lists, I won't decide this on my own.

[soif]

hi @aparcar

Hi, I think it's great work and I'm happy to replace the current implementation.

Thank you. Happy that you like it 😎

Could you create a PR against the existing implementation?

Where to push it?

So with the current approach this comment is then magically converted to HTML/JS

I'm not sure where to put it ?

I thinks the deployment job on GitHub isn't used after all..

Confused 😕 😕 😕

From my side it's fine to drop the current views of the wiki and add toh.openwrt.org, serving this page like we do with firmware-selector.openwrt.org.

IMHO, it sounds to be the best option. See my previous proposal...

If you want that, please write an email to the email lists, I won't decide this on my own.

Would you (OpenWrt team members) please discuss among yourselves, if you're interested/willing to use my work.
If yes, please come back to me (ie in this issue), so we can discuss the way you would like to integrate my code into the OpenWRT website....

Then, I would obsiouly comply to your wishes, and push the code to the appropriate place!