Typsium Roadmap and Ideas

[Ants-Aare]

We want to streamline how people use chemistry in their papers and publications, protocolls, notes, lab journals, study material, slides and more.
The vision:
All molecules and elements used in the paper should be variables. This way you can declare your compounds at the beginning of the file and reference them throughout. The variables should contain as much useful information as possible, molecular weights, elemental percentages, and more will be automatically calculated. Typsium packages should follow the principle "Declare once -> Use everywhere"

[Ants-Aare]

Formula parser

The formula parser should convert any and all formulas to their correctly formatted look, both manually written formulas as well as from variables.

Reactions should look professional and clean, multiple different reaction arrows with descriptions on top or bottom, phase states, endo and exothermic reactions, delta H, etc etc. Reactions should be auto alignable just like math mode aligning.
Maybe it could make sense to add warnings if stochiometry doesn't work(?) maybe not as useful though.

[capichou]

This is done

Does that mean it's time to move from whalogen to typsium? I saw you recommended whalogen somewhere due to stability issues.

[Ants-Aare]

Should all be fixed, do let me know if you encounter anything or open up an issue! I recently wrote the manual (or at least started to write) so we're almost feature complete for that package. If you're interested in contributing, the documentation would be the best way to start! There's a bunch of tests in the test folder and those could be converted to examples or feature showcases.

[capichou]

With the holiday coming up, I'll take a break from typst for a while... But I'll consider contributing to documentation afterwards!

A few bugs / strange behaviour / improvements :

• Iodine atoms are treated as charges. #ce("C_15H11I4NO_4") renders correctly with whalogen but not with typsium.

• Wildcard import breaks arrow usage. Or all symbols? Haven't tried. If I import everything from typsium (#import "@preview/typsium:0.3.0": *) and not only ce, I cannot use this anymore to insert a single arrow somewhere. $arrow.r$ is not longer recognized

• Dashes (#ce("-CH2 - CH3")) are kept with typsium while they are transformed into long dashes with whalogen. I think the whalogen version is better (see picture - whalogen version). (sorry I'm not sure about the right terms between dashes/hyphens and so on).

[Ants-Aare]

This was a known issue and was fixed recently. Just update your import to Typsium:0.3.1 without changing anything else. What you mention about arrow is interesting and I know why it's happening: we have an arrow element that you can invoke manually (all typsium building blocks can be added manually. This makes it easier to integrate with packages etc.) it's like if you declare a method called arrow, then if you write arrow.r, expecting it to draw an arrow, it will actually call the method and try to access the r element inside the return of that method. That's why our arrow is blocking it.

I didn't think people just wildcard entire packages all the time, I know it's maybe in the examples, but it's not best practice, just import ce or do typsium as ts etc. I can update the package if it's too confusing though. I don't think it's good to change our function name to typsium-arrow though, simply because that's exactly what the namespace import etc is for, you should rather do import typsium as typsium and then write typsium.arrow

About the dashes we can totally make that happen. We need to be careful though, since dashes are also used to add negative charges. So HSO4- will show with count four and charge - if we want to display single bonds we need users to add a space or something like that so that the program can distinguish it from charges. We don't need to show actual spaces in the output. We could do it without spaces so If you want I can add an exception so that if dashes appear inside a molecule they will appear as bonds and if they appear at the end they will appear as charges(?) would need more setup, but totally possible. I'm just afraid that it might make reading it less clear. Also should = be double bond? What should triple bond be? There's not really a good easy to reach symbol on the normal keyboard. We can also make bond length configurable with show rules.

[capichou]

Ok for the update (it works, sorry, my bad) and the wildcard best practice.

About your last paragraph, I'd try to follow the same "conventions" as in chemistry packages from LaTex. We can expect (hope?) that those packages respect IUPAC recommendations, and that would ease migration from LaTex to Typst.
https://www.overleaf.com/learn/latex/Chemistry_formulae
(by the way, chemfig is really impressive for organic chemistry)

[Ants-Aare]

Molecule Database
It makes sense to have a database of maybe a couple thousand most commonly used chemicals, along with their smiles, CAS, H&P, etc. this way people can simply do this:
#let ethanol = get-molecule(“ethanol”)

instead of

#let ethanol = define-molecule(common-name:"Ethanol", iupac-name: "ethanol", formula:"C2H6O",smiles:"CCO", ghs:(2,7), h-p:("H225, H319", "P210, P233, P240, P241, P242, P243, P264+P265, P280, P303+P361+P353, P305+P351+P338, P337+P317, P370+P378, P403+P235, P501"))

Data can come from PubChem or Wikipedia and we can just save it as a csv. text is doesn't take that much storage, so we should have no problem just shipping 1000-3000 common chemicals. What is considered a common chemical is a bit harder to define though...

[capichou]

I suppose it's not possible within Typst, but making an external small script (python?) that fills the csv with data from pubchem or wikipedia would both remove the burden of choosing the chemicals, and would allow everyone to get his own list of important chemicals.

[Ants-Aare]

Yeah, I looked into it a while ago, apparently this is a conscious design decision of the typst team. I spoke with them and everything will stay sandboxed for the foreseeable future. There is a package that offers external sources integration with typst called prequery: https://forum.typst.app/t/prequery-get-outside-information-into-your-documents-shell-escape-for-typst/6023 but it won’t work for the online version. So if people in the browser import the package and try using it they will get unexpected behaviour/noop, which is a nogo for me personally. That’s the main reason ( and only reason) why I even suggested the offline csv data approach. For a lot of chemistry databases there are even explicit disclaimers and warnings that you MUST NOT build your own database by scraping the data. Partly because they don’t like it and partly because it’s bad practice and datasets get updated and revised often. I already did this for the Typsium-ghs package for all the H and P statements, but even those get revised every now and then. Until now I refrained from doing it for the molecules because of these reasons. Ideally I’d prefer a threepronged approach: have an offline database for the most commonly used chemicals and functional groups directly in the main package. This would be necessary anyways, so that we can properly locally calculate molecular weights and counts etc for the mixed notation (EtOH for ethanol for example). Then offer „extension dataset“-packages on the universe each with a dataset specific to organic chemistry, inorganic, medical or biotechnology/biology, maybe also physics with all the isotopes, alongside with the option for people to easily load in their own datasets that is specific for their lab. I think it’s important that like 90% of casual web users will be happy this way, and then still offer the prequery option for pro users who are too lazy to import molecules manually.

I’d really appreciate it if someone experienced in chemistry would help with defining the core database for the functional groups at least. Also defining what values are important to include. EN, density, molecular weights, counts, etc is obvious, but first ionisation energy? Is that important? Different Atomic radii? Idk… we’re also currently working on a physical chemistry package and ideally it would interoperate with that and sometimes you need some specific random values like thermal conductivity etc.

[capichou]

IMHO, there are two approaches : included everything you can find, or focus only on "universally needed" values. In between, it would end in an endless debate about the importance of something. Or if you release a scraper, you let people build their own database?

[Ants-Aare]

Hazard and Precautionary Statements
Typst is very good at generating pdfs at scale, one document format that is super easily streamlined are safety datasheets. So data can just be provided from a commandline interface, or from csv sheets, layout is handled by one or multiple selectable templates, and the H&P statements, as well as GHS pictograms are part of the package and can be converted to formatted content.

[Ants-Aare]

this is done!
https://github.com/Typsium/typsium-ghs
https://typst.app/universe/package/typsium-ghs

[Ants-Aare]

Smiles and organic chemistry
since each molecule already has a smiles code attached, it makes sense to be able to easily display the rendered out structures.
Ideally if someone has a method call to render a reaction, they should be able to turn the #reaction into a #reaction-smiles and everything will work just the same, except the organic structures will be displayed.

[capichou]

AFAIK, we still need to use an external software to draw organic molecules. Indeed, using SMILES to display organic structures would be excellent.

Bonus points:

• Option for explicitely showing carbons or not
• Rotation angle. For some molecules (steroïds for example) there are some "conventions" on how to represent them, and rotating the representation helps.

Whatever: Typsium is an awesome project and excellent job! I'm no developer, but I can try to help with suggestions and bug report.

[Ants-Aare]

We have the developer of alchemyst onboard, he wrote the Lewis structure renderer for typst. We have a project started that should parse smiles codes (and ideally Inchi) using wasm and return data that can be rendered by alchemyst, but the project stalled. It’s a bit harder than expected, and even though I’m a developer myself I don’t think I could do it so easily. https://github.com/Typsium/typsium-smiles

the bonus points would be independent of the smiles parsing, so that would be no problem at all and we could probably even do it with show rules. If you want to help, maybe there’s even an off the shelf smiles parser somewhere out there. I saw this one, but haven’t looked much into it: https://duartegroup.github.io/autodE/reference/smiles/parser.html

keep in mind that this is a community driven project, so it might take some time until all features are ready, but everything is already in place for this kind of workflow.

[Ants-Aare]

Elements and Atoms
Getting information about specific elements (including their ions and isotopes) like electron configurations, shell occupation, weight, boiling points, melting points, phase, valence electrons, electronegativity, CAS number.

It should also be possible to display atoms according to bohr model, KLMNO shells, atomic orbitals spdf and their shapes. maybe even some molecule orbitals like sp3 or sigma/pi bonds etc so people can add them onto their cetz graphics if they so choose

[Ants-Aare]

Units
Chemists usually work with moles, meaning that gram->mole or mole->litre conversions are commonplace. If we already have the molecule definitions, including their weight, we can just do all the calculations in the background.

#let hydrogen = get-molecule("hydrogen") // H2
#let oxygen = get-molecule("oxygen")     // O2
#let water = get-molecule("water")       // H2O

#let mol-H = 2
#let mol-O = 1

#reaction[#mol-H #hydrogen + #mol-O #oxygen -> #mol-H #water]

We mix #mol(mol-H, hydrogen) with #mol(mol-O, oxygen).

output:
2H₂ + O₂ -> 2H₂O
We mix 4g of hydrogen with 32g of oxygen.

[Ants-Aare]

If someone is interested in making a periodic table generator I think that would be cool as well!
We have data on everything that goes into a periodic table, all we would need is a table builder method. We can pass in a method that generates the squares element => box(fill: gray, align(center + horizon, element.symbol)), etc etc this way we can have one or two default looks, but users can decide to override it with the information they want. Maybe also options for enabling and disabling certain groups such as transition metals or f-block. or just rendering a specific period or group. Probably most useful for education context, etc.

[tryptophawa]

I had some nice discussion with a experienced typst package developer in my community. He suggest me to try write a document with loads of chemical equations and consider the future development base on that.

Personally, I holp the main package (typsium) could be clean for only easy and modern chemical parser, so that any issue could be fix when I met them. Also this would help other users to use the package that they won't confused because so many functions in the package. As a result, I wish to move all code to a new, temporary repo except for the main function about chemical formula parsering.

Some breaking change could happen when I finish the trial on our package. I am trying to use wasm, which returns a json file, with @sjfhsjfh in typsium to replace the regex rules.

[sjfhsjfh]

1. Converting from content to IR to content is obviously slow

2. In math mode when you call a function without the #, it would naturally pass the content sequence to as the only positional argument. So it does work and works well.

I think we should use the example document to iterate the syntax design in a draft PR.

[sjfhsjfh]

Also, ideally $ce("A" + "B")$ and $ce("A + B")$ should be exactly the same. This can be one of the test cases for the parser and IR -> content conversion tool.

[tryptophawa]

hey tryptophawa, I think there must have been some kind of miscommunication. I sent you a couple of emails, but haven't heard back from you in a while. Also who's @sjfhsjfh and why does he seem to know more than me? If you are working on a wasm refactor on the side, why is none of it public on github? I tried checking your or his github for any activity, but I can't see anything.

I was battling with my English exam recently. Sorry for that.

[tryptophawa]

> > ```typst > #let y = string-to-ir("H2SO4 ->[$Delta H^0$ <- Mathstuff! ][H2O ->[Arrowception][Bottom Text] OH- + 5C[NO]4] ABC-") > ``` > > For example a couple days ago I thought adding reactions on top of or below reaction arrows could be useful so I made a mock of how it would look like written out and a mock method returning the intermediate representation and it looks pretty okay, so next step would be to implement it and add the unit test. especially tricky is the square bracket inside the reaction, so it doesn't misinterpret it as closing the reaction top and bottom content. But everything works now and isn't in the way when not used.

Wow, this looks great. I actually try this a few weeks ago, but didn't success.

[Ants-Aare]

Ah I think it wasn't compiling because I put your example outside of a math block and it was giving errors like "unexpected arrow" and "The character ^ is not valid in code" but now it compiles.

$ce("A" + "B")$ and $ce("A + B")$ should be exactly the same

I'm not able to reproduce this though, first is sequence([A], [ ], [+], [ ], [B]) and the second is "A + B"
Basically the first is exactly the same what I get when doing it with #ce[A + B] (this returns sequence([A], [ ], [+], [ ], [B]) as well).

Converting from content to IR to content is obviously slow

I haven't seen any drawbacks yet, I just move around the content and it's children. As a crude example of what it's basically doing:

#let test(body1, body2) = {
  //assign the content to variables inside ir
  let ir = (
    "body1":body1,
    "body2":body2,
  )

  // display the content from ir and transform the layout etc
  ir.body2
  linebreak()
  ir.body1
}
#test[Test1][Test2]

output: Test2 Test1

as far as I know this is intended behaviour and a bunch of packages use analysing content a lot. Touying for example takes the content, iterates over it's children and places them on slides. This is what I was trying to say when I explained why I wrote it the ir implementation in typst, because pushing the content through a wasm/rust pipeline could definitely slow things down and could cause all sorts of bugs.

The default math mode doesn't look that good for reactions. putting lr(), attach(), stretch(), etc in a bunch of places just makes things hard to read.

I agree with the making a syntax UX reference document. We can also remove some methods as @tryptophawa wanted.
On another note, I've been recently looking a lot into lilaq and what they're doing with custom elements using elembic. It probably makes most sense to wait for an official types and elements implementation, but I love how typst-y the show and set rule syntax feels when writing lilaq, it's just a very nice UX and I've been thinking about implementing it in the molecule displayer. And what I love most about it is that you can define your rules once and then every package that uses typsium as a dependency will have those rules apply to it as well

[tryptophawa]

any interesting things recently?

[Ants-Aare]

not really, I looked through the rust implementation and it looks nice, although it has some weird results sometimes and the rendering implementation needs a lot of work. I'm not sure... maybe it does make sense to use wasm. I've been back to university and daily driving the IR implementation and it holds up quite well. I need to add hydrate groups and oxidation numbers support, as well as maybe some better handling of Rest syntax R` so that it looks better. There have been lots of issues added to the alchemist repo, and half of them are about formula rendering instead of structure rendering. And robotechnic is gradually adding the features step by step. I don't think it makes sense though, because the implementation there isn't particularly good and I have a feeling people will continue requesting stuff. So I think it makes sense to add typsium in there. But in the current state things are changing too much and I'd like it to be more stable first. Also I'm working on figuring a way out to make styled formulas work well. I already have the rendering figured out, but it needs a bit more I think. The code is really awful, and applying two styling functions doesn't work yet, but I got it to output colored charges and counts, striking through and canceling, of individual symbols or molecules, over/underbrace, etc. italic & bold... and I took some of the suggestions from @sjfhsjfh to make the syntax more typst reliant. If you want I'd love to brainstorm some ideas with you on how it makes most sense to pull this off. let me know if you'd like to videocall sometime.

[tryptophawa]

Got it. I will be back on the 11th of June.

[tryptophawa]

https://github.com/ParaN3xus/typst-snippets/blob/main/indenta/indenta.typ
maybe we can check how content is recovered in this repo.