diff --git a/.vscode/ltex.hiddenFalsePositives.en-US.txt b/.vscode/ltex.hiddenFalsePositives.en-US.txt new file mode 100644 index 000000000000..7d982039a85e --- /dev/null +++ b/.vscode/ltex.hiddenFalsePositives.en-US.txt @@ -0,0 +1 @@ +{"rule":"EN_UNPAIRED_BRACKETS","sentence":"^\\QThe simplest way would be to set up an intrinsic template like \"subject-verb-object\"^1 in order to offset portion of the probabilistic model's support^3 to not put any probability mass to structures other than \"subject-verb-object\".\\E$"} diff --git a/_bibliography/papers.bib b/_bibliography/papers.bib index 9b5f3eaf6d13..d0f0a9da28a4 100644 --- a/_bibliography/papers.bib +++ b/_bibliography/papers.bib @@ -4,7 +4,9 @@ @article{anantonio2023 author={An, Xuelong and Vergari, Antonio}, year={2023}, poster={sassy-clevr-poster-beta.pdf}, - preview={nesy_models_framework_verbeta.png} + slides={nesy-slides.pdf}, + preview={nesy_models_framework_verbeta.png}, + pdf={final_draft_chart_nesy_reasoner-gamma.pdf} } diff --git a/_posts/2023-02-08-workload.md b/_posts/2023-02-08-workload.md new file mode 100644 index 000000000000..e69de29bb2d1 diff --git a/_posts/2023-04-26-path.md b/_posts/2023-04-26-path.md new file mode 100644 index 000000000000..e69de29bb2d1 diff --git a/_posts/2023-06-28-lecunn.md b/_posts/2023-06-28-lecunn.md new file mode 100644 index 000000000000..e69de29bb2d1 diff --git a/_posts/2023-09-01-nesy-summer-school.md b/_posts/2023-09-01-nesy-summer-school.md index b5137c7a1d64..d4dbdab46e4e 100644 --- a/_posts/2023-09-01-nesy-summer-school.md +++ b/_posts/2023-09-01-nesy-summer-school.md @@ -19,7 +19,7 @@ I attended the NeuroSymbolic (NeSy) Summer School 2023 held [virtually](https:// They mostly talked about the use of logic, propositional or first-order, to shape a problem domain and drive neural networks to make explainable predictions, discussing the strenghts and drawbacks of this approach. -I am personally skeptical of the use of logic to encode knowledge or describe relations. This is because there exists infinite knowledge, and thus this entails that in order to solve real-world problems that involve complex systems, e.g. predicting cell progression in a tissue, may involve writing infinite amount of rules encompassing cell physiology or pathogenesis. It is also disputable what are the putative rules to encode in order to solve problems, and rarely two experts on the same field may converge on what rules should be encoded into a model for solving a problem like cell trajectory analysis. On a tangent, I also liked the point raised by Prof. Benjamin Grosof on defeasible knowledge, referring to rules which are only true circumstantially. For example, for a rule like 'every cell has a nucleus', this is only true except when it doesn’t such as during anaphase of mitosis or if it's a bacterium. One can relax the "absoluteness" of rules by encoding probabilistic knowledge, i.e., rules with probability distributions on their truth values attached to them. However, this only exacerbates the inherent combinatorial search space related to the search for rules that can answer a query. +I am personally skeptical of the use of logic to encode knowledge or describe relations. This is because there exists infinite knowledge, and thus this entails that in order to solve real-world problems that involve complex systems, e.g. predicting cell progression in a tissue, may involve writing infinite amount of rules encompassing cell physiology or pathogenesis. It is also disputable what are the putative rules to encode in order to solve problems, and rarely two experts on the same field may converge on what rules should be encoded into a model for solving a problem like cell trajectory analysis. On a tangent, I also liked the point raised by Prof. Benjamin Grosof on defeasible knowledge, referring to rules which are only true circumstantially. For example, for a rule like 'every cell has a nucleus', this is only true when the cell is not going through anaphase of mitosis or if it's a bacterium. One can relax the "absoluteness" of rules using temporal logic ('something is true during certain timeframes') or by encoding probabilistic knowledge, i.e., rules attached with probability distributions on their truth values. However, these only exacerbate the inherent combinatorial search space related to the search for rules that can answer a query. Let us illustrate with the following hypothetical logic program: @@ -51,9 +51,11 @@ My criticism towards this kind of logic is that it doesn’t reflect the complex Rather, I believe that the brain stores operations/programs which operate on the world to obtain knowledge. In other words, it doesn’t store knowledge, it scaffolds from the environment to obtain knowledge. For example, it is not the case that a Herbrand Base of some logic program is stored in my brain, rather, inside it there are programs, such as [modus pollens and modus tollens](https://human.libretexts.org/Bookshelves/Philosophy/A_Miniguide_to_Critical_Thinking_(Lau)/01%3A_Chapters/1.08%3A_Patterns_of_Valid_Arguments) that allow to process this logic program that is written on the screen in front of you. What is interesting about this is that, for example, Shirley the person is not stored in my mind, she exists on the real world. What is actually in my mind is an operation like modus pollens, which when applied on her and her smoking habit, allows me to deduce she is a smoker. Perhaps the rule of whether she is a smoker is encoded in prior exposure to social dynamics of friends and smokers. Here, the encoding procedure, the manipulation of social dynamics and smoking habits can all be thought of as programs. -Another source of beauty with programs is that one doesn't need to write rules, but rather a domain-specific language with grammar rules for the machine to use it as its language of thought to interpret incoming data. As an analogy, I like to think of it as defining a programming language, where a machine learns to understand the world by manipulating code written in it. +Another source of beauty with programs is that one doesn't need to write rules, but rather a domain-specific language with grammar rules for the machine to use it as its language of thought to interpret incoming data. As an analogy, I like to think of it as defining a programming language, where a machine learns to understand the world by writing and editing its own code, a form of self-reference. -Furthermore, I hypothesize that perhaps we need a vector embedding for primitive programs and their compositions. Empirically, it has been observed that continuous representation of discrete symbols/concepts tend to speed up their manipulation, such as searching over them. +Furthermore, I hypothesize that perhaps we need a vector embedding for primitive programs and their compositions. Empirically, it has been observed that continuous representation of discrete symbols/concepts tend to speed up their manipulation by exploiting GPU parallelism, such as searching over them. + +Interesting work revolving on program synthesis include [neural module networks](https://arxiv.org/pdf/1511.02799.pdf), and an [improvement](https://arxiv.org/abs/1704.05526) over this architecture. @@ -147,9 +149,9 @@ I think it depends on the context. For proving mathematical theorems, humans arg A member of the audience appealed to the bird-airplane thought experiment: will something similar happen to the brain-computer relationship, i.e., we can design an algorithm ran on a computer that is devoid of the flaws of the brain, such as the brain's limited lifespan, biological constraints, and obtain a tool that is far more powerful than the brain itself. This is analogous to how an airplane, despite having its architecture inspired by the phenotype of the bird, is many scales more efficient at flying and carrying cargo than any existing bird. -First, I think the analogies are flawed at its core if we want to discuss on the topic of intelligence. While the airplane succeeds at a particular task of flying over long distances, it is devoid of intelligence and is dependant on the human flying it. Without a human operator, an airplane that is about to crash, or goes low on fuel, has absolutely no way to adapt to changes, and formulate a plan to get out of its predicament by itself. Birds, on the other hand, can engage in simple problem solving concerning survival, reproduction and caring for the young, all while adapting to circumstantial events in their environment. One striking example that is often cited by [Prof. Song Zhu-Chun](https://arxiv.org/pdf/2004.09044.pdf) is of a crow that can take advantage of incoming cars to [break nut shells](https://www.youtube.com/watch?v=NenEdSuL7QU) and eat them. +First, I think the analogies are flawed at its core if we want to discuss on the topic of intelligence. While the airplane succeeds at a particular task of flying over long distances, it is devoid of intelligence and is dependant on the human flying it. In some regard, intelligence perhaps is _not just about solving a task_. Without a human operator, an airplane that is about to crash, or goes low on fuel, has absolutely no way to adapt to changes, and formulate a plan to get out of its predicament by itself. Birds, on the other hand, can engage in simple problem solving concerning survival, reproduction and caring for the young, all while adapting to circumstantial events in their environment. One striking example that is often cited by [Prof. Song Zhu-Chun](https://arxiv.org/pdf/2004.09044.pdf) is of a crow that can take advantage of incoming cars to [break nut shells](https://www.youtube.com/watch?v=NenEdSuL7QU) and eat them. -However, once we have grasped the dark matter of intelligence in a model, then I think scaling it up will lead to wonders that we can now only envision: supercomputers able to [manage cities](http://www.wadjeteyegames.com/games/technobabylon/), accelerate scientific discovery or take us to Mars. +However, once we have grasped the dark matter of intelligence in a model, then I think scaling it up will lead to wonders that we can now only envision: supercomputers able to [manage cities](http://www.wadjeteyegames.com/games/technobabylon/), accelerate scientific discovery such as in the biological sciences and take us to Mars. ## What is hardcoded in the brain? diff --git a/_posts/2023-10-31-sgd.md b/_posts/2023-10-31-sgd.md index 0ddc79c47368..c968caab94d6 100644 --- a/_posts/2023-10-31-sgd.md +++ b/_posts/2023-10-31-sgd.md @@ -7,11 +7,9 @@ tags: food-for-thought creative-work categories: blog-post disqus_comments: true related_posts: true -thumbnail: /assets/img/sgd4life.png +thumbnail: /assets/img/sgd4life.PNG.png toc: sidebar: left - -# related_publications: einstein1950meaning, einstein1905movement --- # A tale of a scientist @@ -40,7 +38,7 @@ The stochasticity of gradient descent is mainly derived from the order and the s Given these precedents, I wanted to depict my desire to see in the future a kind of "structured" gradient descent (which in English coincides with the same acronym as stochastic gradient descent):
- Sorry, an unanticipated error occured and the image can't load. + Sorry, an unanticipated error occured and the image can't load.
Not an actual depiction of structured gradient descent, but nonetheless a cool logo.
@@ -54,6 +52,6 @@ The unanswered questions that follow would include: - Most importantly, would more intricately labeled data compensate for low data?
- Sorry, an unanticipated error occured and the image can't load. + Sorry, an unanticipated error occured and the image can't load.
A (hopefully) cool logo of SGD
\ No newline at end of file diff --git a/_posts/2023-11-2-mechanistic-subtypes-parkinson-copy.md b/_posts/2023-11-2-mechanistic-subtypes-parkinson-copy copy.md similarity index 100% rename from _posts/2023-11-2-mechanistic-subtypes-parkinson-copy.md rename to _posts/2023-11-2-mechanistic-subtypes-parkinson-copy copy.md diff --git a/_posts/2023-11-3-biophysical.md b/_posts/2023-11-3-biophysical.md new file mode 100644 index 000000000000..6eeba1aa3e6d --- /dev/null +++ b/_posts/2023-11-3-biophysical.md @@ -0,0 +1,47 @@ +--- +layout: distill +title: "Review of the paper Learning biophysical determinants of cell fate with deep neural networks" +date: 2023-11-02 19:42 +description: comments on a paper that leverages deep learning to classify epithelium cell fate by observing its live image trajectory. +tags: research +categories: blog-post +disqus_comments: true +related_posts: true +authors: + - name: Xuelong An +toc: + - name: Brief summary + - name: My comments and future research directions +thumbnail: /assets/img/biophysical-mitosis.png +bibliography: deep-med.bib +--- + +# Brief summary + +The paper by leverages deep learning architectures to solve a pentanary classification task given either a cell's tabular features or images. The five independent classes are one healthy control and four subtypes of Parkinson's Disease: familial proteinopathy (SNCA), environmental proteinopathy ($$\alpha$$-Syn oligomer), and two subtypes characterized by different mitochondria dysfunction pathways. These pathologies were chemically induced on stem cells. Fifty-six phenotypical features of them were extracted automatically and recorded as tabular data, along with images of the cells extracted via microscopy. Both data modalities were labeled with one of the five classes. + +The research team trained separately a dense feedforward neural network (DNN) to classify on the tabular data, as well as a convolutional neural network (CNN) to classify on image data. The test classification accuracy achieved by the DNN reached around 83%, while the CNN 95%. + +
+ Sorry. Image couldn't load. +
Two separate models are trained on different datasets on the same task of Parkinson subtype classification. Figure extracted from the original research article at https://www.nature.com/articles/s42256-023-00702-9
+
+ +# My comments and future research directions + +Generally, in the deep learning literature, it is acknowledged that the usage of DNNs comes at the expense of poor explainability. Despite achieving high classification accuracy, these models are black-boxes. Nonetheless, there are ways to identify what are the features that the neural networks pay the most attention when deciding on a classification label, mainly by looking at its last layer's activation and tracing back to the input space which input feature is associated to it. In CNNs, the technique employed by the research team is called the ShAP (SHapley Additive exPlanations) method. + +The authors managed to identify in both the DNN and CNN that the mitochondria, lysosome and the interaction of both features mainly contributed to the classification decisions of both models. + +One future research direction I am interested is exploring whether by integrating both data sources can improve performance and yield explainability, because the original work trains separate models, trained on different datasets. + +One source of inspiration is from , where they integrate image data along with a mouse's behavioral features to predict its neural responses collected from neural recordings. Another source of inspiration is drawn from concept-bottleneck models . There, a CNN in charge of processing images doesn't learn to output a classification label, but instead to output features that are relevant to the image. These features, in turn, are annotations of the image stored in tabular: + +
+ Sorry. Image couldn't load. +
A depiction of the pipeline of a concept-bottleneck model. The first half outputs a set of concepts given an image, which can be learnt from intricate annotations, or metadata, of the image. The second half outputs a classification label. Figure extracted from the original paper
+
+ +Altogether, with regards to the work by , one interesting extension to their CNN is to have it not predict a Parkinson subtype, but rather learn to predict the cell's physiological features stored as tabular data given image input. Subsequently, use the features to train a multi-class regressor using standard softmax to output a classification label. The prospect is that this hybrid model can leverage the high accuracy prediction of the CNN, whilst being explainable thanks to the logistic regressor. + +As a further improvement, we can use a [Slot Transformer](https://arxiv.org/abs/2210.11394) instead of the CNN with the hope of learning a disentangled representation given the image with its annotations. However, the architecture will be more computationally expensive. A pretrained Slot Transformer that already learnt to disentangle CLEVR-Scenes may be more powerful than training it from scratch. \ No newline at end of file diff --git a/_posts/bayesianism.md b/_posts/bayesianism.md new file mode 100644 index 000000000000..063df47f731d --- /dev/null +++ b/_posts/bayesianism.md @@ -0,0 +1,11 @@ +# una breve historia + +En secundaria, uno puede ser introducido al teorema de Bayes. +# Obteniendo la distribución posterior + + +# Muestreo de una distribución +# Study tips + +- What are the types of variables? + - function, scalar, vector, matrix, or else \ No newline at end of file diff --git a/_posts/halicin.md b/_posts/halicin.md new file mode 100644 index 000000000000..44d3866a974e --- /dev/null +++ b/_posts/halicin.md @@ -0,0 +1,114 @@ +\section{Technical background, research approach, and results of the paper} + +\subsection{Motivation} +\citet{stokes_2020_a}, hereon referred to as "the authors", address the global health concern of the proliferation of antibiotic-resistant bacteria by leveraging artificial intelligence (AI) for large-scale, high-throughput drug screening. + +% However, the Achilles's heel of medicine is that bacteria, and cells in general, mutate and adapt to a precarious environment \cite{swings_2017_adaptive}. These can then pass antibiotic-resistant determinants that render existing antibiotics powerless to fight infections. Traditional means of screening can not scale beyond millions of compounds \citep{liu_2023_deep}, a tiny fraction of the estimated search space in the order of $10^{60}$\citep{wang_2023_scientific}. + +Antibiotics are amongst the essential tools to fight against microbial infections. However, the Achilles's heel of medicine is that existing antibiotics can pressure bacteria to adapt to them through mutation and passing antibiotic-resistant determinants, rendering them useless. Thus, re-purposing and discovering new drugs to mitigate the proliferation of them are urgent to prevent deaths associated to antibiotic-resistant infections \citep{stokes_2020_a}. + +There is a vast chemical space (in the order of $10^{60}$ compounds) to explore for possible candidates \cite{pavel_2022_the}. Nonetheless, most of this search space consist of non-usable biochemicals which can not be anticipated beforehand, thus would render its exploration and testing a waste of resources. Traditional means of screening can not scale beyond millions of compounds, and may suffer from the de-replication problem: same compounds are repeatedly discovered. A tangential problem is to find compounds structurally similar to existing ones, which could be deleterious in the long-term because bacteria that developed resistance to a drug may well be resistant to analogues\cite{liu_2023_deep}. An alternative that can bypass this flaw resorts to \textit{in silico} methods, i.e., computer simulations, in particular deep-learning to exploit its feature-extraction capabilities to model complex relationships \citep{jimnezluna_2020_drug}. \textit{In silico} methods vectorize molecules to obtain a representation that can be processed by a machine, and can conveniently scale. These features can be handcrafted based on domain-expertise, denoted as "molecular fingerprints", and they can be obtained from Dragon descriptors, Morgan fingerprints or using the open-source package RDKit \citep{yang_2019_analyzing}. However, domain-knowledge is often disputable, and experts may disagree on what are the putative features of a molecule. Another approach is to have a graph representation of a molecule whereby its hidden state is learnt via a deep graph convolutional neural network in a downstream, prediction task. The strength of a graph representation includes retaining the geometrical information (e.g spatial atom-atom bonding) of the molecule that could be relevant to determine its function. + + +\subsection{Model architecture and dataset} +The authors adopt a hybrid architecture, called Chemprop\footnote{Code available at: \url{https://github.com/chemprop/chemprop/tree/master}}, that leverages both molecular fingerprints and learn a hidden representation for each molecule, combining the strengths of both worlds: the incorporation of expert knowledge, and flexibility of learning task-dependent, global hidden representations. It is a Directed Message-Passing Neural Network (DMPNN), a variant of the Message-Passing Neural Network, where message passing is asymmetrical, and is done among bonds instead of atoms in order to avoid redundant messages\citep{yang_2019_analyzing}. +The authors frame drug discovery as a binary function classification task given a molecule, and validate their model's findings through rigorous wet-lab testing (see Figure \ref{fig:pipeline}). + +\begin{figure} +\centering{\includegraphics[width=\textwidth]{pipeline.png}} +\caption{\text{a)} A depiction of the DMPNN representing a molecule. Each vertex is an atom, and each edge is a bond. Messages of hidden states are passed along edges (e.g. the yellow and read arrows at the top). \text{b)} denotes the training and validation phase of the DMPNN, making predictions for $10^8$ molecules. \text{c) and d)} describe the screening of such molecules based on prediction scores, structural similarity and toxicity to filter the most promising candidates, along with experimental validation in the wet lab. Figure edited and extracted from \citep{stokes_2020_a}} \label{fig:pipeline} +\end{figure} + +First, they train the DMPNN in a supervised setting to identify molecules that can inhibit the growth of _Escherichia coli_. They collect a dataset $\mathcal{D} = \{\textbf{X}, \textbf{y}\}$ consisting of $|\textbf{X}| = 2335$ unique molecules, each annotated with $y \in \{0, 1\}$ using $80\%$ growth inhibition as a cut-off. This results in an imbalanced dataset with only 120 molecules with growth inhibitory activity. It is split according to a ratio of 80%/20%/20% into training/validation/testing sets. + +A molecule is a group of atoms held by bonds. Each is represented as a directed graph $G=(V, E)$, where each $v \in V$ is an atom, and each $e_{vw} \in E$ is an edge between vertices $v, w$ representing a bond, where $e_{vw} \neq e_{wv}$. Both atom and bond have molecular fingerprints, as well as associated hidden representations $h_v, h_{vw}$ that are obtained via learnable matrices $\textbf{W}=\{W_i, W_m, W_a\}$[^1]. The goal of Chemprop, as described by \citet{yang_2019_analyzing}, is to learn the optimal hidden representations that can be used to predict a functional property of the molecule, which in this work is growth inhibition of \textit{E. coli}. A forward computation and training iteration of the network for a single molecule (Figure \ref{fig:pipeline}a) is described as follows: + +% works as follows : + +\begin{enumerate} +\item Hidden state features for each bond are initialized at timestep $t = 0$: + +$ +h_{vw}^0 = \tau(W_i \text{cat}(v, e_{vw}) +$, where $v$ is the RDKit feature for the atom, and $e_{vw}$ is the RDKit feature for a bond. $W_i \in \mathbb {R}^{h\times h_i} $ is a learnable matrix of parameters associated to the hidden state of some edge $e_i$, \text{cat($\cdot$)} is a function that concatenates the atom and bond features, and $\tau$ is the ReLU activation function. +\item Messages between bonds $m_{vw}^t$ and hidden states $h_{vw}^t$ are passed and updated, respectively, given simple heuristics: + +$ +m_{vw}^{t+1} = \sum\limits_{\substack{k\in{N(v) \setminus w}}} h_{kv}^t +$, where the message is an aggregation of hidden representations, and $N(v)$ are the neighbors of atom $v$. + +$ +h_{vw}^{t+1} = \tau(h^0_{vw} + W_mm^{t+1}_{vw}) +$, where $W_m \in \mathbb {R}^{h\times h} $ is a learnable matrix. +\item Such message passing occurs for $t \in {1,...,T }$ through the whole graph, followed by a final message $m_v$ that returns the hidden representation $h_v$ for an atom $v$ of the molecule by summing the bond features as per: + +$ +m_{v} = \sum\limits_{\substack{k\in{N(v)}}} h_{kv}^T \\ +h_{v} = \tau(W_a\text{cat}(v, m_v)) +$, where $W_a \in \mathbb {R}^{h\times h} $ is a learnable matrix. + +\item The hidden representations for all atoms are obtained and aggregated to $h$. +$ +h = \sum\limits_{\substack{v\in{V}}} h_v +$ + +\item The output $\hat y$ of the D-MPNN is then computed as a function of $h$. In order to ensure generalization, this prediction is made by also incorporating 200 global features $h_f$ obtained via RDKit: + +$ +\hat y = f(\text{cat}(h, h_f)) +$, where $f(\cdot)$ is a feed-forward neural network. + +\item A loss function, in this case the binary cross-entropy, is computed based on the predicted output $\hat{y}$ and the ground truth value $y$, where $y \in \textbf{y}$. Then, its gradient is backpropagated to learn the optimal parameters $W_i, W_m, W_a$. + +\end{enumerate} + +\subsection{Results} + +The authors' final prediction is an average of an ensemble of 20 classifiers trained with different parameter initializations. Hyperaparameters are estimated using Bayesian optimization. Despite the class skewness, the model achieves a high test accuracy measured by the ROC-AUC of 89.6\%, evidencing its robustness. This is further reassuring given how their model is the highest performing in ablation studies examining different molecular fingerprints and architectures. + +Then, the authors use the DMPNN to screen more than 6000 molecules from the Drug Repurposing Hub (Figure \ref{fig:pipeline}cd). The most promising candidate according to prediction score, structural dissimilarity to known antibiotics, and predicted toxicity is named as halicin. They further validate it with multiple assays on a range of bacteria, as well as through rat animal models, observing long-term, broad-spectrum antibacterial activity \cite{marchant_2020_powerful}. + +\section{Critical analysis: limitations and future research directions} +\subsection{Efficient high-throughput screening} +The authors successfully leverage geometric deep learning as spatial-aware, pattern extractors in order to tackle an extremely challenging problem of drug repurposing, given the highly heterogeneous behavior of a drug's biochemicals and the sheer scale of their search space. They successfully overcome the bottleneck of traditional means as evidenced by how they then screened more than 107 million molecular structures from the ZINC15 database in a matter of 4 days, thus greatly reducing the cost of filtering potential candidates through conventional means. This has several real-world applications such as aiding biochemical labs in highly-efficient, fast screening of drugs to fight disease. +%thus greatly expanding the capabilities of drug screening and reducing associated costs. +% From thousands of high-scoring candidates, they shortlist 23 based on , +% Leveraging DMPNN to assess hundreds of millions to billions of molecules in a matter of days is an exponential improvement over traditional screening programs limited to testing a few million molecules (Liu, Gary (2023) . This includes finding it is structurally-divergent to existing antibiotics, and thus would not be subject to the long-term worry of antibiotic-resistant determinants, as well as low toxicity according to animal models, coupled with broad-spectrum antiobiotic efficacy. +Furthermore, extensive in-silico and wet-lab testing ensure the potential and safety of the predicted halicin. In addition to the above characteristics such as being structurally divergent, halicin has also been touted for its unconventional mechanism of action. It disrupts the flow of protons across the cell membrane, instead of more traditional approaches like blocking enzymes involved in protein synthesis \cite{marchant_2020_powerful}. This is an unanticipated gain that could arguably be only predicted by a deep learning system that can extract patterns beyond human comprehension from the training data. + +% The authors also rigorously tested its safety and efficacy through laborious wet-lab experiments, showing its broad-spectrum antibiotic efficacy. + +\subsection{Black-box architecture} +Despite these strengths, their model has a major flaw: its predictions remain elusive to interpretation by the biomedical personnel. This is concerning, given that the authors can not guarantee that their model is not learning spurious correlations \cite{jimnezluna_2020_drug} from the training data, e.g., maybe halicin was a top candidate because an irrelevant bond frequent in training was observed. Furthermore, the model's parameters can not explain how physico-chemical properties of halicin correlate to its functional properties. +% Furthermore, sometimes, chemists would be willing to sacrifice prediction accuracy in favour of explainable models that correlate biological effects with physicochemical properties (Luna, 2020). + +One powerful approach to mitigate this is semi-supervision: to employ generative pretraining over molecular databases\footnote{There are many datasets of molecules, such as those benchmarked in \cite{yang_2019_analyzing}} so that the model can learn a-priori a global latent representation of what are molecules. This graph autoencoder can then be finetuned to a downstream task of function classification, borrowing its internal representation to guide learning. Ad-hoc processing of such task-dependent latent representation, using techniques such as principal component analysis as in \citep{soelistyo_2022_learning, soelistyo_2023_virtual}, coupled with SHAP value methods that explore correlations between the input space and hidden activations of the model can yield mechanistic insight into \textit{why} it predicts certain compound. For example, maybe the presence of certain subgraph of atoms is biologically essential to inhibit bacterial growth. The latent representation could also help cluster drugs with similar properties, enabling the model to make predictions beyond a binary label. For more explainable methods please see \citet{jimnezluna_2020_drug}. +% \footnote{Such approach would be more expressive as one can use the latent representation to cluster drugs with similar properties, instead of specifically identifying a single function like growth inhibition of \textit{E. coli}} +Such pretraining could also yield additional benefits such as robustness to the the original dataset's small size and heavy skewness towards samples with no inhibition activity. This is important since despite achieving high test accuracy on the original dataset, the authors later report only $51.5\%$ when evaluated on the Drug Repurposing Hub. +% . that +% this would also help identify molecules which have inhibitory effect against harmful bacteria whilst avoiding those that synergize with the human body. For example, it is well-known +% it can also help save resources in validating the safety of molecules and making explainable predictions beyond a single label of whether it has an inhibitory property. This is +% .could thus improve explainability (Luna, Jose (2020)). T, which could have also Furthermore, while the authors did extensive wet-lab validation of the drug halicin, some work could have been simplified had they observed what are the "atoms" or "bond" characteristic the model is attending to that could yield some mechanistic insight into \textit{why} the model predicted halicin. +% and explainability. Additional benefits could span over the +% The model would have the advantage of learning a latent representation of what molecules are that could improve performance. +% Furthermore, it would have been useful to screen a few more candidates other than invest resources to validate one. +% \begin{itemize} +% \item focus on interpretability +% \end{itemize} +% such as where the molecule is supposed to act, or how neighboring molecules could influence its mechanism of action. Furthermore, +\subsection{Multimodal integration for contextualized predictions} +% \begin{itemize} +% \item focus on multimodal integration +% \end{itemize} +% In addition to addressing the above black-box limitations as potential future work, +Even if the black-box nature of deep learning is mitigated, it is noted that authors' adopted approach can only make context-agnostic predictions of a molecule's ability to inhibit \textit{E. coli}'s growth. For example, halicin may not universally inhibit its growth, such as when it lives in the human gut system repleted with other microorganisms. An exciting line of research is to integrate multiple modalities of data in order to make contextualized predictions of a molecule's functional property. This is a great opportunity for chemoinformatics given the need to unify the deeply fragmented public biochemical databases available, spanning datasets over drug-repositioning, drug-target prediction, drug-drug interaction datasets \cite{pavel_2022_the}, as well as a drug's side effects \cite{li_2022_graph}. + +Such effort to train models for contextualized predictions synergize well with the demands of transparency because a prediction would then be beyond a single probability value of a label. It would also depend on the aforementioned factors with potential benefits such as identifying molecules that selectively target harmful strains of \textit{E. coli}. This is important because it is well known that most strains of \textit{E. coli} are harmless and aid the digestive system of humans \cite{worldhealthorganization_2018_e}, while others can cause food poisoning. +Therefore, halicin may not be a good candidate if it indiscriminately kills \textit{E. coli}. + +It is clear that a lot of work is yet to be done on building transparent models for drug repurposing beyond highly performing black-boxes. +% The adoption Future research directions include augmenting the model's prediction capabilities, not restricting it to predict First, There needs to be more effort in multimodal data integration in order to +% such as a molecule's environment, molecule-molecule interactions derived from biomedical knowledge graphs, and clinical relevant data (such as a molecule's known side effects when interacting with another) could also augment the expressivity of the classifier. +The materialization of explainable models that can provide contextualized outputs can revolutionalize biomedical research, as they earn the trust of researchers whilst being highly performing. They can be deployed into real-world settings like clinical labs to aid rapid and efficient scientific discovery of drugs to tackle diverse global health concerns. In addition to fighting antibiotic resistance, applications can include repurposing existing drugs to fight viral variants, or mitigate neurological diseases like Alzheimer's or Parkinson's. + +[^1]: this is analogous to word embeddings that can be obtained via GLoVE and word2vec, where their hidden representations are learnt in a recurrent neural network's layers of abstraction. \ No newline at end of file diff --git a/_posts/lang-biology.md b/_posts/lang-biology.md new file mode 100644 index 000000000000..c22b308b3605 --- /dev/null +++ b/_posts/lang-biology.md @@ -0,0 +1,56 @@ +# An interesting analogy between language and biology + +# On the interesting parallels of language and biology + +Reading an interesting paper by Lauren, + +Hi Antonio, I have a question on VAEs. + +There are interesting parallels between the problems surfacing in natural language processing (NLP), as well as computational biology. The simplest common denominator is the analysis of sequences: algorithms and modelling paradigms that help process a sequence of words to solve a task like classification can surprisingly work with sequences of genes to classify disease. + +Another concern embeddings. Much as how words have embeddings which can be extracted from open-source libraries like GLoVE and word2Vec, molecules also have expert-coded molecular fingerprints as embeddings. + +Another Imagine that you want to generate sentences. You have a collection of sentences to train a probabilistic model on. Because we can't control what the probabilistic model learns from the given data, how do you make sure you don't generate an ungrammatical sentence like "jogging Octopie went", where we have a "verb-subject-verb" structure. The simplest way would be to set up an intrinsic template like "subject-verb-object"[^1] in order to offset portion of the probabilistic model's support[^3] to not put any probability mass to structures other than "subject-verb-object". Given this, this will first guarantee that sentences follow the correct grammar rules, and second guide [learning](https://awxlong.github.io/blog/2023/sgd/) since the model is not exploring over the whole parameter space, but only that which conform to the supplied template. + +Consider the following analogous task, given some atoms like hydrogen, carbon, oxygen, among others, you want to train a probabilistic model to predict the angle of rotation when such input atoms are assembled together into a compound. This could be relevant for a 3D reconstruction of an assembled molecule. You have a dataset of different atom-atom compounds with their associated rotation angles. If you only rely on learning, then the resulting probabilistic model may put probability mass on biologically implausible rotation angles, even if the probability mass is negligible. In the end, statistics is just about counting and averaging, meaning that even if you only see that some compound like water has a bond angle of around [105 degrees](http://witcombe.sbc.edu/water/chemistrystructure.html), the model's support would still put probability mass in-between 0 or 200 degrees [^2]. Enumerating all possible rules concerning rotation angles would be impractical, so one can resort to biomedical ontologies. + +Arguably, this is how DeepMind's AlphaFold ensures that proteins generated from DNA input sequences are biologically plausible, as can be observed in the model's pipeline. In it, the input sequence first goes through a database search that is subsequently converted into embeddings used to generate the protein structure. Without this first phase, this probabilistic model may output proteins that resemble the ones observed during training, at the risk of following impossible configurations. + + +
+ Sorry, an unanticipated error occured and the image can't load. +
Structural constraints in the first phase concerning genetic database search and structure database search ensures that rotation angles of bonds in the compounds of the protein are biologically plausible, setting structure in learning. Figure extracted from https://www.nature.com/articles/s41586-021-03819-2
+
+ +Generating implausible proteins, or invalid angle rotations between bonds is akin to generating sentences that violate grammar rules. It is debatable whether the rules can be learnt from data. Rather, the role of rules and constraints might be to shape learning. + +For further thought experiments, imagine whether a model can learn what are the rules of programming languages by only observing code, or infer the rules of grammar from just observing written text. + +Structural + +So I know that autoencoders are able to learn embeddings of the training data, however the training data is usually "static". Imagine now that the training data has a temporal dimension, and measures features that change over time of, say, a cell. Is there work that trains a "recurrent" VAE such that it can learn embeddings that are dependent on time? so if I want to visualize the embeddings, they change depending the time step in which I access them. + +Does this make sense? + +Lorenzo recommended VAEs that disentangle features of the input, such as a $$\beta$$-VAE, and see whether it can disentangle time. however, the paper he shared mainly worked with celebA. While there are multiple images of faces, each face is just a "snapshot", so maybe the VAE learns to disentangle noses from eyes from mouths, but not necessarily how they can change over time. Also, I'm not thinking of working with images. I'm thinking of tabular data that measures the features of multiple cells across time. + +# NLP inspiring future biomedical research directions + + +A dictionary with mostly indisputable grammar rules are akin to biological ontologies like the Gene Ontology +An interesting thought experiment hence, is can this grammar be expanded given the text available, analogous to asking whether + +
+ Sorry, an unanticipated error occured and the image can't load. +
.
+
+ +Mutations as contradictions? -> this sentence "this sentence has five words" has five words + +For example, challenges in NLP involve processing a sequence not only forward, but also backward. Consider the sentence: "shift two positions backward of each letter of the word 'trapelo' per the alphabet to decode it", where it is needed to read the sentence back and forth to process the word. Do we have to read a genome forward and backwards? + +If you have answers, share thoughts, you can leave a comment or please email me! + +[^1]: Sentences don't follow this simple template, but it helps get the idea across that placing structure into the support of a probabilistic model helps guide learning. +[^2]: While it is true that some bonds between atoms like a carbon-carbon bond have no restricted rotation angle, others like a [hydrogen peroxide](https://www.sciencedirect.com/science/article/pii/S0022285217302990) is constrained to a setting of rotation degrees (with some uncertainty). +[^3]: A probabilistic model's support refers to the domain of values for which the output of the model is non-zero. \ No newline at end of file diff --git a/_posts/on_representation.md b/_posts/on_representation.md index 5f77b406392f..65215fbf8cea 100644 --- a/_posts/on_representation.md +++ b/_posts/on_representation.md @@ -9,4 +9,16 @@ what if we step back and say, we don't need a representation. Instead, the world in other words, we don't have an internal representation for every object, we have an universal parsing mechanism that can parse objects, resulting in representations that we denote as internal representations -to give an example, remember Song Zhu Chun's and-or graphs. we used it to parse bottles when we were talking that afternoon: \ No newline at end of file +to give an example, remember Song Zhu Chun's and-or graphs. we used it to parse bottles when we were talking that afternoon: + +representation is not a reflection of the natural stimuli. + - for example a water molecule. A representation of it would not be the precise 3D model of it, accounting for scale. That would be the water molecule itself. + -t could be an abstract template which wich we measure + - such template can be obtained via averaging prior observations. average is just a weighted aggregation. a latent that is the average + - such abstraction is often of lower dimension and complexity than the reference stimuli. Again with the example of a water molecule, an abstraction of it may be a vectors like [2, 1] to denote 2 hydrogens and 1 oxygen (please see research on molecular fingerprints) or a symbol like `H_2O`. Both the siluette + - abstractions should be dynamic and are related to other abstractions. + - are some representations more powerful than others? + - graph vs. molecular fingerprints? why not use both. +unanswered questions: + + - can we control the representation that a machine learns so that it is scientifically meaningful. diff --git a/_posts/thoughts-deepmind-cell.md b/_posts/thoughts-deepmind-cell.md index 7dfe79823874..cda00d68d78e 100644 --- a/_posts/thoughts-deepmind-cell.md +++ b/_posts/thoughts-deepmind-cell.md @@ -10,7 +10,7 @@ We have no control on what artifacts of the dataset a deep learning model is loo Prof. Xavier Trepat offers a very good [account](https://www.nature.com/articles/d41586-018-07246-8) on the complementarity of top-down and bottom-up processes. Consider the following thought experiment: Top-down allows us to explain traffic jams, while bottom-up processes at most help us know how cars work. -In biomedicine, however, if we want to validate scientific discovery, mechanistic insight is indispensable. In that regard, AI can't further scientific discovery, only aid. +In biomedicine, however, if we want to validate scientific discovery, mechanistic insight is indispensable. In that regard, deep learning can't further scientific discovery, only aid. It is still the human scientists deciding what sequence of genes to decode. @@ -18,11 +18,15 @@ At least in coursework, More challenging benchmarks, then should involve modific In the future, however, I have confidence that AI can inspire future directions of research. -### On the science of intelligence and how it impacts unrelated fields +# Neuro-symbolic AI for building next-generation virtual cells -NeSy, despite pursuing the understanding of human intelligence, can yield unexpected insights into tackling problems in completely unrelated fields +In one of the most interesting articles I've read recently on [Building the next generation of virtual cells to understand cellular biology](https://www.sciencedirect.com/science/article/pii/S0006349523002369) -The above statement is supported by anecdotal evidence where scientific research translated into overarching technological applications for social good on completely unrelated fields. Our confidence on such an idea can come from Prof. Geoffrey Hinton's [interview with CBS](https://www.youtube.com/watch?v=qpoRO378qRY), among the endeavors of other researchers pursuing science whose work was converted into technologies benefiting humankind. Efforts include: +### A small note on the science of intelligence and how it impacts unrelated fields + +By themselves, NeSy AI methods are not designed to solve biomedical tasks like single-cell data analysis, in a way such as a sequence alignment algorithm is tailored for identifying similarities in gene sequences or a deep network architecture is assembled to solve a medical imaging reconstruction task. At its core, research on NeSy AI is to understand how the brain works. Despite pursuing how human intelligence functions, this can nonetheless can yield unexpected insights into tackling problems in completely unrelated fields like network biology or cell science. + +The above statement is supported by anecdotal evidence where scientific research translated into overarching technological applications for social good on completely unrelated fields. My confidence on such an idea can come from Prof. Geoffrey Hinton's [interview with CBS](https://www.youtube.com/watch?v=qpoRO378qRY), among the endeavors of other researchers pursuing science whose work was converted into technologies benefiting humankind. Efforts include: - [Prof. Katalin Karikó](https://arstechnica.com/health/2023/10/after-being-demoted-and-forced-to-retire-mrna-researcher-wins-nobel/), whose pursuit in understanding the central dogma of biology, namely how messenger-RNA translates into proteins, laid the foundation for the design of mRNA vaccines against the coronavirus during the COVID-19 pandemic. - Prof. Yann LeCun, Prof. Yoshua Bengio, Prof. Jürgen Schmidhuber and Prof. Geoffrey Hinton himself, whose research into how the human brain works, such as how to simulate the visual cortex, how to reproduce human language, how can a machine achieve self-reference or how machines can learn like human beings, kickstarted the Third Revolution of Artificial Intelligence that can translate into a plethora of applications spanning over biomedicine - Prof. Noam Chomsky's work on a universal grammar underlying Human thought for understanding human language, which helped inspire work like the design of the [FORTRAN programming language](https://en.wikipedia.org/wiki/Noam_Chomsky#Reception_and_influence) diff --git a/assets/bibliography/drl1.bib b/assets/bibliography/drl1.bib new file mode 100644 index 000000000000..c5d619ca1423 --- /dev/null +++ b/assets/bibliography/drl1.bib @@ -0,0 +1,135 @@ +@article{stokes_2020_a, + author = {Stokes, Jonathan M. and Yang, Kevin and Swanson, Kyle and Jin, Wengong and Cubillos-Ruiz, Andres and Donghia, Nina M. and MacNair, Craig R. and French, Shawn and Carfrae, Lindsey A. and Bloom-Ackerman, Zohar and Tran, Victoria M. and Chiappino-Pepe, Anush and Badran, Ahmed H. and Andrews, Ian W. and Chory, Emma J. and Church, George M. and Brown, Eric D. and Jaakkola, Tommi S. and Barzilay, Regina and Collins, James J.}, + month = {02}, + pages = {688–702.e13}, + title = {A Deep Learning Approach to Antibiotic Discovery}, + doi = {10.1016/j.cell.2020.01.021}, + url = {https://www.cell.com/cell/fulltext/S0092-8674(20)30102-1?_returnURL=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS0092867420301021%3Fshowall%3Dtrue#secsectitle0020}, + volume = {180}, + year = {2020}, + journal = {Cell} +} + +@article{yang_2019_analyzing, + author = {Yang, Kevin and Swanson, Kyle and Jin, Wengong and Coley, Connor and Eiden, Philipp and Gao, Hua and Guzman-Perez, Angel and Hopper, Timothy and Kelley, Brian and Mathea, Miriam and Palmer, Andrew and Settels, Volker and Jaakkola, Tommi and Jensen, Klavs and Barzilay, Regina}, + month = {07}, + pages = {3370-3388}, + title = {Analyzing Learned Molecular Representations for Property Prediction}, + doi = {10.1021/acs.jcim.9b00237}, + volume = {59}, + year = {2019}, + journal = {Journal of Chemical Information and Modeling} +} + +@article{swings_2017_adaptive, + author = {Swings, Toon and Van den Bergh, Bram and Wuyts, Sander and Oeyen, Eline and Voordeckers, Karin and Verstrepen, Kevin J and Fauvart, Maarten and Verstraeten, Natalie and Michiels, Jan}, + month = {05}, + title = {Adaptive tuning of mutation rates allows fast response to lethal stress in Escherichia coli}, + doi = {10.7554/elife.22939}, + volume = {6}, + year = {2017}, + journal = {eLife} +} + +@article{soelistyo_2022_learning, + author = {Soelistyo, Christopher J and Vallardi, Giulia and Charras, Guillaume and Lowe, Alan R}, + month = {06}, + pages = {636-644}, + publisher = {Nature Portfolio}, + title = {Learning biophysical determinants of cell fate with deep neural networks}, + doi = {10.1038/s42256-022-00503-6}, + urldate = {2023-10-20}, + volume = {4}, + year = {2022}, + journal = {Nature Machine Intelligence} +} + +@article{soelistyo_2023_virtual, + author = {Soelistyo, Christopher J and Charras, Guillaume and Lowe, Alan R}, + month = {07}, + publisher = {Cold Spring Harbor Laboratory}, + title = {Virtual perturbations to assess explainability of deep-learning based cell fate predictors}, + doi = {10.1101/2023.07.17.548859}, + year = {2023}, + journal = {bioRxiv (Cold Spring Harbor Laboratory)} +} + +@article{pavel_2022_the, + author = {Pavel, Alisa and Saarimäki, Laura A. and Möbus, Lena and Federico, Antonio and Serra, Angela and Greco, Dario}, + month = {01}, + pages = {4837–4849}, + title = {The potential of a data centred approach & knowledge graph data representation in chemical safety and drug design}, + doi = {10.1016/j.csbj.2022.08.061}, + url = {https://www.sciencedirect.com/science/article/pii/S2001037022003956}, + urldate = {2023-10-05}, + volume = {20}, + year = {2022}, + journal = {Computational and Structural Biotechnology Journal} +} + +@article{wang_2023_scientific, + author = {Wang, Hanchen and Fu, Tianfan and Du, Yuanqi and Gao, Wenhao and Huang, Kexin and Liu, Ziming and Chandak, Payal and Liu, Shengchao and Van Katwyk, Peter and Deac, Andreea and Anandkumar, Anima and Bergen, Karianne and Gomes, Carla P. and Ho, Shirley and Kohli, Pushmeet and Lasenby, Joan and Leskovec, Jure and Liu, Tie-Yan and Manrai, Arjun and Marks, Debora and Ramsundar, Bharath and Song, Le and Sun, Jimeng and Tang, Jian and Veličković, Petar and Welling, Max and Zhang, Linfeng and Coley, Connor W. and Bengio, Yoshua and Zitnik, Marinka}, + month = {08}, + pages = {47–60}, + title = {Scientific discovery in the age of artificial intelligence}, + doi = {10.1038/s41586-023-06221-2}, + url = {https://www.nature.com/articles/s41586-023-06221-2}, + urldate = {2023-08-20}, + volume = {620}, + year = {2023}, + journal = {Nature} +} + +@article{jimnezluna_2020_drug, + author = {Jiménez-Luna, José and Grisoni, Francesca and Schneider, Gisbert}, + month = {10}, + pages = {573-584}, + title = {Drug discovery with explainable artificial intelligence}, + doi = {10.1038/s42256-020-00236-4}, + urldate = {2021-10-18}, + volume = {2}, + year = {2020}, + journal = {Nature Machine Intelligence} +} + +@article{marchant_2020_powerful, + author = {Marchant, Jo}, + month = {02}, + title = {Powerful antibiotics discovered using AI}, + doi = {10.1038/d41586-020-00018-3}, + url = {https://www.nature.com/articles/d41586-020-00018-3}, + year = {2020}, + journal = {Nature} +} + +@article{liu_2023_deep, + author = {Liu, Gary and Catacutan, Denise B. and Rathod, Khushi and Swanson, Kyle and Jin, Wengong and Mohammed, Jody C. and Chiappino-Pepe, Anush and Syed, Saad A. and Fragis, Meghan and Rachwalski, Kenneth and Magolan, Jakob and Surette, Michael G. and Coombes, Brian K. and Jaakkola, Tommi and Barzilay, Regina and Collins, James J. and Stokes, Jonathan M.}, + month = {05}, + pages = {1342–1350}, + title = {Deep learning-guided discovery of an antibiotic targeting Acinetobacter baumannii}, + doi = {10.1038/s41589-023-01349-8}, + url = {https://www.nature.com/articles/s41589-023-01349-8}, + volume = {19}, + year = {2023}, + journal = {Nature Chemical Biology} +} + +@article{li_2022_graph, + author = {Li, Michelle M. and Huang, Kexin and Zitnik, Marinka}, + month = {10}, + title = {Graph representation learning in biomedicine and healthcare}, + doi = {10.1038/s41551-022-00942-x}, + url = {https://pubmed.ncbi.nlm.nih.gov/36316368/}, + urldate = {2022-12-12}, + year = {2022}, + journal = {Nature Biomedical Engineering} +} + +@misc{worldhealthorganization_2018_e, + author = {World Health Organization}, + month = {02}, + title = {E. coli}, + url = {https://www.who.int/news-room/fact-sheets/detail/e-coli}, + year = {2018}, + organization = {World Health Organization} +} \ No newline at end of file diff --git a/assets/img/alphafold-pipelin.png b/assets/img/alphafold-pipelin.png new file mode 100644 index 000000000000..6db711df8370 Binary files /dev/null and b/assets/img/alphafold-pipelin.png differ diff --git a/assets/img/biophysical-mitosis.png b/assets/img/biophysical-mitosis.png new file mode 100644 index 000000000000..827e3ec002c6 Binary files /dev/null and b/assets/img/biophysical-mitosis.png differ diff --git a/assets/img/dog-cat.jpg b/assets/img/dog-cat.jpg new file mode 100644 index 000000000000..ddb4b380993d Binary files /dev/null and b/assets/img/dog-cat.jpg differ diff --git a/assets/img/expand-ontology.png b/assets/img/expand-ontology.png new file mode 100644 index 000000000000..974a9e22f55c Binary files /dev/null and b/assets/img/expand-ontology.png differ diff --git a/assets/pdf/autonomous-ml-lecunnpdf.pdf b/assets/pdf/autonomous-ml-lecunnpdf.pdf new file mode 100644 index 000000000000..259b947e85f5 Binary files /dev/null and b/assets/pdf/autonomous-ml-lecunnpdf.pdf differ diff --git a/assets/pdf/deep-dark.pdf b/assets/pdf/deep-dark.pdf new file mode 100644 index 000000000000..264d13e57369 Binary files /dev/null and b/assets/pdf/deep-dark.pdf differ diff --git a/assets/pdf/feb_8th_presentation.pdf b/assets/pdf/feb_8th_presentation.pdf new file mode 100644 index 000000000000..864aa23e8ca0 Binary files /dev/null and b/assets/pdf/feb_8th_presentation.pdf differ diff --git a/assets/pdf/final_draft_chart_nesy_reasoner-gamma.pdf b/assets/pdf/final_draft_chart_nesy_reasoner-gamma.pdf new file mode 100644 index 000000000000..b3c42a000a7d Binary files /dev/null and b/assets/pdf/final_draft_chart_nesy_reasoner-gamma.pdf differ