Doc formatting draft 1 (#1286)

* initial commit to migrate the mtllm documentation into jac-lang.org

* Doc formatting draft 1

* Pre commit done and draft mtllm documentation updated

* Formatting update

---------

Co-authored-by: Thamirawaran <Thamirawaran@users.noreply.github.com>
Co-authored-by: savini98 <savikashmira@gmail.com>

Author: Thamirawaran

jac/support/jac-lang.org/docs/learn/coders/mtllm/docs/building_blocks/Multimodality.md

@@ -0,0 +1,74 @@
+# <span style="color: orange">Multimodality
+
+For MTLLM to have actual neurosymbolic powers, it needs to be able to handle multimodal inputs and outputs. This means that it should be able to understand text, images, and videos. In this section, we will discuss how MTLLM can handle multimodal inputs.
+
+## <span style="color: orange">Image
+
+MTLLM can handle images as inputs. You can provide an image as input to the MTLLM Function or Method using the `Image` format of mtllm. Here is an example of how you can provide an image as input to the MTLLM Function or Method:
+
+```jac
+import:py from mtllm.llms, OpenAI;
+import:py from mtllm, Image;
+
+glob llm = OpenAI(model_name="gpt-4o");
+
+enum Personality {
+   INTROVERT: 'Person who is shy and reticent' = "Introvert",
+   EXTROVERT: 'Person who is outgoing and socially confident' = "Extrovert"
+}
+
+obj 'Person'
+Person {
+    has full_name: str,
+        yod: 'Year of Death': int,
+        personality: 'Personality of the Person': Personality;
+}
+
+can get_person_info(img: 'Image of Person': Image) -> Person
+by llm();
+
+with entry {
+    person_obj = get_person_info(Image("person.png"));
+    print(person_obj);
+}
+```
+
+Input Image (person.png):
+![person.png](https://preview.redd.it/g39au73fdir01.jpg?auto=webp&s=cef8394b639af82ba92d6ab084935f7adc8e841d)
+
+```python
+# Output
+Person(full_name='Albert Einstein', yod=1955, personality=Personality.INTROVERT)
+```
+
+In the above example, we have provided an image of a person ("Albert Einstein") as input to the `get_person_info` method. The method returns the information of the person in the image. The output of the method is a `Person` object with the name, year of death, and personality of the person in the image.
+
+## <span style="color: orange">Video
+
+Similarly, MTLLM can handle videos as inputs. You can provide a video as input to the MTLLM Function or Method using the `Video` format of mtllm. Here is an example of how you can provide a video as input to the MTLLM Function or Method:
+
+```jac
+import:py from mtllm.llms, OpenAI;
+import:py from mtllm, Video;
+
+glob llm = OpenAI(model_name="gpt-4o");
+
+can is_aligned(video: Video, text: str) -> bool
+by llm(method="Chain-of-Thoughts", context="Mugen is the moving character");
+
+with entry {
+    video = Video("mugen.mp4", 1);
+    text = "Mugen jumps off and collects few coins.";
+    print(is_aligned(video, text));
+}
+```
+
+Input Video (mugen.mp4):
+[mugen.mp4](https://github.com/Jaseci-Labs/jaseci/blob/main/jac-mtllm/examples/vision/mugen.mp4)
+
+```python
+# Output
+True
+```
+
+In the above example, we have provided a video of a character ("Mugen") as input to the `is_aligned` method. The method checks if the text is aligned with the video. The output of the method is a boolean value indicating whether the text is aligned with the video.

jac/support/jac-lang.org/docs/learn/coders/mtllm/docs/building_blocks/functions_methods.md

@@ -0,0 +1,159 @@
+# <span style="color: orange">Functions and Methods
+
+Functions and methods play a crucial role in implementing various functionalities in a traditional GenAI application. In jaclang, we have designed these functions and methods to be highly flexible and powerful. Surprisingly, they don't even require a function or method body thanks to the MTLLM `by <your_llm>` syntax. This section will guide you on how to effectively utilize functions and methods in jaclang using MTLLM.
+
+## <span style="color: orange">Functions
+
+Functions/Abilities in jaclang are defined using the `can` keyword. They can be used to define a set of actions. Normal function looks like this in jaclang:
+
+```jac
+can <function_name>(<parameter : parameter_type>, ..) -> <return_type> {
+    <function_body>;
+}
+```
+
+In a traditional GenAI application, you would make API calls inside the function body to perform the desired action. However, in jaclang, you can define the function using the `by <your_llm>` syntax. This way, you can define the function without a body and let the MTLLM model handle the implementation. Here is an example:
+
+```jac
+can greet(name: str) -> str by <your_llm>();
+```
+
+In the above example, the `greet` function takes a `name` parameter of type `str` and returns a `str`. The function is defined using the `by <your_llm>` syntax, which means the implementation of the function is handled by the MTLLM.
+
+Below is an example where we define a function `get_expert` that takes a question as input and returns the best expert to answer the question in string format using mtllm with openai model with the method `Reason`. `get_answer` function takes a question and an expert as input and returns the answer to the question using mtllm with openai model without any method. and we can call these function as normal functions.
+
+```jac
+import:py from mtllm.llms, OpenAI;
+
+glob llm = OpenAI(model_name="gpt-4o");
+
+can get_expert(question: str) -> 'Best Expert to Answer the Question': str by llm(method='Reason');
+can get_answer(question: str, expert: str) -> str by llm();
+
+with entry {
+    question = "What are Large Language Models?";
+    expert = get_expert(question);
+    answer = get_answer(question, expert);
+    print(f"{expert} says: '{answer}' ");
+}
+```
+
+Here's another example,
+
+```jac
+import:py from mtllm.llms, OpenAI;
+
+glob llm = OpenAI(model_name="gpt-4o");
+
+can 'Get a Joke with a Punchline'
+get_joke() -> tuple[str, str] by llm();
+
+with entry {
+    (joke, punchline) = get_joke();
+    print(f"{joke}: {punchline}");
+}
+```
+
+In the above example, the `joke_punchline` function returns a tuple of two strings, which are the joke and its punchline. The function is defined using the `by <your_llm>` syntax, which means the implementation is handled by the MTLLM. You can add semstr to the function to make it more specific.
+
+
+## <span style="color: orange">Methods
+
+Methods in jaclang are also defined using the `can` keyword. They can be used to define a set of actions that are specific to a class. Normal method looks like this in jaclang:
+
+```python
+obj ClassName {
+    has parameter: parameter_type;
+    can <method_name>(<parameter : parameter_type>, ..) -> <return_type> {
+        <method_body>;
+    }
+}
+```
+
+In a traditional GenAI application, you would make API calls inside the method body to perform the desired action while using `self` keyword to get necessary information. However, in jaclang, you can define the method using the `by <your_llm>` syntax. This way, you can define the method without a body and let the MTLLM model handle the implementation. Here is an example:
+
+```jac
+obj Person {
+    has name: str;
+    can greet() -> str by <your_llm>(incl_info=(self));
+}
+```
+
+In the above example, the `greet` method returns a `str`. The method is defined using the `by <your_llm>` syntax, which means the implementation of the method is handled by the MTLLM. The `incl_info=(self.name)` parameter is used to include the `name` attribute of the `Person` object as an information source for the MTLLM.
+
+In the below example, we define a class `Essay` with a method `get_essay_judgement` that takes a criteria as input and returns the judgement for the essay based on the criteria using mtllm with openai model after a step of `Reasoning`. `get_reviewer_summary` method takes a dictionary of judgements as input and returns the summary of the reviewer based on the judgements using mtllm with openai model. `give_grade` method takes the summary as input and returns the grade for the essay using mtllm with openai model. and we can call these methods as normal methods.
+
+```jac
+import:py from mtllm.llms, OpenAI;
+
+glob llm = OpenAI(model_name="gpt-4o");
+
+obj Essay {
+    has essay: str;
+
+    can get_essay_judgement(criteria: str) -> str by llm(incl_info=(self.essay));
+    can get_reviewer_summary(judgements: dict) -> str by llm(incl_info=(self.essay));
+    can give_grade(summary: str) -> 'A to D': str by llm();
+}
+
+with entry {
+    essay = "With a population of approximately 45 million Spaniards and 3.5 million immigrants,"
+        "Spain is a country of contrasts where the richness of its culture blends it up with"
+        "the variety of languages and dialects used. Being one of the largest economies worldwide,"
+        "and the second largest country in Europe, Spain is a very appealing destination for tourists"
+        "as well as for immigrants from around the globe. Almost all Spaniards are used to speaking at"
+        "least two different languages, but protecting and preserving that right has not been"
+        "easy for them.Spaniards have had to struggle with war, ignorance, criticism and the governments,"
+        "in order to preserve and defend what identifies them, and deal with the consequences.";
+    essay = Essay(essay);
+    criterias = ["Clarity", "Originality", "Evidence"];
+    judgements = {};
+    for criteria in criterias {
+        judgement = essay.get_essay_judgement(criteria);
+        judgements[criteria] = judgement;
+    }
+    summary = essay.get_reviewer_summary(judgements);
+    grade = essay.give_grade(summary);
+    print("Reviewer Notes: ", summary);
+    print("Grade: ", grade);
+}
+```
+
+## <span style="color: orange">Ability to Understand Typed Inputs and Outputs
+
+MTLLM is able to represent typed inputs in a way that is understandable to the model. Sametime, this makes the model to generate outputs in the expected output type without any additional information. Here is an example:
+
+```jac
+import:py from mtllm.llms, OpenAI;
+
+glob llm = OpenAI(model_name="gpt-4o");
+
+
+enum 'Personality of the Person'
+Personality {
+   INTROVERT: 'Person who is shy and reticent' = "Introvert",
+   EXTROVERT: 'Person who is outgoing and socially confident' = "Extrovert"
+}
+
+obj 'Person'
+Person {
+    has full_name: 'Fullname of the Person': str,
+        yod: 'Year of Death': int,
+        personality: 'Personality of the Person': Personality;
+}
+
+can 'Get Person Information use common knowledge'
+get_person_info(name: 'Name of the Person': str) -> 'Person': Person by llm();
+
+with entry {
+    person_obj = get_person_info('Martin Luther King Jr.');
+    print(person_obj);
+}
+```
+
+```python
+# Output
+Person(full_name='Martin Luther King Jr.', yod=1968, personality=Personality.INTROVERT)
+```
+
+In the above example, the `get_person_info` function takes a `name` parameter of type `str` and returns a `Person` object. The `Person` object has three attributes: `full_name` of type `str`, `yod` of type `int`, and `personality` of type `Personality`. The `Personality` enum has two values: `INTROVERT` and `EXTROVERT`. The function is defined using the `by <your_llm>` syntax, which means the implementation is handled by the MTLLM. The model is able to understand the typed inputs and outputs and generate the output in the expected type.

jac/support/jac-lang.org/docs/learn/coders/mtllm/docs/building_blocks/language_models.md

@@ -0,0 +1,106 @@
+# <span style="color: orange">Language Models
+
+Language models is the most important building block of MTLLM. Without it we can't achieve neuro-symbolic programming.
+
+Let's first make sure you can set up your language model. MTLLM support clients for many remote and local LMs. You can even create your own as well very easily if you want to.
+
+## <span style="color: orange">Setting up a LM client
+
+In this section, we will go through the process of setting up a OpenAI's `GPT-4o` language model client. For that first makesure that you have installed the necessary dependancies by running `pip install mtllm[openai]`.
+
+```jac
+import:py from mtllm.llms.openai, OpenAI;
+
+my_llm = OpenAI(model_name="gpt-4o");
+```
+
+Makesure to set the `OPENAI_API_KEY` environment variable with your OpenAI API key.
+
+## <span style="color: orange">Directly calling the LM
+
+You can directly call the LM by giving the raw prompts as well.
+
+```jac
+my_llm("What is the capital of France?");
+```
+
+You can also pass the `max_tokens`, `temperature` and other parameters to the LM.
+
+```jac
+my_llm("What is the capital of France?", max_tokens=10, temperature=0.5);
+```
+
+## <span style="color: orange">Using the LM with MTLLM
+
+Intented use of MTLLM's LMs is to use them with the `jaclang`'s `BY_LLM` Feature.
+
+### <span style="color: orange">With Abilities and Methods
+
+```jac
+can function(arg1: str, arg2: str) -> str by llm();
+```
+
+### <span style="color: orange">With Classes
+
+```jac
+new_object = MyClass(arg1: str by llm());
+```
+
+### <span style="color: orange">You can parse following attributes to the `by llm()` feature:
+
+- `method` (default: `Normal`): Reasoning method to use. Can be `Normal`, `Reason` or `Chain-of-Thoughts`.
+- `tools` (default: `None`): Tools to use. This is a list of abilities to use with ReAct Prompting method.
+- `model specific parameters`: You can pass the model specific parameters as well. for example, `max_tokens`, `temperature` etc.
+
+## <span style="color: orange">Enabling Verbose Mode
+
+You can enable the verbose mode to see the internal workings of the LM.
+
+```jac
+import:py from mtllm.llms, OpenAI;
+
+my_llm = OpenAI(model_name="gpt-4o", verbose=True);
+```
+
+## <span style="color: orange">Remote LMs
+
+These language models are provided as managed services. To access them, simply sign up and obtain an API key. Before calling any of the remote language models listed below.
+
+> <span style="color: orange">**NOTICE**
+>
+> make sure to set the corresponding environment variable with your API key. Use Chat models for better performance.
+
+```jac
+llm = mtllm.llms.{provider_listed_below}(model_name="your model", verbose=True/False);
+```
+
+1. `OpenAI` - OpenAI's gpt-3.5-turbo, gpt-4, gpt-4-turbo, gpt-4o [[model zoo]](https://platform.openai.com/docs/models)
+2. `Anthropic` - Anthropic's Claude 3 & Claude 3.5 - Haiku ,Sonnet, Opus [[model zoo]](https://docs.anthropic.com/en/docs/about-claude/models)
+3. `Groq` - Groq's Fast Inference Models [[model zoo]](https://console.groq.com/docs/models)
+4. `Together` - Together's hosted OpenSource Models [[model zoo]](https://docs.together.ai/docs/inference-models)
+
+## <span style="color: orange">Local LMs
+
+### <span style="color: orange">Ollama
+
+Initiate an ollama server by following this tutorial [here](https://github.com/ollama/ollama). Then you can use it as follows:
+
+```jac
+import:py from mtllm.llms.ollama, Ollama;
+
+llm = Ollama(host="ip:port of the ollama server", model_name="llama3", verbose=True/False);
+```
+
+### <span style="color: orange">HuggingFace
+
+You can use any of the HuggingFace's language models as well. [models](https://huggingface.co/models?pipeline_tag=text-generation)
+
+```jac
+import:py from mtllm.llms.huggingface, HuggingFace;
+
+llm = HuggingFace(model_name="microsoft/Phi-3-mini-4k-instruct", verbose=True/False);
+```
+
+>  <span style="color: orange"> **NOTICE**
+>
+> We are constantly adding new LMs to the library. If you want to add a new LM, please open an issue [here](https://github.com/Jaseci-Labs/jaseci/issues).