Build with Watson.md

IBM Watson services on IBM Cloud

The figure shows the Watson services on IBM Cloud circa 2020. These services will continue to be enhanced and evolve in the future.

The following services are available as REST APIs and SaaS tools that developers can use to build AI solutions or add AI features to their applications. See the website https://www.ibm.com/watson/products-services/.

AI Assistant (Chatbot): Integrate diverse conversation technologies into your application: Watson Assistant: Quickly build a chat bot by using tools and dialog trees.

Data: Collect, organize and analyze your data, then achieve trust, scale, and automation across your full AI lifecycle:

• Watson Studio: collaborative environment with AI tools that a team can use to collect and prepare training data, and to design, train, and deploy machine learning models.

• Watson Machine Learning: Enables users to perform two fundamental operations of machine learning: training and scoring.

• Watson Knowledge Catalog: Machine learning data catalog (MLDC) that enables you to access, curate, categorize and share data, knowledge assets and their relationships, wherever they reside.

Knowledge: Get insights through accelerated data optimization capabilities:

• Discovery: Unlock hidden value in data to find answers, monitor trends, and surface patterns.
• Discovery News: Explore news and blogs with smarter news from Watson that includes concepts, sentiment, relationships and categories.
• Natural Language Understanding: NLP for advanced text analysis.
• Knowledge Studio: Teach Watson to discover meaningful insights in unstructured text.

Speech: Converts text and speech with the ability to customize models:

• Speech to Text: Easily converts audio and voice into written text.
• Text to Speech: Converts written text into natural-sounding audio.

Language: Analyzes text and extracts metadata from unstructured content:

• Language Translator: Translates text from one language to another.
• Natural Language Classifier: Interprets and classifies natural language. Applies natural language processing and machine learning techniques to return the best matching classes for a sentence or phrase.

Empathy: Understands tone, and emotional state:

• Natural Language Understanding: Extract sentiment and emotions from text.
• Tone Analyzer: Understands emotions and communication style in text.

Watson Assistant

• Adds a natural language interface to your application Dto automate interactions with your users.
• Build the conversation flow and train the service by using an easy-to- use web interface.
• Integrate Watson Assistant with other Watson services to enrich the chatbot interaction experience.
• Example applications: Virtual agents and chat bots.

Watson Assistant

IBM Watson Assistant is a cognitive bot that you can customize for your business needs, and deploy across multiple channels to bring help to your customers where and when they need it.

You can add a natural language interface to your application to automate interactions with your users.

Example applications include virtual agents and chat bots that can integrate and communicate on any channel or device, including mobile devices, messaging platforms, and robots.

Train the Watson Assistant service by using an easy-to-use web interface so that you can quickly build natural conversation flows between your apps and users, and deploy scalable and cost-effective solutions.

Watson Discovery

• Adds cognitive search and content analytics to applications to identify patterns, trends, and insights.
• Unlocks actionable insights into unstructured data.
• Unifies structured and unstructured data.
• Uses simple query language to eliminate the need for manual filtering of results.
• Includes the Discovery API and Discovery tooling.
• Example application: Find answers to FAQs

Watson Discovery

Add a cognitive search and content analytics engine to applications to identify patterns, trends, and actionable insights that drive better decision-making. Rapidly build cognitive, cloud-based exploration applications that unlock actionable insights that are hidden in unstructured data.

Securely unify structured and unstructured data with pre-enriched content, and use a simplified query language to eliminate the need for manual filtering of results.

With Discovery, you can prepare your unstructured data, create a query that will pinpoint the information you need, and then integrate those insights into your new application or existing solution.

The Discovery services includes:

• Discovery API: The Discovery service supports a number of SDKs to simplify the development of applications. The SDKs are available for many popular programming languages and platforms, including Node.js, Java, and Python.
• Discovery tooling: The Discovery service includes a complete set of online tools - the Discovery tooling - to help you quickly setup an instance of the service and populate it with data. The Discovery service tooling has been designed to save time by eliminating the need to use APIs to configure and populate your service.

With Discovery you can build applications that extract the correct answers to FAQs by enriching and searching data collections.

Natural Language Understanding

• Analyze semantic features of text input, including the following items:

• Concepts, Entities, Keywords, Categories, Sentiment, Emotion, Relations, Semantic roles

• Categorize content.

• Develop custom annotation models to identify domain-specific entities and relations in unstructured text by using Knowledge Studio.

• Example applications: Categorize news articles and blog posts and sort them based on general concepts, keywords, and entities.

Natural Language Understanding

Analyze text to extract metadata from content such as concepts, entities, keywords, categories, sentiment, emotion, relations, semantic roles. Custom annotation models are developed by using Watson Knowledge Studio to identify industry- and domain-specific entities and relations in unstructured text. Example applications: Categorize news articles and blog posts and sort them based on general concepts, keywords, and entities.

Natural language Classifier

• Applies AI techniques to return the best matching classes for a sentence or phrase.

• You create a classifier instance by providing a set of representative strings and a set of one or more correct classes for each training.

• After training, the new classifier can accept new questions or phrases and return the top matches with a probability value for each match.

• Example applications:

• Banking and finance: Classify investments, risks, and transactions.

• Services: Categorize service queries, messages, and responses to help address problems and deploy solutions quicker.

• Email: Classify mail as spam or non-spam.

Natural Language Classifier

The Natural Language Classifier service applies AI techniques to return the best matching classes for a short text (sentence or phrase). For example, you submit a question and the service returns keys to the best matching answers or next actions for your application.

You create a classifier instance by providing a set of representative strings and a set of one or more correct classes for each training. After training, the new classifier can accept new questions or phrases and return the top matches with a probability value for each match.

Example applications:

• Banking and finance: Classify investments, risks, and transaction
• Services: Categorize service queries, messages, and responses to help address problems and deploy solutions quicker
• Email: Classify mail as spam or non-spam.

Speech to Text

• Converts human voice into the corresponding text.

• Uses machine intelligence to combine information about grammar and language structure with knowledge of the composition of the audio signal to generate an accurate transcription.

• Provides APIs that you can use to add speech transcription capabilities to your applications.

• The languages that are currently supported are: Arabic (Modern Standard), Brazilian Portuguese, Chinese (Mandarin), Dutch, English (Australian, United Kingdom, and United States), French (French and Canadian), German, Italian, Japanese, Korean, Spanish.

• Example applications:

• Voice control of applications, embedded devices, and vehicle accessories

• Transcribing meetings and conference calls

• Dictating email messages and notes

Speech to Text

The Speech to Text service converts the human voice into the corresponding text. Use this service to convert audio and voice into text for quick understanding of content. It can be used anywhere there is a need to bridge the gap between the spoken word and their written form, including voice control of embedded systems, transcription of meetings and conference calls, and dictation of email and notes. This easy-to-use service uses machine intelligence to combine information about grammar and language structure with knowledge of the composition of the audio signal to generate an accurate transcription.

The following languages are currently supported: Arabic (Modern Standard), Brazilian Portuguese, Chinese (Mandarin), Dutch, English (Australian, United Kingdom, and United States), French (French and Canadian), German, Italian, Japanese, Korean, Spanish (Argentinian, Castilian, Chilean, Colombian, Mexican, and Peruvian).

Text to Speech

• Converts written text into natural sounding audio in various languages and voices.

• The currently available languages are Arabic, Brazilian Portuguese, Chinese (Mandarin), Dutch, English (Australian, United Kingdom, and United States dialects), French, German, Italian, Japanese, Korean, Spanish (Castilian, Latin American, and North American dialects).

• Example applications: Voice-driven and screenless applications, where audio is the preferred method of output:

• Interfaces for the disabled, such as assistance tools for the vision-impaired

• Reading text and email messages aloud to drivers

• Video-script narration and video voice over

• Reading-based educational tools

• Home-automation solutions

Text to Speech

Use the Watson Text to Speech API to convert written text into natural sounding audio in various languages and voices. The Text to Speech service processes text and natural language to generate synthesized audio output with the appropriate cadence and intonation.

The service supports voices in the following languages: Arabic, Brazilian Portuguese, Chinese (Mandarin), Dutch, English (Australian, United Kingdom, and United States dialects), French, German, Italian, Japanese, Korean, Spanish (Castilian, Latin American, and North American dialects). The service offers at least one male or female voice, sometimes both, for each language.

The service is appropriate for voice-driven and screenless applications, where audio is the preferred method of output:

• Interfaces for the disabled, such as assistance tools for the vision-impaired
• Reading text and email messages aloud to drivers
• Video-script narration and video voice over
• Reading-based educational tools
• Home-automation solutions

This service can be used in practical applications such as these:

• Create audio narration from a written script for a variety of applications, such as online tutorials for an e-learning course, audio books, and so on.
• Provide callers with information, such as company location, store hours, and account information that is extracted from a database or organization’s documentation and converted to audible speech.
• Develop interactive educational material for children.
• Communicate directions, hands-free.

Language Translator

• Identifies the language of text and translates it into different languages programmatically.

• High-quality, domain-specific text translation.

• The following domains and languages are supported:

• The News domain to translate English to and from French, Spanish, Portuguese, or Arabic.

• The Conversational domain to translate English to and from Spanish or Portuguese.

• The Patent domain to translate Spanish, Portuguese, Japanese, or Korean to English.

• Extend provided models to learn custom terms and phrases.

• Example application: Enable a help desk representative to assist international customers through chat.

Language Translator

This service provides high-quality and domain-specific text translation from one language to another. It is one of the few offerings that focuses on domain-specific Statistical Machine Translation.

The following domains and languages are supported:

•            The News domain to translate English to and from French, Spanish, Portuguese, or Arabic.

•            The Conversational domain to translate English to and from Spanish or Portuguese.

•            The Patent domain to translate Spanish, Portuguese, Japanese, or Korean to English

You can create your own custom translation model. Most of the provided translation models in Language Translator can be extended to learn custom terms and phrases or a general style that is derived from your translation data.

Tone Analyzer

• Tone impacts the effectiveness of communication in different contexts.

• Watson Tone Analyzer uses linguistic analysis to identify various tones at both the sentence and document level.

• It detects three types of tones from text:

• Emotion (anger, disgust, fear, joy, and sadness)

• Social tendencies (openness, conscientiousness, extroversion and introversion, agreeableness, and emotional range)

• Language styles (analytical, confident, and tentative)

• Example applications:

• Understand how written communications are perceived and then improve the tone of the communications.

• Businesses can learn the tone of their customers' communications and then respond appropriately to each customer.

Tone Analyzer
People show various tones, such as joy, sadness, anger, and agreeableness, in daily communications. Such tones can impact the effectiveness of communication in different contexts.

Watson Tone Analyzer uses linguistic analysis to identify various tones at both the sentence and document level. This insight can then be used to refine and improve communications.

It detects three types of tones:

• Emotion (anger, disgust, fear, joy, and sadness)
• Social propensities (openness, conscientiousness, extroversion and introversion, agreeableness, and emotional range)
• Language styles (analytical, confident, and tentative) from text

Use the Watson Tone Analyzer API in your applications to understand emotions, social tendencies, and perceived writing style.

Watson Studio

• Collaborative environment with AI tools to collect and prepare training data, and to design, train, and deploy machine learning models.
• It is a SaaS solution that is delivered on IBM Cloud.
• Watson Studio AI tools support popular frameworks, including: TensorFlow, Caffe, PyTorch, and Keras.
• The architecture of Watson Studio is centered around the project.

    

Watson Studio

IBM Watson Studio is a collaborative environment with AI tools that a team can use to collect and prepare training data, and to design, train, and deploy machine learning models.

It is a SaaS solution that is delivered on IBM Cloud.

Watson Studio provides a suite of tools for data scientists, application developers, and subject matter experts (SMEs) to work collaboratively and easily with data. They can then use that data to build, train, and deploy models at scale. These tools are preconfigured so that builders do not have to spend time installing, setting up, and maintaining them. The built-in catalog function enables knowledge sharing and retention. Watson Studio can infuse AI into your business.

It enables you to analyze data by using RStudio, Jupyter, and Python in a configured and collaborative environment that includes added value, such as managed Spark and IBM Watson Machine Learning.

The architecture of Watson Studio is centered around the project. A project is where you organize your resources and work with data.

You can think of Watson Studio AI tools in these categories:

• Natural language classification
• Machine learning
• Deep learning

Watson Machine Learning

• Machine Learning is a service on IBM Cloud with features for training and deploying machine learning models and neural networks. It provides:

• Interfaces for building, training, and deploying models: Python client library external link, Command line interface, REST API external link.

• Deployment infrastructure for hosting your trained models.

• Hyperparameter optimization for training complex neural networks.

• Distributed deep learning for distributing training runs across multiple servers.

• GPUs for faster training.

Watson Machine Learning

IBM Watson Machine Learning is a full-service IBM Cloud offering that makes it easy for data scientists and developers to work together to integrate predictive analytics with their applications.

The Watson Machine Learning service enables your organization to use the models in your

end-to-end solutions without the impact of licensing, installation, and configuration that is required by the same products when they are installed on-premises. You can use machine learning and deep learning models in production. Use an automated and collaborative workflow to grow intelligent business applications easily and with more confidence.

Using Watson Machine Learning, you can build sophisticated analytical models, trained with your own data, that you can deploy for use in applications.

Machine Learning is a service on IBM Cloud with features for training and deploying machine learning models and neural networks:

• Interfaces for building, training, and deploying models: Python client library external link, Command line interface, REST API external link
• Deployment infrastructure for hosting your trained models. After you create, train, and evaluate a model, you can deploy it. When you deploy a model you save it to the model repository that is associated with your Watson Machine Learning service. Then, you can use your deployed model to score data and build an application.
• Hyperparameter optimization for training complex neural networks : You can run your experiments with HPO to easily find the best quality model. Hyperparameter Optimization (HPO) is a mechanism for automatically exploring a search space of potential Hyperparameters, building a series of models and comparing the models using metrics of interest. To use HPO you must specify ranges of values to explore for each Hyperparameter.
• Distributed deep learning for distributing training runs across multiple servers. Deep learning models training can be significantly accelerated with distributed computing on GPUs.
• GPUs for faster training : IBM Watson Machine Learning deep learning simplifies the process to train models in parallel with an on-demand graphics processing units (GPU) compute cluster that you can scale to your specific needs.

IBM Watson knowledge Studio

• Enables developers and domain experts to collaborate and create a machine learning model that understands the linguistic nuances, meaning, and relationships specific to an industry or domain.
• Provides easy-to-use tools for annotating unstructured domain literature.
• The annotations are used to create a custom machine learning model that understands the language of the domain.
• The model can be deployed directly to Watson Natural Language Understanding and Watson Discovery.

IBM Watson Knowledge Studio

IBM Watson Knowledge Studio enables developers and domain experts to collaborate and create a machine learning model that understands the linguistic nuances, meaning, and relationships specific to an industry or domain.

Provides easy-to-use tools for annotating unstructured domain literature, and uses those annotations to create a custom machine-learning model that understands the language of the domain.

These annotators can identify mentions and relationships in unstructured data and be easily administered throughout their lifecycle by using one common tool.

Annotator components can be deployed directly to Watson Natural Language Understanding and Watson Discovery.

The diagram illustrates how it works: ![[Pasted image 20230623192944.png]] 1.     Based on a set of domain-specific source documents, the team creates a type system that defines entity types and relation types for the information of interest to the application that will use the model.

2.     A group of two or more human annotators annotates a small set of source documents to label words that represent entity types, to identify relation types where the text identifies relationships between entity mentions, and to define coreferences, which identify different mentions that refer to the same thing, that is, the same entity. Any inconsistencies in annotation are resolved, and one set of optimally annotated documents is built, which forms the ground truth.

3.     Watson Knowledge Studio uses the ground truth to train a model.