Using Natural Language Processing and Machine Learning
Sumit Raj
Purchase Now !
Just with Paypal
Just with Paypal
Book Details
Price
|
4.00 USD |
|---|---|
Pages
| 205 p |
File Size
|
5,305 KB |
File Type
|
PDF format |
ISBN-13 (electronic) ISBN-13 (pbk)
| 978-1-4842-4096-0 978-1-4842-4095-3 |
Copyright
| 2019 by Sumit Raj |
Sumit Raj is a techie at heart, who loves coding and building
applications. He is a Python expert with a keen interest in
Machine Learning and Natural Language Processing. He
believes in the idea of writing code that directly impacts the
revenue of the company.
Sumit has worked in multiple domains, such as personal
finance management, real estate, e-commerce, and revenue
analytics, to build multiple scalable applications. He has
helped various early age startups with their initial design
and architecture of the product, which was later funded by investors and governments.
He comes with a good experience of cutting-edge technologies used in high-volume
internet/enterprise applications for scalability, performance tuning, and optimization
and cost-reduction.
He has been mentoring students/developers on Python programming all across the
globe. He has mentored over 1000 students and professionals using various online and
offline platforms and channels on programming languages, data science, and for career
counseling. Sumit likes to be a part of technical meetups, conferences, and workshops.
He never likes to miss a chance to attend hackathons. His love for building applications
and problem solving has won him multiple awards and accolades. He is regularly invited
to speak at premier educational institutes of India. He is also a speaker at PyLadies
meetup group, ladies who code in Python, which is led by one of the former director of
PSF (Python Software Foundation).
In his free time, he likes to write on his blog and answer questions on computer
programming, chatbots, Python/Django, career advice, and web development on Quora,
having over 1 million views together. Feel free to A2A on his Quora profile.
Currently, Sumit is working as Senior Solutions Architect at GeoSpark R&D in
Bangalore, India, building a developer platform for location tracking. You can get to
know more about him from his website (https://sumitraj.in). Readers can also ask
their questions and discuss at, https://buildingchatbotswithpython.sumitraj.in/.
About the Technical Reviewer
Nitin Solanki has extensive experience in Natural Language
Processing, Machine Learning, and Artificial Intelligence
Chatbot development. He has developed AI chatbots in
various domains, such as healthcare, e-commerce, education
and law firms, and more. He has experience working
on NLP libraries, data mining, data cleansing, feature
engineering, data analytics and visualization, and machine
learning algorithms. Nitin loves to make things simple and
automated. In his spare time, his mind starts chattering
about ideas to make money. Therefore, he keeps his mind
busy in exploring technologies and in writing codes.
Introduction
This book has been written with immense care to keep the teachings from this book
very pragmatic and results-oriented. Building chatbots is not just about completing a
tutorial or following a few steps—it’s a skill in itself. This book will certainly not bore you
with lots of text and process to be read; rather, it takes the learning-by-doing approach.
You must have used at least one chatbot to do something in your life by now. Whether
you are a programmer or not, once you go through this book you will find the building
blocks of chatbots, and all the mysteries will be uncovered. Building chatbots may
seem difficult from the outside, but this book makes it so easy for you. Our brain is not
designed to directly process the complex concepts; rather, we learn step-by-step. When
you are reading this book, from the first chapter through the last chapter, you will find
how clearly things are progressing. Although you can directly go to any chapter, I highly
recommend you start from the first chapter, as it is bound to bolster your thoughts.
This book is like a web series where you won’t be able to resist the next chapter after
completing one. Any chatbot that you interact with after going through this book will
create a picture in your mind on how that chatbot is designed and built internally.
Who This Book Is For
This book will serve as a great resource for learning the concepts related to chatbots and
learning how to build them. Those who will find this book useful include:
• Python web developers looking to expand their knowledge or career
into chatbots development
• Students and aspiring programmers wanting to acquire a new skill
set by hands-on experience to showcase something and stand out in the crowd
• Natural Language enthusiasts looking to learn how to build a chatbot
from scratch
• Budding entrepreneurs with a great idea but not enough technical
feasibility information on how to go about making a chatbot
• Product/Engineering managers planning for a chatbot-related project
Table of Contents
About the Author ..................................................................................................... xi
About the Technical Reviewer ............................................................................... xiii
Acknowledgments ...................................................................................................xv
Introduction ...........................................................................................................xvii
Chapter 1: The Beloved Chatbots
Popularity of Chatbots Usage ........................................................................................................ 2
The Zen of Python and Why It Applies to Chatbots? ...................................................................... 3
The Need for Chatbots .................................................................................................................. 5
The Business Perspective ....................................................................................................... 5
The Developer’s Perspective ................................................................................................. 10
Industries Impacted by Chatbots ................................................................................................ 12
Brief Timeline of Chatbots ........................................................................................................... 13
1950 ...................................................................................................................................... 13
1966 ...................................................................................................................................... 14
1972 ...................................................................................................................................... 14
1981 ...................................................................................................................................... 14
1985 ...................................................................................................................................... 14
1992 ...................................................................................................................................... 14
1995 ...................................................................................................................................... 14
1996 ...................................................................................................................................... 15
2001 ...................................................................................................................................... 15
2006 ...................................................................................................................................... 15
2010 ...................................................................................................................................... 15
2012 ...................................................................................................................................... 15
2014 ...................................................................................................................................... 15
2015 ...................................................................................................................................... 16
2016 ...................................................................................................................................... 16
2017 ...................................................................................................................................... 16
What Kind of Problems Can I Solve Using Chatbots? .................................................................. 16
Can the Problem be Solved by Simple Question and Answer or Back-and-Forth
Communication? ................................................................................................................... 17
Does It Have Highly Repetitive Issues That Require Either Analyzing or Fetching of Data? ....17
Can Your Bot’s Task be Automated and Fixed? ...................................................................... 18
A QnA Bot .................................................................................................................................... 18
Starting With Chatbots ................................................................................................................ 20
Decision Trees in Chatbots .......................................................................................................... 20
Using Decision Trees in Chatbots .......................................................................................... 21
How Does a Decision Tree Help? ........................................................................................... 21
The Best Chatbots/Bot Frameworks ........................................................................................... 25
Components of a Chatbot and Terminologies Used .................................................................... 26
Intent ..................................................................................................................................... 27
Entities ................................................................................................................................... 27
Utterances ............................................................................................................................. 27
Training the Bot ..................................................................................................................... 28
Confidence Score .................................................................................................................. 28
Chapter 2: Natural Language Processing for Chatbots
Why Do I Need to Know Natural Language Processing to Build a Chatbot? ............................... 29
What Is spaCy? ........................................................................................................................... 31
Benchmarks Results of spaCy ............................................................................................... 31
What Does spaCy Provide? .................................................................................................... 32
Features of spaCy ....................................................................................................................... 32
Installation and Prerequisites ................................................................................................ 33
What Are SpaCy Models? ...................................................................................................... 35
Fundamental Methods of NLP for Building Chatbots .................................................................. 37
POS Tagging ........................................................................................................................... 37
Stemming and Lemmatization ............................................................................................... 42
Named-Entity Recognition ..................................................................................................... 44
Stop Words ............................................................................................................................ 47
Dependency Parsing .............................................................................................................. 49
Noun Chunks ......................................................................................................................... 54
Finding Similarity .................................................................................................................. 55
Good to Know Things in NLP for Chatbots ................................................................................... 58
Tokenization .......................................................................................................................... 59
Regular Expressions .............................................................................................................. 60
Summary .................................................................................................................................... 61
Chapter 3: Building Chatbots the Easy Way
Introduction to Dialogflow ........................................................................................................... 63
Getting Started ............................................................................................................................ 65
Building a Food-Ordering Chatbot ......................................................................................... 65
Deciding the Scope ............................................................................................................... 65
Listing Intents ........................................................................................................................ 66
Listing Entities ....................................................................................................................... 66
Building a Food Ordering Chatbot ............................................................................................... 66
Getting Started With Dialogflow ............................................................................................ 67
Points to Remember When Creating Intents .......................................................................... 71
Creating Intents and Adding Utterances ................................................................................ 72
Adding Default Response to the Intent .................................................................................. 72
Item Description Intent and Belonging Entities ..................................................................... 73
Understanding and Replying Back to the User ...................................................................... 77
Deploying Dialogflow Chatbot on the Web .................................................................................. 82
Integrate Dialogflow Chatbot on Facebook Messenger .............................................................. 86
Setting Up Facebook ............................................................................................................. 86
Creating a Facebook App ....................................................................................................... 87
Setting Up the Dialogflow Console ........................................................................................ 88
Configuring Webhooks ........................................................................................................... 90
Testing the Messenger Bot .................................................................................................... 91
Fulfillment ................................................................................................................................... 96
Enabling Webhook ................................................................................................................. 98
Checking the Response ....................................................................................................... 101
Summary .................................................................................................................................. 103
Chapter 4: Building Chatbots the Hard Way
What Is Rasa NLU? .................................................................................................................... 106
Why Should I Use Rasa NLU? .............................................................................................. 106
Diving Straight Into Rasa NLU ............................................................................................. 107
Training and Building a Chatbot From Scratch ......................................................................... 109
Building a Horoscope Bot .................................................................................................... 109
Conversation Script Between the Horoscope Bot and the User .......................................... 110
Preparing Data for Chatbot .................................................................................................. 111
Training the Chatbot Model ................................................................................................. 116
Predicting From the Model .................................................................................................. 119
Dialog Management Using Rasa Core ....................................................................................... 121
Understanding More on Rasa Core and Dialog System ....................................................... 122
Understanding Rasa Concepts ............................................................................................ 125
Creating Domain File for the Chatbot .................................................................................. 127
Writing Custom Actions of the Chatbot ..................................................................................... 130
Data Preparation for Training the Bot ........................................................................................ 133
Creating Story Data ............................................................................................................. 134
Interactive Learning ............................................................................................................. 136
Exporting Conversations As Stories ..................................................................................... 150
Testing the Bot .......................................................................................................................... 152
Test Case 1 .......................................................................................................................... 152
Test Case 2 .......................................................................................................................... 153
Summary .................................................................................................................................. 153
Chapter 5: Deploying Your Chatbot
First Steps ................................................................................................................................. 155
Rasa’s Credential Management ................................................................................................ 155
Deploying the Chatbot on Facebook ......................................................................................... 157
Creating an App on Heroku .................................................................................................. 157
Setting Up Heroku on Your Local System ............................................................................ 158
Creating and Setting Up an App at Facebook ...................................................................... 158
Creating and Deploying Rasa Actions Server App on Heroku .............................................. 161
Creating Rasa Chatbot API App ............................................................................................ 163
Creating a Standalone Script for Facebook Messenger Chatbot ......................................... 164
Verifying the Deployment of Our Dialog Management App on Heroku ................................ 167
Integrating Webhook With Facebook ................................................................................... 167
Post-Deployment Verification: Facebook Chatbot ................................................................ 169
Deploying the Chatbot on Slack ................................................................................................ 171
Creating a Standalone Script for Slack Chatbot .................................................................. 171
Editing your Procfile ............................................................................................................ 175
Final Deployment of Slack Bot to Heroku ............................................................................ 175
Subscribe to Slack Events ................................................................................................... 175
Subscribe to Bot Events ...................................................................................................... 176
Post-Deployment Verification: Slack Bot ............................................................................. 177
Deploying the Chatbot on Your Own .......................................................................................... 178
Writing a Script for Your Own Chatbot Channel ................................................................... 179
Writing the Procfile and Deploying to the Web .................................................................... 181
Verifying Your Chatbot APIs ................................................................................................. 181
Creating the Chatbot UI ....................................................................................................... 183
Summary .................................................................................................................................. 187
Index ..................................................................................................................... 189
Bookscreen
How Do I Approach This Book?
Remember this book is not written like other books are written. This book is written
keeping in mind that once you are done with this book, you can build a chatbot yourself
or teach someone how to build a chatbot. It’s very important to keep a few points in
mind before approaching this book like any other book:
• This book covers almost everything that you need to build a chatbot,
rather than what exists.
• This book is about spending more time in doing things on your
system, with this book by your side. Make sure you execute each code
snippet and try to write the code; do not copy and paste.
• Make sure you follow the steps as they appear in the book; don’t
worry if you don’t understand something. You will get to know about
that later in the chapter.
• Use the source code and Jupyter Notebook provided with this book
for your reference.