4/4 ABBYY FlexiCapture 11 - FlexiLayout Studio & FormDesigner at #ABBYYSummit16
Four-Wheel Drive: ABBYY FlexiCapture & Mobile Captureimages.abbyy.com/Partner...
Transcript of Four-Wheel Drive: ABBYY FlexiCapture & Mobile Captureimages.abbyy.com/Partner...
Olga Titova, Andrew Zyuzin
Four-Wheel Drive: ABBYY FlexiCapture & Mobile Capture
ABBYY FlexiCapture Enables Complex Automation Environments to Work
2© ABBYY Confidential
Systems and RobotsEnterprise applications and
digital robotic processes
designed to run businesses
DatabasesRobotic Process Automation (RPA)
ECM ERP, CRM, BPM
ABBYY provides the intelligence to turn unstructured content into structured, actionable information, transforming processes for a better customer experience, lower costs, lower risk and competitive advantage.
Customer InteractionsReal-time, Content Centric Processes
Structured & Unstructured Content
FAXMFP
SCANWEB
MOBILE
Digitization occurs at the Knowledge Worker’s point of need (not IT mandate) – the platform is for making business units more effective in serving their customers, facilitating meaningful change
3
New Opportunities with State-of-the-art AI
© ABBYY Content IQ Conference. Bologna 2019
Intelligent Image Capture
Customer Onboarding
Advanced Classification
Mortgage, Intent Detection
Semi-structured
P2P/O2C, Expenses
NLP
Commercial Real Estate, Healthcare
Intelligent Image Capture
© ABBYY Content IQ Conference. Bologna 2019
Quality matters!
5© ABBYY Content IQ Conference. Bologna 2019
Automatic image capture
• estimates camera movement, frame sharpness and looks for document boundaries
• captures the best document image from video stream
Crop document and make it look as rectangle
Get the best image possible
6© ABBYY Content IQ Conference. Bologna 2019
Mobile usage scenarios
7
• Customer registration/account opening
On-device, cloud based or on premise server side back end solution• Customers self-service
trailing document
© ABBYY Confidential
No need to download an app, just open a webpage
For user looks like camera working in web-browser
Works on both Android and iOS
Mobile Web Capture: Loan application
8© ABBYY Confidential
LIVE DEMO
9© ABBYY Content IQ Conference. Bologna 2019
Advanced Classification
© ABBYY Content IQ Conference. Bologna 2019
11
• A major financial sector in US.
• 66% homeowners have mortgage today
• In 2018 $1.75 trillion mortgages originated
• Mortgage refinancing
• 400+ doc types
• Speed of the approval is key for origination
• Depth of analysis – to refinance
Mortgage Challenge
Market Key Players Capture Challenge
© ABBYY Content IQ Conference. Bologna 2019
• Origination
• 40% banks
• 51% non-depository lending institution
• Refinance - 46% GSEs
FlexiCapture 12How Advanced Classification works
12© ABBYY Confidential
А
Input Classification
text-based
image-based
Ensemble
Result
Confidence89%
IMG
TXT
CNN
RNN & NLP
SVM
?
Class A 90%Class B 50%Class C 10%
Class A 40%Class B 80%Class C 75%
ID
ID
ID
ID
15© ABBYY Confidential
Intent:AccountingPay invoice # <attached>Due date: 2019/07/30
Banks, insurance companies, public sector, service desk of any company receive a huge amount of repetitive messages every day….
When running a video in ADC 3.0 I have recently lost my sound. I have attached screenshots of my MAC sound system settings to get started.Regards,Andy Smith
Intent:Support ticketSound issue
Intent 1:Insurance.Car insurance originationCustomer: Andy Smith
Please calculate the cost of insurance for my car. Documents for the car attached.Andy Smith
Intent Detection with ABBYY FlexiCapture
Hi, see the invoice attached, please pay by the end of the month.
Invoices with Deep Learning
© ABBYY Content IQ Conference. Bologna 2019
17
ABBYY FlexiCapture for InvoicesPowered by Deep Neural Networks
• Imitates how human reads the invoice
• Quality boost from 80 to 90+%for invoices header fields
• No Master Data
• Extended country support
© ABBYY Confidential
18
P2P/O2C with ABBYY FlexiCapture
© ABBYY Content IQ Conference. Bologna 2019
Buyer areas
Supplier areas
Purchase order
Order confirmation
Delivery note
Invoice
Remittance advice
Payment
SALESProcess incoming POCreate a sales order
LOGISTIC EXECUTIONDeliver goods or services
ACCOUNTS RECIEVABLECreate outgoing invoice
Process RA and find open itemsClear open items
ACCOUNTS PAYABLEProcess incoming invoicesCreate remittance advice
Pay invoice
LOGISTIC EXECUTIONProcess incoming delivery
notes
PURCHASINGCreate purchase order
Process confirmation
Various departments
Demand, need or requirementCreate a requisition
Procurement to pay and order to cash Buyer's and supplier's
documents:
• Purchase Orders
• Delivery Note
• Sales Orders
• Remittance
Shipment documents:
• Bill of lading
• Waybill
19© ABBYY Confidential
A-Za-z0-9.,
A-Za-z0-9,
A-Za-z0-9.,
A-Za-z.,/
A-Za-z0-9,
A-Za-z0-9,A-Z0-9/.-
• Works perfect for similar layouts, like invoices from 1 supplier, 1 invoice is enough
ABBYY FlexiCapture 12On-premise ML – One-shot Learning
The algorithm automatically splits incoming documents into groups with similar looking document layouts.
Extraction models are trained for each group.
Master Data is not required.
Finally, more accurate learning and less training.
20
ABBYY FlexiCapture 12Unsupervised Clustering Enables On-premise ML
R3
NLP for Unstructured
© ABBYY Content IQ Conference. Bologna 2019
What can be addressed with NLP
22
bill of lading pay stub noticeappraisal
NLPFlexilayout
Healthcare: research articles analysis
Use Cases for ABBYY FlexiCapture NLP
© ABBYY Confidential 23
Financial services for real estate
Contract approval automation
Financial solutions for Commercial Real Estate (US)
24© ABBYY Confidential
Real estate total return is around 7% in 2018 to 8% in 2019
16M loan agreements in 2018at origination and secondary market
Portfolio Management AnalystCommercial Real EstateSalary: well paidLevel of expertize: high
• Reviewing and analyzing complex loan documents, inter-creditor documents and deal documents
• Knowledge and understanding of credit and risk management principles
Scenarios: Loan scoring, Risk assessment, Loan
underwriting, Compliance regulations
Pain: struggle to extract huge amount of data from
Loans and supporting docs
Service Provider for real estate finance industry
Main Documents in Commercial Real Estate
25© ABBYY Confidential
Leases Loans Loan Applications
Data Extraction from Lease AgreementsNLP assists to portfolio analyst to extract data from hundreds of leases
26
1. Tenant Name: Collis Agency, LLC (“Tenant”)2. Suite Number: Suite No. 9083. Lease Commencement Date: December 1, 20164. Lease Expiration Date: January 31, 20195. Lease Annual Rent:
12/01 /2016-01/31/2017 $846.00/mo02/01 /2017-01/31/2018 $871.00/mo02/01/2018-01/31/2019 $897.00/mo
6. Security Deposit: $798.007. Pro Rata Share: (i) Tenant’s Proportionate Share: .45%,
which is the percentage by dividing (a) the 716 rentable square feet in the Premises by (b) the 158,397 rentable square feet in the Project.
8. Lease Commencement Date definition: If the Commencement Date is not the first day of a calendar month, then the Term shall be extended by the time between the Commencement Date and the first day of the next month. If this Lease is executed before the Premises become vacant or otherwise available and ready for occupancy by Tenant, or if any present occupant of the Premises holds over and Landlord cannot acquire possession of the Premises before the Commencement Date, then (a) Tenant’s obligation to pay Rent
9. Square Footage: 1,35210. Original Occupancy Date: January 12, 201611. Dates of Each Renewal / Extension: Every year12. Insurance: Cost of any insurance or insurance related
expense applicable to the Project and Landlord’s personal property used in connection therewith
…
NLP Model for Leases
• Lease agreement contains 27-35 pages
• 22-37 fields should be extracted
• NLP model was train on 15docs
• 1 min to process 1 lease agreements
NLP Data Extraction from Loan AgreementsLooking for reduction in time, now data are processed manually
27
1. Origination Date: March 29, 20182. Entity Name – role Lender: DEUTSCHE BANK AG, NEW
YORK BRANCH3. Entity Address: 60 Wall Street, 10
thFloor, New York,
New York, 100054. Entity Name - role Borrower: DW PORTLAND LLC5. Entity name with role Co-Borrower: DW OMAHA SUB
LLC6. Entity Address: 640 Fifth Avenue, 17
thFloor, New York,
New York, 100197. Entity Overflow Name: Russell Gimelstob8. Business Day: “Business Day” shall mean any day other
than a Saturday, a Sunday or a legal holiday on which national banks are not open for general business in (i) the State of New York or (ii) the state where the servicing offices of the Servicer are located.
9. Debt Service Coverage Ratio: “Debt Service Coverage Ratio” shall mean, a ratio, as reasonably determined by Lender in which:
(a) the numerator is the Annualized Underwritten Net Cash Flow; and(b) the denominator is the Annualized Debt Service.
10. Debt Yield: “Debt Yield” shall mean, as of any date of determination, a ratio, expressed as a percentage, as reasonably determined by Lender, obtained by dividing:
(a) the Annualized Underwritten Net Cash Flow determined as of such date; and(b) the Aggregate Outstanding Principal Balance.
…
NLP Model for Loans
• Loan agreement contains 200-300 pages
• 50-70 fields should be extracted
• 3 iterations to clarify requirements
• 2 weeks to understand the contact language and to mark-up 23 docs
NLP in ABBYY FlexiCapture
Segment contains terms
Segment contains liability
28
ABBYY FlexiCapture NLP
Start Date: SEPTEMBER 26, 2007
End Date: JUNE 30, 2008
Maximum liability: $6,000
Segmentation enables us:• to improve accuracy of field extraction• to increase speed of processing
29
ABBYY FlexiCapture NLP
NLP understands document as it “knows” a lot about each word in a sentence and relations between them
Patient Leaflet
Pharmacovigilance (PV)
30© ABBYY Confidential
Medical literature: articles in journals related to pharmaceutical products
Pharmacovigilance (PV), also known as drug safety, is the pharmacological science relating to the collection, assessment, monitoring, and prevention of adverse effects with pharmaceutical products.
31
400 doctors and pharmacists Salary: well paidLevel of expertize: high
• 700 drugs are tracked
• 30-200 articles about each drug weekly
• Severe adverse events (death, related to newborns, pregnancy) should be reported less than in 14 days
• Report to a pharmaceutical company and to the local drug regulatory authority
Service Provider for Pharmacovigilance Individual Case Safety Reports (ICSR) detection
© ABBYY Content IQ Conference. Bologna 2019
Pharmacovigilance (PV)
Age 59-year-old Gender Female Adverse reactions Loss of muscle strengthMedication Simvastatin Dosage 40 mg PO qDay in the eveningMedication Fluconazole Dosage 150 mg PO as a single dose Adverse reactions Rhabdomyolysis
Thank you!
© ABBYY Content IQ Conference. Bologna 2019