Large-Scale Machine Learning at PayPal Risk Zhang Pengshan © 2015 PayPal Inc. All rights reserved. Confidential and proprietary. 1 TO DECLINE, OR NOT DECLINE? 7:16pmCEST PDT 4:16am 7:15pm PDT €20 EUR $100 USD 7:18pm PDT 12:18pm AEST $65 AUD 7:15pm: Card holder lives in US - His wife paid a bill online using their home laptop 4:16am CEST: Card holder’s son studies in Europe - He bought Angry Birds on iPad instead of studying 12:18pm AEST: Card holder travels to Australia - He just paid for lunch at a POS © 2015 PayPal Inc. All rights reserved. Confidential and proprietary. 4 AGENDA PayPal & PayPal Risk Large Scale Machine Learning Solution Feature Engineering & Modeling Future Plan © 2015 PayPal Inc. All rights reserved. Confidential and proprietary. 5 AGENDA PayPal & PayPal Risk Large Scale Machine Learning Solution Feature Engineering & Modeling Future Plan © 2015 PayPal Inc. All rights reserved. Confidential and proprietary. 6 © 2015 PayPal Inc. All rights reserved. Confidential and proprietary. 7 PayPal: Leading the Digital Payments Revolution STRONG FOUNDATION 165 Million Active Customer Accounts $235 Billion Total Payment Volume STRONG MOMENTUM +19 Million Active Customer Accounts Gained in 2014 +26% Total Payment Volume Growth YoY $8 Billion +19% 4 Billion +22% Revenue Payment Transactions © 2015 PayPal Inc. All rights reserved. Confidential and proprietary. Total Revenue Growth YoY Payment Transactions Growth YoY 8 PAYPAL RISK CONSUMER SELLER TRUST PAYPAL RISK SAVE LOSS BUYER/SELLER PROTECTION © 2015 PayPal Inc. All rights reserved. Confidential and proprietary. ENABLE BUSINESS 9 TRADITINAL FRAUD TYPES IN PAYPAL 404 STOLEN FINACIAL INR: SNAD: INR & SNAD * Item Not Received Significantly Not as Described © 2015 PayPal Inc. All rights reserved. Confidential and proprietary. 10 “NEW” FRAUD TYPES IN PAYPAL MONEY LAUNDRY © 2015 PayPal Inc. All rights reserved. Confidential and proprietary. COLLUSION 11 AGENDA PayPal & PayPal Risk Large Scale Machine Learning Solution Feature Engineering & Modeling Future Plan © 2015 PayPal Inc. All rights reserved. Confidential and proprietary. 12 TRADE-OFF IN RISK DECISION PLATFORM Accuracy, speed & robustness are conflicting requirements! ariables v f o ts lo d e Ne rate to make accu decisions… Reduce real-tim e data load… … but that impa cts decision accura cy pacts … but that im ed decision spe Speed © 2015 PayPal Inc. All rights reserved. Confidential and proprietary. 13 RISK MODELING ARCHITECTURE Online System Offline System Decision Services Logging System (Kafka + Storm) Variable Services Monitor System Model Services Models Online Variables (NoSql DB) Offline Variable Data warehouse (HDFS) Model Training Platform Offline Variable Computation Online Variable Simulation Offline Variables (NoSql DB) © 2015 PayPal Inc. All rights reserved. Confidential and proprietary. 14 RISK MODELING ARCHITECTURE Online System Offline System Decision Services Logging System (Kafka + Storm) Variable Services Monitor System Model Services Models Online Variables (NoSql DB) Offline Variable Data warehouse (HDFS) Model Training Platform Offline Variable Computation Online Variable Simulation Offline Variables (NoSql DB) © 2015 PayPal Inc. All rights reserved. Confidential and proprietary. 15 SHIFU: COMBINING FEATURE ENGNEERING AND DATA MODELING Shifu is an open-source, end-to-end machine learning and data mining framework built on top of Hadoop. NEW INIT STATS NORMALIZE EVAL VARSELECT Built on Guagua* POSTTRAIN* POSTTRAIN TRAIN Built on Guagua* *Guagua is an iterative computing framework for both Hadoop MapReduce and Hadoop YARN © 2015 PayPal Inc. All rights reserved. Confidential and proprietary. 16 SCALABLE FEATURE ENGINEERING Feature Statistics © 2015 PayPal Inc. All rights reserved. Confidential and proprietary. Feature Binning Feature Transform 17 LARGE-SCALE MODELING Cross Validation Feature Selection Model Ensemble #44566874232 Training Set(60%) #65888123344 #231889456 1 . #2223644887 Cross Validation Dataset(20%) Test Set(20%) © 2015 PayPal Inc. All rights reserved. Confidential and proprietary. 18 AGENDA PayPal & PayPal Risk Large Scale Machine Learning Solution Feature Engineering & Modeling Future Plan © 2015 PayPal Inc. All rights reserved. Confidential and proprietary. 19 A DATA EXAMPLE Category Target ID-Like © 2015 PayPal Inc. All rights rese